作者：崔晓兵

从一个线上问题说起

最近在线上遇到了一些[HMDConfigManager remoteConfigWithAppID:]卡死

初步分析

观察了下主线程堆栈，用到的锁是读写锁

随后又去翻了下持有着锁的子线程，有各种各样的情况，且基本都处于正常的执行状态，例如有的处于打开文件状态，有的处于read状态，有的正在执行NSUserDefaults的方法…通过观察发现，出问题的线程都有QOS:BACKGROUND标记。整体看起来持有锁的子线程仍然在执行，只是留给主线程的时间不够了。为什么这些子线程在持有锁的情况下，需要执行这么久，直到主线程的8s卡死？一种情况就是真的如此耗时，另一种则是出现了优先级反转。

解决办法

在这个案例里面，持有读写锁且优先级低的线程迟迟得不到调度（又或者得到调度的时候又被抢占了，或者得到调度的时候时间已然不够了），而具有高优先级的线程由于拿不到读写锁，一直被阻塞，所以互相死锁。iOS8之后引入了QualityOfService的概念，类似于线程的优先级，设置不同的QualityOfService的值后系统会分配不同的CPU时间、网络资源和硬盘资源等，因此我们可以通过这个设置队列的优先级 。

方案一：去除对NSOperationQueue的优先级设置

在 Threading Programming Guide 文档中，苹果给出了提示：

Important: It is generally a good idea to leave the priorities of your threads at their default values. Increasing the priorities of some threads also increases the likelihood of starvation among lower-priority threads. If your application contains high-priority and low-priority threads that must interact with each other, the starvation of lower-priority threads may block other threads and create performance bottlenecks.

苹果的建议是不要随意修改线程的优先级，尤其是这些高低优先级线程之间存在临界资源竞争的情况。所以删除相关优先级设置代码即可解决问题。

方案二：临时修改线程优先级

在 pthread_rwlock_rdlock(3pthread) 发现了如下提示：

Realtime applications may encounter priority inversion when using read-write locks. The problem occurs when a high priority thread “locks” a read-write lock that is about to be “unlocked” by a low priority thread, but the low priority thread is preempted by a medium priority thread. This scenario leads to priority inversion; a high priority thread is blocked by lower priority threads for an unlimited period of time. During system design, realtime programmers must take into account the possibility of this kind of priority inversion. They can deal with it in a number of ways, such as by having critical sections that are guarded by read-write locks execute at a high priority, so that a thread cannot be preempted while executing in its critical section.

尽管针对的是实时系统，但是还是有一些启示和帮助。按照提示，对有问题的代码进行了修改：在线程通过pthread_rwlock_wrlock拿到_rwlock的时候，临时提升其优先级，在释放_rwlock之后，恢复其原先的优先级。

- (id)remoteConfigWithAppID:(NSString *)appID
{
    .......
    pthread_rwlock_rdlock(&_rwlock);
    HMDHeimdallrConfig *result = ....... // get existing config
    pthread_rwlock_unlock(&_rwlock);
    
    if(result == nil) {
        result = [[HMDHeimdallrConfig alloc] init]; // make a new config
        pthread_rwlock_wrlock(&_rwlock);
        
        qos_class_t oldQos = qos_class_self();
        BOOL needRecover = NO;
        
        // 临时提升线程优先级
        if (_enablePriorityInversionProtection && oldQos < QOS_CLASS_USER_INTERACTIVE) {
            int ret = pthread_set_qos_class_self_np(QOS_CLASS_USER_INTERACTIVE, 0);
            needRecover = (ret == 0);
        }
            
        ......

        pthread_rwlock_unlock(&_rwlock);
        
        // 恢复线程优先级
        if (_enablePriorityInversionProtection && needRecover) {
            pthread_set_qos_class_self_np(oldQos, 0);
        }
    }
    
    return result;
}

值得注意的是，这里只能使用pthread的api，NSThread提供的API是不可行的

Demo 验证

为了验证上述的手动调整线程优先级是否有一定的效果，这里通过demo进行本地实验：定义了2000个operation（目的是为了CPU繁忙），优先级设置NSQualityOfServiceUserInitiated，且对其中可以被100整除的operation的优先级调整为NSQualityOfServiceBackground，在每个operation执行相同的耗时任务，然后对这被选中的10个operation进行耗时统计。

for (int j = 0; j < 2000; ++j) {
    NSOperationQueue *operation = [[NSOperationQueue alloc] init];
    operation.maxConcurrentOperationCount = 1;
    operation.qualityOfService = NSQualityOfServiceUs