Is Parallel Programming Hard, And, If So, What Can You Do About It?

324 APPENDIX F. ANSWERS TO QUICK QUIZZESAnswer:Absolutely none!!! The code in rcu_read_unlock()interactswiththescheduling-clockinterrupthandlerrunning on the same CPU, and is thus insensitiveto reorderings because CPUs always see their ownaccesses as if they occurred in program order. OtherCPUs do access thercu_flipctr, but because theseother CPUs don’t access any of the other variables,ordering is irrelevant.Quick Quiz D.61:What problems could arise in rcu_read_unlock()if irqs were not disabled?Answer:1. Disabling irqs has the side effect of disablingpreemption. Suppose that this code were tobe preempted in the midst of line 17 betweenselecting the current CPU’s copy of the rcu_flipctr array and the decrement of the elementindicated by rcu_flipctr_idx. Executionmight well resume on some other CPU. Ifthis resumption happened concurrently with anrcu_read_lock() or rcu_read_unlock() runningon the original CPU, an increment ordecrement might be lost, resulting in either prematuretermination of a grace period, indefiniteextension of a grace period, or even both.2. Failing to disable preemption can also defeatRCU priority boosting, which relies on rcu_read_lock_nesting to determine which tasksto boost. If preemption occurred between theupdate of rcu_read_lock_nesting (line 16)and of rcu_flipctr (line 17), then a grace periodmight be stalled until this task resumed.But because the RCU priority booster hasno way of knowing that this particular taskis stalling grace periods, needed boosting willnever occur. Therefore, if there are CPUboundrealtime tasks running, the preemptedtask might never resume, stalling grace periodsindefinitely, and eventually resulting in OOM.Of course, both of these situations could be handledby disabling preemption rather than disablingirqs. (The CPUs I have access to do not show muchdifference between these two alternatives, but othersmight.)Quick Quiz D.62:Suppose that the irq disabling in rcu_read_lock()was replaced by preemption disabling. What effectwould that have on GP_STAGES?Answer:No finite value of GP_STAGES suffices. The followingscenario, courtesy of Oleg Nesterov, demonstratesthis:Suppose that low-priority Task A has executedrcu_read_lock() on CPU 0, and thus has incrementedper_cpu(rcu_flipctr,0)[0], which thushas a value of one. Suppose further that Task Ais now preempted indefinitely.Given this situation, consider the following sequenceof events:1. Task B starts executing rcu_read_lock(), alsoon CPU 0, picking up the low-order bit ofrcu_ctrlblk.completed, which is still equal tozero.2. Task B is interrupted by a sufficient numberof scheduling-clock interrupts to allow the currentgrace-period stage to complete, and also besufficient long-running interrupts to allow theRCU grace-period state machine to advance thercu_ctrlblk.complete countersothatitsbottombit is now equal to one and all CPUs haveacknowledged this increment operation.3. CPU 1 starts summing the index==0 counters,starting with per_cpu(rcu_flipctr,0)[0], which is equal to one due to Task A’s increment.CPU 1’s local variablesum is thereforeequal to one.4. Task B returns from interrupt, resuming itsexecution of rcu_read_lock(), incrementingper_cpu(rcu_flipctr,0)[0], which now hasa value of two.5. Task B is migrated to CPU 2.6. Task B completes its RCU read-side criticalsection, and executes rcu_read_unlock(),which decrements per_cpu(rcu_flipctr,2)[0], which is now -1.7. CPU 1now addsper_cpu(rcu_flipctr,1)[0]and per_cpu(rcu_flipctr,2)[0] to its localvariable sum, obtaining the value zero.8. CPU 1 then incorrectly concludes that all priorRCU read-side critical sections have completed,and advances to the next RCU grace-periodstage. This means that some other task mightwell free up data structures that Task A is stillusing!