Is Parallel Programming Hard, And, If So, What Can You Do About It?

268 APPENDIX E. FORMAL VERIFICATION1 static int2 dyntick_save_progress_counter(struct rcu_data *rdp)3 {4 int ret;5 int snap;6 int snap_nmi;78 snap = rdp->dynticks->dynticks;9 snap_nmi = rdp->dynticks->dynticks_nmi;10 smp_mb();11 rdp->dynticks_snap = snap;12 rdp->dynticks_nmi_snap = snap_nmi;13 ret = ((snap & 0x1) == 0) && ((snap_nmi & 0x1) == 0);14 if (ret)15 rdp->dynticks_fqs++;16 return ret;17 }Figure E.22: Saving Dyntick Progress Counters1 static int2 rcu_implicit_dynticks_qs(struct rcu_data *rdp)3 {4 long curr;5 long curr_nmi;6 long snap;7 long snap_nmi;89 curr = rdp->dynticks->dynticks;10 snap = rdp->dynticks_snap;11 curr_nmi = rdp->dynticks->dynticks_nmi;12 snap_nmi = rdp->dynticks_nmi_snap;13 smp_mb();14 if ((curr != snap || (curr & 0x1) == 0) &&15 (curr_nmi != snap_nmi || (curr_nmi & 0x1) == 0)) {16 rdp->dynticks_fqs++;17 return 1;18 }19 return rcu_implicit_offline_qs(rdp);20 }Figure E.23: Checking Dyntick Progress CountersCPU’s dynticks and dynticks_nmi counters.Lines 8 and 9 snapshot these two variables to locals,line 10 executes a memory barrier to pair with thememory barriers in the functions in Figures E.19,E.20, and E.21. Lines 11 and 12 record the snapshotsfor later calls to rcu_implicit_dynticks_qs,and 13 checks to see if the CPU is in dynticks-idlemode with neither irqs nor NMIs in progress (inother words, both snapshots have even values),hence in an extended quiescent state. If so, lines 14and 15 count this event, and line 16 returns true ifthe CPU was in a quiescent state.Figure E.23 shows dyntick_save_progress_counter, which is called to check whether a CPUhas entered dyntick-idle mode subsequent to a callto dynticks_save_progress_counter(). Lines 9and 11 take new snapshots of the correspondingCPU’sdynticks anddynticks_nmi variables, whilelines 10 and 12 retrieve the snapshots saved earlierby dynticks_save_progress_counter(). Line 13then executes a memory barrier to pair with thememory barriers in the functions in Figures E.19,E.20, and E.21. Lines 14 and 15 then check tosee if the CPU is either currently in a quiescentstate(currandcurr_nmihavingevenvalues)orhaspassedthroughaquiescentstatesincethelastcalltodynticks_save_progress_counter() (the valuesof dynticks and dynticks_nmi having changed).If these checks confirm that the CPU has passedthrough a dyntick-idle quiescent state, then line 16counts that fact and line 16 returns an indication ofthis fact. Either way, line 19 checks for race conditionsthat can result in RCU waiting for a CPU thatis offline.Quick Quiz E.20: This is still pretty complicated.Why not just have a cpumask_t that has abit set for each CPU that is in dyntick-idle mode,clearing the bit when entering an irq or NMI handler,and setting it upon exit?E.8.6 DiscussionA slight shift in viewpoint resulted in a substantialsimplification of the dynticks interface for RCU.The key change leading to this simplification wasminimizing of sharing between irq and NMI contexts.The only sharing in this simplified interfaceis references from NMI context to irq variables (thedynticks variable). This type of sharing is benign,because the NMI functions never update this variable,so that its value remains constant through thelifetime of the NMI handler. This limitation of sharingallows the individual functions to be understoodone at a time, in happy contrast to the situation describedin Section E.7, where an NMI might changeshared state at any point during execution of the irqfunctions.Verification can be a good thing, but simplicity iseven better.E.9 SummaryPromela is a very powerful tool for validating smallparallel algorithms. It is a useful tool in the parallelkernel hacker’s toolbox, but it should not bethe only tool. The QRCU experience is a case inpoint: given the Promela validation, the proof ofcorrectness, and several rcutorture runs, I now feelreasonably confident in the QRCU algorithm andits implementation. But I would certainly not feelso confident given only one of the three!Nevertheless, if your code is so complex that youfind yourself relying too heavily on validation tools,you should carefully rethink your design. For example,a complex implementation of the dynticks inter-