Is Parallel Programming Hard, And, If So, What Can You Do About It?

256 APPENDIX E. FORMAL VERIFICATIONentry to an interrupt or NMI handler (in rcu_irq_enter()) and is decremented upon exit (inrcu_irq_exit()). In addition, the pre-existingin_interrupt() primitive is used to distinguish betweenan outermost or a nested interrupt/NMI.Interrupt entry is handled by the rcu_irq_entershown below:1 void rcu_irq_enter(void)2 {3 int cpu = smp_processor_id();45 if (per_cpu(rcu_update_flag, cpu))6 per_cpu(rcu_update_flag, cpu)++;7 if (!in_interrupt() &&8 (per_cpu(dynticks_progress_counter,9 cpu) & 0x1) == 0) {10 per_cpu(dynticks_progress_counter, cpu)++;11 smp_mb();12 per_cpu(rcu_update_flag, cpu)++;13 }14 }Line 3 fetches the current CPU’s number, whilelines 5 and 6 increment the rcu_update_flag nestingcounter if it is already non-zero. Lines 7-9 checkto see whether we are the outermost level of interrupt,and, if so, whether dynticks_progress_counter needs to be incremented. If so, line 10 incrementsdynticks_progress_counter, line 11 executesa memory barrier, and line 12 incrementsrcu_update_flag. As with rcu_exit_nohz(), thememory barrier ensures that any other CPU thatsees the effects of an RCU read-side critical sectionin the interrupt handler (following the rcu_irq_enter() invocation) will also see the increment ofdynticks_progress_counter.Quick Quiz E.7: Why not simply incrementrcu_update_flag, and then only incrementdynticks_progress_counter if the old value ofrcu_update_flag was zero???Quick Quiz E.8: But if line 7 finds that we arethe outermost interrupt, wouldn’t we always need toincrement dynticks_progress_counter?Interrupt exit is handled similarly by rcu_irq_exit():1 void rcu_irq_exit(void)2 {3 int cpu = smp_processor_id();45 if (per_cpu(rcu_update_flag, cpu)) {6 if (--per_cpu(rcu_update_flag, cpu))7 return;8 WARN_ON(in_interrupt());9 smp_mb();10 per_cpu(dynticks_progress_counter, cpu)++;11 WARN_ON(per_cpu(dynticks_progress_counter,12 cpu) & 0x1);13 }14 }Line 3 fetches the current CPU’s number, as before.Line 5 checks to see if the rcu_update_flag is non-zero, returning immediately (via fallingoff the end of the function) if not. Otherwise,lines 6 through 12 come into play. Line 6 decrementsrcu_update_flag, returning if the resultis not zero. Line 8 verifies that we are indeedleaving the outermost level of nested interrupts,line 9 executes a memory barrier, line 10 incrementsdynticks_progress_counter, and lines 11and 12 verify that this variable is now even. Aswith rcu_enter_nohz(), the memory barrier ensuresthat any other CPU that sees the incrementof dynticks_progress_counter will also see the effectsof an RCU read-side critical section in the interrupthandler (preceding the rcu_irq_exit() invocation).These two sections have described how thedynticks_progress_counter variable is maintainedduring entry to and exit from dynticks-idlemode, both by tasks and by interrupts and NMIs.The following section describes how this variable isusedbypreemptable RCU’s grace-periodmachinery.E.7.1.3 Grace-Period InterfaceOf the four preemptable RCU grace-period statesshown in Figure D.63 on page 236 in Appendix D.4,onlythercu_try_flip_waitack_state()andrcu_try_flip_waitmb_state() states need to wait forother CPUs to respond.Of course, if a given CPU is in dynticks-idlestate, we shouldn’t wait for it. Therefore, just beforeentering one of these two states, the precedingstate takes a snapshot of each CPU’s dynticks_progress_counter variable, placing the snapshot inanother per-CPU variable, rcu_dyntick_snapshot.This is accomplished by invoking dyntick_save_progress_counter, shown below:1 static void dyntick_save_progress_counter(int cpu)2 {3 per_cpu(rcu_dyntick_snapshot, cpu) =4 per_cpu(dynticks_progress_counter, cpu);5 }The rcu_try_flip_waitack_state() state invokesrcu_try_flip_waitack_needed(), shownbelow:1 static inline int2 rcu_try_flip_waitack_needed(int cpu)3 {4 long curr;5 long snap;67 curr = per_cpu(dynticks_progress_counter, cpu);8 snap = per_cpu(rcu_dyntick_snapshot, cpu);9 smp_mb();10 if ((curr == snap) && ((curr & 0x1) == 0))11 return 0;12 if ((curr - snap) > 2 || (snap & 0x1) == 0)13 return 0;