Is Parallel Programming Hard, And, If So, What Can You Do About It?

E.8. SIMPLICITY AVOIDS FORMAL VERIFICATION 2671 void rcu_nmi_enter(void)2 {3 struct rcu_dynticks *rdtp;45 rdtp = &__get_cpu_var(rcu_dynticks);6 if (rdtp->dynticks & 0x1)7 return;8 rdtp->dynticks_nmi++;9 WARN_ON_RATELIMIT(!(rdtp->dynticks_nmi & 0x1),10 &rcu_rs);11 smp_mb();12 }1314 void rcu_nmi_exit(void)15 {16 struct rcu_dynticks *rdtp;1718 rdtp = &__get_cpu_var(rcu_dynticks);19 if (rdtp->dynticks & 0x1)20 return;21 smp_mb();22 rdtp->dynticks_nmi++;23 WARN_ON_RATELIMIT(rdtp->dynticks_nmi & 0x1, &rcu_rs);24 }Figure E.20: NMIs From Dynticks-Idle ModeLine 6 ensures that any prior memory accesses(which might include accesses from RCU read-sidecriticalsections)areseenbyotherCPUsbeforethosemarking entry to dynticks-idle mode. Lines 7 and 12disable and reenable irqs. Line 8 acquires a pointerto the current CPU’s rcu_dynticks structure, andline 9 increments the current CPU’s dyntickscounter, whichshouldnowbeeven, giventhatweareentering dynticks-idle mode in process context. Finally,line 10 decrements dynticks_nesting, whichshould now be zero.The rcu_exit_nohz() function is quite similar,but increments dynticks_nesting ratherthan decrementing it and checks for the oppositedynticks polarity.E.8.3 NMIs From Dynticks-IdleModeFigure E.20 show the rcu_nmi_enter() and rcu_nmi_exit() functions, which inform RCU of NMIentry and exit, respectively, from dynticks-idlemode. However, if the NMI arrives during an irqhandler, then RCU will already be on the lookoutfor RCU read-side critical sections from this CPU,so lines 6 and 7 of rcu_nmi_enter and lines 19and 20 of rcu_nmi_exit silently return if dynticksis odd. Otherwise, the two functions incrementdynticks_nmi, with rcu_nmi_enter() leaving itwith an odd value and rcu_nmi_exit() leaving itwith an even value. Both functions execute memorybarriers between this increment and possible RCUread-side critical sections on lines 11 and 21, respectively.1 void rcu_irq_enter(void)2 {3 struct rcu_dynticks *rdtp;45 rdtp = &__get_cpu_var(rcu_dynticks);6 if (rdtp->dynticks_nesting++)7 return;8 rdtp->dynticks++;9 WARN_ON_RATELIMIT(!(rdtp->dynticks & 0x1), &rcu_rs);10 smp_mb();11 }1213 void rcu_irq_exit(void)14 {15 struct rcu_dynticks *rdtp;1617 rdtp = &__get_cpu_var(rcu_dynticks);18 if (--rdtp->dynticks_nesting)19 return;20 smp_mb();21 rdtp->dynticks++;22 WARN_ON_RATELIMIT(rdtp->dynticks & 0x1, &rcu_rs);23 if (__get_cpu_var(rcu_data).nxtlist ||24 __get_cpu_var(rcu_bh_data).nxtlist)25 set_need_resched();26 }Figure E.21: Interrupts From Dynticks-Idle ModeE.8.4 Interrupts From Dynticks-IdleModeFigure E.21 shows rcu_irq_enter() and rcu_irq_exit(), which inform RCU of entry to and exitfrom, respectively, irq context. Line 6 of rcu_irq_enter() increments dynticks_nesting, and if thisvariable was already non-zero, line 7 silently returns.Otherwise, line 8 increments dynticks, which willthen have an odd value, consistent with the fact thatthis CPU can now execute RCU read-side criticalsections. Line 10 therefore executes a memory barrierto ensure that the increment of dynticks is seenbefore any RCU read-side critical sections that thesubsequent irq handler might execute.Line 18 of rcu_irq_exit decrements dynticks_nesting, andiftheresultisnon-zero, line19silentlyreturns. Otherwise, line 20 executes a memory barrierto ensure that the increment of dynticks online 21 is seen after any RCU read-side critical sectionsthat the prior irq handler might have executed.Line 22 verifies that dynticks is now even, consistentwith the fact that no RCU read-side critical sectionsmay appear in dynticks-idle mode. Lines 23-25check to see if the prior irq handlers enqueued anyRCUcallbacks, forcingthisCPUoutofdynticks-idlemode via an reschedule IPI if so.E.8.5 Checking For Dynticks QuiescentStatesFigure E.22 shows dyntick_save_progress_counter(), which takes a snapshot of the specified