Is Parallel Programming Hard, And, If So, What Can You Do About It?

D.3. HIERARCHICAL RCU CODE WALKTHROUGH 201counts the irq nesting level, otherwise it is onegreater than the irq nesting level.dynticks: This int counter’s value is even if thecorrespondingCPUisindynticks-idlemodeandthere are no irq handlers currently running onthat CPU, otherwise the counter’s value is odd.In other words, if this counter’s value is odd,then the corresponding CPU might be in anRCU read-side critical section.dynticks_nmi: This int counter’s value is odd ifthe corresponding CPU is in an NMI handler,but only if the NMI arrived while this CPUwas in dyntick-idle mode with no irq handlersrunning. Otherwise, the counter’s value will beeven.This state is shared between the rcu and rcu bhimplementations.D.3.1.2 Nodes in the HierarchyAs noted earlier, the rcu_node hierarchy is flattenedinto the rcu_state structure as shown in FigureD.13 on page 191. Each rcu_node in this hierarchyhas fields as follows:lock: This spinlock guards the non-constant fieldsin this structure. This lock is acquired fromsoftirq context, so must disable irqs.Quick Quiz D.19: Why not simply disablebottomhalves(softirq)whenacquiringthercu_data structure’slock? Wouldn’t this be faster?The lock field of the root rcu_node has additionalresponsibilities:1. Serializes CPU-stall checking, so that agiven stall is reported by only one CPU.This can be important on systems withthousands of CPUs!2. Serializes starting a new grace period, sothat multiple CPUs don’t start conflictinggrace periods concurrently.3. Prevents new grace periods from startingin code that needs to run within the confinesof a single grace period.4. Serializes the state machine forcingquiescent states (in force_quiescent_state()) in order to keep the number ofreschedule IPIs down to a dull roar.qsmask: This bitmask tracks which CPUs (for leafrcu_node structures) or groups of CPUs (fornon-leaf rcu_node structures) still need to passthrough a quiescent state in order for the currentgrace period to end.qsmaskinit: This bitmask tracks which CPUs orgroups of CPUs will need to pass through aquiescent state for subsequent grace periods toend. The online/offline code manipulates theqsmaskinit fields, which are copied to the correspondingqsmask fields at the beginning ofeach grace period. This copy operation is onereason why grace period initialization must excludeonline/offline operations.grpmask: This bitmask has a single bit set, and thatis the bit corresponding to the this rcu_nodestructure’s position in the parent rcu_nodestructure’s qsmask and qsmaskinit fields. Useof this field simplifies quiescent-state processing,as suggested by Manfred Spraul.Quick Quiz D.20: How about theqsmask andqsmaskinit fields for the leaf rcu_node structures?Doesn’t there have to be some way towork out which of the bits in these fields correspondsto each CPU covered by the rcu_nodestructure in question?grplo: This field contains the number of the lowestnumberedCPUcoveredbythisrcu_nodestructure.grphi: This field contains the number of thehighest-numbered CPU covered by this rcu_node structure.grpnum: This field contains the bit number inthe parent rcu_node structure’s qsmask andqsmaskinit fields that this rcu_node structurecorresponds to. In otherwords, given a pointerrnptoagivenrcu_nodestructure,itwillalwaysbe the case that 1ULgrpmask. Thegrpnum field is used only for tracingoutput.level: This field contains zero for the root rcu_node structure, one for the rcu_node structuresthatarechildrenoftheroot, andsoondownthehierarchy.parent: This field is a pointer to the parent rcu_node structure, or NULL for the root rcu_nodestructure.