Is Parallel Programming Hard, And, If So, What Can You Do About It?

D.3. HIERARCHICAL RCU CODE WALKTHROUGH 203that is ready to wait for the next graceperiod, or is equal to the RCU_WAIT_TAILelement if there is no such callback.RCU_NEXT_TAIL=3: This element references thenext field of the last callback in the list,or references the nxtlist field if the list isempty.Quick Quiz D.24: Why not simply have multiplelists rather than this funny multi-tailedlist?qlen: This field contains the number of callbacksqueued on nxtlist.blimit: Thisfieldcontainsthemaximumnumberofcallbacks that may be invoked at a time. Thislimitation improves system responsiveness underheavy load.dynticks: This field references the rcu_dynticksstructure for the corresponding CPU, which isdescribed in Section D.3.1.1.dynticks_snap: This field contains a past valueof dynticks->dynticks, which is used to detectwhen a CPU passes through a dynticksidle state when this CPU happens to be in anirq handler each time that force_quiescent_state() checks it.dynticks_nmi_snap: This field contains a pastvalue of dynticks->dynticks_nmi, which isused to detect when a CPU passes through adynticks idle state when this CPU happens tobe in an NMI handler each time that force_quiescent_state() checks it.dynticks_fqs: This field counts the number oftimes that some other CPU noted a quiescentstateonbehalfoftheCPUcorrespondingtothisrcu_data structure due to its being in dynticksidlemode.offline_fqs: This field counts the number of timesthat some other CPU noted a quiescent state onbehalf of the CPU corresponding to this rcu_data structure due to its being offline.Quick Quiz D.25: So some poor CPU hasto note quiescent states on behalf of each andevery offline CPU? Yecch! Won’t that result inexcessive overheads in the not-uncommon caseof a system with a small number of CPUs buta large value for NR_CPUS?resched_ipi: This field counts the number of timesthat a reschedule IPI is sent to the correspondingCPU. Such IPIs are sent to CPUs that failto report passing through a quiescent states ina timely manner, but are neither offline nor indynticks idle state.n_rcu_pending: This field counts the number ofcallstorcu_pending(), whichiscalledonceperjiffy on non-dynticks-idle CPUs.n_rcu_pending_force_qs: This field holds athreshold value for n_rcu_pending. Ifn_rcu_pending reaches this threshold, thatindicates that the current grace period has extendedtoo long, so force_quiescent_state()is invoked to expedite it.D.3.1.4 RCU Global StateThe rcu_state structure contains RCU’s globalstate for each instance of RCU (rcu and rcu bh). Itincludes fields relating to the hierarchy of rcu_nodestructures, including thenode array itself, thelevelarray that contains pointers to the levels of thehierarchy, the levelcnt array that contains thecount of nodes at each level of the hierarchy, thelevelspread arraythatcontains thenumberofchildrenper node for each level of the hierarchy, and therda array of pointer to each of the CPU’s rcu_datastructures. The rcu_state structure also containsa number of fields coordinating various details ofthe current grace period and its interaction withother mechanisms (signaled, gpnum, completed,onofflock,fqslock,jiffies_force_qs,n_force_qs, n_force_qs_lh, n_force_qs_ngp, gp_start,jiffies_stall, and dynticks_completed).Each of these fields are described below.node: This field is the array of rcu_node structures,with the root node of the hierarchy beinglocated at ->node[0]. The size of thisarray is specified by the NUM_RCU_NODES C-preprocessor macro, which is computed fromNR_CPUS and CONFIG_RCU_FANOUT as describedin Section D.3.1.5. Note that traversing the->node array starting at element zero has theeffect of doing a breadth-first search of thercu_node hierarchy.level: This field is an array of pointers into thenode array. The root node of the hierarchy isreferenced by ->level[0], the first node of thesecond level of the hierarchy (if there is one) by->level[1], and so on. The first leaf node isreferenced by ->level[NUM_RCU_LVLS-1], and