Is Parallel Programming Hard, And, If So, What Can You Do About It?

D.3. HIERARCHICAL RCU CODE WALKTHROUGH 227D.3.8.2 Handling Offline and Holdout CPUsFigure D.50 shows the code for rcu_implicit_offline_qs(), which checks for offline CPUs andforcing online holdout CPUs to enter a quiescentstate.Line 3 checks to see if the specified CPU is offline,and, if so, line 4 increments statistical counter ->offline_fqs (which is used only for tracing), andline 5 returns non-zero to indicate that the CPU isin an extended quiescent state.Otherwise, the CPU is online, not in dynticks-idlemode (or this function would not have been calledin the first place), and has not yet passed througha quiescent state for this grace period. Line 7checks to see if the holdout CPU is the current runningCPU, and, if not, line 8 sends the holdoutCPU a reschedule IPI. Otherwise, line 10 sets theTIF_NEED_RESCHED flag for the current task, forcingthe current CPU into the scheduler. In either case,the CPU should then quickly enter a quiescent state.Line 11 increments statistical counter resched_ipi,which is again used only for tracing. Finally, line 12returns zero to indicate that the holdout CPU is stillrefusing to pass through a quiescent state.D.3.8.3 Scanning for Holdout CPUsFigure D.51 shows the code for rcu_process_dyntick(), which scans the leaf rcu_node structuresin search of holdout CPUs, as illustrated bythe blue arrow in Figure D.52. It invokes the functionpassed in through argument f on each suchCPU’s rcu_data structure, and returns non-zero ifthegraceperiodspecifiedbythelastcompargumenthas ended.Lines 13 and 14 acquire references to the first andthe last leaf rcu_node structures, respectively. Eachpass through the loop spanning lines 15-38 processesone of the leaf rcu_node structures.Line 16 sets the local variable mask to zero. Thisvariable will be used to accumulate the CPUs withinthe current leaf rcu_node structure that are in extendedquiescent states, and can thus be reportedas such. Line 17 acquires the current leaf rcu_nodestructure’s lock, and line 18 checks to see if the currentgrace period has completed, and, if so, line 19releases the lock and line 20 returns non-zero. Otherwise,line 22 checks for holdout CPUs associatedwith this rcu_node structure, and, if there are none,line 23 releases the lock and line 24 restarts the loopfrom the beginning on the next leadrcu_node structure.Execution reaches line 26 if there is at least oneholdout CPU associated with this rcu_node struc-1 static int2 rcu_process_dyntick(struct rcu_state *rsp,3 long lastcomp,4 int (*f)(struct rcu_data *))5 {6 unsigned long bit;7 int cpu;8 unsigned long flags;9 unsigned long mask;10 struct rcu_node *rnp_cur;11 struct rcu_node *rnp_end;1213 rnp_cur = rsp->level[NUM_RCU_LVLS - 1];14 rnp_end = &rsp->node[NUM_RCU_NODES];15 for (; rnp_cur < rnp_end; rnp_cur++) {16 mask = 0;17 spin_lock_irqsave(&rnp_cur->lock, flags);18 if (rsp->completed != lastcomp) {19 spin_unlock_irqrestore(&rnp_cur->lock, flags);20 return 1;21 }22 if (rnp_cur->qsmask == 0) {23 spin_unlock_irqrestore(&rnp_cur->lock, flags);24 continue;25 }26 cpu = rnp_cur->grplo;27 bit = 1;28 for (; cpu grphi; cpu++, bit rda[cpu]))31 mask |= bit;32 }33 if (mask != 0 && rsp->completed == lastcomp) {34 cpu_quiet_msk(mask, rsp, rnp_cur, flags);35 continue;36 }37 spin_unlock_irqrestore(&rnp_cur->lock, flags);38 }39 return 0;40 }Figure D.51: Scanning for Holdout CPUs