Is Parallel Programming Hard, And, If So, What Can You Do About It?

Chapter 5Partitioning and SynchronizationDesignThis chapter describes how to design software totake advantage of the multiple CPUs that are increasinglyappearing in commodity systems. It doesthis by presenting a number of idioms, or “designpatterns” that can help you balance performance,scalability, and realtime response. As noted in earlierchapters, the most important decision you willmake when creating parallel software is how to carryout the partitioning. Correctly partitioned problemslead to simple, scalable, and high-performance solutions,while poorly partitioned problems result inslow and complex solutions.@@@ roadmap @@@P5P1P25.1 Partitioning ExercisesP4P3This section uses a pair of exercises (the classic DiningPhilosophersproblemandadouble-endedqueue)to demonstrate the value of partitioning.5.1.1 Dining Philosophers ProblemFigure 5.1 shows a diagram of the classic DiningPhilosophersproblem[Dij71]. Thisproblemfeaturesfive philosophers who do nothing but think and eata “very difficult kind of spaghetti” which requirestwo forks to eat. A given philosopher is permittedto use only the forks to his or her immediate rightand left, and once a philosopher picks up a fork, heor she will not put it down until sated.Theobjectistoconstructanalgorithmthat, quiteliterally, prevents starvation. One starvation scenariowould be if all of the philosophers picked uptheir leftmost forks simultaneously. Because none ofthem would put down their fork until after they ate,and because none of them may pick up their secondfork until at least one has finished eating, they allstarve.Figure 5.1: Dining Philosophers ProblemDijkstra’ssolutionusedaglobalsemaphore, whichworks fine assuming negligible communications delays,an assumption that has become invalid in theensuingdecades. Therefore, recentsolutionsnumberthe forks as shown in Figure 5.2. Each philosopherpicks up the lowest-numbered fork next to his or herplate, then picks up the highest-numbered fork. Thephilosopher sitting in the uppermost position in thediagram thus picks up the leftmost fork first, thenthe rightmost fork, while the rest of the philosophersinstead pick up their rightmost fork first. Becausetwoofthephilosopherswillattempttopickupfork1first, and because only one of those two philosopherswill succeed, there will be five forks available to fourphilosophers. At least one of these four will be guaranteedtohavetwoforks,andthusbeabletoproceedeating.