Is Parallel Programming Hard, And, If So, What Can You Do About It?

12.2. MEMORY BARRIERS 12712.2.10.2 Implicit Memory BarriersThereareacoupleoftypesofimplicitmemorybarriers,socalledbecausetheyareembeddedintolockingprimitives:1. LOCK operations and2. UNLOCK operations.LOCK Operations A lock operation acts as aone-way permeable barrier. It guarantees that allmemory operations after the LOCK operation willappear to happen after the LOCK operation withrespect to the other components of the system.Memory operations that occur before a LOCK operationmay appear to happen after it completes.A LOCK operation should almost always bepaired with an UNLOCK operation.UNLOCK Operations Unlock operations alsoact as a one-way permeable barrier. It guaranteesthat all memory operations before the UNLOCK operationwill appear to happen before the UNLOCKoperation with respect to the other components ofthe system.Memory operations that occur after an UNLOCKoperationmayappeartohappenbeforeitcompletes.LOCK and UNLOCK operations are guaranteedto appear with respect to each other strictly in theorder specified.The use of LOCK and UNLOCK operations generallyprecludes the need for other sorts of memorybarrier (but note the exceptions mentioned in thesubsection ”MMIO write barrier”).Quick Quiz 12.10: What effect does the followingsequence have on the order of stores to variables“a” and “b”?a = 1;b = 1;12.2.10.3 What May Not Be AssumedAbout Memory Barriers?There are certain things that memory barriers cannotguarantee outside of the confines of a given architecture:1. There is no guarantee that any of the memoryaccesses specified before a memory barrier willbecomplete bythecompletionofamemorybarrierinstruction; the barrier can be consideredto draw a line in that CPU’s access queue thataccesses of the appropriate type may not cross.2. There is no guarantee that issuing a memorybarrier on one CPU will have any direct effecton another CPU or any other hardware in thesystem. The indirect effect will be the orderin which the second CPU sees the effects ofthe first CPU’s accesses occur, but see the nextpoint.3. There is no guarantee that the a CPU will seethe correct order of effects from a second CPU’saccesses, even if the second CPU uses a memorybarrier, unless the first CPU also uses amatchingmemorybarrier(seethesubsectionon”SMP Barrier Pairing”).4. There is no guarantee that some interveningpiece of off-the-CPU hardware 7 will not reorderthe memory accesses. CPU cache coherencymechanisms should propagate the indirect effectsof a memory barrier between CPUs, butmight not do so in order.12.2.10.4 Data Dependency BarriersThe usage requirements of data dependency barriersare a little subtle, and it’s not always obvious thatthey’re needed. To illustrate, consider the followingsequence of events, with initial values {A=1,B=2,C=3,P=&A,Q=&C}:CPU 1 CPU 2B = 4;P = &B;Q = P;D = *Q;There’s a clear data dependency here, and itwould seem intuitively obvious that by the end ofthe sequence, Q must be either &A or &B, and that:(Q == &A) implies (D == 1)(Q == &B) implies (D == 4)Counter-intuitive though it might be, it is quitepossible that CPU 2’s perception of P might be updatedbefore its perception of B, thus leading to thefollowing situation:(Q == &B) and (D == 2) ????7 This is of concern primarily in operating-system kernels.For more information on hardware operations and memoryordering, see the files pci.txt, DMA-mapping.txt, andDMA-API.txt in the Documentation directory in the Linuxsource tree [Tor03].