Is Parallel Programming Hard, And, If So, What Can You Do About It?

12.2. MEMORY BARRIERS 125then be used:*A = 5;x = *D;but this might show up as either of the followingtwo sequences:STORE *A = 5, x = LOAD *Dx = LOAD *D, STORE *A = 5the second of which will almost certainly result inamalfunction, sinceitsettheaddressafter attemptingto read the register.12.2.9 GuaranteesThere are some minimal guarantees that may be expectedof a CPU:1. Onany given CPU, dependentmemory accesseswill be issued in order, with respect to itself.This means that for:Q = P; D = *Q;the CPU will issue the following memory operations:Q = LOAD P, D = LOAD *Qand always in that order.2. Overlapping loads and stores within a particularCPU will appear to be ordered within thatCPU. This means that for:a = *X; *X = b;the CPU will only issue the following sequenceof memory operations:a = LOAD *X, STORE *X = bAnd for:*X = c; d = *X;the CPU will only issue:STORE *X = c, d = LOAD *X(Loads and stores overlap if they are targettedat overlapping pieces of memory).3. A series of stores to a single variable will appearto all CPUs to have occurred in a single order,thought this order might not bepredictablefrom the code, and in fact the order might varyfrom one run to another.And there are a number of things that must ormust not be assumed:1. It must not be assumed that independent loadsandstoreswillbeissuedintheordergiven. Thismeans that for:X = *A; Y = *B; *D = Z;we may get any of the following sequences:X = LOAD *A, Y = LOAD *B, STORE *D = ZX = LOAD *A, STORE *D = Z, Y = LOAD *BY = LOAD *B, X = LOAD *A, STORE *D = ZY = LOAD *B, STORE *D = Z, X = LOAD *ASTORE *D = Z, X = LOAD *A, Y = LOAD *BSTORE *D = Z, Y = LOAD *B, X = LOAD *A2. It must be assumed that overlapping memoryaccesses may be merged or discarded. Thismeans that for:X = *A; Y = *(A + 4);we may get any one of the following sequences:X = LOAD *A; Y = LOAD *(A + 4);Y = LOAD *(A + 4); X = LOAD *A;{X, Y} = LOAD {*A, *(A + 4) };And for:*A = X; Y = *A;we may get either of:STORE *A = X; Y = LOAD *A;STORE *A = Y = X;12.2.10 What Are Memory Barriers?As can be seen above, independent memory operationsare effectively performed in random order, butthis can be a problem for CPU-CPU interaction andfor I/O. What is required is some way of interveningto instruct the compiler and the CPU to restrict theorder.