Is Parallel Programming Hard, And, If So, What Can You Do About It?

282 APPENDIX F. ANSWERS TO QUICK QUIZZESexcept that it absolutely must distinguish perfectlybetween non-zero and zero values. However, onceit has read out a zero value, it must act to keepthe value at zero until it has taken some action toprevent subsequent threads from gaining access tothe device being removed. See Section 4.5.Quick Quiz 4.6:But doesn’t the ++ operator produce an x86 addto-memoryinstruction? And won’t the CPU cachecause this to be atomic?Answer:Although the ++ operator could be atomic, thereis no requirement that it be so. Furthermore, theACCESS_ONCE() primitive forces most version ofgcc to load the value to a register, increment theregister, then store the value to memory, which isdecidedly non-atomic.Quick Quiz 4.7:The 8-figure accuracy on the number of failuresindicates that you really did test this. Why wouldit be necessary to test such a trivial program,especially when the bug is easily seen by inspection?Answer:There are no trivial parallel programs, and mostdaysIamnotsosurethattherearetrivialsequentialprograms, either.No matter how small or simple the program, ifyou haven’t tested it, it does not work. And even ifyou have tested it, Murphy says there are at least afew bugs still lurking.Furthermore, while proofs of correctness certainlydo have their place, they never will replace testing,including the counttorture.h test setup used here.After all, proofs can have bugs just as easily has canprograms!Quick Quiz 4.8:Why doesn’t the dashed line on the x axis meet thediagonal line at y = 1?Answer:Because of the overhead of the atomic operation.The dashed line on the x axis represents the overheadof a single non-atomic increment. After all,an ideal algorithm would not only scale linearly, itwould also incur no performance penalty comparedto single-threaded code.This level of ideality may seem severe, but if it isgood enough for Linus Torvalds, it is good enoughfor you.Quick Quiz 4.9:But atomic increment is still pretty fast. Andincrementing a single variable in a tight loop soundspretty unrealistic to me, after all, most of theprogram’s execution should be devoted to actuallydoing work, not accounting for the work it has done!Why should I care about making this go faster?Answer:In many cases, atomic increment will in fact be fastenough for you. In those cases, you should by allmeans use atomic increment. That said, there aremany real-world situations where more elaboratecounting algorithms are required. The canonicalexample of such a situation is counting packets andbytes in highly optimized networking stacks, whereit is all too easy to find much of the execution timegoing into these sorts of accounting tasks, especiallyon large multiprocessors.In addition, counting provides an excellent viewof the issues encountered in shared-memory parallelprograms.Quick Quiz 4.10:But why can’t CPU designers simply ship theoperation to the data, avoiding the need to circulatethe cache line containing the global variable beingincremented?Answer:It might well be possible to do this in some cases.However, there are a few complications:1. If the value of the variable is required, then thethread will be forced to wait for the the operationto be shipped to the data, and then for theresult to be shipped back.2. If the atomic increment must be ordered withrespect to prior and/or subsequent operations,then the thread will be forced to wait for theoperation to be shipped to the data, and for anindication that the operation completed to beshipped back.3. Shipping operations among CPUs will likely requiremore signals, which will consume more diearea and more electrical power.