Is Parallel Programming Hard, And, If So, What Can You Do About It?

Chapter 15Conflicting Visions of the FutureThis chapter presents some conflicting visions ofthe future of parallel programming. It is not clearwhich of these will come to pass, in fact, it is notclear that any of them will. They are neverthelessimportant because each vision has its devoted adherents,and if enough people believe in somethingfervently enough, you will need to deal with thatthing’s existence in the form of its influence on thethoughts, words, and deeds of its adherents. Besideswhich, it is entirely possible that one or moreof these visions will actually come to pass.Therefore, the following sections give an overviewof transactional memory, shared-memory parallelfunctional programming, and process-based parallelfunctional programming.15.1 Transactional MemoryThe idea of using transactions outside of databasesgoes back many decades [Lom77], with the key differencebetween database and non-database transactionsbeing that non-database transactions dropthe “D” in the “ACID” properties defining databasetransactions. The idea of supporting memory-basedtransactions, or “transactional memory” (TM), inhardware is more recent [HM93], but unfortunately,support for such transactions in commodity hardwarewas not immediately forthcoming, despiteother somewhat similar proposals being put forward[SSHT93]. Not long after, Shavit and Touitouproposed a software-only implementation of transactionalmemory (STM) that was capable of runningon commodity hardware, give or take memoryorderingissues. This proposal languished for manyyears, perhaps due to the fact that the research community’sattention was absorbed by non-blockingsynchronization (see Section 12.3).But by the turn of the century, TM started receivingmore attention [MT01, RG01], and by themiddle of the decade, the level of interest can onlybe termed “incandescent” [Her05, Gro07], despite afew voices of caution [BLM05, MMW07].The basic idea behind TM is to execute a sectionofcodeatomically, sothatotherthreadsseenointermediatestate. As such, the semantics of TM couldbe implemented by simply replacing each transactionwith a recursively acquireable global lock acquisitionand release, albeit with abysmal performanceand scalability. Much of the complexity inherent inTMimplementations, whetherhardwareorsoftware,is efficiently detecting when concurrent transactionscan safely run in parallel. Because this detectionis done dynamically, conflicting transactions can beaborted or “rolled back”, and in some implementations,this failure mode is visible to the programmer.Because transaction roll-back is increasingly unlikelyas transaction size decreases, TM might becomequite attractive for small memory-based operations,such as linked-list manipulations used forstacks, queues, hash tables, and search trees. However,it is currently much more difficult to make thecase for large transactions, particularly those containingnon-memoryoperationssuchasI/Oandprocesscreation. The following sections look at currentchallenges to the grand vision of “TransactionalMemory Everywhere” [McK09b].15.1.1 I/O OperationsOne can execute I/O operations within a lock-basedcritical section, and, at least in principle, fromwithin an RCU read-side critical section. What happenswhen you attempt to execute an I/O operationfrom within a transaction?The underlying problem is that transactions maybe rolled back, for example, due to conflicts.Roughly speaking, this requires that all operationswithin any given transaction be idempotent, so thatexecuting the operation twice has the same effect asexecuting it once. Unfortunately, I/O is in generalthe prototypical non-idempotent operation, making