Is Parallel Programming Hard, And, If So, What Can You Do About It?

PrefaceThe purpose of this book is to help you understandhow to program shared-memory parallel machineswithout risking your sanity. 1 By describingthe algorithms and designs that have worked well inthe past, we hope to help you avoid at least someof the pitfalls that have beset parallel projects. Butyou should think of this book as a foundation onwhich to build, rather than as a completed cathedral.You mission, if you choose to accept, is to helpmake further progress in the exciting field of parallelprogramming,progressthatshouldintimerenderthis book obsolete. Parallel programming is not ashard as it is reputed, and it is hoped that this bookmakes it even easier for you.Thisbookfollowsawatershedshiftintheparallelprogrammingfield, from being primarily the domainof science, research, and grand-challenge projects tobeing primarily an engineering discipline. In presentingthis engineering discipline, this book will examinethespecificdevelopmenttaskspeculiartoparallelprogramming, and describe how they may bemost effectively handled, and, in some surprisinglycommon special cases, automated.This book is written in the hope that presentingthe engineering discipline underlying successfulparallel-programming projects will free a new generationof parallel hackers from the need to slowly andpainstakingly reinvent old wheels, instead focusingtheir energy and creativity on new frontiers. Althoughthe book is intended primarily for self-study,it is likely to be more generally useful. It is hopedthat this book will be useful to you, and that the experienceof parallel programming will bring you asmuch fun, excitement, and challenge as it has providedthe authors over the years.1 Or, perhaps more accurately, without much greater riskto your sanity than that incurred by non-parallel programming.Which, come to think of it, might not be saying allthat much. Either way, Appendix A discusses some importantquestions whose answers are less intuitive in parallel programsthan they are in sequential program.