COPYRIGHT 2008, PRINCETON UNIVERSITY PRESS

370 chapter 14The problem affects parallelism: You must analyze your problem in terms ofhow and when data are used, how much computation is required for eachuse, and the type of problem architecture:• Perfectly parallel: The same application is run simultaneously on differentdata sets, with the calculation for each data set independent (e.g., runningmultiple versions of a Monte Carlo simulation, each with different seeds,or analyzing data from independent detectors). In this case it would bestraightforward to parallelize with a respectable performance to be expected.• Fully synchronous: The same operation applied in parallel to multiple partsof the same data set, with some waiting necessary (e.g., determining positionsand velocities of particles simultaneously in a molecular dynamics simulation).Significant effort is required, and unless you balance the computationalintensity, the speedup may not be worth the effort.• Loosely synchronous: Different processors do small pieces of the computationbut with intermittent data sharing (e.g., diffusion of groundwater fromone location to another). In this case it would be difficult to parallelize andprobably not worth the effort.• Pipeline parallel: Data from earlier steps processed by later steps, with someoverlapping of processing possible (e.g., processing data into images and theninto animations). Much work may be involved, and unless you balance thecomputational intensity, the speedup may not be worth the effort.14.12.1 High-Level View of Message PassingAlthough it is true that parallel computing programs may become very complicated,the basic ideas are quite simple. All you need is a regular programminglanguage like C or Fortran, plus four communication statements: 3send: One processor sends a message to the network. It is not necessary toindicate who will receive the message, but it must have a name.receive: One processor receives a message from the network. This processordoes not have to know who sent the message, but it has to know the message’sname.myid: An integer that uniquely identifies each processor.numnodes: An integer giving the total number of nodes in the system.Once you have made the decision to run your program on a computer cluster,you will have to learn the specifics of a message-passing system such as MPI(Appendix D). Here we give a broader view. When you write a message-passingprogram, you intersperse calls to the message-passing library with your regularFortran or C program. The basic steps are1. Submit your job from the command line or a job control system.3 Personal communication, Yuefan Deng.−101COPYRIGHT 2008, PRINCET O N UNIVE R S I T Y P R E S SEVALUATION COPY ONLY. NOT FOR USE IN COURSES.ALLpup_06.04 — 2008/2/15 — Page 370