Parallel Processing: A KISS Approach - University of North Dakota

More documents

Recommendations

Info

Table of Contents Chapter 1: Introduction to Parallel Processing .............................................................3 Chapter 2: CRCRAM ..................................................................................................16 Chapter 3: Serial Template #1 (serial) ........................................................................21 Chapter 4: Master-Slave Template #1 (MST) ............................................................51 Chapter 5: Coordinating Peers Template #1 (PTP) ....................................................60 Chapter 6: Serial Template #2 (serial_2) ....................................................................80 Chapter 7: Master-Slave Template #2 (MST_2) ........................................................90 Chapter 8: Coordinating Peers Template #2 (PTP_2) ..............................................119 2
Chapter 1: Introduction to Parallel Processing 1. Introduction. The intent of this tutorial is to provide design heuristics and sample programs to help UND researchers port their compute intensive research to the Computational Research Center’s (CRC) High Performance Computing Cluster (HPCC) also known as “Shale”. What is Computational Science why it is important for UND? Computational Science is the application of mathematical simulations and models to understand complex phenomena. When combined with high performance computing, the use of simulations and models allow the scientist to manipulate the phenomena in ways that are beyond any reach using traditional scientific methods. Thus, according to the National Science Foundation, Computational Science which was once viewed as a mere adjunct to theoretical (1 st branch) and experimental (2 nd branch) scientific approaches, is now emerging as a principal means of scientific research and considered to be the 3 rd branch of scientific endeavor. It is an excepted fact that without access to high-end computational capability, many important discoveries in the fields of chemistry, biology, oceanography, meteorology, and many others could not have been made. Finally, according to heralded science journalist George Johnson “As research on so many fronts is becoming increasingly dependant on computation, all science, it seems, is becoming computer science.” Unfortunately, like many modern scientific disciplines, Computational Science has become a diverse field and it would take volumes to properly describe each domain. Even scientific computing, the domain that focuses on simulation and modeling software development, includes many paradigms of software development, including: • Iterative parallelism – where a program has several identical processes wrapped in a loop. • Recursive parallelism – where a program uses recursion to work on different parts of the data. • Producers and Consumers – where processes communicate or share data. • Clients and Servers – where clients request services and servers provide services. • Interacting peers – distributed systems that support many other paradigms: 1. Master-slave – where 1 machine coordinates the processing. 2. Coordinating peers – where interacting processes periodically exchange data (without any forced timing constraints). 3. Heartbeat algorithms – where interacting processes periodically exchange data (with forced timing constraints). 4. Pipeline algorithms – where data flows from 1 process to another. 5. Broadcast algorithms – used for decentralized decision making. 6. Token-passing algorithms – used for decentralized decision making. 7. Probes and receives – to disseminate and gather data from trees and graphs. 8. Replicated server processes – used to manage multiple instances of resources. 3
Page 1: Parallel Processing: A KISS Approac
Page 5 and 6: 2.2 PVM vs MPI. When developing an
Page 7 and 8: independently from every other task
Page 9 and 10: We now have what could be considere
Page 11 and 12: only required a single pass of the
Page 13 and 14: will reduce the time and cost of de
Page 15 and 16: consider are those that specify a s
Page 17 and 18: fi . ~/.bashrc # User specific envi
Page 19 and 20: 1. “#PBS -S” Sets the execution
Page 21 and 22: Chapter 3: Serial Template #1 (seri
Page 23 and 24: serial: $(CC) $@.cpp $(OFLAG) $(INC
Page 25 and 26: * Verbose is used to display log me
Page 27 and 28: ***********************************
Page 29 and 30: * these with their own variable dec
Page 31 and 32: Chapter 4: Master-Slave Template #1
Page 33 and 34: 1 R K (2) 1 1 ' j, i = ∑∑R( j+
Page 35 and 36: alancing in particular), and to str
Page 37 and 38: $(CC) $@.cpp $(OFLAG) $(INCP) $(LIB
Page 39 and 40: 7. Source code follows: /**********
Page 41 and 42: } if (masterNodeFile == NULL) { pri
Page 43 and 44: } /********************************
Page 45 and 46: ***********************************
Page 47 and 48: ***********************************
Page 49 and 50: * modify this code to fit their app
Page 51 and 52: } } M[j][i] = 0.0; // Set location
Page 53 and 54:
***********************************
Page 55 and 56:
SPAWN_PVM_NODES(); /***************
Page 57 and 58:
Accumlate user data. ACCUMULATE_M_B
Page 59 and 60:
} } } PVM_SEND(TID[j*nodesWanted+n]
Page 61 and 62:
Hot spot Cold spot Figure 7. This p
Page 63 and 64:
Note that it is common (required?)
Page 65 and 66:
$(CC) $@.cpp $(OFLAG) $(INCP) $(LIB
Page 67 and 68:
Hot spot Cold spot Figure 11. If we
Page 69 and 70:
***********************************
Page 71 and 72:
* 1. Shuts down MPI. */ /* */ /* !N
Page 73 and 74:
***********************************
Page 75 and 76:
int i; for (i = 0; i < cellWidth; i
Page 77 and 78:
} if (myID == 0) { MPI_Recv(haloTop
Page 79 and 80:
} for (i = 0; i < cellWidth; i++) {
Page 81 and 82:
' 1 ' 1 Vx i = ( Vxi + * t) and Vy
Page 83 and 84:
***********************************
Page 85 and 86:
***********************************
Page 87 and 88:
} difference = 0.0; for (j = 0; j <
Page 89 and 90:
Records the stopping time of the pr
Page 91 and 92:
#define Width 4 // Width of problem
Page 93 and 94:
PATH=$PATH:$HOME/bin export PATH #
Page 95 and 96:
4. The mstSlave_2.cpp file: The mst
Page 97 and 98:
template was designed to manage, yo
Page 99 and 100:
#define slavePath "/home/rmarsh/MST
Page 101 and 102:
} nodesWanted++; } // Write PVMhost
Page 103 and 104:
int cc, j, n; int tid; // Initializ
Page 105 and 106:
double xForce, yForce; double PARTI
Page 107 and 108:
* terminate! */ /******************
Page 109 and 110:
} fscanf(data, "%lf", &DATA[j][i]);
Page 111 and 112:
* modify this code to fit their app
Page 113 and 114:
int iteration; double newDifference
Page 115 and 116:
* data sent to them for processing
Page 117 and 118:
} /********************************
Page 119 and 120:
Chapter 8: Coordinating Peers Templ
Page 121 and 122:
Even though we have partitioned the
Page 123 and 124:
define user definable parameters th
Page 125 and 126:
6. Source code follows: /**********
Page 127 and 128:
* !No changes should be made to thi
Page 129 and 130:
***********************************
Page 131 and 132:
double localDifference, newDifferen
Page 133 and 134:
} if (convergence < 0.00001 || iter
show all

Parallel Processing: A KISS Approach - University of North Dakota

Create successful ePaper yourself

Delete template?

Save as template?