Parallel Implementation of a Sandpile Model - Oak Ridge National ...

Parallel Implementation of a
Sandpile Model
Nathaniel D. Sizemore
Westminster College
Vickie E. Lynch
Oak Ridge National Laboratory
Fusion Energy Division

Goals for the Project
Sandpile models are useful for modeling a
variety of processes ranging from power
transmission systems to the transport of
particles in plasmas used in fusion reactions.
However, on PCs and workstations, the
simulations often take too long to run. The
goal for the project was to reduce the model’s
run time.
Fusion Energy Division

What is a Sandpile?
• Sandpile models are a subset of cellular
automata (such as Conway’s Life
simulations)
• The simulation runs by applying a set of
rules to each cell over a given number of
time steps.
Fusion Energy Division

How a Sandpile Works
• Particles are “dropped” into the simulation.
• When a cell grows high enough to be
unstable, it falls over, giving part of its mass
to neighboring cells. This is called a flip.
• Flips can cause other flips -- this process is
called an avalanche.
Fusion Energy Division

• Z n = h n - h n±1
• If Z n Z crit then
➛h n = h n - N f
“Rules of the Game”
➛h n ±1 = H n ±1 + N f
Fusion Energy Division

How does it work?
Fusion Energy Division

Parallelization
• Ways to speed up a job
– Work harder (faster machines)
– Work smarter(optimizing)
– Find more workers
• Parallelization helps speed the program up
by using many processors to work on the
problem
Fusion Energy Division

Optimizing the Serial Code
• Reduced costly file I/O by combining
several output files into one
• Loops combined where possible
• Instead of storing the sandpile in several
arrays, a single array of a C++ cell class
was used
• Cell objects contained pointers to their
neighbors, similar to a doubly-linked list
Fusion Energy Division

Going from Serial to Parallel
• Each processor was given L/p of the
sandpile
• MPI was used, so global functions (such as
determining the mass of the sandpile) were
already optimized
Fusion Energy Division

mcurie -- the Cray T3E
Fusion Energy Division

mcurie -- the Cray T3E
• 696 DEC Alpha 450 MHz processing
elements (nodes)
• 900 Mflops/node peak performance
• Distributed memory: 256 Mb/node
Fusion Energy Division

Comparing Serial Algorithms
execution time (min)
120
100
80
60
40
20
BAC vs. NDS Serial Algorithms
0
- 500 1,000 1,500 2,000 2,500 3,000 3,500
Number of cells
BAC serial code
NDS serial code
time steps = 1,000,000
Fusion Energy Division

Run Time Increases Linearly with Number of Time Steps
execution time (min)
120
100
80
60
40
20
Time Steps vs. Execution Time
0
0.00E+00 2.00E+05 4.00E+05 6.00E+05 8.00E+05 1.00E+06 1.20E+06
length of simulation (time steps)
L=3200
BAC serial code
NDS serial code
sandpileP.x (p=5)
Fusion Energy Division

execution time (min)
120
100
80
60
40
20
Serial vs. Parallel
Execution TIme Comparison
0
- 500 1,000 1,500 2,000 2,500 3,000 3,500
number of cells
BAC (on Mac)
NDS Serial (on Mac)
NDS Serial (on T3E)
parallel; p=21
time steps = 1,000,000
Fusion Energy Division

Optimum Number of Processors
execution time
9
8
7
6
5
4
3
2
1
0
Processor Number vs. Execution Time
0 5 10 15 20 25 30 35 40 45
number of processors
L=1000
L=3200
Fusion Energy Division

Conclusions
Parallelization was a very effective means of
improving the run times of sandpile
simulations. However, at some point the
communication between processes actually
degrades performance. A rule of thumb
suggested for this code is to choose p such
that each processor handles 250-350 cells.
Fusion Energy Division

Future Improvements
•Adding tracer particles to track grains from
insertion to ejection
•further optimization of the parallel source
code
•Using PVM to have the program itself
calculate and create the optimum number of
processes for a given sandpile
Fusion Energy Division