Iraqi Kurdistan All in the Timing

As a rough simplification, the modern
supercomputer is divided up into nodes. Each
node may have multiple processors which are
the hearts of the node. Each processor has
access to memory on its node – but does not
have access to the memory on the other nodes.
Communication of information can occur
between processors within or across nodes.
Communication across nodes takes place
through a high-speed network, but is usually
considerably slower than communication
within a node. Data storage has to be accessible
from all nodes in order to input and output
data. In order to benefit from a supercomputer,
the programs need to utilise the resources
of multiple nodes in parallel. The network
connects the nodes to make larger parallel
computer clusters.
Node 1 Node 2 Node 3 Node n
Memory Memory Memory Memory
p1 p2 p3 pn p1 p2 p3 pn p1 p2 p3 pn p1 p2 p3 pn
processors processors processors processors
NETWORK
independent sub-problems, and some
sort of communication must take place
between nodes. A common example
is a problem where each processor is
assigned an independent computation,
but where the results from all processors
must be combined to form the solution.
A simple example of such a problem
is the evaluation of an area under a
curve. Numerically, this is usually
solved by dividing the area up into small
rectangles: computing the area for each
rectangle and then summing together all
individual rectangle areas. Table 1 shows
an excerpt from a C program performing
this computation on a single processor.
Basically the programmer has used a loop
to perform the computation of the area of
each small rectangle, starting with the first
rectangle and sequentially adding together
all rectangles to form the total area.
Solving the same problem in a parallel
way on a supercomputer with many
processors requires some tools not
present in conventional programming
languages. Although a large number of
computer languages have been designed
and implemented for this purpose, one
of the most popular approaches consists
of using a conventional programming
language, but extending the language
Table 1: An excerpt from a program written in
the C language to compute the area under a
curve defined by the function f(x)=4/(1+x 2 ) in
the interval [0,1] using the rectangle (midpoint)
method; n is the number of rectangles used and
h is the width of each rectangle. The third line is
the start of a loop doing the area computation
and i is a variable looping from 1 and up to n. The
expression i++ means that i is incremented by
one for each time the loop is executed. The last
line computes the final answer, which in this case
is the first few digits of the famous number, pi.
with a small collection of functions
capable of sending and synchronising
data or messages between processors, a
so-called message passing interface or
commonly referred to as MPI. It turns
out that by using this simple approach a
large number of parallel problems can be
efficiently programmed.
The example given in the first table
can be parallelised by the computer
n=1000000;
h=1/n;
for (i = 1; i