Synergy User Manual and Tutorial. - THE CORE MEMORY

Synergy User Manual and Tutorial
Existing Tools for Parallel Processing
The parallel programming systems discussed, PVM, MPI and Linda, are implemented
with libraries of function calls that are coded directly into either C or Fortran source code
and compiled. There are two primary types of communication used: message passing
(PVM and MPI) and tuple space (Linda and Synergy). In message passing a participating
process may send messages to any other process, directly, which is somewhat similar to
inter-process communication (IPC) in the Linux/UNIX operating system. In fact, both
message passing and tuple space systems are implemented with sockets in the
Linux/UNIX environment. A tuple space is a type of distributed shared memory that is
used by participating processes to hold messages. These messages can be posted or
obtained by any of the participants. All of these programs function by the use of
“master” and “worker” designations. The master is generally responsible to break the
task into pieces and to assemble the results. The workers are responsible to complete
their piece of the task. These systems are communicate over computer networks and
typically have some type of middleware to facilitate cooperation between machines, such
as the cluster discussed below.
Computer Clusters
Computer clusters, sometimes referred to as server farms, are groups of connected
computers that form a parallel computer by working together to complete tasks. Clusters
were originally developed in the 1980’s by Digital Equipment Corporation (DEC) to
facilitate parallel computing and file and peripheral device sharing. An example of a
cluster would be a Linux network with some middleware software to implement the
parallelism. Well established cluster systems have procedures to eliminate single point
failures, providing some level of fault tolerance. The four major types of clusters are:
• Director based clusters—one machine directs or controls the behavior of the
cluster and usually implemented to enhance performance
• Two-node clusters—two nodes perform the same part of the task or one serves as
a backup in case the other fails to ensure fault tolerance
• Multi-node clusters—may have tens of clustered machines, which are usually on
the same network
• Massively parallel clusters—may have hundreds or thousands of machines on
many networks
Currently, the fastest supercomputing cluster is Earth Simulator at 35.86 TFlops, which is
15 TFlops faster than the second place machine. The main reason for cluster based
66