Parallel Processing: A KISS Approach - University of North Dakota

Obviously, the development of scientific simulation and modeling software is a very
complex task. Fortunately, many scientific simulation and modeling tasks intended for
execution on a HPCC can be represented by the interacting peer paradigm. Therefore, I
will use the KISS approach to describe the development of two common HPCC software
architectures, have provided code examples for both architectures, and the associated
documentation. I have also provided a utility for implementation of a special class of
HPCC problem and have provided the associated documentation.
2. Background.
Before we can delve into the software development process we must have a basic
understanding of the different computing platforms that exist, the intraprocess
communication requirements, and the intraprocess management styles that are common.
2.1. High performance computing platforms.
Each platform type requires the developer to abide by a particular set of design criteria
and may put limitations on the software’s performance that the software developer must
understand. In keeping with the KISS approach, we will briefly review only 2 classes of
platforms:
• The first class is the classic “supercomputer.” A multiple CPU shared memory
platform. Supercomputers tend to be expensive due to extreme memory
bandwidth requirements. However, supercomputers are very flexible and the
software development is much less constrained.
• The second class includes the cluster, Network Of Workstations (NOWs), or
GRID. Clusters are much less expensive than supercomputers because they use
commodity personal computers linked together with commodity networking.
However, clusters cannot deliver the level of performance of a supercomputer and
have more software development design constraints. Yet, for many scientific
problems, clusters provide an affordable way to achieve high performance
computing. Since the Computational Research Center’s High Performance
Computing Cluster is a cluster, we will assume this class of machine for all
software development covered by this tutorial.
To make the best use of a cluster, the researcher must be able to spread their
workload across the different computers (we will refer to the individual
computers making up a cluster as nodes) making up the cluster. It is this
parallelism that achieves high performance computing. However, when
developing an application for a cluster, the communication costs associated with
having to spread the computations across many different nodes and the
synchronization of the computations across those different nodes introduces
several software design constraints that are unique to clusters.
4