Magellan Final Report - Office of Science - U.S. Department of Energy

Magellan Final Report
As message size increases, overhead related to latency and connection instantiation become less significant.
The results shown in Figure 9.12 demonstrate that while the raw hardware cluster has neared its practical
maximum bandwidth at 16 KB, the virtual cluster is still 50 kbps below its average at 2 MB. Of particular
note, variability in the results on the virtual cluster was incredibly high; with a 2 MB message size, a
maximum reported bandwidth value of 181.74 kbps and a minimum of 32.24 kbps were reported.
The difference between clusters in zero-length message bandwidth was also of concern. While an average
of 329,779 messages passed between ranks every second on the raw hardware cluster, the virtual cluster could
only manage an average of 21,793 per second. This is a strong indictment of the virtual network model as
it pertains to MPI workload performance.
Figure 9.13: Phloem: Selected MPI method tests, average performance ratio (16 ranks, 8 ranks per node).
Figure 9.13 shows the performance penalties associated with individual one-to-many and many-to-many
MPI calls, as measured by the Phloem mpiBench Bcast utility, run on a 16-rank 2-node cluster to maintain
relevance with regard to the above SQMR tests. These are the averages of the per-method virtual cluster
over raw hardware cluster time-to-solution ratios, chosen because they remain remarkably consistent across
varying message sizes. This demonstrates that, in aggregation, the high point-to-point latency illustrated by
the SQMR tests will result in a consistently reproducible multiplier for general parallel coding methods.
Figure 9.14: Phloem: Selected MPI method tests, average performance ratio across 1000 trials.
Figure 9.14 shows the scaling characteristics of the virtual cluster as compared to raw hardware, again
62