Magellan Final Report - Office of Science - U.S. Department of Energy

Magellan Final Report
Table 9.6: Comparison of STAR performance and cost across different Amazon instance types and Magellan
Instance type m1.large x2.4xlarge Magellan
cores 2 8 8
EC2 units/core 2 3.25 -
EC2 units/instance 4 26 -
Cost ($)/instance hour $0.38 $2.28 -
Cost ($)/hour/EC2 unit $0.10 $0.09 -
Wall hours/job 2.119 1.242 1.269
CPU hours/job 1.500 1.240 1.263
Cost ($)/job 0.403 0.354 -
cluster was 2 to 3 times slower than on bare metal. Due to the performance hit and the need to move data
in and out of the virtual cluster, which did not have access to the NERSC Global File System, the user did
not consider this a resource environment worth pursuing at the current time.
9.5.5 EnergyPlus
Another type of application that was tested in the “Little Magellan” environment was a code called EnergyPlus
(E+) that does building simulations, such as estimating the energy needs of a building before it
is built. The user had been tasked with improving the performance, and used OpenMP with great results
when comparing serial with 1, 2, 4 and 8 OpenMP threads. Code ran in 10 different configurations (each
of them 1, 2, 4 and 8 threads) on both virtual cluster and bare metal. Results for 1, 2 and 4 threads were
identical. The results for 8 threads were 1015% slower.
9.5.6 LIGO
LIGO was another application that used the Magellan testbed through the “Little Magellan” setup. The
application is described in greater detail in Section 11.2.3. LIGO users stated that they saw no difference
between running through the serial queue on Magellan and on the virtual machines. The nature of the LIGO
workload, i.e., minimal data and limited communication, makes it a suitable candidate to use virtualized
cloud resources. There was negligible difference in CPU performance between bare metal and virtualized
resources for this application.
9.6 Workload Analysis
Our benchmarking results show that applications with minimal communication and I/O do well in cloud
environments. In order to further understand the range of HPC applications that might benefit from cloud
environments, we analyzed the workflows of researchers who were using the Magellan batch queue system at
NERSC.
Workload profile information was collected using the Integrated Performance Monitoring (IPM) analysis
infrastructure. IPM is a low-overhead profiling infrastructure that provide a high-level report on the execution
of a job. IPM reports hardware counters data, MPI function timings, and memory usage. It provides a lowoverhead
means to generate scaling studies or performance data. IPM is available to users on NERSC
systems using modules.
On the Magellan systems, we force IPM profiling information to be collected for every job. The user gets
normal output at the end of the job, and a log file is written to a shared space. To aid with the analysis,
we built a generic infrastructure using MongoDB, also a cloud technology. MongoDB is a scalable, open
source, document-oriented data store that supports dynamic schemas. An existing parsing script in IPM
was modified to output a JSON-style document that was then used to populate MongoDB.
78