Magellan Final Report - Office of Science - U.S. Department of Energy

Chapter 5
Magellan Testbed
As part of the Magellan project, a dedicated, distributed testbed was deployed at Argonne and NERSC. The
two sites architected, procured, and deployed their testbed components separately, although the resources
were chosen to complement each other. Deploying a testbed (versus acquiring services on existing commercial
cloud systems) provided the flexibility necessary to address the Magellan research questions. Specifically our
hardware and software were configured to cater to scientific application needs, which are different from the
typical workloads that run on commercial cloud systems. For example, the ability to adjust aspects of
the system software and hardware allowed the Magellan team to explore how these design points impact
application performance and usability. In addition, the diverse user requirements for cloud computing,
ranging from access to custom environments to the MapReduce programming model, led to a requirement
for a dynamic and reconfigurable software stack. Users had access to customized virtual machines through
OpenStack (at Argonne only) and Eucalyptus (at both sites), along with a Hadoop installation that allowed
users to evaluate the MapReduce programming model and the Hadoop Distributed File System. Both
OpenStack and Eucalyptus provide an application programming interface (API) that is compatible with
the Amazon EC2 API, enabling users to port between commercial providers and the private cloud. Access
to a traditional batch cluster environment was also used at NERSC to establish baseline performance and
to collect data on workload characteristics for typical mid-range science applications that were considered
suitable for cloud computing. The hardware and software deployed within the testbed are described in the
following section.
5.1 Hardware
The Magellan testbed hardware was architected to facilitate exploring a variety of usage models and understanding
the impact of various design choices. As a result, the testbed incorporated a diverse collection
of hardware resources, including compute nodes, large-memory nodes, GPU servers, and various storage
technologies. In the fall of 2011, Magellan will be connected to the 100 Gb network planned for deployment
by the DOE-SC-funded Advanced Networking Initiative (ANI). A portion of the Magellan resources will
remain available after the cloud research is completed in order to support the ANI research projects.
Both Argonne and NERSC deployed compute clusters based on IBM’s iDataplex solution. This solution
is targeted towards large-scale deployments and emphasizes energy efficiency, density, and serviceability.
Configurations for the iDataplex systems are similar at both sites. Each compute node has dual 2.66 GHz
Intel Quad-core Nehalem processors, with 24 GB of memory, a local SATA drive, 40 Gb Infiniband (4X
QDR), and 1 Gb Ethernet with IPMI. The system provides a high-performance InfiniBand network, which
is often used in HPC-oriented clusters but is not yet common in mainstream commercial cloud systems.
Since the network has such a large influence on the performance of many HPC and mid-range applications,
the ability to explore the range of networking options from native InfiniBand to virtualized Ethernet was an
23