Magellan Final Report - Office of Science - U.S. Department of Energy

Magellan Final Report
for optimizations necessary for particular hardware). In cloud systems, the end-user is responsible for
managing all aspects of the operating system and other software that might be required by the application.
The user has to identify, manage, operate, and maintain the operating system and dependent libraries
in addition to their specific application software. This requires end-users to have system administration
skills themselves, or additional support. Thus centers will need to provide tools and support for managing a
diverse set of kernels and operating systems that might be required by specific groups. The clear separation
of responsibilities for software upgrades and operating system patches no longer exists, and sites will need
mechanisms to bridge the gap between supporting user-supported images and site security policies.
Tools exist to bundle a running operating system and upload it to the cloud system. However, some
customization is typically required. Users need to have an understanding of standard Linux system administration,
including managing ssh daemons, ntp, etc. Furthermore, debugging and testing can be tedious,
since it often requires repacking and booting instances to verify the correct behavior. This also requires
experimentation to determine what applications and data are best to include in the image or handle through
some other mechanism. The simplicity of bundling everything in the image needs to be balanced with the
need to make dynamic changes to applications and data. This process is complex and often requires users to
carefully analyze what software pieces will be required for their application, including libraries, utilities, and
supporting datasets. If the application or supporting datasets are extremely large or change quickly, then
the user stores the data outside of the image due to limits on image size and its impact on virtual machine
boot-up times and memory that is available to the application at run-time. All our scientific users who had
to create images ranked it as medium or hard.
In order to help user groups try out cloud computing without the setup costs, we set up a virtual cluster
with a PBS queue that users could submit to. This largely made the virtual environment look similar to
the HPC environments. This environment allowed users to try out the virtual environment and compare it
to bare-metal performance. The success of this approach can be seen from one of the users who remarked,
“Everything works beautifully on the cluster! I deployed our software on the gatekeeper and ran a job
through PBS. I didn’t hit a single problem. From my perspective it makes no difference.”
11.4.3 On-Demand Bare-Metal Provisioning
As illustrated through the use cases presented in Section 11.1, on-demand bare-metal provisioning through
serial queues, reservations, and HaaS can meet many of the needs of scientific users seeking the extra
features of cloud environments such as custom environments. Bare-metal provisioning provides a number of
advantages, including access to high-performance parallel file systems, better performance for applications
that cannot tolerate the virtualization overhead, and access to specialized hardware for which virtualization
does not currently exist (e.g. GPUs).
11.4.4 User Support
It is important to understand that a large number of the users of Magellan had already used cloud resources,
had significant experience with virtual environments and system administration, or had computer scientists
or IT personnel on the project helping. In spite of previous experience, these users faced numerous challenges.
Scientific users with less experience in cloud tools require additional user support and training to help them
port and manage application in these environments; 96% of the users who responded to the survey (including
computer science and IT personnel) said they needed help from the Magellan staff. Most Magellan users were
able to overcome the challenges of using these environments and would consider using these environments
again.
11.4.5 Workflow Management
Another challenge in using cloud systems is developing a mechanism to distribute work. This is complicated
by the fact that cloud systems like Amazon Web Services are inherently ephemeral and subject to failure.
115