Magellan Final Report - Office of Science - U.S. Department of Energy

Magellan Final Report
7.4 Summary
User support models for technical computing users remains one of the open issues for cloud systems. Technical
computing users have grown accustomed to a high level of support from HPC center personnel. In this setting,
purpose built systems are constructed for a specific use case of large scale parallel applications. As a result,
many users have not developed the skills that are needed to effectively work in the more flexible environment
that clouds provide. Several possible solutions are discussed below.
Users could be educated to become comfortable designing and deploying infrastructure. This education
effort would be time-consuming and is unlikely to be sufficient, leaving some technical computing users unable
to use large scale cloud computing resources. This approach would also require that scientists develop skills
which are not directly related to their area of study, most likely reducing their scientific output.
Another potential solution is for an analogous HPC-tuned environment to be made available on a private
or public cloud. This solution is the most similar to current production computing centers, and would be
likely comparable in cost to current HPC center support activities. In fact, this approach was explored
at NERSC using a virtual version of an existing cluster (“Little Magellan” described in Chapter 3) which
was used by users to easily measure the overhead of running in a virtualized cloud environment. ALCF
also worked with the Argonne Laboratory Computing Resource Center (LCRC), which provides the large
laboratory mid-range computing cluster, to provide a virtual version of the existing Fusion cluster within
the Magellan public OpenStack cloud. In addition, the ALCF/LCRC team was able to set up the Fusion
resource manager so that jobs could expand into the Magellan public cloud in a secure fashion with a virtual
Fusion environment, providing dynamic expansion into the cloud when the Fusion resources were fully used.
Another approach would involve building user communities around common domains and applications. In
this model, sophisticated users would provide VM images which are optimized for a particular science domain
and are used by a large number of users. Many of these communities would mirror virtual organizations
that are already in place. This model would work well for large, established organizations, but could present
challenges for smaller communities composed primarily of novice users.
39