Magellan Final Report - Office of Science - U.S. Department of Energy
Magellan Final Report - Office of Science - U.S. Department of Energy
Magellan Final Report - Office of Science - U.S. Department of Energy
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>Magellan</strong> <strong>Final</strong> <strong>Report</strong><br />
role in addressing the computing requirements <strong>of</strong> scientific user groups. We recommend that DOE<br />
resource providers investigate mechanisms to provide on-demand resources to user groups.<br />
3. A number <strong>of</strong> DOE collaborations rely on complex scientific s<strong>of</strong>tware pipelines with specific library and<br />
version dependencies. Virtual machines are useful for end users who need customizable environments.<br />
However, the overheads <strong>of</strong> virtualization are significant for most tightly coupled scientific applications.<br />
These applications could benefit from bare-metal provisioning or other approaches to providing custom<br />
environments. DOE resource providers should consider mechanisms to provide scientists with tailored<br />
environments on shared resources.<br />
4. DOE resource providers are cost and energy efficient. However, the commercial sector is constantly innovating,<br />
and it is important that DOE resource providers should track their cost and energy efficiencies<br />
in comparison with the commercial cloud sector.<br />
5. Private cloud virtualization s<strong>of</strong>tware stacks have matured over the course <strong>of</strong> the project. However,<br />
there are significant challenges in performance, stability and reliability. DOE resource providers should<br />
work with the developer communities <strong>of</strong> private cloud s<strong>of</strong>tware stacks to address these deficiencies before<br />
broadly deploying these s<strong>of</strong>tware stacks for production use.<br />
6. There are gaps in implementing DOE-specific accounting, allocation and security policies in current<br />
cloud s<strong>of</strong>tware stacks. Cloud s<strong>of</strong>tware solutions will need customization to handle site-specific polices<br />
related to resource allocation, security, accounting, and monitoring. DOE resource providers should<br />
develop or partner with the developer communities <strong>of</strong> private cloud s<strong>of</strong>tware stacks to support sitespecific<br />
customization.<br />
7. User-created virtual images are powerful. However, there is also a need for a standard set <strong>of</strong> base<br />
images and simple tools to reduce the entry barrier for scientists. Additionally, scientists <strong>of</strong>ten require<br />
pre-tuned libraries that need expertise from supercomputing center staff. DOE resource providers<br />
should consider providing reference images and tools that simplify using virtualized environments.<br />
8. The cloud exposes a new usage model that necessitates additional investments in training end-users<br />
in the use <strong>of</strong> these resources and tools. Additionally, the new model necessitates a new user-support<br />
and collaboration model where trained personnel can help end-users with the additional programming<br />
and system administration burden created by these new technologies. DOE resource providers should<br />
carefully consider user support challenges before broadly supporting these new models.<br />
<strong>Science</strong> Groups. Cloud computing promises to be useful to scientific applications due to advantages<br />
such as on-demand access to resources and control over the user environment. However, cloud computing<br />
also has significant impact on application design and development due to challenges related to performance<br />
and reliability, programming model, designing and managing images, distributing the work across compute<br />
resources, and managing data. We make specific recommendations to science groups that might want to<br />
consider cloud technologies or models for their applications.<br />
1. Infrastructure as a Service provides an easy path for scientific groups to harness cloud resources while<br />
leveraging much <strong>of</strong> their existing application infrastructure. However, virtualized cloud systems provide<br />
various options for instance types and storage classes (local vs block store vs object store) that<br />
have different performance and associated price points. <strong>Science</strong> groups need to carefully benchmark<br />
applications with the different options to find the best performance-cost ratio.<br />
2. Cloud systems provide application developers the ability to completely control their s<strong>of</strong>tware environments.<br />
However, there is currently a limited choice <strong>of</strong> tools available for workflow and data management<br />
in these environments. Scientific groups should consider using standard tools to manage these environments<br />
rather than developing custom scripts. Scientists should work with tool developers to ensure<br />
that their requirements and workflows are sufficiently captured and understood.<br />
vii
Change language