Magellan Final Report - Office of Science - U.S. Department of Energy

Magellan Final Report
pacity.
Sporadic Demand. One of the more common cases for using commercial cloud offerings is when the demand
is highly variable, especially if there are also time sensitive requirements for the service. In the analysis
of DOE centers above, the demand is still highly variable. However, scientists can typically tolerate reasonable
delays in the start of an application, especially if this results in access to more cycles. For cases where
the demand must be quickly met, the ability to quickly add additional resources can mean the difference
in being able to complete the project or not. One example is a DOE lab science project that is conceived
suddenly that requires more resources than can be obtained quickly from a DOE HPC Center. DOE HPC
Centers typically do not have a viable way to add the necessary resources in a short timescale, and the
existing resources are heavily utilized, so the only way to make room for the new project would be to push
out existing projects. Other examples are ones with real time-critical requirements, e.g., modeling an oil
spill to direct efforts to contain the spill, hurricane tracking, or simulating a critical piece of equipment when
it fails and causes an expensive resource to go down. Additionally, some experiments such as those at light
sources and accelerators require or benefit from analysis resources in real-time. If those experiments only
run a fraction of the time, they may benefit from on-demand cloud-type models. These kinds of opportunity
costs can be difficult to quantify but should be considered when considering whether to move to a cloud model.
Facility Constrained. Some sites are severely infrastructure limited. This could be due to building restrictions,
insufficient power at the site, or other limitations. In these cases, commercial offerings may be the only
reasonable option available. However, if expansion is an option, the long-term cost should be considered.
While infrastructure expansion can be costly, those costs can be amortized over a long period, typically 15
years.
The potential cost savings to the customer for these cases come from a few common sources. One is
the potential ability to avoid purchasing and deploying computing resources when the demand is unclear.
The other is operating resources that are only needed for infrequent periods of time which results in very
low utilization. Once a project can maintain a reasonably high utilization of a resource, the cost savings
typically vanish.
12.6 Late Update
As this report was being finalized, Amazon announced several important updates. Due to the timing of
these announcements, we have left most of the analysis unchanged, but we considered it was important to
discuss the impact of these changes on the analysis. There were three significant developments: an updated
Top500 entry from Amazon, the release of a new instance type which was used for the Top500 entry, and
new pricing. We will discuss each of these and their impact on the analysis.
Amazon’s Top500 entry for November 2011 achieved 240 TF and number 42 on the list. More interesting
than the absolute numbers or position is the improvement in efficiency to 68% of peak. On previous Top500
entries, Amazon had achieved approximately 50% of peak. This is likely due to better tuning of the Linpack
execution. Traditional HPC systems typically achieve between 80%-90% of peak. Eventually virtualized
systems may approach these efficiencies through improved integration with the interconnect and continued
improvements in the virtualization stacks.
In parallel with the new Top500 result, Amazon announced a new instance type, cc2.8xlarge. This
instance type is notable for several reasons. It is the first significant deployment of Intel Sandy Bridge EP.
In addition to increasing the number of cores per socket to eight cores from four cores compared with the
Nehalem processor used in the previous cluster compute instance type, the Sandy Bridge processor also
effectively doubled the number of floating point operations per cycle. However, the processors used in the
new instance type run at slightly lower clock rate (2.6 GHz versus 2.95 GHz). As a result of these differences,
the new instance has a theoretical peak FLOP rate that is approximately 3.5x larger than the previous cluster
126