Magellan Final Report - Office of Science - U.S. Department of Energy

Magellan Final Report
method to the Open Science Grid (OSG). An important aspect of porting Einstein@Home to run on OSG
was strengthening of fault-tolerance and implementation of automatic recovery from errors. Fixing problems
manually at scale simply isn’t practical, so Einstein@OSG eventually automated the process.
LIGO used the Magellan serial queue and the Eucalyptus setup at NERSC to understand the advantages
of running in cloud environments. A virtual cluster was set up in the NERSC Eucalyptus deployment for
LIGO to access the virtual resources. The virtual cluster was set up to serve an OSG gatekeeper on a head
virtual machine to provide the user a seamless transition between the traditional batch queue submission
and the virtual cluster.
Our work with LIGO shows how center staff might set up virtual clusters for use by certain groups.
To keep the image size small, the gatekeeper stack (approximately 600 MB) was installed on a block store
volume. In order to work around the virtual nature of the head node, it was always booted with a private IP;
and a reserved public IP, which is static, was assigned after the instance was booted. The gsiftp server had
to be configured to handle the private/public duality of the IP addressing. No other changes were required
to the LIGO software stack. LIGO was able to switch from running through the NERSC Globus gatekeeper
to running through the virtual gatekeeper without major difficulties, and ran successfully.
11.2.4 ATLAS
Figure 11.6: ATLAS CloudCRV job flow.
ATLAS is a particle physics experiment at the Large Hadron Collider (LHC) at CERN. The ATLAS
detector is searching for new discoveries in the head-on collisions of protons of extraordinarily high energy.
ATLAS used Magellan resources at NERSC to build a highly scalable compute cluster on the Eucalyptus
cloud. The configuration of the cluster is similar to a normal ATLAS compute cluster: Panda is used as the
primary job distributer, where a user can submit jobs into a queue. A storage gateway is installed in the
cluster to move data into and out of the cluster. A Hadoop File System (HDFS) is installed on the worker
nodes so that each worker node also acts as a storage node.
The key difference between the Magellan cluster and a conventional ATLAS cluster is that it fully utilizes
the agility provided by the cloud. The cluster can automatically adjust the number of workers according to
the number of jobs waiting in the queue. When there are many jobs waiting in the queue, new worker nodes
are automatically added to the cluster. When the jobs are finished, idle workers are terminated automatically
to save cost.
To address many of the challenges in working with virtual clusters that we discussed in earlier case studies,
the ATLAS project developed and deployed a tool called CloudCRV. CloudCRV can automatically deploy
and configure individual VMs to work as a cluster. It can also scale the size of the cluster automatically
according to demand. As shown in Figure 11.6, the lifetime of all core services and the worker nodes are
controlled by CloudCRV.
110