Magellan Final Report - Office of Science - U.S. Department of Energy

Magellan Final Report
greater than three months that were not possible with earlier versions. Many of our users commented favorably
on the stability and performance of the system. Newer releases show even more promise, as developers
continue to add features and improve stability.
Infrastructure. OpenStack’s management infrastructure (shown in Figure 6.2) is comprised of a number
of individual services which communicate with each other via asynchronous message passing and a MySQL
database. The API service handles remote client interaction and the serving of instance metadata to hypervisor
nodes. The object store and volume services manage the storage and access of virtual machine images
and EBS volumes, respectively, and the scheduler handles the allocation of compute resources to spread load
evenly across the hypervisor cluster. The networking service is responsible for all networking tasks, both
local and externally-routable, and the compute service runs on the hypervisors themselves, performing the
local management of individual KVM instances.
User Management. All users on an OpenStack cluster belong to one or more projects, and are assigned
roles both globally and within their projects. These roles govern their level of access, with the union of
global and project roles determining the precise privileges. Each user/project combination requires its own
set of client credentials e.g., a user belonging to three projects will require three different sets of credentials.
Each project is given its own local subnet and has project-wide instance firewall rules and access privileges
to published images and EBS volumes. The Nova management tools allow administrators to apply quotas
to user projects, limiting the number of instances a user may have allocated simultaneously, or the number
or total size of volumes they may create. User interaction with the OpenStack API is accomplished via the
eucatools client in the same manner as Eucalyptus; as such it is API-compatible with both Eucalyptus and
Amazon EC2.
Image and Volume Management. User virtual machine images are managed by the object store service,
which is an Amazon S3-like bucket storage application dedicated solely to storing and serving images. Image
access privileges are project-specific, meaning that should a user’s roles within a project allow it, they have
access to use and modify all of the project’s private images. Unlike Eucalyptus, users are also allowed to register
kernels and ramdisk images; making customization of existing images and addition of new ones far easier
for the user and reducing the support burden on the administrators. EBS volumes are handled by the volume
service, which creates, stores, exports and deletes iSCSI volumes. All volume privileges are bound to the user.
Machine Virtualization and Scheduling. The OpenStack compute service runs on each hypervisor
node, and is responsible for the creation and destruction of virtual machine instances and all associated
hypervisor-local resources, such as project-LAN networking and virtual disks. The underlying virtualization
technology in use at Argonne is KVM, though others are supported as well, such as Xen, UML, and the
relatively new LXC. The hypervisor for a group of instances is controlled by the Scheduling service, which
spreads the compute load evenly across the hypervisor cluster. The versions of OpenStack used by the
Magellan project did not support live migration, and compute resources are acquired by the first user to
request them. This has the effect of spreading even small single-core instances across the hypervisor cluster
and subsequently blocking a number of large 8-core jobs from running.
Accounting. All information pertinent to instances launched on an OpenStack cluster is recorded in the
OpenStack database. The information in the database is available to administrators. For example, the
database includes the date and time of instance creation and destruction, the user and project responsible
for the instance, the current state of the instance, the image from which the instance was created.
Network. All networking for OpenStack instances is managed by a Network service. Both project-LAN and
externally-routable IPs are assigned to instances via this service. Each project-LAN is given its own VLAN
in order to separate the broadcast domains of projects from each other. DHCP and an intricately designed
NATing chain is used to accomplish this task. Externally-routable addresses are broadcast to a border router
33