TGQR 2010Q2 Report.pdf - Teragridforum.org

NSF Extensible Terascale Facility
TeraGrid
Report for
Q2: April 1 st , 2010 through June 30 th , 2010
Ian Foster (GIG)
Phil Andrews
Jay Boisseau
John Cobb
Michael Levine
Rich Loft
Honggao Liu
Richard Moore
Carol Song
Rick Stevens
Craig Stewart
John Towns
Principal Investigators (GIG and RPs)
University of Chicago/Argonne National Laboratory (UC/ANL)
University of Tennessee (UT-NICS)
Texas Advanced Computing Center (TACC)
Oak Ridge National Laboratory (ORNL)
Pittsburgh Supercomputing Center (PSC)
National Center for Atmospheric Research (NCAR)
Louisiana Optical Network Initiative/Louisiana State University
(LONI/LSU)
San Diego Supercomputer Center (SDSC)
Purdue University (PU)
University of Chicago/Argonne National Laboratory (UC/ANL)
Indiana University (IU)
National Center for Supercomputing Applications (NCSA)
i

TeraGrid Forum (TGF)
John Towns (NCSA)
Matt Heinzel (UC/ANL)
Phil Andrews (UT-NICS)
Jay Boisseau (TACC)
John Cobb (ORNL)
Honggao Liu (LONI/LSU)
Michael Levine (PSC)
Rich Loft (NCAR)
Richard Moore (SDSC)
Mike Papka (UC-ANL)
Carol Song (PU)
Craig Stewart (IU)
Grid Infrastructure Group
Ian Foster (GIG) PI of TeraGrid GIG
Matt Heinzel (UC) Director of the TeraGrid GIG
Tim Cockerill (NCSA) Project Management Working Group
Kelly Gaither (TACC) Visualization
David Hart (SDSC) User Facing Projects
Christopher Jordan (TACC) Data Analysis
Daniel S. Katz (UC/ANL) GIG Director of Science
Scott Lathrop (UC/ANL) Education, Outreach and Training; External Relations
Elizabeth Leake (UC) External Relations
Lee Liming (UC/ANL) Software Integration and Scheduling
Amit Majumdar (SDSC) Advanced User Support
J.P. Navarro (UC/ANL) Software Integration and Scheduling
Mike Northrop (UC) GIG Project Manager
Jeff Koerner (UC/ANL) Networking, Operations and Security
Sergiu Sanielevici (PSC) User Services and Support
Nancy Wilkins-Diehr (SDSC) Science Gateways
Carolyn Peters (ANL) Event Coordinator/Administration
ii

Working Group Leaders
Accounting
Advanced User Support
Allocations
Common User Environment
Core Services
Data
Education, Outreach and Training
External Relations
Extreme Scalability
Networking
Operations
Project Management
Quality Assurance
Scheduling
Science Gateways
Security
Software
User Facing Projects
User Services
Visualization
David Hart (SDSC)
Amit Majumdar (SDSC)
Kent Milfield (TACC)
Shawn Brown (PSC)
Dave Hart (SDSC)
Christopher Jordan (TACC)
Scott Lathrop (UC/ANL)
Elizabeth Leake (UC)
Nick Nystrom (PSC)
Linda Winkler (ANL)
Jeff Koerner (UC/ANL)
Tim Cockerill (NCSA)
Kate Ericson (SDSC), Shava Smallen (SDSC)
Warren Smith (TACC)
Nancy Wilkins-Diehr (SDSC)
Jim Marsteller (PSC)
Lee Liming (UC/ANL), JP Navarro (UC/ANL)
Dave Hart (SDSC)
Sergiu Sanielevici (PSC)
Kelly Gaither (TACC), Mike Papka (UC/ANL)
iii

TERAGRID QUARTERLY REPORT
Contents
1 Overview ........................................................................................................................... 7
1.1 Scope .............................................................................................................................. 7
1.2 Communities Served ...................................................................................................... 8
1.3 TeraGrid’s Integrated, Distributed Environment ........................................................... 9
1.4 Organizational Architecture ........................................................................................... 9
2 Science and Engineering Highlights ............................................................................. 10
2.1 List of Science and Engineering Highlights ................................................................. 10
2.2 Science and Engineering Highlights ............................................................................ 11
3 Software Integration and Scheduling ........................................................................... 21
3.1 Highlights ..................................................................................................................... 21
3.2 Advanced Scheduling Capabilities .............................................................................. 21
3.3 Capability Development and Expansion ...................................................................... 22
3.4 Operational Issues ........................................................................................................ 22
4 Science Gateways ............................................................................................................ 22
4.1 Highlights ..................................................................................................................... 22
4.2 Targeted Support .......................................................................................................... 23
4.3 Gateway Infrastructure and Services ........................................................................... 28
4.4 RP Operations: SGW Operations ................................................................................. 30
5 Users and User Support ................................................................................................. 31
5.1 Highlights ..................................................................................................................... 31
5.2 User Engagement ......................................................................................................... 32
5.3 Frontline User Support ................................................................................................. 33
5.4 RP Operations: User Services ...................................................................................... 36
5.5 Advanced User Support ............................................................................................... 36
6 User Facing Projects and Core Services ....................................................................... 59
6.1 Highlights ..................................................................................................................... 59
6.2 Enhanced TeraGrid User Access ................................................................................. 59
6.3 Allocations Process and Allocation Management ........................................................ 61
6.4 RP Integration and Information Sharing ...................................................................... 62
6.5 RP Operations: Accounting/Core Services .................................................................. 63
6.6 User Information Presentation ..................................................................................... 64
iv

6.7 RP Operations: User Facing Projects ........................................................................... 65
6.8 Information Production and Quality Assurance ........................................................... 69
7 Data and Visualization ................................................................................................... 69
7.1 Highlights ..................................................................................................................... 69
7.2 Data .............................................................................................................................. 70
7.3 RP Operations: Data..................................................................................................... 71
7.4 Visualization ................................................................................................................ 72
7.5 RP Operations: Visualization ....................................................................................... 73
8 Network Operations and Security ................................................................................ 75
8.1 Highlights ..................................................................................................................... 75
8.2 Networking .................................................................................................................. 75
8.3 RP Operations: Networking ......................................................................................... 77
8.4 Inca ............................................................................................................................... 78
8.5 Grid Services Usage ..................................................................................................... 78
8.6 RP Operations: HPC Operations .................................................................................. 85
8.7 GIG Operations ............................................................................................................ 88
8.8 Security ........................................................................................................................ 88
8.9 RP Operations: Security ............................................................................................... 89
8.10 Common User Environment ........................................................................................ 90
8.11 System Performance Metrics ....................................................................................... 90
9 Evaluation, Monitoring, and Auditing ......................................................................... 98
9.1 Highlights ..................................................................................................................... 98
9.2 XD – TIS: Technology Insertion Services ................................................................... 98
9.3 Quality Assurance ...................................................................................................... 100
9.4 XD – TAS: Technology Audit Services for TeraGrid ............................................... 100
10 Education, Outreach, and Training; and External Relations .................................. 106
10.1 Highlights ................................................................................................................... 106
10.2 Training, Education and Outreach ............................................................................. 111
10.3 External Relations ...................................................................................................... 118
10.4 Enhancement of Diversity .......................................................................................... 122
10.5 International Collaborations ....................................................................................... 122
10.6 Broader Impacts ......................................................................................................... 123
11 Project Management .................................................................................................... 138
11.1 Highlights ................................................................................................................... 138
11.2 Project Management Working Group ........................................................................ 138
v

12 Science Advisory Board Interactions.......................................................................... 139
13 Collaborations with Non-TeraGrid Institutions ........................................................ 140
13.1 Grid Partnerships/Infrastructures ............................................................................... 140
13.2 Science Projects and Science Gateways .................................................................... 140
13.3 Data Systems .............................................................................................................. 140
13.4 Science Projects ......................................................................................................... 140
A Publications Listing ...................................................................................................... 142
A.1. TeraGrid Staff Publications ............................................................................................. 142
A.2. Publications from TeraGrid Users ................................................................................... 143
B Performance on CY2010 Milestones ........................................................................... 177
vi

1 Overview
The TeraGrid is an open cyberinfrastructure that enables and supports leading-edge scientific
discovery and promotes science and technology education. The TeraGrid comprises
supercomputing and massive storage systems, visualization resources, data collections, and
science gateways, connected by high-bandwidth networks integrated by coordinated policies and
operations, and supported by computational science and technology experts,.
TeraGrid’s objectives are accomplished via a three-pronged strategy: to support the most
advanced computational science in multiple domains (deep impact), to empower new
communities of users (wide impact), and to provide resources and services that can be extended
to a broader cyberinfrastructure (open infrastructure). This “deep, wide, and open” strategy
guides the development, deployment, operations, and support activities to ensure maximum
impact on science research and education across communities.
1.1 Scope
TeraGrid is an integrated, national-scale computational science infrastructure operated in a
partnership comprising the Grid Infrastructure Group (GIG), eleven Resource Provider (RP)
institutions, and six Software Integration partners, with funding from the National Science
Foundation’s (NSF) Office of Cyberinfrastructure (OCI). Initially created as the Distributed
Terascale Facility through a Major Research Equipment (MRE) award in 2001, the TeraGrid
began providing production computing, storage, and visualization services to the national
community in October 2004. In August 2005, NSF funded a five-year program to operate,
enhance, and expand the capacity and capabilities of the TeraGrid to meet the growing needs of
the science and engineering community through 2010, and then extended the TeraGrid an
additional year into 2011 to provide an extended planning phase in preparation for TeraGrid
Phase III eXtreme Digital (XD).
Accomplishing this vision is crucial for the advancement of many areas of scientific discovery,
ensuring US scientific leadership, and increasingly, for addressing important societal issues.
TeraGrid achieves its purpose and fulfills its mission through a three-pronged strategy:
Deep: ensure profound impact for the most experienced users, through provision of the
most powerful computational resources and advanced computational expertise;
Wide: enable scientific discovery by broader and more diverse communities of
researchers and educators who can leverage TeraGrid’s high-end resources, portals and
science gateways; and
Open: facilitate simple integration with the broader cyberinfrastructure through the use
of open interfaces, partnerships with other grids, and collaborations with other science
research groups delivering and supporting open cyberinfrastructure facilities.
The TeraGrid’s deep goal is to enable transformational scientific discovery through leadership
in HPC for high-end computational research. The TeraGrid is designed to enable high-end
science utilizing powerful supercomputing systems and high-end resources for the data analysis,
visualization, management, storage, and transfer capabilities required by large-scale simulation
and analysis. All of this requires an increasingly diverse set of leadership-class resources and
services, and deep intellectual expertise in the application of advanced computing technologies.
The TeraGrid’s wide goal is to increase the overall impact of TeraGrid’s advanced
computational resources to larger and more diverse research and education communities
7

through user interfaces and portals, domain specific gateways, and enhanced support that
facilitate scientific discovery by people without requiring them to become high performance
computing experts. The complexity of using TeraGrid’s high-end resources will continue to grow
as systems increase in scale and evolve with new technologies. TeraGrid broadens the scientific
user base of its resources via the development and support of simpler but powerful interfaces to
resources, ranging from establishing common user environments to developing and hosting
Science Gateways and portals. TeraGrid also provides focused outreach and collaboration with
science domain research groups, and conducts educational and outreach activities that help
inspire and educate the next generation of America’s leading-edge scientists.
TeraGrid’s open goal is twofold: to enable the extensibility and evolution of the TeraGrid by
using open standards and interfaces; and to ensure that the TeraGrid is interoperable with
other, open standards-based cyberinfrastructure facilities. While TeraGrid only provides
(integrated) high-end resources, it must enable its high-end cyberinfrastructure to be more
accessible from, and even federated or integrated with, cyberinfrastructure of all scales. That
includes not just other grids, but also campus cyberinfrastructures and even individual researcher
labs/systems. The TeraGrid leads the community forward by providing an open infrastructure that
enables, simplifies, and even encourages scaling out to its leadership-class resources by
establishing models in which computational resources can be integrated both for current and new
modalities of science. This openness includes open standards and interfaces, but goes further to
include appropriate policies, support, training, and community building.
The TeraGrid’s integrated resource portfolio includes 19 high-performance computational (HPC)
systems, several massive storage systems, remote visualization resources, and a dedicated
interconnection network. This infrastructure is integrated at several levels: policy and planning,
operational and user support, and software and services. Policy and planning integration
facilitates coordinated management and evolution of the TeraGrid environment and allows us to
present—as much as possible—a single cyberinfrastructure to the user community. Operational
and user support integration allows the user community to interact with one or more of the many
distinct resources and HPC centers that comprise TeraGrid, through a common service, training,
and support organization—masking the complexity of a distributed organization. In addition, this
allows the national user community to request allocations through a single national review
process and use the resources of the distributed facility with a single allocation. Software and
services integration creates a user environment with standard service interfaces, lowering barriers
to porting applications, enabling users to readily exploit the many TeraGrid resources to optimize
their workload, and catalyzing a new generation of scientific discovery through distributed
computing modalities. Science Gateways, web-based portals that present TeraGrid to specific
communities, represent a new paradigm in high-performance computing with the potential to
impact a wide audience. Gateways benefit significantly from the integration of software and
services across the TeraGrid.
1.2 Communities Served
The national, and global, user community that relies on TeraGrid has grown tremendously during
the past four years, from fewer than 1,000 users in October 2005 to nearly 5,000 active users and
more than 10,000 total lifetime users at the end of 2009.
In Q2 2010, TeraGrid continued to exceed new thresholds for many user-related metrics. Notably,
in May, TeraGrid surpassed 1,400 individuals charging jobs in a single month. TeraGrid also
surpassed 6,000 active users at the quarter’s end. Finally, 662 new individuals joined the ranks of
TeraGrid users, the second highest quarterly total to date. This elevated workload is made
possible by prior efforts to streamline the new-user process and reduce the central staff effort.
8

Further details can be found §6.2.1.
To support the great diversity of research activities and their wide range in resources needs, our
user support and operations teams leverage the expertise across the eleven TeraGrid Resource
Providers (§5, §8). In addition, users benefit from our coordinated education, outreach, and
training activities (§9).
1.3 TeraGrid’s Integrated, Distributed Environment
TeraGrid’s diverse set of HPC resources provides a rich computational science environment.
TeraGrid RPs operate more than 25 highly reliable HPC resources and 10 storage resources that
are available via a central allocations and accounting process for the national academic
community. More detailed information is available on compute resources at
www.teragrid.org/userinfo/hardware/resources.php and on storage resources at
www.teragrid.org/userinfo/hardware/dataresources.php.
In Q2 2010, TeraGrid again saw further increases in delivered NUs. TeraGrid compute resources
delivered 12.1 billion NUs to users, 0.3% as TeraGrid Roaming NUs (Figure 8-11). Roaming
NUs continued to decline as that allocation feature is phased out. The 12.1 billion NUs represent
a 20% increase over the NUs delivered in Q1 2010. Nearly 100% of those NUs were delivered to
allocated users. The Q2 2010 NUs delivered are 1.7x the NUs delivered in Q2 2009. Further
details can be found §8.11.
1.4 Organizational Architecture
The coordination and management of the TeraGrid partners and resources requires organizational
and collaboration mechanisms that
are different from a classic
organizational structure for single
organizations. The existing
structure and practice has evolved
from many years of collaborative
arrangements between the centers,
some predating the TeraGrid. As
the TeraGrid moves forward, the
inter-relationships continue to
evolve in the context of a
persistent collaborative
environment.
Figure 1-1: TeraGrid Facility Partner Institutions
The TeraGrid team (Figure 1-1) is
composed of eleven RPs and the
GIG, which in turn has subawards to the RPs plus six additional Software Integration partners.
The GIG provides coordination, operations, software integration, management and planning. GIG
area directors (ADs) direct project activities involving staff from multiple partner sites,
coordinating and maintaining TeraGrid central services.
TeraGrid policy and governance rests with the TeraGrid Forum (TG Forum), comprising the
eleven RP principal investigators and the GIG principal investigator. The TG Forum is led by an
9

elected Chairperson, currently John Towns. This position facilitates the functioning of the TG
Forum on behalf of the overall collaboration.
TeraGrid management and planning is coordinated via a series of regular meetings, including
weekly project-wide Round Table meetings (held via Access Grid), weekly TeraGrid AD and
biweekly TG Forum teleconferences, and quarterly face-to-face internal project meetings.
Coordination of project staff in terms of detailed technical analysis and planning is done through
two types of technical groups: working groups and Requirement Analysis Teams (RATs).
Working groups are persistent coordination teams and in general have participants from all RP
sites; RATs are short-term (6-10 weeks) focused planning teams that are typically small, with
experts from a subset of both RP sites and GIG partner sites. Both types of groups make
recommendations to the TeraGrid Forum or, as appropriate, to the GIG management team.
2 Science and Engineering Highlights
The TeraGrid publishes science impact stories to inform and educate the public about the
importance and impact of high-end computing technologies in advancing scientific research. The
fourth annual TeraGrid Science Highlights document was distributed at the SC09 conference,
providing information about 20 of the many science projects that have achieved significant
impact by using the TeraGrid. These and other science impact stories are available on the
TeraGrid web site.
Below we provide 14 examples of TeraGrid-enabled science: first in a summary list, then as short
summaries of each project. These highlights have been selected from the much larger collection
of important scientific research activity supported on the TeraGrid to demonstrate the breadth of
scope across scientific disciplines, research institutions, and sizes of research groups.
2.1 List of Science and Engineering Highlights
2.1.1
2.1.2
2.1.3
2.1.4
2.1.5
2.1.6
2.1.7
2.1.8
2.1.9
Analytical and Surface Chemistry: Cleaner coal through computation (Jennifer Wilcox,
Stanford University)
Bioengineering: The role of anatomical structure in ventricular and atrial arrhythmias
(Elizabeth Cherry, Cornell)
Biophysics: Molecular dynamic study of the conformational dynamics of HIV-1 protease
subtypes A, B, C, and F (Adrian Roitberg, U. Florida
Condensed Matter Physics: First-principles investigation of elastic constants of Sn-based
intermetallics and thermodynamics of Fe-based multicomponent alloys (Gautam Ghosh,
Northwestern)
Chemistry: Mechanisms of bioorganic and organometallic cyclization reactions (Dean
Tantillo, U. California, Davis)
Education: ModelThis! Science Olympiad initiative (Edee Wiziecki, NCSA)
Extragalactic Astronomy and Cosmology: The formation and evolution of X-ray clusters,
galaxies, and the first stars (Greg Bryan, Columbia)
Materials Research: Structural order and disorder in semiconducting nanocrystals
studied by combined DFT calculations and solid-state NMR (Bradley Chmelka, U.
California, Santa Barbara)
Meteorology: Center for analysis and prediction of storms (CAPS) (Ming Xue,
Oklahoma)
10

2.1.10
2.1.11
2.1.12
2.1.13
2.1.14
Molecular and Cellular Biosciences: Solution structure of PNA and DNA double-helices
with and without metal substitutions; and Study of the interactions between mutant RT
and novel anti-HIV drugs (Marcela Madrid, Pittsburgh Supercomputing Center)
Physical Chemistry: Actin filament remodeling by actin depolymerization factor/cofilin
(Greg Voth, U. Chicago; Jim Pfaendtner, U. Washington)
Physics: The MIMD lattice computation (MILC) collaboration (Bob Sugar, U.
California, Santa Barbara)
Molecular and Cellular Biosciences: Blueprint for the affordable genome (Aleksei
Aksimentiev, U. Illinois at Urbana-Champaign)
Physics: The Universe’s magnetic personality (Stanislav Boldyrev, U. Wisconsin)
2.2 Science and Engineering Highlights
2.2.1
Analytical and Surface Chemistry: Cleaner coal through computation (Jennifer Wilcox,
Stanford University)
Pollution control devices known as scrubbers, installed to restrict the amount of nitrogen oxide
and sulfur dioxide release from coal-fired power plants, may have helped to reduce acid rain, but
they haven’t made those plants safe. About 5,000 tons of mercury are released worldwide from
coal-fired power plants each year, according to the
Environmental Protection Agency (EPA). Scientists have
determined that this mercury travels long distances and
ends up in remote regions, like the Arctic, where it enters
the food chain and accumulates within organisms. As a
consequence, the people of the Arctic display elevated
levels of these toxic metals in their blood, which is
correlated to birth defects and other ailments.
In order to reduce the release of trace metals, Wilcox
simulates the interactions of these particles using Ranger
at TACC. Her studies are helping to improve on current
technologies and design new ones that can remove heavy
metals from the coal combustion process. Specifically,
Wilcox’s simulations show how the size, pore structure,
and composition of a material affect its success as an
oxidizer of heavy metals. The goal is to create a structure
that will bind trace metal molecules and convert them
into a water-soluble form that can be easily removed.
Wilcox’s solutions can take the form of changes in the
Figure 2-1. Understanding trace metal
adsorption aids in determining the pathway
by which the trace metals are emitted into
the atmosphere. In this example, elemental
selenium is adsorbed onto an iron oxide
nanoparticle through a surface oxidation
mechanism. [Selenium: blue-colored atom;
oxygen: red-colored atoms such that
hatched and solid correspond to surface
and bulk, respectively; iron: rust-colored
atoms]
spacing of pores in a charcoal filter, or the creation of a never-before-seen alloy of palladium and
gold that, Wilcox predicts, will better oxidize heavy metals in gasification.
In 2011, the EPA is expected to release regulations limiting the release of mercury and other
heavy metals into the atmosphere. Power companies will be forced to adopt more efficient and
effective technologies to protect the environment and avoid fines. By developing the theory and
methods needed to produce optimal capture devices, Wilcox is leading the way and helping to
create a cleaner energy future.
11

2.2.2
Bioengineering: The role of anatomical structure in ventricular and atrial arrhythmias
(Elizabeth Cherry, Cornell)
New technologies for low-energy defibrillation (applying shocks to persons undergoing heart
attack at much lower voltage than is the current standard) hold promise to avoid undesirable sideeffects
— including tissue damage and pain — of current defibrillation. These emerging
technologies, which apply shocks that may be below the pain threshold, also hold for “witness
defibrillation” — shocks administered by a layperson who witnesses collapse. Elizabeth Cherry
and co-PIs (Flavio Fenton and Jean Bragard) perform whole-organ simulation of cardiac
electrical dynamics.
They have developed
parallel codes that
implement a threedimensional
model of
electrical wave
dynamics in atria and
ventricles, to
implement models of
cellular
electrophysiology in
Figure 2-2. Top: Idealized simulated cardiac tissue with conductivity discontinuities
created by two inexcitable obstacles. With increasing field strengths, first the tissue
edge, then a single conductivity discontinuity, and finally both conductivity
discontinuities are excited. Bottom: Realistic simulated cardiac tissue with two
inexcitable obstacles. Increasing the field strength recruits more conductivity
discontinuities as virtual electrodes.
realistic geometries.
This is computationally
challenging because of
the range of space and
time scale. During the
past year, they reported
on simulations (using
BigBen at PSC) that
show the feasibility of a
novel method of lowenergy
defibrillation.
This report appeared in
Circulation, the leading research journal in cardiology. In this procedure, low-voltage shocks are
applied via far-field pacing to terminate arrhythmias. Modeling of this method requires inclusion
of small-scale discontinuities in conductivity, such as blood vessels and other non-conducting
tissue. The researchers incorporated these effects into their bidomain model of cardiac electrical
activity. A bidomain model, as opposed to monodomain models that treat interstitial space as
grounded, is crucial for accurate simulation of defibrillation. Their simulations are correlated with
experimental studies. Newer publications highlighting these findings, including a book chapter,
are in press.
2.2.3 Biophysics: Molecular dynamic study of the conformational dynamics of HIV-1 protease
subtypes A, B, C, and F (Adrian Roitberg, U. Florida
Human Immunodeficiency Virus (HIV) is still one of the most prominent diseases in the world
with infection rates continuously rising in certain areas. There are two groups of HIV, which are
divided into HIV-1 and HIV-2. HIV-1 is more prevalent than HIV-2 and accounts for the
majority of infections. HIV-1 can further be divided into three subgroups M, N, and O. Subgroup
M is the largest subgroup and can be further divided into several subtypes. Roitberg group’s work
focuses on four subtypes of subgroup M, specifically A, B, C, and F. Subtype A is generally seen
in West and Central Africa. Subtype B is predominant in North and South America, Australia and
12

Figure 2-3. Illustration of the three main flap conformations of HIV-1 Protease, from left to right, closed, semi-open, wideopen.
Western Europe. Subtype C is found in East and Southern Africa and throughout Eastern Asia.
Subtype F is predominantly found in South America.
The HIV protease has become an attractive target for drug design due to its role of cleaving the
gag and gag-pol polyprotein precursors. Furthermore, inhibition of the protease’s natural
biological function would prevent the maturation of the HIV, hence preventing the infection of
neighboring cells. The flap domain is the most mobile of all domains, largely attributed to the
number of Gly residues found in this region. Due to the importance of the flaps having to open
and close in order for catalytic activity to occur, it has been suggested by previous works that the
development of a new class of protease inhibitors that instead target the flap domain or other
essential domains might be more effective than the original idea of developing a protease
inhibitor that works through competitive inhibition. In this current study, Roitberg and his
students investigate conformational dynamics of the flaps, and the size of active site in order to
correlate how the different sequences of subtypes A, B, C and F allow for different conformations
of the protease.
The Molecular Dynamics simulations were performed on Kraken using a version of the AMBER
software optimized specifically for this problem as part of a TeraGrid Advanced User Support
project between Prof. Roitberg and
SDSC. Each of the simulations was run
for 220 ns in which between 204 and 256
processors were used for each
independent run.
The results indicate that the different
subtypes have different mobilities,
particular in the protein flaps, which in
term affect substrate and inhibitor access
to the active site. Given these differences,
one can rationalize for instance that
certain inhibitors, designed to work
against subtype B of HIV Protease (the
subtype prevalent in Europe and North
America) do not bind as well to subtype
A and C (prevalent in Africa and
responsible for 60% of the overall HIV
infections). Using this information, we
will work on designing inhibitors better
tailored to the less studied subtypes.
Figure 2-4. Snapshots from the simulation of subtype C
superimposed, which illustrate the mobility of the flaps.
13

2.2.4
This team’s general area of interest is the calculation of
cohesive properties in solid-state systems using firstprinciples
methods. They have used Cobalt and the nowretired
Mercury at NCSA, Pople at PSC, the retired IBM
IA-64 Linux cluster at SDSC, and Kraken at NICS in two
main areas of computational research: (1) thermodynamic
stability, elastic constants and elastic anisotropy, and
thermal expansion of intermetallics (binary and ternary)
relevant to modern solder interconnects in
microelectronic packages, and (2) calculation of bulk
thermodynamic properties of multicomponent alloys,
precipitate/matrix interfacial energies, and elastic
constants of Fe-base solid solutions to facilitate
computational design and microstructure simulation in
ferritic superalloys. They have collaborated with the
member companies of Semiconductor Research
Corporation and provided them basic materials properties
relevant to modeling constitutive relations in solder
interconnects. This in turn led to an interdisciplinary and
multiscaled approach to integrate intermetallic properties
into a predictive reliability model for area-array solder
interconnects. Their work was published in Acta Materialia.
2.2.5
Condensed Matter Physics: First-principles investigation of elastic constants of Sn-based
intermetallics and thermodynamics of Fe-based multicomponent alloys (Gautam Ghosh,
Northwestern)
Terpenes are a prominent class of natural products whose diverse members consist of most
everything from flavor molecules to anti-cancer drugs, a prominent example being taxol, a
therapeutic agent derived from
tree bark used especially in
treatment of breast cancer. The
mechanisms by which terpenes
are produced in nature have
been debated for decades, but
are still far from fully
understood. Tantillo has used
GAUSSIAN on up to 128
processors of Pople at PSC to
explore the potential energy
surfaces and map the transition
states of these cascade
reactions, which are among the
most challenging in nature.
The key factor here is that one
enzyme works on the substrate
Figure 2.n. Orientation dependence of
shear modulus of AuSn 4 , based on the
elastic constatnts (C ij s) calculated from
first-principles at LDA and GGA levels,
where the shear direction is rotated from
[010] to [001] in the shear plane (100) or
from [010] to [101] in the shear plane [10-
1]. Calculated results demonstrate the
extent of anisotropy of shear modulus.
Chemistry: Mechanisms of bioorganic and organometallic cyclization reactions (Dean
Tantillo, U. California, Davis)
Figure 2-5. Product distributions for cyclizations of biological cations
such as that derived from farnesyl diphosphate
to produce one terpene, and another enzyme works on the same substrate to produce a different
terpene, resulting in about 400 separate terpene products from one reactant. More often in nature,
only one enzyme works with any one reactant to make one product. Tantillo’s recent work has
14

mapped out pathways from the bisaboyl cation to a diverse array of complex sequiterpenes. His
findings, reported in Nature Chemistry (August 2009) and JACS (March 2010), conclude that: (a)
correct folding of the substrate, farnesyl diphosphate, alone doesn’t always dictate the structure of
which products will be formed, (b) secondary carbocations are often avoided as intermediates in
these rearrangements, and (c) most of the reactions don’t involve significant conformational
changes for the intermediates, which may contribute significantly to product selectivity.
2.2.6
Recently, the NCSA cybereducation team deployed the
first national science Olympiad event using computer
modeling. The Science Olympiad National Tournament
2010 competition was held on the University of Illinois
campus May 21-22 for students from 49 states. Ninetytwo
middle school and 50 high school students
representing the top competitors in their state competed
in the Model This! competition using Vensim and
NetLogo to answer questions related to predator-prey
interactions and epidemiology, respectively. Two-person
teams used laptop computers and modeling software to
gain insights into their topic; they interpreted data and
used it to answer questions. Previously, in April, dozens
of middle- and high-school students from across Illinois
competed in the Illinois Science Olympiad State Finals Competition at NCSA. Model This! is the
first Olympiad competition to use computer modeling. It was a trial challenge, meaning it is now
being evaluated and may become an official Science Olympiad event. Edee Wiziecki, director of
NCSA cybereducation programs, is shepherding the introduction of Model This! into the Science
Olympiad competition. The initiative, which is supported by NCSA, the TeraGrid, and the
Institute for Chemistry Literacy Through Computational Science (ICLCS), integrates computer
technology, the Internet, quantitative data analysis and computer modeling.
2.2.7
Education: ModelThis! Science Olympiad initiative (Edee Wiziecki, NCSA)
Figure 2-6. Students work on their tasks in
the ModelThis! Competition at NCSA.
ModelThis! is the first Science Olympiad
competition to use computer modeling.
Extragalactic Astronomy and Cosmology: The formation and evolution of X-ray clusters,
galaxies, and the first stars (Greg Bryan, Columbia)
It appears that the first generation of stars to form in the universe were probably very massive
(about 100 times the mass of the sun), and lived short lives before exploding in supernovae
explosions. However, during the time that they did shine, they were so bright that they ionized the
gas for thousands of light-years around. Greg Bryan and his team have carried out adaptive-mesh
refinement simulations using Abe at NCSA to investigate the impact of this ionizing radiation on
the ability of the gas to form later generations of stars. A typical run on Abe, notes Bryan, uses
around 500 processors and runs for approximately 100 hours. They demonstrated that there are
two separate significant effects: one that tends to enhance later star formation, and one that
suppresses it. They found that these two effects are (coincidently) nearly balanced. There is
generally a slight net suppression but this disappears after a relatively short period of time (80
million years). The result is that previous simple models that neglected both of these effects are
(by chance) reasonably accurate predictors of the star formation rate.
They have also looked at galaxies. Galaxies tend to group together, forming galaxy clusters.
When a galaxy falls into a cluster for the first time, it plows into this hot cluster gas at high
velocity, which can result in cold galaxy gas being stripped off. This is important because the
cold gas is the fuel for new star formation in the galaxy. Bryan’s team has carried out the highest
resolution simulations ever conducted of this process. This results in significantly more realistic
simulations, and they can make extensive comparisons to observations, making, for the first time,
15

ealistic "simulated" maps that can be directly compared to observations. Their work was
published in the Monthly Notices of the Royal Astronomical Society, The Astrophysical Journal,
and Astrophysical Journal Letters.
2.2.8
Materials Research: Structural order and disorder in semiconducting nanocrystals
studied by combined DFT calculations and solid-state NMR (Bradley Chmelka, U.
California, Santa Barbara)
The research team of Bradley Chmelka is motivated by the need to understand at a molecular
level the fabrication and functions of new catalysts, adsorbents, porous ceramics, and
heterogeneous polymers. These categories of
technologically important materials are linked by their
crucial dependencies on local order/disorder, which often
govern macroscopic process or device performance. The
team is observing many common molecular features
among these diverse systems, which provide new insights
for materials chemistry and engineering.
Using Cobalt at NCSA, two-bond scalar 2 J( 29 Si-O- 29 Si)
spin-spin couplings between 29 Si atoms connected via
bridging oxygen atoms in complicated nanoporous
zeolite frameworks were calculated with high accuracies
by density functional theory (DFT) and compared against
experimentally measured values obtained from twodimensional
29 Si{ 29 Si} nuclear magnetic resonance
(NMR) spectroscopy. Such calculations, based on singlecrystal
or powder X-ray diffraction (XRD) structures of
siliceous zeolites Sigma-2 and ZSM-12 and a surfactanttemplated
layered silicate, were determined to be
extremely sensitive to local framework structures. The
calculations were compared against experimentally
measured 2 J( 29 Si-O- 29 Si) couplings obtained with the
Figure 2-7. Different O-centered clusters
extracted from the structure of zeolite
Sigma-2 (as determined by single-crystal
XRD analyses25) and centered on the
29 Si(1)–O- 29 Si(4) fragment with different
bond terminations. Si, O, and H atoms are
displayed in blue, red, and white,
respectively.
highest accuracies yet achieved for zeolitic solids using state-of-the-art solid-state 2D doublequantum
29 Si{ 29 Si} NMR. The use of 2 J( 29 Si-O- 29 Si) couplings to probe, evaluate, and refine the
local structures of zeolites and layered silicates was demonstrated, including for frameworks that
had previously been unsolved.
In combination, such 2D NMR, XRD, and DFT analyses enable new opportunities for the
development of protocols that integrate new and sensitive J coupling constraints into structuredetermination
or -refinement protocols. Such local order (or disorder) has important influences on
the macroscopic adsorption, reaction, mechanical, photophysical, and/or stability properties of
these diverse classes of materials, which can now be better understood and correlated with their
structures at a more detailed molecular level. The team’s work, in collaboration with Sylvian
Cadars (now at the Centre National de la Recherche Scientifique in Orléans, France) and Darren
Brouwer at the National Research Council of Canada, was presented at several conferences, has
been published in Physical Chemistry Chemical Physics in 2009 and has been submitted to the
Journal of the American Chemical Society in 2010.
2.2.9 Meteorology: Center for analysis and prediction of storms (CAPS) (Ming Xue,
Oklahoma)
16

The Center for Analysis and Prediction of Storms (CAPS) used Athena at NICS in dedicated
mode for 6 hours per night, 5 nights a week, to run their 2010 CAPS Spring Experiments from
April to June. A combination of
jobs using up to 12804 cores
filled up most of Athena’s
18048 cores during these
periods. Several computational
models were used, including
WRF, ARW, NMM and ARPS,
using 51 vertical levels, with
different horizontal grid sizes;
there were 4 km grids forecasts,
and a single 1 km grid forecast,
all of them encompassing the
whole continental USA. A total
of 4.7 Million computational
core hours were used throughout
the duration of this project. The
codes were significantly
improved from the ones used
during last year’s CAPS Spring
Experiments on Kraken.
Figure 2-8. Computational domains for the 2010 CAPS Spring Season.
The outer thick rectangular box represents the domain for performing
3DVAR (Grid 1 – 1200×780). The red dot area represents the WRF-
NMM domain (Grid 2 – 790×999). The inner thick box is the domain for
WRF-ARW and ARPS and also for common verification (Grid3 –
1160×720 at 4 km grid spacing; 4640×2880 at 1 km grid spacing)
The stormscale ensemble and 1
km CONUS systems continue to break new ground in advancing high resolution modeling for
operational applications. The Spring Experiment this year included convective aviation-impacts
and quantitative precipitation forecast (QPF)/heavy rain components in partnership with the
Aviation Weather Center (AWC) and the Hydrometeorological Prediction Center (HPC),
respectively, in addition to their usual severe thunderstorm focus. The more wide-ranging
convective storm focus allowed others in the operational community to become more familiar
with the CAPS models, leading the HPC to say that the storm-scale ensemble forecasting system
was a "transformational" event for warm season QPF.
Ming Xue, Director of CAPS, said, “Obviously, the success of our forecasting effort this and last
year is due in large part to the incredible support that we have received from NICS, University of
Tennessee. Without their making available to us a dedicated machine (with >18,000 cores) and
their support of a dedicate team that usually stays up late into night to fix many system related
problems, all of these would not have been possible. So we deeply appreciate their efforts. As in
the past, a lot of new science will result from such efforts, as a stream of abstracts is being
submitted to the Severe Local Storm Conference.“
More than 10 abstracts were submitted to the American Meteorological Society (AMS)
Conference for Severe Local Storms (October 2010), based on the analysis of the data sets. More
journal articles will follow. An overview article is planned for Bulletin of the AMS, which has the
largest readership among AMS journals.
17

2.2.10
By means of X-ray crystallography and calculations
done on Pople at PSC, the team of Joanne I. Yeh,
Boris Shivachev, Srinivas Rapireddy, Matthew J.
Crawford, Roberto R. Gil, Shoucheng Du, Marcela
Madrid, and Danith H. Ly has determined the
structure of a PNA-DNA duplex. This structure
represents the first high-resolution view of a hybrid
duplex containing a contiguous chiral PNA strand
with complete γ-backbone modification (“γPNA”).
The new structure (see Figure 2.a) illustrates the
unique characteristics of this modified PNA,
possessing conformational flexibility while
maintaining sufficient structural integrity to ultimately
adopt the preferred P-helical conformation upon
hybridization with DNA. The unusual structural
adaptability found in the γPNA strand is crucial for
enabling the accommodation of backbone
modifications while constraining conformational
states. These results provide unprecedented insights
into how this new class of chiral γPNA is preorganized and stabilized, before and after
hybridization with a complementary DNA strand. Such knowledge is crucial for the future design
and development of PNA for applications in biology, biotechnology and medicine.
2.2.11
Molecular and Cellular Biosciences: Solution structure of PNA and DNA double-helices
with and without metal substitutions; and Study of the interactions between mutant RT
and novel anti-HIV drugs (Marcela Madrid, Pittsburgh Supercomputing Center)
Physical Chemistry: Actin filament remodeling
by actin depolymerization factor/cofilin (Greg
Voth, U. Chicago; Jim Pfaendtner, U.
Washington)
Voth and Pfaendtner have extended prior work on
actin, a ubiquitous protein with a vital role in
cytoskeletal dynamics and structure. The cycle of actin
polymerization into filaments that undergird the
cytoskeleton and depolymerization of these filaments
has been implicated in many cancers, in particular
breast cancer. The researchers used molecular
dynamics simulations on BigBen at PSC and Ranger at
TACC to investigate how the severing protein, actin
depolymerization factor (ADF)/cofilin, modulates the
structure, conformational dynamics, and mechanical
properties of actin filaments. Their findings, reported
in PNAS (April 2010), show that actin and cofilactin
filament bending stiffness and corresponding
persistence lengths obtained from all-atom simulations
are comparable to values obtained from analysis of
thermal fluctuations in filament shape. Filament
flexibility is strongly affected by the nucleotide-linked
conformation of the actin subdomain 2 DNase-I
binding loop (DB loop) and the filament radial mass
Figure 2-9. (a) A view of the γPNA strand
(the DNA strand is omitted for clarity). The
“water bridges” are present in between all the
backbone amide NH and nucleobases
(purine-N3 or pyrimidine-O2) enhancing the
stability of the γPNA helix. (b) γPNA
interactions with surrounding solvent
molecules (γPNA strand, residues 6 to 9).
Figure 2-10. This is a representative
snapshot of two actin subunits
(grey/green) and one bound ADF/cofilin
(yellow). The parts that look like a rough
surface are those regions of the actin
protein that bind the ADF/cofilin protein
(the ADF/cofilin binding surface). The
red regions are those regions which were
implicated in experiments to be important
for ADF/cofilin binding to actin.
18

density distribution. ADF/cofilin binding between subdomains 1 and 3 of a filament subunit
triggers reorganization of subdomain 2 of the neighboring subunit such that the DB loop moves
radially away from the filament. Repositioning of the neighboring subunit DB-loop significantly
weakens subunit interactions along the long-pitch helix and lowers the filament bending rigidity.
Lateral filament contacts between the hydrophobic loop and neighboring short-pitch helix
monomers in native filaments are also compromised with cofilin binding. These results provide a
molecular interpretation of biochemical solution studies documenting the disruption of filament
subunit interactions and also reveal the molecular basis of actin filament allostery and its linkage
to ADF/cofilin binding. In sum, their findings, corroborated through collaboration with prominent
actin structural biologists Thomas Pollard and Enrique De La Cruz of Yale, show how slight
structural change in the DB loop translates to altered flexibility of the cytoskeleton, pointing the
way to drug design possibilities for cancer therapies.
2.2.12 Physics: The MIMD lattice computation (MILC) collaboration (Bob Sugar, U.
California, Santa Barbara)
The MIMD Lattice Computation (MILC) Collaboration used approximately 8.66M core-hours on
Athena at NICS in May 2010 for their Dedicated Athena Project research in Quantum
Chromodynamics (QCD). This research addressed fundamental questions in high energy and
nuclear physics, and is directly related to major experimental programs in these fields,
particularly as they relate to the strong (nuclear) forces. The objectives of their work are to
calculate some of the least well determined parameters of the standard model, and, in concert
with major experiments in high energy physics, to test the standard model and search for new
physics that goes beyond it.
The particular calculations carried out on Athena represented a milestone because they are the
first simulations which include four quarks, up, down, strange and charm, all with their physical
masses. This had not been possible before because computational resources required to carry out
QCD simulations grow as the masses of the up and down quarks decrease. Furthermore, the
ensemble of gauge configurations worked on with Athena is the most challenging generated to
date with the Highly Improved Staggered Quark (HISQ) formulation of lattice quarks. A total of
81 gauge configurations were generated with Athena resources, using two different algorithms as
a check on the results. The jobs running on Athena ranged from 1536 cores to 16384 cores. The
MILC group also performed measurements of the static quark potential, the masses of the lightest
strongly interacting particles, and the leptonic decay constants of pi, K, D and Ds mesons on over
100 configurations, including ones generated on Athena as well as on other TeraGrid computers.
The resulting gauge configurations will enable major improvements in the precision of a wide
range of physical quantities of importance in high energy and nuclear physics.
The ensemble of gauge configurations generated Athena must be combined with those from other
ensembles to obtain final results before the work can be published. A talk on this work was
presented at the XXVIII International Conference on Lattice Field Theory in Villasimius,
Sardinia in June 2010.
2.2.13
Molecular and Cellular Biosciences: Blueprint for the affordable genome (Aleksei
Aksimentiev, U. Illinois at Urbana-Champaign)
As “next-generation” gene sequencers begin to make their mark on the life sciences, teams
around the world are racing to develop new and improved DNA sequencers that can ingest a
strand of nucleotide bases and directly read a person’s genetic code at a cost of less than $1,000.
19

Aleksei Aksimentiev, a computational physicist at the U.
Illinois at Urbana-Champaign, is one of the creators of a
new kind of sequencer that may make the dream of an
affordable, personal genome a reality. This type of
sequencer uses an electric field to drive a strand of DNA
through a small hole, or “nanopore,” in a membrane, and
reads each base pair in order by measuring the change in
current as the pair moves through the hole in the
membrane.
Since experiments can’t show exactly what’s going on
inside a nanopore, Aksimentiev produces atom-by-atom
models of nanopore designs and runs them on Ranger at
TACC. His simulations revealed the atomic-level
movements of DNA as it wriggles through the nanopore,
leading to insights into how to improve the design of the
system.
Aksimentiev’s nanopore design promises a dramatic reduction in the cost of gene sequencing, and
a commensurate increase in speed. The development of such a sequencer would have important
ramifications for medicine, biology and human health, with the potential to lead to breakthroughs
in the study of disease and evolution, and the development of personalized medicine. The
National Institute of Health (NIH) has set a goal of producing a $1,000 genome using nanopore
sequencers by 2013.
2.2.14
Physics: The Universe’s magnetic personality (Stanislav Boldyrev, U. Wisconsin)
Plasma, or ionized gas, is the most common form of
matter in the universe. Stars are balls of plasma held
together by gravity, and even the space between the stars
is filled with plasma, albeit sparsely. Astrophysicists
have long believed that this plasma is magnetized.
Likewise, they’ve understood that the cosmos is
turbulent, filled with non-linear flows and eddies that
generate motion and heat.
What they have recently discovered is that for many
astrophysical systems, magnetic turbulent energy is
comparable to, and at some scales exceeds, kinetic
energy, making it one of the most powerful forces
shaping the system’s dynamics. In fact, certain
astronomical observations, like the behavior of accreting
disks and the signature of radio waves from the
interstellar medium, only make sense after accounting for
magnetic turbulence.
Unlike regular hydrodynamic turbulence, which can be
observed in the natural world and tested in laboratories
or wind tunnels, those who study
magnetohydrodynamics, or MHD, have little tangible
evidence to go on. Since scientists can’t generate large
Figure 2-11. Large-scale molecular
dynamics simulations reveal the
microscopic mechanics of DNA transport
through solid-state nanopores. [Courtesy of
Aleksei Aksimentiev, University of Illinois
at Urbana-Champaign]
Figure 2-12. Solar wind turbulence is
imbalanced, with more Alfven waves
propagating along the magnetic field lines
from the Sun than toward the Sun. Such
turbulence is reproduced in numerical
simulations on Ranger. Red and blue
regions correspond to streams of nonlinear
Alfven waves propagating along the
magnetic field (z-axis) in opposite
directions.
magnetized turbulent flows in our laboratories, they have to rely on numerical simulations. Over
the last several years, Boldyrev and his colleague, Jean Carlos Perez (U. Wisconsin), developed
the theory and methodology required to represent the magnetic turbulence of plasma. Using
20

Ranger at TACC, they simulated the dynamics believed to be characteristic of most plasma flows
with higher resolution than ever before, revealing its fine-scale dynamics.
Through his simulations on Ranger, Boldyrev and his collaborators have shown that magnetism
cannot be ignored when drafting the history of the Universe. In fact, magnetic turbulence may be
the key to explaining some basic astrophysical phenomenon, like the clumping of interstellar gas
to form stars, and the scintillation, or twinkling, of pulsars that scientists now recognize as
evidence of the fluctuations produced by a turbulent and magnetized interstellar medium.
Boldyrev says, “Understanding magnetized plasma turbulence is crucial for explaining very
important astrophysical observations.”
3 Software Integration and Scheduling
The software integration area covers the operational tasks associated with keeping TeraGrid’s
common software up-to-date and consistently configured across all HPC resources that support
them, as well as a number of small capability extension projects. In PY5, these projects include
improving our meta-scheduling capabilities, enhancing our information service, supporting NSF’s
software cyber-infrastructure development and maintenance programs (SDCI, STCI), improving
our documentation, and formalizing our application hosting capabilities.
3.1 Highlights
The scheduling working group is continuing to transition the batch queue prediction capability
from QBETS to Karnac in order to ensure long-term production-quality support for the capability.
We also are beginning the process of releasing historical scheduling workload data for public use
in external R&D activities.
An increasing number of services (most recently at NCAR and ORNL) are publishing new kinds
of system data into TeraGrid’s Integrated Information Service (IIS), including directories of
locally available software packages and science gateways.
3.2 Advanced Scheduling Capabilities
TeraGrid’s current automatic resource selection (metascheduling) capabilities are provided by
MCP software (developed at SDSC) and Condor-G with matchmaking software (from Univ. of
Wisconsin-Madison). We made incremental improvements to MCP, including a new portlet that
displays the nodes and CPUs/node for TeraGrid machines. We added documentation on Condor-
G with matchmaking to the TeraGrid documentation website. We continued to assess the
possibility of using Moab as a metascheduling system for TeraGrid. NICS is conducting an
evaluation of Moab’s performance when metascheduling across at least two Moab clusters. NICS
has obtained permission to use the FutureGrid system as a second site (beyond NICS) for the
experiments. We also conducted a study to evaluate the performance of QBETS, Condor-G, and
MCP for automatic resource selection.
TeraGrid’s batch queue prediction service is currently provided by UC-Santa Barbara’s QBETS
service. Our quality assurance working group recommended that this service be described in
documentation as an “experimental” service, which we implemented this quarter. We released a
“beta” version of a new batch queue prediction service based on TACC’s Karnak software, which
we believe will eventually replace QBETS and become a production queue prediction service.
SDSC implemented basic Inca tests for the beta Karnak service. We also distributed a new
version of TeraGrid’s GLUE2 capability kit, which provides the data used by our Condor-G
matchmaking and Karnak services.
21

The scheduling working group also began new plans to release scheduling workload data from
the live TeraGrid systems to the public. TeraGrid operators use this data to evaluate different
cluster and grid scheduling strategies. It will also be useful to researchers and developers to help
them develop algorithms and tools that work on systems like TeraGrid.
3.3 Capability Development and Expansion
On Indiana University’s Sierra cluster, users can run HPC jobs and can utilize the nimbus cloud.
On IU’s India cluster, users can run HPC jobs and utilize the eucalyptus cloud.
The McStas simulation package, used by the Neutron Science TeraGrid Gateway (NSTG)
managed at ORNL is installed on the NSTG cluster and advertised via the IIS. Currently jobs
dispatched from the Neutron Science Portal (NSP) Application Manager are done via explicitly
Globus/Gram calls, but future use is anticipated both possibly from the NSP as well as a separate
portal infrastructure, Orbiter, being developed by TechX, Buffalo, from a SBIR grant.
NCAR deployed TeraGrid’s new Local HPC Software Publishing capability and has begun
advertizing its local software in TeraGrid’s Integrated Information Service (IIS).
A new version of Globus GRAM software, which provides TeraGrid’s remote job submission
capability, has been tested on TACC and LONI systems in preparation for a TeraGrid-wide
deployment. We identified issues with the software configuration, usage reporting, and our
installation tools that will be corrected before the TeraGrid-wide deployment in Q3.
3.4 Operational Issues
LONI upgraded the MyCluster, Condor, and Condor-G software on the QueenBee system.
The TACC system operators identified issues with TeraGrid’s new security policies related to
TAGPMA and certificate authority (CA) certification. Specifically, the initial policies regarding
user sites were too restrictive, and this issue has been corrected.
TACC also raised a concern arising from non-TeraGrid sites who would like to use TeraGrid’s
CTSS software distributions with non-TeraGrid (and non-TAGPMA) certificate authorities. This
issue is still open, but it is not strictly relevant to TeraGrid operations, either.
CTSS’s remote login, data movement, application and parallel application development and
runtime support, and TeraGrid core integration capabilities are all used frequently (often heavily)
on all TeraGrid systems. Use of the CTSS remote compute capability (which allows remote job
submission) varies widely with heavy use at three sites, frequent use at four sites, and infrequent
use at two sites. Only two resource providers offer the data visualization capability but both
report frequent use. The data management capability is used relatively infrequently at the three
sites where it is available. The science workflow support capability is available on most resources
and is used relatively infrequently, but each use results in many job submissions, making it
difficult to compare to the other capabilities.
4 Science Gateways
4.1 Highlights
The CIPRES (Cyberinfrastructure for Phylogenetic Research) gateway, with 637 users in this
reporting period, had more users running simulations on the TeraGrid than all gateways combined
in 2009. The group received a 2.7M hour award at the June allocations meeting and was cited as a
model gateway proposal. Demand is expected to far outstrip this award though, which was
limited only by resource availability. The team is looking into having top users make individual
allocation requests.
22

Presentations at gateway telecons featured Ultrascan (section 4.2), gateway software listing
(section 4.3), the Dark Energy Survey (DES) gateway, TeraGrid’s Common User Environment,
GRAM5 (section 4.3), and the Expressed Sequence Tag (EST) gateway (section 4.3). Unfunded
efforts such as DES highlight projects making effective use of TeraGrid without staff assistance.
4.2 Targeted Support
CCSM/ESG.
Version 1.0 of the NCAR Earth Science Knowledge Environment – Science Gateway Framework
(ESKE-SGF) has been released. ESKE-SGF is a major software development effort aimed at
building shared cyberinfrastructure for supporting several important data and knowledge
management initiatives. The gateway currently supports dissemination of IPCC AR4 data, and in
the near future will support the IPCC Fifth Assessment Report (AR5). Current holdings of the
ESKE-SGF total about 290 TB. Total download activity from the site is about 10 TB per month,
of which about 2 TB per month are transferred from deep storage at partner sites using the
TeraGrid. Holdings for AR5 are anticipated to be about 400 TB of rotating storage at NCAR. The
NCAR gateway is anticipated to contain references to petabyte-scale holdings at collaborating
sites.
NCAR and Purdue are developing an Environmental Science Gateway on the TeraGrid that
integrates elements of the Purdue Climate Data Model Portal, the Earth System Grid (ESG)
infrastructure at NCAR, and the multi-agency Earth System Curator effort. Currently,
infrastructural iRODS elements are being set up, an ESG data node has been set up at Purdue, and
publication of test data is underway to expose the test data in the NCAR ESG gateway. ESMF is
currently being modified to include CIM metadata and to output the CIM XML. CCSM4 will be
modified to include CIM attributes. CIM metadata for simulations has been manually coded in
OWL and has been tested in a Development Portal user interface. A test instance CCSM4 was set
up on the IBM Power 575 Bluefire computer at NCAR.
Task 1 for this reporting period was to provide interfaces for remote model run invocation.
CCSM4 final release with the ESMF library has been installed on the Steele system at Purdue.
The install is being validated. Basic test runs were successful. Web Services interfaces for
CCSM4 simulations are available including support for the Purdue CCSM modeling portal and
the NOAA Curator workflow client, SOAP and RESTFUL interfaces, and access through either a
community account or individual’s TeraGrid allocation via MyProxy. The group will maintain
one code base by adapting Purdue’s current CCSM3 implementation and modifying it to meet
ESGC’s needs such as authentication and running on multiple sites. Purdue is working on open
source licensing for the Web Service code. Design is finalized. Implementation is in progress.
Next quarter the team will install the CCSM attribute patch on Steele, finish model validation and
complete the CCSM4 web service implementation.
Task 2 is to publish Purdue’s CCSM archive data to ESG. Data publishing has been tested to
PCMDI (Program for Climate Model Diagnosis and Intercomparison) gateway from the Purdue
ESG data node. Data publishing to the NCAR prototype gateway is almost complete (Figure 4-1).
Example metadata has been published to the Curator developmental portal
23

Figure 4-14 CCSM output in ESG
Next quarter the team will publish example CCSM output data to NCAR prototype gateway and
investigate two paths for harvesting iRODS datasets: the FUSE file system with the ESG
Publisher, or PyRODS with the same.
Task 3 is to collect and publish CCSM4 model data and metadata back to ESG. The
CCSM4/ESMF installation has been compiled and tested at Purdue. Several build issues were
resolved with help from the ESMF support team. A group demonstration of METAFOR CIM
(Common Metadata for Climate Modeling Digital Repositories), which Curator is building ESG
interfaces for, and will be output by CCSM4 later this year. Good progress has been made on
instrumenting CCSM4 such that it can produce a METAFOR-compliant CIM document
(WCRP/CMIP5 metadata).
Plan for next quarter include the evaluation of the CCSM4/ESMF CIM output capability. Scripts
will be tested for collecting and publishing the metadata for CCSM4 model runs to ESG and
evaluating the ATOM feed of CCSM-CIM into ESG. The design will be refined for an overall
system architecture that supports publishing CCSM4 model output data to ESG.
New features added to the CCSM portal include secure FTP download of CCSM model output.
This is convenient for downloading large volumes of data and is integrated with the Purdue
LDAP server and the portal account database. Online video tutorials have been added. Install of
CCSM on Queenbee is in progress. The goal here is to dynamically submit to a TeraGrid site
based on resource availability.
In addition, the Purdue under RP funding has contributed to this work by developing new features
for the TeraGrid CCSM modeling portal based on the feedback from previous class use.
This team’s overarching goals for next quarter are continued work on software licensing and
publishing the first Purdue CCSM dataset into ESG.
GISolve.
The GISolve team evaluated a hydrological modeling prototype to study the management
strategies of water resources in Illinois and integrated a data-intensive map projection function
from the USGS National Map project. Next quarter the team continues to explore new
applications/communities and will integrate a data transfer service for moving data between the
24

USGS National Map infrastructure and TeraGrid. They will also study the integration of a land
use model.
RENCI Science Portal.
The RENCI team wrote numerous web service clients which help the learning curve involved in
writing/using web service clients that call into the RENCI Science Portal (RSP) infrastructure.
Clients include Autodock (NCBR), UrQMD (Duke), BLAST+ (WUSTL), KlustaKwik (Duke),
MEME/MAST (UNC), MAQ (Duke) and ProteinEffects (Duke). Several new applications have
been installed: ProteinEffects, SamTools (pileup) and KlustaKwik.
The team has continued working with various researchers to create large BLAST runs using the
BLASTMaster Desktop and RSP web service infrastructure. The Genome Center at WUSTL
continues to make use of the RSP system via web services. Web service clients have been honed
to work cohesively with Jython based scripts at WUSTL. This enables bridging different
programming expertise (Java at RENCI and Python at WUSTL). The RSP has been used as a
proxy for launching jobs using the RSP web service infrastructure (business logic on third-party
web site calls into RSP web services).
Next quarter work concludes for this team. They will continue to work with researchers to utilize
TeraGrid and maintain account credentials across all configured clusters (TeraGrid, OSG, and
local RENCI clusters). They will refactor the backend software used to submit glidein jobs out to
remote resources using the next generation Portal Glidein Factory. This will be plugin-based
(OSGi). Granularity of pre-staging data and binary building will be improved.
UltraScan.
UltraScan (PI Prof. Borries Demeler, U Texas Health Science Center at San Antonio) is a
gateway supporting the computationally intense analysis of data collected on macromolecules by
an analytical ultracentrifuge. The gateway is used by researchers throughout the world and also
makes use of computational resources throughout the world. While some end users have their
own centrifuges and use the analysis capabilities of the gateway, others send samples to use both
the centrifuge and the analysis software. The IU gateways team completed initial integration of
Open Grid Computing Environment tools (application management, registration, and monitoring
services) with Ultrascan portal to submit jobs on Queenbee and Ranger. The team implemented
job monitoring for job status, job cancel method based on requests from the UltraScan
development team. Future work includes the addition of checkpointing for long-running jobs, the
use of a community account on TeraGrid resources. Resource discovery methods developed
through TeraGrid’s advanced support work with the GridChem project were applied in this
project as well. Final production integration of OGCE tools with UltraScan's production
environment depends on final GRAM5 stability validation on Ranger.
Gateways built with many different technologies have been able to make effective use of OGCE.
The LEAD gateway is portlet-based. GridChem is a Java swing client side application and
Ultrascan is php and perl-based. All have been able to effectively use OGCE. In addition, a large
MPI application that forks off many independent runs will be a focus for improvement through
TeraGrid’s ASTA program.
Finally, the Ultrascan gateway infrastructure has been replicated on IU’s Quarry hosting service
so that improvements can be made without impacting the production environment.
25

Arroyo.
Figure 4-2 Ultrascan infrastructure.
The Arroyo gateway supports the use of the widely used Arroyo adaptive optics code. Adaptive
optics is used to sharpen images and comes into play in the building of actuators and sensing
systems. End user groups consulted in the design of the gateway include astronomers at Caltech
and the Jet Propulsion Lab. Typically a 4 day run on a workstation is needed for 4 seconds of
optical imagery. The code requires a very complicated set of input parameters and one goal of the
gateway is to simplify that with the web interface. Django has been used for gateway
development and mysql to keep track of users and jobs. A daemon makes http calls to webserver
asking if there are new jobs, then copies local information and uses Java CoG to push jobs to
Queenbee. Users are updated on the progress of jobs continually throughout the process.
Figure 4-3 Adaptive optics make wavefront corrections.
26

Figure 4-4 Effect of adaptive optics.
RP Activities
IU: The Expressed Sequence Tag (EST) Pipeline is a bioinformatics portal for processing
expressed sequence tags that has been described in previous reports. The increased availability of
next generation sequencing devices will result in considerably higher volumes of data,
necessitating the use of high performance computing for analysis. During the current period of
performance, the IU gateways team added Cobalt, Ranger, Queenbee and Abe to the EST
pipeline resource pool significantly reducing turn around time. Pipeline applications
(RepeatMasker, PACE, CAP3) were installed on the above resources. The team performed initial
evaluations of benchmarked performance of pipeline applications on resources and selected
resources more suitable for applications. The pipeline was tested with 150,000, 200,000, 300,000,
1,000,000 and 2,000,000 sequence inputs to test scaling and validate the underlying OGCE job
management software. The EST Pipeline is based on the SWARM Web Service that provides a
web service interface to clients and also manages the bulk job submission using the Birdbath API
to submit to Condor. Jobs can range in length from milliseconds to days. Short jobs are run
locally, longer jobs are submitted to remote supercomputers using Condor-G. SWARM load
balances based on queues on the remote machines. The workflow that is to be executed by the
pipeline is configured using a PHP based gateway that allows users to upload input data and
select programs to run.
OLAS: This is a new effort in collaboration with Prof. Craig Mattocks, University of North
Carolina, to support Gateway-based hurricane and storm surge modeling. Dr. Mattocks's ASTA
request has been approved, and the IU team is awaiting official notification.
BioVLAB: This bioinformatics gateway led by Prof. Sun Kim at IU will rely heavily on OGCE
job management and workflow tools. During the current period, the IU gateways team assisted
Prof. Kim's startup allocation request and assisted his team with pilot TeraGrid jobs. Prof. Kim
plans to submit a larger allocation and advanced support request proposal during Quarter 3.
ORNL: The ORNL Resource Provider (RP) continues to provided specific support to the
Neutron Science TeraGrid Gateway as well as the Earth Systems Grid (ESG) effort. For the
NSTG, the main effort has focused on preparing the NSTG (gateway and cluster) to transition to
long-term, stable, contributory operations within the Spallation Neutron Source (SNS). For the
ESG, the entire collaboration is finalizing preparations for support for the next round of very high
profile IPCC runs. ESG plans to be the principal source for simulation data and collection support
for the IPCC-5 effort.
27

Purdue: For the Year 5 GIG TeraGrid Climate Science Gateway project, the Purdue/NCAR are
making progress and on track according to the project plan. Bi-weekly conference calls were held
between Purdue and NCAR/NOAA collaborators to discuss project design and implementation
details. Purdue RP has successfully installed the CCSM4 final release with ESMF libraries on the
Steele cluster and is validating the installation. A common set of CCSM4 web service interfaces
has been defined and is being implemented to support both the CCSM modeling portal at Purdue
and the Curator workflow client at NOAA. A shared SVN repository has been set up for
collaborative code development of Purdue/NCAR. Purdue and NCAR have investigated iRODS
features in the design of the mechanism for publishing CCSM archive data to the Earth System
Grid gateway. Purdue has also set up an ESG data node on a production server and successfully
published example CCSM model data to the ESG PCMDI gateway
UC/ANL: The U Chicago/Argonne RP has continued maintenance of the TeraGrid Visualization
Gateway. The team also continued previous work done with the SIDGrid Gateway, which
generalized the application framework to support Swift for the execution and management of
large-scale science and engineering workflows. Using this gadget-workflow framework, the U
Chicago/Argonne team has continued development and support of the Open Protein Simulator
(OOPS) gateway, a gadget-based science portal for a science community that focuses on protein
3D structure simulation and prediction. Scientists in this community have been using this
gateway to participate in the on-going CASP9 protein structure prediction challenge, sponsored
by the US National Library of Medicine (NIH/NLM).
4.3 Gateway Infrastructure and Services
Documentation and Gateway registry.
Most effort from the SDSC team has been on bringing TeraGrid Liferay up to speed. The high
number of bad links has been brought under control. No substantive changes to content have
been implemented in Science Gateways. Science Gateways are represented in the
Comprehensive Software Search, but requires further population from gateways. Only the RENCI
science portal software is registered at this time.
The RENCI team has developed Flex based XML generation web applications which generates
XML which can be digested by TeraGrid integrated information services (IIS) software listings
registry at http://www.renci.org/~jdr0887/kit-registration/index.html. The source is available at
http://www.renci.org/~jdr0887/git-repo/kit-registration.git. Next quarter the team will work with
TeraGrid to resolve any issues arising from the Gateway Application Web Service Registry. They
will continue to support other Gateways in publishing their web service capabilities into
TeraGrid’s IIS and continue to publish RENCI software listing information into TG IIS
Gateway User Count.
Science Gateway credential management documentation was updated to describe use of shortlived
versus long-lived certificates and MyProxy by gateways
(http://teragridforum.org/mediawiki/index.phptitle=Science_Gateway_Credential_Management)
. The NCSA user count team worked with Yan Liu (NCSA) on security improvements to
SimpleGrid. “TeraGrid Science Gateway AAAA Model: Implementation and Lessons Learned”
was accepted for TeraGrid 10. GRAM proxy refresh was implemented in the jglobus GramJob
API to enable jglobus-based gateways to renew proxy credentials for running GRAM jobs.
Next quarter the team will present their TeraGrid 10 paper. They will assist with inclusion of
GRAM5 and SSH support for gateway attributes in CTSS and will support gateway delivery of
attributes to RPs. Current status of this work is available at:
http://teragridforum.org/mediawiki/index.phptitle=Science_Gateway_Credential_with_Attribute
s_Status and http://info.teragrid.org/web-apps/html/kit-reg-v1/science-gateway.teragrid.org-
28

4.2.0/. The NCSA team will support AMIE integration by RP accounting administrators. This has
been completed by NCSA, NICS, LONI/LSU, and NCAR; integration at other RPs is planned.
The team will also support use of community shell
(commsh, http://teragridforum.org/mediawiki/index.phptitle=Community_Shell) by RPs.
At ORNL, the Neutron Science TeraGrid Gateway (NSTG) does support attributes at this point
for jobs dispatched from the Neutron Science portal however, the NSTG cluster (where most
gateway jobs were queued) did not complete the deployment of the software to support this useful
reporting to the TGCDB until early in the third quarter.
SimpleGrid.
Usability improvements include integrating interactive command access and tutorial instructions
into a single Web interface. The team has developed a generic Web interface generation module
to streamline the process of creating gateway application Web service and Web user interface by
leveraging advanced Web 2.0 technologies (i.e., Yahoo UI and inputEx). They have developed a
configuration tool to automate the management of package installation as well as server and Web
portal account as a set of system command scripts. The team has explored a solution using
TeraGrid information services for pulling usage data from the TeraGrid Central Database
(TGCDB). The enhancement of SimpleGrid Web interface and package management enables
more efficient delivery of the knowledge of gateway application development and the tutorial
package, respectively.
Next quarter the team will continue to improve the usability of SimpleGrid by evaluating the
effectiveness of newly added features from user experience and feedback collected in handstutorials.
They will continue to explore solutions to gateway user-level resource usage reporting
and work with TeraGrid information service group to extract gateway user-level data from
TGCDB.
Helpdesk support.
Staff resolved a ticket (#185005) from the University of Michigan on emulating Windows
applications on TeraGrid using Wine technology. They continued to support CIPRES gateway
development on using local job scheduler features and continued to support NBCR gateway
development on solving a meta-scheduler problem.
Improved standardization of community accounts.
The NICS team has tested the commsh community tool shell on Kraken with GRAM4. They have
completed the commsh configuration and testing for GridAMP gateway. The community account
“gridamp” now has commsh as the production login shell for gateway access. Use of commsh
will provide a reasonably secure “sandbox” for gateway developers by limiting the commands
and scripts that can be run by community account.
Next quarter the team will investigate the use of chroot with commsh and conduct security testing
of GRAM4 with commsh in an attempt to identify any additional vulnerabilities. A presentation
on gateway security and commsh will be given on TG Science Gateway call. A paper will be
completed on installation, configuration and the use of commsh. Securing community accounts is
an important function for TeraGrid and the gateway program. This work furthers the work of
securing gateways and their use of community accounts.
Summer Interns. IU's Science Gateway Advanced Support Team added two summer interns
during the period of performance: Suresh Kumar Deivasigamani and Patanachai Tangchaisin.
Both are enrolled as IU Computer Science Master's Students. Deivasigamani is developing
JavaScript client libraries, REST services, and iGoogle Gadgets to provide interactive interfaces
to TeraGrid information services. Tangchaisin is assisting with GRAM5 testing (see below),
29

developing and running realistic test suites and developing iGoogle Gadgets for visualizing
results.
GRAM 5 preparation.
The IU gateways team developed a testing suite and ran tests on the GRAM5 hosts Ranger,
Lonestar and Queenbee with different batch sizes (5,10,20,50,100,500,1000) to assist with
validation. Initial problems were found (specifically, resources associated with failed jobs were
not being released) with Ranger and Queenbee installations. These problems were resolved with
the Globus team in multiple iterations of test. Tests on the latest GRAM5 releases on Ranger and
Queenbee are running adequately, but the IU team is continuing 2-3 stability tests per day. The
harness is being used for the Gram5 testing.
The testing suite is packaged and made available on the Open Grid Computing Environments'
SourceForge site for any gateway's usage. The testing suite is currently being used for scalability
testing but will be expanded to run reliability tests if a gateway chooses to augment Inca testing.
The testing suite is modularized to extend to other job manager including Condor JDRAMMA
interfaces.
The larger TeraGrid team has made use of the TeraGrid User Portal’s user forum capability for
GRAM5 deployment discussions and has provided feedback to the TGUP team on the utility of
the user forum. GRAM5 experiences have been exchanged with the Open Science Grid as well.
4.4 RP Operations: SGW Operations
During this quarter RPs worked with developers and providers of Scientific Gateways on specific
RP resources in order to ensure that operation of each gateway is secure and that there is a means
for tracking accountability for gateway usage.
IU: IU added two new virtual machine images.
TACC: TACC added support for the ogce community account.
NCAR: While the Asteroseismic Modeling Portal (AMP) is not receiving advanced support in
PY5, NCAR staff have consulted with NICS staff on the implementation of commsh and its use
by gateway developers.
ORNL: During this quarter, the ORNL RP continued its efforts to support the Neutron Science
TeraGrid Gateway. The ORNL RP focused on making dispatching jobs to the TeraGrid a more
routine execution target for the Neutron Science and Orbiter portals. Following a DOE internal
assessment, the SNS (actually the ORNL enterprise) network has instituted default deny for
outgoing network connections. Thus exceptions from SNS host(s) to TeraGrid subnets must now
be requested on a pair wise case-by-case basis. In the long term this is not tenable. It would be
easier if ORNL did not have this new policy; if the TeraGrid had a common address space (i.e. a
single router rule for a FW exception) or the number of ports required for common TeraGrid
services such as Gram and GridFTP could be reasonably restricted to a smaller number of high
numbered ports. A long-term solution is not clear at this time, but is still being worked.
Purdue: The Purdue CCSM portal has been successfully used in a graduate class (POL 520/EAS
591, Purdue) in the fall semester, 2009. After the class, the Purdue RP staff worked closely with
the students and instructors to collect feedback on how to improve the portal to better meet the
instructional needs. In Q2 2010, several new features were developed and deployed to the portal
including secure FTP download of model output and online video tutorials demonstrating the
capabilities of the portal. In addition, progress has been made in setting up the model at Queenbee
to allow dynamic job submission based on resource availability. These new capabilities will
30

significantly improve the user experience and ensure better support for another graduate class in
the fall semester 2010.
UC/ANL: The UChicago/Argonne RP hosts and provides operating system, security and other
software support for the two machines that act as the production and development Visualization
Gateway resources, as well as the machine that serves the Open Life Science Gateway. The
UChicago/Argonne team has continued work on their gadget-workflow framework. Initial work
on a package builder for the framework, which enables developers to reuse the framework to plug
in their domain-specific workflows has been completed. To simplify the handling of input, two
intuitive and easy-to-use GUIs were developed. Users can simply type in command-line examples
or upload their Swift scripts to create XML descriptions for their applications. Also implemented
was support for determining output products from workflow descriptions and representing them
in the portal graphically. The output view also includes an output file browser, execution
parameters view, and workflow progress view. Users can customize the output views when
defining their workflows by selecting the appropriate output files and modifying the general
transform style sheets. Based on their experience of developing the gadget-workflow space, the
UChicago/Argonne RP team wrote the paper "Accelerating Science Gateway Development with
Web 2.0 and Swift" which was submitted, and accepted, to the TeraGrid 2010 Conference.
5 Users and User Support
User support in the TeraGrid comprises interlocking activities in several project areas:
documentation, allocations and accounting services in the User Facing Projects and Core
Services area (Section 6 of this report); the TeraGrid Operations Center in the Networking,
Operations and Security area (Section 8); user outreach and training in the Education, Outreach
and Training/External Relations area (Section 10); and the Frontline User Support and Advanced
User Support services described in the current section. In addition, specialized user support
services are provided by staff from the Science Gateways (Section 4) and Data and Visualization
(Section 7) areas. The synergy of all these aspects of user support is coordinated by the User
Interaction Council which consists of the GIG Director of Science and the directors of the above
mentioned areas.
5.1 Highlights
The User Support (US) and Advanced User Support (AUS) staff continued their highly rated
services for TeraGrid users at every level. As needed, US and AUS staff collaborated with the
Data and Visualization, Science Gateways, EOT and User-Facing Projects and Core Services
(UFC) staff to provide comprehensive support for TeraGrid users.
An important collaboration between UFC and US staff has resulted in the final test version of a
new “User Services Contact Administration and Logging” LifeRay portlet, which will enable US
staff to manage the assignment of personal contacts not only to TRAC grant awardees after each
quarterly meeting, but also to Startup and Education awardees whose usage patterns indicate a
possible need for proactive assistance. Building on an application previously developed by UFC
for AUS project planning and reporting, this web page will also enable consultants to take notes
of their interactions with the users, greatly enhancing the User Engagement function of the User
Champions program. The application is now in final testing, and has been used for assigning
contacts to the awardees of the June 2010 TRAC meeting.
Based on work done on ASTA and other AUS projects, AUS staff continued to present papers at
conferences, and submit and publish journal papers. Multiple AUS and US staff were involved in
organizing the TG10 conference. This included submitting abstracts/papers/tutorials for the
31

conference, reviewing abstracts/papers for the conference, selecting invited speakers for the
science track etc.
As a product of the Advanced User Support project on Performance and Analysis Tools, Mahin
Mahmoodi of PSC created an online tutorial TAU: Parallel Performance Profiling and Tracing
Tool that was made available on the TeraGrid AUS projects website during the last week of June.
TAU is an open-source and advanced performance tool that can be used in profiling scientific
applications. The tutorial provides TAU usage information for Kraken and Ranger. It introduces
commonly used features, provides quick references, and assists users to successfully start using
TAU on the systems. The tutorial also provides a guide for navigating visualization features of
TAU which are essential in extracting and analyzing performance data. The material is available
at https://www.teragrid.org/web/user-support/tau.
The SDSC User Support team helped move Dash into production by installing software,
developing documentation, and setting up the user environment (using modules). Support has
been provided to users of Dash to help with porting codes and running jobs using Dash’s unique
features (flash memory, vSMP). User support personnel also tested performance of several codes
on Dash (NAMD, GAMESS, VASP, ABAQUS).
The TACC Visualization and Data Analysis (VDA) group has been assisting Dr. Clint Dawson’s
(UT Austin, Center for Subsurface Modeling) visualization efforts related to the Deepwater
Horizon oil spill. Dawson’s team is simulating the path of oil from the BP Deepwater Horizon oil
spill using simulations of different past hurricanes (IKE, GUSTAV, RITA) computed using the
ADCIRC coastal circulation model coupled to the unstructured SWAN wave model. The
visualization effort by TACC staff has been focused on the overlay of particle movement and
satellite or aerial imaging data. The particles in the visualization represent the oil spill and their
position is either hypothetical or reflect the position of the oil on the surface. The data has been
visualized using Longhorn and MINERVA, which is an open source geospatial software
primarily developed by TACC members. The data is generated daily and it's approximately 100
GB is size.
Members of the TACC VDA group also assisted Dr. Tom Fogal at SCI (University of Utah) as he
recently used Longhorn to perform some of the largest visualizations ever accomplished. Dr.
Thomas Fogal wrote, “Longhorn is an impressive machine: using only 256 processes, we
performed some of the largest-ever published volume renderings. This was in major part due to
the large number of GPUs available, in addition to helpful staff that aided us in accessing them
for our visualization work.”
5.2 User Engagement
User engagement is done by various means. Under the User Champions program, RP consultants
are assigned to each TRAC award right after the results of an allocations meeting become known.
The assignment takes place by discussion in the user services working group, taking into account
the distribution of an allocation across RP sites and machines, and the affinity between the group
and the consultants based on expertise, previous history, and institutional proximity. The assigned
consultant contacts the user group as their champion within the TeraGrid, and seeks to learn about
their plans and issues. We leverage the EOT area’s Campus Champions program (Section 10) to
fulfill this same contact role with respect to users on their campuses, especially for Startup and
Education grants. Campus Champions are enrolled as members of the user services working
group, and thus being trained to become “on-site consultants” extending the reach of TeraGrid
support. To help users efficiently scale their codes and ancillary data processing flows to the tens
of thousands of processes (on Ranger and Kraken), the Advanced User Support and User Support
areas jointly operate the extreme scalability working group. Led by Nick Nystrom (PSC), this
team brings together staff experts from all RP sites, external partners working to develop
32

petascale applications and tools, as well as TeraGrid users and awardees of the NSF PetaApps,
SDCI and STCI programs.
Purdue RP Staff, in response to an increasing number of tickets requesting assisting with setting
up login access to Steele and Condor as well as input from the Campus Champions group for a
“simple” method to log into all resources for new users, created three step-by-step tutorials
(including graphics) that show how to download and use the GSI-SSHTerm Java application as
well as how to customize and save the resource settings and use the built-in sftp file transfer
application. Two of these tutorial documents are finalized and currently located on the Campus
Champions Wiki under the “Helping TeraGrid Users” heading
(http://www.teragridforum.org/mediawiki/index.phptitle=Campus_Champions#Helping_TeraGr
id_users). These tutorials will also be made available soon on the local Purdue TeraGrid website.
The third document is currently in review and will be published soon. Purdue RP Staff also
identified a need to assist Campus Champions with identifying and matching their users’ research
requirements to TeraGrid resources. Based on this need, Purdue RP Staff created and presented a
session on “Matching Users to Resources on the TeraGrid” for the Campus Champions.
5.3 Frontline User Support
In this section we describe the frontline user support provided by TeraGrid user support staff. In
addition, TeraGrid provides online documentation, a knowledgebase and training support for
“self-serve” help and this is explained in detail in §6. By means of the User Interaction Council,
which is chaired by the GIG Director of Science, Frontline and Advanced support functions are
operationally integrated with the TeraGrid Operations Center (TOC) and the “self-serve” online
help team.
The TOC help desk is the first tier of user support for the TeraGrid. It is staffed 24X7 at NCSA.
All TeraGrid users are asked to submit problem reports to the TOC via email to
help@teragrid.org, by web form from the TeraGrid User Portal, or via phone (866-907-2382).
The TOC creates a trouble ticket for each problem reported, and tracks its resolution until it is
closed. The user is automatically informed that a ticket has been opened and advised of the next
steps, as well as of a ticket number enabling follow-up.
If a ticket cannot be resolved within one hour at the TOC itself, it gets passed on to the second
tier of user support. It is referred to the RP where the problem has been experienced, or, if the
problem appears to be TeraGrid-wide, it is reported to the full user services working group.
There, it is assigned to a user consultant who begins by discussing the matter with the user, in
order to narrow the diagnosis and to understand if the issue lies with the user’s application, with
the RP or TeraGrid-wide infrastructure, or both. Tiger teams are formed by the user services
working group as necessary, to diagnose and solve complex problems that implicate several RP
sites or fields of expertise. The personal contacts assigned to TRAC awarded projects (via the
User Champions program), as well as the Campus Champions, proactively seek to develop
porting and optimization plans with their assigned users. They act as proxies for “their” users
with respect to managers, systems administrators and vendors at their local RP site and (via the
user services working group) at other RP sites, ensuring support for any special arrangements that
may be required for debugging, testing and benchmarking new or upgraded user codes.
As needed, consultants may request the assistance of advanced user support staff (the third tier of
user support), systems, or vendor experts. The immediate goal is to ensure that the user can
resume his or her scientific work as soon as possible, even if addressing the root cause requires a
longer-term effort. When a proposed root cause solution becomes available, we contact the
affected users again and request their participation in its testing. When a user’s problem requires
active assistance from a M.S.- or Ph.D.-level staff exceeding approximately one person-month of
33

effort, the issue is referred to the third tier of user support, i.e. for Advanced User Support
described in §5.5 following the guidance of the allocations process. In terms of experience and
qualifications, frontline support consultants range from the B.S. level to the Ph.D. level, with
many of the latter dividing their effort between frontline and advanced user support.
The framework for coordinating frontline support across the TeraGrid is the user services
working group which assembles staff from all the RP sites under the leadership of the Area
Director for User Support. It enables each site’s staff to leverage the experience and expertise of
all others, and a concerted approach to solving cross-site user problems based on sharing and
adopting best practices. The working group functions by means of an email list and weekly
conference calls. It coordinates the User Champions and Campus Champions programs that, as
we mentioned above, provide user engagement at the personal level. By involving the Champions
in resolving tickets submitted by “their” users, we can improve the promptness and quality of
ticket resolution, and often prevent problems from occurring in the first place, or from recurring
repeatedly.
5.3.1 Helping users port software into a TeraGrid environment
In this section, in table 5.1 below, we provide some instances of assistance provided by our
consultants to enable or improve users’ work on TeraGrid resources, which did not rise to the one
FTE-month involvement level that defines “advanced” user support.
Table 5.1. Examples of Frontline User Consulting Porting Software on TeraGrid Resources
PI Name RP(s) Involved Name of software Nature of improvement
Steve Rounsley, U.
Arizona
PSC Abyss Helped a group new to
TeraGrid get genome
assembly software running
on Pople.
Greg Voth, U. Utah PSC, TACC Data generated from
NAMD runs
Mark Ellison,
Ursinus College
Assisted user in
transferring large dataset
(> 1 TB) from retiring
Bigben and PSC archival
storage to the archive at
TACC.
PSC Gaussian Helped user overcome a
problem with unexpected
Gaussian termination by
setting a fault-tolerant
Gaussian runtime option.
Doran Naveh, CMU TACC Siesta 3.0-b Parallel versions of Siesta
and Transiesta compiled
with support for Scalapack,
GotoBlas and netCDF.
Code tested and made
available to users via a
module with access control
for licensing purposes.
Jonas Baltrusaitis,
University of Iowa
Purdue TURBOMOLE Installed and modified run
script to work in the Steele
34

Srinivasa
Jampani,
University
Rao
Drexel
Peter Peumans,
Stanford University
Alenka Luzar,
Virginia
Commonwealth
University
Shahriar Afkhami,
New Jersey Institute
of Technology
environment.
Purdue Gromacs Assisted the user in setting
up a script to run Gromacs
correctly on Steele.
Purdue Lumerical Assisted the user in
installing and configuring
the software to run on
Steele using the user’s
license.
Purdue LAMMPS Assisted the user in
installing and customizing
a specific version of the
software.
Purdue Custom Code Assisted the user in
compiling and running
custom 32 bit serial code
on Steele.
5.3.2 More Frequent User Support Issues/Questions
Among the system or site specific user issues referred to the RPs for resolution, the most frequent
had to do with login/access and account management issues (e.g. TeraGrid Portal password
versus resource specific passwords, password resets, security credentials, adding and removing
users, locating environment variables, allocations on machines leaving and joining the TeraGrid).
In decreasing order of frequency, issues with job queuing (e.g. which queues to use for various
types of jobs, throughput issues, scripting and execution issues); software availability and use
(finding, setting up, building, optimizing and running); system availability and performance
problems (e.g. memory limit questions, compute node failures, I/O timeouts, requests to kill
jobs); file systems (e.g. permissions and quotas, corrupted files), and wide area data transfer and
archive systems questions were also encountered.
5.3.3 Trouble Ticket Statistics
Table 5.2 shows how long tickets remained open during this quarter, that is, how long it took for
TeraGrid user support staff to provide a diagnostic, workaround or solution.
Table 5.2. Number of Q2, 2010 tickets that remained “open” for the indicated time bins.
Open for 1 hour, 1 day, 2 days, 1 week, 2 weeks, 4 weeks 190
35

5.4 RP Operations: User Services
5.4.1 Consulting Services
Ongoing daily operational user support was conducted at the RPs. Consulting is the second tier of
TG’s three-tier user support, with the first tier being the TG Operations Center and the third tier
being the TG Advanced User Support. TG RP Consulting staff provide real-time, frontline
support for their local RP’s resources in response to users’ requests to the TeraGrid Operations
Center and RP consulting systems. This support includes assistance such as allocations issues,
initial login issues for new users, debugging jobs, data transfer, and compiler support. Consulting
support typically engages a user for a relatively short period of time, and may pass a user on to
Advanced Support if longer-term assistance is required. Some examples of site-specific assistance
achievements and their impact on users’ research progress follow.
Under the User Champions program (section 5.2) TACC staff contacted 51 PIs with new or
renewal awards granted for the allocation period beginning April 1, 2010. Responses were
received from 13 PIs and TACC staff members resolved any user issues or referred them to the
appropriate site for resolution. TACC also initiated contact with 151 “startup” principal
investigators whose allocations began in March, April, and May of 2010. Responses were
received from 39 PIs and TACC staff members resolved any user issues or referred them to the
appropriate site for resolution. TACC staff continued to facilitate special user requests, including
resource usage outside normal queue limits, high priority in order to meet deadlines, and special
allocation requests to prevent user research projects from coming to a halt.
SDSC User Support staff tested various research (NAMD, GAMESS, VASP) and commercial
(ABAQUS) codes on the Dash computational resource. Tests included checking performance
benefits of using Dash’s unique features (flash memory and vSMP technology). Preliminary tests
with ABAQUS showed an improvement of up to 50% with use of flash for I/O. Information from
early user experiences and SDSC support staff testing were used to develop an introductory
tutorial which has been accepted for the TeraGrid 2010 conference.
Purdue RP staff member Kim Dillman took a leading role in the testing of the new User Services
Contact Administration and Logging web page (see Section 5.1). She currently serves as a
“volunteer” Startup Reviewer for new TeraGrid startup/renewal/supplemental allocation requests
and has reviewed 63 startup requests during Q2. Leveraging her experience working as the RP
support expert and her contact with many campus champions, Kim also assisted Campus
Champions with questions/issues in support of their institution. The issues range from managing
their TeraGrid allocations for campus users to identifying appropriate TeraGrid resources for
specific research projects from the campuses that include Bingham University, University of
Arkansas, and Pittsburg State University (Kansas).
5.4.2 Helpdesk
TG RPs provided local helpdesk first-tier support that integrates with the TeraGrid Operations
Center. This support works to provide rapid response email and phone responses for questions
and trouble reports from the user community. Helpdesk staff direct users’ queries to Consulting
or Advanced user support in cases where second-tier or third-tier expertise is required.
5.5 Advanced User Support
Advanced User Support (AUS) staff are located at the RP sites and are coordinated by the AUS
Area Director. The highest level of long-term user support for a TeraGrid user or user group is
provided by the AUS staff. The overall advanced support efforts are categorized by three sub-
36

efforts: Advanced Support for TeraGrid Applications (ASTA), Advanced Support for Projects
(ASP), and Advanced Support for EOT (ASEOT).
Operations of the AUS area is coordinated by the AUS Area Director jointly with the AUS Point
of Contacts (POC) who are in most cases the managers or group leaders of the scientific
computing applications groups at each of the RP sites. All the management and coordination
issues such as analyzing TRAC ASTA review reports, contacting PIs for further understanding of
ASTA plans, matching of appropriate AUS staff to ASTAs, discussing/initiating new ASPs,
matching of AUS staff to ASPs, discussing ASEOT topics, preparing program plan and annual
reports, etc., are handled by this group. They hold biweekly teleconferences for the management
and coordination issues and continue discussions via email as needed.
On alternate bi-weeks a technical tele/web-conference is hosted using ReadyTalk for all the AUS
technical staff as well as the AUS POCs. At this technical tele/web-conference 1-2 technical
presentations are made by AUS technical staff discussing progress of ASTA projects, progress of
ASPs, and any other technical issue that is of interest to all the AUS technical staff. ReadyTalk
allows all the AUS staff, attending these bi-weekly tele/web-conferences to view the slides and
listen and participate in the technical presentations. This fosters an environment for collaboration
among the AUS staff located at the various RP sites and also allows sharing of technical insight
that may have impact on multiple ASTAs or ASPs. The presentation slides and audio recordings
are archived and made publicly available for interested staff as well as users at:
http://teragridforum.org/mediawiki/index.phptitle=Technical_presentations .
5.5.1 Advanced Support for TeraGrid Applications (ASTA)
Advanced Support for TeraGrid Applications (ASTA) efforts allow AUS staff to work with a user
for a period of a few months to a year. Activities include porting applications, implementing
algorithmic enhancements, implementing parallel programming methods, incorporating math
libraries, improving the scalability of codes to higher core counts, optimizing codes to utilize
specific resources, enhancing scientific workflows, providing help for science gateways and
performing visualization and data analysis projects. To receive ASTA support, TeraGrid users
submit a request as a part of their annual, Supplemental, or Startup resource allocation proposal.
The recommendation score provided by the reviewers is taken into account and upon discussion
with the user regarding a well-defined ASTA workplan, AUS staff provide ASTA support to the
user. This support often impacts a larger number of users who are part of the PI’s team. Since late
2008, in addition to the quarterly TRAC allocations, ASTA support is also available to those
requesting Supplements or Startup allocations. The TeraGrid-wide AUS effort improves the
ability to optimally match AUS staff to an ASTA project by taking into account the reviewers’
recommendation score, the AUS staff(s) expertise in a domain science/HPC/CI, the ASTA project
workplan, and the RP site where the user has a resource allocation. In addition to providing longterm
benefits to the user or the user team, projects are also beneficial to the TeraGrid as a whole.
Results of ASTA projects provide insights and exemplars for the general TeraGrid user
community and these are included in documentation, training and outreach activities. In the table
5.3 we list 47 ASTA projects that were active and 6 ASTA projects that were completed during
the quarter. This list also includes Startup and Supplemental ASTA projects. Quarterly updates of
ASTA work for most of these ASTA project are provided after the table.
ASTA Project
The Effect of
Gravitational
Instabilities on
Table 5.3. List of continuing and completed ASTA projects for the quarter.
Site(s)
Involved
Status
Indiana Continuing through
09/10
Notes
37

Planet Migration in
Protoplanetary
Disks, PI Durisen, IU
DNS of Spatially
Developing
Turbulent Boundary
Layers, PI Ferrante,
U. Washington
Multiscale Analysis
of Size Dependence
of Deformation and
Fracture of Hierarchy
Protein Materials, PI
Buehler, MIT
An Earthquake
System Science
Approach to
Physics-based
Seismic Hazard
Research, PI Jordan,
USC
Direct Numerical and
Large-eddy
Simulation of
Complex Turbulent
Flows, PI Mahesh,
U. Minnesota
Modeling Studies of
Nano
and
Biomolecular
Systems, PI
Roitberg, U. Florida
Efficient
Implementation of
Novel MD Simulation
Methods in
Optimized MD
Codes, PI Voth, U.
Chicago
Flow and Nutrient
Transport in 3D
Porous Scaffolds
Used for Bone
Tissue Growth, PI
Papavassiliou, U.
Oklahoma
Biomechanics of
Stem Cells, PI Finol,
SDSC, NCSA,
TACC, NICS
Continuing
03/11
through
SDSC Continuing through
12/10
SDSC,
TACC,
NCSA
PSC,
NICS,
Continuing
09/10
through
SDSC Continuing through
09/10
SDSC,
NICS
NCSA,
PSC, NICS,
TACC, NCSA
Continuing
09/10
Continuing
09/10
through
through
PSC Continuing through
06/11
PSC Continuing through
09/10
38

CMU
Global Kinetic
Simulations of the
Magnetosphere, PI
Karimabadi, UCSD
EpiSims, PI Roberts,
RTI
International;NekTar-
G2, PI Karniadakis,
Brown U.; GENIUS,
PI Coveney, UCL
Enabling Neutron
Science Research
with the Neutron
Science TeraGrid
Gateway, PI Cobb,
ORNL
Simulation of Liquid
Fuel Combustors, PI
Mashayek, UIC
Multichannel
Scattering Theory
via the Modified
Faddeev Equation,
PI Hu, CSULB
Turbulence
Simulations Towards
the PetaScale:
Intermittency,
Mixing, Reaction and
Stratification, PI
Yeung, Georgia
Tech
First-Principles
Molecular Dynamics
for Petascale
Computers, PI Gygi,
UCD
UNRES Force-field
for Simulation of
Large Molecular
Systems, PI
Scheraga, Cornell
Petascale Adaptive
Computational Fluid
Dynamics, PI
Jansen, RPI
SDSC, NICS Continuing through
06/11
NIU,
NCSA,
TACC,
Purdue
SDSC,
ANL,
IU,
Continuing
09/10
through
ORNL Continuing through
03/11
NCSA Continuing through
09/10
TACC Continuing through
09/10
NICS,
TACC
SDSC,
Continuing
09/10
through
NICS, TACC Continuing through
12/10
PSC Continuing through
12/10
NICS,
PSC
TACC,
Continuing
12/10
through
This is also part of the
Advanced Support
Project of MPig and
cloud computing (see
section 5.5.2)
39

Insight into
Biomolecular
Structure, Dynamics,
Interactions and
Energetics from
Simulation, PI
Cheatham, U. Utah
Large Eddy
Simulation of
Particle-Turbulence
Interactions in
Complex Flows, PI
Apte, Oregon State
Nonlinear Evolution
of the Universe, PI
Cen, Princeton
Modeling Global
Climate Variability
with the Multi-scale
Modeling
Framework: The
Boundary-layer
Cloud Problem, PI
Helly, SIO/UCSD
Lattice Gauge
Calculation of
Hadronic Physics, PI
Liu, University of
Kentucky
Leveraging
Supercomputing for
Large-scale Gametheoretic
Analysis, PI
Sandholm, CMU
Astrophysical
Applications of
Numerical Relativity:
Coalescing Binary
Systems and Stellar
Collapse, PI
Schnetter, LSU
Hierarchical
Bayesian Models for
Relational Learning,
PI Gordon, CMU
Fractals in Elasto-
Plasticity, PI Ostoja-
Starzewski, UIUC
SDSC Continuing through
12/10
TACC Continuing through
03/11
TACC,
NICS
SDSC,
Continuing
09/10
through
Purdue, SDSC Continuing through
06/11
PSC, NICS Continuing through
03/11
PSC Continuing through
03/11
PSC,
SDSC
NCSA,
Continuing
03/11
PSC Completed 04/10
through
NCSA Continuing through
01/11
40

Heat Transfer
Simulation of
Suspension
Micro/nano Flow
Using a Parallel
Implementation of
Lattice-Boltzman
Method, PI Joshi,
Georgia Tech
The CIPRES
Science Gateway, PI
Miller, SDSC/UCSD
Coupling of
Turbulenct
Compressible Solar
Convection with
Rotation, Shear and
Magnetic Fields, PI
Toomre, U. Colorado
Effects of Star
Formation and
Supernova
Feedback on Galaxy
Formation, PI Choi,
U. Nevada, Las
Vegas
High-Resolution
Modeling of
Hydrodynamic
Experiments, PI
Demeler, U. Texas
Health Science
Center
Imine Formation in
an Enzyme Active
Site, PI LLoyd,
Oregon Health and
Science University
The Roles of
Vortices and Gravity
Waves in Plane
Formation in
Protoplanetary Disks
and in Jupiter’s
Atmosphere, PI
Marcus, U. California
Berkeley
Magnetized
Astrophysical
Purdue, TACC,
PSC
Completed 06/10
SDSC Continuing through
06/11
PSC,
TACC
NICS,
Completed 06/10
TACC Continuing through
06/10
IU,
Purdue
PSC,
Continuing
03/11
through
SDSC Continuing through
09/10
TACC, SDSC Continuing through
09/10
NCSA, TACC Continuing through
09/10
Also see Science
Gateways section of the
quaterly report
(UltraScan)
41

Plasmas with
Anisotropic
Conduction,
Viscosity, and
Energetic Particles,
PI Parrish, U.
California Berkeley
On the Decay of
Anisotropic
Turbulence, PI
Perot, U. of
Massachusetts,
Amherst
Simulations of
Coarsed Grained
Macromolecular
Fluids, PI Guenza,
U. Oregon
ExtremeOpenMP –
A Programming
Model for Productive
High
End
Computing, PI Tafti,
Virginia Polytechnic
Inst and State U.
High Throughput
Modeling of
Nucleosome
Stability, PI Bishop,
Tulane U.
Center for Integrated
Space Weather
Modeling, PI Quinn,
Boston U.
Parallelization of a
Finite Element Code
for Petascale
Dynamics Modeling,
PI Cui, SDSC
Computational
Modeling of Thermal
Stripping in Nuclear
Reactors, PI Kimber,
U. Pittsburgh
Simulation of
Complex Organic
Mixture
Biotransformations in
Natural Systems, PI
NCSA Continuing through
09/10
SDSC Completed 06/10
NCSA Completed 06/10
SDSC Completed 06/10
NICS, TACC Continuing through
12/10
SDSC Continuing through
08/10
PSC Continuing through
12/10
PSC Continuing through
04/11
42

VanBriesen, CMU
Villous Motility as a
Critical Mechanism
for Efficient Nutrient
Absorption in the
Small Intestine, PI
Brasseur, Penn
State
Systematics of
Nuclear Surface
Vibrations in
Deformed Nuclei, PI
Engel, U. North
Carolina
Simulations of
Molecular Clouds
and Protoplanetary
Disks, PI Mac Low,
American Museum
of Natural History
Structural and
Functional
Characterization of
Glu-Plasminogen
and
Lys-
Plasminogen, PI
Kim, CMU
Design
of
Communicating
Colonies of
Biomimetic
Microcapsules, PI
Kolmakov, Univ. of
Pittsburgh
Computational
Modeling of Ultracold
Molecular Collisions
and Reactions
Involving Clusters
and Nanoparticles,
PI Naduvalath,
UNLV
Development of
Novel HIV Entry
Inhibitors Using the
ROCS Shaped
Based Matching
Algorithm and
Molecular Dynamics
NICS, SDSC Continuing through
03/11
NICS, TACC Continuing through
03/11
PSC Continuing through
03/11
PSC Continuing through
03/11
TACC,
NCSA
SDSC,
Continuing
03/11
through
NICS, PSC Continuing through
03/11
PSC Continuing through
03/11
43

Studies of gp120
Envelop Proteins, PI
LaLonde, Bryn Mawr
College
Simulation and Data
Analysis of Macro-
Molecular Systems,
PI Blaisten-Barojas,
George Mason
University
Applications of Bispeptide
Nanostructures, PI
Schafmeister,
Temple University
Finite-elements
Modeling of Singleand
Poly-Crystalline
Material Removal
Including
Crystallographic
Effects,
PI
Ozdoganlar, CMU
PSC Continuing through
03/11
NICS Continuing through
04/11
PSC Continuing through
03/11
PI: Durisen (Indiana U, Astronomy). The Effect of Gravitational Instabilities on Planet Migration
in Protoplanetary Disks. Continuing through 09/10. The ASTA staff for this project are Don
Berry and Robert Henschel from the High Performance Applications group of Research
Technologies at Indiana University. In this quarter the ASTA staff continued to work with faculty
from the IU astronomy department to refine the requirements for the proposed automated
workflow system. Work continued on all the applications of the package and to produce standard
data products. Performance tests of the code was done specially for postprocessing routines on
the clusters.
PI: Ferrante (U Washington, Fluids, Particulate and Hydraulic Systems). DNS of Spatially
Developing Turbulent Boundary Layers. Continuing through 03/11. The ASTA team for this
project consists of Darren Adams (NCSA), David Bock (NCSA), John Peterson (TACC), Lonnie
Crosby (NICS) and Dmitry Pekurovsky (SDSC). The team continued to collaborate with the PI
on the integration of a Parallel HDF5 file format with the PI’s existing simulations code. TACC
staff continued to provide support for the SVN repository for user’s codes. Further support
activities such as compiling, porting, scaling and general help running the code etc. are also
occasionally provided. A shift to more visualization support is anticipated as new data is
generated using a new HDF5 file format.
Some software engineering and other ASTA assistance has basically been completed with some
minimal ongoing support as stated below:
• Improvements were made to the src code for scalability, speed, and portability
• Source code was split from a single F77 code into many files and some were converted to
F90 with improved interfaces
44

• Subversion source management has resulted in better collaboration
• ASTA work has allowed runtime parameters control grid size and process count as well
as other IO-related settings
• SPRNG random number library was added
• A project wiki space was created for the project on NCSA wiki: wiki.ncsa.uiuc.edu and
the wiki is being used to develop a code developer/user guide. This is very beneficial as
there are always new students who need to get up to speed with the (rapidly changing)
software
• It is anticipated that the wiki will help with documentation and legacy support once the
active development stage of the project collaboration has completed
• 3D transpose routines were integrated with the existing code
Darren’s main focus was to design a new HDF5 date format and optimize I/O routines. The new
HDF5 code, developed at NCSA for the PI’s DNS data has these basic properties:
• A lightweight c library (dubbed “h5dns”) with Fortran interface routines. The library is
designed to be used both as a writer (from the dns code) and a reader (from dns code and
visualization codes)
• The c library only requires HDF5 (usually with MPI –“PHDF5”)
• All hdf I/O routines will be optimized for parallel I/O when they are compiled with MPI
• Files are intended to be “self-describing” with attributes and metadata datasets
• The file format will be used for both 2D and 3D Rectilinear Grid data consisting of both
scalar and 3d vector variables
• Information about the per MPI-process layout of the data is preserved as supplementary
metadata to the main datasets. The datasets themselves are single array spanning the
global computational grid.
Initial testing of basic I/O functionality and unit testing of the library are happening currently. It
is expected that runs will be done in early to mid August to get new visualization dump via the
HDF5 routines and it will allow to explore the data in both VisIt and with David Bock’s
visualization tools.
Following many tasks remain to be completed:
• Fully integrate HDF5 routines with the main code
• Test new data format for correctness
• Create reader routine for VisIt
• Integrate Isotropic turbulence code into new executable
• Update code to perform 2d domain decomposition
• Optimize I/O patterns both by system and by taking special measure when running at
very large scale
PI: Buehler (MIT, Mechanics and Materials). Multiscale Analysis of Size Dependence of
Deformation and Fracture of Hierarchy Protein Materials. Continuing through 12/10. The
project is lead by Ross Walker (SDSC). The PI used the executables, build by Walker
45

previously, to run calculations. ASTA help was provided to produce some ray-traced pictures for
the PI.
PI: Jordan (University of Southern California, Earth Sciences). SCEC PetaScale Research: An
Earthquake System Science Approach to Physics-based Seismic Hazard Research. Continuing
through 09/10. The ASTA team consists of Yifeng Cui (SDSC), John Urbanic (PSC), Byoung-Do
Kim (TACC), Kwai Wong (NICS), Mark Vanmoer (NCSA), and Amit Chourasia (SDSC).
Highlights of ASTA effort for the quarter include:
• Support for SCEC M8 dynamic rupture simulation on NICS Kraken using the AWM
code; the input for M8 wave propagation simulation was prepared on ORNL Jaguar
machine
• Time was spent to debug and test compiler issues on Kraken for AWM runs at very large
scale
• Publication: Cui, Y., Olsen, K., Jordan, T., Lee, K., Zhou, J., Small, P., Ely, G., Roten,
D., Panda, DK, Levesque, J., Day, S. and Maechling, P.: Scalable Earthquake Simulation
on Petascale Supercomputers, submitted to SC10, New Orleans, Nov, 2010
• Hercules code work: Additional large scale (100K PE) optimization was done. In
particular, Lustre limitations have required that checkpointing also now be scaled using
techniques related to those used for the IO of scientific results IO. This work is in
progress as full checkpointing at very large scale pushes the envelope of Lustre
capability. The opportunity was also taken to refactor all related part of the code.
• Visualization work: Support continued by creating visualization of M8 simulations.
PI: Roitberg (U. Florida, Biophysics). Modeling Studies of Nano and Biomolecular Systems.
Continuing through 09/10. The ASTA team consists of Ross Walker (SDSC), Sudhakar
Pamidighantam (NCSA) and Christian Halloy (NICS). During this quarter Walker continued
work with Prof Roitberg to further develop a GPU accelerated version of AMBER. In April 2010
we version 11 of the AMBER software was released and this included support for NVIDIA GPU
acceleration by default. Prof Roitberg is now using this for his simulations. ASTA work is now
ongoing with him to develop a version of AMBER that runs on multiple GPUs using MPI. This
will likely take the next 3 months or so and then Prof Roitberg will be renewing his TRAC
proposal and requesting an extension to the AUS project in order to finish this work. During this
quarter a joint NSF SSE proposal was submitted and it requested funds to extend this project
further to support QM/MM simulations and additional more advanced MD features such as
Thermodynamic Integration, Replica Exchange etc. A manuscript describing the GPU
implementation is in preparation.
PI: Voth (University of Chicago, Physical Chemistry). Efficient implementation of novel MD
simulation methods in optimized MD codes. Continuing through 09/10. The ASTA team consists
of Phil Blood (PSC), John Peterson (TACC), and Lonnie Crosby (NICS). Recently the Voth
group completed the initial port of their MS-EVB molecular dynamics code to LAMMPS
molecular dynamics simulation package (LAMMPS EVB). The MS-EVB method allows the
calculation of quantum effects like proton transport from within an MD simulation much more
efficiently than traditional quantum methods. Nevertheless, the algorithm presents extreme
scaling challenges since the long-range electrostatics (requiring a 3D FFT with all-to-all
communication) must be recalculated many times each time step, as opposed to a single time for
regular MD simulation. During this quarter, Blood has continued work to find ways to overcome
these severe scaling challenges and completed an initial analysis of the opportunities to save
communication time in the LAMMPS EVB code. Based on this initial analysis, it appears that it
46

may be necessary to use a different method for the long-range electrostatics calculation that can
avoid the need for an all-to-all communication. Recently, a scientist that the PSC devised a
method for performing a 3D FFT without all-to-all communication. The ASTA team is exploring
the possibility of putting this code into LAMMPS to help address the scaling problem. In
addition, a group at Temple has recently developed an experimental version of LAMMPS that
uses hybrid MPI/OpenMP programming to achieve better balance between computation and
communication at high processor counts in the regular version of LAMMPS
(http://sites.google.com/site/akohlmey/software/lammps-icms). During this quarter, Blood
ported the LAMMPS EVB code to work with the new hybrid version of LAMMPS. Initial tests
indicate that this may help alleviate some of the scaling problems, although additional effort, as
described above, will be needed to achieve good scaling.
PI: Papavassiliou (U. Oklahoma, Chemical, Thermal Systems). Investigation of Flow and
Nutrient Transport in PorousScaffolds Used for Bone Tissue Growth. Continued through 06/11.
ASTA staff for this project is Raghu Reddy (PSC). Interaction is ongoing with PI’s student
regarding profiling runs of PI’s codes.
PI: Finol (CMU, Mechanical and Biomedical Engineering). Biomechanics of Stem Cells.
Continuing through 09/10. The ASTA consultant is Anirban Jana (PSC). In this project, the
extruded mesh for the arterial wall using prism elements (code developed inhouse, see last
quarterly report) was tested with ADINA. However, it was discovered that for nonlinear isotropic
and anisotropic materials, the prism element is not a good choice. Quadratic brick elements are
better suited for this. Hence an algorithm that split each prism element into 3 brick elements was
incorporated into the extrusion code. Another outstanding issue is that isoparametric element
formulation, that allows an element to have curved edges, may improve the accuracy of the
simulations with fewer elements. Anirban and the PI’s team have been looking into the pros and
cons of incorporating this into the extrusion code. Finally, in the latest ADINA versions (8.6
onwards), the interface for user-defined Mooney-Rivlin type hyperelastic materials has changed.
Effort is underway on updating the user defined subroutines accordingly. .
PI: Karimabadi (UCSD, Magnetospheric Physics). Global Kinetic Simulations of the
Magnetosphere. Continuing through 06/11. The ASTA staff for this project are Mahidhar
Tatineni (SDSC) and Glenn Brook (NICS). The ASTA effort focused on performing various tests
for I/O performance on the Lustre filesystem of Kraken system and analysis of those results.
Multiple tests runs were made to see the impact of the filesystem on I/O timing variability. I/O
methods were implemented where “gating” of processors were done i.e. allowing a set of
processors, instead of all the processors, to write at a time to the Lustre parallel file system and
I/O performance enhancement was achieved.
PI: Roberts (RTI International, Computer Science) EpiSims; PI: Karniadakis (Brown U., CFD)
NekTar-G2; PI Coveney (University College of London, Chemistry) GENIUS. Continuing
through 09/10. The lead, regarding MPIg research and implementation, for these ASTA projects
is Nick Karonis (NIU, ANL) and he works with user support consultants from NCSA, TACC,
and SDSC, and more recently with Indiana, Purdue, and the University of Chicago. Based on
conversations with the user, ASTA effort is focusing on aspects found in the MPI standard and
other more ubiquitous technologies, namely, multithreading and MPI-2's RMA operations.
Karonis group have continue to work with Karniadakis and Grinberg to identify the
computational bottlenecks in the application. Next step is focus on those sections with the goal of
introducing multithreading, where possible, and then possibly RMA operations. The primary
motivation for adopting this new strategy is to try and find performance gains from memory
hierarchies.
47

PI: Mashayek (University of Illinois Chicago, CFD). Simulation of Liquid Fuel Combustors.
Continuing through 09/10. The ASTA consultant is Ahmed Taha (NCSA). Work continued to do
scalability study for parallel Fluent v.12.1.4 version for the following cases:
i. Comparing the LES simulations without microjets to the experiment data of the cold flow;
ii. Comparing the LES simulations with microjets to the experiment data of the cold flow;.
PI: Hu (California State University, Long Beach, Theoretical Physics). Applications of the
Multichannel scattering theory via the ModifiedFadeev Equation. Continuing through 09/10.
Gabriele Jost and Bill Barth (TACC) are providing ASTA support for this project. > 1. The issue
regarding excessive time spent in reading the input and distributing the data has been resolved by
changing the IO procedure. In the original code the master process reads all of the data, then
distributes the data to the other processes. This has been changed such that now all processes read
the data they require in parallel. Dr. Hu has tested the modified code and confirmed that it
reduces the run time to be acceptable. Two tasks are ongoing currently. (i) Work is ongoing with
Dr. Hu on changing the currently used dense matrix solver based on Scalapack to a sparse direct
solver. ASTA staff are investigating the usefulness of the SuperLU and Mumps packages. (ii)
Another work is on setting up the execution of multiple sequential runs using the Ranger launcher
utility. Dr, Hu is currently testing the suggested procedure for this and will report back soon.
Dr. Hu has also sent a TRAC request to extend the project and has provided more details on the
work that was done in the TRAC allocation proposal.
PI: Yeung (Georgia Tech, Fluids). Turbulence Simulations Towards the PetaScale: Intermittency,
Mixing, Reaction and Stratification. Continuing through 09/10. The ASTA team consists of Kwai
Wong (NICS), Dmitry Pekurovsky (SDSC), Bill Barth (TACC) and visualization experts from
TACC. In this quarter partial ASTA effort was provided in porting the 3D FFT kernel to IBM
Power7 with OpenMP threads.
PI: Gygi (UCD, Materials Science). First-Principles Molecular Dynamics for Petascale
Computers. Continuing through 12/10. The ASTA staff for this project are BD Kim (TACC) and
Bilel Hadri (NICS). Kim provided help with Qbox application and specifically on using gotoblas,
scalapack and MKL. The PI was interested in suggestion regarding better scalability. The use of
scalapack and gotoblas gave minor performance improvement compared to MKL case. While
helping the PI on this, some other issues were resolved and that allowed to move on to the next
step. PI was also trying to do scalability test of the hybrid version of Qbox on Ranger, and Kim
set him up for the large queue and provided previous test cases with some other hybrid
applications. Hadri provided help regarding optimizing the Qbox code with OpenMP directives
and about multi-threaded linear algebra libraries. Help was also provided to enable uuid library
on Kraken by fixing the code and providing directions regarding how to link dynamically the
code with uuid library.
PI: Scheraga (Cornell U., Biophysics). UNRES Force-field for Simulation of Large Molecular
Systems. Continuing through 12/10. The ASTA team consists of Phil Blood and Mahin
Mahmoodi (PSC). ASTA team stayed in touch with the PI’s group but no major support issues
were needed this quarter.
PI: Jansen (RPI, Fluids). Petascale Adaptive Computational Fluid Dynamics. Continuing
through 12/10. ASTA team for this project consists of Glenn Brook (NICS), David O’Neal
(PSC), and John Peterson (TACC). There is no major update for this quarter.
PI: Cheatham (U. Utah, Biochemistry). Insight into Biomolecular Structure, Dynamics,
Interaction and Energetics from Simulation. Continuing through 12/10. The ASTA team consists
of Ross Walker (SDSC). During this quarter work continued to develop the parallel version of
48

Prof. Cheatham’s ptraj MD analysis program. This has included adding support for additional
parallel ‘actions’ as well as fixing several bugs related to reading and writing compressed files in
parallel that were not apparent until the code received wide spread usage. The first version of
parallel ptraj was released as Open Source as part of the AmberTools 1.4 package and is available
for download from http://ambermd.org/ .
PI: Apte (Oregon State University, Fluid, Particulate, and Hydraulic Systems). Large Eddy
Simulation of Particle-Turbulence Interactions in Complex Flows. Continuing through 03/11.
The ASTA staff for this project is Carlos Rosales (TACC). In the previous year several
benchmarking of PI’s code was done and currently ASTA staff is waiting to hear back from the
PI for further interaction.
PI: Liu (University of Kentucky, Elementary Particle Physics). Lattice Gauge Calculation of
Hadronic Physics. Continuing through 03/11. The ASTA team consists of Raghu Reddy (PSC)
and Haihang You (NICS). Raghu continued some of the profiling work initiated by You. IPM
was used to get performance and communication information for PI’s code. PI’s group were
interested in more detailed performance information and TAU is being used for this. ASTA staff
are dealing with issues related to TAU and some potential bug in the code and this is being
investigated. Following two papers were posted by the PI’s group and they summarize the
scientific work of the PI’s group:
http://arxiv.org/abs/1005.5424
http://arxiv.org/abs/1005.4158.
PI: Helly (UCSD, SIO, Atmospheric Sciences). Modeling Global Climate Variability with the
Multi-scale Modeling Framework New parameterizations of Cloud Micro-physics and
Developing Community Community Accounts Portal for Running MMF. Continuing through
06/11. Phil Cheeseman (Purdue), Adam Jundt (SDSC) and Mahidhar Tatineni (SDSC) are the
ASTA team for this project. There is no major update on this project for this quarter.
PI: Sandholm (CMU, Computer and Computation Research). Leveraging Supercomputing for
Large-scale Game-theoretic Analysis. Continuing through 03/11. Joel Welling and David O’Neal
(PSC) are working with the PI on this project. The PI’s team won a prize in a computer poker
tournament as described in the following news report:
A Carnegie Mellon team headed by Computer Science Professor Tuomas Sandholm won the
heads-up, no-limit bankroll category in the annual Computer Poker Competition at the
Association for the Advancement of Artificial Intelligence 2010 conference in Atlanta. The team
included Sam Ganzfried, a Ph.D. student in computer science, and Andrew Gilpin (CS09), an
adjunct assistant professor of computer science. For more:
http://www.cmu.edu/news/blog/2010/Summer/all-aces.shtml
Prof. Sandholm has a new grad student and is generating a new code; David and Joel as well as
John Urbanic (PSC) have consulted with the student regarding this. The previous program uses a
custom algorithm to solve what is essentially multiplication by a Kronecker product matrix; the
grad student was studying literature on Kronecker product algorithms. They are also considering
a new algorithm which is essentially Monte Carlo. Sandholm has applied for a new grant with
the intention of continuing to run the old code while these new codes are developed.
PI: Schnetter (LSU, Gravitational Physics). Astrophysical Applications in Numerical Relativity:
Coalescing Binary Systems and Stellar Collapse. Continuing through 03/11. The ASTA team
consists of Mahin Mahmoodi (PSC), Rick Kufrin and Rui Liu (NCSA), and Wayne Pfeiffer
(SDSC). There is no major update on this project for this quarter.
49

PI: Ostoja-Starzewski (UIUC, Mechanics and Materials) Fractals in Elasto-Plasticity.
Continuing through 01/11. The ASTA staff for this project is Seid Koric (NCSA). Preliminary
studies with the Abaqus Explicit Finite Element solver in 3D have been done, and the team is
now putting together a code to run on TeraGrid. The code will allow to determine the formation
of fractal patterns of plastic grains at elastic-plastic transitions in polycrystals. Given the fact that
(i) one needs very large lattices to reliably assess fractal dimensions, and (ii) large lattices require
very large memory allocations and powerful number crunching, the ASTA support from NCSA is
most helpful. There was a slight delay due to a change of a graduate student working on the
project, but results of publication quality will be produced this summer.
PI: Miller (SDSC/UCSD, System and Population Biology) The CIPRES Science Gateway.
Continuing through 06/11. ASTA support is being provided by Wayne Pfeiffer (SDSC). Wayne
Pfeiffer attended HiCOMB 2010 and IPDPS 2010 in Atlanta and gave a paper at the former. The
paper was titled "Hybrid MPI/Pthreads Parallelization of the RAxML Phylogenetics Code" and
was coauthored with Alexis Stamatakis of the Technical University of Munich.
To help users with special requirements, long phylogenetic analyses were manually run on Dash.
Specifically, long MrBayes jobs were run for
. Jennifer Thomas, formerly of Rutgers and now at the University of London, and
. Joel Ledford of the California Academy of Sciences and UC Berkeley.
PI Miller has also received a Startup allocation on the SDSC Dash machine and the title of the
project is Improving the Efficiency of Eukaryotic Genome Sequence Assembly Using Dash.
Wayne Pfeiffer performed various tests using BFAST and SHRiMP, two codes for mapping
short DNA sequence reads to a large reference genome. In advance of the tests, both codes were
installed on Dash as well as on Triton (a SDSC machine) PDAF. BFAST is parallelized with
Pthreads, while SHRiMP is parallelized with OpenMP.
DNA read data were obtained from Sam Levy of the Scripps Translational Science Institute
(STSI), and baseline runs were made for both codes mapping the reads to the reference human
genome. The run time for both codes on 8 cores of Dash is similar, while SHRiMP runs about
about 2.4x faster when using 32 cores on Triton PDAF. However, the percentage of the reads that
are mapped is higher with BFAST than SHRiMP, so the solution quality may be better with
BFAST. This is being investigated further by Ashley Van Zeeland of STSI.
Both codes do a fair amount of I/O, and hence tests using flash memory instead of disk were done
on a Dash I/O node. The speedup from using flash memory was only a few percent for both
codes, evidently because their I/O is sequential rather than random. Moreover, the compute time
as well as I/O time improved, which suggests that the I/O node is slightly faster than a compute
node, independent of I/O. This was subsequently confirmed by HPCC benchmark tests, though
the reason for the better I/O node performance has not yet been determined.
Tests of the vSMP virtual shared memory software on Dash were also done for SHRiMP. This
code uses a lot of memory, and analyses can exceed the memory available on Dash for some
choices of input parameters. Unfortunately, SHRiMP ran several times slower with vSMP than
without, so vSMP does not appear useful. Analyses with the default parameters will run on Dash
without vSMP, while analyses requiring extra memory can be run on Triton PDAF.
PI: Toomre (U. Colorado, Astronomy) Coupling of Turbulenct Compressible Solar Convection
with Rotation, Shear and Magnetic Fields. Completed 06/10. The ASTA team consists of Raghu
Reddy (PSC) and Haihang You (TACC). There has been no interaction from the PI’s team and
the project has ended.
50

PI: Choi (U. Nevada, Las Vegas, Astronomy and Cosmology) Effects of Star Formation and
Supernova Feedback on Galaxy Formation. Continuing through 06/10. ASTA support is
provided by Lars Koesterke (TACC). There is no update for this quarter.
PI: Demeler (U. Texas Health Science Center, Biochemistry and Molecular Structure and
Function) High-Resolution Modeling of Hydrodynamics Experiments. Continuing through 03/11.
ASTA support is provided by Suresh Marru, Raminder Singh (IU), Roberto Gomez (PSC), and
Phil Cheeseman (Purdue). The ASTA team consists of staff from both the Science Gateway area
and the Advanced User Support area since the PI’s project requires advanced support for
gateways as well as MPI optimization. Further description of activities is provided in the Science
Gateways section.
PI: Lloyd (Oregon Health and Science University, Molecular Bioscience) Imine Formation in an
Enzyme Active Site. Continuing through 09/10. Ross Walker (SDSC) is providing the ASTA
support. Between March 2010 and June 2010 Walker worked with Robert Lloyd and Bud Dodson
to complete their QMMM calculations. This involved fixing several bugs in their code relating to
the constraint of hydrogen atom positions around the edge of the quantum region in the proton
transfer reaction they are attempting to study. These bugs only materialized when running on
more than 16 cores and were due to incorrect reduction of the force array in parallel. These fixes
allowed the PIs to complete their calculations and they are now writing a manuscript for
biochemistry entitled "Imine Reactions in a DNA Repair Enzyme Active Site: Carbinolamine
Formation and Dehydration". Prof Lloyd and Dodson have been grateful of ASTA contributions
and are acknowledging that they couldn't have completed this work without Walker’s
contributions by including his name on the manuscript. Walker will be working over the next
month or so to help revise this manuscript.
PI: Marcus (U. California Berkeley, Astronomy) The Roles of Vortices and Gravity Waves in
Plane Formation in Protoplanetary Disks. Continuing through 09/10. The ASTA staff are Carlos
Rosales (TACC) and Dmitry Pekurovsky (SDSC). PI’s interest is for a better performing 2D
parallel FFT. ASTA team is waiting to have further interaction from the PI’s team.
PI: Parrish (U. California Berkeley, Astronomy) Magnetized Astrophysical Plasmas with
Anisotropic Conduction, Viscosity, and Energetic Particles. Continuing through 09/10. The
ASTA staff are Darren Adams (NCSA) and Paul Navratil (TACC). There is no update regarding
I/O work for this quarter.
PI: Perot (U. Massachusetts, Amherst, CFD) On the Decay of Anisotropic Turbulence.
Continuing through 09/10. ASTA staff is Jay Alameda (NCSA). There is no update for this
quarter.
PI: Tafti (Virginia Polytechnic Inst and State University, Advanced Scientific Computing).
ExtremeOpenMP – A Programming Model for Productive High End Computing. Completed
06/10. Rick Kufrin and Rui Liu, from NCSA participated in this ASTA effort along with the PI
Danesh Tafti and Amit Amritkar. This ASTA project was completed successfully, with the major
outcome that the performance degradations observed in the code have been located and
addressed. The work involved was documented in a paper that has been submitted to the
International Journal of High Performance Computing ("Extreme OpenMP in CFD", Amritkar,
A., Tafti, D., Liu, R., Kufrin, R., Chapman, B.). The team also summarized their work in a talk
presented to the TeraGrid AUS group on May 6, 2010
(http://www.sdsc.edu/~majumdar/AUS_Recordings/NCSA_RICK_5_6_2010/RickAUSRecordin
g/Recording/lib/playback.html).
PI: Bishop (Tulane University, Biophysics). High Throughput Modeling of Nucleosome Stability.
Completed 06/10. The ASTA staff for this project is Adam Jundt (SDSC, previously at LONI).
51

This project has a large number of jobs to run (some with dependencies on previous job runs) and
multiple machines available. The PI is looking for a way to manage multiple systems, plus
multiple jobs efficiently, and is currently using a python package - titled /Many Jobs/ - to help
accomplish this. The PI has been working closely with the developer of these scripts (they are
both at the same university), and does not seem to need any further assistance from AUS staff.
PI: Quinn (Boston University, Solar Terrestrial Research). Center for Integrated Space Weather
Modeling. Continuing through 12/10. The ASTA staff for this project are Lonnie Crosby, Glenn
Brook (NICS), and Carlos Rosales-Fernandez (TACC). There is no further update for this quarter.
PI: Cui (SDSC, Computer and Computation Research). Parallelization of a Finite Element Code
for Petascale Dynamics Modeling. Continuing through 08/10. The ASTA staff for this project is
Jun Zhou (SDSC). The MaFE code supported by ASTA project has been successfully scaled up
to 48,000 cores on both TACC Ranger and NICS Kraken. The MPI/OpenMP Hybrid version has
been also implemented to improve the parallel efficiency. However, the hybrid solver performs
even worse than pure MPI for large-scale benchmark run, where MPI communication and
synchronization overhead dominate the simulation time based on our experiments. Currently we
are redesigning asynchronous model and trying to reduce communication time via overlap
implementation. We will also further investigate optimization algorithm to reduce the main
computation time towards higher peak performance.
PI: Kimber (University of Pittsburgh, Thermal Systems). Computational Modeling of Thermal
Striping in Nuclear Reactors. Continuing through 12/10. Anirban Jana from PSC is the ASTA
staff for this project. As the computational lead, Anirban continued to guide the postdoc on
systematically simulating in Fluent a series of cases with growing complexity, on the local
machine at Kimber's lab. The following cases has been simulated in Fluent this quarter (in order
of growing complexity):
a) Single laminar isothermal round jet, 2d axisymmetric
b) Single laminar non-isothermal round jet, 2d axisymmetric, without and with temperature
dependence of fluid properties.
c) Single laminar isothermal round jet, 3d
d) Single turbulent isothermal round jet, 3d, RANS and LES
Anirban and the postdoc completed rigorous mesh independence and domain independence tests
and validation with known (analytical) results for case (a) and will repeat them for the other cases
next. The ultimate goal is to perform fully 3d, non-isothermal, LES simulations in Fluent of
single and multiple jets in order to study their thermal mixing characteristics.
The project has also been expanded to include the investigation of boiling heat transfers in
nuclear reactors. The plan to use the Lattice Boltzmann Method as the primary tool here, since
they will be interested in mesoscopic effects. They have submitted a request to add to this TG
account Dan Roseum, a PhD student in the Dept of Mechanical Engr, University of Pittsburgh,
who will be working on this.
PI: VanBriesen (CMU, Chemical and Reaction Processes). Simulation of Complex Organic
Mixture Biotransformations in Natural Systems. Continuing through 04/11. Anirban Jana from
PSC is collaborating on this ASTA project. In this project, the aim is to perform Monte Carlo
simulations to investigate the dechlorination likelihoods of a class of chemicals known as
polychlorinated biphenyls or PCBs. The Monte Carlo runs themselves are embarrassingly
parallel, but the outputs of each MC run needs to be stored in n-dimensional arrays so that at the
end, relevant statistics can be computed. Hence clearly, both memory requirements and
computational time increase as the number of MC runs increase.
52

Amanda Hughes, a PhD student of Dr. Van Briesen, had developed a MATLAB code that could,
on her laptop, run 330 MC simulations before running out of memory. Over the last quarter, her
original serial MATLAB code was ported to the MATLAB Star-P platform on Pople and
parallelized. As part of the porting process:
1) The original code structure was thoroughly reviewed.
2) The code was restructured so that the parallelizable parts of the code were consolidated into a
single for loop.
3) During the restructuring, additional vectorization and other code tweaks were performed to
improve the serial performance.
4) Finally the code was parallelized with Star-P by moving the contents of the parallelizable for
loop into a functions and doing a ppeval on the function.
This has now enabled simulations with up to 10000 MC runs. The scaling of computational time
with increasing # cores was found to be very satisfactory too.
Note that because the code was restructured to put everything to be parallelized into a single for
loop, it will be trivial to run this in parallel on *other parallel MATLAB platforms*. For example,
if using the Mathworks Parallel Computing Toolbox, instead of a ppeval, the for loop just needs
to be replaced by the parfor loop.
PI: Brasseur (Penn State, BioPhysics).Villous Motility as a Critical Mechanism for Efficient
Nutrient Absorption in the Small Intestine. Continuing through 03/11. Lonnie Crosby (NICS), for
computational work, and Amit Chourasia (SDSC), for visualization work, are involved in this
ASTA project. In this quarter effort focused on computational work by Lonnie. This group's code,
Intestine3D, initially produced a large amount of metadata traffic on the file system. This large
traffic volume was due to the I/O pattern employed by the application in which files are opened
and closed for each read/write operation. Twelve files were identified which could benefit from
remaining open during the course of the application's runtime. This approach was able to
decrease the application's runtime by 33%.
A full profile of application performance was performed and sent to members of the project. This
profile identified an inherent load imbalance in the application. Addressing this load imbalance
could improve performance by an estimated 10%. Additionally, the implementation of MPI-IO
parallel I/O in order to produce concatenated binary files instead of individual (one per process or
group) ANSCI files can improve application performance by about 10%. Suggestions and
instructions for implementing MPI-IO parallel-IO were sent to members of the project.
PI. Engel (U. North Carolina, Physics). Systematics of Nuclear Surgace Vibrations in Deformed
Nuclei. Continuing through 03/11. ASTA staff for this project are Meng-Shiou Wu (NICS) and
Victor Eijkhout (TACC). During the first quarter of this ASTA project focus was on gathering
detail specifications and understanding the code structure of the project. Meng-Shiou has been
working with the group in order to explore possible approaches to improve the code efficiency on
Kraken. Jointly with the PI’s group they have identified and discussed why their approach to use
ScaLapack was not working, and what are possible choices for them to conduct diagonalization
on a multi-core XT5 node. Both shared memory approach (use threaded libraries) and distributed
memory approach (use sub-communicator and re-design memory management in their code)
were discussed. Several scientific libraries that support multi-core architecture were tested
(Libsci of Cray, AMD's ACML and ATLAS), but very limited or no performance improvement
was observed. Currently work is on integration of their code with a code segment provided by
another research team that use a master-slave style programming to utilize MPI's subcommunicator
for the project.
53

PI. Mac Low (American Museum of Natural History,PlanetaryAstronomy). Simulations of
Molecular Clouds and Protoplanetary Disks. Continuing through 03/11. Roberto Gomez (PSC) is
working the PI’s team for this ASTA project. The users have been doing development work on a
code derived from Gadget-2, with a different hydrodynamics and MHD algorithm, which they
think is now "more settled and starting to do useful stuff". They have not pushed the number of
cores much past 128. Recently (end of June) they made contact requesting ASTA help in
profiling/optimizing their code. Roberto received an account on the SVN server at AMNH and
had just pulled the code to test that he could build and run a model that was of interest to the
user, working at first on PSC’s local hardware (Pople) to get a handle on it. After discussion with
the user, a procedure was worked out to run the initialization part, and write a restart file, and
resume the run from the restart file. The intention is to profile after the restart so as to avoid
timing the initialization, which takes several minutes, and concentrate instead on profiling the
time evolution, which is where most of the run time will be spent. This profiling effort is just
beginning.
PI. Kim (CMU, Biochemistry and Molecular Structure and Function). Structural and Functional
Characterization of Glu-Plasminogen and Lys-Plasminogen. Continuing through 03/11. ASTA
effort is provided by Phil Blood (PSC). Phil is in touch with the PI’s group, but no activities to
report yet.
PI. Kolmakov (Univ. of Pittsburgh, Materials Research).Design of Communicating Colonies of
Biomimetic Microcapsules. Continuing through 03/11. Carlos Rosales-Fernandez (TACC), Jay
Alameda (NCSA), Dongju Choi, Amit Majumdar (SDSC) are providing ASTA support for this
project. Carlos Rosales-Fernandez from TACC worked with user German V. Kolmakov from
Pittsburg University in porting his current Lattice Boltzmann Method to GPUs. Initially a group
of consultants (Jay Alameda, Carlos Rosales-Fernandez, Dongju Choi, Amitava Majumdar)
suggested using accelerator regions defined within the PGI compiler framework to improve
performance, but the limitations of this method prompted the user to try a straight port to CUDA.
An initial version of the code was ported to CUDA by the user, with a performance that was
benchmarked to be around twice as slow as that of the pure CPU code when running on a Tesla
C870 system and to only match the CPU code speed when run on a Quadroplex 2200 S4 card on
the Longhorn system at TACC. Subsequent improvements were made to the code, focusing on
memory access. Improved shared memory and register usage and improved memory access
patterns led to a 16X performance improvement, representing a reduction from 205 ms/iteration
to 13 ms/iteration for the benchmarked example.
Since this optimization of a simplified version of the code has been successful, the AUS
consultant will continue to work with the user in order to complete the port of the production
code. This work will include the parallelization of the production GPU code over multiple GPUs.
PI. Naduvalath (UNLV, Chemistry).Computational Modeling of Ultracold Molecular Collisions
and Reactions Involving Clusters and Nanoparticles. Continuing through 03/11. ASTA effort is
provided by Bilel Hadri (NICS) and Mahin Mahmoodi (PSC). Initial contact has been made with
the PI and the ASTA team is waiting as the PI needs little bit more time to start the interaction.
PI. LaLonde (Bryn Mawr College, Biochemistry and Molecular Structure and Function).
Development of Novel HIV Entry Inhibitors Using the ROCS Shaped Based Matching Algorithm
and Molecular Dynamics Studies of gp120 Envelop Proteins. Continuing through 03/11. ASTA
staff of this project is Phil Blood (PSC) and did quite a bit of work with the PI before the ASTA
was officially initiated. There is no major update for this quarter.
PI. Blaisten-Barojas (George Mason University, Physical Chemistry). Simulation and Data
Analysis of Macro-Molecular Systems. Continuing through 03/11. ASTA staff for this project is
54

Yang Wang (PSC). Initial contact was made with the PI’s team and due to being busy with other
work, the PI will contact ASTA staff back in August for help.
PI. Schafmeister (Temple University, Organic and Macromolecular Chemistry). Applications of
Bis-peptide Nanostructures. Continuing through 04/11. ASTA staff for this project is Bilel Hadri
(NICS). In this quarter help was provided regarding running the boost library with PGI on the
Kraken machine and the PI switched to gnu compiler; both gnu and Pathscale compilers worked
in this case. The PI was provided guidance for running performance tools such as PAPI, Craypat
and IPM and these are expected to help the PI understand the performance as well as memory
usage.
PI. Ozdoganlar (CMU, Manufacturing Processes and Equipment).Finite-Elements Modeling of
Single- and Poly-Crystalline Material Removal Including Crystallographic Effects. Continuing
through 03/11. ASTA staff for this project is Anirban Jana (PSC). Anirban has worked with PI’s
student, Shin Hyung Song, and run parallel ABAQUS simulations of the plastic deformations of
crystals on Pople, using user defined subroutines written in Fortran.
5.5.2 Advanced Support for Projects (ASP)
Under the ASP sub-area, AUS staff work on projects that directly benefit a large number of
TeraGrid users, as opposed to the ASTA projects for which AUS staff work, following the
guidance of the TRAC review process, with an individual user on his/her applications such that
TeraGrid resources can be used effectively by the user. The different categories of ASPs are:
• Installation and maintenance of domain science codes and software as well as HPC and
CI related software and tools on suitable HPC machines located at the RP sites
• Projects, decided in collaboration with users, that have the potential to impact large
number of users
• Creation of a software and associated database to monitor libraries being used by users
• Implementation and testing of software allowing cross-site MPIg based simulations to
run and associated interaction with cloud computing
Among these four categories, the second one involves more effort, from AUS staff, than the other
three.
5.5.2.1 Installation and maintenance of domain science codes and software as well as HPC
and CI related software and tools on all the HPC machines located at the RP sites
For this category of ASP tasks, AUS staff from all the RP sites continued to participate in this
effort. This is considered a foundation type work that is performed in an ongoing basis and is
needed for TeraGrid users who routinely use these codes and software for their science
simulations. Many of these codes and software are also needed and used as a part of various
ASTA projects by AUS staff. Examples of these codes and software include scientific
applications in domains sciences such as chemistry, biochemistry, materials science, engineering,
numerical mathematics, visualization and data, and in HPC/CI. AUS effort continued in this
quarter involving maintenance and proper installation of these codes and software, optimization
of the software for an architecture, debugging the software, interaction with and training for users
regarding usage of these codes and software. This is carried out by AUS staff that have Ph.D. and
M.S. level expertise in specific domain sciences and computer science.
55

5.5.2.2 Projects, decided in collaboration with users, that have the potential to impact large
number of users
Under the second category of ASP efforts, AUS staff work on projects that have the potential to
benefit a large number of users (at least 10 user groups) within a specific domain science or
TeraGrid users in general, utilizing the unique expertise and experience that AUS staff hold. The
projects are identified based on impact it will have on the user community, input from users, input
from AUS staff and other TeraGrid experts. Expertise is available within the AUS staff pool,
across the RP sites, to undertake such projects that have broader impact. AUS staff from multiple
RP sites jointly collaborated on these projects. AUS staff continued to pursue five such projects.
Two of these were pursued by the AUS staff that have domain science as well as HPC expertise
in chemistry/biochemistry and materials science and are users and, in some cases, developers of
molecular dynamics and materials science codes. Large numbers of TeraGrid users use the
molecular dynamics and materials science codes on various TeraGrid machines. The third project,
investigating hybrid programming model, was pursued by AUS staff that has expertise in the area
of HPC programming paradigm. The fourth project involves investigation of PGAS languages
and comparison of performance with MPI based programming models. Initially kernels for linear
system solvers are being explored and later other kernels/codes will be added. AUS staff from
other sites can join once the project progresses. The fifth project involves developing tutorials of
profiling and tracing tools for users.
The first project involved porting, optimizing (using the best compiler options, machine specific
environment variables), benchmarking, and profiling widely used molecular dynamics (MD)
codes on various TG machines. Currently AUS staff from PSC, SDSC, and TACC are
participating in this project. AMBER and NAMD were benchmarked on Ranger and Kraken for
various input cases and utilizing various compiler and environment options. Benchmarks were
also done using all the cores within a node, and partial number of cores within a node to see the
impact on performance and scaling. The results were presented from the AUS benchmark
webpage and feedback was received from some users of these codes. During the quarter AUS
staff worked with the documentation group to do final edit of these benchmark pages. All of the
technical information are available for the MD user community via web-based documentation and
can also be used for TeraGrid training and workshops. We believe that as such detailed MD
benchmark data are made available to the TeraGrid MD users, it will greatly benefit this large
community. In addition to always having the optimally installed MD codes on various TG
machines, this project will enable the MD community to (1) gain knowledge about performance
and scaling characteristics of MD codes on various machines, (2) know about the optimal
compiler and libraries to use on TG machines, (3) install their own versions of MD codes on
TeraGrid machines, and (4) justify resource requests in allocation proposals. This will also help
TRAC reviewers to cross-check users’ requests for resources in their allocation proposals. The
profiling results will be made available to the MD code developers and enable them to tune the
future released versions of the MD codes for TeraGrid machines. Feedback regarding the
usefulness of this project, was sought from about ten prominent long time TeraGrid MD users and
strong support for this project was received from the users.
The second project is similar to the MD project described above and targets codes that are used
more for materials science type simulations, such as VASP, and CPMD. AUS staff, from NCSA,
LONI, PSC and NICS, that have expertise in these codes and in the field of materials science, are
carrying out this second project. AUS staff have worked on benchmarking VASP on various
TeraGrid machines for various input cases and parameter settings. AUS staff worked with the
TeraGrid documentation experts to do final edits of these results. Also benchmark effort for the
newer version of VASP is being pursued. The results are available for the materials science user
56

community. User feedback received, from materials science users, for this project also showed
strong support for this project.
The third project involves evaluation and benchmarking of the hybrid (MPI-OpenMP/pthreads)
programming model. Due to the emergence of multi-core processors, hybrid programming, using
MPI tasks and multiple OpenMP threads or pthreads per tasks, is of interest to users. This model
may provide better performance than straight MPI programming. In addition hybrid programming
may allow some simulations that are otherwise not possible due to limited memory available per
core and hence provide a science capability not achievable by straight MPI codes on multi-core
architectures. In order to investigate these issues, AUS staff from PSC, NICS, TACC and SDSC
are comparing performance and usage of codes written in hybrid mode versus straight MPI mode.
The codes being targeted are an astrophysics code, a hybrid benchmarking code, phylogenetics
codes, FFTW, and a Monte Carlo code and work is ongoing. AUS staff are planning to write up
the results comparing straight MPI versus MPI-OpenMP/pthreads performance for TeraGrid user
community. Raghu Reddy (PSC) continued to work with University of Colorado researchers on
measuring the hybrid performance of their code. One of them members of the group will be
presenting the results at TG10. This work also resulted in the following
submission:http://arxiv.org/abs/1003.4322 .
As mentioned earlier the fourth project involves investigating PGAS languages. Progress to-date
includes the implementation of a 2D finite difference kernel in UPC and MPI. Testing of said
kernel demonstrated expected MPI performance and pretty poor UPC performance. Interactions
have began with the Berkeley UPC group, and the TACC project team is still waiting on a
response from them on a few items before they move on to implementation of other kernels. A
Chapel implementation of the same kernel is underway. Benchmarks were also run with MUPC
with similar results.
The fifth project involved developing TAU tutorial and this is done by AUS staff from PSC. This
tutorial and documentation were linked from the AUS projects page (i.e. the same page from
where the other projects of benchmark results and hybrid programming results are published).
The TAU tutorial will give users information about TAU installation on Ranger and Kraken and
other TeraGrid resources, and will be a practical guide for TAU. It will allow users to learn about
the commonly used features of the TAU software tool and will enable users to successfully start
using TAU on TeraGrid resources. Fig 5.1 shows one the images from the tutorial.
Figure 5.1 TAU Startup Screen as Shown in the TAU Tutorial.
57

5.5.2.3 Creation of a software and associated database to monitor libraries being used by
users
For the third category of ASP, AUS staff at NICS are implementing and testing “ld” and “aprun"
as a part of the libraries database project on Kraken. This database will provide user support staff
knowledge about library usage such as which ones are used the most and how frequently. The
information obtained will be used to improve the software management, procurement and
maintenance requirements. Also, this information may prove useful in the planning and
development stages of new resources as they are brought to production. After testing on Kraken,
this will be made available to other RP sites that may be interested in using this. During this
quarter a beta release of the software was made. This beta software was shared with the new
NOAA center at ORNL.
5.5.2.4 Implementation and testing of software allowing cross-site MPIg based simulations to
run and associated interaction with cloud computing
Under the fourth category of ASP, the project being done involves both ASTA and ASP and it
has been described in §5.5.1 (PI Roberts (RTI International, Computer Science) EpiSims; PI:
Karniadakis (Brown U., CFD) NekTar-G2; PI Coveney (University College of London,
Chemistry) GENIUS)
5.5.3 Advanced Support for EOT (ASEOT)
AUS staff from all the RP sites contribute to preparing and delivering advanced HPC and CI
technical contents for trainings and workshops that are offered throughout the year at all the RP
sites. This is done in coordination with the EOT area. List of workshops, which were offered are
listed in the EOT section of this report.
Under the ASEOT effort, AUS staff continued to participate in the Extreme Scalability (XS)
working group, which specifically focuses on architecture, performance, tools and software
related to petascale-level simulation. AUS staff are participating in projects identified by the XS
working group, and in many cases these projects allow interaction with researchers/users funded
by NSF PetaApps, SDCI, and STCI grants. Interactions within the extreme scalability working
group also help AUS staff to identify ASPs that may impact large number of users. The hybrid
MPI-OpenMP programming model project and the TAU tools project, as stated in §5.5.2.2, are
joint projects between the AUS area and the XS working group.
AUS experts, from all the RP sites, regularly present technical talks at the bi-weekly AUS
technical tele/web-conferences. These technical talks, based on ASTA work or other AUS work,
are recorded and the slides and the audio recordings are available from:
http://teragridforum.org/mediawiki/index.phptitle=Technical_presentations. The EOT area
director and other members of the EOT area are going over these talks and plan to utilize these
for development of future HPC/CI technical training materials.
AUS staff are engaged in organizing and planning of the TG10 conference to be held in
Pittsburgh in August, 2010. Multiple AUS staff served as reviewers of abstracts/papers and in
selecting and inviting invited speakers for science track. Multiple AUS staff are regular members
of the TRAC review committee and in addition many AUS staff review TRAC proposals that
request smaller amount of SUs and hence do not require review by regular members of the TRAC
review committee.
58

6 User Facing Projects and Core Services
The User-Facing Projects and Core Services (UFC) area continued its work of providing the
central access and information dissemination points for TeraGrid users and to present a single
interface to the collective efforts of the TeraGrid RPs. The GIG supports and coordinates effort in
this area from staff at IU, NCSA, PSC, SDSC, TACC, and UC/ANL and coordinates the
participation by and integration of all RPs into the UFC environment and processes.
6.1 Highlights
662 new users were added to TeraGrid making this the second busiest quarter ever. The June
TRAC meeting reviewed requests for more than 700 million SUs and the TeraGrid team, for the
first time, implemented an automated process to roll back recommended awards to fit within the
available SU capacity.
The ongoing work to permit unvetted users to create their TeraGrid User Portal (TGUP) accounts
led to a completed and approved implementation document; work will proceed in Q3. The TGUP
now allows users to view their submitted tickets and see activity on those tickets, and the POPS
system now has a significantly streamlined resource request page. The accounting system
(TGCDB and AMIE) added two significant features. First, the system allows sites to include user
uid and project gid information, in support of the Lustre wide-area file system effort. Second, the
TGCDB now permits sites to send usage information associated with allocations on storage
systems. The team has also begun collaborating with the XD-TAS team, both to identify the best
way to transfer data from TGCDB into the XD-TAS system and to ensure metrics portlet effort
by the Documentation team does not duplicate features of the XD-TAS system.
6.2 Enhanced TeraGrid User Access
This objective supports the creation and distribution of new user logins across TeraGrid RPs for
the various resources, as well as enhancements to this process to integrate with campus
authentication infrastructures. Within the TeraGrid User Portal, users are also provided ways to
maintain their user profiles, customize their interaction with TeraGrid, and access TeraGrid
resources and services remotely.
The TGUP team completed the implementation plan for creating “unvetted” new user accounts
and received sign-off from almost all working groups; following a discussion with the securitywg
lead in July, implementation activities are getting underway in anticipation of final sign-off
from that working group early in Q3 and targeting completion by the end of the calendar year.
Also in Q2, the NOS team provided the TGUP team with a prototype implementation and
documentation of Shibboleth integration with Liferay, and work is progressing on this integration
effort.
A TGUP (and TG Mobile) portlet to list a user’s help tickets was deployed in production. The
ticket information is being provided via a REST service to the ticketing system database. A
revised add/remove user form was released in June that supports multiple add/remove requests
with one submission.
UFC staff have also prototyped a TGUP portlet for accessing the MCP metascheduling capability
on TeraGrid. The developer is continuing to extract info from TG IIS and other sources, to
populate MCP submission info. The MCP portlet should be ready for testing in early Q3.
The TGUP team also prepared to release a new file-sharing feature within the portal and TG
Mobile site. The new features offer comprehensive file management and collaborative file
sharing in the My TeraGrid section of the Mobile TGUP via the Files category and collaborative
file sharing within the TGUP File Manager. The new collaborative file sharing features are
59

enabled by the TeraGrid Share service. TeraGrid Share provides users with 2GB of secure,
personal space for collaborating with other TeraGrid users and the world. The new mobile file
management feature allows users to manage files across the TeraGrid while away from their
desktop. Users can browse through data, download files, create folders, rename and delete files
and folders, copy files between systems, and publish files to their TeraGrid Share accounts.
6.2.1 Operational Activities
TeraGrid continued to exceed new thresholds for many user-related metrics. Notably, in May,
TeraGrid surpassed 1,400 individuals charging jobs in a single month. TeraGrid also surpassed
6,000 active users at the quarter’s end. Finally, 662 new individuals joined the ranks of TeraGrid
users, the second highest quarterly total to date. This elevated workload is made possible by prior
efforts to streamline the new-user process and reduce the central staff effort.
TeraGrid users numbered 6,482 at the end of the quarter, including 5,414 individuals with current
TeraGrid accounts and 1,068 reported gateway users. Of the 5,414 individuals, 2,037 (38%) were
associated with jobs recorded in the TeraGrid central accounting system. Figure 6-1 shows the
growth of the TeraGrid community since 2004. (Active Users = Users Charging Jobs + Gateway
Users; TeraGrid Users = Current Accounts + Gateway Users.)
The TGUP’s access capabilities continue to be among the most highly used features, with users
accessing the file manager portlet 18,061 times this quarter, and 1,221 users visiting the GSI-SSH
portlet 23,685 times. The new ticket system portlet made the list of most-visited portlets, with 350
visits, even though it only entered production in June. The automated password reset function was
used 292 times. Table 6-1 shows the most highly visited TGUP applications for this quarter.
Figure 6-1. TeraGrid User Trends, Q2 2010
60

Table 6-1. Q2 2010 Most Visited TGUP Applications
Application # Users Q2 2010 Visits
Knowledge Base 790 45,834
Allocation Usage 2,037 33,197
GSI-SSH 1,221 23,685
System Accounts 1,477 20,239
Data Collections 91 17,873
File Manager 89 18,061
System Monitor 1,267 12,925
Science Gateways 101 5,072
Consulting 675 4,571
Add/Remove User Form 619 3,798
User Profile 787 2,115
Queue Prediction 124 1,327
Wait Time Prediction 124 1,327
DN Listing 513 915
Feedback 33 719
Community Accounts 430 694
*Ticketing System 99 350
6.3 Allocations Process and Allocation Management
This objective encompasses the allocations process, both for Startup/Education allocations as well
as the merit-review TRAC Research request process, the POPS system for request handling and
management, mechanisms by which allocation PIs manage allocations through transfers,
extensions and so on, and interfaces by which allocation PIs manage the users who are authorized
to use their allocations. Operationally, this objective includes the TRAC review process, the
Startup allocations review and decision process, and the maintenance and operations of the POPS
system.
The most significant effort was completing the definition of the formulas and process for
“Recommend and Available Allocation Reconciliation.” This work was initiated in recognition of
the fact that oversubscription—that is, TRAC recommendations exceeding available RP SUs—
was fast becoming a pressing issue within TeraGrid. The previous methods were too informal to
fairly and transparently deal with a situation in which awards have to be reduced by potentially
hundreds of millions of SUs; in addition, NSF directives required TeraGrid to factor funding
sources into the equation. The resulting calculations, performed within Excel, optimize awards
based on a complex formula incorporating funding source information (with NSF awardees
getting cut less than non-NSF awardees) and size of award (with smaller awards cut less than the
largest awards).
The POPS team rolled out a new version of POPS with a streamlined Resource Request page.
Rather than requiring users to answer low-level questions associated with each resource requested
(data that were never reviewed or used in the allocations process), users now just enter request
amounts, optional comments, and answer three general questions about their planned allocation
use. The three questions were defined by the TeraGrid Director of Science and added in support
of the usage modalities effort. Work began on efforts to streamline the entry screens for POPS to
reduce the opportunities for confusion and error by submitters.
The team also supported a new process by which Advanced User Support allocations will now be
associated with a project in TGCDB. A process that uses existing POPS functionality (transfer
61

equests) was defined to allow the AUS Area Director to have ASTA “allocations” associated
with the respective projects in TGCDB. These account for a number of the Transfer requests
made this quarter.
6.3.1 Operational Activities
The allocations staff also managed the June 2010 TRAC meeting, in Arlington, VA, at which 102
requests for TeraGrid resources were reviewed. Of more than 730 million core-hours requested,
more than 328 million core-hours of compute time and more than 2.4 PB of data storage were
awarded. Due to the high request levels, the TRAC recommended awards that in aggregate
exceeded the available computing time available. Thus, the allocations team applied, for the first
time, the process to reduce recommended awards to fit within the available SUs.
Preparations to support a number of new resources were completed, including the TeraGrid Data
Replication Service, Advanced User Support Service, TeraGrid Wide-Area File System, Nautilus,
and Wispy.
The allocation staff continues to receive in excess of two new/renewal startup requests each day.
TeraGrid received 223 Startup requests (193 approved) and 8 Education requests (8 approved).
Table 6-2 shows the overall allocations management activity handled by POPS and the
allocations staff. Note that for Transfers, the table shows only the positive side of the transaction
to show the total transfer level; there is a corresponding negative amount, adjusted for resource
exchange rates.
Table 6-2. POPS Requests and Awards, Q2 2010
Research Startup Education
# Req SU Requested # Awd SUs Awarded # Req SU Requested # Awd SUs Awarded # Req SU Requested # Awd SUs Awarded
New 49 176,959,526 45 57,229,483 197 28,266,856 171 18,901,423 6 240,165 6 242,006
Prog. Rept 3 2,978,000 3 1,862,005 n/a n/a
Renewal 52 554,374,301 50 247,248,274 26 2,970,341 22 4,815,025 2 307,512 2 307,513
Advance 25 41,777,755 24 23,213,752 n/a n/a
Justification 7 42,643,530 6 11,647,120 0 0 0 0 0 0 0 0
Supplement 32 72,568,631 23 22,512,633 41 4,934,738 33 3,245,508 4 270,012 4 270,012
Transfer 85 23,533,630 84 23,852,996 51 2,249,371 48 1,208,215 1 20,000 1 20,000
Extension 40 n/a 37 n/a 52 n/a 48 n/a 0 n/a 0 n/a
6.4 RP Integration and Information Sharing
This objective ensures that RPs are able to provide information about their resources and services
and that decisions made centrally (such as allocation awards) are communicated to the RP sites.
This objective includes the accounting infrastructure of TeraGrid Central Database (TGCDB) and
AMIE. For operations, the TeraGrid accounting infrastructure requires ongoing operational
support and updates to the TGCDB and AMIE systems to support evolving policies and
requirements for project, user and usage creation and exchange.
Two significant enhancements to the AMIE packet transport software and TGCDB accounting
procedures were implemented and deployed during the quarter.
• Both AMIE and TGCDB were updated to support the optional inclusion of a site’s uid
and gid for a user and project. This feature was added in support of the Lustre global file
system. The uid and gid fields for existing users were back-populated in the TGCDB.
• Procedures to implement storage accounting were implemented and deployed in May
along with AMIE changes to extend the NPU packet definition to support storage usage.
Rather than an “additive” accounting model (as used for compute accounting, in which
the cost of new jobs is added to the previous job charges), the storage accounting uses a
62

“replacement” model, in which the charges sent by the site replace the prior amounts for
a given user. NCSA has implemented and tested storage NPUs and soon intends to start
sending them in earnest.
The documentation team continued work on Liferay portlets to display current and past
allocations information. The usage map from the TeraGrid usage web site (TGU) has been ported
in both Google Map and Google Earth form; the team is working on identifying a production
home for these portlets within the TeraGrid web site.
The TGCDB team is also working with XD-TAS to identify the best way to get TGCDB data into
their system. The XD-TAS effort at the University of Buffalo needs to query the TGCDB in order
to perform its auditing function. In case the XD-TAS database queries would interfere with
normal day-to-day TGCDB operation, Rob Light of PSC has been asked to develop a means for
the TAS team to have a current copy of TGCDB. The model used by the TGCDB failover system
may not be appropriate because both the primary and failover systems need to be identical.
Light’s plan is to send a copy of the up-to-date backup database that resides on PSC’s TGCDB
failover system to Buffalo nightly, which the TAS team can re-store into a running server. This
concept will be discussed further with the TAS team and developed during the next quarter. The
documentation team will also coordinate with XD-TAS on future metrics-related portlets to avoid
duplication of effort.
The SDSC team identified student effort available to proceed with the migration of the Resource
Catalog to integrate with the Resource Description Repository (RDR) developed at PSC. Work
will proceed during the summer.
6.4.1 Operational Activities
Operationally, the Accounting Working Group prepared for a coordinated upgrade to a new
AMIE version that supports packet exchange via .tar files. The new version required all sites to be
running the latest AMIE version and a coordinated final step to deploy the .tar version. In June,
all sites deployed the latest pre-tar version of AMIE. The final upgrade step was scheduled for
and completed on July 14.
Preparations to support a number of new resources were completed, including the TeraGrid Data
Replication Service, Advanced User Support Service, TeraGrid Wide-Area File System, Nautilus,
and Wispy. Quarterly resource utilization metrics appear in Section 8 of this report.
No TGCDB outages occurred in the quarter, and the source of a recurring Inca “false alarm” was
identified and corrected. A query from the TeraGrid portal was causing excessive slow-down,
which triggered the Inca alarm. The portal query has been rewritten and thus far the problem has
not resurfaced. The failover system continues to sync with the production TGCDB server. The
regular quarterly test of TGCDB restoration procedures was completed successfully in less than
two hours.
The staff at SDSC and NCSA are also preparing for a postgreSQL software upgrade for TGCDB.
The current TGCDB software version (postgreSQL 8.1) has been declared end of life as of
November 2010. A test server and install of postgreSQL 8.4 has been deployed at SDSC to
evaluate and prepare for a postgreSQL upgrade.
6.5 RP Operations: Accounting/Core Services
TG RP staff provided local allocations and account management support for users including
account creation/deletion, password reset, and local system usage accounting. These staff
interacted with the TG Account Management transaction system (AMIE), which is a transactionbased
grid account management system that is central to all users having access to the TeraGrid.
63

Four sites are now supporting gateway user tracking through the GRAM Audit Database
integration: NICS, LONI, NCAR, and NCSA. More than 16,000 jobs have been tagged with
GRAM and gateway user data. TACC accounting staff contributed to the effort to facilitate the
tracking of TeraGrid resource usage through science gateways. This effort will result in the
ability to track individual user activities, resulting in increased security and user accountability.
6.5.1 SDSC
SDSC RP staff integrated the new Dash system with the TeraGrid allocations, account creation,
and usage accounting systems. Dash was made available for Startup requests starting in April
2010, and in Q2, the first Startup allocations were created, and jobs from several projects have
been reported to TGCDB. The Dash system was included on the list of available resources that
can be requested for Research (TRAC) allocations starting October 2010.
6.5.2 TACC
TACC accounting staff contributed to the effort to facilitate the tracking of TeraGrid resource
usage through science gateways. This effort will result in the ability to track individual user
activities, resulting in increased security and user accountability. Additional TACC staff
members continued to contribute to activities related to User Facing Projects and Core Services.
Maytal Dahan led TGUP portal development activities including the production release of a
ticketing interface for TGUP and TGUP Mobile. This enables users to manage their tickets online
via an easy-to-use portal interface and view historical tickets. Furthermore, the user portal team
enhanced the add/remove user capability to enable easier user allocation management. Katie
Cohen, Kent Milfeld, and Margaret Murray coordinated the TRAC review process and organized
the allocation review meeting held within the reporting period. In addition, the TACC team
organized and chaired the weekly allocations working group meetings, and processed all advance,
startup (276) and supplement allocation requests during the reporting period.
6.6 User Information Presentation
This objective ensures that users are provided with current, accurate information from across the
TeraGrid in a dynamic environment of resources, software and services. Operationally, this
objective ensures high reliability of the presentation mechanisms, including quarterly operational
support for the TGUP and TeraGrid web site. Support for TeraGrid internal web environments,
notably the Wiki, is included here. The UFC team spent a week working with Liferay consulting
staff to identify, troubleshoot, and resolve ongoing issues. The effort has significantly reduced the
frequency and nature of the problems that have been encountered with Liferay thus far.
During Q2, the User News application was migrated successfully from SDSC’s Oracle database
infrastructure to the TGCDB and postgreSQL. This effort eliminates the application’s dependence
on two separate databases and will allow the User News service to be transitioned as needed to
the XD program in 2011.
6.6.1 Operational Activities
The TGUP and TeraGrid Web presence continued to increase its user base and support large
numbers of visitors and page hits. Table 6-3 shows hits, visitors and logins for the TGUP, Web
site and Wiki. Reported Liferay metrics for May and June were incomplete because the awstats
script stopped running during part of those months; the issue has been resolved. The Liferay
environment experienced one significant unscheduled outage on June 8, when a 30-minute
scheduled downtime to permit a hardware move at TACC turned into a six-hour outage due to a
switch failure. Liferay preventive maintenance and update periods are being scheduled for offhours
until the fully redundant two-server system is deployed at TACC.
64

The TGUP responded to 106,097 portlet requests from registered users. The Knowledgebase
(accessible via both the TGUP and Web site) delivered 39,668 documents in the quarter. The
Liferay environment now permits tracking usage of various web site portlets, and Table 6-4
shows the usage and hits for the Resource, Software, Data Resource, and Gateway catalogs.
Table 6-3. TeraGrid Web traffic, Q2 2010.
Web Portal Mobile Wiki KB
Unique Visitors Hits Unique Visitors Unique Logins Hits Logins Hits Unique Visitors Pages Edits Docs Hits
April 8,785 527,675 3,987 1,250 444,103 47 3,078 2,404
738 20,124
May n/a n/a n/a 1,122 n/a 38 1,449 2,459 7,737 45,629 744 15,494
June n/a n/a n/a 1,177 n/a 56 1,114 2,526 749 20,857
Table 6-4. User information catalog visits
Catalog
Total Visits
Software Search 2,572
Resource Catalog 2,528
Gateway List 989
ASTA List 524
Data Resources 308
CTSS by Resource 121
6.7 RP Operations: User Facing Projects
TG RP staff maintained and updated local documentation for their resources. Some of this
activity resulted from user feedback and interaction with the TeraGrid Operations Center, RP help
desk support personnel, and members of the advanced user support team. Hardware and software
upgrades on HPC, visualization, and storage resources also resulted in local documentation
activities.
6.7.1 Indiana University
Summary statistics for the TeraGrid Knowledge
• Total number of TG KB documents available in TGKB end of 2 nd quarter 749
o The total number of TG KB documents including those archived is 788
• New documents created during 2 nd quarter 22
• Documents modified during 2 nd quarter 169
• Total documents retrieved 182,348
• Real hits 56,475 (documents retrieve minus bots, crawler, scrappers, link checkers). This
number is IU server hits only because TeraGrid server data is unavailable at this time.
On average for 2009, the TG server added an additional 16%, so a conservative estimate
of real hits for the 2 nd quarter would be around 65,500
65

Figure 6-2. TeraGrid KB Number of Documents, Q2 2010
Figure 6-3. TeraGrid KB Document Editing, Q2 2010
66

Most new documentation came as a direct result of conversations within the User Services group.
We are also monitoring the new Forums in Liferay which spawned a document. As a general
practice, the growth of the TG KB is now focused of support driven documentation and less on
quota base contributions.
Figure 6-4. TeraGrid KB Hits, Q2 2010
As noted above, only partial data was available from the TeraGrid server so these numbers do not
include documents opened from the TeraGrid interface. Also note that these counts represent
number of documents opened excluding, as best we can, hits from bots, crawlers, etc.
Content in the TeraGrid Knowledge Base repository can be accessed in a number of ways. The
major sources for legitimate hits (documents opened excluding bots, scrappers, etc.) are the
search interfaces within the TeraGrid web space (missing from this analysis), the IU KB search
interface, links within TGKB documents, and Google searches. A large number of records in the
weblogs do not provide information in the “referrer” field, which prevents determination of the
originating page. Of those hits for which the weblogs provide referrer information, Google
searches are the predominant source.
67

Figure 6-5. TeraGrid KB Source of hits, Q2 2010
Looking at the most frequently accessed documents across the second quarter compared to the
first quarter, there was less overlap. In the 1 st quarter 16 documents appeared among the top 20
documents in all 3 months. In the 2 nd quarter there were only 8 in common across the top 25.
April and May were more alike in that almost 75% of the documents most frequently access were
the same. April had 7 unique documents among the most frequently accessed. May had only 4.
June was the most unique among the three months having 15 documents in the top 25 that did not
appear in the list for April and May.
68

6.8 Information Production and Quality Assurance
This objective includes efforts to provide documentation for TeraGrid activities, provide a
knowledgebase of answers to user questions, and maintain and update general web site content,
all within the TeraGrid Web environment. This activity requires close interaction with RPs and
other GIG areas as subject-matter experts and sources of information. As part of UFC operations,
this objective focuses on updates and Quality Assurance (QA). These ongoing processes and
efforts ensure that existing user information remains correct, is coordinated across both the GIG
and RP subject-matter experts, is in sync with evolution of TG environment (via collaboration
with the CUE working group), and contains minimal errors.
After largely (but not entirely) solving editing and workflow issues with the Liferay environment,
the UFC team began adding more content, particularly new AUS-related pages, updates to the
TG10 web site. A revision to the TeraGrid home page that takes advantage of Liferay capabilities
remains in progress. The Knowledgebase team created 22 new documents and modified another
169. The knowledgebase now holds 749 TeraGrid articles.
6.8.1 Operational Activities
The documentation team continued to work to maintain the correctness of internal and external
links through out the TeraGrid web site, with a goal of keeping documentation link errors to 3%
or less. Issues with the Web Link Validator returning many false broken links in Liferay were
resolved, and the team has once again begun using WLV for tracking link issues. The number of
broken links has crept above 3%, in part due to Liferay issues but also significant increases in the
amount of content with the addition of the AUS performance pages, as reflected in the slow
growth in the number of links on the site. The team continues to bring down the percentage of
broken links.
7 Data and Visualization
7.1 Highlights
Table 6-5. Web Link Validator report for Q2 2010.
Date All links Good links Broken % Broken
14-Apr-10 1,153 1,064 65 5.6%
12-May-10 1,187 1,114 47 4.0%
19-May-10 1,200 1,128 49 4.1%
25-May-10 1,180 1,128 29 2.5%
4-Jun-10 1,234 1,170 41 3.3%
7-Jun-10 1,242 1,180 39 3.1%
14-Jun-10 1,309 1,246 40 3.1%
21-Jun-10 1,266 1,202 41 3.2%
28-Jun-10 1,279 1,215 41 3.2%
Highlights for this quarter include new efforts to support research on the Deepwater Horizon
spill, extension of Lustre-WAN capabilities in production, and continued efforts to integrate with
other large data infrastructures.
Initial hardware setup was completed and configuration and testing began for the new distributed
Lustre-WAN. The production file system will have a petabyte of total capacity, and will be
deployed in production on multiple TeraGrid resources by the end of the program year. Metadata
services for the file system were brought up at PSC, and data servers are under construction at
Indiana, NCSA, NICS, SDSC, and TACC.
69

Deepwater Horizon oil spill related research activities at the Center for Space Research (CSR) at
The University of Texas at Austin have resulted in a significant increase in the size of the TG data
collection managed by CSR. The CSR MAGIC group now manages approximately 23.4
terabytes of data on Corral using the data management system. All CSR MAGIC data is
accessible over the web, with over 15,000 web requests for data from the collection during the
reporting period (from over 2500 separate systems.)
Researcher Scott Michael was offered compute cycles at Mississippi State University to begin his
analysis of the data generated on Pople at PSC and Cobalt at NCSA. To avoid him having to
move 30 TB of simulation data to MSU, IU's Data Capacitor Wide Area File system was used to
bridge campuses. The file system was mounted on compute nodes at MSU. In doing this,
Michael was able to begin his analyses right away without the time consuming step of bundling
his data and moving it from the TeraGrid to a non-TeraGrid institution. Michael has written a
paper about this experience to be presented at the TeraGrid '10 conference.
SDSC continued work on preparing Lustre-WAN for production. DC-WAN (IU Lustre) and
GPFS-WAN (SDSC GPFS) have been mounted on our Dash cluster.
In Q1 Purdue RP migrated existing its TG data collections from the old SRB system to the new
iRODS system. In Q2 Purdue RP modified the Purdue environmental data portal to connect with
the new data management system. The new site is currently being tested.
The ORNL RP continued the active interaction with the REDDnet collaboration and its new
initiative, the Data Logistics Toolkit (DLT). REDDnet was recognized at the Spring Internet2
meeting (http://events.internet2.edu/2010/spring-mm/) with an IDEA award
(http://www.internet2.edu/idea/2010/) for providing distributed working storage in assistance of
projects with large data movement needs.
7.2 Data
7.2.1 Data Movement
Data movement infrastructure based on GridFTP remains stable within the TeraGrid, with
monitoring performed by Inca and the Speedpage. Interactions with the REDDnet collaborators
are expected to result in development of additional mechanisms to facilitate data movement into
and out of TeraGrid resources.
7.2.2 Wide-Area Filesystems
Work continued to deploy the new distributed Lustre-WAN file system using storage at 6 sites.
Indiana, NCSA, NICS, PSC, and TACC all received new hardware to deploy storage servers, and
SDSC began work on repurposing over 200TB of existing storage for use in the new Lustre-
WAN file system. During the quarter the new hardware was deployed and mostly integrated into
the network and power infrastructures of the various RPs, and efforts to prepare for software and
network testing were underway.
In addition to adding MSU as a non-TeraGrid site mounting DC-WAN, IU began work on the
TeraGrid wide area filesystem. IU has written and worked with system administrators at PSC the
UID mapping code that will permit a unified UID space for the TeraGrid. By the end of Q2,
hardware had been delivered, racked, and the disk array had been zoned for production.
PSC continues to operate Lustre 2.0-based test infrastructure to investigate future mechanisms to
enhance wide-area file system security and compatibility. Developments in the quarter include
70

initial testing of NFS4 as a gateway mechanism to a Lustre 2.0 file system and examinations of
the use of Kerberos for authentication in this scenario.
7.2.3 Data Collections
Data Collections support was enhanced for allocated users by the addition of the Data Replication
Service, including metadata management capabilities, and initial discussions were held with user
representatives, as well as both of the present DataNet awardees, to examine ways in which
TeraGrid infrastructure can support additional Data Collections activities.
7.2.4 Archival Storage
The Data Replication Service now provides a capability to move and replicate data across 5 of the
TeraGrid RP archive systems, plus additional disk systems. It is hoped that this capability will
grow to include more archive storage and to play a larger role in user management of archived
data.
7.2.5 Data Architecture
Data Architecture implementation continues with the deployment of the Data Replication Service
and the distributed Lustre-WAN system, both of which will offer persistent storage resources in
support of Data Architecture goals.
7.3 RP Operations: Data
7.3.1 Distributed Data Management
TG RP staff maintained, administered, and operated data resources including archival systems,
GPFS-WAN, Lustre-WAN, database resources, explicit disk/tape allocations, and provision of
scientific databases.
SDSC continued work on preparing Lustre-WAN for production, including the purchase orders of
new servers and storage reassignment.
The ORNL RP continued its participation in the REDDnet collaborative as well as hosting a node
of the Earth Systems Grid (ESG) at the ORNL NSTG TeraGrid enclave.
PSC made significant progress on the distributed Lustre-WAN deployment. PSC set up the
metadata server environment for Lustre-WAN 1.8 deployment. This step allows other sites to
participate in the Lustre-WAN by configuring their servers to use the MDS. PSC has worked
closely with IU to incorporate their user ID mapping code into the deployment. PSC recruited a
“friendly user” whose use uncovered a bug in the IU code, which the IU staff promptly fixed.
To support the user ID mapping, it was necessary to make changes to the TGCDB and have a
new AIME release propagated to all partner sites. PSC led a TeraGrid-wide effort to identify the
changes and get them incorporated into TGCDB; this was a great example of TeraGrid teamwork
that accomplished a lot in a short amount of time.
PSC has established a framework for network testing between the LustreWAN components at the
partner sites. Preliminary tests have been run between TACC and PSC. The results of the testing
are being posted to the Lustre WAN Deployment TeraGrid wiki page. See
http://teragridforum.org/mediawiki/index.phptitle=Distributed_Lustre-
WAN_Deployment#Network_performance_tuning.
71

Separately, PSC is continuing testing of the Kerberos-enabled security release of Lustre 2.0.
Cross realm testing is underway. PSC routinely participates in the Lustre WAN deployment
conference calls to discuss all the Lustre-WAN activities with TeraGrid partners.
The Purdue RP staff has completed the migration of data services which enable programmatic
access to the iRODS data collections from several client systems including the Purdue
Environmental Data Portal and the IndianaView Portal.
Purdue RP staff integrated Purdue storage in the TeraGrid distributed Data Replication Service
and continues to provide direct disk storage to the Data Replication Service. The RP is
investigating the addition of the Hadoop distributed filesystem access to the replication service.
Purdue login nodes continue to mount the Data Capacitor via Lustre-WAN, and continue making
it available to TG users. The RP staff is continuing to investigate vendor tools to also mount DC-
WAN on the workers nodes on Steele as those nodes are on a private IP network.
Purdue RP staff continues to evaluate Cycle Computing's CloudFS, a version of the Apache
Hadoop distributed filesystem. The current testbed is approximately 30TB, and Purdue staff is
working with users from the CMS (Compact Muon Solenoid) project and a social sciences
application to evaluate the use of CloudFS/Hadoop to support high-throughput computing.
Performance testing was conducted which showed that this deployment over the Purdue radon
cluster had negligible impact on PBS jobs running in the cluster, in comparison to other software
such as dCache. In addition, Purdue RP has deployed a small MapReduce cluster with 16 nodes
and is working with instructors of a class at Purdue interested in using it.
7.3.2 Data Collections
Data collection services at the RP sites included the maintenance, administration, and operation
of resources hosting data collections.
Digital Preservation and Chronopolis at NCAR: SDSC, NCAR, and the University of Maryland
(UMD) continued to advance the Chronopolis Consortium with the goal of pioneering new
capabilities in distributed digital data preservation using TeraGrid cyberinfrastructure. Testing of
the iRODS/iCAT system for data transmission has begun, and it is expected to replace the current
SRB/MCAT system. Chronopolis holdings are currently approximately 22 TB in 10 million files.
In Q2 Purdue RP incorporated a new group of Landsat remote sensing data into the TeraGrid
LARS data collection.
7.4 Visualization
The University of Chicago/Argonne and TACC RP teams co-lead the TeraGrid Visualization
working group, which meets bi-weekly. This group leverages the efforts and expertise of the RPs
to provide users with fundamental support in the area of visualization. Work over the last quarter
in the visualization working group has been focused on finishing the petascale visualization
whitepaper. This paper was distributed to the Science Advisory Board for feedback.
NCAR develops and distributes the visual data analysis package, VAPOR, which is funded in
part through GIG PY5&6 funds. Progress was made on both of the major milestones of the
PY5&6 award: 1) Research code supporting wavelet coefficient prioritization-based progressive
data access was integrated into the VAPOR code base. 2) Evaluation of parallel netCDF APIs
72

(pnetCDF and netCDF4) was completed in preparation for providing a parallel (MPI-callable)
interface to the VAPOR data model.
7.5 RP Operations: Visualization
Visualization services at the TG RPs included the maintenance, administration, and operation of
visualization resources, including visualization-specific systems.
SDSC continued advanced visualization support to SCEC on M8 simulations and continued
research and development work on visualizing vector fields.
NCAR operated the DAV cluster, Twister. NCAR also supported the visual data analysis
package, VAPOR.
The Purdue RP made good progress in developing 3D visualization services for super resolution
NEXRAD Level II radar data for research and education users. The implementation for a new
hierarchical data structure and GPU CUDA ray-guided Level-of-Detail rendering has been
completed. The code for data processing and rendering has been optimized to meet real-time
performance requirements for rendering large radar volumes over the entire United States at
multiple resolutions interactively. A prototype end-to-end data processing and visualization
pipeline has been implemented which includes (1) data retrieval from iRODS, (2) data
preprocessing to build hierarchical data structure, (3) CUDA rendering on a Tesla server, (4) data
post-processing to generate top image tiles for the entire US and rotation/tilt views for several hot
spots at different resolutions, and (5) a Google gadget flash viewer for real time interactive radar
data visualization. The gadget viewer will be released to the general public in the fall of 2010.
The UChicago/Argonne RP continued its collaboration with PI Michael Norman on his Projects
in Astrophysical and Cosmological Structure Formation. The RP team produced high-resolution
images and animations of large-scale (4096 3 , and larger) Enzo simulation data using vl3, a
hardware-accelerated volume rendering application. During the past quarter the RP and the Enzo
teams produced animations that were submitted, and accepted, to both the TeraGrid 2010
conference’s Visualization Showcase and to the SciDAC 2010 conference’s Electronic
Visualization Night. The UChicago/Argonne RP continued to work with the Parallel Simulations
of Blood Flow in the Human Cranial Tree project (PI: George Karniadakis), and in collaboration
with a PetaApps award, on the visualization of their data, using a combination of ParaView and
custom-built code. The RP team has extended their previously developed custom ParaView
reader plug-in, which loads data in native NekTar format and pushes it into the ParaView
pipeline. Additionally, the plug-in now also computes derived quantities from the native NekTar
data, as well as curved surfaces at higher resolutions, where appropriate. The NekTar team has
begun integrating LAMMPS code with their simulations, producing multi-scale data sets. The
RP team has been working to visualize the resulting red blood cell data, as well as velocity
continuum and particle data. The UChicago/Argonne RP has been collaborating with the
Southern California Earthquake Center (SCEC) on the visualization of Frechet Kernel data.
Because the SCEC researchers simulate thousands of these kernels, which then need to be
investigated, the RP team has been exploring methods for automating the process for visualizing
and presenting this data to the researchers. With some initial set-up configuration, early efforts
have produced images and animations for dozens of such kernels, organized and presented on a
Web page. In addition, the RP has integrated geophysical data, including Moho discontinuity
surface, into visualizations of the simulation data. The UChicago/Argonne RP has made
numerous enhancements to vl3, their hardware-accelerated volume rendering software, in support
of TeraGrid users. Several customizations to its animation capabilities enabled the production of
several ENZO animations. New methods for data management have lead to larger data capacities
73

and increased resource utilization, and an improved compositing algorithm has resulted in faster
frame rates, particularly at larger scales. The vl3 software has been installed and tested on
Longhorn, the new TeraGrid XD Visualization resource at TACC. Vl3 has been used to
visualize the SCEC kernel data at smaller scales (~100MB), and will be used for larger simulation
data (~100GB+) in the near future. The UChicago/Argonne RP is also actively pursuing
additional TeraGrid users that can benefit from the volume rendering of large-scale datasets.
The NCSA visualization group worked with A. Masud and R. Calderer from the Department of
Civil and Environmental Engineering at the University of Illinois to visualize vortex shedding
created near a cylinder (left), forced-isotropic turbulence (middle), and turbulent channel flow
(right).
Recently, NCSA visualization has also begun working with E. Mehnert from the Institute of
Natural Resource Sustainability at the University of Illinois to visualize results from CO2 storage
modeling simulations.
The TACC team has been continuing to support both Spur and Longhorn to provide remote
visualization resources and services. In addition, the TACC visualization team has worked to
produce numerous visualizations in collaboration with users, including Clint Dawson who is
using his ADCIRC model to simulate the evolution of the BP oil spill data. Dawson's team used
Spur to create images from the simulations that run on Ranger. With the assistance of TACC’s
visualization specialists, their visualization run-time was reduced from three hours to 15 minutes.
While Dawson's team is using Longhorn to visualize forecast and simulation data such as wind
velocity, water elevation, and oil particle position, TACC’s visualization specialists are focused
on the overlay of oil particle movement and satellite data. The file system accessible from
Ranger, Spur, and Longhorn make it easy to access and visualize the data. TACC has also
completed year 1 training materials for Longhorn and used them for a training held January 27
and 28 at TACC. TACC held the first of the quarterly trainings on April 12, 2010. The training,
“Introduction to Scientific Visualization on Longhorn” was held at the TACC training facilities
and 20 attendees. The science disciplines ranged from Computational Plasma Physics to Image
Registration and Computational Fluid Dynamics. The course included an overview of computer
graphics and scientific visualization, and hands-on labs on Longhorn for visualizing with
ParaView and VisIt. The attendees were encouraged to bring their own data to the course to
74

provide a path to becoming Longhorn users. The second training for Longhorn was held at TACC
on June 3, 2010 with 16 on-site attendees and 4 attendees participating via web-casting. Future
tentative dates are September 3 and December 10 2010. These trainings will be publicized
through both the TACC and TeraGrid venues. Longhorn training modules are included as part of
the TACC Summer Supercomputing Institute to be held the week of July 19, 2010. John Clyne,
NCAR has provided training materials for VAPOR as a part of the TACC Summer
Supercomputing Institute and will be on site to provide the training in person.
8 Network Operations and Security
8.1 Highlights
A portion of SDSC’s Dash system was made available for production via Startup Allocations on
April 1. The initial deployment is two 16-node partitions, one with vSMP and one without to
allow for comparative benchmarking.
8.2 Networking
The TeraGrid continued operation of the backbone network consisting of core routers in Chicago,
Denver and Los Angeles, connected with dedicated 10 Gigabit/sec links and two levels of
redundancy in the form of alternative paths via NLR. TeraGrid Resource providers maintain their
respective site border routers and dedicated links to one of the three core routers. The dedicated
links consist of at least one 10Gb/s link .
The following two graphs show total usage and 20-second average peaks on the TeraGrid
backbone between Chicago and Denver.
75

Figure 8-1: Daily peak bandwidth on Chicago-Denver link.
Bandwidth is measured at 20s granularity.
76

Figure 8-2: Total transfer (GB) per day on Chicago-Denver link.
Total theoretical capacity is ~100,000 GB/day.
8.3 RP Operations: Networking
During this quarter, TG RP staff provided support and operation of the local RPs network
connectivity to the TeraGrid backbone. Duties include daily monitoring and maintenance of the
network, planning and/or implementing changes to RP connections to the TG backbone, direct
user support for network-specific issues, switch firmware upgrades, and HPC system
interconnects. TG RP staff also kept the user community apprised of interruptions in network
connectivity via local and TeraGrid user news.
During the second quarter of the 2010 calendar year, there were zero (0) unscheduled outages and
zero (0) scheduled maintenance events that directly affected Indiana University's connection to
TeraGrid. There were one hundred thirty one thousand forty (131,040) minutes during the second
quarter of 2010 calendar year, yielding a 100% uptime. If there were outage and maintenance
events, Indiana University's TeraGrid connectivity would have failed over to our Internet2 and
NLR connectivity, giving an actual site-to-site TeraGrid uptime of 100%.
[FutureGrid] The SDSC network connection was upgraded to 10Gb.
The ORNL RP spent a great deal of its effort planning a physical move (scheduled for Q3) on the
NSTG cluster and associated service nodes to the Spallation Neutron Source (SNS) site. This will
77

allow better integration with SNS’s ongoing operations and enable new network connectivity
with shorter hop counts to SNS data production nodes, pending resolution of ORNL network
policy revisions. The network topology will remain unchanged. The only substantial change
being replacing a short-haul link with a long haul link. The NSTG will continue to connect to the
Oak Ridge border router that serves as the main TeraGrid connection point for NSTG, NICS,
RDAV, and Keeneland.
8.4 Inca
The Inca team continued to support and maintain the Inca deployment on TeraGrid and helped
system administrators to debug failures on their machines. In response to IIS changes and
requests from the gig-pack, security and QA groups, existing Inca test suites were modified.
Three TeraGrid tests were also modified and one new test was written, configured and deployed.
At the time of this report 2,846 pieces of test data are being collected across TeraGrid platforms.
No machines were added or removed from the Inca TeraGrid deployment during this time. A
depot redundancy upgrade to the Inca TeraGrid deployment is still planned, which will allow data
to continue to be collected and queried at another site if the SDSC server fails. The depot
redundancy feature is currently being tested with TeraGrid Inca data.
The Inca team also continued work with an UCSD undergraduate student to develop a
visualization of Inca’s TeraGrid deployment using the Processing Toolkit. During this quarter,
the student added some interactive text interfaces and continued to work on code and cleanup.
This visualization work is ongoing and will be utilized by the Inca team and for demo purposes.
8.5 Grid Services Usage
The following figures show the usage statistics for the Grid services included in CTSS and
deployed at RP sites. An update of the Globus usage collection tools was performed in
anticipation of receiving new Gram 5 packets in Q3 after the new Globus Toolkit release is goes
live. This update was necessary as the current Gram 4 usage collector was not capable of
receiving Gram 5 packets. Both the Gram 4 and Gram 5 usage collectors will be utilized and
maintained going forward.
78

8.5.1 Data Movement (GridFTP) Statistics
Figure 8-3: Total GridFTP transfers (GB) for the quarter per site.
79

Figure 8-4: Total GridFTP transfers (GB) for the quarter per site.
80

Figure 8-5: Total number of GridFTP transfers per site.
81

Figure 8-6: Total number of GridFTP transfers per day per site.
82

8.5.2 Job Submission (GRAM) Statistics
Figure 8-7: Number of GRAM submissions per resource per day.
83

Figure 8-8: Total number of GRAM submissions per resource for the quarter.
84

Figure 8-9: GRAM-submitted jobs as a fraction of the total jobs on each resource. Note that not all GRAM
job submissions are submitted to the queue systems and end up in the allocations database. Hence in some
cases, number of GRAM jobs may exceed total number of reported jobs (the percentages for Frost, Bigred
and NSTG are certainly exaggerated due to this).
8.6 RP Operations: HPC Operations
TG RPs provided the personnel and infrastructure support for the local TG RP resources that are
made available to the user community. System administration duties include daily monitoring of
the health of the system, 24 hour on-call where provided, basic filesystem support, machinespecific
parallel filesystem support, and operating system software maintenance and upgrades.
Additionally, this activity provides for CTSS kit (grid middleware) deployment, administration
and support. RP staff also assisted local and TeraGrid user services personnel to provide timely
announcements to the user community related to the status of all TG resources.
85

8.6.1 IU
[FutureGrid] The Sierra and India clusters passed acceptance testing. An LDAP server for use
across all FutureGrid machines has been implemented.
8.6.2 LONI
SAGA 1.5 is nearing release, and work is progressing with NICS to support remote job
submission to Kraken. A planned storage upgrade for Queen Bee had to be delayed due to
funding cycle issues. 5.6M SUs were delivered to 51.4K TeraGrid jobs serving 129 to 173 unique
users per month. The system had only 1 planned outage for a total of 11hrs of down-time for
storage maintenance.
8.6.3 NCSA
NCSA increased the storage capacity of the Abe/Lincoln clusters by over 10% for a current total
of 344 TB of usable scratch and project space.
8.6.4 ORNL
The largest activity for the ORNL-RP’s NSTG node was planning and preparing for the physical
move to the SNS site in Q3. This included some preparatory configuration changes, many
discussions with staff and labor as well as coordination with the Main campus machine room staff
and the SNS machine room staff. In addition, highlights of other activities performed during the
quarter as part of the computing operations included; investigating, implementing, and testing
propagation of gateway credentials; coping with the need for increasing numbers of firewall
exceptions between SNS machines and TeraGrid subnets in response to ORNL default deny
policy on outbound network session; continued effort to understand periodic performance
degradation in PVFS2 deployment on NSTG cluster; finalization of last remaining items from
previous OS upgrade; ESG operations; replacement of several failing or failed disks; and
preparing for a AMIE upgrade that occurred TG-wide in July.
8.6.5 PSC
The National Institute of General Medical Sciences (NIGMS), part of the National Institutes of
Health (NIH), awarded a two-year, $2.7 million grant to the National Resource for Biomedical
Supercomputing (NRBSC) at PSC to host a specialized supercomputer for biomolecular
simulation designed by D. E. Shaw Research (DESRES). The machine, called Anton, will be
made available without cost by DESRES for non-commercial research use by universities and
other not-for-profit institutions commencing in late 2010. (D. E. Shaw Research is an independent
research laboratory that conducts basic scientific research in the field of computational biology.)
Anton was designed to dramatically increase the speed of molecular dynamics (MD) simulations
compared with the previous state of the art, allowing biomedical researchers to understand the
motions and interactions of proteins and other biologically important molecules over much longer
time periods than have previously been accessible to computational study. The machine and the
novel algorithms it employs were designed by a team of researchers led by David E. Shaw, chief
scientist of DESRES. Anton has run simulations extending for more than a millisecond of
biological time — about 100 times longer than the longest previously published all-atom MD
simulation.
NRBSC has invited U.S. biomedical researchers to submit proposals for allocations of time on the
Anton machine. The deadline for the first round of proposals has already passed. A peer review
committee to be convened by the National Research Council will review proposals.
86

As part of the NIGMS award, NRBSC also will install a new data storage and analysis subsystem,
including nearly half a petabyte of online disk capacity, to host and make widely available the
tremendous amount of data that will be produced by MD simulations on Anton.
PSC has been actively involved in the coordination of implementation of the first phase of the
Common User Environment (CUE) on the AQSL (Abe,QueenBee,Steele,Lonestar) complex. We
helped define and have monitored the installation and testing of a standardized version of
modules software across this complex. This software was installed by Resource Provider liaisons
under coordination of the common user environment working group lead by PSC and contains
commonly defined core modules as well as common environment variables consistent across all
platforms. We are in the process of having this environment expanded to include all TeraGrid
platforms.
The PSC MyProxy Certificate Authority (CA) has received final TAGPMA (The Americas Grid
Policy Management Authority) accreditation, has been added to the collection of TeraGrid-
Accepted CAs and is in full production. It will serve as an alternate/backup MyProxy credential
service for TeraGrid, providing short-lived credentials to users when the default TeraGrid
MyProxy CA at NCSA is unavailable. The fail-over capability has been tested in the production
environment.
PSC Scientific Computing Performance Specialist Mahin Mahmoodi installed TAU-2.19.1 on
Kraken, which was necessary to overcome problems with TAU-2.19 for C codes.
Staff members of the Center for Analysis and Prediction of Storms at the University of Oklahoma
(OU/CAPS) reported that attempts to copy large files from the OU grid node (non-TeraGrid) to
PSC’s archiver via TeraGrid-specific GridFTP addresses were failing. As it turned out, any non-
TeraGrid node attempting to copy large files to these GridFTP addresses was affected. Provision
of an alternate address for non-TeraGrid nodes was established. We are incorporating these
changes into our user documentation. This work represented a more general repair for all non-
TeraGrid nodes, not just OU/CAPS.
8.6.6 Purdue
Purdue’s Condor pool continues to support individual users and gateway uses. During Q2 Purdue
RP staff continued to extend the performance of Puffin, a platform of web-services developed at
Purdue to promote widening the Condor user base, e.g., use by gateway applications, and
increasing Condor usage by existing users through automation and simplification of job
submission and control. Efforts for Puffin development this quarter focused on maximizing job
submission throughput. The code which handles communication between Puffin and the
underlying Condor engine was refined and rewritten in some cases for increased stability and
efficiency. Analytical systems were also implemented to allow tracking of performance metrics
throughout these efforts and to aid future improvements of the Puffin platform. Currently the
Puffin web service Condor submission supports up to 5000 simultaneous job submissions. Purdue
RP will work with end users and gateway applications in the next phase.
Purdue continues work on supported its cloud resource Wispy. The RP continues to work with
ANL’s science cloud in supporting users.
The RP continues with performance turning on its central BlueArc filesystems to improve
responsiveness of scratch storage for TeraGrid user jobs.
87

8.6.7 SDSC
SDSC put our Dash resource into production on April 1 for Startup allocations. Dash is
precursor experimental system for Gordon coming in 2011.
8.6.8 UC/ANL
The UChicago/Argonne RP has continued to host repo.teragrid.org, which includes the TeraGridwide
CVS repository.
8.7 GIG Operations
The GRAM5 usage collector software was updated on globus-usage.teragrid.org. Ongoing
support of MyProxy and Kerberos servers for TeraGrid at NCSA, the mail.teragrid.org server, as
well as the MyProxy, GSI-OpenSSH, and GSI-SSHTerm software used by TeraGrid continued.
This includes regular maintenance as well as addressing incoming support tickets. The CIC IDM
WG TeraGrid Pilot, worked with the TeraGrid Security WG to obtain sign-off on the Federated
IDM Security Incident Response Policy. After updating the policy to address the TG Security
WG's comments, the TG Security WG voted to approve the policy (8 in favor, 3 abstain). The
policy was also presented at the InCommon Campus Authentication & Middlware Planning
meeting on June 22, 2010.
8.8 Security
8.8.1 Security Working Group
Summary of Security Incidents: There were five compromised user accounts and two login node
compromises. A machine used by science gateway developers was compromised which lead to
other account credentials being captured. In some instances the attackers inserted unauthorized
SSH keys into users directories to allow alternate methods of access. The Incident Response
team has disabled all of the accounts involved and audited TeraGrid productions machines for the
known fraudulent SSHKeys. Securing Community Account
Revisited: Victor Hazlewood has been documenting community account usage on the TG.
Currently, documentation of the use cases for 22 science gateways is provided at Teragrid Forum
wiki Science Gateway Use Cases site
A summary of software used and security methods employed for the user accounts used at the
RPs is available in the Summary Table at this website. We are planning on setting up a meeting to
discuss community account management and implementation in the near future with Victor and
Nancy. Goal is to establish some standards or commonly accepted positions relating to
community accounts, for example: Community account cannot have an interactive shell session
on productions machines).
Accessing the community account information page: With the move of the Teragrid webserver to
LifeRay complete, the access control permissions for specific areas of the website needed to be
re-defined. The docs group contacted Jim Marsteller to ask who should have access to the science
gateway info page. Since this page is used to look up contact information for community
accounts, ever member of the incident response team requires access to this page. Jim supplied
the docs group the names of all of those on the TG security contact list so they have access.
Vetted/Unvetted Process Review: The Core2 working group team has been working on an
implementation document for Vetted & Unvetted TGUP accounts in order to move forward with
improving the account creation process with TGUP. The security working group was asked to
review the document and provide comments. Jim sent Maytal the security working group's
comments on May 35th. One of the major questions is whether the pops reconciliation process
will still have a TG staff member vetting the authenticity of a user generated account request.
88

The second issue related to adding a user to an allocation, although the PI may "check off" on an
addition request electronically, it may create a situation where PIs approve additions
automatically without proper review. One suggestion is to have the PI specify the means of
validating the relationship with the user to the project. In this case an error could cost the PI
allocations so they SHOULD have an interest in reviewing additions.
8.8.2 Expanded TeraGrid Access
Development to integrate Shibboleth/InCommon authentication with the TeraGrid User Portal
(TGUP) was completed. Documentation and source code was provided to the TGUP team, and
work is proceeding to deploy the new functionality in the production TGUP. The GIG assisted
PSC in successfully completing TAGPMA accreditation for their MyProxy CA, which now
serves as the backup for TGUP authentication in the event the NCSA MyProxy CA is offline. The
GIG also assisted the TGUP team in deploying the fail-over functionality between the NCSA and
PSC MyProxy servers.
8.9 RP Operations: Security
During this quarter, TG RPs provided expert staff in support of local and TG-wide collaborative
security. TG RP local security operations include intrusion detection, examining log data,
investigating unusual use patterns, and incident response for all hardware and networks on a 24-
hour on-call basis. All TG RP partners are required to participate in the TG-wide Security
Working Group which coordinates security operations and incident response across the project.
8.9.1 IU
[FutureGrid] A VM running on the India cluster was broken into due to a weak root password.
The machine was taken down and because of the stateless nature of the image it did not need to
be cleaned. The password has since been changed to something stronger.
8.9.2 LONI
There were no security incidents this quarter.
8.9.3 NCAR
NCAR disabled five accounts that had been implicated in three different incidents at other RPs.
None of these accounts had been used for unauthorized access, and no compromised ssh keys
needed to be removed from our systems.
8.9.4 NCSA
The NCSA security team responded to three different incidents involving TeraGrid users or
resources in the second quarter of 2010. All three of these incidents involved remotely
compromised user accounts. Each the incidents were detected with the current security
monitoring done within NCSA. On two of the intrusions they were able to escalate to root
privileges on the machines. The downtime for the machines for each incident was less than an
hour. NCSA worked with users in each incident to get their remote systems cleaned and access
back to NCSA re-enabled within a day.
8.9.5 ORNL
During the second quarter of calendar 2010, there were no root compromises on the NSTG
computing nodes. There were no users who lost control of their credentials from activities on
NSTG machines. TeraGrid users who were the victims of stolen credentials at other TeraGrid
sties did not use those credentials to log on to NSTG machines. NSTG staff took timely action to
de-activate accounts of users whose credentials were stolen and users whose credentials may have
89

een exposed during a known incident. During the second quarter, the tripwire footprint was
expanded to monitor hashes for a larger set of files on NSTG and ESG (at ORNL) machines.
8.9.6 PSC
PSC had no security incidents during the quarter.
The PSC website now supports an automated password reset function. It is based on the TeraGrid
password reset model that the Security working group developed. It also enforces new password
“strength” requirements (see http://www.psc.edu/general/policies/passwords/), which are the
same as the ones that Carnegie Mellon University is adopting for InCommon membership.
8.9.7 Purdue
During Q2 Purdue RP disabled 10 TeraGrid accounts on the Purdue TG resources due to
compromises at other RP sites.
8.9.8 SDSC
SDSC has had zero major incidents, zero serious incidents, and one minor incident this quarter.
The minor incident consisted of a user account that was compromised at other sites. The account
was suspended at SDSC to prevent possible misuse. No unauthorized activity was observed prior
to locking the account.
8.9.9 TACC
Within the reporting period four accounts were locked out at TACC due to a compromise on a
non-TACC system that occurred on 16Apr10 - including the 2nd security incident involving a
specific community account.
Also, TACC staff discovered a side-effect related to the qb4 compromise on a non-TACC system
that occurred on 30Apr10. A username and password were harvested from another system
located at the site of the compromised system. Most likely attack vector: two user accounts
known compromised during the qb4 incident had sudo access to the compromised system.
8.10 Common User Environment
The CUE has been finalizing the implementation of the Common User Environment Management
System (CUEMS) and the Common User Environment Variable Collection (CUEVC). The
"modules" program has been installed on all machines throughout the TeraGrid, and a common
set of modules have been defined to provide users with an interface that is similar across the
resources. Members of the working group have been coordinating with RP systems personnel on
implementation. The implementation is defined in a document outlining the necessary
components in order to be in compliance with the CUE. Mechanisms were put into place to allow
users to opt into the CUE by making modules their default EMS and allowing the CUE modules
to be loaded automatically. Working with the Documentation group, a new document describing
the use of modules has been drafted, with clear instructions on how to activate the CUE.
8.11 System Performance Metrics
8.11.1 Resource Usage Summary
TeraGrid again saw further increases in delivered NUs. TeraGrid compute resources delivered
12.1 billion NUs to users, 0.3% as TeraGrid Roaming NUs (Figure 8-11). Roaming NUs
continued to decline as that allocation feature is phased out. The 12.1 billion NUs represent a
20% increase over the NUs delivered in Q1 2010. Nearly 100% of those NUs were delivered to
allocated users. The Q2 2010 NUs delivered are 1.7x the NUs delivered in Q2 2009.
90

Figure 8-9. TeraGrid Computational Resource Usage
8.11.2 Summary of Usage / Users / Allocations by Discipline
Figure 8-12 shows the relative fraction of PIs, current accounts, active users (individuals charging
jobs), allocations, and NUs used according to discipline. The 10 disciplines with greater than 2%
of NUs used are listed in the figure. Staff usage is excluded, and users that appear on more than
one allocated project may be counted more than once, if those projects are associated with
different fields of science. Consistent with past quarters, the same disciplines dominant the NUs
used column, though there have been some changes in the ordering. Astronomical Sciences ranks
second, and Chemical and Thermal Systems ranks 8th. As in past quarters, more than 20% of PIs
and current accounts, as well as 15% of active users, are associated with the 20 “other”
disciplines, collectively representing about 2% of total quarterly usage.
91

Figure 8-10. TeraGrid Usage Summary by Discipline
8.11.3 Summary of Top PIs / Projects, Overall and by Discipline
882 different projects charged usage on TeraGrid computational resources, an increase of almost
10% over Q1 2010. Figure 8-13 shows the top 20 PIs by total usage in NUs. The top 20 PIs
consumed 52% of the NUs used and the remaining 862 projects consumed the other half. Table
8-1 shows the top PIs (>2% of the discipline’s usage) in each discipline with more than 2% of
total usage; the final section shows the top PIs for all other disciplines. The overall rank of each
PI among the 882 projects charged is also listed.
92

Figure 8-11. Top 20 TeraGrid PIs, Q2 2010.
Table 8-1. Top PIs by Discipline for Q2 2010
Physics - 2,613,313,083
PI, Institution
NUs Rank
Colin Morningstar, CMU 842,401,191 2
Robert Sugar, UC Santa Barbara 448,720,301 4
Erik Schnetter, Louisiana State U 420,618,820 6
Klaus Bartschat, Drake U 128,603,597 16
Keh-Fei Liu, U Kentucky 123,841,843 18
Mathieu Luisier, Purdue U 109,675,314 23
Konstantinos Orginos, College of Wm & Mary 90,603,726 28
Stanislav Boldyrev, U Wisconsin-Madison 62,943,681 40
Cary Forest, U Wisconsin-Madison 55,752,242 44
George Fleming, Yale U 52,579,520 49
All 53 Others 459,011,257
Astronomical Sciences - 2,422,240,376
PI, Institution NUs Rank
Juri Toomre, U Colorado 435,129,692 5
Tiziana Di Matteo, CMU 297,016,473 8
Robert Harkness, UC San Diego 214,062,477 11
Roger Blandford, Stanford U 137,160,363 15
Adam Burrows, Princeton U 128,001,206 17
Michael Norman, UC San Diego 115,572,718 22
Thomas Quinn, U Washington 92,977,957 27
Andreas Berlind, Vanderbilt U 68,804,123 34
Renyue Cen, Princeton U 61,833,758 41
Dean Townsley, U Alabama 61,322,982 42
Bronson Messer, U Tennessee 54,935,993 45
Richard I. Klein, UC Berkeley 53,524,110 47
Paul Shapiro, UT Austin 50,376,033 50
All 57 Others 470,118,205
Molecular Biosciences - 2,229,651,467
PI, Institution NUs Rank
Klaus Schulten, UIUC 367,290,273 7
Michael Klein, Temple U 232,303,319 10
Gregg Beckham, NREL 160,429,658 12
Aleksei Aksimentiev, UIUC 149,745,363 14
Emad Tajkhorshid, UIUC 118,522,530 21
Thomas Cheatham, U Utah 90,237,671 29
Carlos Simmerling, SUNY Stony Brook 73,069,966 30
Wonpil Im, U Kansas 71,903,651 31
Cheng Zhu, Georgia Tech 65,427,129 38
Michael Feig, Michigan State U 63,094,958 39
Stephen White, UC Irvine 52,669,275 48
All 148 Others 784,189,969
Earth Sciences - 1,069,997,914
PI, Institution NUs Rank
Thomas Jordan, USC 912,466,883 1
Omar Ghattas, UT Austin 42,853,665 61
Burkhard Militzer, UC Berkeley 30,272,571 70
Magali Billen, UC Davis 23,085,626 88
All 20 Others 61,319,169
Atmospheric Sciences - 1,016,010,959
PI, Institution NUs Rank
Homayoun Karimabadi, UC San Diego 613,303,983 3
Ming Xue, U Oklahoma 157,263,205 13
Fuqing Zhang, Penn State U 67,526,309 37
Nikolai Pogorelov, U Alabama, Huntsville 58,627,158 43
James Kinter, COLA 24,906,577 82
All 39 Others 94,551,366
Materials Research - 725,105,565
PI, Institution NUs Rank
Chris Van de Walle, UC Santa Barbara 70,902,277 33
Stefano Curtarolo, Duke U 48,035,319 52
Peter Cummings, Vanderbilt U 44,419,815 57
Giulia Galli, UC Davis 44,307,422 58
Lubos Mitas, NC State U 44,223,953 59
Julia Medvedeva, Missouri S&T 30,467,783 69
Marvin L. Cohen, UC Berkeley 30,130,898 71
James R. Chelikowsky, UT Austin 28,942,090 74
Dallas Trinkle, UIUC 28,784,357 75
Stephen Paddison, U Tennessee 26,331,592 79
Bhagawan Sahu, UT Austin 24,551,679 84
John D. Joannopoulos, MIT 23,085,547 89
Gautam Ghosh, Northwestern U 21,721,675 96
Ivan Oleynik, U South Florida 18,055,539 110
Sergey Barabash, UCLA 17,023,083 115
Dane Morgan, U Wisconsin-Madison 16,850,475 116
Alexie Kolpak, Yale U 16,290,849 118
Axel Vandewalle, Caltech 14,808,563 123
All 87 Others 178,200,151
Chemistry - 631,769,408
PI, Institution NUs Rank
Gregory A. Voth, U Chicago 289,850,125 9
Guillermina Estiu, U Notre Dame 71,733,514 32
Enrique Batista, LANL 23,771,817 86
93

Arun Yethiraj, U Wisconsin-Madison 14,459,419 125
Rommie Amaro, UC Irvine 14,348,026 126
All 136 Others 216,423,081
Chemical, Thermal Systems - 627,882,647
PI, Institution NUs Rank
Madhusudan Pai, Stanford U 123,140,649 19
Kenneth Jansen, RPI 93,190,694 26
Daniel Bodony, UIUC 68,332,121 35
Rajat Mittal, Johns Hopkins U 49,668,491 51
Cyrus K. Aidun, Georgia Tech 24,787,473 83
Olivier Desjardins, U Colorado 23,016,499 90
Andreas Heyden, U South Carolina 22,171,412 92
George E. Karniadakis, Brown U 19,384,553 105
Pui-kuen Yeung, Georgia Tech 18,636,476 107
John Kim, UCLA 17,184,641 114
Krishnan Mahesh, U Minn 15,917,735 121
Stephen Pope, Cornell U 14,277,061 127
All 66 Others 138,308,990
Advanced Scientific Computing - 339,387,474
PI, Institution NUs Rank
Martin Berzins, U Utah 107,137,049 24
Ali Uzun, Florida State U 103,536,693 25
Tonny Oyana, SIU Carbondale 67,983,386 36
Xiaowen Wang, UCLA 12,081,320 139
Richard Loft, NCAR 10,287,221 155
David LeBard, Arizona State U 8,958,801 176
Bilge Yildiz, MIT 7,282,956 191
Celso Mendes, UIUC 6,788,128 203
All 34 Others 15,331,920
8.11.4 Usage by Institution
Mathematical Sciences - 197,054,478
PI, Institution NUs Rank
Clinton N. Dawson, UT Austin 122,336,390 20
Daniel Bernstein, U Illinois, Chicago 54,444,478 46
Nicholas Zabaras, Cornell U 6,021,641 218
All 21 Others 14,251,955
All Others - 207,867,295
PI, Institution (Discipline) NUs Rank
Mark Miller, SDSC (Environmental Biology) 34,817,821 68
Teresa Chereskin, SIO (Ocean Sciences) 22,671,992 91
Tony Keaveny, UC Berkeley (Biological and Critical
Systems) 20,097,602 103
Huajian Gao, Brown U (Mechanical and Structural Systems) 13,443,461 133
Server Yilmaz, U Pittsburgh (Science and Engineering
Education) 9,162,671 172
Shaowen Wang, UIUC (Social and Economic Science) 8,447,801 183
Dan Stanzione, UT Austin (Environmental Biology) 6,994,064 197
Pranav Shrotriya, Iowa State U (Mechanical and Structural
Systems) 6,543,656 210
Christian Lastoskie, U Michigan (Biological and Critical
Systems) 5,490,875 229
James McWilliams, UCLA (Ocean Sciences) 5,324,979 232
Markus Buehler, MIT (Mechanical and Structural Systems) 4,818,131 241
Kim Dillman, Purdue U (Cross-Disciplinary Activities) 4,790,619 244
Jun Song, Brown U (Mechanical and Structural Systems) 4,765,960 246
All 119 Others 60,085,396
Users from 316 institutions (up from 305 in Q1 2010) across the U.S. and abroad were associated
with the usage delivered by TeraGrid resources. The top 47 institutions (by NUs consumed) were
responsible for 80% of the total NUs delivered. Table 8-2 lists the 316 institutions, the number of
users responsible for the usage at each site, and the NUs consumed.
Table 8-2. TeraGrid Usage by Institution, Q3 2009
Institution
Users
UIUC 216 901,704,211
CMU 43 797,379,040
UC San Diego 56 630,643,099
LANL 7 607,263,917
U Utah 34 487,224,754
SDSC 11 471,986,218
Jefferson Lab 2 469,628,380
U Arizona 7 444,016,918
U Colorado 52 338,540,787
NCAR 12 272,519,944
U Wisconsin-Madison 35 197,349,398
Stanford U 36 195,219,737
Princeton U 9 179,483,989
Temple U 9 176,903,419
U Oklahoma 33 165,266,150
U Washington 51 162,434,751
Georgia Tech 68 157,574,223
NREL 9 156,631,958
UC Berkeley 45 143,887,943
UT Austin 41 143,312,473
Purdue U 47 137,721,649
Caltech 18 117,216,799
SUNY Stony Brook 10 114,656,664
Duke U 5 113,885,534
UC Davis 37 113,418,320
U Maryland, College Park 26 110,989,827
MIT 56 110,964,628
U Wyoming 6 109,111,424
Institution
Users
UCLA 43 108,339,265
Florida State U 6 107,115,316
College of Wm & Mary 5 105,473,430
U Notre Dame 11 97,444,719
MPI Gravitationsphysik
(Germany) 3 96,237,353
RPI 14 94,953,661
UC Santa Barbara 33 92,852,016
NC State U 24 92,731,546
Drake U 4 90,582,365
NYU 14 89,248,370
Penn State U 16 88,022,414
Vanderbilt U 16 84,995,576
Louisiana State U 26 84,259,899
U Kansas 13 72,766,013
UC Irvine 20 72,403,341
SIU Carbondale 5 68,021,934
Yale U 9 66,690,691
Cornell U 45 65,544,695
Michigan State U 12 63,568,384
U Illinois, Chicago 9 63,453,682
U Chicago 31 61,604,754
RIT 7 59,895,362
U Kentucky 7 58,963,374
Harvard U 15 58,774,850
U Alabama, Huntsville 2 58,627,158
U Rochester 5 58,616,755
U Tennessee 9 56,078,910
94

Institution
Users
Colorado Sch of Mines 3 55,310,241
MPI Plasmaphysik
(Germany) 1 54,675,602
U Virginia 15 52,824,012
U Pennsylvania 10 52,119,075
Johns Hopkins U 7 52,021,344
Albert Einstein Institute
(Germany) 6 50,313,712
Florida Atlantic U 5 48,403,403
Brown U 52 45,822,874
Washington U 12 44,176,412
U Pittsburgh 35 41,713,031
NASA Goddard 2 41,118,432
U Iowa 11 41,106,481
Missouri S&T 10 40,117,869
University of Sussex 2 40,035,946
LLNL 3 33,271,697
Columbia U 23 32,886,615
SIO 3 32,543,567
UNC 8 32,352,522
Arizona State U 11 31,199,428
U Toronto (Canada) 6 30,124,658
Northwestern U 22 30,111,368
U Michigan 20 29,583,351
UC Santa Cruz 5 29,093,294
U Minn 6 28,485,578
Indiana U 10 27,377,290
Texas A&M 4 26,803,770
Drexel U 19 26,796,194
U South Florida 13 26,647,804
Rice U 9 26,071,989
Mt Sinai Sch of Med 10 24,676,235
ANL 4 24,494,718
Weill Cornell Med College 4 24,443,709
U South Carolina 6 23,692,574
Univ of Vienna 1 23,255,638
UNLV 8 23,206,201
UC San Francisco 10 21,938,554
U Heidelberg (Germany) 4 21,771,525
U Alabama 2 19,773,826
Tel Aviv U (Israel) 1 19,559,445
Iowa State U 8 16,937,104
Toyota Tech Inst, Chicago 5 16,271,963
Bioprocessing Tech Inst
(Singapore) 1 15,961,927
U Florida 19 14,551,198
U Richmond 1 14,279,732
Lock Haven U 5 14,178,679
Keldysh Inst of Applied
Math (Russia) 1 14,059,645
Brandeis U 14 13,834,860
Boston U 18 13,569,041
U Mass, Amherst 15 12,567,797
U Maryland, Baltimore Co 14 11,457,514
Duquesne U 5 10,961,027
Wesleyan U 2 10,885,793
USC 4 10,457,138
NC Central U 2 9,953,865
Institution
Users
MPI Astrophysik (Germany) 2 9,944,249
Oxford U (United Kingdom) 1 9,507,236
UC Riverside 5 9,277,423
Virginia Tech 7 9,221,329
Georgetown U 10 9,220,039
Auburn U 6 8,847,212
U Jena (Germany) 2 8,845,034
Rockefeller U 1 8,751,384
St Louis U 16 8,249,773
WPI 2 8,064,218
U Delaware 5 7,973,918
Long Island U 3 7,872,687
CIW 5 7,613,867
U Puerto Rico System 7 7,481,881
College of Charleston 2 7,053,134
Montana State U 5 6,929,795
U Maryland, Baltimore 2 6,693,038
U Texas Hlth Sci Ctr
Houston 7 6,654,504
U Connecticut 5 6,491,143
Rutgers U 20 6,471,322
U New Hampshire 7 6,392,655
Chosun U (South Korea) 1 6,359,204
USGS 2 6,355,677
Perimeter Inst (Canada) 2 6,245,100
University College, London
(United Kingdom) 5 6,003,945
Desert Res Inst 3 5,879,367
Corning, Inc. 2 5,866,082
La Jolla Bioeng Inst 3 5,856,424
CENTRA - Inst Sup Tecnico
(Portugal) 1 5,798,510
UT San Antonio 1 5,583,566
CWRU 1 5,553,287
LBNL 2 5,413,040
Indiana U, Kokomo 1 5,174,163
U Tenn, Oak Ridge 1 5,168,864
U Michigan, Flint 1 5,050,633
Tulane University 4 4,949,138
Colorado State U 12 4,853,494
U Illes Balears (Spain) 1 4,513,554
Princeton Plasma Physics
Lab 2 4,339,410
U Puerto Rico, Mayaguez 2 4,297,589
U Houston 2 4,216,479
U Puerto Rico, Rio Piedras 1 4,129,085
NICS 2 4,085,546
Institute for Advanced Study 2 4,083,681
Florida A&M U 4 3,772,159
Predictive Science, Inc. 3 3,674,712
Clark U 3 3,584,226
AMNH 6 3,457,463
U North Texas 10 3,391,689
San Diego State U 8 2,974,537
Connecticut College 1 2,956,926
UT Southwestern Med Ctr 2 2,853,814
ND State U 2 2,818,084
CSU-Long Beach 2 2,752,978
95

Institution
Users
NIST 1 2,723,312
Atmospheric Tech Services
Co. 1 2,723,284
U New Mexico 5 2,702,950
Lehigh U 3 2,500,322
Kingsborough Comm
College 1 2,483,868
UT El Paso 22 2,394,797
U Mass, Boston 1 2,369,654
Truman State U 2 2,364,790
Burnham Inst (a) 1 2,265,038
Armstrong Atlantic State U 4 2,258,348
Southwest Res Inst 1 2,161,587
U Mass, Dartmouth 1 1,989,696
SUNY at Buffalo 2 1,908,922
Albany State U 4 1,901,445
CERN 1 1,893,101
U Sciences, Philadelphia 1 1,865,710
Fisk U 2 1,744,073
Applied Sci Research 1 1,660,617
Hamilton College 4 1,654,928
U Oregon 12 1,635,819
Cold Spring Harbor Lab 2 1,565,451
IIT 6 1,358,508
University of Nebraska
Medical Center 1 1,334,422
PSC 2 1,285,900
UT Med Branch 2 1,284,402
Illinois State U 2 1,245,899
Washington State U 3 1,198,799
Michigan Tech 9 1,173,522
ORNL 12 1,157,036
George Mason U 3 1,117,224
U Arkansas 4 1,110,040
UNC System Office 1 1,013,605
U Zurich (Switzerland) 2 944,599
U Pittsburgh Sch of Med 2 933,782
Natl Hellenic Res Fnd
(Greece) 2 929,414
Baylor College of Med 1 913,976
Eastern Michigan U 1 903,040
University of Wisconsin-Eau
Claire 1 828,329
U Georgia 8 799,927
Texas Tech U 3 796,661
Kansas State U 2 784,519
IIMCB (Poland) 1 737,543
CUNY City College 1 688,370
Queens U, Belfast (United
Kingdom) 1 610,980
Howard U 1 602,172
Ill. State Water Survey 2 592,707
U Central Florida 3 580,310
Natl Space Sci & Tech Ctr 1 579,722
U North Dakota 15 572,679
UC Merced 1 566,090
Vorcat, Inc. 1 561,442
Clark Atlanta University 1 558,075
Institution
Users
Ursinus College 2 554,428
U Idaho 2 537,435
University of Montana 1 532,591
CSU-Fullerton 2 528,227
Lamar University-Beaumont 5 490,808
Southern Methodist U 2 476,482
U Pisa (Italy) 1 466,001
Wichita State U 5 437,593
LSST Corp. 2 396,539
Queensborough Comm
College 2 373,340
Ill. Geological Survey 1 368,019
RENCI 3 359,237
NJIT 1 352,720
U Houston-Clear Lake 2 339,052
U Hawaii, Honolulu 1 335,292
Ohio U 2 298,041
Marquette University 3 293,900
UT Arlington 3 293,202
Computational Sci & Eng 2 281,542
Naval Postgrad School 1 272,686
Vienna U of Technology
(Austria) 1 228,740
U Missouri, Columbia 1 215,562
U Hawaii, Manoa 2 214,664
US FDA 1 197,191
Marmara U (Turkey) 1 184,930
Active Site Dynamics, LLC 1 177,320
Old Dominion U 1 173,543
Clarkson U 1 170,212
Jackson State U 1 167,647
Virginia Commonwealth U 4 156,017
Alfred University 1 153,794
Wayne State U 2 146,367
Rosalind Franklin U 1 139,526
Macalester College 6 129,123
U Missouri 2 122,992
New College of Fla. 1 121,998
Monell Chemical Senses Ctr 1 120,948
Inst HEP (China) 1 118,193
U Minn, Twin Cities 1 107,969
Tufts U 7 95,870
Marshall U 5 94,427
UT System 1 86,360
Climate Central 1 77,879
POSTECH (South Korea) 1 64,389
U Detroit Mercy 14 57,801
SD State U 4 39,977
U Hawaii Research Corp 1 39,127
Technische Universitaet
IImenau 1 35,957
U Canterbury (New Zealand) 1 34,679
Shippensburg U 13 34,369
U Cordoba (Spain) 1 34,199
Marian College 1 32,522
Francis Marion U 1 28,386
Hope College 1 27,570
Wilkes U 3 27,025
96

Institution
Users
CSU-San Bernardino 1 25,770
BYU 1 23,485
SUNY at Binghamton 4 21,359
Westminster College of Salt
Lake City 1 19,228
United Technologies 1 13,800
U New Orleans 1 11,356
Syracuse U 1 10,540
Boise State U 1 10,419
Allegheny Gen Hosp 1 9,034
Richard Stockton College of
New Jersey 1 8,402
Julich Supercomputing
Centre (Germany) 1 7,456
Kent State U 1 6,587
Rose-Hulman Inst of Tech 1 6,303
Widener U 2 6,287
Slovak Acad of Sci
(Slovakia) 2 6,085
UT Permian Basin 1 3,726
Washington U School of
Med 1 3,650
CUNY 1 3,012
Monash U (Australia) 2 2,918
U Houston-Downtown 1 2,596
U Paul Sabatier (France) 1 2,351
San Francisco State U 2 1,638
Fond du Lac Tribal College 2 1,635
Florida Intl U 2 1,056
Central Michigan U 1 1,001
Albany College of Pharmacy
& Health Sci 2 993
U Miami 2 779
SUNY ESF 1 750
Northeastern U 3 621
Oregon State U 2 300
U Wisconsin-Milwaukee 2 290
George Washington U 1 210
KEK Japan (Japan) 1 200
St Vincent College 1 191
National U 1 147
Santa Clara U 2 54
APS 1 38
Centre College 1 29
Wofford College 4 27
Earlham College 1 21
PNL 1 21
U Mass, Lowell 1 20
Indian Institute of Science
(India) 1 19
UNC, Asheville 1 3
IFPRI 1 1
Villanova U 1 0
97

9 Evaluation, Monitoring, and Auditing
9.1 Highlights
This is a new section to the TG quarterly report, resulting from the first NSF XD awards for TIS
(Technology Insertion Service) and TAS (Technology Audit Service). This section of the TG
report was developed with a flow in mind – from evaluation and recommendation of technologies
(TIS), to assuring the quality of those technologies via monitoring (QA), and finally the oversight
and quality control of technologies (TAS).
9.2 XD – TIS: Technology Insertion Services
National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign
John Towns, PI
Program Start Date: 7/1/2010
9.2.1 Technology Insertion Service Program Goals
The purpose of XSEDE TIS (the eXtreme Science and Engineering Discovery Environment,
Technology Insertion Service) funded under the NSF TeraGrid Phase III: eXtreme Digital
Resources for Science and Engineering program is to execute comprehensive, expert
technology evolution of the world’s most advanced collection of integrated digital services in
scale and diversity, helping to maximize its impact and advance it over the project lifetime. The
XSEDE partnership includes NCSA, NICS, PSC, TACC, and the Univeristy of Virginia. XSEDE
will ensure that XD’s architecture, resources, and services remain at the forefront of technology
and are reliable, usable, and offer maximum possible performance, thus enabling transformational
science across domains, while advancing integrated distributed environments, high end
computing, data management, and other technologies for computational science.
There are two components to the XSEDE TIS – a Technology Evaluation component and a
Usage, Satisfaction, and Impact Evaluation component.
9.2.1.1 Technology Evaluation
The Technology Evaluation team has the task of identifying hardware and software technologies
that can help XD improve its operations and impact. We anticipate the CMS awardee will have a
periodic deployment schedule in which this service will participate. In Yr1 we will participate
with TeraGrid as appropriate based on knowledge of the technologies being deployed.
The Technology Insertion team will develop and maintain an open, web-front-ended database of
technology projects of potential value to XD sites and users known as XTED (XSEDE
Technology Evaluation Database).
A Technology Evaluation Laboratory will be operated to help ensure that any technology changes
to the infrastructure are thoroughly evaluated and tested before recommended for inclusion into
the XD production environment. The laboratory will leverage the Futuregrid resources and
environment for this testing. We will design all tests, coordinate their execution, evaluate and
document their outcomes, and iterate until we can provide our recommendation regarding
deployment.
XSEDE will provide documentation of the processes and procedures for the evaluation of
technologies, and will offer code consulting services to software developers to help promote code
quality necessary for production usage. Specifically, proposed deployment objects will first be
evaluated and tested on the Futuregrid, in cooperation with their project team. The deployment
98

objects that pass Futuregrid testing must then be tested by the CMS for interoperability and
appropriateness in the defined environment. Finally, technologies deployed into XD must
undergo deployment and acceptance testing on XD production resources. To enable this final
phase, we will work with XD CMS and Service Providers to configure a suitable set of resources
(node partitions and/or reserved time slots) so that production user impact is minimized.
All technologies recommended for deployment by the Technology Insertion team will be
documented including fields that describe roles, rationale, responsibilities and usage context for
the deployment object, in addition to the specific deployment contexts, security requirements,
dependencies and tests and measures of correctness and performance used to validate (audit) the
deployments. XSEDE TIS will work closely with the XD management team to coordinate a
deployment strategy team with members including system administrators, operators, site user
consultants, and technical writers from each of the XD resource providers.
9.2.1.2 Usage, Satisfaction, and Impact Evaluation
This activity is the assessment of user productivity by evaluation of the user environment, usage
of code instrumentation and profiling tools, and collecting feedback on user satisfaction and
impact. These data will be evaluated to determine if the vision and goals of XD are being met and
to identify transformational science being accomplished due to technology insertion. Feedback
will be derived from multiple sources including the annual User Survey and user focus groups.
9.2.2 Progress Report
In this first TIS quarterly progress report we discuss project staffing, communications, and initial
progress. The reporting period is the three months prior to the award state date of July 1, so much
of the activity was in preparation for the start of the award.
9.2.3 Project Staffing
The TIS management team consists of John Towns as the project PI with Tim Cockerill (NCSA)
as project manager, Jay Boisseau (TACC) heading up the Technology Evaluation team, and
Sergiu Sanielevici (PSC) leading the Usage, Satisfaction, and Impact Evaluation team. The
XSEDE TIS team is staffed by members from NCSA, NICS, PSC, TACC, and the University of
Virginia. During this reporting period (prior to the July 1 award date), we identified staff
members at each site that will execute the project.
9.2.4 Communications
A meeting schedule was developed by the management and initial organizational meetings were
held. Full team meetings are initially weekly, with biweekly management meetings. Email lists
were set up for the different areas of the project. The team also set up a wiki space for internal
project communications.
9.2.5 Initial progress
Early deliverables include the XTED, a technology evaluation process, and user focus group
meetings. The XTED group is in the early design phase – gathering requirements for database
content, expected output, and user interface needs. The technology evaluation process group is in
the early stages of developing a procedure including technology selection, metrics, evaluation,
99

and reporting. The usage, satisfaction, and impact evaluation group did some preliminary data
mining of the TG ticket system data and user surveys to identify the first user focus groups.
9.3 Quality Assurance
The QA group worked to test and debug CTSS services that the group considered to be most
relevant for users. After reviewing CTSS usage and reliability last quarter, QA decided to focus
on GRAM and GridFTP testing and debugging. In particular, evaluating the upcoming GRAM5
release was a high priority. The QA group collaborated with the Gateway working group in
ongoing debugging discussion of GRAM5 experiences and participated in the TeraGrid user
portal forum thread related to GRAM5 debugging. QA also monitored the activities of others
involved in the GRAM5 release – mainly the Software and Gig-pack working groups. The QA
group gathered its own early GRAM5 test results using Inca and compared them to results from
other Gateway testing (the nanoHUB grid probes). While reviewing test errors the QA group
discussed how errors are logged and subsequently submitted requests for improved logging to
Globus developers. The GRAM5 test sources yielded different information – although the Inca
testing was mostly successful because it was fairly lightweight, it did indicate a problem with
lingering zombie processes that has since been addressed. The nanoHUB testing was successful
overall on Lonestar but was mostly failing on Queenbee. The Queenbee issues are still being
debugged and led to the administrator upgrading the GRAM5 software. Some of the differences
between the Inca and nanoHUB testing are that Inca tests are submitted to the local GRAM
service while nanoHUB submits them remotely, and nanoHUB doesn't take downtimes into
account as Inca does. The QA group is currently beginning to leverage some of the previous
Gateway GRAM scalability testing to run tests on a Ranger test node, where a GRAM5
installation was deployed for QA and is also now being used for Gateway testing. As the Ranger
node will soon become a production login node, the QA group is investigating the possibility of
using FutureGrid to continue testing.
In addition to the GRAM5 testing, the QA group created new tests for Condor-g matchmaking
and Karnak. Tests for Condor were also updated as well as the usage reporter for GRAM. The
group continued to monitor the CUE working group’s progress on its upcoming module release
and will test CUE deployments when available. Also, the QA group recommended that the
QBETS service be labeled as experimental in the user portal and documentation due to
intermittent failures experienced by users and a lack of available tests to detect failures.
Finally, the QA group continued work to create a sustainable process for collecting and analyzing
usage data in order to determine which services are most important to users. A member of the
group wrote a script to automate the reporting of CTSS processing accounting data and the group
created an informational wiki article to help explain the process of doing this:
http://www.teragridforum.org/mediawiki/index.phptitle=QA_Process_Accounting. Efforts to
further automate this process and to generate consistent and meaningful reports are ongoing.
9.4 XD – TAS: Technology Audit Services for TeraGrid
Center for Computational Research, University at Buffalo
Thomas R. Furlani, PI
Program Start Date: 7/1/2010
100

9.4.1 Technology Audit Services Program Goals
The Technology Audit Service (TAS) will both provide quality assurance and quality control for
TG (XD) to help maximize the positive impact of the infrastructure investment on science and
engineering research. Coupled with a strong user needs and usability analysis program, TAS will
be leveraged to help ensure that users’ needs are met, with particular emphasis on improving the
effectiveness of major users, bringing novice users up to speed rapidly and attracting nontraditional
users. By identifying user needs early on and providing the tools and information that
they need, TAS will allow them to efficiently and more effectively utilize TG resources. TAS will
also provide necessary inputs to other stakeholders, namely, NSF, resource providers (RP) and
the TG user community represented by the Science Advisory Board, interested in the optimal
usage of TG (XD) resources. TAS will also strive to provide quantitative and qualitative metrics
of performance rapidly to all of these stakeholders.
9.4.2 Progress Report
In this first TAS quarterly progress report we discuss project staffing, infrastructure put into place
to facilitate communication and documentation within the group, organizational meetings and
initial technical progress.
9.4.3 Project Staffing
• University at Buffalo: We are pleased to report that four professionals were hired and all
have started work on the program. The new hires include: Mr. Ryan J. Gentner, scientific
programmer who will work on the XDMoD (Extreme Digital Metrics on Demand) portal
development, Dr. Charng-Da Lu, computational scientist who will work on application
kernel development, Mr. Amin Ghadersohi, scientific programmer who will work on the
XDMoD portal development and XDMoD Data Warehouse, and Dr. Robert L. DeLeon
who will act as the TAS Program Manager. In addition to the new hires, existing CCR
staff members, who will have major programmatic responsibilities, have begun work on
the TAS program. These include Dr. Thomas R. Furlani, who will act as the PI and
oversee the entire TAS program, Dr. Matthew D. Jones who will oversee the technical
aspects of the program as well as the application kernel development; Mr. Steven M.
Gallo, who will lead development and implementation of the XDMoD portal and
backend infrastructure; and Mr. Andrew Bruno, developer of the UB Metrics on Demand
(UBMoD) portal, who will play a central role in development of the XDMoD portal.
• Indiana University: The Indiana University subcontract will be lead by Dr. Gregor von
Lazewski and Dr. Lizhe Wang. Initial efforts include design and implementation of the
custom report builder and support for the XDMoD design.
• University of Michigan: The University of Michigan subcontract effort will be lead by
Dr. Thomas Finholt. Dr. Finholt will be responsible for determining user assessment of
XDMoD and obtaining critical user feedback to optimize XDMoD to best meet user
needs.
101

9.4.4 Organizational Meetings and Events
Weekly meetings of the TAS Working Group were held. The TAS Working Group includes:
• University at Buffalo: Dr. Thomas R. Furlani, PI, Dr. Matthew D. Jones (Co-PI and
Technical Project Manager), Mr. Steven Gallo (lead software engineer), Mr. Andrew
Bruno (scientific programmer), Dr. Charng-Da Lu (computational scientist), Mr. Ryan
Gentner (scientific programmer), Mr. Amin Ghadersohi (scientific programmer), and Dr.
Robert L. DeLeon (Project Manager)
• Indiana University: Dr. Gregor von Lazewski and Dr. Lizhe Wang
• University of Michigan: Dr. Thomas Finholt.
9.4.5 Coordination with External Projects
Through Indiana University, we have conducted several initial outreach activities to identify
opportunities for collaboration and leveraging. As part of this we are working intensely together
with the NSF funded FutureGrid to leverage on their efforts on Single Sign-On and the
integration with InCommon. We will integrate these efforts with our own requirements and
collaboratively explore this path. In addition we have also contacted the NSF funded OGCE
project that was just recently funded in order to evaluate potential collaboration by leveraging
gadgets and their containers. These are ongoing activities and will continue for the foreseeable
future.
9.4.6 Technical Progress
• BASECAMP Project Management System and TAS Wiki: In order to facilitate
communication between the TAS team and provide a repository for project documents,
we are leveraging BASECAMP project management software. BASECAMP is a webbased
project management system that facilitates communication and collaboration to
allow a team to work effectively on multidisciplinary projects. It allows us to post files
for mutual use, send messages, track milestones, assign tasks, and communicate via
write-boards. It has already proven to be a very useful tool in supporting communication
between team members and keeping the TAS program moving forward. All project
personnel, including the sub-contract awardees, are using BASECAMP. An internal TAS
wiki has been set up in addition to BASECAMP to facilitate organization and sharing of
information.
• XDMoD User Interface: A flexible web-application shell for Extreme Digital Metrics
on Demand (XDMoD) interface is being finalized and will serve as the User Interface
(UI) for the XDMoD prototype. The shell will implement a session-based authentication
mechanism, role-based user authorization, extendable user profiles, and a user
administration interface. A content template system will allow customized content to be
delivered to users based on their role within XDMoD (e.g., Program Officer, Center
Director, PI, User, etc.). We have begun the initial user interface designs for the
prototype. We have finished a sitemap diagram of all the pages within the application and
three wireframe designs which will constitute the basic look and feel of the application.
These wireframes will help drive both the development and the UI design of the site and
provide a concrete specification document for future changes to application and user
interface. The wireframe diagrams include layouts for the general TeraGrid User as well
as the Program Officer. The user administration tools for local user accounts (i.e., user
accounts not associated with a TeraGrid account) and the basic functionality pertaining to
102

user management is complete. Local user creation includes the assignment of one or more
portal roles, including identification of the user’s primary role which is used in
determining the initial view of the data that will be presented to the user upon login.
Portal accounts are verified via a confirmation email containing an activation url that is
sent to the user. The portal administrator has the ability to manage role assignments, user
profile information, and enable/disable accounts. In the case of a forgotten password, a
facility is provided to reset a user’s password.
An administrative interface to XDMoD has been developed that will provide developers
with the necessary functionality for creating local users and mapping those users to usage
data for actual TeraGrid users. This will simplify development and testing.
Implementation of the portal wireframes are 80% complete with the remainder to be
completed within the next reporting period. Initial versions of identified charts and
visualizations generated with data from the TGCdb are currently being integrated into the
portal. Figure 1. below is an example of an XDMoD screen displaying summary
information for a particular TG User. We are working with a graphic designer at UB to
develop a look-and-feel for the portal. We have presented a prototype of the new
interface at TeraGrid 10, the annual TG users meeting in Pittsburgh (August 2-5, 2010).
Figure 1. Example XDMoD interface.
• XDMoD Data Warehouse: We have begun exploration of the data contained in the
TeraGrid Central Database (TGCdb). To date, queries have been implemented to explore
user, organization, allocation, resource, and job information. Design work has begun on
an abstraction layer for querying and extracting data from the TGCdb. In order to
provide rapid visualizations of TGCdb data for the XDMoD prototype, the Google Maps
and Chart APIs are being employed to develop a number of initial data views. These will
103

display aggregate information geographically with the ability to drill down to more
detailed information as well as aggregate information such as “Job Distribution by
Organization.” In our continuing exploration of the data contained in the TeraGrid
Central Database (TGCdb), queries are being developed to feed the initial visualizations
for the User and Program Officer roles. To date, functionality has been created for
converting query results to JSON data structures and using this data to generate charts,
images and maps using the various Google Chart APIs. We are addressing performance
issues when querying the larger TGCdb tables (i.e., acct.jobs) and are examining ways to
mitigate these issues and improve performance in a production system. Several sample
data visualizations have been created including reports on usage, CPU consumption vs.
Job Size, total jobs run on each resource, and allocation reports. A set of classes
encapsulating the functionality for dynamically generating the queries is being built. We
have begun to develop an in-house database to hold data ingested from various data
sources (initially TGCdb) and support the construction of visualizations via the portal.
Scripts have been written to ingest data for a targeted subset of TeraGrid users that will
be used for testing purposes. Various methods for drawing charts have been investigated
for maximum cross-browser compatibility. We have made contact with Rob Light (PSC)
regarding mirroring of the TGCdb. Rob will be assisting us in identifying the tools
required to mirror a copy of the TGCdb locally at CCR.
• TeraGrid Central Database (TGCdb) Findings/Limitations: In general, the TGCdb
records are very good, however in the course of constructing early versions of XDMoD
we discovered that there are some areas where data is apparently missing and other areas
where there is room for added or improved record storage. Our initial assessment of the
TGCdb finds some inconsistency in batch job usage reporting. Two notable examples are
that job exit status is not reported and not all computational resources report usage by
processor core as well as by node. Indeed, reporting is not always consistent for jobs on
an individual resource. We anticipate recommending a uniform taxonomy and more
extensive collection of resource utilization statistics.
• Monitoring Framework: The Inca User-Level Grid Monitoring tool has been installed
and configured locally at CCR and is currently deployed on our 2112 node compute
cluster. Efforts are underway to develop a number of example reporter modules that can
be used as a basis for deploying custom Application Kernels within Inca. A welldocumented
process will be provided for other team members. Exploration of the
structure of the Inca Depot is underway with the assistance of Shava Smallen (SDSC).
Queries to extract detailed reporter information from the Depot and display it are being
developed. A web-based exploration tool for the Inca data depot is being built. This will
allow the team to quickly explore the results of the custom application kernels that are
being developed.
• Application Kernel Development: Initial application kernel requirements have been
documented and the kernel classification scheme aligned with the “Collella dwarfs” that
describe application processing and data requirements. Deployment will leverage
existing Inca reporting infrastructure, and the first (dense matrix) kernel and reporter is
ready for testing. Two micro benchmarks: STREAM, which measures processor to
memory bandwidth, and BLAS (DGEMM), which measures single-node floating
pointing performance in the context of a dense matrix problem, are now running within
the local Inca testbed. We will coordinate and collaborate with FutureGrid staff and Inca
104

developers to see what they have already deployed on FutureGrid and leverage their
existing codebase for early and easily adopted application kernels. Inca’s current cluster
batch wrapper needs to be augmented to facilitate running kernels of varying size
(especially important for lightweight versions of the various application kernels), and we
will need to develop a meta-scheduling scheme to manage the flow of kernel jobs through
the various batch queuing systems in TG/XD (prevent accumulation of jobs, duplication,
etc.).
• Miscellaneous: TeraGrid accounts have been requested and received for two developers
(Charng-Da Lu and Steven Gallo) under the Network/Operations/Security (NOS)
allocation by Shava Smallen and Jeff Koerner. These accounts will be used for Inca
Monitoring and Application Kernel development and testing. Additionally, a subversion
repository has been set up for TAS software management.
• Hardware Ordered for XDMoD development: Hardware was purchased to host the
development and production environments for XDMoD including a mirror of the TGCdb
database. The hardware includes two Dell R710 servers and an Equilogic PS6010XV
high performance storage array with 9.6TB of raw storage.
• University of Indiana Subcontract: IU tasks include: (a) Identify the current available
TG monitoring and resource management service, for example, access interface (API,
command line, web service). (b) Assist in the development of the XDMoD (XD Metrics
on Demand) portal. (c) Collaboratively design and develop a Custom Report Builder that
can be incorporated in the XDMoD portal, allowing users to easily generate custom
reports in several formats, including PDF and excel. (d) Participate in the development
and deployment of application test kernels. (e) Interface with the Futuregrid and TIS
(Technology Insertion Service) projects.
• University of Michigan Subcontract: The University of Michigan subcontract was
finalized.
• The TAS Technical Advisory Committee: This committee will meet regularly to
provide feedback and guidance on the TAS program. The current committee members
are listed below. We plan on adding a representative from the XD resource providers
once the XD award has been established. The first meeting will tentatively be held in
November 2010.
TAS Technical Advisory Committee:
1. Dr. Jerzy Bernholc (Atomistic QM, Physics, NC State)
Email: bernholc@ncsu.edu; http://chips.ncsu.edu/~bernholc/
2. Dr. P.K. Yeung (CFD, Georgia Tech)
Email: pk.yeung@ae.gatech.edu; http://soliton.ae.gatech.edu/people/pyeung/index.html
105

3. Dr. Martin Berzins (Chair Computer Science at Utah)
Email: mb@cs.utah.edu; http://www.cs.utah.edu/personnel/mb/
4. Dr. Craig Stewart (Executive Director, Pervasive Technology Institute, Indiana),
Email: stewart@iu.edu; http://www.indiana.edu/~ovpit/bios/cstewart.html
5. Dr. Richard Brower (Lattice QCD, Boston University)
Email: brower@bu.edu;; http://physics.bu.edu/~brower
6. Dr. Sara Graves (Director, Information Technology and Systems Center; Univ of
Alabama, Huntsville), Email: agraves@itsc.uah.edu;; http://www.itsc.uah.edu
10 Education, Outreach, and Training; and External Relations
The TeraGrid collective efforts have engaged at least 4,145 people through at least 110 events as
shown in the table below. The number of participants were not recorded at all events. Details on
the events are included in section 10.6.
# Participants # Underrepresented
people
# events
Workshops 988 423 48
Tutorials 98 9
Async Tutorials 1,750 29
Forum and Tours 1,161 471 7
Conference
Presentations
86 27 9
Courses 122 21 10
Total 4,205 942 113
10.1 Highlights
Campus Champions
The following e-mail is from Jeff Pummill, a Campus Champion at the University of Arkansas.
A grad student just finished using Ranger for a computer science project involving the following:
Geoscience research has experienced an explosion of data in recent years. Streams of data from
satellites, unmanned aerial vehicles, airplanes, and people need to be accurately georeferenced (i.e.
placed into a global map). When performed using existing methods, georeferencing is a timeconsuming
process that requires significant human interaction. My research is part of a project
working towards using computer vision techniques and algorithms to quickly and autonomously
106

GIG
place and orient new data. I parallelized the Scale Invariant Feature Transform (SIFT) which was
originally ill-equipped to work with terabyte-sized data sets./
Our campus resources were in heavy use and also did not have the required core count for the
project.
The student sent me this note regarding the experience:
Jeff,
The help from teragrid was very prompt, the person understood my problem and quickly
helped me resolve it. TeraGrid CC is an important job. If you were not on campus, I may
never have even thought about getting on TeraGrid. Also, helping me get an account quickly
allowed me to run very large scale tests and publish those results in time for a deadline.
Having lots of SUs is also good. Overall, this has been a good experience, keep up the good
work.
All my scripts used the wall time of 1:30:00. The spreadsheet has the queue times.
I'd consider this yet another success story for the TeraGrid, the Campus Champions, AND the
stellar staff at TACC ;-)
TeraGrid helped support 3 workshops for faculty to assist them with integrating computational
science tools, resources and methods into their courses. The courses included Introduction to
Computational Thinking, and Parallel and Cluster Computing. These events were jointly
supported by Blue Waters and the National Computational Science Institute
(www.computationalscience.org).
Quote from a participant:
LONI
From: "Heeler,Phil"
To: Henry Neeman
Subject: RE: NCSI Intmd Parallel Programming Workshop
Henry,
Thank you for your diligent efforts in conducting a challenging
workshop. The material was definitely harder but at the same time
very interesting. I did learn a lot. Your team and you are to be
congratulated on doing a fine job. Bob Panoff would be proud.
Hopefully, I will see you at SC10 and maybe at the OU
symposium,if I can twizle my schedule!
Thank you again,
Phil Heeler
LONI/LSU offered 8 tutorials, each 2-4 hours long, on the LSU campus and via Access Grid. A
3-day workshop was also sponsored on campus for advanced users. All tutorials and the
workshop involved hands-on learning opportunities covering development tools in HPC
environments. These included MPI, OpenMP, and hybrid programming methods plus debugging
and profiling for optimization. Other topics included functioning in a UNIX environment, account
and allocation management, and job submission and management.
NCAR
107

NCAR’s 2010 Summer Internships in Parallel Computational Science (SIParCS) program got
under way in this quarter, and it includes several TeraGrid-related projects. This year 20 students
are participating in the program from 17 different universities from across the country. Some
have signed up for the Virtual School of Computational Science and Engineering (VSCSE)
petascale applications development, to be held online July 6-9, 2010.
NCSA
The following is an e-mail from a user about the CI-Tutor system of tutorials provided by NCSA.
NICS
From: "Srirangam V Addepalli"
To: training@ncsa.uiuc.edu
Sent: Friday, May 28, 2010 10:41:03 AM
Subject: NCSA Cyberinfrastructure Tutor
Hello,
I am currently in the process of setting up a training page for our users and came across your
website. This website has ranked the best and comprises of the most comprehensive collection of
training materials and tracking training progress. Would it ok if we redirect our users for training
to your website.
Sincerely
Rangam
Srirangam Addepalli; Programmer Analyst – III; High Performance Computing Center
Texas Tech University
NICS efforts in EOT were highlighted by two large workshops, the NICS/OLCF XT Hex-Core
Workshop May 10-12, and the Computational Thinking for Educators workshop June 12-14. In
addition to these large workshops, six college classes from Colorado to Massachusetts to South
Carolina were given access and direct support during the Spring 2010 semester. Considerable
work was also completed on a CI-Tutor course in Parallel I/O, and a new course for the catalog
was started.
PSC
PSC led the planning for the Student Program for the TeraGrid’10 Conference. This included
securing funding from NSF for students to participate in the Conference as well as organizing
sessions to engage the students.
Purdue
Greg Howes, University of Iowa professor of physics and astronomy, contacted TeraGrid looking
for a system on which his students participating in the Iowa High Performance Computing
Summer School course could practice parallel computing. Purdue RP stepped up as TeraGrid
resource partner to accommodate Professor Howe’s needs by providing access to Moffett,
Purdue’s SiCortex supercomputer. Graduate students from 14 departments in fields as diverse as
biochemistry and physics, economics and astronomy, geography and electrical and computer
engineering participated in the two-day, hands-on session of running and debugging parallel code.
The over 4,500 processors and extraordinarily fast custom interconnect fabric of the SiCortex
system proved to be a perfect fit for what Professor Howes was trying to accomplish. The Iowa
graduate students ran 1,420 jobs and used 16,721 CPU hours.
TACC
TACC’s education and outreach programs include the scientific computing courses (2 during this
reporting period) and tours of TACC’s facility at the J.J. Pickle Research Campus (about 10 miles
108

north of UT Austin’s main campus) and the ACES Visualization Laboratory at the UT Austin
main campus. The Austin Forum continues to attract a broad audience from the university, the
city, and the wide range of technology industries around Austin. TACC’s Scientific Computing
courses were all fully subscribed this semester, with 35 students in each of the courses. TACC
continues a dialogue with the community using Web 2.0 technologies like Facebook and Twitter.
TACC continues to experiment with webcasting training courses, and will expand the course
availability in the coming months.
Describe any high-impact EOT activities that resulted from use of the TeraGrid resources, and
TeraGrid EOT activities.
NICS
Training. The NICS/OLCF XT Hex-Core workshop (May 10-12, 2010) was presented at Oak
Ridge National Laboratory and served 65 users of both the NICS Kraken machine and the similar
DOE machine at ORNL. Daniel Lucio and Lonnie Crosby of NICS delivered training material,
in a program that also included DOE presenters and experts from vendors to give insights to their
hardware and tools. The workshop was intermediate to advanced in scope, and NICS staff from
its Scientific Computing and User Support groups provided consulting to the attendees during
hands-on laboratory sessions.
Education. NICS planned and co-hosted a 2.5-day workshop on Computational Thinking for
Educators, held at the Oak Ridge Associated Universities facility in the town of Oak Ridge, TN.
Twenty educators took part in the workshop, ranging from lower middle school teachers, through
middle and high school math & science teachers, up through two faculty from Tuskegee
University. Attendees hailed from Alabama, Georgia, North Carolina, and Tennessee. Robert
Panoff of Shodor provided one day of excellent instruction on their various tools for educators,
three Master Teachers from Tennessee teamed to provide content on the 2 nd day, while Jim
Ferguson of NICS taught basic programming techniques through the use of a programmable car
which the attendees got to take back to their schools with them. Hands-on activities designed for
students (and their teachers’ lessons) were constant throughout the workshop. Initial surveys
after the course were overwhelming positive, and a follow-up survey is planned into the next
school semester to see how the instructors are making use of what they learned.
Education/Outreach/Training. Six college classes were supported by NICS during the Spring
semester of 2010. Classes at Colorado (Prof Jessup), Tufts (Prof Grinberg), Wofford (Prof
Shiflet), Brown (Prof Grinberg), and two at Tennessee (Professors Banks and Dongarra) were
given access to Kraken, in-person or online (WebX) training, and follow-up user support. There
were approximately 105 students who made use of NICS resources, along with their professors
and teaching assistants.
Training. Mikhail Sekachev and Jim Ferguson are collaborating with other NICS staff and with
Sandie Kappes of NCSA to produce a CI-Tutor training module covering Parallel I/O issues on
the latest systems. Drawing on materials and expertise produced locally at NICS and at DOE’s
OLCF, this module is being shaped to give students a solid overview of the issues at hand
regarding parallel I/O, as well as some specific tools to use on Cray XT and other hardware
platforms. Final editing and polishing is underway, and the finished product will be put up
3Q2010 at the CI-Tutor site.
Training. Jim Ferguson has been heavily involved with locally organizing and recruiting
speakers from the upcoming multi-point summer school in Petascale Computing, being put on by
the Virtual School for Computational Science and Engineering. Ferguson consulted with Scott
Lathrop on the overall agenda for the summer school, and recruited four speaker-trainers at NICS
109

and OLCF (Lonnie Crosby, Rebecca Hartman-Baker, Sean Ahern, Scott Simmerman) to help fill
out that agenda with top people. The workshop is scheduled for July 6-9, 2010.
**Excerpt from a report written by Professor Angela Shiflet regarding NICS, her Spring class,
and the visit of Bruce Loftis and Jim Ferguson to Wofford to give a seminar.**
“The class and I had wonderful support NCSA, NICS, and the TeraGrid. The NCSA tutorial
Introduction to MPI was invaluable to me for lecture material and examples. The TeraGrid staff
promptly answered our questions at all hours of the day and night. One of the TeraGrid
computers we used was NICS' Kraken. NICS staff presented an hour-and-a-half online tutorial
on Kraken designed for the Wofford class. Two scientists from NICS, Jim Ferguson and Bruce
Loftis, volunteered to present a talk at Wofford on “Scientific Computing at a National
Supercomputing Center.” Approximately thirty students and faculty members attended the talk,
including computational science students from the Data and Visualization class (COSC 370).
The talk was the only exposure the COSC 370 students had to HPC during the semester. Some of
their comments about the excellent talk follow:
"I thought the talk by Bruce Loftis and Jim Fergus was extremely interesting. I had never
heard of UT having supercomputers, much less the Kraken. I also did not know that we
had computers capable of operating 100,000 processors at once, or that anyone had
achieved a petaflop yet….The HIV model particularly interested me because we have spent
an entire lecture on that in Immunology, including how it infects T cells, etc. I had heard of
parallel processing before, but never gave much thought to how it worked. The discussion
on how all the processors work at once was the first education I have had on the topic, and I
found it interesting to finally learn a little bit about how multiple 'brains' of the computer
can simultaneously find solutions to the same problem."
"Before this talk I knew next to nothing about the capabilities of such machines, but when I
left I was yearning to learn more. The possibilities for scientific discovery, in most every
field, seem endless with this immense computing power. Impressive images of a supernova
core collapse, detailed earthquake simulations, and immature virion would not be
possible without such technology. Personally, it seems as though we have just scratched the
surface of this quickly advancing field, and I cannot wait to see where the future takes us."
After the talk, members of the HPC class went to lunch with the NICS scientists. The discussions
lead to a "dream internship" at NICS for one of the class members, Glenn Hope.”
SDSC
Planning Underway for Grand Challenges in Data Intensive Sciences
Planning is well underway for a conference at UCSD on October 26-29, 2010. The conference on
“Grand Challenges in Data-Intensive Sciences” will include three days of presentations by leaders
in data-intensive sciences in a wide range of fields. The fourth day of the conference will be a
hands-on workshop using Dash for attendees who wish to try their own code on the machine,
working closely with the workshop presenters in small tutorial teams. SDSC’s TeraGrid EOT,
User Services, and AUS staff are all involved in this upcoming event.
The purpose of the conference is to:
• Identify "Grand Challenges" in data-intensive science across a broad range of topics
• Identify applications and disciplines that will benefit from Gordon's unique architecture
and capabilities
110

• Invite potential users of Gordon to speak and participate
• Make leaders in data-intensive science aware of what SDSC is doing in this space
• Raise awareness among disciplines poorly served by current HPC offerings
• Better understand Gordon's niche in the data-intensive cosmos and potential usage modes
Topic areas are to include astronomy, biology, computer science, earth sciences, engineering,
economics, medicine, neuroscience, tools and technologies and arts, humanities and social
sciences. Each topic area will include a senior plenary speaker laying out the scientific frontiers
and challenges, and an invited speaker who will discuss a current data-intensive project that
ideally could use Gordon. One panel is scheduled for each afternoon on a cross-cutting topic in
data-intensive research (e.g., visualization, data mining, data management).
10.2 Training, Education and Outreach
10.2.1 RP Operations: Training
NCSA
RP-funded training activities included hands-on workshops, training sessions via the Access Grid,
support of online and in-person tutorials. TeraGrid RP staff participate in both local site-specific
training as well as TG-wide training.
PSC
PSC’s John Urbanic and Tom Maiden gave a presentation at the 5th Annual High Performance
Computing Day at Lehigh on April 23. The 3½-hour presentation included an overview of the
multicore computing hardware landscape and an overview of software computing techniques
including examples of MPI, OpenMP, and GPGPU languages like CUDA and Open CL. There
were 20 attendees. For more information, see:
http://www.lehigh.edu/computing/hpc/hpcday/2010/
http://insidehpc.com/2010/05/03/lehigh-hosts-fifth-hpc-day/.
PSC staff conducted the quarterly TeraGrid web conference New User Training Session on
Friday, April 30, using the TeraGrid User Services ReadyTalk line. There were 22 participants
for the 80-minute session. Since audio was not saved for this session, the website notes that audio
and corresponding slides are available for February’s session, and has a link to that session. PSC
people involved were Marcela Madrid – presenter, Phil Blood - support for presenter, Tom
Maiden - troubleshooter for participants and Laura McGinnis - question router for participants.
PSC staff members created a workstation cluster installation process to support an introductory
High Performance Computing workshop hosted at Chatham University, a predominantly female
undergraduate school, on May 19-20, 2010. The goal was to use existing computers in a Chatham
student workstation lab to create a cluster. Since these systems were in active use by Chatham
students, the workshop cluster needed to reside next to the existing software on the workstations
in a non-intrusive way.
The PSC people explored several alternatives and developed a virtual machine (VM) model that
could be run from a memory stick on a head node and create a cluster on all the other available
workstations from the head node. They added in the capability of “netbooting” the nodes since
system administrators would have full control over the software load. (“Netboot” is a machine
build and replication service that PSC developed many years ago and is in use for most of PSC’s
in-house clusters.) With this netboot process, they created a cluster with PBS and OpenMP that is
capable of dynamic scaling of the number of nodes. These techniques reduced the time needed to
111

uild a simple cluster down to a matter of a few hours. The cluster was used for a two-day MPI
workshop at Chatham. The working cluster remains at Chatham and is being used as a learning
resource. Dr. Larry Viehland, the Chatham faculty member who hosted the workshop, is running
production codes on the cluster.
Six people from Chatham and one from Pitt attended the workshop. The workshop covered
Introduction to PSC, Introduction to Chatham Cluster, Introduction to Parallel Computing, MPI
Basics, Scalable Coding (Laplace Solver Case Study), OpenMP, and plenty of hands-on
exercises.
Mahin Mahmoodi of PSC created an online tutorial TAU: Parallel Performance Profiling and
Tracing Tool that was made available to the public on the TeraGrid website during the last week
of June. TAU is an open-source and advanced performance tool that can be used in profiling
scientific applications. The tutorial provides TAU usage information for Kraken and Ranger. It
introduces commonly used features, provides quick references, and assists users to successfully
start using TAU on the systems. The tutorial also provides a guide for navigating visualization
features of TAU which are essential in extracting and analyzing performance data. The material is
available at https://www.teragrid.org/web/user-support/tau.
Purdue
Purdue RP created a set of step-by-step online video tutorials on how to use the Purdue CCSM
modeling portal to create, configure and run CCSM simulations. It also covers advanced topics
such as how to manage jobs, how to post process model output, how to configure customized
input data using real world simulation examples.
SDSC
RP-funded training activities included hands-on workshops, training sessions via the Access Grid,
support of online and in-person tutorials. TeraGrid RP staff participate in both local site-specific
training as well as TG-wide training.
TACC
Staff members at the Center for Advanced Computing at Cornell University continue to maintain,
support, and develop new modules for the Ranger Virtual Workshop. Available through both the
TeraGrid User Portal and the TACC User Portal, the workshop provides users access to fifteen
training modules with six new modules currently under development.
Available Modules
Parallel Programming Concepts and High-Performance Computing
Ranger Environment
Message Passing Interface (MPI)
MPI Point-to-Point Communications
MPI Collective Communications
MPI One-Sided Communication
OpenMP
Hybrid Programming with OpenMP and MPI
Profiling and Debugging
Optimization and Scalability
Computational Steering
Large Data Visualization
ParaView
VisIt
Using Databases
112

Modules Under Development
Advanced MPI
Data Analysis with Python on Ranger
Distributed Debugging Tool (DDT)
Multi-core
Multi-node Map Reduce
Profiling with mpiP
10.2.2 RP Operations: Education
NCSA
TG RP Education activities included activities to communicate the relationships between science,
technology, and society, and the role that scientific computing plays. TG RP staff reached out to
the K-20 community in an effort to encourage student interest in math and science during the
education process, to members of diverse, underrepresented groups in order to broaden
participation in high performance computing, and to the general public in order to promote
awareness of role of high performance computing to advance scientific discovery. Some RP sites
contribute to the scientific computing curriculum at their institution, creating and maintaining
"course packages" that are available to higher education faculty to produce their own courses.
ORNL
Jim Rome at the ORNL RP established a working relationship with the Tennessee Governor’s
Academy for Mathematics and Science (TGA) to teach a weeklong course on programming (in
Java). The TGA (http://tga.tennessee.edu/) is a residential secondary school focusing on advanced
science and engineering located in Knoxville with students drawn from across the state of
Tennessee in a highly selective process. Under the direction of UT’s nationally acclaimed College
of Education, Health and Human Sciences, the Academy provides an integrated curriculum that
embeds key elements of the social sciences, language arts, and the humanities in rich and relevant
problem-based modules. With a strong physics and calculus foundation, the Academy students
are also paired with a scientist at the Oak Ridge National Lab to enhance the experiential nature
of his/her science and math instruction. In discussions with the academy, it became clear that the
advanced curriculum of the academy did not include enough hands-on programming instruction
or experience. Consequently, Dr. Rome, with support from TG-EOT and the ORNL-RP, will be
coordinating a week-long introduction to programming in Java course to be taught before the first
week of school in August. In Q2 of 2010, a considerable amount of planning and preparation
occurred in order to be ready for the course in August.
Henri Monti, a graduate student at Virginia Tech, visited Oak Ridge and collaborated with staff at
the ORNL RP and other areas of ORNL. He conducted research on developing methods to
checkpoint HPC codes to memory instead of disk. For Petascale (and later Exascale) HPC
platforms, the time to dump memory to disk for a checkpoint can be long, minutes even. This
means that the act of checkpointing is very expensive in terms of SU’s. An active area of research
is checkpointing to unused memory on other cores either as a way to develop more resilient
modes of computing or as a way to offload the I/O associated with checkpointing and thus hide
the latency usually associated with saving state. The results were promising. The plan is to submit
one of more papers from this work for publication.
PSC
Cheryl Begandy, Pallavi Ishwad and Robin Flaus of PSC met with the technology coordinators of
the Pennsylvania Cyber Charter School; see http://www.pacyber.org/. They reviewed PSC’s slate
of K-12 programs and there was a good deal of interest. PSC can clearly provide curriculum
113

content to the Cyber Charter School’s students, who all attend over the internet, and also help
their instructors find additional content on the web. Another meeting is planned which will also
include representatives from the National Network of Digital Schools.
PSC has selected four high school students to work as non-paid interns with NRBSC staff on a
bioinformatics research project. The internship program runs from June 14 to July 23, 2010. In
addition to learning how to conduct a research project in bioinformatics, the interns will develop
career relationships and receive career guidance from the national and international participants
(both professors and university students) of PSC’s MARC Summer Institute in Bioinformatics.
There are two students from two of the three schools that piloted BEST (Better Educators of
Science for Tomorrow), which is a high school bioinformatics course that was developed over the
last three years as a combined effort between PSC scientists and multidisciplinary high school
teachers. Funding support came from the Buhl Foundation. BEST is taught from the overall
viewpoint of training practicing biologists who will use bioinformatics techniques and tools in
their biology careers. The internships will provide experience in bioinformatics from another
viewpoint, that of a computer scientist applying the problem solving and programming skills of
computer science to biological sequence data. The program began with two weeks of intense
instruction in the Python programming language. The interns are now working collaboratively on
applying Python, with guidance from PSC senior scientists, to biological sequence data. There is
an opportunity for the interns to have scientific publications.
Alex Arrico, a Physics undergraduate at Duquesne University, asked PSC’s Yang Wang to
mentor his application for one of the Pennsylvania Space Grant Consortium Research
Scholarships (http://pa.spacegrant.org/psgc-fellowship-program) which are awarded by a program
administered by the Pitt Department of Physics and Astronomy to support undergraduate
research. The goal of this NASA Space Grant program is to encourage students to enter STEM
fields. The proposal they developed, “Explore the use of CUDA GPU for High Performance 3-D
FFT Applications”, was successful, and Mr. Arrico has become a PSC intern under Dr. Wang’s
mentorship. The internship runs from Monday, May 24 to Friday, July 30. They have received a
Startup grant on NCSA Lincoln and will also seek to use the GPGPU cluster at the Center for
Molecular and Materials Simulations at Pitt. The project is to test Fast Fourier Transform
packages, targeted to Astrophysics applications, and compare the performances between CPU and
GPGPU. Mr. Arrico has finished the code development for the tests and is now focusing on
getting the timing data.
PSC has two additional interns this summer, Matt Lambert, a sophomore at the College of
Wooster in Ohio, and Amy Pfeifer, an incoming freshman from Carnegie Mellon, who are
working on the HPC University Computational Question of the Week, with a team of developers
at Shodor (in Raleigh, NC). The short-term goal is to get the project back up to date and verify
the problem sets and solutions that have already been entered, in time for the TeraGrid ’10
Student Program. Longer term goals include enhancing the problem sets, adding more detailed
solutions, including discussion forums for each problem, and possibly adding computational
resources (possibly a virtual machine environment) that could be used as a platform for solving
the problems.
Purdue
Purdue RP staff collaborated with faculty at Marquette University about utilizing the Wispy cloud
resource in fall 2010 coursework about cloud computing. We expect students to utilize Wispy and
Hadoop/Mapreduce.
Additionally, Purdue has continued to integrate student system administrators into RP operations,
providing real-life, hands-on experience with operating a large scale computing infrastructure.
114

Purdue RP has recruited two undergraduate students, Talia Ho and Evan Albersmeyer, as summer
interns to work with the TeraGrid team since May. Talia is working with RP user support expert
to create step-by-step tutorials on frequently asked topics. She has learned to access and use
various TeraGrid resources and tools, such as the TG user portal, gsissh tools, as well as the user
issues, and how to make effective help material. Evan is developing a utility tool to help manage
the virtual machines deployed in Purdue’s Condor pool and report usage data to the users of the
desktop computers that contribute cycles to the pool. Both students have found many benefits in
learning about TeraGrid, high end computing and scientific applications in general, learning to
work with end users, learning to present advanced technology to beginners, and working with a
collaborative team.
SDSC
TG RP Education activities included activities to communicate the relationships between science,
technology, and society, and the role that scientific computing plays. TG RP staff reached out to
the K-20 community in an effort to encourage student interest in math and science during the
education process, to members of diverse, underrepresented groups in order to broaden
participation in high performance computing, and to the general public in order to promote
awareness of role of high performance computing to advance scientific discovery. Some RP sites
contribute to the scientific computing curriculum at their institution, creating and maintaining
"course packages" that are available to higher education faculty to produce their own courses.
TACC
TACC Scientific Computing Curriculum Courses: Since the fall of 2006, over 400 students have
enrolled in our courses. A total of 68 undergraduate and graduate students enrolled in the two
spring 2010 courses, Introduction to Scientific Programming (focused on FORTRAN and C) and
Parallel Computing for Scientists & Engineers. Four TACC scientists are teaching these courses.
In each course, enrollment reached the maximum. Engineering students continue to hold the
greatest percentage of students (40%), compared to the natural sciences, computer sciences,
geosciences, and liberal arts. The majority of students are involved in research. Introduction to
Scientific Programming continues to attract the greatest percentage of undergraduates per course.
TACC is evaluating the quality and effectiveness of the courses with pre- and post-course surveys
based on the content, learning objectives, and the computational science competencies from the
Ralph Regula School of Computational Science. The Spring 2010 course data indicates
statistically significant gains in students’ knowledge and skills. There are a few interesting
findings: Introduction to Scientific Programming attracts mostly undergraduates, while graduates
are the overwhelming majority in the Parallel Computing course. The majority of students in the
programming class were from engineering and physics, but two liberal arts majors enrolled in the
class as well. Student achievement in the learning objectives for the parallel computing course,
based on the competencies developed by the Ralph Regula School of Computational Science,
increased from little or no experience to proficient/very experienced over the course of the
semester. An encouraging highlight is 75% of students in the scientific programming course
indicated that they are applying their new programming capability in other courses.
TACC Scientific Computing Curriculum Package: Production of the Scientific Computing
Curriculum Package continues. Instructors are now reviewing the revised course materials, with
an eye toward distribution of materials at SC’10. . The presentation slides contain the bulk of the
courses’ content that students use on a regular basis, thus the content accuracy and quality of the
slides are critical to maximize impact and sustainability. Students have commented to our
instructors about the high quality of the slides. The package content extends beyond the course
slides to ensure that faculty at other institutions may readily adopt or integrate the materials for
115

creating their own courses or enriching existing courses. Additional content includes assignments,
tests, projects, references, and instruction guides.
10.2.3 RP Operations: Outreach
NCSA
NCSA provided tours of its facilities and an overview of how the center supports science and
engineering discovery and innovation to more than 2,000 people. More than 1,000 of these
visitors attended our June 17, 2010, Community Day event at the new National Petascale
Computing Facility, the building that will house the Blue Waters sustained-petaflops
supercomputer and other advanced infrastructure.
ORNL
The ORNL-RP continued coordination and outreach to the DataONE DataNet project continued
in many areas including attending the DataONE core cyberinfrastructure Team meeting in
Albuquerque, NM. The suggestion was made that DataONE should seek closer partnership with
TeraGrid. TeraGrid and DataONE have complementary services for data with TeraGrid
concentrating on low level protocols and infrastructure while DataONE provides higher-level
software infrastructures. Beyond broad collaboration, specific actions in the DataONE working
group for scientific exploration, visualization, and analysis include deploying new, larger, data
intensive studies correlating and predicting bird migration patterns as a function of observed
environmental, ecological, and climatic data. The TeraGrid is now a prime target computational
platform for these studies.
PSC
Marcela Madrid of PSC gave an Introduction to TeraGrid Resources talk at the fourth PA-OH-
WV Simulators Meeting at CMU, on June 15, 2010. She covered the resources and services
offered by the TeraGrid, and how to apply for an allocation through POPS. There were 48
attendees. Ten people signed up for more information about the TeraGrid, one from Case Western
Reserve University, one from University of Pennsylvania, three from West Virginia University,
three from Carnegie Mellon University and two from University of Pittsburgh. For more
information on the meeting see http://ntpl.me.cmu.edu/ntplwiki/index.php/PA-OH-
WV_Simulators_Meeting.
PSC’s Cheryl Begandy and Lynn Layman presented an overview of opportunities that PSC and
the SuperComputing Science Consortium, (SC) 2 provide for companies at an outreach event
organized by the Erie Technology Incubator and eBizITPA. (SC) 2 is a partnership of the National
Energy Technology Laboratory, the Pittsburgh Supercomputing Center, Carnegie Mellon
University, the University of Pittsburgh, West Virginia University, the West Virginia Office of
Technology, Duquesne University, and Waynesburg University which improves the ability of the
partners to advance energy and environment technologies through the application of high
performance computing and communications. The Erie Technology Incubator
(http://erietech.org/) “provides the resources and mentoring, the nurturing and business acumen to
help technology-based start-up companies take that next step both in business development or
funding.” The mission of eBizITPA (http://www.ebizitpa.org/) is “to assist Pennsylvania
businesses and organizations with understanding, using and developing information technologies
to grow the economy by providing resources to enable company formation and growth,
developing technology talent and expertise, and facilitating collaborative efforts to stimulate
innovation.”
PSC’s Cheryl Begandy and Dave Moses met with Justin Driscoll, who is the Director of STEM
activities for the Pittsburgh Technology Council and also the Greater Oakland KIZ (Keystone
Improvement Zone) coordinator. As a result of this meeting, Mr. Driscoll is adding another
116

eak-out session to the regional STEM Summit (August 26) to feature PSC K-12 programs along
with two other developers of curriculum material. Mr. Driscoll also encouraged PSC to both
exhibit and present at the annual TRETC (Three Rivers Educational Technology Conference) in
November. He is involved in organizing both of these events.
PSC co-organized the Symposium on Applications of Graphics Processing Units (GPUs) in
Chemistry and Materials Science (June 28-30, University of Pittsburgh;
http://www.sam.pitt.edu/education/gpu2010.php). Nick Nystrom delivered the welcome on the
morning of June 30, in which he discussed PSC, TeraGrid and allocations, Anton (a specialized
supercomputer for biomolecular simulation designed by D. E. Shaw Research that is available for
non-proprietary research at PSC; see §8.6.7), and valuable applications of large shared memory.
The symposium was sponsored by QChem, Penguin Computing, and PETTT, and it was webcast
to DoD researchers.
Purdue
Purdue RP staff is actively contributing to the TeraGrid Campus Champion program. Kay Hunt
and Scott Lathrop presented a position paper sharing the campus champion program perspective
with the NSF sponsored Campus Bridging workshop in Indianapolis, April 2010. In her unique
position of being both an RP user support specialist and the Purdue campus champion, Kim
Dillman helps support many requests from campus champions at a number of campuses. She has
also created tutorials to provide users with detailed learning material in addressing the frequently
asked topics. She is also leading the effort in evaluating technologies for a campus champion’s
portal, She was the Campus Champion’s representative on the Roaming RAT group and
contributed to the construction of a user survey, result analysis, the proposed for meeting the
needs identified in the survey.
SDSC
TG RP Outreach activities included activities to communicate the relationships between science,
technology, and society, and the role that scientific computing plays. TG RP staff also reach out
to the scientific and professional communities in an effort to encourage interest in the use of
TeraGrid resources and services. Many RP sites contribute to broadening participation in the
scientific computing community that should be covered in this section.
SDSC’s Education Director has served as Co-Chair of the Education Program for the TG’10
conference in Pittsburgh, Pa., and has been responsible for recruiting presenters, reviewers, and
overseeing the review process. SDSC’s Education Program Manager has been responsible for
coordinating the student volunteer program logistics.
SDSC’s Education Programmer/Instructor has been responsible for further development of the
Campus Champions web portal, responding to feedback from Campus Champions and the
TeraGrid CC team.
SDSC’s Education Programmer/Instructor has been responsible for initial development and
testing of the MSI-CIEC web portal, responding to feedback from the MSI-CIEC community.
SDSC’s Education Programmer/Instructor has been responsible for administering and
maintaining the TeacherTECH Community Portal and the Discover Data Education Portal in
support of online instruction.
Dash became a TeraGrid resource on April 1. At SDSC, the Dash/Gordon User Support team has
worked closely with a number of early users from a variety of disciplines. The first table below
summarizes highlights from this early user outreach. The outcomes are current for this quarter,
though the outreach has been launched, in some cases, prior to the current quarter.
117

The second table (below) summarizes the currently active Dash allocations, which reflect
outreach to new communities and research groups that have data-intensive computing challenges
that might be amenable to faster time to solution using the new Dash memory-rich architecture.
TACC
TACC hosted three of its monthly Austin Forum events with invited speakers from areas of
interest focused on science and technology. The goal of “The Austin Forum on Science,
Technology & Society” is to engage and educate the Austin community about the numerous ways
in which science and technology enhance the quality of their everyday life as well as the health,
prosperity, and security of the nation. Two hours are devoted to a networking reception,
presentation, and Q&A discussion among the speaker and guests, which reinforces the broader
impact and intellectual merit of the program. Speakers and topics within this reporting period
focused on:
• Tony Befi and Brad McCredie, IBM: “IBM and Austin: Partners is a Smarter Planet.”
Two of IBM Austin's leaders shared how IBM Austin is developing smarter systems for
meaningful change and described the company's view of how technology is addressing
major infrastructure issues. 89 attendees.
• Dr. Jessica Hanover, Dr. Steven Leslie, Greg W. Hartman, Joe Skraba: “BioTech:
The Next Big Thing.” Home to 3,300 technology companies, national recognized medical
facilities, and a top-tier research university, Austin is emerging as a best-in-class life
sciences community. The convergence of biology and technology allows Austin to
capitalize on its global reputation as a technology hub. Collaboration between
universities, hospitals, businesses, and others include curriculum planning, research,
medical training, and support of life science initiatives. This talk highlighted the benefits
and challenges of Austin as a community for biotech. 150 attendees.
• Brewster McCracken, “Creating a Dynamic Clean Energy Economy.” Brewster is
executive director of Pecan Street Project Inc., a clean energy/smart grid R&D
organization headquartered at The University of Texas at Austin. Pecan Street is
developing and implementing smart grid and clean energy technologies and business
models. This talk stressed why Austin is uniquely qualified to lead in the 21 st century
energy industry. 101 attendees.
TACC openly advertises tours of our facilities to the Austin and central Texas area. Tour groups
include visitors in K-12 (mainly high school), higher education (training class participants,
university students, etc), industry, government, and the general public. A few events and impacts
stand out with regard to TACC’s outreach efforts: 1) TACC hosted its first “Explore Creativity in
Digital Space” event which showcased digital media, photography and video works in TACC's
Visualization Laboratory; and 2) TACC was selected by the UT Central Development office to
participate in Professional Advisors Day, a unique event that hosts professional advisors
(attorneys, CPAs, and others who serve as financial advisors and estate planners). An overview of
TeraGrid and its contributions to STEM research were given at each event.
10.3 External Relations
External Relations Highlights
The TeraGrid External Relations (ER) working group issued 29 stories and press releases, and
provided media relations, event planning, and other related activities in Q210. Both TACC and
LONI issued stories about how TeraGrid resources were used to mitigate the oil spill in the gulf,
which began in late April and continued throughout the quarter. Efforts to work with NSF-OLPA
118

to coordinate press for all related NSF-funded research were taken. For all TeraGrid-related news,
see www.teragrid.org/news.
This quarter, there were 172 news “events” where TeraGrid was mentioned in a unique way on
the World Wide Web, as was determined by Google Alerts for the term “TeraGrid.” This number
is steadily increasing each quarter. There were 330 events in all of 2009. There is more ‘buzz’
about TeraGrid among bloggers and professional organizations that are now choosing to feature
our news—especially internationally. TeraGrid is achieving broader exposure as we have made
an increased effort to release news items via our outreach list to related professional
organizations. TeraGrid’s FaceBook page has become more utilized and the number of ‘friends’
has increased from 64 in Q209 to 109 in Q210. See:
http://www.facebook.com/pages/TeraGrid/106634599376129ref=search#!/group.phpgid=2562
777259&ref=ts
Plan Year Six (PY6) officially began in April, 2010. Within the Integrated Project Plan for the
plan year, Leake had identified the need for additional help with science writing, and graphic
design. To support this need, .20 FTE of additional effort had been funded for graphic design
services by Shandra Williams (PSC) and .30 FTE allocated for science writing by Jan Zverina
(SDSC). Each began to provide additional assistance this quarter—Williams helped with web
needs, documentation, data facilitation, and design assistance with TG’10 efforts. Zverina began
to customize stories so that they were less RP-specific, and more relevant to TeraGrid in general.
Zverina also took a leading role with the TG’10 communication committee efforts.
Science Highlights
Sixteen stories were identified for the 2010 Science Highlights book. The book will be printed
and delivered in time for the November Supercomputing conference. Once again, Warren
Froelich (SDSC) will serve as editor, and Carlton Bruett has been contracted to provide design
services. Most of the ER team provides stories and assist with editorial efforts for this
publication, as well as the 2010 Education, Outreach, and Training book, edited by EOT Working
Group Member Ange Mason (SDSC).
Discussion ensued about how to produce the 2011 Science Highlights publication. The incumbent
TG-ER team will continue to identify story prospects for 2011 purposes, but the winning XD
team will assume ownership of the production once the award is announced in April 2011. The
printing and design will be funded with XD funds, and the production schedule will be
determined by the winning XD team. We would anticipate that it will be delivered in time for
SC’11. The piece will showcase XD science and technology.
TeraGrid’10 Conference
Elizabeth Leake (ER Team Leader, UC-GIG) and Jan Zverina (SDSC) served as co-chairs for the
TeraGrid’10 Communication Committee. Faith Singer-Villalobos (TACC), Bill Bell (NCSA),
Daphne Seifert (IU), Trish Barker (NCSA), Shandra Williams (PSC), and Michael Schneider
(PSC) also contributed. Warren Froelich served as deputy chair for the conference, and most of
the ER team had a role with the communication efforts of the conference.
TeraGrid Resource Allocation Committee ER Team Involvement (TRAC)
Efforts to identify stories via the TeraGrid Resource Allocation Committee (TRAC) meetings
continued. In April, Elizabeth Leake and Dan Katz began to share the responsibility on behalf of
TeraGrid External Relations so that it wasn’t necessary for both to attend all meetings. A printed
letter from Leake and Katz is now included in reviewer packets for each TRAC meeting. The
letter reminds reviewers to identify those proposals that are especially transformational in their
field, or that use technology in new or unique ways. They are also asked to watch for projects that
119

demonstrate collaborations with other grids, across several domains, or with international
partners.
The TRAC Deadline Announcement is disseminated by Leake each quarter about two weeks
prior to the deadline. The announcement is published in HPCWire, iSGTW, Supercomputing
Online, and other relevant external media, as well as TeraGrid User News and the TG.org news
site. We typically wait until we have the statistics from the previous cycle, and include those in
the announcement. Therefore, it serves a dual purpose—deadline announcement and summary of
the previous cycle’s applications and awards. Although TeraGrid’s allocation process is
becoming progressively more oversubscribed, we have been increasing efforts to push this
announcement in hopes of engaging a broader user base with TeraGrid resources. The increased
competitive element could result in more creative uses of TeraGrid resources, better justification
of resources, and better allocation applications overall.
Misc. External Relations Working Group Activity in Q210
All TG-ER team members conduct external relations efforts for their RP sites, and disseminate
Science Highlights, represent TeraGrid, and write related content for local, regional, and national
sources. Many attend professional conferences on behalf of TeraGrid.
In April, Leake attended the fifth Enabling Grids for E-SciencE (EGEE) user conference in
Uppsala Sweden, upon the invitation of Bob Long, director of EGEE. Since this was EGEE’s
final conference, just prior to their transition to the European Union’s European Grid Initiative
(EGI), the event paralleled the NSF-funded TeraGrid’s transition to the eXtreme Digital (XD)
phase of cyberinfrastructure . Leake had the opportunity to discuss their external relations
experience with navigating the transition with their administration and the European Editor of
International Science Grid This Week (iSGTW). Leake wrote two stories highlighting topics
relevant to both TeraGrid and EGEE users. One story, about Science Gateways and Portals,
highlighted middleware bridging efforts—what it takes for users to successfully leverage both
American and European CI’s. The second story identified the global need for long-term storage
solutions that will serve generations to come.
TeraGrid’s content management system, LifeRay, continued to have issues which delayed the
wiki transition, and ER training efforts we had hoped to undergo this quarter. Leake was granted
minimal content management rights, but changes were limited, and often stalled, due to ongoing
problems with LifeRay. Efforts to train Williams will be made in Q310 so that she can take an
active role in a website facelift, and wiki look-and-feel (the content will continue to be
maintained by Dudek and he will transition wiki content to the LifeRay environment once
Williams designs a new look-and-feel (skin) and organization of content). Both Dudek and
Williams will interface with the User Facing Projects working group in Q310.
TeraGrid’s collaboration with International Science Grid This Week produced a statement of
work. Efforts to identify funding sources were made by OSG and iSGTW administration.
In late April, Leake was invited to Marquette University, Milwaukee, Wisconsin, to talk with
their administration and staff about TeraGrid in conjunction with a CI Days event. Faculty from
the University of Wisconsin at Milwaukee also attended. Marquette has a new computational
science program, and two new TeraGrid campus champions. They were looking for ways to
leverage TeraGrid in the classroom. Since their Fall ’10 curriculum included coursework about
cloud computing, Leake suggested they contact Carol Song and others at Purdue University, to
explore the use of Wispy—TeraGrid’s first cloud production environment. They did, and the
exchange resulted in a Marquette TeraGrid cloud computing course (utilizing
Hadoop/Mapreduce) that is now optimized for the classroom beginning Fall ’10.
120

ER-RP Reports for Q210:
NCSA
TeraGrid RP local external relations staff developed local site-specific content as well as
contributed to TeraGrid-wide activities and publications. External relations staff provide web and
print media outlets detailing the successes of the TeraGrid project and its user community. They
also develop materials for Supercomputing and TeraGrid conferences.
NICS
NICS developed two feature articles focusing on scientific breakthroughs enabled by computing
on NICS resources. The first of these articles highlights researcher Stephen Paddison and his
team as they uncover the fundamental properties of Proton Exchange Membrane fuel cells. The
other feature focused on researchers Greg Voth and Gary Ayton, and their work on understanding
the behavior of the HIV virus, the precursor to Acquired Immune Deficiency Syndrome. NICS
participated along with other TeraGrid external relations staff in nominating, reviewing, and
selecting science stories to be included in the upcoming TeraGrid Science Highlights booklet, to
be released at SC10 in New Orleans.
PSC
PSC Senior Computer Scientist Art Wetzel worked on a 360° Gigapanorama photo of the
Pittsburgh area from the top of the USS Tower. Other team members are David Bear of Carnegie
Mellon University’s Studio for Creative Inquiry, Randy Sargent, Paul Heckbert, Dror Yaron, and
Goutham Mani of the Create Lab at Carnegie Mellon University, Fran Flaherty of CMU Digital
Print Lab, and Ruth Karlin. See http://www.gigapan.org/gigapans/47373/ for the view and more
about production of the Gigapanorama.
Rebecca Horne, Photo Editor of Discover, has asked for and received permission to use various
PSC supercomputing visualizations in a “supercomputing portfolio” planned for an issue of
Discover magazine this fall. PSC provided high resolution versions of Jacobo Bielak’s earthquake
image that was used for PSC’s 2010 calendar and two images of Leopold Grinberg’s brain
arteries simulation data.
SDSC
TeraGrid RP local external relations staff developed local site-specific content as well as
contributed to TeraGrid-wide activities and publications. External relations staff provide web and
print media outlets detailing the successes of the TeraGrid project and its user community. They
also develop materials for Supercomputing and TeraGrid conferences.
SDSC’s Communications Director has served as Deputy Chair for the TG’10 conference in
Pittsburgh, Pa., and has been responsible for preparing committee agendas and notes, while
overseeing specific tasks as assigned by the event’s co-chairs. In addition, SDSC’s Media
Manager has served as co-chair of the TG’10 Communications committee, while SDSC’s graphic
designer has provided graphic support for this meeting, including the development of its graphic
identity (brand) used for the website, electronic and print items.
SDSC’s Communications Director also is serving as the editor of TG’s annual Science
Highlights. During the past quarter, he has led a subcommittee of the TG External Relations team
charged with reviewing and selecting stories for this year’s issue.
SDSC’s communications team also has actively participated in TG ER activities and meetings,
including assistance with press releases, media relations, event planning, and other related
activities.
TACC
121

Below are five feature articles (with links) published on the TACC website within the reporting
period that highlight TeraGrid resources and projects:
• Cleaner Coal through Computation Stanford researcher uses Ranger to design better
pollution controls
• The Universe's Magnetic Personality Simulations by University of Wisconsin
researchers explore turbulence at the cosmic scale
• Blueprint for the Affordable Genome Ranger helps University of Illinois researchers
simulate nanopore gene sequencer
• Catching Evolution in Action UT biologist studies coral and worm RNA to learn how
new traits develop
• In Search of Tomorrow's Cures University of Texas researchers set out to benchmark
and develop computational approaches for accurate, efficient drug discovery
10.4 Enhancement of Diversity
NCAR
After student qualifications and rankings have been established, the SIParCS summer internship
program considers diversity factors when matching students with mentors, projects, and funding.
In this way, two additional students from minority-serving institutions were supported by the
SIParCS program.
TACC
In the spring and summer, TACC accommodates a wide range of tour requests from school
programs and summer camps for under-represented students. Highlights include the FirstByte
summer camp, a one-week program that introduces high school girls to the field of computer
science; the My Introduction to Engineering (MITE) summer camp for high school sophomores
and juniors from under-represented minorities; the final presentations by the Del Valle High
School ROBOTech team (80% under-represented); and the Consider Every Option (CEO)
Program for high school women to explore how engineering benefits society and impacts the
world. Each of these groups was given a presentation about TACC and a tour of the visualization
laboratory, with hands-on activities. TeraGrid was described in each presentation.
10.5 International Collaborations
Indiana
In early June Alcatel-Lucent announced the first commercial 100 Gb/s link connecting
Technische Universitaet Dresden (TUD) and Bergakademie Freiberg. Because of the expertise
that Indiana University has with wide area filesystems and the long standing collaboration that IU
has with TUD, IU will be involved with the testing and deployment of a filesystem bridging both
German campuses.
On April 21, Indiana University and the Center for Information Services and High Performance
Computing (ZIH) at the Technische Universität co-hosted a hands-on workshop on Vampir, a tool
designed to conduct performance analyses and diagnose problems in serial and parallel
supercomputing applications. The full day workshop had joint presentations by members of ZIH
and IU's High Performance Applications (HPA) group in morning followed by a hands-on tutorial
in the afternoon. Vampir will be a fundamental component of FutureGrid, a collaborative grid and
cloud computing test-bed funded by the NSF and developed under the leadership of the PTI
Digital Science Center.
ORNL
122

The ORNL RP continues to collaborate with the Swiss NSF funded Sinergia project focused on
the fitting of diffuse neutron and x-ray scattering patterns in order to characterize the molecular
structure of materials exhibiting diffuse scattering features such as streaks.
PSC
PSC Senior Scientific Specialist Yang Wang has accepted an invitation from Prof. Jianguo Hou,
Co-Director, International Center for Quantum Design of Functional Materials (ICQD) and
President, University of Science and Technology of China in Hefei to visit ICQD as a
Distinguished Guest Scientist during July. He will present several lectures and interact with
faculty members and ICQD scientists to promote deep understanding and effective collaborations
in the fields of multiscale materials science modeling and simulation. While there, Dr. Wang will
present an invited talk at the International Workshop on Nanomagnetism and Spintronics:
Current Status and Outlook in Linfen.
TACC
TACC continues to support researchers and students participating in the UT Austin-Portugal
Collaboratory. A member of the TACC Visualization and Data Analysis group presented handson
sessions at the Summer School on e-Science with GPUs. Students were provided training
accounts and allocation on the XD-Vis system Longhorn. The hands-on sessions were held on
June 16-17 in Braga, Portugal.
http://utenportugal.org/2010/04/ibergrid%E2%80%992010-and-the-summer-school-in-e-sciencewith-many-core-cpugpu-processors-in-braga-this-may-and-june/
10.6 Broader Impacts
NICS
NICS has an ongoing relationship with the University of Tennessee-Knoxville’s chapter of the
Society of Women Engineers. NICS Assistant Director Patricia Kovatch has given talks at
campus meetings of the group, and NICS will be assisting SWE when it hosts its Regional
Conference in the Spring of 2011.
PSC
Kristopher Hupp, a social studies teacher at Cornell High School, in the Coraopolis School
District near Pittsburgh, asked PSC to help the 39 members of the senior class participate in a
county-wide competition to design and code an electronic voting system to be implemented in
high schools across Pennsylvania for a mock election in the fall of 2010. However, since none of
the students had any programming experience and the competition timeline was short, PSC staff
people thought developing a voting system would be too ambitious for them. Instead, PSC
offered to talk to the students about the election process with an emphasis on security and how
that could relate to electronic voting systems. As an alternative to a lecture, PSC opted to use a
role-playing activity to illustrate opportunities for fraud and to encourage critical thinking. Mr.
Hupp sent a nice letter of thanks in which he said, “The activities you put together for our
students not only engaged them, but also helped me begin the dialogue with my students on the
importance of voting and election security. The feedback I got from my students and my
colleague, Megan Fuga, was extremely positive.”
TACC
TACC is endeavoring to broaden the scope of the research and education projects that it supports,
including outreach to the Humanities, Arts & Social Sciences. The first Digital Media & Arts
event, held on April 29, 2010, attracted over 200 artists and creative experts from the Austin
community. In the Vislab, attendees were introduced to advanced computing and visualization
123

techniques and technologies. This event was the first in a planned series which will broaden
TACC’s reach in the Arts & Humanities. In addition, a workshop is being planned for Social
Sciences researchers in the fall.
10.6 Detailed Information on Events and Attendance
Indiana
Type Title Location Date(s) Hours Number of
Participan
ts
Workshop
Performance Analysis
Using the Vampir
Toolchain
IU
Bloomin
gton
April 21,
2010
Number
of
Underrepresen
ted
people
Method
9 13 1 Synchrono
us
LONI
Type Title Location Date(s) Hours Number of
Participan
ts
Workshop
LONI HPC Workshop
at LSU
LSU 26-28
Apr 2010
Tutorial Introduction to Ruby LSU 15 Apr
2010
Tutorial Introduction to Cactus LSU 21 Apr
2010
Number
of
Underrepresen
ted
people
24 9 S
2 14 S
2 14 S
Tutorial Introduction to CPMD LSU 28 Apr 2 3 S
Tutorial Profiling with TAU LSU 6 May
2010
Tutorial Intro to Linux and VI LSU 3 Jun
2010
Tutorial Intro to HPC LSU 10 Jun
2010
Tutorial
Intro To HPC
Environments
LSU
17 Jun
2010
Tutorial Intro to HPC Job Mgmt LSU 24 Jun
2010
2 6 S
3 15 S
3 9 S
3 10 S
3 7 S
Method
124

NCAR
Type Title Location Date(s) Hours Number of
Participan
ts
NCAR:
Conference
presentatio
n
Campus, Regional, and
National HPC:
Infrastructure and
services for research
and education
Colorado
School of
Mines,
Golden,
Colo.
May 7,
2010
Number
of
Underrepresen
ted
people
1 50 5 S
Method
NCAR:
Conference
presentatio
n
Digitizing the Planet:
The role of
cyberinfrastructure in
understanding how our
world works
NCAR,
Boulder,
Colo.
June 16,
2010
1 10 2 S
NCAR:
Tutorial
The 11 th Annual WRF
Users’ Event
NCAR,
Boulder,
Colo.
June 21-
25, 2010
2 20 0 S
NCSA
Type Title Location Date(s) Hours Number of
Participan
ts
Online
Tutorial
Online
Tutorial
Online
Tutorial
Number
of
Underrepresen
ted
people
Access Grid Tutorials CI-Tutor Ongoing N/A 34 Unknown A
BigSim: Simulating
PetaFLOPS
Supercomputers
Debugging Serial and
Parallel Codes
CI-Tutor Ongoing N/A 10 Unknown A
CI-Tutor Ongoing N/A 53 Unknown A
Method
125

Online
Tutorial
Online
Tutorial
Online
Tutorial
Online
Tutorial
Getting Started on theTeraGrid CI-Tutor Ongoing N/A 39 Unknown A
Intermediate MPI CI-Tutor Ongoing N/A 104 Unknown A
Introduction to MPI CI-Tutor Ongoing N/A 557 Unknown A
Introduction to Multicore
Performance
CI-Tutor Ongoing N/A 70 Unknown A
Online
Tutorial
Introduction to
OpenMP
CI-Tutor Ongoing N/A 245 Unknown A
Online
Tutorial
Introduction to
Visualization
CI-Tutor Ongoing N/A 35 Unknown A
Online
Tutorial
Multilevel Parallel
Programming
CI-Tutor Ongoing N/A 48 Unknown A
Online
Tutorial
Parallel Computing
Explained
CI-Tutor Ongoing N/A 91 Unknown A
Online
Tutorial
Parallel Numerical
Libraries
CI-Tutor Ongoing N/A 36 Unknown A
Online
Tutorial
Performance Tuning
for Clusters
CI-Tutor Ongoing N/A 37 Unknown A
Online
Tutorial
Tuning Applications
for High Performance
Networks
CI-Tutor Ongoing N/A 23 Unknown A
NICS
Type Title Location Date(s) Hours Number of
Participan
ts
Workshop
NICS/OLCF XT Hex-
Core Workshop
Oak
Ridge
National
May 10-
12, 2010
Number
of
Underrepresen
ted
people
24 hrs 65 20 Live
Method
126

Lab, TN
Workshop
Computational
Thinking for Educators
ORAU,
Oak
Ridge,
TN
Jun 12-
14, 2010
21 hrs 20 17 Live
College
Class
U.Tennessee-
Knoxville:
Structures
Data
Knoxvill
e, TN
2010
Spring
semester
45 16 Live
College
Class
U.Tennessee-
Knoxville:
HPC class
graduate
Knoxvill
e, TN
2010
Spring
semester
18 3 Live
College
Class
Wofford College
Spartanb
urg, SC
2010
Spring
semester
6 2 Live
College
Class
Tufts University
Boston,
MA
2010
Spring
semester
12 Unknown Live
College
Class
Brown University
Providen
ce, RI
2010
Spring
semester
11 Unknown Live
College
Class
University of Colorado
Boulder,
CO
2010
Spring
semester
14 Unknown Live
ORNL
PSC
Type Title Location Date(s) Hours Number of
Participan
ts
Presentation
Computational
Thinking in K-12
Teaching
SITE (Soc. Tech. Inf
and Teacher Ed.)
International
San
Diego,
CA
Number
of
Underrepresen
ted
people
4/1/10 2 30 16 S
Method 1
1 Methodology: Synchronous (e.g., face to face in classroom or live instruction over the Internet) or asynchronous
(e.g., via the Internet using WebCT or other authoring software.) S=synchronous; A=asynchronous.
127

Workshop
Workshop
Workshop
Workshop
Workshop
Presentation
Workshop
Workshop
Workshop
Symposium
Conference
SafeNet Cyber Safety
Train the Trainer
Workshop
Computational
Biophysics using
NAMD and VMD
Better Educators of
Science for Tomorrow
(BEST) Teachers Day
Summer Institute in
Bioinformatics
Better Educators of
Science for Tomorrow
(BEST)
5 th Annual High
Performance
Computing Day
TeraGrid New User
Training
High Performance
Computing Workshop
Introduction to TG
Resources at the 4 th
PA-OH-WV
Simulators Meeting
Applications of GPUs
in Chemistry and
Material Science
PSC
Pittsburg
h, PA
PSC
Pittsburg
h, PA
PSC
Pittsburg
h, PA
PSC
Pittsburg
h, PA
PSC
Pittsburg
h, PA
Lehigh
Universit
y
Lehigh,
PA
TG
Ready
Talk line
Chatham
Universit
y
Pittsburg
h PA
Carnegie
Mellon
Pittsburg
h PA
Universit
y of
Pittsburg
h
Pittsburg
h PA
5/26/10 6 17 11 S
5/10-
5/14/10
45 28 8 S
5/19/10 8 5 5 S
6/14-
6/25/10
6/14-
6/25/10
95 30 23 S
80 5 5 S
4/23/10 4 20 5 S
4/30/10 2 22 S
5/19 –
5/20/10
16 7 0 S
6/15/10 1 48 S
6/28 –
6/29/10
16 S
Presenta- Overview of HPC for Erie 5/7/10 2 11 4 S
128

tion Industry Technology
Incubator
Erie PA
Purdue
Type Title Location Date(s) Hours Number of
Participan
ts
Graduate
Course
Presentatio
n
Paper
Presentatio
n
Iowa High
Performance
Computing Summer
School, Bill Whitson
Matching Users to
Resources on the
TeraGrid, Kim Dillman
NSF Campus Bridging
Technologies
conference at IUPUI,
Kay Hunt, Scott
Lathrop
The
Universit
y of Iowa
25-
MAY-
2010 and
26-
MAY-
2010
Purdue 12-
MAY-
2010
Indianap
olis, IN
April 6-
7, 2010
Number
of
Underrepresen
ted
people
48 16 (unknown) S
Method
1 Readytalk
0.5 n/a n/a
Presentatio
n
Presentatio
n
“Intro to TeraGrid and
Campus Champions” at
the CI Days event at
Notre Dame, Kim
Dillman and Kay Hunt
“Virtual Breast Project
for Training and
Research”, Purdue
Center for Cancer
Research, Dave Braun
Fort
Wayne,
IN
West
Lafayette
, IN
April 29,
2010
May 14,
2010
1
.5 30
SDSC
Type Title Location Date(s) Hours
Number of
Participan
ts
Number
of
Underrepresen
ted
people
Method
129

W
W
W
W
W
W
W
W
W
W
W
W
Exploring the World of
Digital Art and Design
BE SMART:
Experience Proteins
Through Lab Activities
and Visualization
Accessible Scientific
Programming with
Java: Visualizing the
Human Genome -Part 3
Water Conservation
Lessons for the
Classroom
Moonwalking: Moon
Exploration - Past,
Present and Future
DNA Analysis 1: DNA
Electrophoresis
Questioning and
Assessment Techniques
to Improve Learning
SMART Team students
present at ASBMB
The Wondrous Ways
Google Earth Can be
Used in the Classroom
Environmental Digital
Photography Project:
Part 2
HTML Programming
for High School
Students Part 1
Bridging the Digital
Divide: What Teachers
Need to Know
SDSC
04/01-
02/10
12 28 13 S
SDSC 04/03/10 3 26 26 S
SDSC 04/-6/10 2 23 8 S
SDSC 04/13/10 2 9 8 S
SDSC 04/17/10 3 44 26 S
SDSC 04/20/1- 2 18 16 S
SDSC 04/20/10 2 12 8 S
Anaheim
. CA
04/24/10 4 18 11 S
SDSC 04/27/10 2 22 13 S
SDSC 05/07/10 2 7 7 S
SDSC 05/08/10 3 19 9 S
SDSC 05/10/10 3 35 21 S
W DNA Analysis 2 SDSC 05/13/10 2 18 16 S
W
W
W
A Virtual Tour of Our
Own California Parks
Beginning SMART
Board workshop for
Pre-service Educators
HTML Programming
for High School
SDSC 05/13/10 3 10 4 S
SDSC 05/19/10 2 17 15 S
SDSC 05/22/10 3 19 9 S
130

Students Part 2
W Proteins in Action 2 SDSC 05/25/10 2 18 16 S
W
W
W
Interactive Digital
Books as a Supplement
to Classroom
Textbooks
SpaceTECH;
Standards-based
Astronomy Activities
HTML Programming
for High School
Students Part 3
SDSC 05/26/10 2 14 12 S
SDSC 05/29/10 3 28 20 S
SDSC 06/-5/10 3 19 9 S
W Proteins in Action 3 SDSC 06/08/10 2 18 16 S
W
C
C
C
Exploring the World of
Digital Art and Design
Gordon: A New Kind of
Supercomputer for
Data-Intensive
Applications
Data-Intensive
Solutions at SDSC
Gordon: An
Architecture for Data-
Intensive Computing
SDSC
Princeton
Inst; for
CS&E
(PIC
SciE)
IEEE
MSST20
10
06/28-
30/10
18 54 24 S
3/29/10 1 UNK UNK S
5/3-
7/2010
1 UNK UNK S
UCSD 4/12/10 1 UNK UNK S
TACC
Type Title Location Date(s) Hours Number of
Participan
ts
Workshop
Workshop
Workshop
Profiling and
Debugging Serial and
Parallel Programs
Introduction to
Scientific Visualization
on Longhorn
Fortran90/95/2003
Programming for HPC
Number
of
Underrepresen
ted
people
TACC 4/8/2010 4 12 1 S
TACC 4/12/201
0
TACC 4/22/201
0
7 15 6 S
3 13 2 S
Method
131

Workshop
Introduction to Parallel
Programming with
OpenMP(PETTT)
TACC 4/30/201
0
6 15 1 S&A
Workshop
Scientific Software
Day 2010
TACC 5/10/201
0
8 22 5 S
Workshop
Introduction to
Scientific Visualization
on Longhorn
Huntsvill
e
,
A
L
5/11-
12/2010
16 8 S
Workshop
Introduction to Parallel
Computing on Ranger
Cornell
5/19-
20/2010
16 35 S
Workshop
Introduction to
Scientific Visualization
on Longhorn
Portugal
5/28/201
0
8 11 S
Workshop
Introduction to
Scientific Visualization
on Longhorn
TACC
6/4/2010 7 22 6 S
Workshop
Introduction to Parallel
Computing on Ranger
and Lonestar
TACC
6/10-
11/2010
17 21 4 S
Workshop
Profiling and
Optimization of
Parallel Programs
(PETTT)
TACC
6/30/201
0
6 9 1 S&A
Table: Ranger Virtual Workshop Usage
Quarter Total Logins Unique Logins
Q1 ‘08 74 20
Q2 ‘08 129 26
Q3 ‘08 198 31
Q4 ‘08 135 31
Q1 ‘09 184 29
132

Q2 ‘09 638 216
Q3 '09 504 403
Q4 ‘09 438 289
Q1 '10 425 349
Q2 '10 657 377
Total 4220 1639
Type Title Location Date(s) Number
of
Participa
nts
The Austin
Forum on
Science,
Technology &
Society
The Austin
Forum on
Science,
Technology &
Society
The Austin
Forum on
Science,
Technology &
Society
Digital Media
event at TACC
Visualization
Laboratory
Ranger
Presentation
UT Austin
Central
Development
Event
Other TACC
and TeraGrid
Tours
Tony Befi and Brad
McCredie: “IBM
and Austin: Partners
is a Smarter Planet”
Dr. Jessica
Hanover, Dr.
Steven Leslie, Greg
W. Hartman, Joe
Skraba: “BioTech:
The Next Big
Thing”
Brewster
McCracken,
“Creating a
Dynamic Clean
Energy Economy”
“Explore Creativity
in Digital Space”
Sun City Computer
Club
“Professional
Advisor Day”
Miscellaneous
AT&T
Conference
Center
AT&T
Conference
Center
AT&T
Conference
Center
TACC
Vislab
Sun City,
Texas
TACC
Vislab
TACC
Vislab and
ROC
4/6/2010 89 38
5/4/2010 150 75
6/1/2010 101 25
4/29/2010 200
5/17/2010 100
5/14/2010 62
April - June 458 333
Number of
Underrepresented
people
Totals 1160 471
133

SDSC
Dash Early Users
Project Title
Principle
Investigat
or
Institution
(s)
Project Overview
Status/Outcomes
Protein
Databank
Phil
Bourne
University
of
California,
San Diego
Protein Data Bank (PDB) is a
worldwide repository of
information about the 3D
structures of large biological
molecules including proteins
and nucleic acids. Alignment-
DB, from the PDB group,
performs predictive science
with queries on pair-wise
correlations and alignments
of protein structures that
predict protein fold space and
other properties.
Initial tests show
speedup of up to
69x on a limited
set of queries
when using flash
drives. Tests are
ongoing to
evaluate the
possibility of
running test for
the full PDB. This
represents an
important use case
for Gordon, i.e., a
semi-persistent
database installed
on SSD’s. This
project is
providing a
mechanism to
assess this
architecture.
Biological
Networks
Mihail
Baitaliuc
University
of
California,
San Diego
Biological Networks
integrates over 100 public
databases for thousands of
eukaryotic, prokaryotic and
viral genomes and provides
software tools needed to
decipher gene regulatory
networks, sequence and
experimental data, functional
annotation, transcriptional
regulatory region analysis.
The software platform
supports biological pathways
analysis, querying and
visualization of gene
regulation and protein
interaction networks,
metabolic and signaling
pathways.
Generated a
synthetic
workload to
observe query
plans using
indexes and
sequential scans.
Initial tests show
speedup of up to
186x for typical
queries when the
database is
installed on
SSD’s.
Palomar
Transient
Peter
Nugent
Caltech,
UC
This project’s goal is to
support the PTF Image
PTF transient
search tested on
134

Factory
Berkeley,
Berkeley
Lab (LBL),
Yale
University
processing pipeline and
enable transient detection.
This requires large, random
queries across multiple
databases. There are
approximately 100 new
transients every minute. The
key is to handle all the I/O
issues related to image
processing.
dash I/O node
with SSD storage.
Increased query
performance up to
161x was
achieved.
Discussions are
underway for
sustained longterm
usage of
Dash (and
Gordon) for
research by PTF.
Multiscale
Systems
Center
(MUSYC
Cathie
Olschano
wsky
UC San
Diego
California.
Study energy consumption of
DASH-IO nodes and provide
general relevancy of IO
performance and energy
consumption.
Studied the
requirements of
both software and
hardware
architecture
required on
DASH to perform
this study. Setup
initial hardware
configuration to
carry out
experiments
needed for
project.
Scalable
Graph
Framework
Sandeep
Gupta
University
of
California,
San Diego
The project, Scalable Graph
Framework (SGF), aims
to develop a graph processing
engine for DASH. It is
designed to addresses
scalability and optimization
challenges faced in
processing massively large
graph in the domain of bioinformatics,
semantic neuroinformatics
and graphmodeling.
Project will
deliver a suite of
preprocessing
strategies/algorith
ms for working
efficiently on bioinformatics
and semantic
neuro-informatics
workloads. The
preprocessing
strategy involves
partitioning the
graph and
distributing
workload across
the cluster.
Algorithms use a
hybrid of sharedmemory
and MPI
to effectively
exploit DASH
135

architecture
Architecture
aware
compiler and
algorithm
development
Allan
Snavely
U. Tenn.;
GATech
Res. Inst.;
ORNL; U.
Colorado;
Portland
Group
All groups have
accounts on Dash
and have been
conducting
preliminary tests
on the machine
RAXML
Wayne
Pfeiffer
University
of
California,
San Diego
Benchmark and port RAXML
code to Dash.
Completed
porting and
benchmarking of
code on Dash.
This includes
testing on the
vSMP node.
Abaqus
Mahidhar
Tatineni
University
of
California,
San Diego
The goal is to evaluate the
potential of using flash
storage to accelerate
commercial finite element
codes like Abaqus,
NASTRAN, ANSYS, etc
which have large I/O
requirements during the runs.
Wide applicability since there
is a large user base for such
codes.
ABAQUS
standard test cases
(S2A1, S4B)
using flash
storage for I/O.
Preliminary
results show
~50%
improvement for
8 core runs.
Table: - Active Dash TeraGrid Allocations
Principle
Project Title
Institution(s) Project Overview
Investigator
Performance
Evaluation
Using the
TAU
Performance
System (R)
Sameer
Shende
University of
Oregon
The project aims to analyze
profiling data from runs on
Kraken (up to 64k cores).
These are very large files that
need to be read into memory
(merged) so that users can
analyze the performance data
and visually examine the
results. The vSMP node is
attractive since it has a lot of
aggregated memory. PIs’
group develops the widely
used TAU profiling and
tracing toolkit for
Status/Outcomes
Application
ported to Dash.
Preliminary
testing conducted
on vSMP node
136

performance analysis of
parallel programs written in
Fortran, C, C++, Java,
Python.
Improving
the
Efficiency of
Eukaryotic
Genome
Sequence
Assembly
Using DASH
Mark Miller
University of
California,
San Diego
The proposal aims to address
the question whether the
BFAST code can be speeded
up by taking advantage of the
combination of SSDs and
high RAM nodes available
on the DASH machine. The
request is for a small
allocation to benchmark the
existing BFAST code on
DASH, and to explore how
BFAST code can be
modified to take further
advantage of the DASH
architecture. One specific
part of the BFAST code that
should be improved
dramatically on the DASH
architecture is the initial file
preparation step. In this step,
very large data sets (typically
8 - 85 GB) are read from disc
storage and written into a
new format (fastq) on disc.
This operation is completely
I/O bound on conventional
machines, and requires 7
hours to complete on the
Triton Cluster at SDSC for
our current data set.
Conducting this operation on
SSDs on should speed this
step significantly and may
immediately cut the run time
nearly in half. The aim is to
create a new body of
knowledge on how data
intensive computing can be
done on a new architecture
and enable new, efficient
approaches to Next
Generation Sequence
assembly.
TG Startup
allocation has
been approved
and preliminary
work underway
on Dash.
Large
Synoptic
Survey
Tim
Axelrod
University of
Arizona
The Large Synoptic Survey
Telescope (LSST), to be
constructed at Cerro Pachon
Allocation
approved,
preliminary
137

Telescope
Data
Challenge 3b
in the Chilean Andes and
slated for first light in 2014
and full operation by 2016,
will produce over 15
terabytes of raw astronomical
data each night, resulting in a
database catalog of 22
Petabytes and an image
archive of 100 Petabytes by
the end of the 10-year
survey. This TG allocated
project addresses the goals of
the second half of the third
data challenge (DC3b),
which focuses on the
production of science data
products from an archive's
worth of data.
discussions
completed w/
SDSC contacts
Data
Transposition
Development
for Exa-scale
Data in
Memory
John Helly
University of
California,
San Diego
Cloud data is voluminous
and this is a pilot study of the
utility of Dash for use in
CMMAP data analysis
activities. Temporal
snapshots of data need to be
read into memory, merged,
and resulting dataset is
analyzed.
Startup allocation
was approved.
Preliminary
discussions
completed. An
OpenMP proof of
concept code was
developed and
tested. Read 256
temporal
snapshots with an
aggregated size
of 128GB in
memory
11 Project Management
11.1 Highlights
A new section of the quarterly report Section 9 Evaluation, Monitoring, and Auditing was added
now that the first XD awards have been made for the XD Technology Audit Service and the XD
Technology Insertion Service.
11.2 Project Management Working Group
11.2.1 Tracking
The IPP/WBS tracking process is continuing with project managers reviewing status with area
directors. The area directors are finding the tracking process to be a useful exercise, providing
138

them with a thorough overview of their area’s status against the plan. The update is included as an
appendix to this report.
11.2.2 Reporting
This report begins the second year of TeraGrid providing a single, integrated quarterly report
representing the activity across all NSF TeraGrid awards. The report template has evolved over
the past year by implementing improvements suggested upon the completion of each quarter’s
report. Additionally, a new section of the quarterly report Section 9 Evaluation, Monitoring, and
Auditing was developed now that the first XD awards have been made for the XD Technology
Audit Service and the XD Technology Insertion Service.
12 Science Advisory Board Interactions
In advance of the TeraGrid Science Advisory Board (SAB) in San Diego in Q3, changes have
been made in the SAB membership as follows.
Will be added to the SAB in July 2010:
• Amy Apon-Arkansas
• John Connolly-Kentucky
• Kevin Franklin-Illinois
• Jerry Ostriker-Princeton
• Carlos Simmerling-Stony Brook
• Ali Uzun-FSU
• Greg Voth-UChicago
Will be removed from the SAB after July 2010:
• Eric Chassignet-FSU
• Dave Kaeli-Northeastern
• Luis Lehner-U. Guelph
• Michael Macy-Cornell
• Pat Teller-UTEP
• Cathy Wu-University of Delaware and Georgetown
139

13 Collaborations with Non-TeraGrid Institutions
13.1 Grid Partnerships/Infrastructures
Purdue continued to work with OSG engagement users on using TeraGrid Steele for “highthroughput
HPC” (HTHPC) computation jobs (single node, 8-core). This effort leverages
previous work with enabling easier submission of MPI jobs in the Purdue OSG effort. In this
quarter, a total of 1765 OSG MPI jobs consumed 138,122 wallclock hours on Purdue Steele.
13.2 Science Projects and Science Gateways
Under the TG-DEISA Interoperability Project (funded as part of the LONI HPCOPS award),
LONI expects to demonstrate interoperability of TG Queen Bee resources with DEISA in August,
with the potential of extending support to all of TG soon after. The science component has begun
using the available interoperability support, with results to be presented at TG’10 and to the UK
e-Science Program.
13.3 Data Systems
IU worked closely with Mississippi State University to facilitate Scott Michael's astronomical
research. By mounting IU's Data Capacitor Wide Area Filesystem at MSU, it was possible to
bridge the campuses and prevent Michael from having to move 30 TB of data to MSU for
analysis. This was an example of using a wide area filesystem to allow non-TeraGrid institutions
to move their data onto the TeraGrid with ease by offering the user a standard filesystem
interface.
ORNL continues to work with the REDDnet collaboration. Three different TeraGrid sites
(ORNL, SDSC, TACC) now have REDDnet nodes.
John Cobb at ORNL continues to work with the DataONE Datanet project. TeraGrid as a whole
has reached out to both awarded DataNet projects (DataONE and the Data Conservancy)
13.4 Science Projects
PSC is a member of two collaborations involving the NIH MIDAS Research Network, as the
Computational Core for the U. Pittsburgh National Center of Excellence (COE) and as a research
partner on the RTI Information Technology Resource. MIDAS is a collaborative network of
research scientists funded by the NIH who use computational, statistical and mathematical
models to understand infectious disease dynamics and thereby assist the nation to prepare for,
detect and respond to infectious disease threats.
1. In support of the U. Pittsburgh COE, PSC established infrastructure to facilitate
development of software projects, including setup and management of a CVS server. The
server deploys both ViewVC for viewing the CVS repository as a webpage and Google's
coderev to generate browsable code reviews. These developments have led to better
organization and software design for the COE's software projects. PSC developers have
actively participated in three software projects: FRED, a new agent-based simulator
specifically tailored to support features such as behavioral, viral evolution, and
emergency preparedness modeling; GAIA, a GIS Information-based visualization tool
designed to represent epidemiological data; and HERMES, a discrete event simulation
engine for modeling supply chain dynamics of vaccine cold-chains in developing nations.
2. In support of the RTI Information Technology Resource, PSC Research Fellow Shawn
Brown provided leadership for the MIDAS Software Working Group. PSC developed a
web based portal, entitled MISSION (MIDAS Software Sharing and Information
Outreach Network), with the purpose of facilitating collaboration between the various
140

MIDAS Network software development efforts. The site provides a collaborative
development environment for over fifty developers in seven different research groups,
and it provides a platform for sharing software and promoting open access to MIDAS
software tools. PSC also hosted the first “MIDAS Software Developers Workshop”
focused on the issues associated with model development and productization of MIDAS
tools, with 16 developers from around the world in attendance.
Pan-STARRS -- the Panoramic Survey Telescope & Rapid Response System -- is an innovative
design for a wide-field imaging facility being developed at the University of Hawaii's Institute for
Astronomy. The immediate goal of Pan-STARRS is to discover and characterize Earthapproaching
objects (both asteroids and comets) that might pose a danger to our planet. For Pan-
STARRS, PSC Senior Scientific Specialist Joel Welling is developing 1) a web utility to look up
available observations by sky coordinates, which he is implementing as a SOAP-based interface
to the Pan-STARRS image database and skycell geometry database; 2) a SOAP-based interface to
the tool which causes the IPP computing infrastructure to generate “postage stamp” images of
particular sky locations; and 3) PHP pages to bridge the two. The utility, running on one of
Michael Wood-Vasey’s machines at the University of Pittsburgh, has already had hits from
Harvard, Queen’s University Belfast, and Denmark. This work develops RP/AUS staff expertise
in large-scale data logistics and analysis.
The Vaccine Modeling Initiative (VMI) is funded by the Bill & Melinda Gates Foundation and
directed by Dr. Donald Burke, Dean of the University of Pittsburgh Graduate School of Public
Health. The VMI is a research consortium between the University of Pittsburgh, Imperial College
London and Princeton University with collaborators in many other institutions including the
Johns Hopkins Bloomberg School of Public Health and the Pittsburgh Supercomputing Center.
For the vaccine modeling initiative (VMI), PSC’s Joel Welling has developed and executed
vaccine distribution simulations for distribution networks in Niger and districts of Thailand. A
short-term goal is reconciling the predictions of HERMES, the code developed at PSC, against
the predictions of the ARENA code developed at Pitt. Dr. Welling has also been working to refactor
HERMES to separate more fully the model from the functional code, which is necessary to
develop models of new distribution networks (e.g., areas other than Niger and Thailand, or
incorporating different aspects) more efficiently and reliably. This work fosters computational
techniques and RP/AUS staff skills in agent based modeling, data analysis and presentation,
knowledge extraction and decision support.
Wayne Peiffer of SDSC began a collaboration with Sam Levy and Ashley Van Zeeland of the
Scripps Translational Science Institute (STSI). This is in conjunction with his Advanced User
Support for Mark Miller's new TeraGrid allocation on Dash. He installed and tested on Dash two
codes—BFAST and SHRiMP—that map short DNA sequence reads to a large reference genome.
TACC continues to participate in the World Community Grid (WCG) effort. TACC resources
contributing to the solution of problems addressed by the WCG include the visualization system
Colt (40 compute cores) and the HPC serial cluster Stampede (1737 compute cores). TACC staff
desktops and laptops also provide cycles to the project. Approximately 44 years of run time have
been accumulated (93,616 results) on TACC resources in the reporting period.
ORNL continues to operate an ESG node in its TeraGrid enclave. ESG is gearing up to support
the next round of IPCC simulation runs for the next IPCC report.
141

A
Publications Listing
A.1. TeraGrid Staff Publications
For Q2 2010, the TeraGrid sites reported the following publications authored in whole or in part
by GIG- or RP-funded staff members.
13.4.1 Science and Engineering Highlights
1. D Katz, Scott Callaghan, Robert Harkness, Shantenu Jha, et al “Science on the TeraGrid”, submitted to
Journal of Computational Methods in Science and Technology [UC/ANL, SDSC, LONI]
13.4.2 Data and Visualization
2. Josephine Palencia, Robert Budden and Kevin Sullivan, “Kerberized Lustre 2.0 over the WAN”,
TeraGrid 2010, Pittsburgh, PA, August 2-5, 2010, accepted for ACM Publication Copyright 2010 ACM
978-1-60558-818-6/10/0. [PSC]
3. 1. C Miceli, M Miceli and S Jha∗, “Understanding Performance of Distributed Data-Intensive
Applications”, Accepted for publication in Philosophical Transactions of the Royal Society of London A
(2010) [LONI]
13.4.3 Users and User Support
4. H. Karimabadi, H. X. Vu, B. Loring, Y. Omelchenko, M. Tatineni, A. Majumdar, and J. Dorelli, "Enabling
Breakthrough Kinetic Simulations of the Magnetosphere Using Petascale Computing," Asia Oceania
Geosciences Society (AOGS) Hyderabad, India, July 5-9, 2010
5. Y Khamra, C White, S Jha, “Modelling Data-Driven CO2 Sequestration Using Distributed HPC
Cyberinfrastructure: A Case Study”, TeraGrid 2010 Conference Proceedings [TACC, LONI]
6. Joanne I. Yeh, Boris Shivachev, Srinivas Rapireddy, Matthew J. Crawford, Roberto R. Gil, Shoucheng
Du, Marcela Madrid, and Danith H. Ly, “Crystal Structure of Chiral γPNA with Complementary DNA
Strand—Insights into the Stability and Specificity of Recognition and Conformational Preorganization”,
Journal of the American Chemical Society, in press. [PSC]
13.4.4 User-Facing Projects and Core Services
7. David Hart, "Measuring TeraGrid: Workload Characterization for an HPC Federation," International
Journal of HPC Applications, 2010, accepted. [SDSC]
13.4.5 Network, Operations, and Security
8. H Kim, Y Khamra, S Jha, M Parashar, “An Autonomic Approach to Integrated HPC Grid and Cloud
Usage”, IEEE E-Science 2009 Conference Proceedings [TACC]
13.4.6 Education, Outreach and Training
9. Richard A. Aló, Diane Baxter, Karl Barnes, Al Kuslikis, Geoffrey Fox, Alex Ramirez, Advancing
Computational Science, Visualization and Homeland Security Research/ Education at Minority Serving
Institutions National Model Promoted/ Implemented by MSI-CIEC (Minority Serving Institutions-
CyberInfrastructure Empowerment Coalition), accepted to the International Conference on
Computational Science, ICCS 2010 for inclusion in the program (May 31 – June 2, 2010, Amsterdam)
and for publication in the new Elsevier, Procedia Computer Science Series. [Indiana, SDSC]
13.4.7 Science Gateways
10. Tang, W., Bennett D. A., and Wang, S. 2010. "A Parallel Agent-based Model of Land Use Opinions."
Journal of Land Use Science, under revision
11. Bennett D. A., Tang, W., and Wang, S. 2010. "Toward an Understanding of Provenance in Complex
Land Use Dynamics." Journal of Land Use Science, accepted
142

12. Wang, S. 2010. "A Cyber-GIS Framework for the Synthesis of Cyberinfrastructure, GIS, and
Spatial Analysis." Annals of the Association of American Geographers, 100 (3): 1-23
A.2. Publications from TeraGrid Users
The following 731 publications were gathered primarily from renewal Research submissions to
the June 2010 TRAC meeting. The extraction process was conducted by a UCSD undergraduate
student at SDSC. The publications are organized by field of science and by the proposal with
which they were associated.
Advanced Scientific Computing
ASC050025
1. A. Bhatele, E. Bohm, and L. V. Kale. Optimizing communication for Charm++ applications by reducing
network contention. Concurrency and Computation: Practice and Experience, 2010.
2. A. Bhatele and L. V. Kale. Quantifying Network Contention on Large Parallel Machines. Parallel
Processing Letters (Special Issue on Large-Scale Parallel Processing), 19(4):553–572, 2009.
3. A. Bhatele, L. Wesolowski, E. Bohm, E. Solomonik, and L. V. Kale. Understanding application
performance via microbenchmarks on three large supercomputers: Intrepid, Ranger and Jaguar.
IJHPCA, 2010.
4. S. Biersdorff, C. W. Lee, A. D. Malony, and L. V. Kale. Integrated Performance Views in Charm ++:
Projections Meets TAU. In Proceedings of The 38th International Conference on Parallel Processing
(ICPP), pages 140–147, Vienna, Austria, September 2009.
5. C. L. M. Esteban Meneses and L. V. Kale. Team-based message logging: Preliminary results. In 3rd
Workshop on Resiliency in High Performance Computing (Resilience) in Clusters, Clouds, and Grids
(CCGRID 2010)., May 2010.
6. F. Gioachin, C. W. Lee, and L. V. Kale. Scalable Interaction with Parallel Applications. In Proceedings
of TeraGrid’09, Arlington, VA, USA, June 2009.
7. P. Jetley, L. Wesolowski, and L. V. Kale. Studies on the performance of scalable GPU-based
hierarchical N-body simulations. Submitted for Publication.
8. E. R. Rodrigues, P. O. A. Navaux, J. Panetta, and C. L. Mendes. A new technique for data privatization
in userlevel threads and its use in parallel applications. In ACM 25th Symposium On Applied Computing
(SAC), Sierre, Switzerland, 2010.
9. G. Zheng, E. Meneses, A. Bhatele, and L. V. Kale. Hierarchical Load Balancing for Large Scale
Supercomputers. Technical Report 10-08, Parallel Programming Laboratory, March 2010.
MCA08X024
10. Lixia Liu and Zhiyuan Li, ʺ A Compiler-automated Array Compression Scheme for Optimizing Memory
Intensive Programsʺ , Proceedings of 24th ACM International Conference on Supercomputing, June
1-4, 2010, to appear.
11. Yingchong Situ, Lixia Liu, Chandra S. Martha, Matthew E. Louis, Zhiyuan Li, Gregory A. Blaisdell and
Anastasios S. Lyrintzis, ʺ Reducing Communication Overhead in Large Eddy Simulation of Jet Engine
Noiseʺ , submitted to IEEE Cluster 2010.
Atmospheric Sciences
ATM090002
12. Pritchard, M. S. and R. C. J. Somerville, 2009. Assessing the Diurnal Cycle of Precipitation in a Multi-
Scale Climate Model. Journal of Advances in Modeling Earth Systems 1, Art. 12.
13. Pritchard, M. S. and R. C. J. Somerville, 2009. Empirical orthogonal function analysis of the diurnal
cycle of precipitation in a multi-scale climate model. Geophysical Research Letters 36 (5), L05812.
ATM090042
14. Sippel, J. A and F. Zhang, 2010: Predictability of Tropical Cyclones -- Understanding the Limits and
Uncertainties in Hurricane Prediction. VDM Verlag, 178pp.
143

15. Fang, J and F. Zhang, 2010: Initial development and genesis of Hurricane Dolly (2008).Journal of the
Atmospheric Sciences, 63, 655-672.
16. Zhang, F., Y. Weng, Y.-H. Kuo, and J. S. Whitaker, 2010: Predicting Typhoon Morakot’s Catastrophic
Rainfall and Flooding With a Cloud-Scale Ensemble System. Weather and Forecasting, submitted.
17. Zhang, F., Y. Weng, J. Gamache, and F. Marks, 2010: Future of Hurricane Prediction: Cloudresolving
Ensemble Analysis and Forecast. Bulletin of American Meteorological Society, submitted.
18. Weng, Y., M. Zhang, and F. Zhang , 2010: Advanced data assimilation for cloud-resolving hurricane
initialization and prediction. Computing in Science and Engineering, submitted.
19. Kieu, C. Q., F. Zhang, J. S. Gall and W. M. Frank, 2010: Tropical cyclone formation from the equatorial
waves. Journal of the Atmospheric Sciences, submitted.
20. Fang, J and F. Zhang, 2010: Evolution of Multi-scale Vortices in the Development of Hurricane Dolly
(2008). Journal of the Atmospheric Sciences, submitted.
21. Fang, J and F. Zhang, 2010: Beta-effect on the evolution of tropical cyclones. Monthly Weather Review,
in preparation.
22. Green, B., F. Zhang, and P. Markowski, 2010: Supercells in Hurricane Katrina (2005). Monthly Weather
Review, in preparation.
23. Poterjoy, J., and F. Zhang, 2010: Dynamics and structures of error covariance in hurricanes. Monthly
Weather Review, in preparation.
24. Weng, Y., and F. Zhang, 2010: Assimilation of airborne Doppler radar observations with an Ensemble
Kalman filter for cloud-resolving hurricane analysis and forecasting. Monthly Weather Review, in
preparation.
25. Zhang, M. and F. Zhang: Coupling EnKF with 4D-Var in a hybrid data assimilation algorithm (4D-Hybrid)
for cloud-resolving hurricane prediction. Monthly Weather Review, in preparation.
ATM090046
26. Karimabadi, H., J. Scudder, V. Roytershteyn, J. Scudder, W. Daughton, Why is reconnection in the
solar wind so different than in other environments, 9th Annual International Astrophysics Conference,
Pickup Ions Throughout the Heliosphere and Beyond, Maui, Hawaii, March 14-19, 2010.
27. Karimabadi, H., W. Daughton, V. Roytershteyn, J. Scudder, Conditions for fast reconnection and its
large-scale structure, EGU conference, Vienna, 2010.
28. Karimabadi. H., H. X. Vu, B. Loring, Y. Omelchenko, M. Tatineni, A. Majumdar, J. Dorelli, Enabling
Breakthrough Kinetic Simulations of the Magnetosphere Using Petascale Computing. AOGS, India,
2010.
29. Karimabadi, H., and J. Dorelli, H. X. Vu, B. Loring, Y. Omelchenko, Is quadrupole structure of out-ofplane
magnetic field evidence of Hall reconnection, to appear in Modern Challenges in Nonlinear
Plasma Physics, editor D. Vassiliadis, AIP conference, 2010.
30. Daughton, W., V. Roytershteyn, H. Karimabadi, L.Yin, B.J. Albright, S.P. Gary and Kevin J. Bowers,
Secondary Island Formation in Collisional and Collisionless Kinetic Simulations of Magnetic
Reconnection, to appear in Modern Challenges in Nonlinear Plasma Physics, editor D. Vassiliadis, AIP
conference, 2010.
31. Karimabadi, H., V. Roytershteyn, C. G. Mouikis, L.M. Kistler, W. Daughton, Flushing Mechanism in
Reconnection: Effects of Minority Species of Oxygen Ions, to appear in Planetary and Space Science,
2010.
ATM100032
32. Jacobsen, D.A., Thibault, J.C., Senocak, I., 2010. An MPI-CUDA implementation for massively parallel
incompressible flow computations on multi-GPU clusters, in: Proceedings of 48th AIAA Aerospace
Sciences Meeting and Exhibit, paper no: 2010-522
MCA95C006
33. Zhao, K. and M. Xue, 2009: Assimilation of coastal Doppler radar data with the ARPS 3DVAR and cloud
analysis for the prediction of Hurricane Ike (2008). Geophy. Res. Letters, 36, L12803,
doi:10.1029/2009GL038658.
34. Potvin, C. K., A. Shapiro, T.-Y. Yu, J. Gao, and M. Xue, 2009: Using a low-order model to characterize
and detect tornadoes in multiple-Doppler radar data. Mon. Wea. Rev., 137, 1230–1249.
35. Dunning, T. H., Jr., K. Schulten, J. Tromp, J. P. Ostriker, K. K. Droegemeier, M. Xue, and P. Fussell,
2009: Science and engineering in the petascale era. Computing Sci. Engineering, 11, 28-36.
144

36. Xue, M., M. Tong, and G. Zhang, 2009: Simultaneous state estimation and attenuation correction for
thunderstorms with radar data using an ensemble Kalman filter: Tests with simulated data. Quart. J.
Royal Meteor. Soc., 135, 1409-1423.
37. Schwartz, C., J. Kain, S. Weiss, M. Xue, D. Bright, F. Kong, K. Thomas, J. Levit, and M. Coniglio, 2009:
Next-day convection-allowing WRF model guidance: A second look at 2 vs. 4 km grid spacing. Mon.
Wea. Rev., 137, 3351-3372.
38. Clark, A. J., W. A. Gallus, Jr., M. Xue, and F. Kong, 2009: A comparison of precipitation forecast skill
between small convection-permitting and large convection-parameterizing ensembles. Wea. and
Forecasting, 24, 1121-1140.
39. Jung, Y., M. Xue, and G. Zhang, 2010: Simulations of polarimetric radar signatures of a supercell storm
using a two-moment bulk microphysics scheme. J. Appl. Meteor. Climatol., 49, 146-163.J. Appl. Meteor.
Climatol., 49, 146-163.
40. Jung, Y., M. Xue, and G. Zhang, 2010: The estimation of microphysical parameters and atmospheric
state using simulated polarimetric radar data and ensemble Kalman filter in the presence of observation
operator error. Mon. Wea. Rev., 138, 539–562.
41. Coniglio, M. C., K. L. Elmore, J. S. Kain, S. Weiss, and M. Xue, 2010: Evaluation of WRF model output
for severe-weather forecasting from the 2008 NOAA Hazardous Weather Testbed Spring Experiment.
Wea. Forecasting, Accepted.
42. Schwartz, C. S., J. S. Kain, S. J. Weiss, M. Xue, D. R. Bright, F. Kong, K. W.Thomas, J. J. Levit, M. C.
Coniglio, and M. S. Wandishin, 2010: Toward improved convection-allowing ensembles: model physics
sensitivities and optimizing probabilistic guidance with small ensemble membership. Wea. Forecasting,
Accepted.
43. Clark, A. J., W. A. Gallus, M. Xue, and F. Kong, 2010: Growth of spread in convection allowing and
convection-parameterizing ensembles Wea. Forecasting, Accepted.
44. Dawson, D. T., II, M. Xue, J. A. Milbrandt, and M. K. Yau, 2010: Comparison of evaporation and cold
pool development between single-moment and multi-moment bulk microphysics schemes in idealized
simulations of tornadic thunderstorms. Mon. Wea. Rev., Accepted.
45. Xue, M., Y. Jung, and G. Zhang, 2010: State estimation of convective storms with a twomoment
microphysics scheme and ensemble Kalman filter: Experiments with simulated radar data Q. J. Roy.
Meteor. Soc, Accepted.
46. Ge, G., J. Gao, K. Brewster, and M. Xue, 2010: Impacts of beam broadening and earth curvature on 3D
variational radar data assimilation with two Doppler radars. J. Atmos. Ocean Tech., 27, 617-636.
47. Clark, A. J., W. A. Gallus, Jr., M. Xue, and F. Kong, 2010: Convection-allowing and convectionparameterizing
ensemble forecasts of a mesoscale convective vortex and associated severe weather.
Wea. Forecasting, Accepted.
48. Schenkman, A., M. Xue, A. Shapiro, K. Brewster, and J. Gao, 2010: Impact of radar data assimilation
on the analysis and prediction of the 8-9 May 2007 Oklahoma tornadic mesoscale convective system,
Part I: Mesoscale features on a 2 km grid. Mon. Wea. Rev., Conditionally accepted.
49. Schenkman, A., M. Xue, A. Shapiro, K. Brewster, and J. Gao, 2010: Impact of radar data assimilation
on the analysis and prediction of the 8-9 May 2007 Oklahoma tornadic mesoscale convective system,
Part II: Sub-storm-scale mesovortices on a 400 m Grid. Mon. Wea. Rev., Conditionally accepted.
50. Kain, J. S., M. Xue, M. C. Coniglio, S. J. Weiss, F. Kong, T. L. Jensen, B. G. Brown, J. Gao, K.
Brewster, K. W. Thomas, Y. Wang, C. S. Schwartz, and J. J. Levit, 2010: Assessing advances in the
assimilation of radar data within a collaborative forecasting-research environment. Wea. Forecasting,
Accepted.
Astronomical Sciences
AST060032
51. @* “Cyberinfrastructure to Support Science and Data Management for the Dark Energy Survey”
52. Ngeow, C. Mohr, J.J., Alam, T., Barkhouse, W.A., Beldica, C., Cai, D., Daues, G., Plante, R. Annis, J.,
Lin, H., Tucker, D. and Smith, C. 2006, SPIE, 6270, 68-79.
53. *“The Dark Energy Survey Data Management System”
54. Mohr, J.J., Adams, D., Barkhouse, W., Beldica, C., Bertin, E., Cai, Y.D., da Costa, L.A.N., Darnell, J.A.,
Daues, G.E., Jarvis, M., Gower, M., Lin, H., Martelli, L., Neilsen, E., Ngeow, C., Ogando, R.L.C., Parga,
A., Sheldon, E., Tucker, D., Kuropatkin, N. & Stoughton, C. for the Dark Energy Survey Collaboration
2008, SPIE, 7016, 17.
145

55. “The Dark Energy Survey Data-Management System: The Processing Framework”
56. Gower, M.; Mohr, J. J.; Adams, D.; Cai, Y. D.; Lin, H.; Neilsen, E. H.; Tucker, D.; Bertin, E.;
da Costa, L. A. N.; Martelli, L.; Ogando, R. L. C.; Jarvis, M.; Sheldon, E.
57. 2009, contribution to the Astronomical Data Analysis Software and Systems XVIII ASP Conference
Series, 411, 14-17
58. “The Dark Energy Survey Data Management System: The Coaddition Pipeline and PSF
Homogenization”
Darnell, T.; Bertin, E.; Gower, M.; Ngeow, C.; Desai, S.; Mohr, J. J.; Adams, D.; Daues, G. E.;
Gower, M.; Ngeow, C.; Desai, S.; Beldica, C.; Freemon, M.; Lin, H.; Neilsen, E. H.; Tucker, D.;
da Costa, L. A. N.; Martelli, L.; Ogando, R. L. C.; Jarvis, M.; Sheldon, E.
59. 2009, contribution to the Astronomical Data Analysis Software and Systems XVIII ASP Conference
Series, 411, 18-21
60. “Application of the Dark Energy Survey Data Management System to the Blanco Cosmology Survey
Data” Ngeow, C.C., Mohr, J.J., Barkhouse, W., Alam, T., Beldica, C., Cai, D., Daues, G., Duda, P.,
Annis, J., Lin, H., Tucker, D., Rest, A., Smith, C., Lin, Y., High, W., Hansen, S., Brodwin, M., Allam, S.
and BCS Collaboration. 2006 American Astronomical Society Meeting, 209, #22.06
61. “The Blanco Cosmology Survey: Data Reduction, Calibration and Photometric Redshift Estimation to
Four Distant Galaxy Clusters Discovered by the South Pole Telescope”
62. Ngeow, Chow Choong; Mohr, J.; Zenteno, A.; Data Management, DES; BCS; SPT Collaborations
63. 2009 American Astronomical Society Meeting, 213, #448.04
64. “A Data Management System for the Dark Energy Survey”
65. Barkhouse, W., Alam, T., Beldica, C., Cai, D., Daues, G., Mohr, J., Ngeow, C., Plante, R., Annis, J., Lin,
H. and Tucker, D. 2006 American Astronomical Society Meeting, 208, #62.02
AST080007
66. Kowal & Lazarian, 2010, “Turbulence in Collisionless Plasmas: Statistical Analysis from Numerical
Simulations with Pressure Anisotropy”, submitted to ApJ
67. Kowal, Falceta-Gon¸calves, D. & Lazarian, 2010, “Velocity Field of Compressible MHD Turbulence:
Wavelet Decomposition and Mode Scalings”, submitted to ApJ
68. Kowal, Lazarian, Vishniac & Otmianowska-Mazur, 2009, “Numerical Tests of Fast Reconnection in
Weakly Stochastic Magnetic Fields”, ApJ, 700, 63
69. Beresnyak & Lazarian, 2010, “Nonlocality of Balanced and Imbalanced Turbulence”, submitted to ApJ,
arXiv:1002.2428
70. Beresnyak, Yan & Lazarian, 2010 “Numerical study of Cosmic Ray Diffusion in MHD turbulence”,
submitted to ApJ, arXiv:1002.2646
71. Beresnyak & Lazarian 2009, “Structure of Stationary Strong Imbalanced Turbulence”, ApJ, 702, 460,
72. Beresnyak & Lazarian 2009, “Comparison of spectral slopes of magnetohydrodynamic and
hydrodynamic turbulence and measurements of alignment effects” ApJ, 702, 1190
AST080028
73. Simulations of Magnetized Disks Around Black Holes: Effects of Black Hole Spin, Disk Thickness, and
Magnetic Field Geometry Authors: Penna, Robert F.; McKinney, Jonathan C.; Narayan, Ramesh;
Tchekhovskoy, Alexander; Shafee, Rebecca; McClintock, Jeffrey E. Journal: Monthly Notices of the
Royal Astronomical Society, 2010, submitted (arXiv:1003.0966) TeraGrid resources used: NICS
Kraken, LONI QueenBee, NCSA Abe Web link: http://www.cfa.harvard.edu/_narayan/TeraGrid
Papers/narayan1.pdf
74. Black Hole Spin and The Radio Loud/Quiet Dichotomy of Active Galactic Nuclei Authors:
Tchekhovskoy, Alexander; Narayan, Ramesh; McKinney, Jonathan C. Journal: The Astrophysical
Journal, 711, 50-63 (2010) TeraGrid resources used: LONI QueenBee Web link:
http://www.cfa.harvard.edu/_narayan/TeraGrid Papers/narayan2.pdf
75. Numerical Studies of Relativistic Jets Authors: Narayan, Ramesh; Tchekhovskoy, Alexander; McKinney,
Jonathan C. Journal: Proceedings of “Accretion and Ejection in AGN: A Global View” (Como, June 22-
26, 2009), ASP Conference Series, Eds: L. Maraschi, G. Ghisellini, R. Della Ceca and F. Tavecchio, in
press, 2010 (arXiv:1001.1355) TeraGrid resources used: LONI QueenBee Web link:
http://www.cfa.harvard.edu/_narayan/TeraGrid Papers/narayan3.pdf
76. Magnetohydrodynamic Simulations of Gamma-Ray Burst Jets: Beyond the Progenitor Star Authors:
Tchekhovskoy, Alexander; Narayan, Ramesh; McKinney, Jonathan C. Journal: New Astronomy, in
146

press, 2010 (arXiv:0909.0011) TeraGrid resources used: LONI QueenBee Web link:
http://www.cfa.harvard.edu/_narayan/TeraGrid Papers/narayan4.pdf
77. Efficiency of Magnetic to Kinetic Energy Conversion in a Monopole Magnetosphere Authors:
Tchekhovskoy, Alexander; McKinney, Jonathan C.; Narayan, Ramesh Journal: The Astrophysical
Journal, 699, 1789-1808, 2009 TeraGrid resources used: LONI QueenBee Web link:
http://www.cfa.harvard.edu/_narayan/TeraGrid Papers/narayan5.pdf
AST100029
78. Armitage, C., and Wandelt, B. D., “PreBeaM for Planck: a Polarized, Regularized, Beam-Deconvolution
Map-Making Method,” arxiv:0807.4179, Astrophysical Journal Supplement, 181, 533-542 (2009)
79. Fendt, W.A., Chluba, J., Rubino-Martin, J.A., Wandelt, B. D., “Rico: An Accurate Cosmological
Recombination Code,” arXiv:0807.2577, Astrophysical Journal Supplement, 181, 627-638 (2009)
80. Jewell, J. B., Eriksen, H. K., Wandelt, B. D., O'Dwyer, I. J., Huey, G., Gorski, K. M., “A Markov Chain
Monte Carlo Algorithm for analysis of low signal-to-noise CMB data,” arXiv:0807.0624, Astrophysical
Journal 697, 258-268 (2009)
81. Elsner, F. and Wandelt, B. D., “Improved simulation of non-Gaussian temperature and polarization CMB
maps,” arXiv:0909.0009, Astrophysical Journal Supplement, 184, 264-270 (2009)
82. Dickinson, C., Eriksen, H. K., Banday, A. J., Jewell, J. B., Gorski, K.M., Huey, G., Lawrence, C. R.,
O'Dwyer, I. J., Wandelt, B. D., “Bayesian component separation and CMB estimation for the 5-year
WMAP temperature data,” arXiv:0903.4311, Astrophysical Journal, 705, 1607-1623 (2009)
83. Biswas, R. and Wandelt, B. D., “Parameters and pitfalls in dark energy models with time varying
equation of state,” arXiv:0903.4311, Physical Review D, in press.
84. Groeneboom, N. E. , Eriksen, H. K., Gorski, K., Huey, G., Jewell, J., Wandelt, B. D., “Bayesian analysis
of white noise levels in the 5-year WMAP data,” arXiv:0904.2554, Astrophysical Journal 702, L87-L90
(2009)
85. Elsner, F. and Wandelt, B. D., and Schneider, M., “Probing Local Non-Gaussianities Within a Bayesian
Framework”, accepted for publication in Astronomy and Astrophysics.
86. Kitaura, F. S., Jasche, J., Li, C., Ensslin, T. A., Metcalf, R. B., Wandelt, B. D., Lemson, G., White, S. D.
M., “Cosmic Cartography of the Large-Scale Structure with Sloan Digital Sky Survey Data Release 6,”
arXiv:0906.3978, accepted for publication in MNRAS.
87. Lavaux, G. and Wandelt, B. D., “Precision cosmology with voids: definition, methods, dynamics,” 15
pages, arXiv:0906.4101, accepted for publication in MNRAS.
88. Jasche, J., F.Kitaura, S., Wandelt, B. D., and Ensslin, T. A., “Bayesian power-spectrum inference for
Large Scale Structure data,” accepted for publication in Monthly Notices of the Royal Astronomical
Society.
89. Biswas, R., Alizadeh, E., and Wandelt, B. D., “Voids as a precision probe of dark energy,” accepted for
publication in Physical Review D.
90. Rubino-Martin, J.A., Chluba, J., Fendt, W.A., and Wandelt, B.D., “Estimating the impact of
recombination uncertainties on the cosmological parameter constraints from cosmic microwave
background experiments,” arXiv:0910.4383, submitted to Monthly Notices of the Royal Astronomical
Society.
91. Huffenberger, K. and Wandelt, B. D., “Fast and Exact Spin-S Spherical Harmonic Transforms,”
submitted to the Astrophysical Journal Supplement.
92. Lavaux, G. and Wandelt, B. D., “Fast CMB lensing using statistical interpolation on the sphere,” 9
pages, arXiv:1003.4984, submitted to the Astrophysical Journal
MCA04N015
93. Ball, N.M., & Brunner, R.J., “Data Mining and Machine Learning in Astronomy”, International Journal of
Modern Physics D., in press.
94. Ball, N.M., Brunner, R.J., Myers, A.D., Strand, N.E., Alberts, S.L., and Tcheng, D., "Robust Machine
Learning Applied to Astronomical Datasets. III. Probabilistic Photometric Redshifts for Galaxies and
Quasars in the SDSS and GALEX", 2008, Astrophysical Journal, 683, 12.
95. Ball, N.M., Brunner, R.J., Myers, A.D., Strand, N.E., Alberts, S.A., Tcheng, D., and Llora, X., "Robust
Machine Learning Applied to Astronomical Datasets. II. Photometric Redshifts of Quasars in a GALEX-
SDSS Federated Dataset.", 2007, Astrophysical Journal, 663, 774
147

96. Ball, N.M., Brunner, R.J., Myers, A.D., and Tcheng, D., "Robust Machine Learning Applied to
Astronomical Datasets. I. Star-Galaxy Classification of the Sloan Digital Sky Survey DR3 Using
Decision Trees", 2006, Astrophysical Journal, 650, 497
97. Lundgren Britt, PhD Thesis, University of Illinois at Urbana-Champaign (2009)
98. Lundgren, B, Brunner, R.J., et al. 2009, “Cross-Correlation of MgII quasar absorption systems and LRG
galaxies in SDSS DR5”, Astrophysical Journal, 698, 819.
99. Myers, A.D., White, M., and Ball, N., 2009, “Incorporating Photometric Redshift Probability Density
Information into Real-Space Clustering Measurements”, Monthly Notices of the Royal Astronomical
Society, 399, 2279.
100. Myers, A.D., Richards, G., Brunner, R.J., et al. 2008, “Quasar Clustering at 25 h-1 kpc from a Complete
Sample of Binaries”, Astrophysical Journal, 678, 635.
101. Myers, A.D., Brunner, R.J., and the SDSS collaboration, "First Measurement of the Clustering Evolution
of Photometrically Classified Quasars", 2006 Astrophysical Journal, 638, 622
102. Myers, A.D., Brunner, R.J., and the SDSS collaboration, "The Clustering Evolution of 300,000
Photometrically Classified Quasars--I. Luminosity and Redshift Evolution in Quasar Bias", 2007
Astrophysical Journal, 658, 85
103. Myers, A.D., Brunner, R.J., and the SDSS collaboration, "The Clustering Evolution of 300,000
Photometrically Classified Quasars--II. The Excess on Very Small Scales", 2007 Astrophysical Journal,
658, 99
104. Myers, A.D., Brunner, R.J., and the SDSS collaboration, "Quasar Clustering at 25 h-1 kpc from a
Complete Sample of Binaries", 2008 ApJ, 678, 635
105. Ross, A.J., Percival, W., and Brunner, R.J., 2010, “Evolution of the Clustering of Photometrically
Selected SDSS Galaxies”, Monthly Notices of the Royal Astronomical Society, in press.
106. Ross, Ashley, PhD Thesis, University of Illinois at Urbana-Champaign (2009)
107. Ross, A.J. and Brunner, R.J., 2009, “Halo-Model Analysis of the clustering of Photometrically Selected
Galaxies from the SDSS”, Monthly Notices of the Royal Astronomical Society, 399, 878.
108. Ross, A.J., Brunner, R.J., and Myers, A.D., “Normalization of the Matter Power Spectrum via Higher
Order Angular Correlations of Luminous Red Galaxies”, Astrophysical Journal, 682, 737 (2008).
109. Ross, A.J., Brunner, R.J., and Myers, A.D., “Higher Order Angular Galaxy Correlations in the SDSS:
Redshift and Color Dependence of Nonlinear Bias”, Astrophysical Journal, 665, 67 (2007).
110. Ross A.J., Brunner R.J. & Myers A.D., “Precision Measurements of Higher-Order Angular Galaxy
Correlations Using 11 Million SDSS Galaxies”, Astrophysical Journal, accepted (2006)
111. Scranton R., Ménard B., Richards G.T., Nichol R.C., Myers A.D., Jain B., Gray A., Bartelmann M.,
Brunner R.J., et al., “Detection of Cosmic Magnification with the Sloan Digital Sky Survey”,
Astrophysical Journal (2005), 633, 589
112. Strand, Natalie, PhD Thesis, University of Illinois at Urbana-Champaign (2009)
113. Strand, N.E., Brunner, R.J., and Myers, A.D., 2008, “AGN Environments in the Sloan Digital Sky Survey
I: Dependence on Type, Redshift, and Luminosity”, Astrophysical Journal, 688, 180
MCA93S005
114. Brown, B.P., Talk, “Rotating convection: Flows in plasma experiments”, Center for Magnetic Self-
Organization workshop on ‘Flow Driven Instabilities and Turbulence in High Beta Plasmas’, Univ.
Wisconsin, Madison, WI, Dec 2009.
115. Gordovskyy, M., Jain, R. & Hindman, B.W. 2009, “The role of mode mixing in the absorption of p-
modes”, Astrophys. J., 694, 1602-1609.
116. Gough, D. & Hindman, B.W. 2010, “Helioseismic detection of deep meridional flow”, Astrophys. J., in
press.
117. Miesch, M.S., Invited talk, “Mean flows and turbulent transport in global solar convection simulations”, in
Program on Dynamo Theory, KITP, Univ Calif Santa Barbara, CA, 2 July 08.
118. Miesch, M.S. & Hindman, B.W., 2010, “Gyroscopic pumping in the solar near-surface shear layer”,
Astrophys. J., to be submitted.
119. Miesch, M.S. & Toomre, J., 2009, “Turbulence, magnetism, and shear in stellar interiors”, Ann. Rev.
Fluid Mech., 41, 317–345, and on-line ARFM appendix, “Tachocline confinement”, 6pp.
120. Rast, M.P., Invited talk, “A visual interface for the analysis of turbulent structure statistics”, in Astronum-
2009, Chamonix, France, 3 July 2009.
148

121. Rast, M.P., Talk, “Precision photometric imaging of the Sun: What have we learned with the PSPT”, in
LASP colloquium, Boulder, Colorado, 10 Sep 2009.
122. Rast, M.P., Talk, “Is there such a thing as quiet-sun”, in SOHO-23, Northeast Harbor, Maine, 22 Sept
2009.
123. Toomre, J., Invited overview talk, “Advanced simulations: Global solar convection and dynamos”,
American Museum of Natural History, New York, NY, 7 Feb 08.
124. Toomre, J., Invited plenary talk, “Joys of highly turbulent solar convection and magnetic dynamos”, at
Intern. Conf. on Turbulent Mixing and Beyond, The Abdus Salam Intern. Center for Theor. Phys.,
Trieste, Italy, 27 July 2009.
125. Toomre, J., Invited talk, “New 3-D helioseismic inversion approaches for ring-diagram data”, HMI
Science Team Meeting, Stanford Univ., Palo Alto, CA, 10 Sep 2009.
126. Brun, A.S., Miesch, M.S. & Toomre, J., 2009, “Evolution of structures in magnetic dynamo action
realized in deep shells of convection”, Astrophys. J., in preparation.
127. Brun, A.S., Miesch, M.S. & Toomre, J., 2010, “Toward a 3-D nonlinear model of the solar interior”,
Astrophys. J., to be submitted.
128. Criscuoli, S., Harder, J.W., Rast, M.P. & Ermolli, I., Poster, “Unresolved magnetic elements and their
implication for cycle variations in the solar spectral irradiance”, in “Recent directions in astrophysical
quantitative spectroscopy and radiation hydrodynamics”, Boulder, Colorado, 30 Mar 2009.
129. DeRosa, M.L., Talk, “Nonlinear force-free magnetic field modeling of AR 10953: A critical assessment”,
given at Amer. Astron. Soc., Solar Phys. Div. meeting, Boulder, CO, June 2009.
130. DeRosa, M.L., Talk, “Aspects of helio- and asterospheric magnetic fields”, in workshop on “Magnetism
in Stars: origins, observations, opportunities”, CITA, Toronto, Oct 2009.
131. DeRosa, M.L., Talk, “Adventures in solar magnetic field modeling”, presentation in the SSL seminar,
Berkeley, CA, Mar 2010.
132. DeRosa, M.L., Talk, “A spherical harmonic analysis of the evolution of the photospheric magnetic field,
and consequences for the solar dynamo”, to appear in the Amer. Astron. Soc., Solar Phys. Div.
meeting, Miami, FL, May 2010.
133. Dikpati, M., Gilman, P.A., Cally, P.S., & Miesch, M.S., 2009, “Axisymmetric MHD instabilities in
solar/stellar tachoclines”, Astrophys. J., 692, 1421–1431.
134. Hindman, B.W., Haber, D.A. & Toomre, J. 2009, “Subsurface circulations within active regions,”
Astrophys. J., 698, 1749- 1760.
135. Hindman, B.W., Invited talk, “Subsurface Circulations within Active Regions”, HAO Seminar, High
Altitude Observatory, Boulder, Colorado, Apr 2009.
136. Hindman, B.W., Talk, “Subsurface Circulations Established by Active Regions”, SHINE meeting,
Wolfville, Nova Scotia, Aug 2009.
137. Hindman, B.W., Talk, “New 3-D inversion modules for ring-diagram data”, HMI Science Team Meeting,
Stanford, Sept 2009.
138. Jain, R., Hindman, B.W., Braun, D.C. & Birch, A.C. 2009, “Absorption of p modes by thin magnetic flux
tubes,” Astrophys. J., 695, 325-335.
139. Hurlburt, N.E., DeRosa, M.L., Augustson, K.C., & Toomre, J., 2010, “Effects of granulation upon largerscale
convection”, Proc. Third Hinode Science Meeting, ASP Conf. Ser. , submitted.
140. Miesch, M.S., Invited seminar, “Magnetism, and the inner turmoil of stars”, Center for Space Physics,
Boston University, Boston, MA, Feb 26 2009.
141. Miesch, M.S., Invited seminar, “Magnetism, and the inner turmoil of stars”, Dept. of Applied
Mathematics, University of Colorado, Boulder, CO, 16 Apr 2009.
142. Miesch, M.S., Invited seminar, “Magnetism, and the inner turmoil of stars”, Plasma Physics Seminar,
University of California San Diego, San Diego, CA, 20 Apr 2009.
143. Miesch, M.S., 2010, “Solar internal flows and dynamo action”, Ch. 5 of “Heliophysics III: evolving solar
activity and the climates of space and earth”, ed. Schrijver, C.J. & Siscoe, G.L., Cambridge Univ. Press,
in press.
144. Miesch, M.S., 2010, “The dynamo dialectic: an inside look at the current solar minimum”, in Proc.
“SOHO-23: understanding a peculiar solar minimum”, ASP Conference Series, in press.
145. Miesch, M.S., Invited talk, “Perspectives on the global solar dynamo; turbulence, convection, and
shear”, in the HELAS workshop: “Synergies between solar and stellar modeling”, Rome, Italy, 25 Jun
2009.
149

146. Miesch, M.S., Invited lecture, “Solar internal flows and dynamo action”, in “NASA heliophysics summer
school, Year 3: the Earth’s climate system and long-term solar activity”, NCAR, Boulder, CO, 23-24 Ju
2009.
147. Miesch, M.S., Invited talk, “Convection, shear, and magnetism in the sun”, in “Magnetism in stars:
origins, observations, and opportunities”, Canadian Institute for Theoretical Astrophysics, Toronto,
Canada, 28 Oct 2009.
148. Miesch, M.S., Invited talk, “WHI and the solar dynamo”, in “Whole heliosphere interval second
workshop”, NCAR, Boulder, CO, 10 Nov 2009.
149. Miesch, M.S., Invited lecture, “The internal rotation of the sun”, CMPD/CMSO Plasma Physics Winter
School, UCLA, Los Angeles, CA, 7 Jan 2010.
150. Miesch, M.S., Browning, M.K., Brun, A.S., Brown, B.P & Toomre, J., Talk, “Mean-field generation in
turbulent convective dynamos: the role of a tachocline”, in the Amer. Astron. Soc., Solar Phys. Div.
Meeting, Boulder, CO, 15 Jun 2009.
151. Miesch, M.S., Li, L., Sofia, S. & Brown, B.P., 2010, “Dynamo-induced variations in solar structure”,
Astrophys. J., to be submitted.
152. Rast, M.P., Session Leader, in “Active region magnetic fields: from the photosphere to the corona,”
SHINE Workshop, Wolfville, Nova Scotia, 6 Aug 2009.
153. Rempel, M., Talk, “Sunspot structures from radiative MHD simulations”, Max-Planck Institut f¨ur
Sonnensystemforschung, Freiburg, Germany, 27 Feb 2009.
154. Rempel, M., Invited talk, “Radiative MHD simulations of sunspot structure”, in HELAS workshop on
“Sunspots and helioseismology”, Berlin, Germany, 12 May 2009.
155. Rempel, M., Talk, “Numerical simulations of sunspot structure”, Amer. Astron. Soc., Solar Phys. Div.
Meeting, Boulder, CO, 14 Jun 2009
156. Rempel, M., Invited talk, “Radiative MHD simulations of sunspot structure”, in conference on “Natural
Dynamos”, Stara Lesna, Slovakia, 29 Aug 2009.
157. Rempel, M., Invited talk, “Numerical MHD modelling of sunspot structure”, colloquium talk, Instituto
Astrofisica de Canarias, Tenerife, Spain, 22 Oct 2009.
158. Rempel, M., Invited talk, “Numerical sunspot models - subsurface structure and helioseismic forward
modeling”, presented at the AGU Meeting, San Francisco, 14 Dec 2009.
159. Toomre, J., Seminar, “Turbulent dynamo processes in stars”, Astron. Dept., Bologna Univ., Bologna,
Italy, 29 July 2009.
160. Toomre, J. Invited colloquium, “Probing the sources of solar magnetism with heliosismology and
simulations”, Dept. of Astron., Yale Univ., New Haven, CT, 16 Oct 2009.
161. Toomre, J., Invited public presentation, “Touching the heart of magnetism in our nearest star”, Hayden
Planetarium, Amer. Museum of Natural History, New York, NY, 19 Oct 2009.
162. Augustson, K., Brown, B.P., Brun, A.S. & Toomre, J., 2010, “Convection and differential rotation in F-
type stars”, Astrophys. J., to be submitted.
163. Brown, B.P., Thesis talk, “Global-scale stellar dynamos and wreaths of magnetism in rapidly rotating
suns without tachoclines”, Amer. Astron. Soc. Meeting 213, Long Beach, CA, Jan 2009.
164. Brown, B.P., Invited talk, “Mysteries of magnetism in rapidly rotating suns without tachoclines”, Univ
Calif Los Angeles, Los Angeles, CA, Jan 2009.
165. Brown, B.P., Talk, “Dynamo action and wreaths of magnetism in rapidly rotating suns”, colloquium at
High Altitude Observatory, Boulder, CO, May 2009.
166. Brown, B.P., Talk, “Wreaths of magnetism built by dynamos without tachoclines”, in Amer. Astron. Soc.,
Solar Phys. Div. meeting 40, American Astronomical Society, Boulder, CO, May 2009.
167. Brown, B.P., Ph.D. thesis, University of Colorado, Aug 2009, “Convection and Dynamo Action in
Rapidly Rotating Suns”, 220 pp.
168. Brown, B.P., Talk, ”Dynamos in rapidly rotating suns”, in Center for Magnetic Self-Organization general
meeting, Madison, WI, Oct 2009.
169. Brown, B.P.,Talk, “Convection and dynamo action in rapidly rotating suns”, in ‘Magnetism in Stars:
Origins, Observations and Opportunities’, Canadian Institute for Theoretical Astrophysics (CITA),
Toronto, Canada, Oct 2009.
170. Brown, B.P.,Talk, “Simulations of magnetism in rapidly rotating stars”, 2010 NSF Astronomy &
Astrophysics Postdoctoral Fellows Symposium, Amer. Astron. Soc, meeting 215, Washington, D.C., Jan
2010.
150

171. Brown, B.P.,Talk, “Solar rotation II: from the sun to the stars”, Center for Multiscale Plasma Dynamics
and Center for Magnetic Self-Organization winter school, UCLA, Los Angeles, CA, Jan 2010.
172. Brown, B.P., Talk, “Simulations of the global-dynamo in stars like the sun”, Seminar in Plasma Physics,
Univ. Wisconsin, Madison, WI, Feb 2010.
173. Brown, B.P., Browning, M.K., Brun, A.S., Miesch, M.S. & Toomre, J., 2010, “Persistant wreaths of
magnetism in a rapidly rotating sun”, Astrophys. J., 711, 424–438 [Copy provided as Appendix E]
174. Brown, B.P., Browning, M.K., Brun, A.S., Miesch, M.S. & Toomre, J., 2010, “Magnetic cycles in a
convective dynamo simulation of a young solar-type star”, Astrophys. J., submitted.
175. Brown, B.P., Browning, M.K., Brun, A.S., Miesch, M.S. & Toomre, J., Poster, “Wreath-building dynamos
in rapidly rotating suns”, Amer. Astron. Soc. meeting 215, Washington, D.C., Jan 2010.
176. Browning, M.K., Invited talk, “Convection and magnetism in stars: How magnetic fields are built, and
what they can do”, in McGill / Universite de Montreal Joint Astrophysics Seminar, Montreal, Canada, 3
Feb 09.
177. Browning, M.K., Colloquium, “Dynamo action in low-mass stars”, MIT Astrophysics, Cambridge, MA, 7
Apr 09.
178. Browning, M.K., Invited talk, “Stellar dynamos”, in meeting on ‘Magnetic fields of fully convective
dwarfs’, Toulouse, France, 16 Apr 09.
179. Browning, M.K., Poster, “Rotation and magnetic fields in fully convective stars”, IAU General Assembly,
Meeting, Rio de Janeiro, 3-14 Aug 2009.
180. Browning, M.K., Invited talk, “Simulations of dynamos with and without tachoclines”, in meeting on
“Natural Dynamos”, Stara Lesna, Slovakia, 1 Sep 09.
181. Browning, M.K., Talk, “Global simulations of stellar dynamos”, in AGU Fall meeting, San Francisco, CA,
13 Dec 09.
182. Browning, M.K., Invited talk, “Momentum transport in stars”, Princeton Plasma Physics Laboratory,
Princeton, NJ, 19 Jan 2010.
183. Browning, M.K., Invited talk, “Convection and dynamo action in stars: problems and perspectives”,
Physics and Astronomy Seminar, Univ. Delaware, Newark, DE, 20 Apr 2010.
184. Browning, M.K., 2010, “Differential rotation in simulations of fully convective stars”, Astrophys. J.,
submitted.
185. Browning, M.K., Basri, G., Marcy, G.W., West, A.A. & Zhang, J., 2010, “Rotation and magnetic activity
in a sample of M-dwarfs,” Astron. J., 139, 504–518.
186. Brun, A.S. & Palacios, A, 2009, “Numerical simulations of a rotating red giant star. I. 3–D models of
turbulent convection and associated mean flows”, Astrophys. J., submitted.
187. Featherstone, N.A., Talk, “Fossil fields in the convective cores of A-stars”, in “Magnetism in stars:
origins, observations, and opportunities”, CITA, Toronto, Canada, 28 Oct 2009.
188. Featherstone, N.A., Browning, M.K., Brun, A.S. & Toomre, J., 2009, “Effects of fossil fields on
convective core dynamos in A-type stars”, Astrophys. J., 705,1000–1018. [Copy provided as Appendix
D]
189. Reiners, A., Basri, G., & Browning, M., 2009, “Saturation of magnetic flux at low Rossby numbers: the M
stars”, in ‘Cool Stars, Stellar Systems, and the Sun: Proc. 15th Cambridge Workshop’, AIP v. 1094, 728.
190. Reiners, A., Scholz, A., Eisloffel, J., Hallinan, G., Berger, E., Browning, M., Irwin, J., Kuker, M., & Matt,
S., 2009 “The rotation-magnetic field relation”, in ‘Cool Stars, Stellar Systems, and the Sun: Proc. 15th
Cambridge Workshop’, AIP v. 1094, 250.
191. Toomre, J., Invited colloquium, “Joys of turbulent convection and dynamos in both stellar envelopes and
cores”, Astrophys. Dept., Amer. Museum of Natural History, New York, NY, 19 Oct 2009.
MCA06N030
192. Pereira, M. & Bryan, G. L., 2010, “Tidal Torquing of Elliptical Galaxies in Cluster Environments”, ApJ, in
press
193. Molnar, S. M., Chiu, I. -., Umetsu, K., Chen, P., Hearn, N., Broadhurst, T., Bryan, G., & Shang, C. 2010,
“Testing Strict Hydrostatic Equilibrium in Abell 1689”, ApJ, submitted (arXiv:1002.4691)
194. Shang, C., Bryan, G. L., & Haiman, Z. 2010, “Supermassive black hole formation by direct collapse:
keeping protogalactic gas H2 free in dark matter haloes with virial temperatures Tvir > 104 K”, MNRAS,
402, 1249
195. Tonnesen, S., & Bryan, G. L. 2010, “The Tail of the Stripped Gas that Cooled: H I, H_, and X-ray
Observational Signatures of Ram Pressure Stripping”, ApJ, 709, 1203
151

196. Mesinger, A., Bryan, G. L., & Haiman, Z. 2009, “Relic HII regions and radiative feedback at high
redshifts”, MNRAS, 399, 1650
197. Joung, M. R., Mac Low, M.-M., & Bryan, G. L. 2009, “Dependence of Interstellar Turbulent Pressure on
Supernova Rate”, ApJ, 704, 137
198. Molnar, S. M., Hearn, N., Haiman, Z., Bryan, G., Evrard, A. E., & Lake, G. 2009, “Accretion Shocks in
Clusters of Galaxies and Their SZ Signature from Cosmological Simulations”, ApJ, 696, 1640
MCA06N053
199. “The correlation function of quasar: evidence for mergers” DeGraf, C., Di Matteo, T., & Springel, V.
2010, submitted
200. “Galaxy morphology, kinematics and clustering in a hydrodynamic simulation of a cold dark matter
universe” Croft,R., Di Matteo,T, Springel, V. & Hernquist, L., 2009 Monthly Notices of the Royal
Astronomical Society, Volume 400, Issue 1, pp. 43-67
201. “Hydrogen Self Shielding in a Quasar Proximity Zone”, Altay, G., Croft, R. A. C. & Di Matteo, Tiziana
2009, eprint arXiv:0910.5233
202. “Faint-end Quasar Luminosity Functions from Cosmological Hydrodynamic Simulations” DeGraf, C., Di
Matteo, T., & Springel, V. 2009, eprint arXiv:0910.1843
203. “Simulations of the Sunyaev-Zel’dovich effect from quasars” Chatterjee, S.; Di Matteo, T.; Kosowsky, A.;
Pelupessy, I. 2008, Monthly Notices of the Royal Astronomical Society, Volume 390, Issue 2, pp. 535-
544
Bioengineering and Environmental Systems
BCS090007
204. Matthew S. Starosta and Andrew K. Dunn. Three-Dimensional computation of focused beam
propagation through multiple biological cells. Optics Express, 17(15):12455–12469, July 2009.
MCA03S012
205. Cui, Y., K. B. Olsen, T. H. Jordan, K. Lee, J. Zhou, P. Small, D. Roten, G. Ely, D.K. Panda, A.
Chourasia, J. Levesque, S. M. Day, P. Maechling (2010), Scalable Earthquake Simulation on Petascale
Supercomputers, Proceedings of the SC10 International Conference for HPC, Networking and Analysis.
(Submitted for publication).
206. Taborda, R., J. López, H. Karaoglu, J. Urbanic and J. Bielak (2010), Speeding Up Wave Propagation for
Large-Scale Earthquake Simulations, Proceedings of the SC10 International Conference for HPC,
Networking and Analysis. (Submitted for publication).
207. Scott Callaghan, Ewa Deelman, Dan Gunter, Gideon Juve, Philip Maechling, Christopher Brooks, Karan
Vahi, Kevin Milner, Robert Graves, Edward Field, David Okaya, Thomas Jordan (2010), Scaling up
workflow-based applications, J. Computer System Science, special issue on scientific workflows, in
press.
208. Graves, R., T. Jordan; S. Callaghan; E. Deelman; E. Field; G. Juve; C. Kesselman; P. Maechling; G.
Mehta; K. Milner; D. Okaya; P. Small; and K. Vahi (2010). CyberShake: A Physics-Based Seismic
Hazard Model for Southern California, Pure Applied Geophys., accepted for publication
209. Olsen, K.B., and J.E. Mayhew (2010). Goodness-of-fit Criteria for Broadband Synthetic Seismograms
with Application to the 2008 Mw5.4 Chino Hills, CA, Earthquake, Seism. Res. Lett., accepted for
publication.
210. Cruz-Atienza, V.M. and K.B. Olsen (2010). Supershear Mach-Waves Expose the Fault Breakdown Slip,
Tectonophysics, in revision for the special volume on 'Supershar Earthquakes'.
211. Rojas, O., E. Dunham, S.M. Day, L.A. Dalguer, and J.E. Castillo (2009). Finite difference modeling of
rupture propagation with strong velocity-weakening friction, Geophys J. Int., in review.
212. Bielak, J., Graves, R., Olsen, K., Taborda, R., Ramirez-Guzman, L., Day, S., Ely, G., Roten, D., Jordan,
T., Maechling, P., Urbanic, J., Cui, Y. and Juve, G.: The ShakeOut Earthquake Scenario: Verification of
Three Simulation Sets, submitted to Geophysical Journal International, March, 2009.
213. Callaghan, S., P. Maechling, E. Deelman, K. Vahi, G. Mehta, G. Juve, K. Milner, R. Graves, E. Field, D.
Okaya, D. Gunter, K. Beattie, and T. Jordan (2008), 'Reducing Time-to-Solution Using Distributed High-
Throughput Mega-Workflows -- Experiences from SCEC CyberShake', Fourth IEEE International
Conference on eScience, IEEE, Indianapolis, IN,
152

214. Cui, Y., Chourasia, A., Moore, R., Olsen, K., Maechling, P., Jordan, T., The TeraShake Computational
Platform, Advances in Geocomputing, Lecture Notes in Earth Sciences 119, DOI 10.1007/978-3-540-
85879-9_7, pp229-278, editor H. Xing, Springer-Verlag Berlin Heidelberg, 2009.
215. Dalguer, L. A., H. Miyake, S. M. Day, and K. Irikura (2008) Calibrated surface and buried dynamic
rupture models constrained with statistical observations of past earthquakes, Bull. Seism. Soc. Am ,
Vol. 98, doi: 10.1785/0120070134, pp. 1147-1161
216. Day, S. M., R. W. Graves, J. Bielak, D. Dreger, S. Larsen, K. B. Olsen, A.Pitarka, and L. Ramirez-
Guzman (2008). Model for basin effects on long-period response spectra in southern California,
Earthquake Spectra (in press).
217. Doser, D.I., K.B. Olsen, F.F. Pollitz, R.S. Stein, and S. Toda (2009). The 1911 M~6.6 Calaveras
earthquake: Source parameters and the role of static, viscoelastic and dynamic Coulomb stress
changes imparted by the 1906 San Francisco earthquake, Bull. Seis. Soc. Am. 99, 1746-1759.
218. Duan, B., and S.M. Day (2008). Inelastic strain distribution and seismic radiation from rupture of a fault
kink J. Geophys. Res. , in review.
219. Ely, G., S.M. Day, and J-B. Minster (2008) Dynamic rupture models for the southern San Andreas fault,
Bull. Seism. Soc. Am., in review.
220. Ely, G. P., S. M. Day, and J. B. Minster (2008) A support-operator method for viscoelastic wave
modeling in 3D heterogeneous media, Geophys. J. Int. , 172, doi: 10.1111/j.1365-246X.2007.03633.x,
331-344
221. Graves, R. W., The seismic response of the San Bernardino Basin region during the 2001 Big Bear
Lake earthquake, Bull. Seismol. Soc. Am., 98, 241– 252, doi:10.1785/0120070013, 2008.
222. Graves, R.W., B. Aagaard, K. Hudnut, L. Star, J. Stewart & T. H. Jordan (2008), Broadband simulations
for Mw 7.8 southern San Andreas earthquakes: Ground motion sensitivity to rupture speed, Geophys.
Res. Lett., submitted, Sept. 2008.
223. Graves, R., S. Callaghan, E. Deelman, E. Field, N. Gupta, T. H. Jordan, G. Juve, C. Kesselman, P.
Maechling, G. Mehta, D. Meyers, D. Okaya and K. Vahi (2008) Physics Based Probabilistic Seismic
Hazard Calculations for Southern California,14th World Conference on Earthquake Engineering,
October, 2008, Beijing China
224. Ma, S., G.A. Prieto, and G. C. Beroza, Testing Community Velocity Models for Southern California
Using the Ambient Seismic Field, Bull. Seismol. Soc. Am., 98, 2694-2714, DOI: 10.1785/0120080947,
2008
225. Maechling, P., E. Deelman, Y. Cui (2009), Implementing Software Acceptance Tests as Scientific
Workflows, Proceedings of the IEEE International Parallel & Distributed Processing Symposium 2009,
Las Vegas Nevada, July, 2009 (in press)
226. Olsen, K. B., Dalguer, L., Day, S., Cui, Y., Zhu, J., Cruz, V.M., Roten, D., Mayhew, J., Maechling, P.,
Jordan, T., Chourasia, A. and Okaya, D. ShakeOut-D: Ground Motion Estimates Using an Ensemble of
Large Earthquakes on the Southern San Andreas Fault With Spontaneous Rupture Propagation,
Geophysical Research Letters, doi:10.1029/2008GL036832, in press, 2009.
227. Olsen, K.B., W.J. Stephenson, and Andreas Geisselmeyer (2008) 3D Crustal Structure and Long-period
Ground Motions From a M9.0 Megathrust Earthquake in the Pacific Northwest Region, Jour. Seismol.
DOI 10.1007/s10950-007-9082-y.
Chemical and Thermal Systems
CTS050016
228. Broadbelt, L.J. and Pfaendtner, J., “Lexicography of Kinetic Modeling of Complex Systems”, AIChE J.,
2005, 51(8), 2112-2121.
229. Pfaendtner, J., Yu, X. and Broadbelt, L.J., “Quantum Chemical Investigation of Low-Temperature
Intramolecular Hydrogen Transfer of Hydrocarbons”, J. Phys. Chem. A, 2006, 110(37), 10863-10871.
230. Pfaendtner, J. and Broadbelt, L.J., “Elucidation of Structure-Reactivity Relationships in Hindered
Phenols Via Quantum Chemistry and Transition State Theory”, Chem. Eng. Sci., 2007, 62(18-20),
5232-5239.
231. Pfaendtner, J., Yu, X. and Broadbelt, L.J., “The 1D Hindered Rotor Approximation”, Theoretical
Chemistry Accounts, 2007, 118(5-6), 881-898.
232. Pfaendtner, J. and Broadbelt, L.J., “Uncovering the Cause of Contrathermodynamic Behavior in
Intermolecular Hydrogen Transfer of Alkylperoxy Radicals, ROO• + R’H”, ChemPhysChem, 2007, 8(13),
1969-1978.
153

233. Yu, X., Pfaendtner, J. and Broadbelt, L.J., “Ab Initio Study of Acrylate Polymerization Reactions: Methyl
Methacrylate and Methyl Acrylate Propagation”, J. Phys. Chem. A, 2008, 112(29), 6772-6782.
234. Yu, X., Levine, S.E. and Broadbelt, L.J., “Kinetic Study of the Copolymerization of Methyl Methacrylate
and Methyl Acrylate using Quantum Chemistry”, Macromolecules, 2008, 41(21), 8242-8251.
235. Adamczyk, A.J., Reyniers, M.-F., Marin, G.B. and Broadbelt, L.J., “Exploring 1,2-Hydrogen Shift in
Silicon Nanoparticles: Reaction Kinetics from Quantum Chemical Calculations and Derivation of
Transition State Group Additivity (TGSA) Database”, J. Phys. Chem. A, 2010, in press.
236. Adamczyk, A.J., Reyniers, M.-F., Marin, G.B. and Broadbelt, L.J., “Kinetics of Substituted Silylene
Addition and Elimination in Silicon Nanocluster Growth Captured by Group Additivity”, ChemPhysChem,
2010, in press.
237. Adamczyk, A.J., Reyniers, M.-F., Marin, G.B. and Broadbelt, L.J., “Hydrogenated Amorphous Silicon
Nanostructures: Novel Structure-Reactivity Relationships for Cyclization and Ring Opening in the Gas
Phase”, Theoretical Chemistry Accounts, 2010, submitted.
CTS060035
238. Qian, D., Sankaranarayanan, K., Sundaresan, S., Kontomaris, K., and McLaughlin, J.B., “Simulation of
bubble breakup dynamics in homogeneous turbulence,” Chemical Engineering Communications 193,
1038-1063 (2006).
239. Jia, X., McLaughlin, J.B., and Kontomaris, K., “Lattice Boltzmann simulations of contact line motion on
uniform surfaces,” Mathematics and Computers in Simulation 72, 156-159 (2006).
240. Derksen, J., Kontomaris, K., McLaughlin, J.B., and Van den Akker, H.E.A., “Large eddy simulation of
flow dynamics and mixing in an industrial crystallizer geometry,” IChemE Chemical Engineering
Research and Design 85, 169-179 (2007).
241. Jia, X., J.B. McLaughlin, G. Ahmadi, and K. Kontomaris, “Lattice Boltzmann simulations of contact line
pinning,” International Journal of Modern Physics C 18, 595-601 (2007).
242. Jia, X., J.B. McLaughlin, and K. Kontomaris, “Lattice Boltzmann simulations of flows with fluid-fluid
interfaces,” Asia-Pacific Journal of Chemical Engineering 3, 124-143 (2008).
243. Jia, X. J.B. McLaughlin, and K. Kontomaris, “Lattice Boltzmann simulations of drops colliding with solid
surfaces,” European Journal of Physics Special Topics 171, 105-112 (2009).
CTS080034
244. A. D. Naiman, S. K. Lele, F. Ham, J. T. Wilkerson, and M. Z. Jacobson. Large eddy simulation of
persistent contrails in wind shear and atmospheric turbulence. American Physical Society Division of
Fluid Dynamics 62nd Annual Meeting, No. BS-8, Minneapolis, MN, 22-24 November 2009a.
245. A. D. Naiman, S. K. Lele, F. Ham, J. T. Wilkerson, and M. Z. Jacobson. Large eddy simulation of
persistent contrails. 2nd International Conference on Transport, Atmosphere, and Climate, Aachen,
Germany and Maastricht, Netherlands, 22-25 June 2009b.
246. A. D. Naiman, S. K. Lele, J. T. Wilkerson, and M. Z. Jacobson. Modeling persistent contrails in a large
eddy simulation and a global climate model. American Geophysical Union 2009 Fall Meeting, No.
A51C-0119, San Francisco, CA, 14-18 December 2009c.
247. A. D. Naiman, S. K. Lele, F. Ham, J. T. Wilkerson, and M. Z. Jacobson. Large eddy simulation of
persistent contrails. Physics of Fluids, in preparation, 2010a.
CTS080042
248. Voronov, R.S., VanGordon, S., Sikavitsas, V.I., and D.V. Papavassiliou, “Efficient Lagrangian scalar
tracking method for reactive local mass transport simulation through porous media,” submitted to Int. J.
of Numerical Methods in Fluids
249. VanGordon, S., Voronov, R.S., Blue, T.B., Shambaugh, R.L., Papavassiliou, D.V., and V.I. Sikavitsas,
V.I., and D.V. Papavassiliou, “Effects of scaffold architecture on preosteoblastic cultures under
continuous fluid shear,” in press, Ind. Eng. Chem.. Res., 2010.
250. Voronov, R., VanGordon, S., Sikavitsas, V.I., and D.V. Papavassiliou, “Local velocity and stress fields
within 3D porous scaffolds used in perfusion bioreactors for bone tissue growth,” in press, J. of
Biomechanics, 2010, DOI: 10.1016/j.jbiomech.2010.01.007
251. Duong, H.M., Yamamoto, N., Bui, K., Papavassiliou, D.V., Maruyama, S., B.L. Wardle, “Morphology
effects on non-isotropic thermal conduction of aligned single- and multi-walled carbon nanotubes in
polymer nanocomposites,” accepted, Journal of Phys. Chemistry C, 2010.
252. Srinivasan, C., and D.V. Papavassiliou, “Backwards and forwards dispersion of a scalar in turbulent wall
flows,” Int. J. Heat Mass Transf. 53, 1023-1035, 2010, DOI: 10.1016/j.ijheatmasstransfer.2009.11.008
154

253. Le, P.M., and D.V. Papavassiliou, “A physical picture of the mechanism of turbulent heat transfer from
the wall,” Int. J. Heat Mass Transf., 52(21-22), 4873-4882, 2009.
254. Duong, H.M., Yamamoto, N., Papavassiliou, D.V., Wardle, B.L., and S. Maruyama, “Inter-Carbon
Nanotube contact in thermal transport of controlled-morphology polymer nanocomposites,”
Nanotechnology, 20(15), 155702, 2009.
255. VanGordon, S.B., Voronov, R.S., Blue, T.B., Shambaugh, R.L., Papavassiliou, D.V., and V.I.
Sikavitsas, “Influence of Polymer Scaffold Architecture on the Shear Stress Distribution and Dynamic
Culture of Preosteoblastic Cells,” Society of Biomaterials, 2010 Annual Meeting and Exposition,
Seattle, WA, April 2010.
256. Voronov, R., VanGordon S., Sikavitsas V.I., and D.V. Papavassiliou, “A Lagrangian Particle Method as
an Alternative to Solute Lattice Boltzmann: Investigation of Mass Transfer in Biological Porous,” paper
148b, AIChE Annual Conference, Nashville, November, 2009.
257. Voronov, R., VanGordon S., Landy, B., Sikavitsas V.I., and D.V. Papavassiliou, “Understanding the
Tissue Growth Process via Fluid Shear and Nutrient Transport Simulation in 3D Porous Scaffolds Used
in a Perfusion Bioreactor,” paper 55f, AIChE Annual Conference, Nashville, November, 2009.
258. Voronov, R., VanGordon S., Landy, B., Sikavitsas V.I., and D.V. Papavassiliou, “Insight into tissue
growth process via local shear stress and nutrient transport simulation in 3D porous scaffolds cultured
in a perfusion bioreactor,” paper 485u, AIChE Conference, Nashville, November, 2009.
259. Papavassiliou, D.V., Voronov, R., Sikavitsas, V.I., and S. VanGordon, “Flow and nutrient transport
through porous scaffolds used for the culture of bone cells in perfusion bioreactors,” paper P15.00011,
March 2009 meeting of the American Physical Society, Pittsburgh, PA, March, 2009.
260. Duong, H.M., Yamamoto, N., Panzer, M., Marconnet A., Goodson, K., Papavassiliou, D.V., Maruyama,
S., and B.L. Wardle “Thermal Properties of Vertically Aligned Carbon Nanotube-Nanocomposites
Boundary Resistance and Inter- Carbon Nanotube Contact: Experiments and Modeling,” paper
J26.00002, March 2009 meeting of the APS, Pittsburgh, PA, March, 2009.
261. Duong, H.M., Papavassiliou, D.V., Yamamoto, N., and B.L. Wardle “Computational study of the thermal
conductivities of single- and multi-walled carbon nanotube-polymer composites with inter-carbon
nanotube contact,” 2009 MRS Spring Meeting, San Francisco, April 2009.
262. Voronov, R., VanGordon S., Sikavitsas V.I., and D.V. Papavassiliou, “Using a Particle Method to
Investigate Mass Transfer in Biological Porous Media,” paper 679f, AIChE Annual Conference,
Philadelphia, November, 2008.
CTS090004
263. Q. Zhang and D. J. Bodony, “Numerical Simulation of Two-Dimensional Acoustic Liners with High
Speed Grazing Flow,” submitted to the AIAA J.
264. J. B. Freund, “Adjoint-based optimization for understanding and suppressing jet noise,” Proceedings of
the IUTAM Symposium on Computational Aero-Acoustics for Aircraft Noise Prediction, (2010).
265. D. J. Bodony, G. Zagaris, A. Reichter, Q. Zhang, “Aeroacoustic predictions in complex geometries”,
Proceedings of the IUTAM Symposium on Computational Aero-Acoustics for Aircraft Noise Prediction
(2010).
266. G. Zagaris, D. J. Bodony, M. D. Brandyberry, M. T. Campbell, E. Shaffer & J. B. Freund, “A collision
detection approach to chimera grid assembly for high fidelity simulations of turbofan noise,” AIAA Paper
2010- 0836 (2010).
267. J. Kim, D. J. Bodony, and J. B. Freund, “A high-order, overset mesh algorithm for adjoint-based
optimization for aeroacoustics control,” AIAA Paper 2010-3818 (2010).
CTS090079
268. M. C. Hagy, S. J. Fitzwater, T. H. Pierce, M. Schure, A. V. Popov and R. Hernandez, “Effects of interparticle
energetic barriers on the dynamics of colloidal aggregation,” in preparation.
269. M. C. Hagy, S. J. Fitzwater, T. H. Pierce, M. Schure, A. V. Popov and R. Hernandez, “On the
structuring of colloidal suspensions under shear.” aggregation,” in preparation.
270. G. Ozer, E. Valeev, S. Quirk and R. Hernandez, “The unfolding dynamics of Neuropeptide Y:
accelerated molecular dynamics and adaptive steered molecular trajectories,” in preparation.
155

Chemistry
CHE040030
271. J.W. Medlin, C.M. Horiuchi, M. Rangan, “Effects of ring structure on the reaction pathways of cyclic
esters and ethers on Pd(111)”, Topics in Catalysis, in press (2010).
272. C.M. Horiuchi, M. Rangan, B. Israel, J.W. Medlin, “Adsorption and ring-opening of 2,5(H)-furanone on
the (111) surfaces of Pd and Pt: Implication for selectivity in reactions of unsaturated cyclic
oxygenates”, Journal of Physical Chemistry C, 113 (2009) 14900-14907.
273. M.T. Schaal, M.P. Hyman, M. Rangan, S. Ma, C.T. Williams, J.R. Monnier, and J.W. Medlin,
“Theoretical and experimental studies of Ag-Pt Interactions for supported Ag-Pt bimetallic catalysts”,
Surface Science 603 (2009) 690-696.
274. D.C. Kershner, M.P. Hyman, J.W. Medin, “DFT study of the oxidation of silicon on Pd(111) and
Pt(111)”, Surface Science 602 (2008) 3603-3610.
275. M.P. Hyman and J.W. Medlin, “Mechanistic Studies of Electrocatalytic Reactions”, in Catalysis (volume
20) pp. 309-337, edited by J.J. Spivey and K.M. Dooley. Royal Society of Chemistry (2007).
276. M.P. Hyman, J.W. Medlin, “The Effects of Electronic Structure Modifications on the Adsorption of
Oxygen Reduction Reaction Intermediates on Model Pt(111)-Alloy Surfaces”, Journal of Physical
Chemistry C 111 (2007) 17052-17060.
277. M.P. Hyman, B.T. Loveless, J.W. Medlin, “A Density Functional Theory Study of H2S Decomposition
on the (111) Surfaces of Model Pd-alloys”, Surface Science 601 (2007) 5383-5394.
278. M.P. Hyman, J.W. Medlin, “A Mechanistic Study of the Electrochemical Oxygen Reduction Reaction on
Pt(111) Using Density Functional Theory”, Journal of Physical Chemistry B 110 (2006) 15338-15344.
CHE080052
279. S.L. Fiedler and A. Violi, “Simulation of Nanoparticle Permeation through a Lipid Membrane,”
Biophysical Journal, 2010, in press.
280. S. Chung and A. Violi, “Peri-condensed aromatics with aliphatic chains as key intermediates for the
nucleation of aromatic hydrocarbons,” Proc. Combust. Inst. 33, 2010, accepted.
281. S.L. Fiedler and A. Violi, “Interactions of carbonaceous nanoparticles with a lipid bilayer membrane: A
molecular study,” Proc. 6th US Nat. Comb. Meeting, Ann Arbor, Michigan, May. 2009.
282. S. Chung and A. Violi, “Molecular dynamics study of nanoparticle nucleation at high temperature,” Proc.
6th US Nat. Comb. Meeting, Ann Arbor, Michigan, May. 2009.
CHE080060
283. H. Dong, M.R. Nimlos, D.K. Johnson, M.E. Himmel and X. Qian, ‘The Role of Water in β-D-xylose
Condensation Reactions’, J. Phys. Chem. A 113, (2009) 8577-8585.
284. J-L Li and X. Qian, ‘Free Energy Landscape for Glucose Condensation Reactions’ (manuscript to be
submitted in June 2010). J. Phys. Chem.
285. D-J Liu and X. Qian, ‘Mechanisms and Energetics of Glucose Conversion to HMF’ (manuscript to be
submitted in June 2010). J. Phys. Chem.
CHE100070
286. Ellison, M. D.; Morris, S. T.; Sender, M. R.; Padgett, N. E.; Brigham, J. J. Phys. Chem. C 2007, 111,
18127.
CHE100074
287. Sun, X.; Lee, J. K. "The Stability of DNA Duplexes Containing Hypoxanthine (Inosine): Gas versus
Solution Phase and Biological Implications," J. Org. Chem., 2010, ASAP article published on Web
February 25, 2010 (DOI: 10.1021/jo9023683)
288. Zhachkina, A.; Lee, J. K. "Uracil and Thymine Reactivity in the Gas Phase: The SN2 Reaction and
Implications for Electron Delocalization in Leaving Groups," J. Am. Chem. Soc. 2009, 131, 18376-
18385.
289. Zhachkina, A.; Liu, M.; Sun, X.; Amegayibor, F. S.; Lee, J. K. "Gas-Phase Thermochemical Properties
of the Damaged Base O-Methylguanine versus Adenine and Guanine," J. Org. Chem. 2009, 74, 7429-
7440.
290. Tantillo, D. J.; Lee, J. K. "Reaction Mechanisms: Pericyclic Reactions," Annu. Rep. Prog. Chem., Sect.
B, 2009, 105, 285-309.
156

291. Liu, M.; Li, T.; Amegayibor, F. S.; Cardoso, D. S.; Fu, Y.; Lee, J. K. "Gas-Phase Thermochemical
Properties of Pyrimidine Nucleobases," J. Org. Chem., 2008, 73, 9283-9291.
292. Rozenberg, A; Lee, J. K. "Theoretical Studies of the Quinolinic Acid to Nicotinic Acid Mononucleotide
Transformation," J. Org. Chem., 2008, 73, 9314-9319.
293. Wepukhulu, W. O.; Smiley, V. L.; Vemulapalli, B.; Smiley, J. A.; Phillips, L. M.; Lee, J. K. "Evidence for
Pre-Protonation in the Catalytic Reaction of OMP Decarboxylase: Kinetic Isotope Effects using the
Remote Double Label Method," Organic and Biomolecular Chemistry, 2008, 6, 4533-4541 (ALSO
FEATURED ON COVER).
294. Tantillo, D. J.; Lee, J. K. "Reaction Mechanisms: Pericyclic Reactions," Annu. Rep. Prog. Chem., Sect.
B, 2008, 104, 260-283.
295. Liu, M.; Xu, M.; Lee, J. K. "The Intrinsic Reactivity of Ethenoadenine and Mechanism for Excision from
DNA," J. Org. Chem., 2008, 73, 5907-5914.
CHE100082
296. I. Y. Lyubimov, J. McCarty, A. Clark, and M. G. Guenza “Analytical Rescaling of Polymer Dynamics
from Mesoscale Simulations” (submitted to J. Chem. Phys.)
297. J. McCarty, and M. G. Guenza “Multiscale Modeling of Polymer Mixtures” (in preparation).
298. I. Y. Lyubimov, and M. G. Guenza “Rescaling of Dynamical Properties from Mesoscale Simulations” (in
preparation)
299. J. McCarty, I, Y, Lyubimov, and M. G. Guenza “Effective Soft-Core Potentials and Mesoscopic
Simulations of Binary Polymer Mixtures” Macromolecules, 2010 (accepted).
MCB080004
300. Fatmi, M.Q, Ai, R., and Chang, C-E. A., Synergistic Regulation and Ligand Induced Conformational
Changes of Tryptophan Synthase Biochemistry, 2009. 48, 9921-9931.
301. 170 - Driving forces behind allosteric and synergistic regulations in tryptophan synthase Authors: Dr.
Muhammad Q Fatmi PhD, Rizi Ai, Prof. Chia-en A Chang PhD Division: COMP: Division of Computers
in Chemistry Date/Time: Tuesday, March 23, 2010 - 03:00 PM Session Info: Molecular Mechanics
(01:30 PM - 05:45 PM)
302. 252 - Discovering drug candidates for the alpha-subunit of tryptophan synthase: From docking,
moleculardynamics simulations, to experiments Authors: Dr. Muhammad Q Fatmi PhD, Wai M Thant,
So-Ning Lee, Dr. Dimitri Niks PhD, Prof. Michael F Dunn PhD, Prof. Chia-en A Chang PhD Division:
COMP: Division of Computers in Chemistry Date/Time: Tuesday, March 23, 2010 - 06:00 PM Session
Info: Poster Session (06:00 PM - 08:00 PM)
MCA94P017
303. Ilan, B., E. Tajkhorshid, K. Schulten, and G. A. Voth. 2004. The mechanism of proton exclusion in
aquaporin channels. Proteins 55:223-228.
304. Jeon, J. G., and G. A. Voth. 2005. The dynamic stress responses to area change in planar lipid bilayer
membranes. Biophys. J. 88:1104-1119.
305. Izvekov, S., and G. A. Voth. 2005. A multiscale coarse-graining method for biomolecular systems. J.
Phys. Chem. B 109:2469-2473.
306. Izvekov, S., and G. A. Voth. 2005. Effective force field for liquid hydrogen fluoride from ab initio
molecular dynamics simulation using the force-matching method. J. Phys. Chem. B 109:6573-6586.
307. Tepper, H. L., and G. A. Voth. 2005. Protons may leak through pure lipid bilayers via a concerted
mechanism. Biophys. J. 88:3095-3108.
308. Xu, J., and G. A. Voth. 2005. Computer simulation of explicit proton translocation in cytochrome c
oxidase: the D-pathway. Proc. Natl. Acad. Sci. USA. 102:6795-6800.
309. Wu, Y., and G. A. Voth. 2005. A computational study of the closed and open states of the influenza A
M2 proton channel. Biophys. J 89:2402-2411.
310. Ka, B. J., and G. A. Voth. 2004. Combining the semiclassical initial value representation with centroid
dynamics. J. Phys. Chem. B 108:6883-6892.
311. Yan, T. Y., C. J. Burnham, M. G. Del Popolo, and G. A. Voth. 2004. Molecular dynamics simulation of
ionic liquids: The effect of electronic polarizability. J Phys Chem B 108:11877-11881.
312. Chu, J. W., and G. A. Voth. 2005. Allostery of actin filaments: Molecular dynamics simulations and
coarse-grained analysis. Proc. Natl. Acad. Sci. U. S. A. 102:13111-13116.
157

313. Izvekov, S., M. Parrinello, C. J. Burnham, and G. A. Voth. 2004. Effective force fields for condensed
phase systems from ab initio molecular dynamics simulation: A new method for force-matching. J.
Chem. Phys. 120:10896-10913.
314. Izvekov, S., and G. A. Voth. 2005. Ab initio molecular-dynamics simulation of aqueous proton salvation
and transport revisited. J. Chem. Phys. 123:044505.
315. Izvekov, S., and G. A. Voth. 2005. Multiscale coarse-graining of liquid state systems. J. Chem. Phys.
123:134105-134117.
316. Blood, P. D., G. S. Ayton, and G. A. Voth. 2005. Probing the molecular-scale lipid bilayer response to
shear flow using nonequilibrium molecular dynamics. J. Phys. Chem. B 109:18673-18679.
317. Iyengar, S. S., M. K. Petersen, T. J. F. Day, C. J. Burnham, V. E. Teige, and G. A. Voth. 2005. The
properties of ion-water clusters. I. The protonated 21-water cluster. J. Chem. Phys. 123:084309.
318. Ayton, G. S., and G. A. Voth. 2004. Mesoscopic lateral diffusion in lipid bilayers. Biophys. J. 87:3299-
3311.
319. Chang, R., G. S. Ayton, and G. A. Voth. 2005. Multiscale coupling of mesoscopic- and atomistic-level
lipid bilayer simulations. J. Chem. Phys. 122:224716.
320. Izvekov, S., and G. A. Voth. 2006. Multiscale coarse-graining of mixed phospholipid/cholesterol
bilayers. J. Chem. Theor. Comp. 2:637-648.
321. Shi, Q., and G. A. Voth. 2005. Multi-scale modeling of phase separation in mixed lipid bilayers. Biophys.
J. 89:2385-2394.
322. Wang, Y. T., and G. A. Voth. 2005. Unique spatial heterogeneity in ionic liquids. J. Am. Chem. Soc.
127:12192-12193.
323. Wang, Y. T., S. Izvekov, T. Y. Yan, and G. A. Voth. 2006. Multiscale coarse-graining of ionic liquids. J.
Phys. Chem. B 110:3564-3575.
324. Chen, H., Y. Wu, and G. A. Voth. 2006. Origins of proton transport behavior from selectivity domain
mutations of the aquaporin-1 channel. Biophys. J. 90:L73-L75.
325. Xu, J., and G. A. Voth. 2006. Free energy profiles for H+ conduction in the D-pathway of cytochrome c
oxidase: a study of the wild type and N98D mutant enzymes. Biochim. Biophys. Acta. 1757:852-859.
326. Shi, Q., S. Izvekov, and G. A. Voth. 2006. Mixed atomistic and coarse-grained molecular dynamics:
simulation of a membrane-bound ion channel. J. Phys. Chem. B 110:15045-15048.
327. Paramore, S., and G. A. Voth. 2006. Examining the influence of linkers and tertiary structure in the
forced unfolding of multiple-repeat spectrin molecules. Biophys. J. 91:3436-3445.
328. Paramore, S., G. S. Ayton, D. T. Mirijanian, and G. A. Voth. 2006. Extending a spectrin repeat unit I.
Linear force-extension response. Biophys. J. 90:92-100.
329. Wang, Y., and G. A. Voth. 2006. Tail aggregation and domain diffusion of ionic liquids. J. Phys. Chem.
B. 110:18601.
330. Blood, P. D., and G. A. Voth. 2006. Direct observation of Bin/amphiphysin/Rvs (BAR) domain-induced
membrane curvature by means of molecular dynamics simulations. Proc. Nat. Acad. Sci. USA
103:15068-15072.
331. Paramore, S., G. S. Ayton, and G. A. Voth. 2006. Extending a spectrin repeat unit II. Rupture behavior.
Biophys. J. 90:101-111.
332. Chen, H., B. Ilan, Y. Wu, F. Zhu, K. Schulten, and G. A. Voth. 2007. Charge delocalization in proton
channels. I. The aquaporin channels and proton blockage Biophys. J. 92:46-60.
333. Chu, J.-W., S. Izvekov, and G. A. Voth. 2006. The multiscale challenge for biomolecular systems:
coarsegrained modeling. Mol. Sim. 32:211.
334. Jiang, W., Y. Wang, and G. A. Voth. 2007. Molecular dynamics simulation of a nanostructural
organization in ionic liquid/water mixtures. J. Phys. Chem. B. 111:4812-4818.
335. Liu, P., and G. A. Voth. 2007. Smart resolution replica exchange: An efficient algorithm for exploring
complex energy landscapes. J. Chem. Phys. 126:045106.
336. Swanson, J. M. J., C. M. Maupin, H. Chen, M. K. Peterson, J. Xu, Y. Wu, and G. A. Voth. 2007. Proton
solvation and transport in aqueous and biomolecular systems: Insights from computer simulations
(Invited feature article). J. Phys. Chem. B. 111:4300-4314.
337. Wu, Y., B. Ilan, and G. A. Voth. 2007. Charge delocalization in proton channels. II. The synthetic LS2
channel and proton selectivity. Biophys. J. 92:61-69.
158

338. Xu, J., M. A. Sharpe, L. Qin, S. Ferguson-Miller, and G. A. Voth. 2007. Storage of an excess proton in
the hydrogen-bonded network of the D-pathway of cytochrome c oxidase: Identification of a protonated
water cluster. J. Am. Chem. Soc. 129:2910-2913.
339. Chen, H., Y. Wu, and G. A. Voth. 2007. Proton transport behavior through the influenza A M2 channel:
Insights from molecular simulation. . Biophys. J. 93:3470-3479.
340. Chu, J.-W., S. Izvekov, G. S. Ayton, and G. A. Voth. 2007. Emerging methods for multiscale simulation
of biomolecular systems. Mol. Phys. 105:167-175.
341. Paesani, F., S. Iuchi, and G. A. Voth. 2007. Quantum effects in liquid water from an ab initio-based
polarizable force field. J. Chem. Phys.:074506.
342. Zhou, J., I. F. Thorpe, S. Izvekov, and G. A. Voth. 2007. Coarse-grained peptide modeling using a
systematic force-matching approach. Biophys. J. 92:4289-4303.
343. Chu, J.-W., and G. A. Voth. 2007. Coarse-grained free energy functions for studying protein
conformational changes: A double-well network model. Biophys. J. 93:3860-3871.
344. Chu, J. W., and G. A. Voth. 2006. Coarse-Grained Modeling of the Actin Filament Derived from
Atomistic-Scale Simulations. Biophys. J. 90:1572-1582.
345. Liu, P., S. Izvekov, and G. A. Voth. 2007. Multiscale coarse-graining of monosaccharides. J. Phys.
Chem. B. 111:11566-11575.
346. Blood, P. D., R. D. Swenson, and G. A. Voth. 2008. Factors influencing local membrane curvature
induction by N-BAR domains as revealed by molecular dynamics simulations. Biophys. J 95:1866-1876.
347. Noid, W. G., P. Liu, Y. Wang, J.-W. Chu, G. S. Ayton, S. Izvekov, H. C. Andersen, and G. A. Voth.
2008. The Multiscale coarse-graining method. II. Numerical implementation for coarse-grained
molecular models. J. Chem. Phys. 128:244115.
348. Thorpe, I., J. Zhou, and G. A. Voth. 2008. Peptide folding using multi-scale coarse-grained model. J.
Phys. Chem B. 112:13079-13090.
349. Wang, F., S. Izvekov, and G. A. Voth. 2008. Unusual "Amphiphilic" Association of Hydrated Protons in
Strong Acid Solution. J. Am. Chem. Soc. 130:3120-3126.
350. Wu, Y., H. Chen, F. Wang, F. Paesani, and G. A. Voth. 2008. An Improved Multistate Empirical Valence
Bond Model for Aqueous Proton Solvation and Transport. J. Phys. Chem. B 112:467-482.
351. Jeon, J., and G. A. Voth. 2008. Gating of the mechanosensitive channel protein MscL: The interplay of
membrane and protein. Biophys. J. 94:3497-3511.
352. Izvekov, S., J. M. J. Swanson, and G. A. Voth. 2008. Coarse-Graining in Interaction Space: A
Systematic Approach for Replacing Long-Range Electrostatics with Short-Range Potentials. J. Phys.
Chem. B 112:4711-4724.
353. Xu, J., and G. A. Voth. 2008. Redox-coupled proton pumping in cytochrome c oxidase: Further insights
from computer simulation. Biochimica et Biophysica Acta (BBA) - Bioenergetics 1777:196-201.
354. Zhang, Z., L. Lu, W. G. Noid, V. Krishna, J. Pfaendtner, and G. A. Voth. 2008. A Systematic
Methodology for Defining Coarse-grained Site in Large Biomolecules. Biophys. J. 95:5073-5083.
355. Shi, Q., P. Liu, and G. A. Voth. 2008. Coarse-graining in Interaction Space: An Analytical Approximation
for the Effective Short-ranged Electrostatics†. J. Phys. Chem. B 112:16230.
356. Lu, L., and G. A. Voth. 2009. Systematic coarse-graining of a multi-component lipid bilayer. J. Phys.
Chem. B 113:1501-1510.
357. Wang, Y., W. G. Noid, P. Liu, and G. A. Voth. 2009. Effective force coarse-graining. Phys. Chem.
Chem. Phys. 11:2002-2015.
358. Izvekov, S., and G. A. Voth. 2008. A solvent free lipid bilayer model using multiscale coarse-graining. J.
Phys. Chem. B 113:4443-4455.
359. Chen, H., J. Xu, and G. A. Voth. 2009. Unusual Hydrophobic Interactions in Acidic Aqueous Solutions.
J. Phys. Chem. B 113:7291.
360. Li, H., H. Chen, T. Zeuthen, C. Conrad, B. Wu, E. Beitz, and G. A. Voth. 2010 Enhancement of Proton
Conductance by Mutations of the Selectivity Filter of Aquaporin-1 in preparation.
361. Knight, C., C. M. Maupin, S. Izvekov, and G. A. Voth. 2010. Force matching reactive force fields using
condensed phase ab initio simulations: Excess proton in bulk water. (in preparation).
362. Wang, D., and G. A. Voth. 2009. Proton Transport Pathway in the CLC Cl-/H+ Antiporter. Biophys. J.
97:121-131.
363. Yamashita, T., and G. A. Voth. 2010. Properties of Hydrated Excess Protons near Phospholipid
Bilayers. J. Phys. Chem. B 114:592-603.
159

364. Ayton, G. S., E. Lyman, V. Krishna, R. Swenson, C. Mim, V. Unger, and G. A. Voth. 2009. New insights
into BAR domain induced membrane remodeling. Biophys. J. 97:1616-1625.
365. Ayton, G. S., E. Lyman, and G. A. Voth. 2010. Hierarchical coarse-graining strategy for proteinmembrane
systems to access mesoscopic scales. Faraday Disc. Chem. Soc. 144:347-357.
366. Ayton, G. S., and G. A. Voth. 2009. A hybrid coarse-graining approach for lipid bilayers at large length
and time scales. J. Phys. Chem. B 113:4413-4424.
367. Ayton, G. S., and G. A. Voth. 2010. Multiscale simulation of the HIV-1 virion. Biophys. J. submitted.
368. Krishna, V., G. S. Ayton, and G. A. Voth. 2010. Role of protein interactions in defining HIV-1 viral capsid
shape and stability: A coarse-grained analysis. Biophys. J. 98:18-26.
369. Lyman, E., C. Higgs, B. Kim, D. Lupyan, J. C. Shelley, R. Farid, and G. A. Voth. 2009. A Role for a
Specific Cholesterol Interaction in Stabilizing the Apo Configuration of the Human A2A Adenosine
Receptor. Structure 17:1660-1668.
370. Pfaendtner, J., D. Branduardi, M. Parrinello, T. D. Pollard, and G. A. Voth. 2009. The Nucleotide-
Dependent Conformational States of Actin. Proc. Nat. Acad. Sci. 106:12723-12728.
371. Pfaendtner, J., E. M. De La Cruz, and G. A. Voth. 2010. Actin filament remodeling by actin
depolymerization factor/cofilin. Proc. Natl. Acad. Sci. DOI:10.1073/pnas.0911675107.
372. Pfaendtner, J., E. Lyman, T. D. Pollard, and G. A. Voth. 2010. Structure and dynamics of the actin
filament. J. Mol. Biol. 396:252-263.
373. Cui, H. S., G. S. Ayton, and G. A. Voth. 2009. Membrane Binding by the Endophilin N-BAR Domain.
Biophysical Journal 97:2746-2753.
374. Lyman, E., H. S. Cui, and G. A. Voth. 2010. Water under the BAR. Biophys. J. submitted.
375. Wang, Y., and G. A. Voth. 2010. Molecular dynamics simulations of polyglutamine aggregation using
solvent-free multiscale coarse-grained models. J. Phys. Chem. B submitted.
376. Hills, R., L. Lu, and G. A. Voth. 2010. Multiscale coarse-graining of the folding energy landscape:
Towards a general protein force field. PLoS Computational Biology submitted.
377. Lai, C. L., K. E. Landgraf, G. A. Voth, and J. J. Falke. 2010. EPR-guided molecular dynamics simulation
successfully predicts the PIP2 lipid binding stoichiometry of membrane-bound PKCalpha C2 domain.
Biophys. J. submitted.
378. Jao, C. C., C. L. Lai, G. S. Ayton, J. L. Gallop, B. J. Peter, H. T. McMahon, G. A. Voth, and R. Langen.
2010. Membrane binding and self-association of the epsin N-terminal homology domain. Biophys. J.
submitted.
379. Larini, L., L. Lu, and G. A. Voth. 2010. The multiscale coarse-graining method. VI. Implementation of
three-body coarse-grained potentials. J. Chem. Phys. in press.
380. Lu, L., S. Izvekov, A. Das, H. C. Andersen, and G. A. Voth. 2010. Efficient, Regularized, and Scalable
Algorithms for Multiscale Coarse-Graining. Journal of Chemical Theory and Computation 6:954-965.
Computer and Computation Research
CCR090035
381. Chiu, C.; Udayana, R. J. K.; Flores, D. T.; Perez, V. D.; Moore, P. B.; Shinoda, W.; Nielsen S. O. A
mean field approach for computing solid-liquid surface tension for nanoscale interfaces. J. Chem. Phys.
2010, 132:054706.
382. Gao. J., Li. Z., “Uncover the conserved property underlying sequence-distant and structure-similar
proteins.” Biopolymers, 2010 93(4). 340-347
383. Chiu, C.; Moore, P. B.; Shinoda, W.; Nielsen, S. O. Size-dependent hydrophobic to hydrophilic transition
for nanoparticles: A molecular dynamics study. J. Chem. Phys. 2009, 131:244706.
384. Vazquez U. OM.; Shinoda, W.; Chiu, C.; Nielsen, S. O.; Moore, P. B. Calculating the surface tension
between a flat solid and a liquid: a theoretical and computer simulation study of three topologically
different methods. J. Math. Chem. 2009, 45:161-174.
385. Kalescky, R. JB.; Shinoda, W.; Chiu, C.; Moore, P. B.; Nielsen, S. O. Area per ligand as a function of
nanoparticle radius: A theoretical and computer simulation approach; Langmuir, 2009, 25:1352-1359.
386. Liu, Z.; Remsing, R. C.; Liu, D.; Moyna, G.; Pophristic, V. Hydrogen bonding in ortho-substituted
arylamides: The influence of protic solvents J. Phys. Chem. B 2009, 113, 7041-7044.
160

387. Galan, J. F.; Brown, J.; Wildin, J.; Liu, Z.; Liu, D.; Moyna, G.; Pophristic, V. Intramolecular hydrogen
bonding in ortho-substituted arylamide oligomers: A computational and experimental study of orthofluoro
and ortho-chloro N-methylbenzamides J. Phys. Chem. B 2009, 113, 12809-12815.
388. Petrik, I.; Remsing, R. C.; Liu, Z.; O'Brien, B. B.; Moyna, G. Solvation of carbohydrates in N,N'-
Dialkylimidazolium ionic liquids: Insights from multinuclear NMR spectroscopy and molecular dynamics
simulations. In ACS symposium book series, 2009.
389. Gao. J., Li. Z. Comparing four different approaches for the determination of inter-residue interactions
provides insight for the structure prediction of helical membrane proteins. Biopolymers, 2009 91(7) 547-
556
390. Pabuwal. V., Li. Z. Comparative analysis of the packing toplogy of structurally important residues in
helical membrane and soluble proteins. Protein Engg. Des. Sel., 2009 22(2) 67-73
391. Nguyen, TH. T.; Rao, N. Z.; Schroeder, W. M.; Moore, P. B. Coarse-grained molecular dynamics of
tetrameric transmembrane peptide bundle within a lipid bilayer. Chem. Phys. of Lipids. (Accepted with
minor revisions)
392. Galan, J. F.; Tang, C. N.; Chakrabarty, S.; Liu, Z.; Moyna, G.; Pophristic, V. Influence of Solvents on
Intramolecular Hydrogen Bonding in Furan and Thiophene Based Amides: A Computational and
Experimental Study. (In preparation)
393. Galan, J. F.; Liu, Z.;Tang, C. N.; Luong, M.; Tooley, C.; Pophristic, V. Relevant structural features that
control the degree of backbone flexibility in aromatic oligoamide foldamers: A Quantum Mechanical
Assessment. (In preparation)
394. Liu, Z.; Ensing, B.; Moore, P. B. Quantitative Assessment of the Performance of Commonly used Force
Fields on both Low Energy and Transition State Regions of the ( ) Conformational Space: Free
Energy Surfaces of Alanine Dipeptide. (In preparation)
395. Liu, Z.; Pophristic, V. A Detailed ab initio Molecular Orbital Study of Intramolecular Hydrogen Bonding in
ortho-substituted Arylamide: Implications in the Parameterization of Molecular Mechanics Force Fields
and the Design of Arylamide Foldamer. (In preparation)
396. Liu, Z.; Petrik, P.; Remsing, R. C.; O’brien, B.; Moyna, G. Cation - Carbohydrate Interactions in Ionic
Liquids, Fact or Fiction A Computer Simulation Study. (In preparation)
397. Coarse-grained molecular dynamics simulations of fullerines interacting with lipid bilayers, Preston
Moore, Steven Nielsen, Russel DeVane, ACS Mid-Atlantic Regional Meeting (MARM): April 10-13, 2010
Wilmington, Oral Presentation
398. Binding pocket analysis of seven helical transmembrane proteins: Is sequence identity alone suitable
for modeling GPCRs as drug targets Vagmita Pabuwal and Zhijun Li, ACS Mid-Atlantic Regional
Meeting (MARM): April 10-13, 2010 Wilmington, Oral Presentation
399. Area per ligand as a function of nanoparticle radius, A theoretical and computer simulation approach,
Preston Moore, Robert JB. Kalescky, Wataru Shinoda, and Steven O. Nielsen, ACS 238th National
Meeting: August 16-20, 2009 Washington DC, Oral Presentation
400. Expanding NMR in the undergraduate curriculum using sodium-23 NMR in foods, Donald A Bouchard
and Guillermo Moyna, ACS 238th National Meeting: August 16-20, 2009 Washington DC, Oral
Presentation
401. Aromatic oligoamide foldamers: Conformational insights from molecular dynamics simulation using
reparameterized molecular mechanics force fields regarding aromatic-amide torsions, Zhiwei Liu,
Jhenny Galan, and Vojislava Pophristic, ACS 238th National Meeting: August 16-20, 2009 Washington
DC, Oral Presentation
402. Modeling of transmembrane proteins and peptides: All-atom and coarse grain molecular dynamics
simulations of helical bundles in palmitoyloleoylphosphatidylcholine (POPC) lipid bilayer, Zhiwei Liu,
Thuy Hien Nguyen, Jhenny Galan, Russell DeVane, and Preston B Moore, ACS 238th National
Meeting: August 16-20, 2009 Washington DC, Oral Presentation
403. Intramolecular hydrogen bonding of ortho-substituted arylamide oligomers: Model compound study,
Jhenny Galan, J Brown, Zhiwei Liu, Jayme L. Wildin, Chi Ngong Tang, and Vojislava Pophristic, ACS
238th National Meeting: August 16-20, 2009 Washington DC, Oral Presentation
404. Investigating intramolecular hydrogen bonding in aromatic oligoamide foldamers, Jessica Geer, Jayme
L. Wildin, Guillermo Moyna, Jhenny Galan, Zhiwei Liu, and Vojislava Pophristic, ACS 238th National
Meeting: August 16-20, 2009 Washington DC, Oral Presentation
405. Coarse grain molecular dynamics simulations of membrane proteins, Jhenny Galan, Zhiwei Liu, Russell
DeVane, and Preston B. Moore, ACS 238th National Meeting: August 16-20, 2009 Washington DC,
Oral Presentation
161

406. Coarse grain paramaterization of drugs, proteins, and lipids for use in molecular dynamics simulations,
Preston B. Moore: Russell DeVane, Jhenny Galan, and Zhiwei Liu, ACS 238th National Meeting:
August 16-20, 2009 Washington DC, Oral Presentation
407. Molecular dynamics of amphiphathic peptides embedded in a lipid bilayer, Thuy Hien T Nguyen, Zhiwei
Liu, Preston B Moore, ACS Mid-Atlantic Regional Meeting (MARM): April 10-13, 2010 Wilmington,
Poster Presentation
408. Mechanism of anesthetic binding to lipid bilayer, Nicolas Chen, Jhenny Galan, Zhiwei Liu, Preston
Moore, Julian Snow, ACS Mid-Atlantic Regional Meeting (MARM): April 10-13, 2010 Wilmington, Poster
Presentation
409. Development of a coarse-grain intramolecular forcefield for proteins, Kenny Nguyen, Jhenny Galan,
Zhiwei Liu, Russell DeVane, Preston Moore, ACS Mid-Atlantic Regional Meeting (MARM): April 10- 13,
2010 Wilmington, Poster Presentation
410. Targeting the human Andorgen Receptor with steroidal CYP17 inhibitors: A Molecular Docking
approach, Eleonora Gianti, Randy J. Zauhar, Puranik Purushottamachar, Vincent C. O. Njar, ACS Mid-
Atlantic Regional Meeting (MARM): April 10-13, 2010 Wilmington, Poster Presentation
411. Computational study of the electronic structure and function of a novel class of cyclic Phosphazenes,
Whelton A. Miller, Vincent S. Pagnotti, Edward R. Birnbaum, Preston B. Moore, ACS Mid-Atlantic
Regional Meeting (MARM): April 10-13, 2010 Wilmington, Poster Presentation
412. Comparative analysis of the packing topology of structurally important residues in helical membrane
and soluble proteins, Vagmita Pabuwal and Zhijun Li, ACS 238th National Meeting: August 16-20, 2009
Washington DC, Poster Presentation
413. Implementation of a novel coarse-grain model using rhodopsin in a lipid bilayer, Kenny Nguyen, Jhenny
Galan, Zhiwei Liu, and Preston Moore, ACS 238th National Meeting: August 16-20, 2009 Washington
DC, Poster Presentation
414. Mechanism of anesthetic binding to lipid bilayer, Nicolas Chen, Thuy Hien T. Nguyen, Julian Snow, and
Preston B. Moore, ACS 238th National Meeting: August 16-20, 2009 Washington DC, Poster
Presentation
415. Molecular dynamics of proteins embedded in a lipid bilayer, Thuy Hien Nguyen and Preston B Moore,
ACS 238th National Meeting: August 16-20, 2009 Washington DC, Poster Presentation
416. Surface tension, contact angle, and line tension in a liquid nanodroplet, Vladimir Perez, Chi-cheng Chiu,
Steven O. Nielsen, and Preston Moore, ACS 238th National Meeting: August 16-20, 2009 Washington
DC, Poster Presentation
417. Molecular dynamics studies of cellobiose in N,N'-dialkylimidazolium ionic liquids, Igor D. Petrik, Richard
C. Remsing, Zhiwei Liu, Christopher Petoukhoff, and Guillermo Moyna, ACS 238th National Meeting:
August 16-20, 2009 Washington DC, Poster Presentation
418. Molecular dynamics studies of N,N'-dialkylimidazolium ionic liquid solvation and aggregation, Igor D.
Petrik, Richard C. Remsing, Brendan B. O'Brien, Zhiwei Liu, and Guillermo Moyna, ACS 238th National
Meeting: August 16-20, 2009 Washington DC, Poster Presentation
Earth Sciences
MCA04N026
419. V. Akcelik, H. Flath, O. Ghattas, J. Hill, B. van Bloemen Waanders, and L. Wilcox, Fast algorithms for
Bayesian uncertainty quanti_cation in large-scale linear inverse problems based on low-rank partial
Hessian approximations. Submitted, 2009.
420. L. Alisic, C. Burstedde, O. Ghattas, M. Gurnis, G. Stadler, E. Tan, L. C. Wilcox, and S. Zhong, Rhea:
Parallel adaptive mantle convection. Manuscript to be submitted to Geophysical Journal International,
http://ccgo.ices.utexas.edu/publications/AlisicBursteddeGhattasEtAl10.pdf, 2010.
421. C. Burstedde, M. Burtscher, O. Ghattas, G. Stadler, T. Tu, and L. C. Wilcox, ALPS: A framework for
parallel adaptive PDE solution, Journal of Physics: Conference Series, 180 (2009), p. 012009.
422. C. Burstedde, O. Ghattas, M. Gurnis, T. Isaac, A. Klockner, G. Stadler, T. Warburton, and L. C.
Wilcox, Extreme-scale AMR. Submitted to the Proceedings of the 2010 ACM/IEEE Conference on
Supercomputing. http://ccgo.ices.utexas.edu/publications/BursteddeGhattasGurnisEtAl10.pdf, 2010.
423. C. Burstedde, O. Ghattas, M. Gurnis, E. Tan, T. Tu, G. Stadler, L. C. Wilcox, and S. Zhong, Scalable
adaptive mantle convection simulation on petascale supercomputers, in SC '08: Proceedings of the
162

2008 ACM/IEEE Conference on Supercomputing, Piscataway, NJ, USA, 2008, IEEE Press, pp. 1{15.
Gordon Bell Prize _nalist.
424. C. Burstedde, O. Ghattas, J. Martin, and L. C. Wilcox, Uncertainty quanti_cation in inverse problems
with stochastic Newton MCMC. In preparation, 2010.
425. C. Burstedde, O. Ghattas, G. Stadler, T. Tu, and L. C. Wilcox, Towards adaptive mesh PDE
simulations on petascale computers, in Proceedings of Teragrid '08, 2008. Winner, NSF TeraGrid
Capability Computing Challenge.
426. ALPS: A framework for parallel adaptive PDE solution, 2009. Best poster at IEEE/ACM SC'09.
427. C. Burstedde, O. Ghattas, G. Stadler, T. Tu, and L. C. Wilcox, Parallel scalable adjoint-based adaptive
solution for variable-viscosity Stokes ows, Computer Methods in Applied Mechanics and Engineering,
198 (2009), pp. 1691{1700.
428. C. Burstedde, L. C. Wilcox, and O. Ghattas, p4est: Scalable algorithms for parallel adaptive mesh
re_nement on forests of octrees. Submitted to SIAM Journal on Scienti_c Computing.
http://ccgo.ices.utexas.edu/publications/BursteddeWilcoxGhattas10.pdf, 2010.
429. T. Isaac, C. Burstedde, L. C. Wilcox, and O. Ghattas, Fast recursive algorithms on a parallel adaptive
forest of octrees. In preparation, 2010.
430. G. Stadler, C. Burstedde, O. Ghattas, T. Tu, and L. C. Wilcox, Towards highly parallel mesh adaptation
for large-scale PDE applications, in Optimal Control of Coupled Systems of PDE, K. Kunisch, G.
Leugering, J. Sprekels, and F. Troltzsch, eds., vol. 13/2008 of Oberwolfach Report, Mathematisches
Forschungsinstitut Oberwolfach, 2008, pp. 645{655.
431. G. Stadler, M. Gurnis, C. Burstedde, L. C. Wilcox, L. Alisic, and O. Ghattas, The dynamics of plate
tectonics and mantle ow: From local to global scales. Submitted to Science,
http://www.gps.caltech.edu/~gurnis/Share/Rhea_papers/Stadler_etal_2010_with_Figures.pdf, 2010.
432. L. C. Wilcox, G. Stadler, C. Burstedde, and O. Ghattas, A high-order discontinuous Galerkin method for
wave propagation through coupled elastic-acoustic media, Journal of Computational Physics, (2010).
Submitted, http://ccgo.ices.utexas.edu/publications/WilcoxStadlerBursteddeEtAl10.pdf.
433. S. Zhong, A. McNamara, E. Tan, L. Moresi, and M. Gurnis, A benchmark study on mantle convection in
a 3-D spherical shell using CitcomS, Geochemistry, Geophysics, Geosystems, 9 (2008), pp. 10017{+.
MCA08X018
434. Kollat, J.B. and P. Reed, Visualizing the Time-Evolving Consequences and Tradeoffs for Investments in
Environmental Data. Expected Submission September 2010, In-Preparation.
435. Hadka, D. and P.M. Reed, Diagnostic Assessment of Search Controls and Failure Modes in Many-
Objective Evolutionary Optimization. IEEE Transactions on Evolutionary Computation, In-Review.
436. Kollat, J.B., P.M. Reed, and R.M. Maxwell, Many-Objective Groundwater Monitoring Network Design
using Bias-Aware Ensemble Kalman Filtering, Evolutionary Optimization, and Visual Analytics. Water
Resources Research, In-Review.
Electrical and Communications Systems
MCA08X012
437. G. Klimeck, M. Luisier, H. Ryu, S. Lee, R. Rahman, L. Hollenberg, B. Weber and M. Simmons,
“Atomistic Electronic Structure and Transport Modeling of Realistically Extended Nanoelectronics
Devices,” Invited to International Conference On Nanoscience and Nanotechnology (ICONN), Sydney,
Feb. 2010.
438. G. Klimeck, “Conceptual Challenges for Simulation in Nanotechnology,” Invited to NSF Nanoscale
Science and Engineering Grantees Conference, Arlington, Dec. 2009.
439. B. Weber, S. Mahapatra, H. Ryu, S. Lee, G. Klimeck and M. Simmons, “Atomistic-scale Si:P dopant
wires,” presented to International Conference On Nanoscience and Nanotechnology (ICONN), Sydney,
Feb. 2010.
440. W. Pok, G. Scappucci, W. Lee, W. Thompson, H. Buech, B. Weber, S. Mahapatra, L. Hollenberg, M.
Simmons, H. Ryu, S. Lee, G. Klimeck, M. Friesen and M. A. Eriksson, “Precision control of tunneling in
STM-patterned Si:P devices,” Silicon Qubit Workshop, University of California at Berkeley, Aug. 2009.
441. H. Ryu, G. Klimeck, S. Lee, R. Rahman, B. Haley, S. H. Park, N. Kharche, Z. Jiang, T Boykin, C.
Wellard, J. Cole, L. Hollenberg, G. Lansbergen, S. Rogge, B. Weber and M. Simmons, “Nanoelectronic
Modeling (NEMO) for High Fidelity Simulation of Solid- State Quantum Computing Gates”, Silicon Qubit
Workshop, University of California at Berkeley, Aug. 2009.
163

442. S. Kim, A. Paul, M. Luisier, T. B. Boykin and G. Klimeck, “Full 3-D Quantum Transport Simulation of
Interface Roughness in Nanowire FETs,” presented to American Physics Society (APS) Meeting, Mar
2010.
443. H. Ryu, S. Lee, B. Weber, S. Mahapatra, M. Y. Simmons, L. C. L. Hollenberg, and G. Klimeck,
“Quantum Transport in Ultra-scaled Phosphorus-doped Silicon Nanowires,” submitted to IEEE Silicon
Nanoelectronics Workshop (SNW), Hawaii, Jun 2010.
444. B. Weber, S. Mahapatra, H. Ryu, S. Lee, G. Klimeck and M. Y. Simmons, “Atomic-scale Si:P dopant
wire,” submitted to IEEE Silicon Nanoelectronics Workshop (SNW), Hawaii, Jun 2010.
445. S. Lee, H. Ryu and G. Klimeck, “Equilibrium Bandstructure of a Phosphorus -doped Layer in Silicon
using a Tight-binding Approach,” submitted to International Conference on Nanotechnology (IEEE
NANO) 2010, Seoul, Aug 2010.
Environmental Biology
DEB090011
446. Pfeiffer, W. and Stamatakis, A. (2010) “Hybrid MPI/Pthreads parallelization of the RAxML
phylogenetics code.” Ninth IEEE International Workshop on High Performance Computational Biology
(HiCOMB 2010) Atlanta, April 19, 2010.
447. Pfeiffer, W. and Stamatakis, A. (2010) “Hybrid parallelization of the MrBayes and RAxML phylogenetics
codes.” http://wwwkramer.in.tum.de/exelixis/Phylo100225.pdf.
448. San Mauro, D., and Agorreta, A. (2010) "Molecular systematics: A synthesis of the common methods
and the state of knowledge." Cell Mol Biol Lett.15(2):311-41. Epub 2010 Mar 5.
449. Regier, J.C., Shultz, J.W., Zwick, A., Hussey, A., Ball, B., Wetzer, R., Martin, J. W., and Cunningham,
C. W. (2010) "Arthropod relationships revealed by phylogenomic analysis of nuclear protein-coding
sequences." Nature 463, 1079-1083.
450. Meyer, A., and Lieb, B. (2010) "Respiratory proteins in Sipunculus nudus - Implications for phylogeny
and evolution of the hemerythrin family." Comparative Biochemistry and Physiology Part B:
Biochemistry and Molecular Biology 155, 171-177.
451. Fritz-Laylin, L.K., Prochnik, S.E., Ginger, M.L., Dacks, J.B., Carpenter, M.L., Field, M.C., Kuo, A.,
Paredez, A., Chapman, J., Pham, J., Shu, S., Neupane, R., Cipriano, M., Mancuso, J., Tu,H., Salamov,
A., Lindquist, E., Shapiro, H., Lucas, S., Grigoriev, I.V., Cande, W.Z., Fulton, C., Rokhsar, D.S., and
Dawson, S.C. (2010) "The Genome of Naegleria gruberi Illuminates Early Eukaryotic Versatility" Cell,
140(5): 631-642.
452. Pang, P.S., Planet, P.J., and Glenn, J.S. (2009) "The Evolution of the Major Hepatitis C Genotypes
Correlates with Clinical Response to Interferon Therapy" PLoS ONE 4(8): e6579.
453. McKenna, D.D., Sequeira, A.S., Marvaldi, A.E., and Farrell, B.D. (2009). "Temporal lags and overlap in
the diversification of weevils and flowering plants." PNAS. 106(17):7083-7088.
454. Van der Merwe, M.M., Kinnear, M.W., Barrett, L.G., Dodds, P.N., Ericson, L., Thrall, P.H., and Burdon,
J.J., "Positive selection in AvrP4 avirulence gene homologues across the genus Melampsora" Proc. R.
Soc. B 276:2913-2922.
455. Zimkus, B.M. (2009) "Patterns of speciation and diversity among African frogs (genus
Phrynobatrachus)." Ph.D. dissertation, Harvard University.
456. Toon, A., Finley, M., Staples, J., and Crandall, K.A. (2009) "Decapod phylogenetics and molecular
evolution" in Decapod Crustacean Phylogenetics (Crustacean Issues) (J. Martin, D. L. Felder, and K. A.
Crandall, eds.). CRC Press, Bocan Raton.
457. Dolapsakis, N. and Economou-Amilli, A. (2009) "A new marine species of Amphidinium (Dinophyceae)
from Thermaikos Gulf, Greece." Acta Protozoologica 48(2):91-190.
458. Klassen, J.L. (2009) "Carotenoid diversity in novel Hymenobacter strains isolated from Victoria Upper
Glacier, Antarctica, and implications for the evolution of microbial carotenoid biosynthesis." PhD thesis,
University of Alberta, Edmonton, Canada.
459. Klassen, J.L. (2009) "Pathway evolution by horizontal transfer and positive selection is accommodated
by relaxed negative selection upon upstream pathway genes in purple bacterial carotenoid
biosynthesis." Journal of Bacteriology, in the press.
460. Spinks, P.Q. and Shaffer, H.B. (2009). "Conflicting mitochondrial and nuclear phylogenies for the widely
disjunct Emys (Testudines: Emydidae) species complex, and what they tell us about biogeography and
hybridization." Syst. Biol. 58(1):1–20.
164

461. Spinks, P.Q., Thomson, R.C., Lovely, G.A. and Shaffer, H.B. (2009) "Assessing what is needed to
resolve a molecular phylogeny: simulations and empirical data from emydid turtles." BMC Evolutionary
Biology 9:56. doi:10.1186/1471-2148-9-56.
462. Penna, A., Fraga, S, Batocchi, C., Casabianca, S., Giacobbe, M.G., Riobò, P. and Vernesi, C. (2009) "A
phylogeographic study of the toxic benthic dinoflagellate genus Ostreopsis Schmidt." Journal of
Biogeography in the press.
463. Hibbett, D.S. and Matheny, P.B. (2009) "The relative ages of ectomycorrhizal mushrooms and their
plant hosts estimated using Bayesian relaxed molecular clock analyses." BMC Biology 7:13
doi:10.1186/1741-7007-7-13.
464. Elias M. and Archibald J. M. (2009) "The RJL family of small GTPases is an ancient eukaryotic
invention probably functionally associated with the flagellar apparatus." Gene 442(1-2):63-72. Epub
2009 Apr 22. doi:10.1016/j.gene.2009.04.011
465. Gile, G.H., Novis, P.M., Cragg D.S., Zuccarello G.C., Keeling P.J. (2009) "The distribution of EF-1alpha,
EFL, and a non-canonical genetic code in the Ulvophyceae: Discrete genetic characters support a
consistent phylogenetic framework." J. Eukaryot. Microbiol. 56(4):367-372.
466. Jabaily, R.S. and Sytsma, K.J. (2009) "Phylogenetics of Puya (Bromeliaceae): Placement, major
lineages and evolution of Chilean species." accepted: American Journal of Botany in the press.
467. Weirauch, C. and Munro, J. (2009) "Molecular phylogeny of the assassin bugs (Hemiptera: Reduviidae),
based on mitochondrial and nuclear ribosomal genes." Molecular Phylogenetics & Evolution 53:
287-299. doi:10.1016/j.ympev.2009.05.039
468. Wilson, N. G., Rouse, G. W., and Giribet, G. (2009) "Assessing the molluscan hypothesis Serialia
(Monoplacophora + Polyplacophora) using novel molecular data." Mol Phylogenet Evol. Jul 30. [Epub
ahead of print]. doi:10.1016/j.ympev.2009.07.028
469. Segade, F. (2009) "Functional evolution of the microfibril-associated glycoproteins." Gene 439:43– 54.
470. Welsh, A.K., Dawson, J.O., Gottfried, G.J., and Hahn, D. "Diversity of Frankia in root nodules of
geographically isolated Arizona alder in central Arizona (USA)." Applied and Environmental
Microbiology (in press).
471. Mirza, B.S., Welsh, A.K., and Hahn, D. (2009) "Growth of Frankia strains in leaf litter-amended soil and
the rhizosphere of a nonactinorhizal plant." FEMS Microbiology Ecology 70:132-141 Epub 2009 Jul 21.
472. Welsh, A.K, B.S. Mirza, J.P. Rieder, M.W. Paschke, D. Hahn. (2009) "Diversity of frankiae in nodules of
Morella pensylvanica grown in soils from five continents." Systematic and Applied Microbiology 32: 201-
210. doi:10.1016/j.syapm.2009.01.002
473. Mirza, B.S., A.K. Welsh, G. Rasul, J.P. Rieder, M.W. Paschke, D. Hahn. (2009) "Variation in Frankia
populations of the Elaeagnus host infection group in nodules of six host plant species after inoculation
with soil." Microbial Ecology 58:384-393. doi:10.1007/s00248-009-9513-0
474. Li, J-t., Che, J., Murphy, R.W., Zhao, H., Zhao, E.-m., Rao, D.-q., and Zhang, Y.-p. (2009) "New insights
to the molecular phylogenetics and generic assessment in the Rhacophoridae (Amphibia: Anura) based
on five nuclear and three mitochondrial genes, with comments on the evolution of reproduction."
Molecular Phylogenetics and Evolution 53: 509–522. doi:10.1016/j.ympev.2009.06.023
475. Murienne, J., Karaman, I., and Giribet, G. (2009) "Explosive evolution of an ancient group of
Cyphophthalmi (Arachnida: Opiliones) in the Balkan Peninsula." J. Biogeography Published online, Oct
14, 2009.
476. Schulte II, J.A., and Cartwright, E.M. (2009) Phylogenetic relationships among iguanian lizards using
alternative partitioning methods and TSHZ1: A new phylogenetic marker for reptiles. Molecular
Phylogenetics and Evolution 50:391-396. doi:10.1016/j.ympev.2008.10.018.
477. Armbruster, W.S., Leed,J., and Baldwind, B.G. (2009) "Macroevolutionary patterns of defense and
pollination in Dalechampia vines: Adaptation, exaptation, and evolutionary novelty" Proc. Natl. Acad. Sci
USA. Oct 19 e-pub ahead of publication.
478. Zimkus, B.M., Rödel, M.-C., and Hillers, A. (2009) Complex patterns of continental speciation: molecular
phylogenetics and biogeography of sub-Saharan puddle frogs (Phrynobatrachus) manuscript submitted.
479. Chen, W.-J., Lheknim, V. and Mayden, R. L. (2009). "Molecular phylogeny of the Cobitoidea (Teleostei:
Cypriniformes) revisited: position of enigmatic loach Ellopostoma Vaillant resolved with six nuclear
genes." Journal of Fish Biology 75, In press.
480. Chen, W.-J. and Mayden, R. L. (2009). Molecular systematics of the Cyprinoidea (Teleostei:
Cypriniformes), the world’s largest clade of freshwater fishes: Further evidence from six nuclear genes.
Molecular Phylogenetics and Evolution 52, 544-549. doi:10.1016/j.ympev.2009.01.006
165

481. Mayden, R. L., Chen, W.-J., Bart, H. L., Doosey, M. H., Simons, A. M., Tang, K. L., Wood, R. M.,
Agnew, M. K., Yang, L., Hirt, M. V., Clements, M. D., Saitoh, K., Sado, T., Miya, M. and Nishida, M.
(2009). "Reconstructing the phylogenetic relationships of the Earth’s most diverse clade of freshwater
fishes — Order Cypriniformes (Actinopterygii: Ostariophysi): a case study using multiple nuclear loci
and the mitochondrial genome." Molecular Phylogenetics and Evolution 51, 500-514.
doi:10.1016/j.ympev.2008.12.015
482. Spinks, P.Q., Thomson, R.C., Shaffer, R.B. and Barley, A.J. (2009) "Fourteen nuclear genes provide
phylogenetic resolution for difficult nodes in the turtle tree of life." Molecular Phylogenetics and
Evolution in the press.
Materials Research
DMR060061
483. “Tuning of Metal-Metal Bonding by Counterion Size in Hypothetical AeTiO2 Compounds” Wen, X.-D.;
Cahill, T. J.; Hoffmann, R.; Miura, A. J. Am. Chem. Soc. (communication). 2009, 131 (41), 14632-
14633.
484. “Playing the Quantum chemical Slot Machine: An Exploration of ABX2 Compounds” Wen, X.-D.; Cahill,
T. J.; Gerovac, N.; Bucknum, M. J.; Hoffmann, R. Inorg. Chem, 2010, 49 (1), 249-260.
485. “A Little Bit of Lithium Does a Lot for Hydrogen” Zurek, E.; Hoffmann, R.; Ashcroft, N. W.; Oganov, A.
R.; Lyakhov, A. O. Proc. Nat. Acad. Sci. 2009, 106 (42), 17640-17643.
486. “Large-Scale Soft Colloidal Template Synthesis of 1.4 nm Thick CdSe Nanosheets” Son, J. S.; Wen,
X.-D.; Joo, J. Chae, J.; Baek, S.; Park, K.; Kim, J. H.; An, K.; Yu, J. H.; Kwon, S. G.; Choi, S.-H.; Wang,
C.; Kim, Y. W.; Kuk, Y.; Hoffmann, R.; Hyeon, T. Angew. Chem. Int. Ed. 2009, 48 (37), 6861-6864.
487. “Exploring Group 14 Structures: 1D to 2D to 3D” Wen, X.-D.; Cahill, T. J.; Hoffmann, R. Chem. Eur. J,
2010, in press.
488. “Segregation into Layers: A General Problem for Structures under Pressure, Exemplified by SnH4”
Gonzalez-Morelos, P.; Hoffmann, R. Ashcroft, N. W. In preparation.
489. “Reconstructing a solid-solid phase transformation pathway in CdSe nanosheets with associated soft
ligands” Wang, Z.; Wen, X.-D.; Hoffmann, R.; Son, J. S.; Li, R.; Fang,C-C.; Smilgies, D-M.; Hyeon, T. In
preparation.
490. “Breaking into the C-C Bond of Cyclopropane on Metal Surfaces” Cahill, T. J.; Hoffmann, R. In
preparation.
DMR090098
491. A. Bagri, R. Grantab, N. V. Medhekar and V. B. Shenoy, Stability and formation mechanisms of
carbonyl- and hydroxyl- decorated holes in graphene oxide, Journal of Physical Chemistry (2010), in
press.
492. N. V. Medhekar, A. Ramasubramaniam, R. S. Ruoff and V. B. Shenoy, Hydrogen bond networks in
graphene oxide composites: structure and mechanical properties, ACS Nano, As Soon As Publishable,
April 9, 2010.
493. A. Bagri, C. Mattevi, M. Acik, Y. J. Chabal, M. Chhowalla and V. B. Shenoy, Carbon and Oxygen
interactions during the reduction of graphene oxide, Nature Chemistry, in press (2010).
494. R. Grantab and V. B. Shenoy, The Role of Nanoscale Surface Facets on the Mechanical Strength and
Failure of Wurtzite and Twin-Plane Superlattice Nanowires, Submitted to Physical Review B (2010).
DMR100058
495. Amarasinghe, P. M., Katti, K. S., & Katti, D. R. (2008). Molecular Hydraulic Properties of
Montmorillonite: A Polarized Fourier Transform Infrared Spectroscopic Study. Applied Spectroscopy,
62(12), 1303-1313.
496. Amarasinghe, P. M., Katti, K. S., & Katti, D. R. (2009). Nature of organic fluid-montmorillonite
interactions: An FTIR spectroscopic study. [Article]. Journal of Colloid and Interface Science, 337(1),
97-105. doi: 10.1016/j.jcis.2009.05.011
497. Katti, D. R., Matar, M. I., Katti, K. S., & Amarasinghe, P. M. (2009). Multiscale modeling of swelling
clays: A computational and experimental approach. [Article]. Ksce Journal of Civil Engineering, 13(4),
243-255. doi: 10.1007/s12205-009-0243-0
166

MCA06N063
498. "Emerging Photoluminescence in Monolayer MoS2", A. Splendiani, L. Sun, Y. Zhang, T. Li, J. Kim, C.-
Y. Chim, G.Galli, and F. Wang, Nano Lett. 2010
499. "First-principles investigations of the dielectric properties of crystalline and amorphous Si3N4 thin films",
T. A. Pham, T. Li, S. Shankar, F. Gygi and G. Galli, Appl. Phys. Lett. , 96, 062902 (2010).
500. "Van der Waals interactions in molecular assemblies from first-principles calculations", Y. Li, D. Lu, H-V
Nguyen and G. Galli, J. Phys. Chem. A, 114, 1944-1952 (2010).
501. "Ab initio calculation of van der Waals bonded molecular crystals", D. Lu, Y. Li, D. Rocca and G. Galli,
Phys. Rev. Lett. 102, 206411 (2009).
502. "Nature and strength of interlayer binding in graphite", L. Spanu, S. Sorella and G. Galli, Phys. Rev.
Lett., 103, 196401 (2009).
503. "High energy excitations in silicon nanoparticles" , A. Gali, M. Vörös, D. Rocca, G. Zimanyi and G. Galli,
Nano Lett. 9, 3780 (2009).
504. "Theoretical investigation of methane under pressure", L. Spanu, D. Donadio, D. Hohl and G. Galli, J.
Chem.Phys. 130, 164520 (2009).
Mathematical Sciences
DMS060014
505. B. Bourdin, D. Bucur, and É. Oudet. Optimal Partitions. SIAM J. Sci. Comp. 31(6), 4100–4114, 2009-
2010.
506. B. Bourdin, G.A. Francfort, and J.-J. Marigo. The Variational Approach to Fracture. Springer, 2008.
507. B. Bourdin, G.A. Francfort, and J.-J.Marigo. The Variational Approach to Fracture. J. Elasticity, 91 (1-
3):1– 148, 2008.
508. B. Bourdin, C.J. Larsen, and C. Richardson. A Variational Approach to Dynamic Fracture of Brittle
Materials I: Modeling and Implementation. Submitted, 2009.
509. B. Bourdin. Numerical Implementation of a Variational Formulation of Quasi-Static Brittle Fracture.
Interfaces Free Bound., 9:411–430, 2007.
510. B. Bourdin. The Variational Formulation of Brittle Fracture: Numerical Implementation and Extensions.
In R. de Borst A. Combescure, T. Belytschko, editor, IUTAM Symposium on Discretization Methods for
Evolving Discontinuities, pages 381–393. Springer, 2007.
511. B. Bourdin. Numerical implementation of the variational brittle fracture formulation In G. Dal Maso, G.
Francfort, A. Mielke and T. Roubicek, editor, Mathematisches Forschungsinstitut Oberwolfach workshop
on Analysis and Numerics for Rate-Independent Processes, 4(1), 2007.
512. May 2010: Mechanical Engineering seminar, California Institute of Technology (Pasadena, CA, USA)
513. Apr. 2010: Chevron Special Projects & Unconventional Resources (Houston, TX, USA).
514. APr. 2010: AppliedMathematics Seminar, Department ofMathematics, University of Utah (Salt Lake
City, UT, USA).
515. Feb. 2010: Applied Mathematics Colloquium, UCLA (Los Angeles, CA, USA).
516. Dec. 2009: Colloquium, Department of Petroleum Engineering, Louisiana State University (Baton
Rouge, LA, USA).
517. Nov. 2009: Colloquium, Institut Jean le Rond d’Alembert, Université Pierre etMarie Curie (Paris,
France).
518. May 2008: SIAM Student seminar, CCT, Louisiana State University (Baton Rouge, LA, USA).
519. Sep. 2008: Lloyd Roeling Mathematics Conference. University of Louisiana (Lafayette, LA, USA).
520. Sep. 2008: IUTAM Symposium on Variational Concepts with Applications to the Mechanics of
Materials, (Bochum, Germany).
521. May 2008: SIAMConference onMathematical Aspects ofMaterials Science, minisymposium “Damage
and Fracture Evolution”, (Philadelphia, PA, USA).
522. Nov. 2007: Mathematics and Computer Science Division Seminar, Argonne National Laboratories
(Argonne, IL, USA).
523. Jul 2007: 9th U.S. National Congress on Computational Mechanics, minisymposium “Numerical
Techniques for the Modeling of Failure in Solids” (San Francisco, CA, USA)
167

524. Apr. 2007: Mini-Workshop: “Shape Analysis for Eigenvalues”, Mathematisches Forschungsinstitut
Oberwolfach (Germany).
525. Jan. 2007: Workshop “Free Discontinuity Problems: From Image Processing to Material Science”, LSU
(Baton Rouge, LA, USA) Organizer.
526. Jan 2007: 113th AMS Annual Meeting, minisymposium “Free Discontinuity Problems: From Image
Processing to Material Science” (New Orleans, LA, USA). Co-organizer (with C.J. LarsenWPI, MA,
USA).
Molecular Biosciences
MCB060011
527. E.I. Goksu, M.I. Hoopes, B.A. Nellis, C. Xing, R. Faller, C.W. Frank, S.H. Risbud, J.H. Satcher, M.L.
Longo, Silica Xerogel/Aerogel - Supported Lipid Bilayers: Consequences of Surface Corrugation,
Biochim Biophys Acta 1798 (2010) 719-729.
528. C. Xing, R. Faller, Density imbalances and free energy of lipid transfer in supported lipid bilayers, The
Journal of Chemical Physics 131 (2009) 175104- 175107.
529. C. Xing, R. Faller, Coarse-Grained Simulations of Supported and Unsupported Lipid Monolayers, Soft
Matter 5 (2009) 4526-4530.
530. C. Xing, R. Faller, What is the difference between a supported and a free Bilayer Insights from
Molecular Modeling on different Scales., in: A. Iglic (Ed.), Advances in Planar Lipid Bilayers and
Liposomes, vol. 12, Academic Press, 2010, p. in press.
MCB060067
531. G. Interlandi and W. Thomas; The catch bond mechanism between von Willebrand Factor and platelet
surface receptors investigated by molecular dynamics simulations; Proteins: Structure, Function and
Bioinformatics; In press
532. G. Interlandi and W. Thomas; Type 2A mutations in von Willebrand Factor A2 domain lower its force
resistance; (In preparation)
533. P. Aprikian_, G. Interlandi_, B. A. Kidd, I. Le Trong, V. Tchesnokova, O. Yakovenko, R. E. Stenkamp, W.
E. Thomas, E. V. Sokurenko; Mechanically optimized structure of the adhesive fimbrial tip of E. coli ;
Under review, attached to the progress report; (_Shared first authorship)
534. I. Le Trong, P. Aprikian, B. A. Kidd, M. Forero-Shelton, V. Tchesnokova, P. Rajagopal, V.
Rodriguez, G. Interlandi, R. Klevit, V. Vogel, R. E. Stenkamp, E. V. Sokurenko, W. E. Thomas;
Mechanical activation of the bacterial adhesive protein FimH involves allostery in a _-sandwich; Cell ; In
press
MCB070059
535. Cameron F. Abrams and Eric Vanden-Eijnden, “Large-scale conformational sampling of proteins using
temperature-accelerated molecular dynamics,” Proc Natl Acad Sci USA :4961-4966 (2010).
536. Harish Vashisth and Cameron F. Abrams, “Docking of insulin to a structurally equilibrated insulin
receptor ectodomain,” Proteins :1531-1543 (2010) (Cover).
537. Heather L. Nyce, Spencer Stober, Cameron F. Abrams, and Michael M. White, “Mapping spatial
relationships between residues in the ligand-binding domain of the 5-HT3 receptor using a ‘molecular
ruler’,” Biophys J (accepted, 2010).
538. Harish Vashisth and Cameron F. Abrams, “All-atom structural models for complexes of insulin-like
growth factor hormones IGF1 and IGF2 with their cognate receptor,” J Mol Biol (under review).
539. Ali Emileh and Cameron F. Abrams, “A mechanism by which binding of the broadly neutralizing
antibody b12 unfolds the inner domain alpha-1 helix in an engineered gp120,” Proteins (under review).
540. Harish Vashisth and Cameron F. Abrams, “‘DFG-flip’ in the insulin receptor kinase is facilitated by a
helical intermediate state of the activation loop,” JACS (under review).
MCB080077
541. Provasi, D., Bortolato A., Filizola, M. “Exploring Molecular Mechanisms of Ligand Recognition by Opioid
Receptors with Metadynamics.” Biochemistry (2009) 48 (42): 10020-10029.
542. Provasi, D., Filizola, M. “Putative Active States of a Prototypic G-Protein Coupled Receptor from Biased
Molecular Dynamics.” Biophysical Journal (2010) in press.
168

543. Zhu, J., Zhu, J., Negri, A., Provasi, D., Filizola, M., Coller, B.S., Springer, T.A. “Structural, Functional,
and Dynamic Characterization of a Novel αIIb-Specific Inhibitor of Integrin αIIbβ3” submitted.
MCB090130
544. Z. Issa, C. Manke, and J. Potoff, “Ca2+ bridging of apposed phospholipid bilayers,” Biophysical J., 2010
(in preparation).
545. Z. Issa, C. Manke, and J. Potoff, “Effect of Ca2+ and Na+ on the interfacial tension of phospholipid
bilayers,” Biochimimica et Biophysica Acta, 2010 (in preparation).
546. N. Bhatnagar, C. Manke, and J. Potoff, “Energetics of SYT binding to phospholipid bilayers in the
presence of Ca2+,” J. Phys. Chem. B. 2010 (in preparation).
MCB090132
547. Shan, J., Khelashvili, G., and H. Weinstein. Large-Scale MD Simulations Reveal Structural Elements of
the Activated State in the 5-HT2a Receptor and their Relation to Cholesterol Dynamics. in Biophysical
Society Meeting. 2010. San Francisco, CA, USA.
548. Khelashvili, G., Pabst, G., Weinstein, H., and D. Harries. Cholesterol Orientation and Tilt Modulus in
DMPC Bilayers. in Biophysical Society Meeting. 2010. San Francisco, CA, USA.
549. Ballesteros, J.A. and H. Weinstein, Integrated methods for the construction of threedimensional models
and computational probing of structure-function relations in G proteincoupled receptors. Methods in
Neurosciences, 1995. 40: p. 366-428
550. Mehler, E.L., Hassan, S. A., Kortagere, S., and H. Weinstein. Ab initio computational modeling of loops
in G-protein-coupled receptors: lessons from the crystal structure of rhodopsin. Proteins, 2006. 64(3): p.
673-90.
551. Khelashvili, G., Grossfield, A., Feller, S. E., Pitman, M. C., and H. Weinstein. Structural and dynamic
effects of cholesterol at preferred sites of interaction with rhodopsin identified from microsecond length
molecular dynamics simulations. Proteins, 2009. 76(2): p. 403-17.
552. Han, Y., Moreira I.S., Urizar, E., Weinstein, H., and J. Javitch. Allosteric communication between
protomers of dopamine class A GPCR dimers modulates activation. Nat Chem Biol, 2009. 5(9): p. 688-
95.
MCA93S028
553. L. G. Trabuco, E. Villa, E. Schreiner, C. B. Harrison, and K. Schulten. Molecular Dynamics Flexible
Fitting: A practical guide to combine cryo-electron microscopy and X-ray crystallography. Methods,
49:174–180, 2009.
554. E. Villa, J. Sengupta, L. G. Trabuco, J. LeBarron, W. T. Baxter, T. R. Shaikh, R. A. Grassucci, P.
Nissen, M. Ehrenberg, K. Schulten, and J. Frank. Ribosome-induced changes in elongation factor Tu
conformation control GTP hydrolysis. Proceedings of the National Academy of Sciences, USA, 106:1063–
1068, 2009.
555. J. Gumbart, L. G. Trabuco, E. Schreiner, E. Villa, and K. Schulten. Regulation of the proteinconducting
channel by a bound ribosome. Structure, 17:1453–1464, 2009.
556. T. Becker, S. Bhushan, A. Jarasch, J.-P. Armache, S. Funes, F. Jossinet, J. Gumbart, T. Mielke, O.
Berninghausen, K. Schulten, E.Westhof, R. Gilmore, E. C. Mandon, and R. Beckmann. Structure of
monomeric yeast and mammalian Sec61 complexes interacting with the translating ribosome. Science,
326:1369–1373, 2009.
557. B. Seidelt, C. A. Innis, D. N. Wilson, M. Gartmann, J.-P. Armache, E. Villa, L. G. Trabuco, T. Becker, T.
Mielke, K. Schulten, T. A. Steitz, and R. Beckmann. Structural insight into nascent polypeptide chainmediated
translational stalling. Science, 326:1412–1415, 2009.
558. J. Frauenfeld, J. Gumbart, M. Gartmann, E. O. van der Sluis, T. Mielke, O. Berninghausen, T. Becker,
K. Schulten, and R. Beckmann. Structure of the ribosome-SecYEG complex in the membrane
environment, 2010. Submitted.
559. L. G. Trabuco, C. B. Harrison, E. Schreiner, and K. Schulten. Recognition of the regulatory nascent
chain TnaC by the ribosome. Structure, 2010. In press.
560. L. G. Trabuco, E. Schreiner, J. Eargle, P. Cornish, T. Ha, Z. Luthey-Schulten, and K. Schulten.
Structure and dynamics of ribosomal L1 stalk:tRNA complexes, 2010. To be submitted.
561. P. L. Freddolino, C. B. Harrison, Y. Liu, and K. Schulten. Challenges in protein folding simulations:
Timescale, representation, and analysis, 2010. Submitted.
562. P. L. Freddolino and K. Schulten. Common structural transitions in explicit-solvent simulations of villin
headpiece folding. Biophysical Journal, 97:2338–2347, 2009.
169

563. A. Rajan, P. L. Freddolino, and K. Schulten. Going beyond clustering in MD trajectory analysis: an
application to villin headpiece folding. PLoS One, 2010. In press.
564. L. Miao and K. Schulten. Probing a structural model of the nuclear pore complex channel through
molecular dynamics. Biophysical Journal, 98:1658–1667, 2010.
565. L. Le, E. H. Lee, K. Schulten, and T. Truong. Molecular modeling of swine influenza A/H1N1, Spanish
H1N1, and avian H5N1 flu N1 neuraminidases bound to Tamiflu and Relenza. PLoS Currents: Influenza,
2009 Aug 27:RRN1015, 2010.
566. L. Le, E. H. Lee, D. J. Hardy, T. N. Truong, and K. Schulten. Electrostatic funnel directs binding of
Tamiflu to H5N1/H1N1pdm neuramindase, 2010. Submitted.
567. D. E. Chandler, J. Gumbart, J. D. Stack, C. Chipot, and K. Schulten. Membrane curvature induced by
aggregates of LH2s and monomeric LH1s. Biophysical Journal, 97:2978–2984, 2009.
568. J. Hsin, D. E. Chandler, J. Gumbart, C. B. Harrison, M. Sener, J. Strumpfer, and K. Schulten. Selfassembly
of photosynthetic membranes. ChemPhysChem, 11:1154–1159, 2010.
569. J. Hsin, C. Chipot, and K. Schulten. A glycophorin A-like framework for the dimerization of
photosynthetic core complexes. Journal of the American Chemical Society, 131:17096–17098, 2009.
570. E. R. Cruz-Chu, T. Ritz, Z. S. Siwy, and K. Schulten. Molecular control of ionic conduction in polymer
nanopores. Faraday Discussion, 143:47–62, 2009.
571. E. R. Cruz-Chu and K. Schulten. The role of protonable surface residues in nanoprecipitation oscillation,
2010. Submitted.
572. A. Arkhipov, W. H. Roos, G. J. L. Wuite, and K. Schulten. Elucidating the mechanism behind irreversible
deformation of viral capsids. Biophysical Journal, 97:2061–2069, 2009.
573. W. H. Roos, M. M. Gibbons, A. Arkhipov, C. Uetrecht, N. Watts, P. Wingfield, A. C. Steven, A. Heck, K.
Schulten, W. S. Klug, and G. J. Wuite. Squeezing protein shells: how continuum elastic models,
molecular dynamics simulations and experiments coalesce at the nanoscale, 2010. Submitted.
574. E. H. Lee, J. Hsin, M. Sotomayor, G. Comellas, and K. Schulten. Discovery through the computational
microscope. Structure, 17:1295–1306, 2009.
575. E. H. Lee, J. Hsin, E. von Castelmur, O. Mayans, and K. Schulten. Tertiary and secondary structure
elasticity of a six-Ig titin chain. Biophysical Journal, 98:1085–1095, 2010.
576. A. Arkhipov, Y. Yin, and K. Schulten. Membrane-bending mechanism of amphiphysin N-BAR domains.
Biophysical Journal, 97:2727–2735, 2009.
577. Y. Yin, A. Arkhipov, and K. Schulten. Multi-scale simulations of membrane sculpting by N-BAR
domains. In P. Biggin and M. Sansom, editors, Molecular Simulations and Biomembranes: From
Biophysics to Function. Royal Society of Chemistry, 2010.
578. R. Gamini, M. Sotomayor, and K. Schulten. Cytoplasmic domain filter function in the mechanosensitive
channel of small conductance, 2010. Submitted.
MCA99S007
579. G.G. Maisuradze, A.Liwo, S. Ołdziej, H.A. Scheraga. Evidence, from simulations, of a single state with
residual native structure at the thermal denaturation midpoint of a small globular protein. J. Am. Chem.
Soc., 2010, 132 (27), pp 9444-9452.
580. U. Kozłowska, A. Liwo. H.A. Scheraga. Determination of side-chain-rotamer and side-chain and
backbone virtual-bond-stretching potentials of mean force from AM1 energy surfaces of terminallyblocked
amino-acid residues, for coarse-grained simulations of protein structure and folding. 1. The
Method. J. Comput. Chem., 2010, 31 (6), pp 1143-1153.
581. U. Kozłowska, G.G. Maisuradze, A. Liwo, H.A. Scheraga. Determination of side-chain-rotamer and sidechain
and backbone virtual-bond-stretching potentials of mean force from AM1 energy surfaces of
terminally-blocked amino-acid residues, for coarse-grained simulations of protein structure and folding.
2. Results, comparison with statistical potentials, and implementation in the UNRES force field. J.
Comput. Chem., 2010, 31 (6), 1154-1167.
582. J.A. Vila, P. Serrano, K. Wüthrich and H.A. Scheraga. Sequential nearest-neighbor effects on computed
13Ca chemical shifts. Journal of Biomolecular NMR, (2010), in press.
170

Physics
AST030030
583. N. F. Loureiro, A. A. Schekochihin, D. A. Uzdensky, S. C. Cowley and R. Samtaney, “Instability of
current sheets and formation of plasmoid chains. Part II”, Phys. Plasmas, submitted (2010).
584. D. A. Uzdensky, N. F. Loureiro and A. A. Schekochihin, “Plasmoid-dominated magnetic reconnection in
large systems”, Phys. Rev. Lett., in preparation.
585. N. F. Loureiro, D. A. Uzdensky, A. A. Schekochihin and R. Samtaney, “Nonlinear plasmoid dynamics in
magnetic reconnection”, Phys., Rev. Lett., in preparation.
586. R. Samtaney, N. F. Loureiro, D. A. Uzdensky, A. A. Schekochihin and S. C. Cowley, “Formation of
Plasmoid Chains in Magnetic Reconnection”, Phys. Rev. Lett 103, 105004 (2009), Arxiv:0903:0542
587. US-Japan Workshop on Magnetic Reconnection, Madison, Wi., USA, 2009
588. International Conference on the Numerical Simulation of Plasmas, Lisbon, Portugal, 2009
589. Yosemite 2010 Interdisciplinary Wokshop on Magnetic Reconnection, Yosemite, Ca., USA, 2010
590. Laboratory of Astrophysics of Toulouse-Tarbes, Toulouse, France, 2010
591. A. B. Iskakov, A. A. Schekochihin, S. C. Cowley, and A. Tsinober, “Small-scale structure of isotropic
magnetohydrodynamic turbulence,” Phys. Rev. Lett., in preparation (2010).
592. Workshop on The Physics of Intracluster Medium: Theory and Computation, Univ. of Michigan, Ann
Arbor, Mi., USA, 2010
593. Workshop on Self-Organization in Turbulent Plasmas and Fluids, Dresden, Germany, 2010
594. Workshop on Large-Scale Magnetic Fields in the Universe, ISSI, Bern, Germany, 2010
595. T. Tatsuno, G. G. Plunk, M. Barnes, S. C. Cowley,W. Dorland, G. G. Howes, R. Numata and A. A.
Schekochihin, “Gyrokinetic simulation of entropy cascade in two-dimensional electrostatic turbulence,”
J. Plasma Fusion Res. SERIES, in press (2010); Arxiv:1003.3933.
596. M. Barnes, W. Dorland and T. Tatsuno, “Resolving velocity space dynamics in continuum gyrokinetics”,
Phys. Plasmas 17, 032106 (2010)
597. G. G. Plunk, S. C. Cowley, A. A. Schekochihin, and T. Tatsuno, “Two-dimensional gyrokinetic
turbulence,” J. Fluid Mech., submitted (2009), Arxiv:0904.0243.
598. T. Tatsuno, W. Dorland, A. A. Schekochihin, G. G. Plunk, M. A. Barnes, S. C. Cowley, and G. G.
Howes, “Nonlinear phase mixing and phase-space cascade of entropy in gyrokinetic plasma
turbulence,” Phys. Rev. Lett. 103, 015003 (2009), Arxiv:0811.2538.
599. A. A. Schekochihin, S. C. Cowley, W. Dorland, G. W. Hammett, G. G. Howes, E. Quataert, and T.
Tatsuno, “Astrophysical gyrokinetics: kinetic and fluid turbulent cascades in magnetized weakly
collisional plasmas,” Astrophys. J. Suppl. 182, 310 (2009), Arxiv:0704:0044.
600. M. Barnes, I. G. Abel, W. Dorland, D. R. Ernst, G. W. Hammett, P. Ricci, B. N. Rogers, A. A.
Schekochihin, and T. Tatsuno, “Linearized model Fokker-Planck collision operators for gyrokinetic
simulations. II. Numerical implementation and tests”, Phys. Plasmas 16, 072107 (2009),
Arxiv:0809.3945
601. I. G. Abel, M. Barnes, S. C. Cowley, W. Dorland, and A. A. Schekochihin, “Linearized model Fokker-
Planck collision operators for gyrokinetic simulations. I. Theory,” Phys. Plasmas 15, 122509 (2008)
602. 12th International Solar Wind Conference, Saint-Malo, France, 2009
603. Laboratoire de Physique Statistique et des Plasmas, Universit´e Libre de Bruxelles, Belgium, 2009
604. 7th General Scientific Assembly of the Asia Plasma and Fusion Association and 9th Asia- Pacific
Plasma Theory Conference, Aomori, Japan, 2009
605. Japan Geoscience Union Meeting, Chiba, Japan, 2010
606. Workshop on Multiscale Physics in Coronal Heating and Solar Wind Acceleration, ISSI, Bern,
Germany, 2010
MCA08X008
607. “Toward TeV Conformality." LSD Collaboration, T. Appelquist, A. Avakian, R. Babich, R. C. Brower, M.
Cheng, M. A. Clark, S. D. Cohen, G. T. Fleming, J. Kiskis, E. T. Neil, J. C. Osborn, C. Rebbi, D.
Schaich, and P. Vranas. Phys. Rev. Lett. 104 (2010) 071601, arXiv:0910.2224 [hep-ph].
608. _ \Lattice Strong Dynamics at the TeV Scale." LSD Collaboration, T. Appelquist, A. Avakian, R. Babich,
R. C. Brower, M. Cheng, M. A. Clark, S. D. Cohen, G. T. Fleming, J. Kiskis, E. T. Neil, J. C. Osborn, C.
Rebbi, D. Schaich, and P. Vranas. In preparation.
171

PHY060019
609. Measurement of the Rate of Muon Capture in Hydrogen Gas and Determination of the Proton’s
Pseudoscalar Coupling gP , V. A. Andreev et al. (MuCap Collaboration), Phys. Rev. Lett. 99, 032002
(2007)
610. Nuclear Muon Capture in Hydrogen and its Interplay with Muon Atomic Physics, P. Kammel (MuCap
Collaboration), arXiv:0803.1892 [nucl-ex]
611. Muon Capture in Hydrogen: First MuCap Results and Future Plans, P. Kammel (MuCAP Collaboration),
Few-Body Systems, 44, 333 (2008)
612. Muon Capture Experiments in Hydrogen and Deuterium, C. Petitjean (MuCAP Collaboration), Nucl.
Instr. Meth. A 600, 338 (2009)
613. Muon Lifetime Program at PSI, P. Kammel (MuCAP Collaboration), PoS(Nufact08)107
614. Deuterium Removal Unit for the MuCap Experiment, I. Alekseev et al., NHA Annual Hydrogen
Conference, Sacramento, 2008, http://nha.confex.com/nha/2008/techprogram/MEETING.HTM
615. Muon Capture in Hydrogen and Deuterium, C. Petitjean (MuCAP and MuSun Collaborations), Hyperfine
Interactions, to be published (2009)
616. MuCap: Muon Capture on the Proton to Determine the Pseudoscalar Coupling gP , B. Kiburg (MuCAP
Collaboration), Tenth Conference on the Intersections of Particle and Nuclear Physics, AIP 1182, 686-9
(2009)
617. Muon Capture in Hydrogen, P. Kammel (MuCAP Collaboration), SSP 2009, Taipei, Taiwan (2009) to be
published in Nucl. Phys. A
618. Muon Capture on the Deuteron, The MuSun Experiment, V.A. Andreev et al. [MuSun Collaboration],
PSI Experiment R-08-01, arXiv:1004.1754v1 [nucl-ex] (2010)
PHY080005
619. “The bottomonium spectrum from lattice QCD with 2+1 flavors of domain wall fermions” S. Meinel Phys.
Rev. D 79, 094501 (2009)
620. “Bottom hadrons from lattice QCD with domain wall and NRQCD fermions” S. Meinel, W. Detmold, C. J.
Lin and M. Wingate arXiv:0909.3837 [hep-lat]
621. “Bottom hadron mass splittings in the static limit from 2+1 flavour lattice QCD” W. Detmold, C. J. Lin
and M. Wingate Nucl. Phys. B 818, 17 (2009)
PHY090070
622. Z. Fodor, K. Holland, J. Kuti, D. Nogradi and C. Schroeder, arXiv:0911.2463 [hep-lat].
623. Z. Fodor, K. Holland, J. Kuti, D. Nogradi and C. Schroeder, Phys. Lett. B 681, 353 (2009)
[arXiv:0907.4562 [hep-lat]].
624. Z. Fodor, K. Holland, J. Kuti, D. Nogradi and C. Schroeder, arXiv:0911.2934 [hep-lat].
625. Z. Fodor, K. Holland, J. Kuti, D. Nogradi and C. Schroeder, JHEP 0908, 084 (2009) [arXiv:0905.3586
[hep-lat]].
626. Z. Fodor, K. Holland, J. Kuti, D. Nogradi and C. Schroeder, JHEP 0911, 103 (2009) [arXiv:0908.2466
[hep-lat]].
PHY090083
627. J.A. Ludlow, T. G. Lee, M. S. Pindzola, F. J. Robicheaux, and J. P. colgan, J. Phys. B 42, 225204
(November 2009).
628. J.P. Colgan, D. C. Griffin, C. P. Balance, and M. S. Pindzola, Phys. Rev. A 80, 063414 (December
2009).
629. A. Muller, S. Schippers, R. A. Phaneuf, S. W. J. Scully, A. Aguilar, A. M. Covington, I. Alvarez, C.
Cisneros, E. D. Emmons, M. F. Gharaibeh, G. Hinojosa, A. S. Schlachter, and B. M. McLaughlin, J.
Phys. B 42, 235602 (December 2009).
630. J. A. Ludlow, T. G. Lee, and M. S. Pindzola, Phys. Rev. A 81, 023407 (January 2010).
631. J. P. Colgan, O. Al-Hagan, D. H. Madison, A. J. Murray, and M. S. Pindzola, J. Phys. B 42, 171001
(September 2009).
632. M. S. Pindzola, J. A. Ludlow, and J. P. Colgan, Phys. Rev. A 80, 032707 (September 2009).
633. M. S. Pindzola, J. A. Ludlow, F. J. Robicheauz, J. P. Colgan, and C. J. Fontes, Phys. Rev. A 80,
052706 (November 2009).
172

634. M. S. Pindzola, J. A. Ludlow, F. J. Robicheauz, J. P. Colgan, and D. C. Griffin, J. Phys. B 42, 215204
(November 2009).
635. B. Sun and M. S. Pindzola, Phys. Rev. A 80, 033616 (September 2009).
636. B. Sun and M. S. Pindzola, J. Phys. B 42, 175301 (September 2009).
637. B. Sun and M. S. Pindzola, J. Phys. B 42, 055301 (March 2010).
638. C. P. Ballance, J. A. Ludlow, M. S. Pindzola, and S. D. Loch, J. Phys. B 42, 175202 (September 2009).
639. S. D. Loch, C. P. Ballance, M. S. Pindzola, and D. P. Stotler, Plasma Phys. And Controlled Fusion 51,
105006 (October 2009).
640. M. S. Pindzola, J. A. Ludlow, C. P. Balance, and D. C. Griffin, J. Phys. B 43, 025201 (January 2010).
641. M. S. Pindzola, C. P. Ballance, J. A. Ludlow, S. D. Loch, and J. P. Colgan, J. Phys. B 43, 065201
(March 2010).
642. J. A. Ludlow, C. P. Ballance, S. D. Loch, and M. S. Pindzola, J. Phys. B 43, 074029 (April 2010).
643. S. Miyake, P. C. Stancil, H. R. Sadeghpour, A. Dalgarno, B. M. McLaughlin, and R. C. Forrey,
Astrophys. Journal 709, L168 (February 2010).
PHY090085
644. 1. S. Agarwal, G. Klimeck, and M. Luisier, “Leakage Reduction Design Concepts for Low Power Vertical
Tunneling Field-Effect Transistors”, accepted for publication in IEEE Elec. Dev. Lett. (2010).
645. M. Luisier and G. Klimeck, “Simulations of Nanowire Tunneling Transistors: from the WKB
Approximation to Full-Band Phonon-Assisted Tunneling” accepted for publication in J. of App. Phys.
(2010).
646. T. B. Boykin, M. Luisier, M. Salmani-Jelodar, and G. Klimeck, “Strain-induced, off-diagonal, same-atom
parameters in empirical tight-binding theory suitable for [110] uniaxial strain applied to a silicon
parameterization”, Phys. Rev. B 81, 125202 (2010).
647. M. Luisier and G. Klimeck, “Numerical Strategies towards Peta-Scale Simulations of Nanoelectronic
Devices”, Parallel Computing 36, 117-128 (2010).
648. G. Klimeck and M. Luisier, “Atomistic Modeling of Realistically Extended Semiconductor Devices with
NEMO /OMEN” (invited), IEEE Computers in Science and Engineering (CISE) 12, 28-35 (2010).
649. M. Luisier and G. Klimeck, “Atomistic full-band simulations of silicon nanowire transistors: Effects of
electron-phonon scattering”, Phys. Rev. B 80, 155430 (2009).
650. M. Luisier, “Atomistic Simulations of Nanoelectronic Devices” (invited), NNIN Workshop on Bridging the
Gap between Theory and Experiment, Stanford, CA, USA, February 2010.
651. M. Luisier and G. Klimeck, "Performance Comparisons of Tunneling Field-Effect Transistors made of
InSb, Carbon, and GaSb-InAs Broken Gap Heterostructures", IEDM 2009, Baltimore MD, USA,
December 2009.
652. N. Kharche, G. Klimeck, D.-H. Kim, J. A. del Alamo, and M. Luisier, "Performance Analysis of Ultra-
Scaled InAs HEMTs", IEDM 2009, Baltimore MD, USA, December 2009.
653. M. Luisier, “OMEN: A quantum transport simulator for nanoelectronic devices” (invited), 33rd Annual
EDS/CAS Activities in Western New York Conference, Rochester Institute of Technology, Rochester
NY, USA, November 2009.
654. M. Luisier and G. Klimeck, "Investigation of InxGa1-xAs Ultra-Thin-Body Tunneling FETs using a Full-
Band and Atomistic Approach", International Conference on Simulation of Semiconductor Processes
and Devices, SISPAD 2009, San Diego CA, USA, September 2009.
655. Cover story on the NICS web page in August-September 2009 entitled “Nano Nano: Researchers
model next generation transistors on Kraken” by Gregory Scott Jones.
PHY090084
656. G. G. Howes, J. M. TenBarge, S. C. Cowley, W. Dorland, E. Quataert, and A. A. Schekochihin. A
Weakened Cascade Model for Turbulence in Astrophysical Plasmas. Phys. Plasmas, 2010. in
preparation.
657. G. G. Howes, J. M. TenBarge, S. C. Cowley, W. Dorland, E. Quataert, A. A. Schekochihin, R. Numata,
and T. Tatsuno. Importance of Nonlocal Interactions in the Dissipation of Plasma Turbulence. Phys.
Rev. Lett., 2010. in preparation.
658. R. Numata, G. G. Howes, T. Tatsuno, M. Barnes, and W. Dorland. AstroGK: Astrophysical Gyrokinetics
Code. ArXiv e-prints, April 2010. submitted to J. Comp. Phys.
173

PHY100031
659. Invited for Proc. Nat. Acad. Sci., Special Issue on Surface Chemistry (2010).
PHY100033
660. P. Ajith et al., 'Complete' gravitational waveforms for black-hole binaries with non-precessing spins
(2009), 0909.2867.
661. D. Pollney, C. Reisswig, E. Schnetter, N. Dorband, and P. Diener, High accuracy binary black hole
simulations with an extended wave zone (2009), 0910.3803.
662. L. Santamaria, F. Ohme, M. Hannam, S. Husa, B. Krishnan, P. Ajith, B. Brugmann, N. Dorband, D.
Pollney, C. Reisswig, et al., A new phenomenological waveform family for modeling non-precessing
spinning black-hole binaries.
663. O. Korobkin, E. Schnetter, N. Stergioulas, and B. Zink, Multipatch methods in general relativistic
astrophysics: Nonaxisymmetric instabilities in self-gravitating disks around black holes, in preparation
(2009).
664. B. Zink, O. Korobkin, E. Schnetter, N. Stergioulas, and K. Kokkotas, Nonaxisymmetric f-modes in
rapidly rotating general relativistic neutron stars, in preparation (2009).
665. D. Pollney and C. Reisswig, The gravitational memory of binary black hole mergers (2010), in
preparation.
666. C. Reisswig and D. Pollney, Aliasing e_ects in the integration of numerical waveforms (2010), in
preparation.
667. C. Reisswig, C. Ott, S. Uli, E. Schnetter, and D. Pollney, Comparing wave-extraction methods for
stellar-collapse simulations (2010), in preparation.
668. F. Lo_er, Numerical Simulations of Neutron Star-Black Hole Mergers, Ph.D. thesis, Max Planck
Institute for Gravitational Physics (Albert Einstein Institute), Am Muhlenberg 1, D-14476 Potsdam,
Germany (2005).
669. C. D. Ott, Stellar Iron Core Collapse in 3+1 General Relativity and The Gravitational Wave Signature of
Core-Collapse Supernovae, Ph.D. thesis, Universitat Potsdam, Potsdam, Germany (2006).
670. E. N. Dorband, Computing and analyzing gravitational radiation in black hole simulations using a new
multi-block approach to numerical relativity, Ph.D. thesis, Louisiana State University, Baton Rouge, USA
(2007).
671. C. Reisswig, Binary black hole mergers and novel approaches to gravitational wave extraction in
numerical relativity, Ph.D. thesis, Leibniz Universitat Hannover (2010).
672. M. Jasiulek, Spin measures on isolated and dynamical horizons in numerical relativity, Master's thesis,
Humboldt-Universitat, Berlin, Berlin (2008).
673. P. Mosta, Puncture evolutions within the harmonic framework, Master's thesis, Universitat Kassel,
Kassel (2008).
674. C. Reisswig, N. T. Bishop, D. Pollney, and B. Szilagyi, Characteristic extraction in numerical relativity:
binary black hole merger waveforms at null in_nity, Class. Quant. Grav. 27, 075014 (2010), 0912.1285.
675. P. Mosta et al., Vacuum Electromagnetic Counterparts of Binary Black-Hole Mergers, Phys. Rev. D81,
064017 (2010), 0912.2330.
676. E. O'Connor and C. D. Ott, A New Open-Source Code for Spherically-Symmetric Stellar Collapse to
Neutron Stars and Black Holes, Class. Quant. Grav. in press, arXiv:0912.2393 [astro-ph] (2010).
677. E. B. Abdikamalov, C. D. Ott, L. Rezzolla, L. Dessart, H. Dimmelmeier, A. Marek, and H. Janka,
Axisymmetric general relativistic simulations of the accretion-induced collapse of white dwarfs, Phys.
Rev. D. 81, 044012 (2010).
678. C. D. Ott, Probing the core-collapse supernova mechanism with gravitational waves, Class. Quant.
Grav. 26, 204015 (2009).
679. C. D. Ott, E. Schnetter, A. Burrows, E. Livne, E. O'Connor, and F. Lo_er, Computational models of
stellar collapse and core-collapse supernovae, J. Phys. Conf. Ser. 180, 012022 (2009).
680. C. D. Ott, TOPICAL REVIEW: The gravitational-wave signature of core-collapse supernovae, Class.
Quant. Grav. 26, 063001 (2009).
681. D. Brown, P. Diener, O. Sarbach, E. Schnetter, and M. Tiglio, Turduckening black holes: an analytical
and computational study, Phys. Rev. D (accepted) (2009), arXiv:0809.3533 [gr-qc].
682. L. Dessart, C. D. Ott, A. Burrows, S. Rosswog, and E. Livne, Neutrino Signatures and the Neutrino-
Driven Wind in Binary Neutron Star Mergers, Astrophys. J. 690, 1681 (2009).
174

683. C. Reisswig et al., Gravitational-wave detectability of equal-mass black-hole binaries with aligned spins,
Phys. Rev. D80, 124026 (2009), 0907.0462.
684. C. Reisswig, N. T. Bishop, D. Pollney, and B. Szilagyi, Unambiguous determination of gravitational
waveforms from binary black hole mergers, Phys. Rev. Lett. 103, 221101 (2009), 0907.2637.
685. D. Pollney, C. Reisswig, N. Dorband, E. Schnetter, and P. Diener, The Asymptotic Fallo_ of Local
Waveform Measurements in Numerical Relativity, Phys. Rev. D80, 121502 (2009), 0910.3656.
686. M. Hannam et al., The Samurai Project: verifying the consistency of black- hole-binary waveforms for
gravitational-wave detection, Phys. Rev. D79, 084025 (2009), 0901.2437.
687. B. Aylott et al., Testing gravitational-wave searches with numerical relativity waveforms: Results from
the _rst Numerical INJection Analysis (NINJA) project, Class. Quant. Grav. 26, 165008 (2009),
0901.4399.
688. B. Aylott et al., Status of NINJA: the Numerical INJection Analysis project, Class. Quant. Grav. 26,
114008 (2009), 0905.4227.
689. P. Ajith et al., A template bank for gravitational waveforms from coalescing binary black holes: I. nonspinning
binaries, Phys. Rev. D77, 104017 (2008), 0710.2335.
690. M. C. Babiuc, S. Husa, I. Hinder, C. Lechner, E. Schnetter, B. Szil_agyi, Y. Zlochower, N. Dorband, D.
Pollney, and J. Winicour, Implementation of standard testbeds for numerical relativity, Class. Quantum
Grav. 25, 125012 (2008), arXiv:0709.3559 [gr-qc], URL http://arxiv.org/abs/0709.3559.
691. T. Damour, A. Nagar, E. N. Dorband, D. Pollney, and L. Rezzolla, Faithful E_ective-One-Body waveforms
of equal-mass coalescing black-hole binaries, Phys. Rev. D77, 084017 (2008), 0712.3003.
692. L. Dessart, A. Burrows, E. Livne, and C. D. Ott, The Proto-Neutron Star Phase of the Collapsar Model
and the Route to Long-Soft Gamma-Ray Bursts and Hypernovae, Astrophys. J. Lett. 673, L43 (2008).
693. H. Dimmelmeier, C. D. Ott, A. Marek, and H.-T. Janka, Gravitational wave burst signal from core
collapse of rotating stars, Phys. Rev. D. 78, 064056 (2008).
694. C. D. Ott, E. Schnetter, G. Allen, E. Seidel, J. Tao, and B. Zink, A case study for petascale applications
in astrophysics: Simulating Gamma-Ray Bursts, in Proceedings of the 15th ACM Mardi Gras
conference: From lightweight mash-ups to lambda grids: Understanding the spectrum of distributed
computing requirements, applications, tools, infrastructures, interoperability, and the incremental
adoption of key capabilities (ACM, Baton Rouge, Louisiana, 2008), no. 18 in ACM International
Conference Proceeding Series.
695. C. D. Ott, A. Burrows, L. Dessart, and E. Livne, Two-Dimensional Multiangle, Multigroup Neutrino
Radiation-Hydrodynamic Simulations of Postbounce Supernova Cores, Astrophys. J. 685, 1069 (2008).
696. L. Rezzolla, P. Diener, E. N. Dorband, D. Pollney, C. Reisswig, E. Schnetter, and J. Seiler, The _nal
spin from coalescence of aligned-spin black-hole binaries, Astrophys. J. 674, L29 (2008),
arXiv:0710.3345 [gr-qc].
697. L. Rezzolla et al., On the _nal spin from the coalescence of two black holes, Phys. Rev. D78, 044002
(2008), 0712.3541.
698. L. Rezzolla et al., Spin Diagrams for Equal-Mass Black-Hole Binaries with Aligned Spins, Astrophys.
J679, 1422 (2008), 0708.3999.
699. E. Schnetter, G. Allen, T. Goodale, and M. Tyagi, Alpaca: Cactus tools for application level
performance and correctness analysis, Tech. Rep. CCT-TR-2008-2, Louisiana State University (2008),
URL http: //www.cct.lsu.edu/CCT-TR/CCT-TR-2008-2.
700. E. Schnetter, C. D. Ott, P. Diener, and C. Reisswig, Astrophysical applications of numerical relativity |
from Teragrid to Petascale (2008), the 3rd annual TeraGrid Conference, TeraGrid '08.
701. E. Schnetter, Multi-physics coupling of Einstein and hydrodynamics evolution: A case study of the
Einstein Toolkit (2008), cBHPC 2008 (Component-Based High Performance Computing).
702. J. Seiler, B. Szilagyi, D. Pollney, and L. Rezzolla, Constraint-preserving boundary treatment for a
harmonic formulation of the Einstein equations, Class. Quant. Grav. 25, 175020 (2008), 0802.3341.
703. T. Z. Summerscales, A. Burrows, L. S. Finn, and C. D. Ott, Maximum Entropy for Gravitational Wave
Data Analysis: Inferring the Physical Parameters of Core-Collapse Supernovae, Astrophys. J. 678, 1142
(2008).
704. J. Tao, G. Allen, I. Hinder, E. Schnetter, and Y. Zlochower, XiRel: Standard benchmarks for numerical
relativity codes using Cactus and Carpet, Tech. Rep. CCT-TR-2008-5, Louisiana State University
(2008), URL http://www.cct.lsu.edu/CCT-TR/CCT-TR-2008-5.
175

705. B. Zink, E. Schnetter, and M. Tiglio, Multi-patch methods in general relativistic astrophysics { I.
Hydrodynamical ows on _xed backgrounds, Phys. Rev. D 77, 103015 (2008), arXiv:0712.0353
[astroph], URL http://arxiv.org/abs/0712.0353.
706. G. Allen, E. Caraba, T. Goodale, Y. El Khamra, and E. Schnetter, A scienti_c application benchmark
using the Cactus Framework, Tech. Rep., Center for Computation & Technology (2007), URL http:
//www.cct.lsu.edu/.
707. P. Ajith et al., Phenomenological template family for black-hole coalescence waveforms, Class. Quant.
Grav. 24, S689 (2007), arXiv:0704.3764 [gr-qc].
708. D. Brown, O. Sarbach, E. Schnetter, M. Tiglio, P. Diener, I. Hawke, and D. Pollney, Excision without
excision, Phys. Rev. D 76, 081503(R) (2007), arXiv:0707.3101 [gr-qc].
709. P. Diener, E. N. Dorband, E. Schnetter, and M. Tiglio, Optimized high-order derivative and dissipation
operators satisfying summation by parts, and applications in three-dimensional multi-block evolutions, J.
Sci. Comput. 32, 109 (2007), gr-qc/0512001.
710. H. Dimmelmeier, C. D. Ott, H.-T. Janka, A. Marek, and E. Muller, Generic Gravitational-Wave Signals
from the Collapse of Rotating Stellar Cores, Phys. Rev. Lett. 98, 251101 (2007).
711. M. Koppitz, D. Pollney, C. Reisswig, L. Rezzolla, J. Thornburg, P. Diener, and E. Schnetter, Recoil
velocities from equal-mass binary-black-hole mergers, Phys. Rev. Lett. 99, 041102 (2007), grqc/0701163.
712. C. D. Ott, H. Dimmelmeier, A. Marek, H.-T. Janka, I. Hawke, B. Zink, and E. Schnetter, 3d collapse of
rotating stellar iron cores in general relativity including deleptonization and a nuclear equation of state,
Phys. Rev. Lett 98, 261101 (2007), astro-ph/0609819.
713. C. D. Ott, H. Dimmelmeier, A. Marek, H.-T. Janka, B. Zink, I. Hawke, and E. Schnetter, Rotating
collapse of stellar iron cores in general relativity, Class. Quantum Grav. 24, S139 (2007), astroph/
0612638.
714. D. Pollney, C. Reisswig, L. Rezzolla, B. Szilagyi, M. Ansorg, B. Deris, P. Diener, E. N. Dorband, M.
Koppitz, A. Nagar, et al., Recoil velocities from equal-mass binary black-hole mergers: a systematic
investigation of spin-orbit aligned con_gurations, Phys. Rev. D 76, 124002 (2007), arXiv:0707.2559 [grqc].
715. E. Schnetter, C. D. Ott, G. Allen, P. Diener, T. Goodale, T. Radke, E. Seidel, and J. Shalf, Cactus
Framework: Black holes to gamma ray bursts, in Petascale Computing: Algorithms and Applications,
edited by D. A. Bader (Chapman & Hall/CRC Computational Science Series, 2008), chap. 24,
arXiv:0707.1607 [cs.DC], URL http://arxiv.org/abs/0707.1607.
716. J. Shalf, E. Schnetter, G. Allen, and E. Seidel, Cactus as benchmarking platform, Tech. Rep. CCT-TR-
2006-3, Louisiana State University (2007), URL http://www.cct.lsu.edu/CCT-TR/CCT-TR-2006-3.
717. D. Stark, G. Allen, T. Goodale, T. Radke, and E. Schnetter, An extensible timing infrastructure for
adaptive large-scale applications, in Parallel Processing and Applied Mathematics (PPAM), Lecture
Notes in Computer Science (2007).
718. B. Szil_agyi, D. Pollney, L. Rezzolla, J. Thornburg, and J. Winicour, An explicit harmonic code for blackhole
evolution using excision, Class. Quantum Grav. 24, S275 (2007), gr-qc/0612150.
719. J. Thornburg, P. Diener, D. Pollney, L. Rezzolla, E. Schnetter, E. Seidel, and R. Takahashi, Are moving
punctures equivalent to moving black holes, Class. Quantum Grav. 24, 3911 (2007), gr-qc/0701038.
720. E. Pazos, E. N. Dorband, A. Nagar, C. Palenzuela, E. Schnetter, and M. Tiglio, How far away is far
enough for extracting numerical waveforms, and how much do they depend on the extraction method,
Class. Quantum Grav. 24, S341 (2007), gr-qc/0612149.
721. B. Zink, N. Stergioulas, I. Hawke, C. D. Ott, E. Schnetter, and E. Muller, Nonaxisymmetric instability
and fragmentation of general relativistic quasitoroidal stars, Phys. Rev. D 76, 024019 (2007), astroph/
0611601.
722. P. Diener, F. Herrmann, D. Pollney, E. Schnetter, E. Seidel, R. Takahashi, J. Thornburg, and J.
Ventrella, Accurate evolution of orbiting binary black holes, Phys. Rev. Lett. 96, 121101 (2006), grqc/
0512108, URL http://link.aps.org/abstract/PRL/v96/e121101.
723. E. N. Dorband, E. Berti, P. Diener, E. Schnetter, and M. Tiglio, A numerical study of the quasinormal
mode excitation of Kerr black holes, Phys. Rev. D 74, 084028 (2006), gr-qc/0608091.
724. F. Lo_er, L. Rezzolla, and M. Ansorg, Numerical evolutions of a black hole-neutron star system in full
general relativity: Head-on collision, Phys. Rev. D 74, 104018 (2006), gr-qc/0606104.
725. E. Schnetter and B. Krishnan, Non-symmetric trapped surfaces in the Schwarzschild and Vaidya spacetimes,
Phys. Rev. D 73, 021502(R) (2006), gr-qc/0511017.
176

726. E. Schnetter, P. Diener, N. Dorband, and M. Tiglio, A multi-block infrastructure for three-dimensional
time-dependent numerical relativity, Class. Quantum Grav. 23, S553 (2006), gr-qc/0602104, URL
http://stacks.iop.org/CQG/23/S553.
727. E. Schnetter, B. Krishnan, and F. Beyer, Introduction to Dynamical Horizons in numerical relativity,
Phys. Rev. D 74, 024028 (2006), gr-qc/0604015.
728. B. Zink, N. Stergioulas, I. Hawke, C. D. Ott, E. Schnetter, and E. Muller, Fragmentation of general
relativistic quasi-toroidal polytropes, in Proceedings of the 11th Marcel Grossmann Meeting (MG11) in
Berlin, Germany, July 23-29, 2006 (2007 (submitted)), arXiv:0704.0431 [gr-qc], URL http://arxiv.org/
abs/0704.0431.
729. B. Zink, N. Stergioulas, I. Hawke, C. D. Ott, E. Schnetter, and E. Muller, Rotational instabilities in
supermassive stars: a new way to form supermassive black holes, in International Scienti_c Workshop
on Cosmology and Gravitational Physics, Thessaloniki, December 15-16, 2005, edited by N. K. Spyrou,
N. Stergioulas, and C. Tsagas (ZITI, Thessaloniki, 2006), pp. 155{160.
730. C. Bona, L. Lehner, and C. Palenzuela-Luque, Geometrically motivated hyperbolic coordinate
conditions for numerical relativity: Analysis, issues and implementations (2005), gr-qc/0509092.
PHY100035
731. P. Zhang, H. R. Sadeghpour and A. Dalgarno, "Structure and spectroscopy of ground and excited
states of LiYb" is now in review in J. Chem. Phys.
B
Performance on CY2010 Milestones
177

Work Package Dependencies Planned
QSR 2010Q1 Report Values
QSR 2010Q1 Report Values
Resources 06/30/10 AD Status 06/30/10 AD Notes 3/31/10 AD Status 03/31/10 AD Notes
12/31/09 AD
Status
QSR 2009Q4 Report Values
12/31/09 AD Notes
Project-ID
WBS
O P M
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
SI 2 Area: Software Integration
SI 2.0
Objective 2.0 Operations: maintain/sustain
current capabilities
SI.Coord 2.0.1 O SI Area Coordination
SI.Coord 2.0.1.3 O SI Area Coordination PY5 Aug-09 Jul-10
UC Ian Foster[3%], Lee
Liming[35%]; ANL JP
Navarro [35%]
SI.Coord 2.0.1.3.1 M PY 5 Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-09 Jul-10 75% 50% 25%
SI.Coord 2.0.1.4 O SI Area Coordination Extension Aug-10 Jul-11
SI.Coord 2.0.1.4.1 M Extension Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-10 Jul-11
SI.Inf 2.0.2 O Operate Infrastructure Services
SI.Inf 2.0.2.3 O Operate Infrastructure Services PY5 Aug-09 Jul-10
UC Ian Foster[3%], Lee
Liming[35%]; ANL JP
Navarro [35%]
ANL JP Navarro [2%];
Eric Blau[2%]; NCSA
Jason Brechin[5%];
TACC Warren Smith
[10%]
SI.Inf 2.0.2.3.1 M Extension Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-09 Jul-10 75% 50% 25%
SI.Inf 2.0.2.4 O Operate Infrastructure Services Extension Aug-10 Jul-10
ANL JP Navarro [2%];
Eric Blau[2%]; NCSA
Jason Brechin[5%];
TACC Warren Smith
[10%]
SI.Inf 2.0.2.4.1 M PY5 Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-10 Jul-11 75% 50% 25%
SI.CTSS 2.0.3 O Maintain Current CTSS Kits
SI.CTSS 2.0.3.3 O Maintain Selected Current CTSS Kits PY5 Aug-09 Jul-10
UC Eric Blau[45%], Lee
Liming[15%], TBD[20%];
NCSA Jason
Brechin[14%]; TACC
Warren Smith[10%];
ANL JP Navarro [10%];
UW TBD[7%], Jaime
Frey[7%], Greg
Thaine[6%]
SI.CTSS 2.0.3.3.1 M PY5 Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-09 Jul-10 75% 50% 25%
SI.CTSS 2.0.3.4 P GT 5 Oct-09 Jul-10 70%
In beta, new release
being prepared for RPs
40% Still on original target 20% New Effort Added 2009Q4
SI.CTSS 2.0.3.5 O Maintain Selected Current CTSS Kits Extension Aug-10 Jul-11
UC Eric Blau[45%], Lee
Liming[15%], TBD[20%];
NCSA Jason
Brechin[14%]; TACC
Warren Smith[10%];
ANL JP Navarro [10%];
UW TBD[7%], Jaime
Frey[7%], Greg
Thaine[6%]
SI.CTSS 2.0.3.5.1 M Extension Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-10 Jul-11
SI.CTSS 2.0.3.4 O GT 5 Oct-10 Jul-11
SI.Pkg 2.0.4 O Package Software
SI.Pkg 2.0.4.3 O Package Software PY5 Aug-09 Jul-10
ANL JP Navarro
[15%];NCSA Jason
Brechin[15%]; UC TBD
[10%], Eric Blau[44%],
Joseph Bester[35%];
UW TBD [10%], Jaime
Frey [10%], Greg
SI.Pkg 2.0.4.3.1 M PY5 Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-09 Jul-10
Th i [10%]
75% 50% 25%
SI.Pkg 2.0.4.4 P Local Compute Oct-09 Mar-10 Completed Completed 50% New Effort Added 2009Q4
SI.Pkg 2.0.4.5 P Distributed Programming Jul-09 Jun-10 Completed On target 75% On target 50% New Effort Added 2009Q4
SI.Pkg 2.0.4.6 O Package Software Extension Aug-10 Jul-11
ANL JP Navarro
[15%];NCSA Jason
Brechin[15%]; UC TBD
[10%], Eric Blau[44%],
Joseph Bester[35%];
UW TBD [10%], Jaime
Frey [10%], Greg
Th i [10%]

SI.Pkg 2.0.4.6.1 M Extension Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-10 Jul-11
SI.Pkg 2.0.4.7 O Continue to package software Aug-10 Jul-11
SI.Pkg 2.0.4.8 O Deliver common TG and OSG packages Aug-10 Jul-11
SI 2.1
Objective 2.1 Projects: Information Services
Enhancements
SI.IS 2.1.4 P Information Services Enhancements PY5 Aug-09 Jul-10
SI.IS 2.1.4.1 P
demonstrate ability for resource operators to register
their own locally supported capabilities
Core2 Aug-09 Jul-10
ANL JP Navarro [30%];
NCSA Jason
Brechin[10%]; SDSC
Ken Yashimoto[15%];
UC Eric Blau[10%],
TBD[20%]
SI.IS 2.1.4.1.2 P Capability documented, and registration designed Nov-09 Jan-10 50%
SI.IS 2.1.4.1.3 P Prototype registration(s) deployed Feb-10 Apr-10
SI.IS 2.1.4.1.4 P Production registration(s) deployed May-10 Jul-10
SI.IS 2.1.4.2 P
evaluate the feasibility of gateways publishing about
themselves to information services
SGW Aug-09 Jul-10
SI.IS 2.1.4.2.4 P Publishing evaluation and next step decision May-10 Jul-10 80%
SI.IS 2.1.4.3 P
respond to data, gateways, visualization, and
scheduling needs for expanded information service
content
From
SGW, DV
SI.IS 2.1.5 Information Services Enhancements Extension Apr-10 Jul-11
SI.IS 2.1.5.1 P
Track2D, Viz, and storage capability deployment
registration
Apr-10 Jul-11
SI.IS 2.1.5.2 P New TG capabilities are registered Apr-10 Jul-11
SI.IS 2.1.5.3 P Community capabilities are registered Apr-10 Jul-11
SI.IS 2.1.5.4 P IIS transition to XD
SI 2.2
Objective 2.2 Projects: Application Service
Hosting Partnership
SI.Host 2.2.2 P Application Hosting Services Partnership V1 PY4 Aug-08 Jul-09
SI.Host 2.2.2.1 M
Application Hosting Service Description &
Implementation Documents
Docs
Team,
SGW
Low priority because
RPs are not requesting
this capability.
Dependent on gateway
team to complete their
part before this can be
completed
50% 50% Draft doc to be circulated to RPS
On-going
Mar-09 Jul-10 75% 75% 50%
ANL JP Navarro [30%];
NCSA Jason
Brechin[10%]; SDSC
Ken Yashimoto[15%];
UC Eric Blau[10%],
TBD[20%]
Kate Keahy[20%],
Lee Liming[5%]
Aug-08 Jul-09 75% Lee to talk to Kate 75% Lee to talk to Kate 75%
SI.Host 2.2.2.2 M Coordinate Inca, IS Schema for Hosting Service NOS Jul-09 Oct-09 Lee Liming[6%] 40% Lee to talk to Kate 40% Lee to talk to Kate 40%
8/3 Update: Percentage 75%, The imp
docs are no longer required. The
description document has mostly been
completed. This still needs a small
amount work to complete
IS Schema proto available, no INCA
test yet
SI.Host 2.2.3 P Application Hosting Services Partnership V2 PY5 Aug-09 Jul-10 UC Kate Keahey[25%]
SI.Host 2.2.3.1 M Documentation V2 and transition plan Aug-09 Jul-10
SI.Host 2.2.3.1.1 M Analyze V1 experiences and produce V2 goals, plan Aug-09 Oct-09 0% Low priority 0% Low priority 0% Low priority
SI.Host 2.2.3.1.2 M Develope V1->V2 transition plan Feb-10 Apr-10 0% Low priority 0% Low priority
SI.Host 2.2.3.1.3 M Documnent V2 May-10 Jul-10 0%
SI.Host 2.2.3.2 M Coordinate changes and deployment To RPs Aug-09 Jul-10
SI.Host 2.2.3.2.1 M Collect V1 RP experiences Aug-09 Oct-09 0% Low priority 0% Low priority 0% Low priority
SI.Host 2.2.3.2.2 M Coordinate V2 testing deployment Feb-10 Apr-10 0%
SI.Host 2.2.3.2.3 M Coodinate V2 production deployment May-10 Jul-10 0%
SI 2.3
Objective 2.3 Projects: Public Build and Test
Service
SI.STCI 2.3.2 P TeraGrid STCI User Support PY5 Aug-09 Jul-10
SI.STCI 2.3.2.1 M Feasibility Evaluation for Community Usage Aug-09 Jan-10
SI.STCI 2.3.2.1.1 M
Analyze early user experiences and summarize, asses
community interest
ANL JP Navarro[8%];
UC Eric Blau[9%]
Aug-09 Oct-09 10%
Due to staff losses and
lag in hiring
10%
replacements these task
are behind schedule.
SI.STCI 2.3.2.1.2 M Produce evaluation report and conclusion Nov-09 Jan-10 0% 0%
SI.STCI 2.3.2.2 M Standalone Build & Test Capability Kit 2.3.2.1 Feb-10 Jul-10
SI.STCI 2.3.2.2.1 M Capability kit beta testing, produce documentation Feb-10 Apr-10
SI.STCI 2.3.2.2.2 M Capability kit production rollout May-10 Jul-10
SI 2.5 Objective 2.5 Projects: Scheduling
SI.SPRC 2.5.2.17 P SPRUCE PY4 To RPs Aug-08 Jul-09
Suman Nadella[100%],
Nicholas Trebon[100%],
Pete Beckman [5%],
Initial discussions have
taken place
10% Initial discussions have taken place

SI.SPRC
2.5.2.17.2
M
On-demand compute capability documented and
integrated into production operations on RP systems
SI.SPRC 2.5.2.18 P SPRUCE PY5 To RPs Aug-09 Jul-10
SI.SPRC 2.5.2.18.3 P
SI.SPRC 2.5.2.18.4 P
Deploy a production service for guaranteed network
bandwidth for urgent computations.
Deploy Spruce for virtualized cloud infrastructure
(collaboration with Eucalyptus).
Work with users from VT to utilize Spruce for H1N1
influenza simulations on cloud infrastructures
Work with Purdue and other cloud providers to support
Spruce.
Maintain Spruce infrastructure (i.e., web service server,
database, user/admin portals).
Update Spruce plug-ins at resource-end as new
schedulers are introduced.
Work to get Spruce supported at new TeraGrid
resources that have replaced retired machines that
supported Spruce (e.g., Frost).
Feb-10
May-10
Apr-10
Jul-10
UC Suman
Nadella[40%], Nicholas
Trebon[35%], Pete
Beckman [5%]
67% 67% 67%
100%
Contigent on Pete's
report
Lee to check with Pete
67%
SI.Skd 2.5.3 O Scheduling Working Group
SI.Skd 2.5.3.3 O Scheduling Working Group Coordination PY5 Aug-09 Jul-10
TACC Warren Smith
[25%]
75% 50%
SI.Skd 2.5.3.4 O Scheduling Working Group Coordination Extension Aug-10 Jul-11
TACC Warren Smith
[25%]
SI.QBETS 2.5.9 P QBETS PY4 moved to PY5 Aug-09 Feb-10 Warren
SI.QBETS 2.5.9.2 P
Queue prediction capability documented and
integrated into TeraGrid's production operations
SI.QBETS 2.5.9.2.1 P Queue Predicition Capability Documented Aug-09 Nov-09 60%
Lee to talk to Warren
about Karnak
deployment
60%
Lee to talk to Warren
about Karnak
deployment
SI.QBETS 2.5.9.2.2 P
Queue Predicition Capability integrated into TG
Not formally complete
Not formally complete
Nov-09 Feb-10 25%
0%
operations
but available for use
but available for use
NICS Troy Baer[6%];
SI.MTS 2.5.10 P Metascheduling PY5 Aug-09 Jul-10
SDSC Kenneth
Yashimoto[35%]; TACC
Warren Smith[55%]
SI.MTS 2.5.10.1 P Detailed PY5 metascheduling plan Aug-09 Oct-09 Completed Completed
8/3 Update: On-demand capability has
already been implement by a number of
RPs, documentation and awareness is
still needed for this implementation.
This will be completed by Mar 10
SI.MTS 2.5.10.2 P Metascheduling capabilities developed Nov-09 Jan-10
Condor-G/class AD
Condor-G/class AD
Completed
metascheduling 80%
metascheduling
SI.MTS 2.5.10.3 P
Testing automated and dynamic (information services
based) metascheduling
Feb-10 Apr-10 Completed Completed
SI.MTS 2.5.10.4 P
Production automated and dynamic (information
services based) metascheduling
May-10 Jul-10 Lee to talk to Warren Lee to talk to Warren
SI.MTS 2.5.11 P Metascheduling Extension Apr-10 Jul-11
NICS Troy Baer[6%];
SDSC Kenneth
Yashimoto[35%]; TACC
Warren Smith[55%]
SI.MTS 2.5.11.1 O
Advanced scheduling tools ported to new TeraGrid
systems as appropriate per RP wishes 13%
SI.MTS 2.5.11.2 P
Best practice documentation for using advanced
scheduling capabilities in conjunction with non-
TeraGrid systems
0% Not started
Improved Coordinated Capability User
SI.ICUD 2.7 P
Documentation PY5
Aug-09 Jul-10 SDSC Fariba Fana[15%]
SI.ICUD 2.7.1 P
High-level documentation layout designed, detailed
JP needs to talk to doc
JP needs to talk to doc
Aug-09 Oct-09
project tasks identified, scheduled, and assigned
0%
team 0%
team
SI.ICUD 2.7.2 P
Capability kit introduction/use case documentation
improvements
Nov-09 Jan-10
0%
SI.ICUD 2.7.3 P
Half of capability kit detailed documentation
improvements including IS integration
Feb-10 Apr-10
0%
SI.ICUD 2.7.4 P
Half of capability kit detailed documentation
improvements including IS integration
May-10 Jul-10
0%
0%
Contigent on LifeRay implementation
dependant
SI
END SI

Work Package Dependencies Planned
QSR 2010Q2 Report Values
QSR 2010Q1 Report Values
Resources 6/30/10 AD Status 6/30/10 AD Notes 3/31/10 AD Status 3/31/10 AD Notes
Project-ID
WBS
OPMD
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
SGW 1 Area: Science Gateways
SGW 1.0
Objective 1.0 Operations: maintain/sustain
current capabilities
SGW.Coord 1.0.1.2 O SGW Area Coordination PY4 Aug-08 Jul-09 Nancy Wilkins-Diehr[50%], Ongoing Ongoing
SGW.Coord
D
Support for Gateways requiring urgent computing.
The LEAD gateway will run jobs that pre-empt
others by using SPRUCE.
Standardize security approach to gateways.
SGW.Coord
D Among gateways with roaming allocations, usage
of multiple TeraGrid resources will increase
Gateway Web services registry. It will be possible
SGW.Coord
D to discover services available through Gateways
programmatically
SGW.Coord 1.0.1.3 O SGW Area Coordination PY5 Aug-09 Jul-10 Nancy Wilkins-Diehr[50%],
SGW.Coord 1.0.1.3.1 M Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-09 Jul-10 Stuart Martin[50%]
Ongoing Ongoing
SGW.GDoc 1.0.2 O Gateway Documentation May-07 Jul-10
SGW.GDoc 1.0.2.1.10 P Use of GRAM audit Dec-07 Mar-08 Delayed Delayed
IU Canceled Deferred Not done, currently no RPs support on
demand computing. Gateways will pursue
an application driver for this in PY6 after
capability is developed by scheduling-wg
In progress In progress Delayed until PY5. Targeted effort to
standardize treatment of community
accounts funded at NICS in PY5
RENCI In progress In progress Delayed. Progress slower than expected.
Work expected to wrap up in the first half of
PY5
SGW.GDoc 1.0.2.1.12 P
How to incorporate TG resources into an OGCE
framework
Dec-07 Mar-08 Canceled Delayed
SGW.GDoc 1.0.2.1.13 P Condor, Globus use in a gateway Dec-07 Dec-07 Canceled Delayed
SGW.GDoc 1.0.2.2 O Gateway Documentation PY4 Aug-08 Jun-09 Diana Diehl[50%]
Provide content for documentation, formation of
SGW.GDoc 1.0.2.2.2 M ideas and solutions to common gateway problems
Aug-08 Sep-08 Delayed
(RENCI);
SDSC Diana Diehl[20%];
SGW.GDoc 1.0.2.3 O Gateway Documentation PY5 Aug-09 Jul-10 UNC Jason Reilly [40%],
John McGee [5%]
SGW.GDoc 1.0.2.2.1 M Up to date documentation on science gateways. Aug-09 Jul-10 Ongoing Ongoing
SGW.GDoc 1.0.2.2.2 M Anatomy of Gateway tutorial Aug-09 Jul-10 Done Done
SGW 1.1
Objective 1.1 Projects: Gateways Targeted
Support Program
SGW.HDN 1.1.2.4 O Helpdesk PY5 Aug-09 Jul-10 NCSA Yan Liu[20%]
Provide helpdesk support for production science
SGW.HDN 1.1.2.5 O
gateways by answering user questions, routing
Aug-09 Jul-10
user requests to appropriate gateway contacts,
and tracking user responses
Ongoing
Ongoing
Provide the helpdesk support of the SimpleGrid
SGW.HDN 1.1.2.6 O online training and prototyping services to gateway
Aug-09 Jan-10
Ongoing
Ongoing
SGW.HDN 1.1.2.7 P
developers and communities
Gather requirements for a knowledge base to
improve gateway helpdesk services.
SGW.Gen 1.1.5 P Genomics gateway, SidGrid, MIT SEAS PY4 Aug-08 Mar-09 Suresh Marru[50%]
Work with potential new gateway including polar
grid (IU);
Feb-10
Jul-10
Effort redirected to
GridChem AUS support
request. This work will not be
picked up
Effort redirected to
GridChem AUS support
request
SGW.Gen 1.1.5.1 M Jan-09 Jul-09 Raminder Singh [100%] Completed In progress
CalTech Julian Bunn [10%],
SGW.CIG 1.1.6.3 P CIG support PY5 Aug-09 Jul-10 John McCorquodale [25%],
Roy Wiliams [5%]
Work with NOAO to develop a TG portal that
SGW.CIG 1.1.6.4 P allows users to run customized galaxy collision
Aug-09 Jul-10 Effort redirected to Arroyo
Effort redirected to Arroyo
simulations using the power of the TeraGrid
gateway, tasks forthcoming
gateway, tasks forthcoming
A scriptable interface to portal services will be
Effort redirected to Arroyo
Effort redirected to Arroyo
SGW.CIG 1.1.6.5 P Aug-09 Oct-09
created.
gateway, tasks forthcoming
gateway, tasks forthcoming
Extend the CIG portal with an easy-to-use
language-agnostic RPC-styleinterface that
SGW.CIG 1.1.6.6 P parallels the functionality of the web-based
Nov-09 Jan-10
Effort redirected to Arroyo
Effort redirected to Arroyo
interactiveportal (and extends it where
gateway, tasks forthcoming
gateway, tasks forthcoming
appropriate)
SGW.CIG 1.1.6.7 P
Provide bindings from this interface into popular
languages (driven by user demand) such as C,
Pythonand Perl
SGW.Bio 1.1.10.3 P Bio web services, registry PY4 Aug-08 Jul-09
SGW.Bio 1.1.10.3.4 M
Work with the various TeraGrid teams to evolve
the registry to meet the broader TeraGrid
requirements (RENCI);
SGW.Bio 1.1.10.3.5 M
Work with other gateways to incorporate their
service offerings into the registry (RENCI);
Feb-10 Apr-10 Effort redirected to Arroyo
gateway, tasks forthcoming
Jason Reilly[50%],
Mats Rynge[25%]
John McGee[5%]
To IS, NOS Aug-08 Jul-09 In progress In progress
Effort redirected to Arroyo
gateway, tasks forthcoming
GRAM5 released Jan, 2010. Waiting for TG
installation from software-wg
Documentation staff picking this back up
after Liferay distractions.
Documentation staff picking this back up
after Liferay distractions.
Documentation staff delayed on this due to
LifeRay migration
Aug-08 Jul-09 In progress In progress
SGW.GC 1.1.12 P GridChem or Other New PY5 Aug-09 Jul-10 IU Suresh Marru [50%]
SGW.GC 1.1.12.1 P
Complete Gateway clients as determined by the
Targeted Support Program
Jan-10 Jul-10 In progress In progress Ultrascan
SGW.GC 1.1.12.2 P
Ongoing support as required for existing Gateway
projects
Jan-10 Jul-10 In progress In progress GridChem, Ultrascan, LEAD

Work Package Dependencies Planned
QSR 2010Q2 Report Values
QSR 2010Q1 Report Values
Resources 6/30/10 AD Status 6/30/10 AD Notes 3/31/10 AD Status 3/31/10 AD Notes
Project-ID
WBS
OPMD
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
SGW.GC 1.1.12.3 P
Assist GridChem gateway in the areas of software
access, data management, improved workflows,
Aug-09 Dec-09
visualization and scheduling
Completed
In progress
SGW.PG 1.1.13 P PolarGrid or Other New PY5 Aug-09 Jul-10 IU SGW TBD1 [100%]
SGW.PG 1.1.13.1 P
Complete Gateway clients as determined by the
Targeted Support Program
Jan-10 Jul-10 Completed In progress
SGW.PG 1.1.13.2 P
Ongoing support as required for existing Gateway
projects
Assist PolarGrid team with TeraGrid integration.
SGW.PG 1.1.13.3 P
May include realtime processing of sensor data,
Aug-09 Jul-10
support for parallel simulations, GIS integration
and EOT components
In progress
SGW.SID 1.1.14 P SIDGrid or Other New PY5 Aug-09 Jul-10 UC Wenjun Wu [100%]
SIDGrid scheduling will be enhanced by taking
advantage of work underway in the Scheduling
SGW.SID 1.1.14.1 P Working Group to enable monitoring of and
Aug-09 Oct-09
submission to multiple TeraGrid resources through
a single interface
SGW.SID 1.1.14.2 P
Improved security model for community accounts
utilized by the gateway
Nov-09 Jan-10 Completed In progress
SGW.SID 1.1.14.3 P
Increased number of analysis codes available
through the gateway
Feb-10 Apr-10 Completed
Isolation of execution within virtual machines or
other constrained environments, and data sharing
SGW.SID 1.1.14.4 P with collaborators, while simultaneously protecting
Feb-10 Apr-10
the users' data and execution environment from
accidental or malicious compromise.
Completed
In progress
SGW.SID 1.1.14.5 P
VDS, the current workflow solution, will be
May-10 Jul-10 Completed
replaced by its successor, SWIFT.
Documentation and code that is generally
applicable to gateway efforts will be made
available to TeraGrid gateway partners
Workflow complete, scheduling in
progress
Jan-10 Jul-10 In progress In progress PolarGrid, Ultrascan
Done Using Swift Done Using Swift
SGW.SID 1.1.14.6 P
May-10 Jul-10
In progress
SGW.OSG 1.1.15 P RENCI-OSG PY5 Aug-09 Jul-10
UNC Jason Reilly [15%],
John McGee [5%]
SGW.OSG 1.1.15.1 P
Prototype of OSG jobs running on TeraGrid
resources via NIMBUS
Aug-09 Oct-09 Not completed Deferred
SGW.OSG 1.1.15.2 P
Successful interactions with and assistance for
gateway developers
Nov-09 Apr-10 Not completed In progress
SGW.ES 1.1.16 P Environmental Science Gateway PY5 Aug-09 Jul-10
SGW.ES 1.1.16.1 P
Release of a demonstration version of the ESGC
Portal capable of invoking CCSM runs on the
TeraGrid
Ability for runs invoked on the Purdue Climate
SGW.ES 1.1.16.2 P Portal or the ESGC Gateway to be published back
to the ESG data holdings
SGW.ES 1.1.16.3 P
Investigate federation of Purdue data holdings
with ESG data archives
SGW 1.2 Objective 1.2 Projects: Gateways User Count
Aug-09
SGW.UCnt 1.2.2 P Gateway User Count PY4 To NOS Aug-08 Jul-09
SGW.UCnt
SGW.UCnt
D
D
End gateway users can be counted
programmatically, as opposed to current manual
aggregation method. By the end of PY5, all jobs
submitted via community accounts will include
attributes identifying the end user of that gateway.
Extend TGCDB to store attributes with job logs.
All jobs submitted via community accounts will be
associated with individual gateway users and
made available to backend processes. Job
information will be stored in the TGCDB, which will
make it possible to produce detailed usage reports
through a TGCDB query
SGW.UCnt 1.2.3 P Gateway User Count PY5 NOS Aug-09 Jul-10
SGW.UCnt 1.2.3.1 M
Develop internal web pages for the security
working group and the science gateway
administration page in support of attribute-based
authentication
Oct-09
NCAR NCAR SGW TBD
[50%] ;Purdue PU SGW
TBD1[50%]
In progress
In progress
Nov-09 Jan-10
In progress
In progress
Feb-10 Jul-10 In progress In progress
Aug-09
Dec-09
Tom Scavo[75%],
Jim Basney[10%],
Nancy Wilkins-Diehr[5%]
Delayed
Pending GRAM5 installs and subsequent
RP work.
In progress
NCSA Done. In progress
NCSA Tom Scavo[50%], Jim
Basney[10%], Terry Fleury
[25%]; SDSC Nancy Wilkins-
Diehr[5%], Michael Dwyer
[10%]
Investigating CCSM v4 rather than
v3
Done
Done
SGW.UCnt 1.2.3.2 M Ubiquitous science gateway adoption. Aug-09 Dec-09 Delayed Waiting for GRAM5 install Delayed Dependent on GRAM5 install
SGW.UCnt 1.2.3.3 M Ubiquitous RP adoption. Feb-09 Dec-09 Delayed Waiting for GRAM5 install Delayed
SGW.UCnt 1.2.3.4 M User count INCA tests Jan-10 Feb-10 Done Done

Work Package Dependencies Planned
QSR 2010Q2 Report Values
QSR 2010Q1 Report Values
Resources 6/30/10 AD Status 6/30/10 AD Notes 3/31/10 AD Status 3/31/10 AD Notes
Project-ID
WBS
OPMD
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
1.2.3.5 M Post-TG architecture documentation. Feb-10 Jul-10 In progress In progress
SGW 1.4
Objective 1.4 Projects: Gateways General
Services Discovery
SGW.SG 1.4.2 P
SimpleGrid, helpdesk, accounting, new
communities PY4
SGW.SG
D
Reusable SijmpleGrid modules. TeraGrid science
gateway program will open an online learning
service for gateway development by integrating
SimpleGrid documentation, progressive online
courses, and common components contributed
from other existing gateways. All the training,
coding, and prototyping exercises will be
conducted through web browser.
To NOS
Aug-08
SGW.SG 1.4.1.1 P
SimpleGrid, helpdesk, accounting, new
communities PY5
Aug-09 Jul-10
Continue to develop and support the SimpleGrid
SGW.SG 1.4.1.1.1 M online training service for building new science
Aug-09 Jan-10
gateways
Develop a SimpleGrid prototyping service to
support virtualized access to TeraGrid by
SGW.SG 1.4.1.1.2 M efficiently converting a new community application
Aug-09 Jul-10
to a science gateway application that provides web
access for sharing within the community.
Develop a streamlined packaging service for new
SGW.SG 1.4.1.1.3 M
communities to create their science gateway
Feb-10 Jul-10
software based on the use of the SimpleGrid
prototyping service
Develop a user-level TeraGrid usage service
SGW.SG 1.4.1.1.4 M
within SimpleGrid based on the community
Feb-10 Jul-10
account model and attributes-based security
services
Work with potential new communities to improve
SGW.SG 1.4.1.1.5 M the usability and documentation of the proposed
Aug-09 Jul-10
gateway support services
Conduct education and outreach work using the
SimpleGrid online training service and related
SGW.SG 1.4.1.1.6 M science gateway technologies in the contexts of
Aug-09 Jul-10
undergraduate, graduate, and K-12 education and
training and document experiences
SGW.CA 1.4.5 P Community Accounts PY5 Aug-09 Jul-10
SGW.CA 1.4.5.1 M Review and documentation of requirements Aug-09 Oct-09
Jul-09
Shaowen Wang[8%],
Yan Liu[80%]
NCSA
NCSA Shaowen Wang[10%],
Yan Liu[80%]
NICS Victor
Hazlewood[30%];SDSC
Nancy Wilkins-Diehr[15%]
Delayed
Ongoing
In progress
In progress
Will be using CI Tutor for this in the
extension year.
In progress
Ongoing
Work will complete in the extension In progress
VM deployment in th extension year In progress
Deferred Waiting for GRAM5 install Deferred
In progress
In progress
Complete SciDAC, TG10 In progress
Complete
In progress
SGW.CA 1.4.5.2 M Communication of requirements Nov-09 Jan-10 Complete In progress
SGW.CA 1.4.5.3 M Specification of a standard deployment model(s) Feb-10 Apr-10 Complete In progress
SGW.CA 1.4.5.4 M Deployment of standard model on one gateway May-10 Jul-10 In progress
SDSC Nancy Wilkins-
SGW.GCD 1.4.6 P Gateways Code Discovery PY5 Aug-09 Jul-10 Diehr[5%]; UNC Josh Coyle
[30%] John McGee [5%]
SGW.GCD 1.4.6.1 P
RENCI Gateway Services advertised appropriately
in TG Info Services
Aug-09 Aug-09 Done Done
SGW.GCD 1.4.6.2 P Schema for Gateway services and capabilities Aug-09 Sep-09 Done Done
SGW SGW END
SimpleGrid portal and tutorial server have
been deployed on TeraGrid gateway cluster
for two gateway developers and new
gateway tutorials (TeraGrid'09 and
SciDAC'09). As SimpleGrid is released with
new versions, the online tutorial is being
improved to incorporate core features of
Web 2.0 user interface technologies. New
tutorial experiences have been adopted in
the online learning service. The online
tutorial is being modified to incorporate
efficient ways for new user support based on
our gateway helpdesk support experience.
We will release the online learning service
in mid-PY05 for targeted evaluation.
Pending attribute implementation,
which is delayed until GT5 installs
complete.
installing @ NICS the Science
Gateway kit, the commsh toolset
and began discussions of
implementation of this with
GridAMP as a start. A survey of
what is being done and what is
needed as requirements is still
needed.

TeraGrid US PY5 Project Plan (as of 1/7/09)
Project-ID
WBS
O P M
Description
Work Package Dependencies Planned
Task
Cross
Area
Start
Date
End
Date
Resources
Name [%FTE]
PY5Q3
Status
Update
US 6.0 Operations: Frontline User Support
US.Coord 6.0.1 O US Area Coordination PSC Sergiu Sanielevici (30%)
0 6.0.1.3 O US Area Coordination PY5
user interface
council
AUS, NOS,
UFP
US.Eng 6.0.2 O User Engagement PY5 Aug-09 Jul-10
Aug-09 Jul-10 ongoing
PSC David O'Neal(25%),
Richard Raymond(20%),
Raghurama Reddy(45%),
Sergiu Sanielevici (30%)
US.Eng 6.0.2.1 O User Champion Coordination Aug-09 Jul-10 ongoing
US.Eng 6.0.2.2 O Campus Champion Coordination Aug-09 Jul-10 ongoing
US.Eng 6.0.2.3 O Startup and education grant support coordination Aug-09 Jul-10 ongoing
US.Eng 6.0.2.4 O Analyze and report usage patterns Aug-09 Jul-10 ongoing
US.Eng 6.0.2.5 P TG User Survey PY5 question input All Aug-09 Jul-10 external contractor DONE
PSC David O'Neal(10%), ongoing
US.Tick 6.0.3 O Share and maintain best practices for ticket resolution across all RPs PY5 Aug-09 Jul-10
Richard Raymond(30%),
Raghurama Reddy(10%),
Sergiu Sanielevici (10%)
US.Tick 6.0.3.1 O
Focus on providing users with a substantive clarification of the nature of the problem and
the way forward
Aug-09 Jul-10 ongoing
US.Tick 6.0.3.2 O Focus on the coordinated resolution of problems spanning RPs and systems Aug-09 Jul-10 ongoing
US.Tick 6.0.3.3 O Stale ticket count reaches zero Aug-09 Jul-10 ongoing
US.Tick 6.0.3.4 O At least 85% survey ratings for promptness and quality Aug-09 Jul-10 ongoing
Estabrook [100%], ongoing
US.Cnslt.RP.NCSA 6.0.5.5 O NCSA RP User Services - Consulting Operations Apr-10 Mar-11 Jackson[100%], John [100%],
US.Cnslt.RP.IU 6.0.5.6 O IU RP User Services - Consulting Operations Apr-10 Mar-11 ongoing
US.Cnslt.RP.LONI 6.0.5.7 O LONI RP User Services - Consulting Operations Apr-10 Mar-11 Jundt [50%], Xu [100%] ongoing
Loftis [50%], Lucio [100%], ongoing
US.Cnslt.RP.NICS 6.0.5.8 O NICS RP User Services - Consulting Operations Apr-10 Mar-11
Sharkey [100%], Wong [50%],
Halloy [50%], Crosby [50%],
TBD [250%]
Blood[25%], Costa[90%], ongoing
US.Cnslt.RP.PSC
US.Cnslt.RP.PU
6.0.5.10
6.0.5.11
O
O
PSC RP User Services - Consulting Operations
PU RP User Services - Consulting Operations
Apr-10
Apr-10
Mar-11
Mar-11
Gomez[45%], Madrid[35%],
Maiden[20%], Nigra[70%],
Raymond[3%],
Sanielevici[2%] Wang[5%]
P. Smith [20%], New person
[100%]
ongoing
US.Cnslt.RP.SDSC 6.0.5.12 O SDSC RP User Services - Consulting Operations Apr-10 Mar-11
Choi [5%], Greenberg [15%],
Tatineni [55%], Wolter [100%]
ongoing
ongoing
US.Cnslt.RP.TACC 6.0.5.13 O TACC RP User Services - Consulting Operations Apr-10 Mar-11
Turner [40%], Wilson [40%],
Sadahiro [50%]
ongoing

TeraGrid AUS Project Plan
Work Package Dependencies Planned
Resources
6/30/10 AD
Status
QSR 2010Q2 Report Values
6/30/10 AD Notes
3/31/10 AD
Status
QSR 2010Q1 Report Values
3/31/10 AD Notes
Project-ID
WBS
OPMD
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
AUS 10 Area: Advanced User Support
10.0 AUS Area Coordination
AUS.Coord 10.0.2 O AUS Area Coordination PY5 Aug-09 Jul-10
Majumdar [75%],Alameda[0%], Sanielevici
[0%],Barth[0%], Cheesman[0%],Jundt[0%],
Crosby[0%]
Total 0.75
10.1 Advanced Support Projects
AUS.LIB 10.1.1 P Library Database PY5 Aug-09 Jul-10
NICS Mark Fahey[10%], Intern[50%], HPC Ops
Staff[10%]
AUS.LIB 10.1.1.1 P implement wrappers for ld and aprun Aug-09 Oct-09 100% done 100% done
AUS.LIB 10.1.1.2 P test wrappers; write mining scripts Nov-09 Jan-10 100% done 100% done
AUS.LIB
AUS.LIB
10.1.1.3 P
10.1.1.4 P
wrappers in production mode; implement internal
web interface; share beta release with other
interested sites Feb-10 Apr-10 100% done
everything in production; begin using as a tool to
help manage software; prepare full release;
share with other interested sites May-10 Jul-10 90% done
AUS.MPI 10.1.2 P MPIg Support PY5 Aug-09 Jul-10
AUS.MPI 10.1.2.1 P
Work with ASTA apps groups to port to MPIg on
TG sites. Continue investigation of running apps
at full scale on Ranger. Aug-09 Oct-09
AUS.MPI 10.1.2.2 P Attempt cross-site runs with MPIg on TG sites. Nov-09 Jan-10
AUS.MPI 10.1.2.3 P
NIU Undergrad[29%], Grad[75%], Nick
Karonis[25%]
100% done
(since we
postponed TG
site runs)
100% done
(since we
postponed TG
site runs)
Work with largest TG RP providers to enable
cross-site runs (this is being postponed due to
unable to use MPIg on TG sites). Instead adding
new task of integraing UPC and MPI for user
apps Feb-10 Apr-10 10% done
Investigate intetrating clouds, possibly with
AUS.MPI 10.1.2.4 P Globus, to run MPIg jobs. May-10 Jul-10 80% done
10.2 Advanced Support Operations
AUS.ASTA 10.2.1 O Advanced Support TG Applications
AUS.ASTA 10.2.1.2 O Advanced Support TG Applications PY5 Aug-09 Jul-10
NCSA 0.6 FTE
Jay Alameda [20%]
Dodi Heryadi [0%]
Seid Koric [0%]
Rick Kufrin[25%]
Sudhakar Pamidighantam [0%]
Mark Straka[15%]
Ahmed Taha[0%]
Michelle Gower[0%]
David Bock [0%]
Mark Vanmoer [0%]
PSC 3.15 FTE
Mahin Mahmoodi [10%]
The tracking database is in
production; the internal web interface
was dropped as a deliverable earlier,
rather a suite of scripts to provide the
same functionality.
( g 95% done
have not completed the license
agreement as we work with the
lawyers.) We have shared the beta
with the new NOAA center coming
online at ORNL. We are using the
tool to help guide our software
maintenance. Full release is not
quite done - licensing agreement and
manual need some more time.
Due to unable to use MPIg on TG
sites we are postponing the task of
running full scale)
Due to unable to use MPIg on TG
sites cross-site runs on TG sitesis
postponed for now
Some of the UPC had been
integrated into the app, but based on
discussions with user, focus has now
shifted from UPC to the more
ubiquitous OpenMP and pthreads.
We are now 10% done in that new
100% done
(since we
postponed TG
site runs)
100% done
(since we
postponed TG
site runs)
direction.
10% done
successfully integrated MPIg and
Globus with NIMBUS and ran
application on cloud. RPs have yet
to embrace cloud computing, moving
on to OpenMP and pthreads.
If/when return to clouds, need to add
vendor MPI integration.
The tracking software is in
production on both kraken and
athena. Some scripts to mine data
have been developed. The manual
is not quite done for the full release.
A beta release with incomplete docs
could be shared with interested
parties.
Due to unable to use MPIg on TG
sites we are postponing the task of
running full scale)
Due to unable to use MPIg on TG
sites cross-site runs on TG sitesis
postponed for now
Redirecting this task to integrate
UPC and MPI

TeraGrid AUS Project Plan
Work Package Dependencies Planned
Resources
6/30/10 AD
Status
QSR 2010Q2 Report Values
6/30/10 AD Notes
3/31/10 AD
Status
QSR 2010Q1 Report Values
3/31/10 AD Notes
Project-ID
WBS
OPMD
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
Raghu Reddy [30%]
John Urbanic [25%]
Joel Welling [50%]
Philip Blood [35%]
Roberto Gomez [40%]
Marcela Madrid [20%]
David O'Neal[50%]
Yang Wang [30%]
Anirban Jana [25%]
SDSC 1.75 FTE
Natasha Balac[15%]
Dong Ju Choi[15%]
Amit Chourasia[10%]
Yifeng Cui[40%]
Dmitry Pekurovsky [25%]
Wayne Pfeiffer[10%]
Mahidhar Tatineni[20%]
Ross Walker[20%]
TBD/Doc etc.[20%]
Purdue 0.45 FTE
Phil Cheeseman[45%]
LSU 0.1 FTE
Adam Jundt [10%]
Honggao Liu /POC[0%]
ORNL 0.2 FTE
John Cobb, backup POC ORNL[0%]
Mei Li Chen, backup POC[10%]
Vickie Lynch, POC ORNL[10%]
NICS 0.8 FTE
Lonnie Crosby, POC NICS[40%]
Christian Halloy[20%]
Kwai Wong[20%]
Bruce Loftis, backup POC[0%]
TACC 3.85 FTE
Bill Barth, POC TACC
John Cazes
Lars Koesterke
B. D. Kim
Hang Liu
Robert McLay
Kent Milfeld
John Peterson
Karl Schulz (backup POC)
TACC total 3.85
IU 0.0 FTE
Don Berry[0%]
Ray Shepperd POC[0%]
ANL 0.31 FTE
Joe Insley[31%]
AUS.ASP 10.2.2 O Advanced Support Projects
AUS.ASP 10.2.2.2 O Advanced Support Projects PY5 Aug-09 Jul-10
Total 11.06
NCSA 0.4 FTE
Jay Alameda [20%]
Dodi Heryadi [0%]
Seid Koric [0%]
Ahmed Taha[0%]
Michelle Gower[0%]
David Bock [0%]
Mark Vanmoer [0%]
Mark Straka[20%]
PSC 1.65 FTE

TeraGrid AUS Project Plan
Work Package Dependencies Planned
Resources
6/30/10 AD
Status
QSR 2010Q2 Report Values
6/30/10 AD Notes
3/31/10 AD
Status
QSR 2010Q1 Report Values
3/31/10 AD Notes
Project-ID
WBS
OPMD
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
Nick Nystrom [15%]
Mahin Mahmoodi [40%]
Raghu Reddy[5%]
Joel Welling [30%]
David O'Neal[15%]
Anirban Jana[10%]
Roberto Gomez[20%]
Yang Wang [15%]
Phil Blood [15%]
SDSC 1.05 FTE
Chourasia [15%]
Pfeiffer[40%]
Dmitry Pekurovsky[10%]
Tatineni[5%]
Balac[10%]
Ross Walker [25%]
Purdue 0.3 FTE
TBD 30%
LSU 0.1 FTE
Adam Jundt [10%]
TACC 1.9 FTE
Bill Barth [X%]
others/TACC
TACC total 1.9
IU 0.0 FTE
Don Berry [0]%
NICS 0.4 FTE
Christian Halloy[20%]
Kwai Wong[20%]
AUS.ASEOT 10.2.3 O Advanced Support EOT
AUS.ASEOT 10.2.3.2 O Advanced Support EOT PY5 Aug-09 Jul-10
Total 5.85
NCSA 0.25 FTE
Jay Alameda [10%]
Sudhakar Pamidighantam [0%]
David Bock [0%]
Mark Vanmoer [0%]
Mark Straka[15%]
Purdue 0.2 FTE
TBD[20%]
PSC 0.7 FTE
Nick Nystrom [10%]
Philip Blood [10%]
Marcela Madrid[30%]
Shawn Brown[10%]
Mahin Mahmoodi [10%]
NICS 0.38 FTE
Lonnie Crosby[38]
SDSC 0.4 FTE
Yifeng Cui[10%]
Dmitry Pekurovsky[15%]
TBD/doc[15%]
TACC 0.85 FTE
Lars Koesterke
Kent Milfeld [X%]
Bill Barth [X%]
TACC total 0.85
LSU 0.05 FTE

TeraGrid AUS Project Plan
Work Package Dependencies Planned
Resources
6/30/10 AD
Status
QSR 2010Q2 Report Values
6/30/10 AD Notes
3/31/10 AD
Status
QSR 2010Q1 Report Values
3/31/10 AD Notes
Project-ID
WBS
OPMD
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
Adam Jundt [5%]
IU 0.0 FTE
Don Berry [0%]
Total 2.88

Work Package Dependencies Planned
Resources
(per GIG budget 7/7/08)
6/30/10 AD
Status
QSR 2010Q2 Report Values
6/30/10 AD Notes
3/31/10 AD
Status
QSR 2010Q1 Report Values
3/31/10 AD Notes
Project-ID
WBS
O P M
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
UFP 9 Area: User-facing and Core Services Projects
UFP 9
Objective: User-facing Projects:
Maintainance and Development
May-07 Jul-10
UFP.Coord 9.0 O UFP Area Coordination Aug-07 Jul-10
UFP.Coord 9.0.3 O UFC Area Coordination PY5 Aug-09 Jul-10 SDSC David Hart [93%] Ongoing Ongoing
UFP.Coord 9.0.3.2 M Quarterly Reporting (9/30, 12/31, 3/31, 6/30)
UFP.Coord
9.0.3.3
Lead UFC-RP working group for coordination
O of RPs with UFC
UFP.Acct
NCSA Steve Quinn[20%],
Michael Shapiro[20%];
Ongoing
Ongoing
9.6 O Accounting Operations PY5 Aug-09 Jul-10
SDSC Leo Carson[44%],
Tiffany Duffield[17%], Henry
Jaime[7%]
UFP.Acct 9.6.1 O TGCDB and AMIE PY5 Aug-09 Jul-10
UFP.Acct
Ongoing operations and maintenance of the
Ongoing
Ongoing
9.6.1.1
central TGCDB-AMIE environment and
O servers Aug-09 Jul-10
Leo Carson, H. Jaime (SDSC)
(0.36 FTE)
UFP.Acct
Update TGCDB tables and code to support
Ongoing
Ongoing
9.6.1.2
evolving TG requirements, including addition
of new RP resources into TGCDB; changes to
O existing resources. Aug-09 Jul-10
Michael Shapiro (NCSA) (0.10
FTE), Steve Quinn (NCSA)
(0.10 FTE)
UFP.Acct
Michael Shapiro (NCSA) (0.10 Ongoing
Ongoing
9.6.1.3
Update AMIE tables and code to support
O evolving TG requirements Aug-09 Jul-10
FTE), Steve Quinn (NCSA)
(0.10 FTE)
UFP.Acct 9.6.2 O tgusage and gx-map PY5
UFP.Acct
Maintain and perform critical fixes to tgusage
utility as directed by trouble tickets and in
Tiffany Duffield (SDSC)
[replacing former staff] (0.17
Ongoing
9.6.2.1 O support of evolving UFC needs. Aug-09 Jul-10 FTE)
UFP.Acct
Maintain and perform critical fixes to gx-map
Ongoing
utility as directed by trouble tickets and in
Leo Carson (SDSC) (0.15
9.6.2.2 O support of evolving UFC needs Aug-09 Jul-10 FTE)
UFP.Acct 9.6.3 O Metrics PY5
UFP.Acct
Maintain web-based TGU query and reporting
Ongoing
system, in current form and later as integrated
9.6.3.1 O part of TGUP. Aug-09 Jul-10
UFP.Acct
Metrics reporting for QSRs, including updated
Ongoing
metrics as capabilities supported in TGCDB
9.6.3.2 O and POPS (9/30, 12/31, 3/31, 6/30) Aug-09 Jul-10
UFP.UDoc 9.1 R TG Documentation
UFP.UDoc 9.1.1 R TG Docs PY4 Aug-08 Jun-09
Fariba Fana[100%],
Diana Diehl[25%]
UFP.UDoc 9.1.1.3
TG Resource Catalog integrated with MDS +
M
enhancements
Core2 Aug-08 Jul-09 Initiated
UFP.UDoc 9.1.1.4 M PI User Services Log integrated with TGCDB Core2 Aug-08 Jun-09 In progress
UFP.UP 9.3 User Portal
UFP.UP 9.3.2 R TG User Portal PY4
Enhanced user views, broken down by field of
Aug-08 Jun-09
John Boisseau[8%],
Maytal Dahan[18%],
Patrick Hurley[35%],
Praveen Nuthalapati[45%],
Steve Mock[25%],
TACC NOS[15%]
science, across the user portal. Users will be
UFP.UP 9.3.2.3 M able to filter systems, software, gateways, data
collections, training, news, information
services, etc. by scientific domain (e.g.
biology chemistry etc )
Aug-08 Mar-09 Cancelled
Expanded personalization and collaboration
features across the user portal, this includes
UFP.UP 9.3.2.4 M allocation management, customized resource
Aug-08 Jun-09 In progress
listing, collaboration of proposals, file space,
etc
UFP.UR 9.7 O User Requests
UFP.UR 9.7.1 O User Requests PY5 Aug-09 Jul-10 NCSA V. Halberstadt[100%]
Student employee identified at SDSC to
pursue this as summer activity.
Being actively tested by services-wg.
Troubleshooting a few Liferay issues.
Effort consumed by Liferay
troubleshooting. Focusing on other
priorities.
File sharing capability added to file
manager portlet in TGUP and TG Mobile.
Friendly user mode in Q2. Released early
Q3. Ticket system interface deployed in
TGUP
Ongoing
Ongoing
Ongoing
Ongoing
Delayed
In progress
Delayed
In progress
Due to ongoing challenges with Liferay
learning curve
Completion delayed due to Liferay
learning curve and operational
challenges
Will be revisited when Liferay situation
under control
Focusing on New User process first.

Work Package Dependencies Planned
Resources
(per GIG budget 7/7/08)
6/30/10 AD
Status
QSR 2010Q2 Report Values
6/30/10 AD Notes
3/31/10 AD
Status
QSR 2010Q1 Report Values
3/31/10 AD Notes
Project-ID
WBS
O P M
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
UFP.UR
9.7.1.1
Create/establish user accounts in TGCDB,
O mail new user packets, collect RP site logins. Aug-09 Jul-10
UFP.UR
Support PI requests for removal of users from
9.7.1.2 O allocations Aug-09 Jul-10
UFP.UR
Vet/review user information for new PIs and co-
9.7.1.3 O PIs Aug-09 Jul-10
UFP.UR
Serve as first point of contact for general user
9.7.1.4 O inquiries re: UFC procedures Aug-09 Jul-10
UFP.AP 9.8 O Allocations Process
UFP.AP
O Allocations Process PY4
UFP.AP
D POPS interaction with RDR
UFP.AP
** This process may change in
late PY4, early PY5
This process will continue in
PY5
TACC Kent Milfeld[20%],
Marg Murray[20%], Valerie
Alvarez[25%]
9.8.1 O Allocations Process PY5
UFP.AP
Quarterly TRAC Meetings (9/09, 12/09, 3/10,
Ongoing
Ongoing
9.8.1.1 M 6/10) Aug-09 Jul-10
UFP.AP
Ongoing
Ongoing
Processing of intermittent TRAC submissions
9.8.1.2 O (supplements, justifications, etc) Aug-09 Jul-10
UFP.AP
Ongoing review and processing of Startup and
Ongoing
Ongoing
9.8.1.3 O Education requests Aug-09 Jul-10
UFP.AP
Maintenance and updates to Allocations policy
Ongoing
Ongoing
9.8.1.4 O and procedures documentation Aug-09 Jul-10
UFP.AP 9.8.1.5 O Recruiting and supporting TRAC members Ongoing Ongoing
UFP.POP 9.9 O Resource Requests (POPS)
UFP.POP
NCSA Steve Quinn[10%],
9.9.1 O Resource Requests (POPS) PY5
Ester Soriano[20%]
UFP.POP
Support for Quarterly TRAC meetings (9/09,
Ongoing
9.9.1.1 M 12/09, 3/10, 6/10) Aug-09 Jul-10
UFP.POP
Operate, maintain and update POPS system
Ongoing
9.9.1.2 O in support of allocations process Aug-09 Jul-10
UFP.IP 9.10 O Information Presentation/Web Presence
UFP.IP
SDSC Michael Dwyer[10%];
Information Presentation/Web Presence
TACC Patrick Hurley[25%];
9.10.1 O PY5 Aug-09 Jul-10 UC Tim Dudek[50%]
UFP.IP
Maintenance and updates for deployed portal
Ongoing
9.10.1.1 O capabilities (SSO, etc.) Aug-09 Jul-10 P. Hurley (TACC)
UFP.IP
Support for RPs entering information into
Ongoing
9.10.1.2 O catalogs, monitors, and news services. Aug-09 Jul-10 M. Dwyer (SDSC) (0.05 FTE)
UFP.IP
Ongoing maintenance, updates, and
Ongoing
operations for existing catalogs: Compute and
9.10.1.3 O data resources, CTSS, Gateways, etc. Aug-09 Jul-10 M. Dwyer (SDSC) (0.05 FTE)
UFP.IP
UFP.IP
UFP.IP
UFP.IP
Maintenance and updates to TeraGrid's Batch
Queue Prediction Service, including addition
9.10.1.4
of new resources, user support and bug fixes,
O by QBETS team at UCSB. Aug-09 Jul-10
G. Obertelli (UCSB) (0.50
FTE)
Provide statistics and metrics for QSR to
assist in measuring success for TG efforts that
appear in web/portal/KB behavior (9/30,
9.10.1.5 O 12/31, 3/31, 6/30) Aug-09 Jul-10 T. Dudek (ANL) (0.25 FTE)
Administration of WebSphere servers and
9.10.1.6 O infrastructure Aug-09 Jul-10 Subcontract
9.10.1.7 O
Administration of TeraGrid wiki for
problems/spam and fix creation of secure
areas/user access etc. Aug-09 Jul-10 T. Dudek (ANL) (0.25 FTE)
UFP.QA 9.11 O User Information Quality Assurance
UFP.QA
UFP.QA
9.11.1 O User Information Quality Assurance PY5 Aug-09 Jul-10
Run Web Link Validator weekly and correct
9.11.1.1 O broken links Aug-09 Jul-10
IU Andy Orahood[10%], Julie
Thatcher[25%], Paul
Brown[50%], Mary
Hrovat[20%], Jonathon
Bolte[25%]; SDSC Diana
Diehl[55%]; UC Tim
Dudek[50%]
D. Diehl (SDSC) (0.55 across
all items here)
Ongoing
Done
Ongoing
Ongoing
Ongoing
Ongoing
Cancelled/M
oved
Ongoing
Ongoing
User News application ported from
SDSC's Oracle to TeraGrid postgreSQL.
Ongoing
Done
Ongoing
Ongoing
Delayed
Ongoing
Ongoing
Ongoing
Ongoing
Ongoing
Ongoing
Ongoing
Cancelled/M
oved
Ongoing
Ongoing

Work Package Dependencies Planned
Resources
(per GIG budget 7/7/08)
6/30/10 AD
Status
QSR 2010Q2 Report Values
6/30/10 AD Notes
3/31/10 AD
Status
QSR 2010Q1 Report Values
3/31/10 AD Notes
Project-ID
WBS
O P M
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
UFP.QA
Respond to user documentation correction
Ongoing
Ongoing
requests and initiate corrections to major
9.11.1.2 O errors within 1 business day. Aug-09 Jul-10 D. Diehl (SDSC)
UFP.QA
D. Diehl (SDSC); IU KB team; Ongoing
Ongoing
9.11.1.3
Monitor and compile user feedback to UFC
web presence and procedures Aug-09 Jul-10
T. Dudek (ANL); M. Dahan
(TACC)
UFP.QA
Respond to content update requests for
Ongoing
Ongoing
9.11.1.4 O www.teragrid.org within 1 business day. Aug-09 Jul-10 T. Dudek (ANL) (0.25 FTE)
UFP.QA
Provide central coordination of TG-wide
Ongoing
Ongoing
documentation contributions and updates
submitted from across the GIG and RP sites,
using capabilities within the Liferay
9.11.1.5 O environment. Aug-09 Jul-10 D. Diehl (SDSC)
UFP.QA
Conduct ongoing series of coordinated
reviews to evaluate, synchronize and update
documentation, KB and other web info related
Ongoing Review and release of AUS performance Delayed
pages into Liferay.
9.11.1.6 O to TG-wide user info areas. Aug-09 Jul-10 D. Diehl (SDSC); IU KB team
UFP.QA
Completion of 250 new KB documents, some
Ongoing
Ongoing
in response to US and user inputs through the
9.11.1.7 O ticket system. Aug-09 Jul-10 IU KB team
UFP.QA
Ongoing updates and special requests for
Ongoing
Ongoing
web content including posting information for
EOT and ER areas, and conference/workshop
9.11.1.8 O sites. Aug-09 Jul-10 T. Dudek (ANL) (0.25 FTE)
UFP.EUA 9.12 P Enhanced TeraGrid User Access
UFP.EUA
Enhanced TeraGrid User Access PY4
Linking Shibboleth login to TGUP login in
NOS provided prototype Shibboleth-
UFP.EUA
D TGUP test instance
Liferay integration in April. Work
In progress progressing by TGUP team.
In progress
Complete user account management,
UFP.EUA
D including end-to-end Shibboleth authentication
in production In progress In progress
Addition of customization and personalization
UFP.EUA
D capabilities to TGUP
Cancelled
In progress
Effort consumed by Liferay
troubleshooting. Focusing on other
priorities.
UFP.EUA
Interactive capabilities for job submission in
D
TGUP Delayed Delayed
UFP.EUA D Job reservations form in TGUP Delayed Delayed
UFP.EUA
UFP.EUA
TGUP features combined into a workflow
D including file management, job submission
and remote visualization
Co-scheduling proof of concept portlet
D
Delayed
In progress
File sharing service deployed in TGUP in
July.
Delayed
MCP portlet prototyped. Staff effort
redirected to other efforts during part of
Q2. Populating with user data from
various sources. Anticipating friendly user
release in Q3
In progress
Catching up with content updates delayed
during the Liferay content freeze.
Focusing on New User process first.
MCP portlet in development. Likely for Q2
release.
UFP.EUA
NCSA Ester Soriano[10%],
9.12.1 P Enhanced TeraGrid User Access PY5 Aug-09 Jul-10
Steve Quinn[10%]; TACC
Praveen Nuthulapati[100%],
Steve Mock [50%]
UFP.EUA 9.12.1.1 P Users able to create TG logins at the TGUP Aug-09 Oct-09 Ongoing Implementation plan approved Ongoing Implementation plan in progress.
UFP.EUA 9.12.1.2 P Job submission interface in TGUP Nov-09 Jan-10 Delayed Delayed
UFP.EUA
Users can use campus credentials to
Apr-10 In progress NOS team delivered prototype Liferay- In progress
9.12.1.3 P authenticate to TGUP
Feb-10
Shibboleth integration.
UFP.EUA
Cancelled
9.12.1.4 P
Lustre-WAN access via TGUP with cross-TG
data movement
UFP.RAM 9.13 P Resource Authorization and Management
UFP.RAM
Mar-10
Jul-10 Cancelled New file sharing feature in TGUP file
manager portlet provides partial
capability, including cross-TG data
movement.
NCSA Ester Soriano[60%],
Steve Quinn[25%], Mike
Shapiro[10%]; TACC Maytal
9.13.1 P
Resource Authorization and Management
PY5 Aug-09 Jul-10
Dahan[50%]; PSC Edward
Hanna[10%], Rob Light[10%]
UFP.RAM 9.13.1.1 P Improved user authorization process Aug-09 Oct-09 Ongoing Ongoing
Full completion will not be possible due to
Lustre-side issues. TGUP file manager
now supports Lustre-WAN access via
systems where Lustre-WAN is mounted.

Work Package Dependencies Planned
Resources
(per GIG budget 7/7/08)
6/30/10 AD
Status
QSR 2010Q2 Report Values
6/30/10 AD Notes
3/31/10 AD
Status
QSR 2010Q1 Report Values
3/31/10 AD Notes
Project-ID
WBS
O P M
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
UFP.RAM
UFP.RAM
9.13.1.2 P
Expanding user capabilities for managing
allocations and usage
Nov-09
9.13.1.3 P POPS integrated with TGUP Feb-10
UFP.RAM
Resource Advisor made available to users
9.13.1.4 P within UFP systems
Mar-10
UFP.IIS 9.14 P RP Integration and Information Sharing
UFP.IIS
RP Integration and Information Sharing
P PY4
UFP.IIS
RDR system moved into production
environment
D
UFP.IIS
Storage usage record transport and
D processing mechanisms implemented
UFP.IIS
Multi-audience TGCDB and POPS reporting
capabilities implemented within TGUP as UFC
D development permits
UFP.IIS
Jan-10 Ongoing New resource request interface deployed Ongoing
in POPS. Work beginning on revised
POPS entry screens.
Apr-10 Cancelled Cancelled Finally admitting this is unlikely to happen,
given current staffing levels and other
demands.
Jul-10 On hold On hold Initial prototype may be merged with
existing Resource catalog interface
Delayed Student effort to integrate RDR and
Resource Catalog identified at SDSC for
Q3.
Delayed Due to continued Liferay Challenges
Completed Deployed in production. Initiated Will be completed in Q2
In progress
Allocations query in progress. SU Map In progress
completed. Working to identify production
location for both.
NCSA Ester Soriano[10%],
Mike Shapiro[10%]; PSC
9.14.1 P
RP Integration and Information Sharing
PY5 Aug-09 Jul-10
Edward Hanna[25%], Rob
Light[25%]
UFP.IIS 9.14.1.1 P RDR-based descriptions in POPS Aug-09 Oct-09 Delayed Delayed
UFP.IIS 9.14.1.2 P RDR-based descriptions in TGCDB Nov-09 Jan-10 Delayed Delayed
UFP.IIS
Apr-10 Delayed Likely to pick up in Q3 as student works Delayed
9.14.1.3 P Hardened, updated RDR Feb-10
to integrate Resource Catalog.
UFP.IIS 9.14.1.4 P RDR includes non-compute resources Mar-10 Jul-10 Completed Completed
UFP.UIP 9.15 P User Information Presentation
UFP.UIP P User Information Presentation PY4
UFP.UIP
UFP.UIP
UFP.UIP
UFP.UIP
UFP.UIP
UFP.UIP
UFP.UIP
UFP.UIP
UFP.UIP
UFP
Post-migration alignment of portal and Web
D site look, feel, and organization.
With WebSphere environment, developing
scalable documentation model for TeraGrid
D
Migration to RDR/MDS as definitive source of
D Resource Catalog info
Enhanced TGUP system monitor with
customizable list of resources and expanded
D resource information
Ongoing Will be an ongoing effort for duration of
TeraGrid.
Delayed
In progress External editor/contributor added for Delayed
TG10 content. Working to expand
contributor pool, but proceeding slowly
due to Liferay issues
In progress With summer student at SDSC Delayed
9.15.1 P User Information Presentation PY5 Aug-09 Jul-10
PSC Edward Hanna[20%],
Rob Light[20%]; SDSC
Michael Dwyer[65%]; TACC
Rion Dooley[75%]
Integration of TeraGrid user portal info
Oct-09 Delayed Delayed
9.15.1.1 P services with RDR
Aug-09
Integration of TG catalogs and User News with
Jan-10 Delayed Delayed
9.15.1.2 P RDR
Nov-09
Enhanced system monitors for HPC, storage,
Apr-10 Delayed Delayed
9.15.1.3 P and vis resources
Feb-10
Enhanced integration of user news in TGUP
Jul-10
9.15.1.4 P and user notification
Mar-10
END
Delayed
Delayed
"SU Map" portlet in prototype mode.
Due to Liferay issues

TeraGrid DV PY5 Project Plan (as of 1/11/09)
Work Package Dependencies Planned
Resources
Project-ID
WBS
O P M X
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
DV 3.0
Objective 3.0 Operations:
maintain/sustain current capabilities
DV.Coord 3.0.1 O DAV Area Coordination
DV.Coord 3.0.1.3 O DV Area Coordination PY5 Aug-09 Jul-10
TACC Kelly Gaither [50%],
Chris Jordan[25%]
ongoing
DV.Coord 3.0.1.3.1
Quarterly Reporting (9/30, 12/31, 3/31,
M Aug-09 Jul-10 ongoing
DV.DDM.RP
DV.DDM.RP.NCSA 3.0.1.4 O
DV.DDM.RP.IU 3.0.1.5 O
DV.DDM.RP.LONI 3.0.1.6 O
DV.DDM.RP.NICS 3.0.1.7 O
DV.DDM.RP.ORNL 3.0.1.8 O
DV.DDM.RP.PSC 3.0.1.9 O
DV.DDM.RP.PU 3.0.1.10 O
DV.DDM.RP.SDSC 3.0.1.11 O
DV.DDM.RP.TACC 3.0.1.12 O
O
6/30)
Aggregate RP Distributed Data
Management Ops
NCSA RP Distributed Data Management
Operations
LONI RP Distributed Data Management
Operations
NICS RP Distributed Data Management
Operations
PSC RP Distributed Data Management
Operations
SDSC RP Distributed Data Management
Operations
TACC RP Distributed Data Management
Operations
Apr-10
Apr-10
Mar-11
Mar-11
Alt [40%], B.Butler [75%],
M.Butler [65%], Cai [40%],
Chen [100%], Cribbs [50%],
Glasgow [90%], Kerner [90%],
Loftus [45%] Long [20%]
ongoing
Apr-10 Mar-11 Martinez [100%] ongoing
Apr-10
Apr-10
Mar-11
Mar-11
Kovatch [50%], Hazlewood
[50%], TBD [100%], TBD
[100%], TBD [50%]
Kar[40%], Litzinger[50%],
Nowoczynski[60], Sorge[15],
Stone[80], Yanovich[80]
ongoing
ongoing
Bennett [75%], Cai [95%], ongoing
Apr-10 Mar-11
Chen, S. [40%], Chen, L.
[80%], Dinh [50%], Hom [75%],
Rivers [65%]
Apr-10 Mar-11 Jones[25%] ongoing
DV.DDM.RP.UCANL 3.0.1.13 O
DV.DC.RP O Aggregate RP Data Collections Ops Apr-10 Mar-11
DV.DC.RP.NCSA 3.0.1.14 O
DV.DC.RP.IU 3.0.1.15 O
DV.DC.RP.LONI 3.0.1.16 O
DV.DC.RP.NICS 3.0.1.17 O NICS RP Data Collections Operations Apr-10 Mar-11 Hazlewood [50%], TBD [70%] ongoing
DV.DC.RP.ORNL 3.0.1.18 O
DV.DC.RP.PSC 3.0.1.19 O
DV.DC.RP.PU 3.0.1.20 O PU RP Data Collections Operations Apr-10 Mar-11
Zhao[50%], 2 half time Grad
Students [100%]
ongoing
DV.DC.RP.SDSC 3.0.1.21 O SDSC RP Data Collections Operations Apr-10 Mar-11 Nunes [10%], Wong [25%] ongoing
PY5Q3 Status Update

DV.DC.RP.TACC 3.0.1.22 O TACC RP Data Collections Operations Apr-10 Mar-11 Urban[50%] ongoing
DV.DC.RP.UCANL 3.0.1.23 O
DV.Viz.RP O Aggregate Visualization Ops Apr-10 Mar-11
DV.Viz.RP.NCSA 3.0.1.24 O
DV.Viz.RP.IU 3.0.1.25 O
DV.Viz.RP.LONI 3.0.1.26 O
DV.Viz.RP.NICS 3.0.1.27 O NICS RP Visualization Operations Apr-10 Mar-11 TBD [60%], TBD [60%] ongoing
DV.Viz.RP.ORNL 3.0.1.28 O
DV.Viz.RP.PSC 3.0.1.29 O PSC RP Visualization Operations Apr-10 Mar-11 Foss[90%], new[10%] ongoing
DV.Viz.RP.PU 3.0.1.30 O PU RP Visualization Operations Apr-10 Mar-11
New Person [100%], 2 half time ongoing
grad students [100%]
DV.Viz.RP.SDSC 3.0.1.31 O SDSC RP Visualization Operations Apr-10 Mar-11 Chourasia [25%], TBD [150%] ongoing
Gaither[10%], Burns[50%], ongoing
DV.Viz.RP.TACC 3.0.1.32 O TACC RP Visualization Operations Apr-10 Mar-11
GregSJohnson[30%],
Schneider[50%],
GregPJohnson[50%]
DV.Viz.RP.UCANL 3.0.1.33 O UCANL RP Visualization Operations Apr-10 Mar-11 Insley[11%] ongoing
DV.DPI 3.2
Objective 3.2 Projects: Data Movement
Performance
DV.DPI 3.2.5 P Data Movement Performance PY5 Aug-09 Jul-10
PSC Kathy Benninger[20%],
Bob Budden[50%], Derek
Simmel[60%]
DV.DPI 3.2.5.2
Production tools for scheduled data
P Apr-10 Mar-11 TBD ONGOING
DV 3.3
movement
Objective 3.3 Projects: Global Wide
Area File Systems
DV.GFS
DV.GFS
3.3.3
3.3.3.1
P Global Wide Area File Systems PY5
P Lustre-WAN in Production
Aug-09
Apr-10
Jul-10
Mar-11
NICS Phil Andrews[5%],
Patricia Kovatch[15%],
Nathaniel Mendoza[15%],
Victor Hazlewood[15%]; PSC
Josephine Palencia[50%];
SDSC Jeff Bennett[25%],
Thomas Guptill[25%]; TACC
GFS Engineer[20%]
ONGOING
DV.GFS 3.3.3.6 P
DV 3.4
Evaluation of pNFS readiness for
production
Objective 3.4 Projects: TeraGrid Wide
Data Architect Design
NICS Phil Andrews[10%],
DV.DWG 3.4.4 P Data Architecture PY5 Aug-09 Jul-10
Bruce Loftis[20%]; PSC J Ray
Scott[25%]; TACC Chris
Jordan[50%]
DV.DWG 3.4.4.3 P Physical instantiation of the design 3.4.3.3 TBD Apr-10 Mar-11 ONGOING
DV 3.5 Objective 3.5 Projects: Visualization
DV.Viz 3.5.3 P Visualization PY5 Aug-09 Jul-10
DV
END
Apr-10
Jul-10
ANL Joe Insley [50%]; NCAR
Alan Norton[15%], Viz
TBD[50%]
DELAYED(SOFTWARE
AVAILABILITY)

TeraGrid NOS Project Plan
Work Package Dependencies Planned
Resources
Project-ID
WBS
O P M X
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
Status
NOS 4.0
Objective 4.0 Operations:
maintain/sustain current capabilities
NOS.Coord 4.0.1 O NOS Area Coordination
NOS.Coord 4.0.1.3 O NOS Area Coordination PY5 Aug-09 Jul-10 UCANL Jeff Koerner [25%] ongoing
NOS.Coord 4.0.1.3.1
Quarterly Reporting (9/30, 12/31, 3/31,
M
6/30)
Aug-09 Jul-10 ongoing
NOS.Net 4.0.2 Networking
NOS.Net 4.0.2.3 O Networking Lead PY5 Apr-10 Mar-11
SDSC Tom Hutton [10%];
ANL Linda Winkler [50%]
ongoing
NOS.Net.RP O Aggregate RP Networking Ops Apr-10 Mar-11
NOS.NET.RP.NCSA 4.0.2.4 O NCSA RP Networking Operations Apr-10 Mar-11 Wefel [10%], Shoop [10%] ongoing
NOS.NET.RP.IU 4.0.2.5 Apr-10 Mar-11 ongoing
NOS.NET.RP.LONI 4.0.2.6 O LONI RP Networking Operations Apr-10 Mar-11 ongoing
NOS.NET.RP.NICS 4.0.2.7 O NICS RP Networking Operations Apr-10 Mar-11
Mendoza [50%], TBD [100%], ongoing
Baer [50%]
NOS.NET.RP.ORNL 4.0.2.8 ongoing
Adams[25%],
ongoing
NOS.NET.RP.PSC 4.0.2.9 O PSC RP Networking Operations Apr-10 Mar-11
Benninger[40%],
Huntoon[40%], Lambert[50%],
Lappa[25%], Rapier[25%]
NOS.NET.RP.PU 4.0.2.10 O Purdue RP Networking Operations Apr-10 Mar-11 Lewis [10%] ongoing
Carlson [45%], Dombrowski
NOS.NET.RP.SDSC 4.0.2.11 O SDSC RP Networking Operations Apr-10 Mar-11 [65%], Hutton [20%], Valente
ongoing
[60%]
NOS.NET.RP.TACC 4.0.2.12 O TACC RP Networking Operations Apr-10 Mar-11 Jones [25%] ongoing
NOS.NET.RP.UCANL 4.0.2.13 O UCANL RP Networking Operations Apr-10 Mar-11 Hedden [13%] ongoing
NOS.Sec 4.0.4 O Security Services Apr-10 Mar-11
NOS.Sec 4.0.4.1 O Operational Security Team PY5 Aug-09 Jul-10 PSC James Marsteller [50%] ongoing
NOS.Sec 4.0.4.1.1 O Incident Response activities Aug-09 Jul-10 ongoing
NOS.Sec 4.0.4.1.2
Coordinating operational security issues
O
across the project
Aug-09 Jul-10 ongoing
NOS.Sec 4.0.4.1.3 O Participate in and help lead TAGPMA Aug-09 Jul-10 ongoing
NOS.Sec.RP O Aggregate RP Security Ops Apr-10 Mar-11 ongoing
NOS.Sec.RP.NCSA 4.0.4.2 O NCSA RP Security Operations Apr-10 Mar-11
Barlow [30%], Brooks [15%],
Sharma [60%]
ongoing
NOS.Sec.RP.IU 4.0.4.3 IU RP Securit Operations Apr-10 Mar-11 Cornet [75%] ongoing
NOS.Sec.RP.LONI 4.0.4.4 ongoing
NOS.Sec.RP.NICS 4.0.4.5 ongoing
NOS.Sec.RP.ORNL 4.0.4.6 ongoing

Bennett[30%],
ongoing
NOS.Sec.RP.PSC 4.0.4.7 O PSC RP Security Operations Apr-10 Mar-11
Marsteller[40%],
Shelmire[60%], Simmel[10%],
new[10]
NOS.Sec.RP.PU 4.0.4.8 ongoing
NOS.Sec.RP.SDSC 4.0.4.9 O SDSC RP Security Operations Apr-10 Mar-11 Sakai [30%], Bennett [50%] ongoing
NOS.Sec.RP.TACC 4.0.4.10 O TACC RP Security Operations Apr-10 Mar-11 Murray [25%] ongoing
NOS.Sec.RP.UCANL 4.0.4.11 O UCANL RP Security Operations Apr-10 Mar-11 Leggett [13%] ongoing
NOS.Sec 4.0.4.3
Security Services (kerb, myproxy, CA)
John Quinn [50%]
O Aug-09 Jul-10
PY5
ongoing
Quarterly Reporting (9/30, 12/31, 3/31,
NOS.Sec 4.0.4.3.1 M
6/30)
Aug-09 Jul-10 ongoing
NOS.TOC 4.0.5 O TOC Services Aug-09 Jul-10 ongoing
NOS.TOC 4.0.5.3 O TOC Services PY5 Aug-09 Jul-10 NCSA Ops Staff [200%] ongoing
NOS.TOC 4.0.5.3.1
Quarterly Reporting (9/30, 12/31, 3/31,
M
6/30)
Aug-09 Jul-10 ongoing
NOS.TOC.RP O Aggregate RP Helpdesk Ops Apr-10 Mar-11 ongoing
Cagle [50%], Nickens [20%], ongoing
NOS.TOC.RP.NCSA 4.0.5.4 NCSA RP Helpdesk Operations Apr-10 Mar-11
Pingleton [45%], Roney [75%],
Sellers [75%], Ten Have
[45%], Wells [60%], Wilson
[55%]
NOS.TOC.RP.IU 4.0.5.5 ongoing
NOS.TOC.RP.LONI 4.0.5.6 ongoing
NOS.TOC.RP.NICS 4.0.5.7 ongoing
NOS.TOC.RP.ORNL 4.0.5.8 ongoing
Hackworth[10%], Maiden[5%], ongoing
NOS.TOC.RP.PSC 4.0.5.9 O PSC RP Helpdesk Operations Apr-10 Mar-11 Raymond[2%],
Sanielevici[3%]
NOS.TOC.RP.PU 4.0.5.10 O Purdue RP Helpdesk Operations Apr-10 Mar-11 New person [50%] ongoing
NOS.TOC.RP.SDSC 4.0.5.11 ongoing
NOS.TOC.RP.TACC 4.0.5.12 ongoing
NOS.TOC.RP.UCANL 4.0.5.13 ongoing
NOS.HPCOps 4.0.8 O HPC Operations ongoing
NOS.HPCOps.RP O Aggregate RP HPC Operations Apr-10 Mar-11 ongoing
Bouvet [85%], Fernsler [50%], ongoing
Hoyenga [80%], Khin [80%],
NOS.HPCOps.RP.NCSA 4.0.8.1 O NCSA RP HPC Operations Apr-10 Mar-11
Lapine [75%], Marcusiu
[65%], Parga [75%],
Pflugmacher [10%], J.Quinn
NOS.HPCOps.RP.IU 4.0.8.2 O IU RP HPC Operations Apr-10 Mar-11
[65%] Scharf [50%]
Lowe [100%], Moore[100%],
Miller[100%]
ongoing
Martinez [100%], Leche
NOS.HPCOps.RP.LONI 4.0.8.3 O LONI RP HPC Operations Apr-10 Mar-11 [100%], Giaime [100%],
ongoing
Scheinine [50%]
Walsh [100%], Baer [50%],
NOS.HPCOps.RP.NICS 4.0.8.4 O NICS RP HPC Operations Apr-10 Mar-11 Kovatch [50%], Jones [50%],
ongoing
Ezell [50%], TBD [300%]
NOS.HPCOps.RP.ORNL 4.0.8.5 O ongoing

Albert[85%], Bennett[45%], ongoing
Budden[90%], Flaus[20%],
NOS.HPCOps.RP.PSC 4.0.8.6 O PSC RP HPC Operations Apr-10 Mar-11
Gill[90%], Graham[25%],
Johanson[80%], Kar[40%],
Kochmar[90%],
Palencia[20%] Perrone[50%]
Scott[50%],
ongoing
Sommerfield[80%],
Sorge[75%], Sullivan[50%],
NOS.HPCOps.RP.PU
4.0.8.6
4.0.8.7
O PSC RP HPC Operations Continued
O Purdue RP HPC Operations
Apr-10
Apr-10
Mar-11
Mar-11
Vargo[90%], Vizino[90%],
Webb[50%], Wozniak[80%],
Cubbison[30%], Lappa[25%],
new[55]
P. Smith [10%], Braun [50%] ongoing
Conway [75%], Diegel [50%], ongoing
NOS.HPCOps.RP.SDSC 4.0.8.8 O SDSC RP HPC Operations Apr-10 Mar-11
Fillez [30%], Furman [60%],
Guptill [20%], Hocks [50%],
Kamrath [65%], Khem [60%],
McNew[25%], Silva [25%],
Smallen [5%], Yoshimoto
[50%]
Timm [50%], Carver [50%],
NOS.HPCOps.RP.TACC 4.0.8.9 O TACC RP HPC Operations Apr-10 Mar-11 Anderson [25%], Walling
ongoing
[25%]
NOS.HPCOps.RP.UCANL 4.0.8.10 O UCANL RP HPC Operations Apr-10 Mar-11
Leggett [16%], Insley [18%],
Hedden [45%], Olson [58%]
ongoing
Objective 4.2 Projects: Expanding
NOS 4.2 P
Secure TG Access
ongoing
NOS.Acc 4.2.2 P Expanding Secure TG Access PY4
To WGs,
Jim Basney [20%],
Aug-08 Jul-09
SGW
Terry Fleury [50%]
ongoing
NOS.Acc 4.2.2.2 M Automated DN distribution compliance Aug-08 Dec-09 status update pending
NOS.Acc 4.2.2.4
Requirements/Design for cross-grid
M
Collab.
Aug-08 Jul-10 ongoing
NOS.Acc 4.2.3 P Expanding Secure TG Access PY5 Aug-09 Jul-10
NOS.Acc 4.2.3.1 P
Ubiquitous Adoption of Gateway
accounting
Aug-09
Jul-10
NCSA Jim Basney [25%], Jon
Siwek [50%], Terry Fleury
[25%]
NOS.Acc 4.2.3.2 P INCA tests for Gateway accounting Aug-09 Jul-10 started
NOS.Acc 4.2.3.3 P
TG/OSG authorization interoperability
functional
NOS.Acc 4.2.3.4 P Shibboleth access for campus to TGUP Aug-09 Jul-10 done
NOS 4.3
Objective 4.3 Projects: Operational
Intrumentation (device tracking)
NOS.Inst 4.3.3 P Operational Instrumentation PY5 Aug-09 Jul-10 NCSA Neil Gorsuch [45%]
NOS.Inst 4.3.3.1
Ongoing Monthly, Quarterly and Annual
O
reporting
Aug-09 Jul-10 ongoing
Upgrade Globus Listener to support
NOS.Inst 4.3.3.3 P
GRAM5
Feb-10 Jul-10 done
NOS.Inst 4.3.3.4 P Integration plan for TAIS May-10 Jul-10 ongoing
NOS.INCA 4.4.3 P INCA Improvements PY5 Aug-09 Jul-10
SDSC Kate Ericson [50%],
Shava Smallen[50%]
ongoing
NOS.INCA 4.4.3.1 P QA/CUE views Aug-09 Jul-10 ongoing
NOS.INCA 4.4.3.2 P Link Tickets to Inca results Nov-09 Jan-10 ongoing
Aug-09
Jul-10
pending gram5 deployment at
all sites
not started - depends on 4.2.2.4
above

NOS.INCA 4.4.3.3 P knowledgbase linkage Jan-10 Jun-10 ongoing
NOS.INCA 4.4.3.4 P integrate inca into TGUP Mar-10 Sep-10 ongoing
NOS 4.6
Objective 4.6 Projects: Annual Risk
Assessment
NOS.Risk 4.6.1 P Risk Assessment Aug-08 Jul-09 Jim Rome [10%]
NOS.Risk 4.6.1.1 M ID assessment area and write plan Aug-08 Sep-09 Complete complete
NOS.Risk 4.6.1.2 M Begin assessment of area Aug-08 Dec-09 started
NOS.Risk 4.6.1.3 M Draft report of findings Aug-08 Mar-10 status update pending
NOS.Risk 4.6.1.4 M Final assessment and recommendations Aug-08 Jun-10 status update pending
NOS.QACUE
NOS.QA
Objective X.X Projects: QA/CUE
PY4 Availability Improvement Aug-08 Jul-09
ANL Joe Insley[17%]; IU Mike
Lowe[50%]; LONI Archit
Kulshrestha[25%]; NCSA Dan
Lapine[25%], Luke
Scharf[25%]; NICS Victor
Hazlewood[50%]; PSC
TBD[100%]; Purdue
TBD[50%]; SDSC Thomas
Guptill[50%], Jerry
Greenberg[50%] TACC Da id
NOS.QA
Organize and identify priorities and
SDSC Kate Ericson[12.5%];
Apr-10 Mar-11
timelines for software testing cycle
Shava Smallen[12.5%]
Ongoing
NOS.QA Identify new and existing tests Apr-10 Mar-11
SDSC Kate Ericson[12.5%];
Shava Smallen[12.5%]
Ongoing
NOS.QA Determine Test baselines Dec-08 Mar-09
SDSC Kate Ericson[12.5%];
Shava Smallen[12.5%]
Complete
NOS.QA TG grid services administration guide Aug-09 Jan-10
SDSC Kate Ericson[12.5%];
Shava Smallen[12.5%]
Complete
NOS.QA PY4 Reliability Improvement Aug-08 Dec-09
NOS.QA Identify common INCA errors and classify Aug-08 Mar-10
SDSC Kate Ericson[12.5%];
Shava Smallen[12.5%]
Ongoing
NOS.QACUE O PY5 QA/CUE Leadership Aug-09 Jul-10
SDSC Kate Ericson[12.5%];
Shava Smallen[12.5%]; PSC
Shawn Brown[20%]
NOS.QACUE P Common User interface roll out Aug-10 Jul-10 Shawn Brown[20%] Ongoing
NOS.QA O PY5 Availability Improvement Aug-09 Jul-10
NOS.Email P Maintain TeraGrid.org email services Apr-11 Mar-11 Neil Gorscuh (5%) Complete
NOS
END

Project-ID
WBS
O P M
Work Package Dependencies Planned
Description
Task
Cross
Area
Start
Date
End
Date
Resources
Name [%FTE]
QSR 2010Q2 Report Values
QSR 2010Q1 Report Values
6/30/10 AD Status 6/30/10 AD Notes 3/31/10 AD Status 3/31/10 AD Notes
12/31/09 AD
Status
QSR 2009Q4 Report Values
12/31/09 AD Notes
EOT 5 Area: Education, Outreach and Training
EOT 5.0
Objective 5.0 Operations:
maintain/sustain current capabilities
EOT.Coord 5.0.1 O EOT Area Coordination
EOT.Coord 5.0.1.7 O EOT Area Coordination PY5 Aug-09 Jul-10 Scott Lathrop[50%]
EOT.Coord 5.0.1.7.1 M
working group meetings; info sharing among
100% 75%
Aug-09 Jul-10
RP EOT staff
50%
EOT.Coord 5.0.1.7.2 M
PY5 Quarterly Reporting (9/30, 12/31, 3/31,
100% 50%
Aug-09 Jul-10
6/30)
25%
EOT.COOR 5.0.1.8 O Support HPCU portal infrastructure Aug-09 Jul-10 Shodor contract 100% 75% 50%
EOT.Coord 5.0.1.9 O EOT Area Coordination Extension Aug-10 Jul-11 1.5 FTE for Extension
EOT.Coord 5.0.1.9.1 M
AD and project coodinator activities for Scott
and Elizabeth
Aug-10 Jul-11
EOT 5.1
Objective 5.1 Projects: Training
(HPC University Workforce Development)
EOT.TS 5.1.2 P Objective 5.1 Projects: Training
US, AUSS,
SG, DIV
Aug-09
Training & Seminars PY5 (training offered
EOT.TS 5.1.2.1
To RPs Aug-09 Jul-10
at RP sites; on-line materials developed)
Development of one new training content for
US, AUSS,
EOT.TS 5.1.2.5 P live, synchronous, and/or on-line delivery each
Jul-09 Jul-10
SG, DIV
quarter
EOT.TS.RP 5.1.2.7 O Aggregate RP Training Operations Aug-09 Mar-10
EOT.TS.RP.
5.1.2.7.1 O NCSA RP Training Operations Aug-09 Mar-10
NCSA
EOT.TS.RP.I
5.1.2.7.2 O IU RP Training Operations Aug-09 Mar-10
U
EOT.TS.RP.
5.1.2.7.3 O LONI RP Training Operations Aug-09 Mar-10
LONI
EOT.TS.RP.
5.1.2.7.4 O NICS RP Training Operations Aug-09 Mar-10
NICS
EOT.TS.RP.
5.1.2.7.5 O ORNL RP Training Operations Aug-09 Mar-10
ORNL
EOT.TS.RP.
PSC
Jul-10
5.1.2.7.6 O PSC RP Training Operations Aug-09 Mar-10
IU Robert Ping [20%]
Scott Tiege[20%];
NCSA Sandie
Kappes[50%]; NICS
James Ferguson[30%],
Bruce Loftis[20%]; PSC
Laura McGinnis[30%],
Robin Flaus[5%], Ryan
Omecene[10%], Tom
Maiden[3%], John
Urbanic[3%], Phil
Blood[3%]; SDSC
Edward Jeff Sale[33%];
TACC Chris
Training team 90% Developing two new online courses. 75%
Arnold [50%], File
[80%], Glick [25%],
Kappes [20%]
Ferguson [50%], TBD
[50%]
100% 75%
100% 75%
100% 75%
100% 75%
McGinnis[4%],
Mahmoodi[8%],
Urbanic[15%], Brown,
S[3%], Madrid[5%],
Maiden[35%] Jana[5%]
100% 75%
100% 75%
EOT.TS.RP.
5.1.7.7
PU
O PU RP Training Operations Aug-09 Mar-10
EOT.TS.RP.
Sale [50%] 100% 75%
5.1.7.8 O SDSC RP Training Operations Aug-09 Mar-10
SDSC
EOT.TS.RP.
Turner [10%], Wilson 100% 75%
5.1.7.9 O TACC RP Training Operations Apr-09 Mar-10
TACC
[10%]
EOT.TS.RP.
75%
5.1.7.10 O UC/ANL RP Training Operations Aug-09 Mar-10
UCANL
Training & Seminars Extension (training
EOT.TS 5.1.8.1
offered at RP sites; on-line materials
To RPs Aug-10 Jul-11
developed)
EOT.TS 5.1.8.1.1 P Develop 3 new classes
US, AUSS,
Training team
Aug-10 Jul-11
SG, DIV
EOT.TS 5.1.8.1.2 P
Webcast Parallell Computing and
US, AUSS,
Training team
Aug-10 Jul-11
Programming Class 2 times
SG, DIV
EOT.TS 5.1.8.1.2 P Produce 4 webcasts
US, AUSS,
Training team
Aug-10 Jul-11
SG, DIV
EOT.TS 5.1.8.1.2 P XD Transition place holder
US, AUSS,
Training team
SG, DIV
EOT.RPTRN 5.1.9
O
Extension - Aggregate RP Training
Aug-10 Jul-11
Operations
EOT.RPTRN
Arnold [50%], File
5.1.9.1 O Extension - NCSA RP Training Operations Aug-10 Jul-11
.NCSA
[80%], Glick [25%],
Kappes [20%]
EOT.RPTRN
5.1.9.2
.IU
O Extension - IU RP Training Operations Aug-10 Jul-11
EOT.RPTRN
5.1.9.3
.LONI
O Extension - LONI RP Training Operations Aug-10 Jul-11
EOT.RPTRN
Ferguson [50%], TBD
5.1.9.4 O Extension - NICS RP Training Operations Aug-10 Jul-11
.NICS
[50%]
McGinnis[4%],
EOT.RPTRN
Mahmoodi[8%],
5.1.9.5 O Extension - PSC RP Training Operations Aug-10 Jul-11
.PSC
Urbanic[15%], Brown,
S[3%], Madrid[5%],
Maiden[35%] Jana[5%]
EOT.RPTRN
5.1.9.6
.PU
O Extension - PU RP Training Operations Aug-10 Jul-11
EOT.RPTRN
Sale [50%]
5.1.9.7 O Extension - SDSC RP Training Operations Aug-10 Jul-11
.SDSC
65% Two more in development
50%
50%
50%
50%
50%
50%
50%
50%
50%

Project-ID
WBS
O P M
Work Package Dependencies Planned
Description
Task
Cross
Area
Start
Date
End
Date
Resources
Name [%FTE]
QSR 2010Q2 Report Values
QSR 2010Q1 Report Values
6/30/10 AD Status 6/30/10 AD Notes 3/31/10 AD Status 3/31/10 AD Notes
12/31/09 AD
Status
QSR 2009Q4 Report Values
12/31/09 AD Notes
EOT.RPTRN
Turner [10%], Wilson
5.1.9.8 O Extension - TACC RP Training Operations Aug-10 Jul-11
.TACC
[10%]
EOT.RPTRN
5.1.9.9 O Extension - UC/ANL RP Training Operations Aug-10 Jul-11
.UCANL
EOT.OTS 5.1.10 Extension - Online Training Aug-10 Jul-11 1.04 FTE
EOT.OTS 5.1.10.1 P Develop 3-4 on-line training courses Aug-10 Jul-11
EOT.OTS 5.1.10.2 P XD Transition place holder Aug-10 Jul-11
EOT.TQA 5.1.11 Training and HPC University Quality Aug-10 Jul-11 0.48 FTE
EOT.TQA 5.1.11.1 P QA of training materials Aug-10 Jul-11
EOT.TQA 5.1.11.2 P XD Transition place holder Aug-10 Jul-11
EOT.COMM 5.1.12 TG Community Aug-10 Jul-11 1.25 FTE
EOT.COMM 5.1.12.1 P
Create a TeraGrid social collaboration
environment for learning and community
Aug-10 Jul-11
building
EOT.COMM 5.1.12.2 P XD Transition place holder Aug-10 Jul-11
EOT 5.2
Objective 5.2 Projects: External Relations
(Science Highlights)
EOT.SH 5.2.3 Science Highlights 2010 Jan-10 Nov-10
UC Elizabeth
Leake[10%]
Spans fiscal years
EOT.SH 5.2.3.1 P Science Highlights Production Apr-10 Nov-10 50%
EOT.SH 5.2.3.2 P Review and select science stories to highlight
Froelich,ER team and 100%
Bruett Consulting
EOT.SH 5.2.3.3 P Initiate production for Science Highlights Apr-10 Jul-10 ER team 25%
EOT.SH 5.2.3 Science Highlights 2011 Apr-10 Nov-11 Spans fiscal years
EOT.SH 5.2.4.1 P
Initiate collection of stories for the winning XD
ER team
Apr-11 Nov-11
team
EOT 5.3 Objective 5.3 Projects: Outreach
EOT.OE 5.3.2.2.1 M Close out books on TG'09 Conference Jul-09 Dec-09 On-going Mike - Need to check with John T. On-going Need to check with Matt. On-going Matt is tallking with Towns to get the data.
EOT.OE 5.3.5 Outreach Events PY5 ALL TG Aug-09 Jul-10
IU Robert Ping[20%];
PSC Laura
McGinnis[10%]; SDSC
Ange Mason[20%]
EOT.OE 5.3.5.1 M
presentations, exhibits at conferences and
100% 75%
Aug-09 Jul-10
outreach events
50%
EOT.OE 5.3.5.3 M TG10 Aug-09 Jul-10
90% 25%
Event is in August so effort increases as
20%
the event approaches.
EOT.OE 5.3.5.3.1 M Close out books on TG'10 Conference Jul-10 Dec-10
EOT.OE 5.3.5.5 M Campus Champions meeting at TG10 Jun-10 Jul-10 Hunt
EOT.OE 5.3.5.6 M Campus Champions meeting Spring 2010 Jan-10 Apr-10 Hunt 100%
EOT.OE 5.3.5.7 M
Campus Champion Visits - 2 per quarter
100% 75%
Aug-09 Jul-10
throughout the year
50%
EOT.OE 5.3.5.8 O Campus Champions Consulting Aug-09 Jul-10 100% 75% 50%
EOT.OE 5.3.5.9 M
Campus Champions monthly training sessions
100% 75%
Aug-09 Jul-10
- conference calls and/or sync delivery tools
50%
EOT.OE 5.3.5.10 P Professional Society Outreach Aug-09 Jul-10 Ping [20%] 100% 75% 50%
EOT.OE 5.3.5.10.1 P
Collect information on professional society
Ping 100% 75%
Aug-09 Jul-10
meetings throughout the year
50%
Review and select future outreach events
Ping 100% 75%
EOT.OE 5.3.5.10.2 P quarterly: e.g. AAAS, APS, AGU, ACS, NSTA,
Aug-09 Jul-10
50%
Tapia etc
EOT.OE 5.3.5.10.3 P
PY5 - presentations, exhibits at conferences
100% 75%
Aug-09 Jul-10
and outreach events throughout the year
50%
EOT.OE 5.3.5.11 O TeraGrid Pathways Aug-09 Jul-10 100% 75% 50%
EOT.OE 5.3.5.11.1 O Consulting of under-represented populations AUSS Aug-09 Jul-10 100% 75% 50%
EOT.OE 5.3.5.11.2 O Fellowships for under-represented populations AUSS Aug-09 Jul-10 100% 75% 50%
EOT.OE 5.3.5.11.3 O Mentoring of under-served populations AUSS Aug-09 Jul-10 100% 75% 50%
EOT.OE 5.3.5.12 P
Develop student competitions and awards in
Wiziecki [20%] 100% 75%
Aug-09 Jul-10
conjunction with National Science Olympiad
50%
EOT.OE 5.3.5.12.1 P
Work with Olympiad to establish
Wiziecki 100% 75%
Aug-09 Dec-09
computational science component
50%
EOT.OE 5.3.5.12.2 P Develop competitions Jan-10 May-10
Wiziecki 100% 1 competition completed, one
remaining in May
50% 1 competition completed, one
remaining in May
EOT.OE 5.3.5.12.3 P Deliver competitions and make awards Apr-10 Jul-10
Wiziecki 100% 1 competition completed, one
remaining in May
50% 1 competition completed, one
remaining in May
EOT.OE 5.3.6 Outreach Events Extension ALL TG Aug-10 Jul-11
PSC Laura
McGinnis[10%]; SDSC
Ange Mason[20%] 0.5
FTE for Extension
IU Robert Ping[20%];
EOT.OE 5.3.6.1 M
Campus Champions monthly training sessions
- conference calls and/or sync delivery tools
Aug-10 Jul-11
EOT.OE 5.3.6.2 P Professional Society Outreach Aug-10 Jul-11 Ping [20%]
EOT.OE 5.3.6.3 P
Collect information on professional society
Ping
Aug-10 Jul-11
meetings throughout the year
Review and select future outreach events
Ping
EOT.OE 5.3.6.4 P quarterly: e.g. AAAS, APS, AGU, ACS, NSTA,
Tapia etc
Aug-10 Jul-11
EOT.OE 5.3.6.5 P
Extension - presentations, exhibits at
conferences and outreach events throughout
Aug-10 Jul-11
the year
EOT.OE 5.3.6.10 P
Develop student competitions and awards in
Wiziecki [20%]
Aug-10 Jul-11
conjunction with National Science Olympiad
EOT.OE 5.3.6.11 P
Work with Olympiad to establish
Wiziecki
Aug-10 Dec-10
computational science component
EOT.OE 5.3.6.12 P Develop competitions Jan-11 May-11 Wiziecki
EOT.OE 5.3.6.13 P Deliver competitions and make awards Apr-11 Jul-11 Wiziecki

Project-ID
WBS
O P M
Work Package Dependencies Planned
Description
Task
Cross
Area
Start
Date
End
Date
Resources
Name [%FTE]
QSR 2010Q2 Report Values
QSR 2010Q1 Report Values
6/30/10 AD Status 6/30/10 AD Notes 3/31/10 AD Status 3/31/10 AD Notes
12/31/09 AD
Status
QSR 2009Q4 Report Values
12/31/09 AD Notes
EOT.Edu 5.4.3 Education PY5 ALL TG Aug-09 Jul-10
NCSA Edee Wiziecki
[20%]; PSC Laura
McGinnin [10%], Robin
Flaus [10%], Phil Blood
[5%], Shawn Brown
[10%]; Purdue KayHunt
[50%], TBD [25%],
SDSC Ane Mason
[20%], Diane Baxter
EOT.EDU 5.4.3.2 M Education workshops May-10 Aug-10 90%
EOT.EDU 5.4.3.3 M monthly newsletter of EOT resources Aug-09 Mar-10 100% 75% 50%
EOT.EDU 5.4.3 P EOT Highlights 2010 Apr-10 Dec-10
EOT.EDU 5.4.3.1 P Request stories from EOT team Apr-10 Jul-10 Mason [20%] 100%
EOT.EDU 5.4.3.2 P Select and write stories for inclusion Jul-10 Oct-10 Mason, EOT team 50%
EOT.EDU 5.4.3.3 P Prepare layout and proof Sep-10 Dec-10
Mason, EOT team, 25%
Bruett Consulting
EOT.EDU 5.4.3.4 P Print Highlights Sep-10 Oct-10 Bruett Consulting
EOT.EDU 5.4.3.5 M PY5 - SC10 Education Program Jan-10 Nov-10 75%
EOT.Edu 5.4.4 Education Extension ALL TG Aug-10 Jul-11
NCSA Edee Wiziecki
[20%]; PSC Laura
McGinnin [10%], Robin
Flaus [10%], Phil Blood
[5%], Shawn Brown
[10%]; Purdue KayHunt
[50%], TBD [25%],
SDSC Ane Mason
[20%], Diane Baxter
[20%] Extension Year
EOT.EDU 5.4.4.1 M Education workshops May-11 Jul-11
EOT.EDU 5.4.4.2 M monthly newsletter of EOT resources Aug-10 Mar-11
EOT.EDU 5.4.4.3 P EOT Highlights 2011 Jan-11 Mar-11
EOT.EDU 5.4.4.3.1 P Request stories from EOT team Jan-11 Mar-11 Mason [20%]
EOT.EDU 5.4.4.8 M Extension - Education Program Jan-11 Jul-11
EOT.EDU.R
Aggregate RP Education & Outreach
5.4.7 O
P
Operations
Aug-09 Mar-10
EOT.EDU.R
100% 75%
5.4.7.1 O NCSA RP Education & Outreach Operations Aug-09 Mar-10
P.NCSA
50%
EOT.EDU.R
100% 75%
5.4.7.2 O IU RP Education & Outreach Operations Aug-09 Mar-10
P.IU
50%
EOT.EDU.R
100% 75%
5.4.7.3 O LONI RP Education & Outreach Operations Aug-09 Mar-10
P.LONI
50%
EOT.EDU.R
Ferguson [50%], TBD 100% 75%
5.4.7.4 O NICS RP Education & Outreach Operations Aug-09 Mar-10
P.NICS
[100%], TBD [50%]
50%
EOT.EDU.R
P.ORNL
5.4.7.5 O ORNL RP Education & Outreach Operations Aug-09 Mar-10
EOT.EDU.R
P.PSC
5.4.7.6 O PSC RP Education & Outreach Operations Aug-09 Mar-10
Budden[5%],
Flaus[15%],
Graham[15%],
Hanna[5%], Light[5%],
McGinnis[25%],
Nystrom[10%], Brown,
S[2%], Cubbison[10%],
Hackworth[15%],
Maiden[10%],
Domaracki[50%],
Czech[50%],
Ishwad[25%],
100% 75%
100% 75%
EOT.EDU.R
5.4.7.7 O PU RP Education & Outreach Operations Aug-09 Mar-10
P.PU
EOT.EDU.R
Baxter [37%], Mason 100% 75%
5.4.7.8 O SDSC RP Education & Outreach Operations Aug-09 Mar-10
P.SDSC
[20%]
EOT.EDU.R
Armosky [25%] 100% 75%
5.4.7.9 O TACC RP Education & Outreach Apr-09 Mar-10
P.TACC
EOT.EDU.R
100% 75%
5.4.7.10 O UCANL RP Education & Outreach Operations Apr-09 Mar-10
P.UCANL
EOT.RPOE 5.4.8 O Education &Outreach Aug-10 Mar-11
0.78 FTE for Extension
Year
EOT.RPOE.
5.4.8.1 O NCSA RP Education & Outreach Operations Aug-10 Mar-11
NCSA
EOT.RPOE.I
5.4.8.2 O IU RP Education & Outreach Operations Aug-10 Mar-11
U
EOT.RPOE.
5.4.8.3 O LONI RP Education & Outreach Operations Aug-10 Mar-11
LONI
EOT.RPOE.
Ferguson [50%], TBD
5.4.8.4 O NICS RP Education & Outreach Operations Aug-10 Mar-11
NICS
[100%], TBD [50%]
Budden[5%],
Flaus[15%],
Graham[15%],
Hanna[5%], Light[5%],
McGinnis[25%],
EOT.RPOE.
Nystrom[10%], Brown,
5.4.8.6 O PSC RP Education & Outreach Operations Aug-10 Mar-11
PSC
S[2%], Cubbison[10%],
Hackworth[15%],
Maiden[10%],
Domaracki[50%],
Czech[50%],
Ishwad[25%],
50%
50%
50%
50%
50%

Project-ID
WBS
O P M
Work Package Dependencies Planned
Description
Task
Cross
Area
Start
Date
End
Date
Resources
Name [%FTE]
QSR 2010Q2 Report Values
QSR 2010Q1 Report Values
6/30/10 AD Status 6/30/10 AD Notes 3/31/10 AD Status 3/31/10 AD Notes
12/31/09 AD
Status
QSR 2009Q4 Report Values
12/31/09 AD Notes
EOT.RPOE.
PU
5.4.8.7 O PU RP Education & Outreach Operations Aug-10 Mar-11
EOT.RPOE.
Baxter [37%], Mason
5.4.8.8 O SDSC RP Education & Outreach Operations Aug-10 Mar-11
SDSC
[20%]
EOT.RPOE.
Armosky [25%]
5.4.8.9 O TACC RP Education & Outreach Aug-10 Mar-11
TACC
EOT.RPOE.
UCANL
5.4.8.10 O UCANL RP Education & Outreach Operations Aug-10 Mar-11
EOT.OCCP 5.4.9 O Outreach: Campus Champions Program Aug-10 Mar-11 1.95 FTE
EOT 5.5 Objective 5.5 Projects: Evaluation
EOT.EVAL 5.5.2 PY5 Evaluation
To SI, RPs,
Abbott - external
Aug-09 Jul-10
NSF
evaluator on consulting
agreement
EOT.EVAL 5.5.2.1 O
All RPs collect data at all events throughout
all RPs 100% 75%
Aug-09 Jul-10
year and share with external evaluator
EOT.EVAL 5.5.2.2 M
Distribute surveys to people attending events
Abbott 100% 10%
Feb-10 Jul-10
held 6 months ago
EOT.EVAL 5.5.2.2 M Quartlery summary of data Dec-09 Jul-10 Abbott 75% 50%
EOT.EVAL 5.5.2.3 M Mid-year analysis of data Jan-10 Feb-10 Abbott Completed Completed
EOT.EVAL 5.5.2.4 M Full analysis report annually Jul-10 Jul-10 Abbott
EOT.EVAL.
5.5.3 PY5 - Evaluations offered by RPs Aug-09 Jul-10
RP
EOT.EVAL.R
100% 75%
5.5.3.1 O Collect data throughout year Aug-09 Jul-10
P.IU
EOT.ER 5.6 P Objective 5.5 External Relations
UC Elizabeth
Leake[100%]
EOT.ER 5.6.2 O
PY5 - working group meetings; info sharing
100% 75%
Aug-09 Jul-10
among RP ER staff
EOT.ER 5.6.4 O
PY5 - Quarterly Reporting (9/30, 12/31, 3/31,
100% 50%
Aug-09 Jul-10
6/30)
EOT.ER.RP 5.6.5 O Aggregate RP External Relations Operations Aug-09 Mar-10
EOT.ER.RP.
Barker [40%], Bell 100% 75%
5.6.5.1 O NCSA RP External Relations Operations Aug-09 Mar-10
NCSA
[25%]
EOT.ER.RP.
100% 75%
5.6.5.2 O IU RP External Relations Operations Apr-09 Mar-10
IU
EOT.ER.RP.
100% 75%
5.6.5.3 O LONI RP External Relations Operations Apr-09 Mar-10
LONI
EOT.ER.RP.
100% 75%
5.6.5.4 O NICS RP External Relations Operations Apr-09 Mar-10
NICS
EOT.ER.RP.
5.6.5.5
ORNL
O ORNL RP External Relations Operations Apr-09 Mar-10
EOT.ER.RP.
Schneider[50%], 100% 75%
5.6.5.6 O PSC RP External Relations Operations Aug-09 Mar-10
PSC
Williams[50%]
EOT.ER.RP.
100% 75%
5.6.5.7 O PU RP External Relations Operations Apr-09 Mar-10
PU
EOT.ER.RP.
Zverina [50%] 100% 75%
5.6.5.8 O SDSC RP External Relations Operations Aug-09 Mar-10
SDSC
EOT.ER.RP.
Singer [25%] 100% 75%
5.6.5.9 O TACC RP External Relations Apr-09 Mar-10
TACC
EOT.ER.RP.
100% 75%
5.6.5.10 O UC/ANL RP External Relations Operations Apr-09 Mar-10
UCANL
EOT.ER 5.6.7 M TG10
Communica
tions and
PR
Booth
planning
all RPs Aug-09 Jul-10
Leake [10%], ER team
members
90% 25%
EOT.ER 5.6.9 M SC10
ADs May-10 Dec-10
Coordinated by Leake 50%
[10%], ER team
Identify new venues for disseminating
Leake [10%] 100% 75%
EOT.ER 5.6.10 P
information about TeraGrid among science
Aug-09 Jul-10
and engineering communities to attract new
users to TeraGrid resources and services
EOT.ER 5.6.10.1 P
Work with ER team, AUS, ADs, and TG Forum
Leake 100% 75%
Aug-09 Jul-10
to identify venues
EOT.RPERC 5.6.5 O Aggregate RP External Relations Operations Aug-10 Mar-11 0.75 FTE for Extension
EOT.RPERC
Barker [40%], Bell
5.6.5.1 O NCSA RP External Relations Operations Aug-10 Mar-11
.NCSA
[25%]
EOT.RPERC
5.6.5.2
.IU
O IU RP External Relations Operations Aug-10 Mar-11
EOT.RPERC
5.6.5.3
.LONI
O LONI RP External Relations Operations Aug-10 Mar-11
EOT.RPERC
5.6.5.4
.NICS
O NICS RP External Relations Operations Aug-10 Mar-11
EOT.RPERC
5.6.5.5
.ORNL
O ORNL RP External Relations Operations Aug-10 Mar-11
EOT.RPERC
Schneider[50%],
5.6.5.6 O PSC RP External Relations Operations Aug-10 Mar-11
.PSC
Williams[50%]
EOT.RPERC
5.6.5.7
.PU
O PU RP External Relations Operations Aug-10 Mar-11
EOT.RPERC
Zverina [50%]
5.6.5.8 O SDSC RP External Relations Operations Aug-10 Mar-11
.SDSC
EOT.RPERC
Singer [25%]
5.6.5.9 O TACC RP External Relations Aug-10 Mar-11
.TACC
EOT.RPERC
5.6.5.10
.UCANL
O UC/ANL RP External Relations Operations Aug-10 Mar-11
EOT.ERCom
5.6.7 M
m
EOT.ERCom
5.6.9 M
m
Write stories about TeraGrid (non RP-specific)
resources and services. Customize existing
stories for NSF-OCI, OLPA, and other
purposes
Graphic Design assistance with the facilitation
and updating of TeraGrid’s web
content—specifically images and graphic files
Aug-10
Aug-10
Leake [10%], ER team
Mar-11
members, Grad
Student, 0.87 FTE for
Extension
Mar-11
50%
50%
50%
25%
50%
50%
50%
50%
50%
50%
50%
50%
50%
20%
50%
50%

Project-ID
WBS
O P M
Work Package Dependencies Planned
Description
Task
Cross
Area
Start
Date
End
Date
Resources
Name [%FTE]
QSR 2010Q2 Report Values
QSR 2010Q1 Report Values
6/30/10 AD Status 6/30/10 AD Notes 3/31/10 AD Status 3/31/10 AD Notes
12/31/09 AD
Status
QSR 2009Q4 Report Values
12/31/09 AD Notes
EOT.ERCom
5.6.10 P
m
EOT.ERCom
5.6.10.1 P
m
EOT.SW 5.7
Graduate assistant to help with research, list
maintenance, and ER XD transition
documentation
Internal relations activities. International
collaboration. Training in support of
international communication
Science Writing (stories written and
distributed)
Aug-10
Aug-10
Aug-09
EOT.SW 5.7.3 O Science Writing PY5 Aug-09 Jul-10
Mar-11
Mar-11
Jul-10
PSC Michael
Schneider[20%],
Shandra Williams[20%]
ER team; coordinated
by Leake
100% 75%
Review and select science stories to highlight,
100% 75%
EOT.SW 5.7.3.1 O
RPs, GIG Aug-09 Jul-10
50%
write, and disseminate throughout year
EOT.SW 5.7.3.2 O Select and write stories each quarter RPs, GIG Aug-09 Jul-10 ER team 100% 75% 50%
Pubish stories quarterly on web; disseminate
ER team 100% 75%
EOT.SW 5.7.3.3 O
GIG, UFP Aug-09 Jul-10
50%
through media - iSGTW, HPCWire, etc.
EOT.SW 5.7.4 O Science Writing Extension Aug-10 Jul-11
PSC Michael
Schneider[20%],
Shandra Williams[20%]
EOT.SW 5.7.4.1 O
Review and select science stories to highlight,
ER team; coordinated
RPs, GIG Aug-10 Jul-11
write, and disseminate throughout year
by Leake
EOT.SW 5.7.4.2 O Select and write stories each quarter RPs, GIG Aug-10 Jul-11 ER team
EOT.SW 5.7.4.3 O
Pubish stories quarterly on web; disseminate
ER team
GIG, UFP Aug-10 Jul-11
through media - iSGTW, HPCWire, etc.
EOT.SW 5.7.4.4 O XD Transition place holder GIG, UFP ER team
EOT.CSPW 5.8
Computational Science Problem of the
Aug-10 Jul-11
Week (EOT.CSPW)
EOT.CSPW 5.8.1 O
Computational Science Problem of the Week
(EOT.CSPW)
Aug-10 Jul-11
0.68 FTE
EOT END
50%