TGQR 2010Q4 Report.pdf - Teragridforum.org

NSF Extensible Terascale Facility
TeraGrid
Report for
Q4: October 1 st , 2010 through December 31 th , 2010
Ian Foster (GIG)
Phil Andrews
Jay Boisseau
John Cobb
Michael Levine
Rich Loft
Honggao Liu
Richard Moore
Carol Song
Rick Stevens
Craig Stewart
John Towns
Principal Investigators (GIG and RPs)
University of Chicago/Argonne National Laboratory (UC/ANL)
University of Tennessee (UT-NICS)
Texas Advanced Computing Center (TACC)
Oak Ridge National Laboratory (ORNL)
Pittsburgh Supercomputing Center (PSC)
National Center for Atmospheric Research (NCAR)
Louisiana Optical Network Initiative/Louisiana State University
(LONI/LSU)
San Diego Supercomputer Center (SDSC)
Purdue University (PU)
University of Chicago/Argonne National Laboratory (UC/ANL)
Indiana University (IU)
National Center for Supercomputing Applications (NCSA)
i

TeraGrid Forum (TGF)
John Towns (NCSA)
Matt Heinzel (UC/ANL)
Phil Andrews (UT-NICS)
Jay Boisseau (TACC)
John Cobb (ORNL)
Honggao Liu (LONI/LSU)
Michael Levine (PSC)
Rich Loft (NCAR)
Richard Moore (SDSC)
Mike Papka (UC-ANL)
Carol Song (PU)
Craig Stewart (IU)
Grid Infrastructure Group
Ian Foster (GIG) PI of TeraGrid GIG
Matt Heinzel (UC) Director of the TeraGrid GIG
Tim Cockerill (NCSA) Project Management Working Group
Kelly Gaither (TACC) Visualization
Christopher Jordan (TACC) Data Analysis
Daniel S. Katz (UC/ANL) GIG Director of Science
Scott Lathrop (UC/ANL) Education, Outreach and Training; External Relations
Elizabeth Leake (UC) External Relations
Lee Liming (UC/ANL) Software Integration and Scheduling
Amit Majumdar (SDSC) Advanced User Support
J.P. Navarro (UC/ANL) Software Integration and Scheduling
Mike Northrop (UC) GIG Project Manager
Jeff Koerner (UC/ANL) Networking, Operations and Security
Jeff Koerner (UC/ANL) User Facing Projects
Sergiu Sanielevici (PSC) User Services and Support
Nancy Wilkins-Diehr (SDSC) Science Gateways
Carolyn Peters (ANL) Event Coordinator/Administration
ii

Working Group Leaders
Accounting
Advanced User Support
Allocations
Common User Environment
Core Services
Data
Education, Outreach and Training
External Relations
Extreme Scalability
Networking
Operations
Project Management
Quality Assurance
Scheduling
Science Gateways
Security
Software
User Facing Projects
User Services
Visualization
David Hart (SDSC)
Amit Majumdar (SDSC)
Kent Milfield (TACC)
Shawn Brown (PSC)
Jeff Koerner (UC/ANL)
Christopher Jordan (TACC)
Scott Lathrop (UC/ANL)
Elizabeth Leake (UC)
Nick Nystrom (PSC)
Linda Winkler (ANL)
Jeff Koerner (UC/ANL)
Tim Cockerill (NCSA)
Kate Ericson (SDSC), Shava Smallen (SDSC)
Warren Smith (TACC)
Nancy Wilkins-Diehr (SDSC)
Jim Marsteller (PSC)
Lee Liming (UC/ANL), JP Navarro (UC/ANL)
Jeff Koerner (UC/ANL)
Sergiu Sanielevici (PSC)
Kelly Gaither (TACC), Mike Papka (UC/ANL)
iii

TERAGRID QUARTERLY REPORT
Contents
1 Overview ........................................................................................................................... 7
1.1 Scope .............................................................................................................................. 7
1.2 Communities Served ...................................................................................................... 8
1.3 TeraGrid’s Integrated, Distributed Environment ........................................................... 9
1.4 Organizational Architecture ........................................................................................... 9
2 Science and Engineering Highlights ............................................................................. 11
2.1 List of Science and Engineering Highlights ................................................................. 11
2.2 Science and Engineering Highlights ............................................................................ 12
3 Software Integration and Scheduling ........................................................................... 26
3.1 Highlights ..................................................................................................................... 26
3.2 Coordination and outreach ........................................................................................... 26
3.3 Capability Expansion and Maintenance ....................................................................... 26
3.4 Advanced Scheduling Capabilities .............................................................................. 27
3.5 Information Services Enhancements ............................................................................ 27
3.6 Operational Issues ........................................................................................................ 27
4 Science Gateways ............................................................................................................ 28
4.1 Highlights ..................................................................................................................... 28
4.2 Targeted Support .......................................................................................................... 29
4.3 Gateway Infrastructure and Services ........................................................................... 34
4.4 RP Operations: SGW Operations ................................................................................. 35
5 Users and User Support ................................................................................................. 35
5.1 Highlights ..................................................................................................................... 35
5.2 User Engagement ......................................................................................................... 36
5.3 Frontline User Support ................................................................................................. 37
5.4 RP Operations: User Services ...................................................................................... 40
5.5 Advanced User Support ............................................................................................... 41
6 User Facing Projects and Core Services ....................................................................... 64
6.1 Highlights ..................................................................................................................... 64
6.2 Enhanced TeraGrid User Access ................................................................................. 64
6.3 Allocations Process and Allocation Management ........................................................ 66
6.4 RP Integration and Information Sharing ...................................................................... 66
iv

6.5 RP Operations: Accounting/Core Services .................................................................. 67
6.6 User Information Presentation ..................................................................................... 67
6.7 RP Operations: User Facing Projects ........................................................................... 68
6.8 Information Production and Quality Assurance ........................................................... 68
7 Data and Visualization ................................................................................................... 72
7.1 Highlights ..................................................................................................................... 72
7.2 Data .............................................................................................................................. 73
7.3 RP Operations: Data..................................................................................................... 74
7.4 Visualization ................................................................................................................ 75
7.5 RP Operations: Visualization ....................................................................................... 76
8 Network Operations and Security ................................................................................ 78
8.1 Highlights ..................................................................................................................... 78
8.2 Networking .................................................................................................................. 79
8.3 RP Operations: Networking ......................................................................................... 79
8.4 INCA ............................................................................................................................ 80
8.5 Grid Services Usage ..................................................................................................... 80
8.6 RP Operations: HPC Operations .................................................................................. 81
8.7 Security ........................................................................................................................ 83
8.8 RP Operations: Security ............................................................................................... 84
8.9 Common User Environment ........................................................................................ 86
8.10 System Performance Metrics ....................................................................................... 86
9 Evaluation, Monitoring, and Auditing ......................................................................... 92
9.1 XD – TIS ...................................................................................................................... 92
9.2 Quality Assurance ........................................................................................................ 96
9.3 XD – TAS .................................................................................................................... 97
10 Education, Outreach, and Training; and External Relations .................................. 114
10.1 Joint EOT-ER Outreach Highlights ........................................................................... 114
10.2 EOT Impact Stories .................................................................................................... 121
10.3 EOT Highlights .......................................................................................................... 123
10.4 Training, Education and Outreach ............................................................................. 124
10.5 RP Operations: Training ............................................................................................ 125
10.6 RP Operations: Education .......................................................................................... 126
10.7 RP Operations: Outreach ........................................................................................... 128
10.8 External Relations ...................................................................................................... 130
10.9 Enhancement of Diversity .......................................................................................... 133
v

10.10 International Collaborations ....................................................................................... 133
10.11 Broader Impacts ......................................................................................................... 135
11 Project Management .................................................................................................... 136
11.1 Highlights ................................................................................................................... 136
11.2 Project Management Working Group ........................................................................ 136
12 Science Advisory Board Interactions.......................................................................... 136
13 Collaborations with Non-TeraGrid Institutions ........................................................ 136
13.1 Data Systems .............................................................................................................. 136
13.2 Science Projects ......................................................................................................... 136
14 TeraGrid/Open Science Grid (OSG) Joint Activity .................................................. 137
14.1 ExTENCI ................................................................................................................... 137
vi

1 Overview
The TeraGrid is an open cyberinfrastructure that enables and supports leading-edge scientific
discovery and promotes science and technology education. The TeraGrid comprises
supercomputing and massive storage systems, visualization resources, data collections, and
science gateways, connected by high-bandwidth networks integrated by coordinated policies and
operations, and supported by computational science and technology experts,.
TeraGrid’s objectives are accomplished via a three-pronged strategy: to support the most
advanced computational science in multiple domains (deep impact), to empower new
communities of users (wide impact), and to provide resources and services that can be extended
to a broader cyberinfrastructure (open infrastructure). This “deep, wide, and open” strategy
guides the development, deployment, operations, and support activities to ensure maximum
impact on science research and education across communities.
1.1 Scope
TeraGrid is an integrated, national-scale computational science infrastructure operated in a
partnership comprising the Grid Infrastructure Group (GIG), eleven Resource Provider (RP)
institutions, and six Software Integration partners, with funding from the National Science
Foundation’s (NSF) Office of Cyberinfrastructure (OCI). Initially created as the Distributed
Terascale Facility through a Major Research Equipment (MRE) award in 2001, the TeraGrid
began providing production computing, storage, and visualization services to the national
community in October 2004. In August 2005, NSF funded a five-year program to operate,
enhance, and expand the capacity and capabilities of the TeraGrid to meet the growing needs of
the science and engineering community through 2010, and then extended the TeraGrid an
additional year into 2011 to provide an extended planning phase in preparation for TeraGrid
Phase III eXtreme Digital (XD).
Accomplishing this vision is crucial for the advancement of many areas of scientific discovery,
ensuring US scientific leadership, and increasingly, for addressing important societal issues.
TeraGrid achieves its purpose and fulfills its mission through a three-pronged strategy:
Deep: ensure profound impact for the most experienced users, through provision of the
most powerful computational resources and advanced computational expertise;
Wide: enable scientific discovery by broader and more diverse communities of
researchers and educators who can leverage TeraGrid’s high-end resources, portals and
science gateways; and
Open: facilitate simple integration with the broader cyberinfrastructure through the use
of open interfaces, partnerships with other grids, and collaborations with other science
research groups delivering and supporting open cyberinfrastructure facilities.
The TeraGrid’s deep goal is to enable transformational scientific discovery through leadership
in HPC for high-end computational research. The TeraGrid is designed to enable high-end
science utilizing powerful supercomputing systems and high-end resources for the data analysis,
visualization, management, storage, and transfer capabilities required by large-scale simulation
and analysis. All of this requires an increasingly diverse set of leadership-class resources and
services, and deep intellectual expertise in the application of advanced computing technologies.
The TeraGrid’s wide goal is to increase the overall impact of TeraGrid’s advanced
computational resources to larger and more diverse research and education communities
through user interfaces and portals, domain specific gateways, and enhanced support that
7

facilitate scientific discovery by people without requiring them to become high performance
computing experts. The complexity of using TeraGrid’s high-end resources will continue to grow
as systems increase in scale and evolve with new technologies. TeraGrid broadens the scientific
user base of its resources via the development and support of simpler but powerful interfaces to
resources, ranging from establishing common user environments to developing and hosting
Science Gateways and portals. TeraGrid also provides focused outreach and collaboration with
science domain research groups, and conducts educational and outreach activities that help
inspire and educate the next generation of America’s leading-edge scientists.
TeraGrid’s open goal is twofold: to enable the extensibility and evolution of the TeraGrid by
using open standards and interfaces; and to ensure that the TeraGrid is interoperable with
other, open standards-based cyberinfrastructure facilities. While TeraGrid only provides
(integrated) high-end resources, it must enable its high-end cyberinfrastructure to be more
accessible from, and even federated or integrated with, cyberinfrastructure of all scales. That
includes not just other grids, but also campus cyberinfrastructures and even individual researcher
labs/systems. The TeraGrid leads the community forward by providing an open infrastructure that
enables, simplifies, and even encourages scaling out to its leadership-class resources by
establishing models in which computational resources can be integrated both for current and new
modalities of science. This openness includes open standards and interfaces, but goes further to
include appropriate policies, support, training, and community building.
The TeraGrid’s integrated resource portfolio includes 19 high-performance computational (HPC)
systems, several massive storage systems, remote visualization resources, and a dedicated
interconnection network. This infrastructure is integrated at several levels: policy and planning,
operational and user support, and software and services. Policy and planning integration
facilitates coordinated management and evolution of the TeraGrid environment and allows us to
present—as much as possible—a single cyberinfrastructure to the user community. Operational
and user support integration allows the user community to interact with one or more of the many
distinct resources and HPC centers that comprise TeraGrid, through a common service, training,
and support organization—masking the complexity of a distributed organization. In addition, this
allows the national user community to request allocations through a single national review
process and use the resources of the distributed facility with a single allocation. Software and
services integration creates a user environment with standard service interfaces, lowering barriers
to porting applications, enabling users to readily exploit the many TeraGrid resources to optimize
their workload, and catalyzing a new generation of scientific discovery through distributed
computing modalities. Science Gateways, web-based portals that present TeraGrid to specific
communities, represent a new paradigm in high-performance computing with the potential to
impact a wide audience. Gateways benefit significantly from the integration of software and
services across the TeraGrid.
1.2 Communities Served
The national, and global, user community that relies on TeraGrid has grown tremendously during
the past four years, from fewer than 1,000 users in October 2005 to nearly 5,000 active users and
more than 10,000 total lifetime users at the end of 2009.
The fourth quarter has typically seen a slight dip in TeraGrid user metrics, and Q4 2010 was no
exception. 515 new individuals joined the ranks of TeraGrid users, and in October, 1,373
individuals charged jobs on TeraGrid resources, a new monthly high. TeraGrid users numbered
6,505 at the end of the quarter, including 5,563 individuals with current TeraGrid accounts. Of
the 5,563 individuals, 1,904 (34%) were associated with jobs recorded in the TeraGrid central
8

accounting system; an additional 942 users submitted jobs via science gateways—that is, nearly
1/3 of TeraGrid users submitting jobs worked through gateways.
Further details can be found §6.2.1.
To support the great diversity of research activities and their wide range in resources needs, our
user support and operations teams leverage the expertise across the eleven TeraGrid Resource
Providers (§5, §8). In addition, users benefit from our coordinated education, outreach, and
training activities (§9).
1.3 TeraGrid’s Integrated, Distributed Environment
TeraGrid’s diverse set of HPC resources provides a rich computational science environment.
TeraGrid RPs operate more than 25 highly reliable HPC resources and 10 storage resources that
are available via a central allocations and accounting process for the national academic
community. More detailed information is available on compute resources at
https://www.teragrid.org/web/user-support/resources and on storage resources at
https://www.teragrid.org/web/user-support/data_and_viz.
TeraGrid saw a slight dip in delivered NUs for Q4 2010. TeraGrid compute resources delivered
13.2 billion NUs to users, down 0.2 billion, or 1%, from Q3 (Figure 8-11). However, the
13.2 billion NUs represent a 1.7x increase in the NUs delivered in Q4 2009. Nearly 100% of
those NUs were delivered to allocated users. Further details can be found §8.11.
1.4 Organizational Architecture
The coordination and management
of the TeraGrid partners and
resources requires organizational
and collaboration mechanisms that
are different from a classic
organizational structure for single
organizations. The existing
structure and practice has evolved
from many years of collaborative
arrangements between the centers,
some predating the TeraGrid. As
the TeraGrid moves forward, the
inter-relationships continue to
evolve in the context of a
persistent collaborative
environment.
Figure 1-1: TeraGrid Facility Partner Institutions
The TeraGrid team (Figure 1-1) is composed of eleven RPs and the GIG, which in turn has
subawards to the RPs plus six additional Software Integration partners. The GIG provides
coordination, operations, software integration, management and planning. GIG area directors
(ADs) direct project activities involving staff from multiple partner sites, coordinating and
maintaining TeraGrid central services.
TeraGrid policy and governance rests with the TeraGrid Forum (TG Forum), comprising the
eleven RP principal investigators and the GIG principal investigator. The TG Forum is led by an
9

elected Chairperson, currently John Towns. This position facilitates the functioning of the TG
Forum on behalf of the overall collaboration.
TeraGrid management and planning is coordinated via a series of regular meetings, including
weekly TeraGrid AD meetings, biweekly project-wide Round Table meetings, biweekly TG
Forum meetings, and quarterly face-to-face internal project meetings. Coordination of project
staff in terms of detailed technical analysis and planning is done through two types of technical
groups: working groups and Requirement Analysis Teams (RATs). Working groups are persistent
coordination teams and in general have participants from all RP sites; RATs are short-term (6-10
weeks) focused planning teams that are typically small, with experts from a subset of both RP
sites and GIG partner sites. Both types of groups make recommendations to the TeraGrid Forum
or, as appropriate, to the GIG management team.
10

2 Science and Engineering Highlights
The TeraGrid publishes science impact stories to inform and educate the public about the
importance and impact of high-end computing technologies in advancing scientific research.
Below we provide 19 examples of TeraGrid-enabled science: first in a summary list, then as short
summaries of each project. These highlights have been selected from the much larger collection
of important scientific research activity supported on the TeraGrid to demonstrate the breadth of
scope across scientific disciplines, research institutions, and sizes of research groups.
2.1 List of Science and Engineering Highlights
2.1.1
2.1.2
2.1.3
2.1.4
2.1.5
2.1.6
2.1.7
2.1.8
2.1.9
2.1.10
2.1.11
2.1.12
2.1.13
2.1.14
2.1.15
2.1.16
2.1.17
Astronomical Sciences: Detecting HI In Emission At z = 1: The Effect of Substructure
(PI: Tiziana Di Matteo, Carnegie Mellon)
Astrophysics: The Formation of Active Regions on the Sun (PI: Matthias Rempel, NCAR)
Atmospheric Sciences: Rapid Intensification of Hurricanes Related to Vertical Velocity
Distributions and Microphysical Processes using WRF simulations in the context of
observations (PI: Greg McFarquhar, U. Illinois)
Atomic, Molecular, and Optical Physics: Optical Absorption Enhancement in Silicon
Solar Cells Using Photonic Crystals (PI: Sang Eon Han, MIT)
Biology: Water under the BAR (Gregory A. Voth, U. Chicago)
Biology & Environment Science: North American bird distribution map input for 2011
State of the Birds (PIs: Steve Kelling, John Cobb, Dan Fink, Cornell and ORNL)
Cell Biology: Complex Formation and Binding Affinity of NHERF1 to C-terminal
Peptides (PI: Peter A. Friedman, U. Pittsburgh School of Medicine)
Chemistry: Molecular Simulation of Adsorption Sites in Metal-Organic Frameworks (PI:
Randall Snurr, Northwestern)
Computer and Computation Research: eaviv—Distributed visualization (PI: Andrei
Hutanu, Louisiana State U.)
Computational Mathematics: Residual Based Turbulence Models for Large Eddy
Simulation on Unstructured Tetrahedral Meshes (PI: Arif Masud, U. Illinois)
Earth Sciences: A TeraGrid Community Climate System Modeling (CCSM) Portal (PI:
Matthew Huber, Purdue)
Fluid, Particulate, and Hydraulic Systems: Fluid and Solid Interaction (FSI) in
Engineering, Biomedical and Other Applications (PI: Cyrus Aidun, GeorgiaTech)
Fluid, Particulate, and Hydraulic Systems: Large Eddy Simulation Investigations of
Aviation Environmental Impacts (PI: Sanjiva Lele, Stanford)
Human Computer Interactions: A Window on the Archives of the Future (PIs: Robert
Chadduck, National Archives and Record Administration; Maria Esteva, and Weijia Xu,
TACC)
Molecular and Cellular Biosciences: Computational Approaches to Understanding Ion
Channel and Transporter Function (PI: Michael Grabe, U. Pittsburgh)
Phylogenetic Research: The CIPRES Science Gateway (PI: Mark Miller, UCSD)
Physical Chemistry: Gas Phase Chemistry of Molecules and Ions of Astrophysical
Interest (PI: Zhibo Yang, U. Colorado)
11

2.1.18
2.1.19
Physics: The Geophysical High Order Suite for Turbulence (GHOST) (PI: Annick
Pouquet, NCAR)
Student Training: Use of TeraGrid Systems in the Classroom and Research Lab (PI:
Andrew Ruether, Swarthmore College)
2.2 Science and Engineering Highlights
2.2.1
Astronomical Sciences: Detecting HI In Emission At z = 1: The Effect of Substructure
(PI: Tiziana Di Matteo, Carnegie Mellon)
Observations show that the cosmic star formation rate has declined by more than one order of
magnitude since redshift z = 1 (about 7.7 billion years ago). However a combined census of the
cold gas (the fuel for star formation), and stellar components is largely missing in observations.
This cold gas fraction is a crucial ingredient in models of galaxy formation and will suggest how
galaxies obtain gas and subsequently convert it to stars. A census of neutral hydrogen (HI) will
put tight constraints on different models of galaxy formation.
Based on current observations, Di Matteo’s group modeled the distribution of HI using dark
matter simulations with 14.6 billion particles at z = 1, using the hybrid MPI/Pthreads and
massively parallel code GADGET on ~22k cores of TeraGrid’s Kraken system at NICS. This
simulation is around 1.5 times larger than the Millennium Simulation (Springel et al. 2005) with 3
times finer mass resolution. This was made possible by their NSF PetaApps project with PSC and
the University of Washington, which has scaled GADGET to ~100k cores on Kraken.
A slice from Di
Matteo’s group’s
simulation
illustrating the
rich network of
nodes, filaments
and voids in the
distribution of
dark matter at z
= 1 is shown in
the left side of
the figure. The
high resolution
and large
volume allow
the group to
resolve the
smallest
subhalos that host HI, residing in the largest dark matter halo. The group makes predictions for
existing surveys and radio interferometers to determine the HI mass function at this redshift. They
predict that small galaxies (undetected in an optical survey) embedded in their parent halo can be
detected through their contribution to the total HI signal when seen in a radio telescope. This is
shown for one of the many large halos in their simulation in the right side of the figure.
Figure 2.1. (Left) A slice from the dark matter simulation at z = 1 enclosing the largest halo. (Right)
Contribution of subhalos to the HI signal. The measured signal (red line) of a single large halo and the
modeled signal (black line). Data points are the contribution of subhalos within pixel, the large halo
giving rise to a discrepancy between the theoretical and the observed signal.
Even though individual halos cannot be detected in HI at z = 1, the group can detect the effect of
substructures by stacking thousands of halos whose redshifts have been determined in an optical
survey. They find that the contribution of substructure can be seen in the cumulative HI signal
12

and that they will be able to determine the HI mass function with existing surveys and radio
telescopes with a detection significance of 5 -10σ across models.
2.2.2
Astrophysics: The Formation of Active Regions on the Sun (PI: Matthias Rempel, NCAR)
Rempel and his team research the formation of active regions of
the sun, which are areas with an especially strong magnetic field.
Their recent work has concentrated on two closely related aspects
of solar active region formation: 1) the sensitivity of flux
emergence processes on initial conditions and 2) the dependence
of penumbral fine structure on the imposed top boundary
condition.
Solar flux emergence processes include the transport of a
magnetic field toward the stellar surface, the formation of
starspots (i.e., sunspots), and the expansion of the magnetic field
into the stellar corona. Using a new boundary condition
artificially enhancing the horizontal magnetic field at the top
boundary of the computational domain, the team was able to
simulate sunspots with extended penumbrae and strong Evershed
effect (outflow of gas from sunspots).
Using Kraken at NICS, the team investigated how active region
formation and evolution depend on the degree of magnetic twist
present. In order to investigate the flux emergence, the team ran 150,000 iterations using 1,536
CPUs, leading to 24 hours of simulated solar time. In order to investigate penumbral fine
structure, the team ran 60,000 iterations using 1,536 CPUs to generate 2.5 hours of solar time.
The highest resolution run was restarted from a previous run and evolved an additional 40
minutes of solar time requiring 4,608 CPUs for 20,000 iterations. The increase of CPUs by a
factor of four leads to a threefold decrease in time-to-solution and helped identify parts of the
code requiring further improvement. The team used information from these runs on Kraken for a
long-duration simulation in which the focus was wave propagation through sunspots to test and
improve helioseismic inversion methods. This simulation was started on Ranger (at TACC),
while the last 25% was finished on Kraken.
The new boundary condition allowed the team to investigate the robustness of penumbral fine
structure and of the underlying magneto-convection mechanism with respect to the numerical
resolution. Identifying the minimum resolution required for properly resolving the physics
underlying large scale outflows around sunspots is essential for future simulation runs since it
allows to optimize the use of the required computing resources.
2.2.3
Figure 2.2. Numerical simulation explains
sunspot fine structure as a consequence of
magneto-convective energy transport in
umbra and penumbra.
Atmospheric Sciences: Rapid Intensification of Hurricanes Related to Vertical Velocity
Distributions and Microphysical Processes using WRF simulations in the context of
observations (PI: Greg McFarquhar, U. Illinois)
It is well known that transformations between phases of water—vapor, liquid, ice—result in
latent heating that ultimately affects tropical cyclone intensity. A quantitative understanding of
the distribution of cloud microphysical processes releasing this energy and relating to the
development of updrafts and downdrafts, however, has not been developed, which challenges
forecasters.
Greg McFarquhar and his colleagues at the University of Illinois at Urbana-Champaign
configured the Weather Research and Forecasting model with one kilometer grid spacing and
two-minute output frequency to simulate Hurricane Dennis (2005), which was observed during
the NASA Tropical Cloud Systems and Processes campaign. NCSA’s Abe provided the
13

computational resources for this simulation, which spanned from
Dennis’s tropical storm through mature hurricane stage and
second landfall over Cuba. In particular, 256 processors were
used for integration of the model and an additional 16 processors
for writing the standard and added latent heat output. It took about
100 hours of real time to complete the 66-hour simulation. NCSA
project space has since proven particularly useful for analyzing
the approximately 12 terabytes of data post-processed from this
simulation.
To identify potential precursors to the rapid intensification of
Dennis, statistical distributions of simulated vertical velocity and
latent heat within various horizontal, vertical, and temporal
frameworks were constructed. The team found that during the
approximately 24-hour period preceding the onset of the most
rapid intensification, upper-tropospheric vertical velocity
frequency distributions continually broadened while the outliers
of these distributions converged toward the simulated tropical
cyclone center. Although a clear increase and concentration of
latent heating toward the tropical cyclone center occurred, it was
only after the onset of and during this most rapid intensification,
when lower-to-mid-level vertical velocity frequency distributions
broadened and more effectively forced the phase changes of water.
The team presented their work at many conferences, including the 29th Conference on Hurricanes
and Tropical Meteorology in 2010.
2.2.4 Atomic, Molecular, and
Optical Physics: Optical
Absorption Enhancement in
Silicon Solar Cells Using
Photonic Crystals (PI: Sang
Eon Han, MIT)
The cost of crystalline silicon solar
cells could be reduced significantly if
the thickness of the silicon layer
could be decreased from a few
hundred micrometers to just a few
micrometers. The material cost of
crystalline silicon takes up about half
the module cost of solar cells; hence
thinner silicon would reduce costs.
However, the poor light absorption of
native crystalline silicon is a major
challenge. Thus, an effective
technique for light trapping in thin
active layers needs to be developed.
The scattering from metal
nanoparticles near localized plasmon
Figure 2.3. Hurricane Dennis bears down
on Cuba in 2005 in this image from
NASA. Researchers simulated this phase
of the hurricane as part of their study of the
cloud microphysical processes that caused
the storm to later intensify as part of a
project on forecasting hurricane intensity.
Figure 2.4. The image shows that the amount of silicon can be reduced by
two orders of magnitude by using photonic crystals to achieve the same
absorption as a flat silicon substrate does. Optical interference concentrates
light in silicon and this achieves absorption that exceeds the conventional
light trapping limit at long wavelengths.
resonance—oscillations that exist at the interface between any two materials—is a promising way
of increasing the light absorption in thin-film solar cells.
14

Enhancements in photocurrent have been observed for a wide range of semiconductors and solar
cell configurations. Photonic crystals, optical nanostructures designed to act on photons in the
same way semiconductors act on electrons, can be used to enhance optical absorption in this way.
Photonic crystals are structures periodic on an optical length scale, meaning they contain
regularly repeating internal regions that allow light waves to propagate or not depending on
wavelengths. This periodicity can modify the density of photon states and absorption can be
altered. Han is investigating silicon photonic crystals as light-absorbing structures for solar
photovoltaics via simulations using the high-throughput Condor pool TeraGrid resource at
Purdue.
Charge carriers (electrons and holes) recombine at the crystal defects. If a thinner layer is used, it
results in better charge carrier collection because the area of defects is less. In photonic crystals,
optical diffraction can happen, which can be understood as an optical interference phenomenon.
Constructive interference increases the electromagnetic field intensity and this can lead to
stronger light absorption. If constructive interference in the photonic crystal regions which silicon
occupies can be induced, enhanced absorption is expected. Accordingly, thin films might be used
both to reduce the cost of solar cells and increase their efficiency. Results from this research
using TeraGrid resources have recently been published in Nano Letters, “Toward the Lambertian
Limit of Light Trapping in Thin Nanostructured Silicon Solar Cells,” Nano Lett., 2010, 10 (11),
pp 4692–4696.
2.2.5
Biology: Water under the BAR (Gregory A. Voth, U. Chicago)
Many cellular processes require the
generation of highly curved regions of
cell membranes by interfacial
membrane proteins. Several
mechanisms of curvature generation
have been suggested, but a quantitative
understanding of the importance of
various potential mechanisms remains
elusive.
Voth’s research in this area is
concentrated on the electrostatic
attraction underlying the scaffold
mechanism of membrane bending in
the context of the N-BAR domain of
the protein amphiphysin. Analysis of
atomistic molecular dynamics
simulations reveals considerable water between the membrane and the positively charged
concave face of the BAR, even when it is as tightly bound as physically possible.
Figure 2.5. (Upper-left) Top view of the starting configuration ofthe sixamphiphysin
N-BAR SBCG simulation (counterions omitted for clarity).
(Upper-right) Side view of the starting configuration of the SBCG
simulations, showing counterions. (Lower-left) Final configu- ration of the
SBCG simulation with the relative dielectric set to 1. (Lower- right) Final
configuration of the SBCG simulation with the relative dielectric set to 14.
This results in significant screening (reduction in strength) of electrostatic interactions, suggesting
that electrostatic attraction is not the main driving force behind curvature sensing. These results
also emphasize the need for care when building coarse-grained models of protein-membrane
interactions. In the coarse-grained simulations, Voth finds that the induced curvature depends
strongly on the dielectric screening.
Calculations were performed on BigBen at PSC, Ranger at TACC, and Kraken at NICS. Phil
Blood and Richard Swenson provided assistance with the simulations. This work has been
published in Biophysical Journal v.99, 1783-1790, Sept. 2010.
15

2.2.6
Biology & Environment Science: North American bird distribution map input for 2011
State of the Birds (PIs: Steve Kelling, John Cobb, Dan Fink, Cornell and ORNL)
The Cornell lab of ornithology’s eBird
(http://www.ebird.org) project has collected
620,000 bird observations reported from
107,000 unique locations. Collaborating with
the scientific exploration, visualization, and
analysis working group of the DataONE
DataNet project, the team has developed a new,
improved, statistical analysis method to predict
the presence of absence of bird species
according to environmental covariates. During
2010, the team collected the covariants into a
data warehouse and with a TeraGrid startup allocation (used mostly on Lonestar at TACC) were
able to calculate bird migration maps with unprecedented detail and accuracy. An example
showing the presence of the Wood Thrush during the Spring is given in the figure. Additional
occurrence maps from these calculations are available from eBird
(http://ebird.org/content/ebird/about/occurrence-maps/occurrence-maps). Maps of this detail are
fine enough to be used in making land use and other policy decisions that affect habitat for
important bird species, including decisions about crucial time periods at specific locations
important for migratory species. Data and occurrence maps from these computations will be used
in the annual state of the birds report, a high level policy document used to set multi-agency and
NGO research and conservation policy agendas. (http://www.stateofthebirds.org/). The scale of
computation required for this analysis is much larger than every before attempted by the eBird
team. Without a TeraGrid resource such as Lonestar, it would have been much more difficult.
Expert assistance at TeraGrid sites at Cornell, ORNL, and TACC were critical to the success of
the computational campaign. In particular the data intensive mature of the calculation requires
many large memory nodes and considerable I/O. Future plans include conducting analyses of six
years of data instead of a single year with a future TeraGrid allocation in order to understand
annual variability of migration patterns for several important reasons including as a “canary in the
coal-mine” indicator of climate change.
2.2.7
Cell Biology: Complex Formation and Binding Affinity of NHERF1 to C-terminal
Peptides (PI: Peter A. Friedman, U. Pittsburgh School of Medicine)
The Na/H Exchange Regulatory Factor-1 (NHERF1) is a protein that is
essential for intracellular signaling and trafficking. Recently, three NHERF1
mutations were found in patients who exhibited renal phosphate wasting
and bone thinning (osteopenia). Two of these mutations (R153Q, E225K)
are located at the NHERF1 subdomain PDZ2. NMR and unfolding studies
show that these mutations decrease the thermodynamic and conformational
stability of the protein. Friedman, et al. performed molecular dynamics and
thermodynamics simulations with AMBER on 32 PEs on Pople at PSC.
These simulations made it possible to determine the different interactions of
the wild type and mutated NHERF1 proteins. They show that the
parathyroid hormone 1 receptor (PTH1R) has higher affinity for the mutant
(R153Q) PDZ2 than for the wild type PDZ2, which may explain the
mechanisms of renal disease caused by the mutation.
Figure 2.6. Migration of Wood Thrush Occurrence 2009.
Corrected for variation in detectability. Detailed predictions
reveal both broad-scale patterns and fine-resolution detail.
Figure 2.7. A snapshot from
the MD simulation of bound
PDZ2.
16

2.2.8
Chemistry: Molecular Simulation of Adsorption Sites in Metal-Organic Frameworks (PI:
Randall Snurr, Northwestern)
Metal-organic frameworks (MOFs) are a new class of nanoporous materials that could be
designed for a variety of purposes. For example, nanoscale pores tailored to attract a crowd of
hydrogen gas molecules, which normally like plenty of elbow room, might be one way to pack
more fuel into the tank of a hydrogen-driven vehicle or a fuel cell-powered mobile phone. Snurr
likens nanoporous materials in this case to a sponge, one self-assembled Tinker Toy fashion by
miniscule metal linker pieces and organic struts that join to form a 3-D structure laced with tiny
channels. Snurr and colleagues simulate adsorption sites in MOFs to predict how the materials
will perform.
A focus in 2010 was trying to develop or screen porous
materials for capturing CO2 from the exhaust gases of
coal-fired power plants. Carbon capture and sequestration
are widely viewed as a possible way to reduce CO2
emissions. The idea is to separate the carbon dioxide out
of flue gas from a coal-fired plant and then store the CO2
far underground in geologic formations. Snurr’s group is
modeling porous materials for their ability to separate
CO2 from the other gases in the flue gas stream. They
have developed a consistent molecular modeling method
and validated it against experimental data for 14
representative MOFs. With this validation, they are
screening large numbers of existing and hypothetical
MOFs.
In addition, the materials might be designed for potential
applications in catalysis, chemical separation and to
reduce auto emissions, among other things. Such
materials could lower the energy demands—and cost—of
energy-intensive processes, such as distillation, making
them “greener.” The materials could be useful in chiral
chemistry applications as well, for instance separating
active and inactive forms of molecules for use in drugs.
The researchers employ Monte Carlo simulations to
understand the materials and their properties, such as the
amount of molecules they adsorb, at an atomic level.
Snurr and colleagues likewise can predict the position and
orientation of adsorption sites. Their predictions match
Figure 2.8. The structure of a metal-organic
framework (MOF) with potential for use in
carbon dioxide capture systems superimposed
behind a silhouette of a coal-fired power plant.
Specially designed MOFs might be used to
separate CO2 from flue gas for storage deep
underground, keeping the CO2 out of the
atmosphere.
well with published results from experimental results using X-ray diffraction. The Monte Carlo
technique employed is particularly useful for evaluating a model with a large number of inputs
and more than a few uncertain parameters over and over again to yield an aggregate result, a
description that fits, say, an atomic-level representation of a metal-organic framework and its
guest molecules. The work uses computing resources at several TeraGrid sites, including Purdue,
NCSA and TACC. Recent papers covering the research include “Screening of Metal-Organic
Frameworks for Carbon Dioxide Capture from Flue Gas Using a Combined Experimental and
Modeling Approach," J. Am. Chem. Soc. 131, 18198-18199 (2009).
17

2.2.9
Computer and Computation Research: eaviv—Distributed visualization (PI: Andrei
Hutanu, Louisiana State U.)
Scientific research is increasingly dependent on simulation and analysis requiring high
performance computers, distributed large-scale data, and high-speed
networks. Andrei Hutanu of LSU leads a team addressing fundamental
issues in distributed visualization design and implementation, where
network services represent a first-class resource. They built a distributed
visualization system, eaviv, and a distributed visualization application
optimized for large data, high-speed networks and interaction. eaviv
connects Spider at LSU’s Center for Computation & Technology
(CCT), Lincoln and accelerator clusters at NCSA, and a cluster at the
Laboratory of Advanced Networking Technologies (SITOLA) at
Masaryk University in the Czech Republic. The Internet2 interoperable
On-demand Network (ION) provides wide-area connectivity to support
dynamic network circuit services, and NCSA’s mass storage system
provided permanent data storage. Applications request and reserve
point-to-point circuits between sites as needed using automated control
software. eaviv supports distributed collaboration, letting multiple users
in physically distributed locations interact with the same visualization to
communicate their ideas and explore the datasets cooperatively. The team implemented a raycasting
parallel volume renderer called Pcaster as the rendering component to demonstrate the
distributed pipeline’s workflow. Compared to parallel volume renderers in existing software,
Pcaster is a purely GPU-based volume renderer and image compositor supporting high-resolution
rendering. Pcaster asynchronously couples with parallel data servers for network-streamed data
input. The team tested Pcaster with data sets up to 64 gigabytes per timestep and achieved
interactive frame rates of five to 10 frames per second on Lincoln to produce render images of
1,024 x 1,024 resolution. The eaviv system’s configurable architecture also allows it to run with a
local data server and a single renderer, making it a desktop tool for small-scale local data.
This research was published in 2010 in Scalable Computing: Practice and Experience and
Computing in Science and Engineering, and was presented at the TeraGrid 10 conference.
2.2.10
Computational Mathematics: Residual Based Turbulence Models for Large Eddy
Simulation on Unstructured Tetrahedral Meshes (PI: Arif Masud, U. Illinois)
Development of computational methods for modeling turbulent
flows is considered a formidable challenge due to the plethora of
associated spatial and temporal scales. Professor Arif Masud and
graduate student Ramon Calderer from the Department of Civil
and Environmental Engineering at the University of Illinois at
Urbana-Champaign used Abe at NCSA to develop residual-based
methods for modeling turbulence in complex fluid flows and
fluid-structure interactions. Their new codes are mathematically
consistent and robust and provide high-fidelity solutions at
reduced computational costs compared to the more traditional
Direct Numerical Simulations (DNS).
In an effort to model flow-induced vibrations in off-shore oil
platforms, the team used their new method to investigate fluidstructure
interaction and turbulence around rigid and oscillating
cylinders. Underwater currents trigger periodic vortices around
risers that are oil pipelines extending from the floating platforms
Figure 2.9. eaviv is a
visualization application with
three distributed components
connected by high-speed
networks: data, rendering, and
viewer.
Figure 2.10. Vortex shedding created by
an oscillating cylinder: the cylinder was
kept motionless and the Reynolds number
increased in order to study the vortical
structures in the cylinder's wake.
18

to the seafloor. Vortex shedding causes energy transfer from fluid to the structure and leads to
high-amplitude vibrations that can trigger fatigue failure of the structural systems. These new
methods will help accurately evaluate, both qualitatively as well as quantitatively, the fatigue
failure phenomena that is triggered by the turbulence in the natural convective flows around
risers, thus providing a clearer and a more precise picture when the failure can happen. This will
help in taking preventive measure in time, and therefore will be important not only for the design
of these components, but also as a method to monitor these deep sea oil pipelines in the Gulf of
Mexico.
To visualize the 3D nature of turbulent flows, the team turned to NCSA’s Advanced Applications
Support visualization team. Visualization programmer Mark Van Moer used ParaView, a parallel
renderer, with custom VTK scripts to create visualizations for the team. “These visualizations
have tremendously benefited us in the method development phase,” says Masud.
This work will be presented at the 16th International Conference on Finite Elements in Flow
Problems, Munich, Germany, in March 2011.
2.2.11
Earth Sciences: A TeraGrid Community Climate System Modeling (CCSM) Portal (PI:
Matthew Huber, Purdue)
Graduate student Aaron Goldner used the
TeraGrid Community Climate System
Modeling Portal (developed at Purdue)
and TeraGrid computational resources to
show potential for wetter, rather than
drier, conditions in the American
Southwest (and, indeed, the entire
American West and East) with projected
climate change. The research, presented
at the American Geophysical Union fall
meeting in December 2010, offered
another indicator of where current
climate models can be refined. Goldner
was able to use TeraGrid resources for
the computationally demanding task of
including tropical cyclone winds in his
simulations. That yielded a different
potential picture of future water cycles
over North America than current
modeling, which doesn’t include
cyclonic winds and projects drought-like,
rather than wetter, conditions in areas
such as the American Southwest.
The graphical user interface of the
CCSM portal made it easier for Goldner
to set up his modeling initially, while
TeraGrid hardware enabled modeling at a
level of detail that still required more
than a month to complete. The CCSM
portal also has been used in combined
political science/earth and atmospheric
sciences classes at Purdue. The goal is to
Figure 2.11. Enhanced tropical cyclone activity within ocean models can
produce feedbacks that affect El Niño cycles. When these changes in El
Niño are used as boundary conditions within a global atmospheric 19 model
they cause increased rainfall over the entire United States by shifting
atmospheric circulation, thus altering hydrologic cycles.

educate students in the use and misuse of modeling across disciplines, so they will be better
builders and consumers of integrated, complex computer models. Besides climate science, such
models are employed in business, engineering, medicine, the military and many other fields for
purposes ranging from predicting terrorist activity to predicting the structure of protein molecules
affecting human health and disease. Students in interdisciplinary teams (including at least one
liberal arts and one science student) received hands-on experience running the NCAR
Community Climate System Model via the TeraGrid. All students were able to perform 100-year
simulations.
That would not have been possible without the TeraGrid CCSM portal, which eliminates many
barriers for class use. Purdue’s TeraGrid staff built in features to speed up secure access to real
climate data and enable online data analysis and visualization. For instance, the portal’s features
include the ability to tap TeraGrid resources through a community account so each student need
not apply for TeraGrid computational time. A set of multimedia, web-based tutorials help bring
novice users up to speed on using the portal and the climate model itself. Students say the portal
allows them to get results faster and spend more time working with those results in their
interdisciplinary teams, while offering hands-on experience beyond the classroom norm. The
class use also wouldn’t have been possible without the hardware available on the TeraGrid.
Climate modeling is demanding computationally and typical model runs by a class full of
students would be difficult, if not impossible, with the computers and data storage available on
most campuses. The CCSM Portal draws on Purdue’s Steele supercomputer, which is partially
dedicated to TeraGrid use, and on TeraGrid resources at LONI and SDSC. Other schools are now
looking at integrating the CCSM Portal and climate modeling into the classroom.
2.2.12
Fluid, Particulate, and Hydraulic Systems: Fluid and Solid Interaction (FSI) in
Engineering, Biomedical and Other Applications (PI: Cyrus Aidun, GeorgiaTech)
Problems in which fluid and solid interact dynamically occur in various engineering and
biological applications. In such problems, the solid and fluid phases are tightly coupled requiring
the simultaneous
solution of both
phases. In addition,
the high spatial and
temporal resolution
required for such
problems tends to
make traditional
numerical methods
computationally
expensive. Using the
lattice-Boltzmann
(LB) method and
finite element (FE)
analysis for
modeling the fluid
and solid phases,
respectively, can
provide a number of
unique features
making the solution of large problems in this field feasible. Still, the calculations require
extensive data processing and storage.
Figure 2.12. A numerical model to experiment comparison for eight channels, such as those in
heart valves. The lighter shade right-side bars are from simulations and the darker shade left-side
bars are from experiments. The modeling and experimental results agree well for all channels.
20

In 2010, Aidun and colleagues made use of TeraGrid resources at Purdue, LONI, NCSA and
TACC. The researchers have examined exceedingly complex noncolloidal suspensions, work
which lends itself to varied applications such as biological systems (blood flow), paper
manufacturing (wood fiber and coating flows), mining and petroleum industries (for waste
trailing), and home products (paint and cosmetics). For example, blood is comprised of elements
such as red blood cells and platelets suspended in plasma. Most computational modeling of blood
flow has simplified blood as pure fluid flow without the presence of actual suspended particles.
Understanding the physics of blood flow requires accurate simulation of the suspended solid
particles to capture the suspension structure and the blood elements on a microscale. This kind of
understanding could be important in, among other things, treating pathological blood clots, a
common cause of heart attacks and strokes, which strike more than 2 million Americans annually.
Clot formation and detachment also plays into such medical issues as the failure of popular
bileaflet mechanical heart valves. Aidun’s group published results from such simulations more
than once in 2010, including “Parallel Performance of a Lattice-Boltzmann/Finite Element
Cellular Blood Flow Solver on the IBM Blue Gene/P Architecture,” Computer Physics
Communications, 181:1013-1020, 2010. Their paper “Numerical Investigation of the Effects of
Channel Geometry on Platelet Activation and Blood Damage,” accepted by Annals of Biomedical
Engineering in October 2010, reports results that may help optimize the design of prosthetic
replacement heart valves. The researchers also are looking at extremely complex flows through
saturated deformable media, work that could provide insight into material properties of foams and
felts used in industrial applications and lead to improvement in the designs of these materials.
2.2.13
Fluid, Particulate, and Hydraulic Systems: Large Eddy Simulation Investigations of
Aviation Environmental Impacts (PI: Sanjiva Lele, Stanford)
One of the impacts about which the climate research community is very uncertain is the extent to
which aircraft change cloud cover. That’s why Sanjiva Lele and his team at Stanford University,
including PhD candidate Alexander Naiman, are studying the impact of commercial aviation on
global climate. The direct effect is known: that airplanes produce
contrails, which are clouds of ice particles, under certain atmospheric
conditions. But Lele and Naiman have shown there’s also an indirect
effect, where jet contrails induce cirrus cloud formation where clouds
might not have formed naturally otherwise. This is a concern, as
increased cloud cover may reduce a region’s temperature as well as
the amount of solar energy a region receives.
The portion of the research that's being conducted on Abe at NCSA is
a Large Eddy Simulation (LES) of contrails using a highly scalable
parallelized LES code that was developed at Stanford. It is a new
model of aircraft exhaust plume dilution that is used as a subgrid scale
model within a large-scale atmospheric simulation, providing
predictive equations for the evolution of individual exhaust plumes
based on parameters provided by the large-scale simulation. The
simulations start with the wake of a commercial airliner at a cruise
condition, one second after the airplane has passed. In addition to
solving the LES fluid equations, they also compute the deposition of
water vapor onto the ice particles that form a contrail, leading to
cirrus cloud formation. The outputs of their simulations include 3D
fields of all the simulation variables at certain time intervals, which
was not previously possible.
Current simulations continue for 1200 seconds (20 minutes), and the
Figure 2.13. This image shows 3D
isosurfaces of two quantities. The
opaque surfaces on the inside are
vorticity magnitude colored by
streamwise vorticity; these show the
development of the aircraft wake
vortices, which are what give contrails
interesting dynamics. The vortices are
initially parallel, then they form loops
and interact chaotically until they
dissipate. The translucent green/blue
outer surface shows the aircraft jet
exhaust scalar, which essentially
defines the outer extent of the contrail.
21

team plan to extend these into the two to three hour range. The end goal is to understand how
atmospheric conditions and aircraft types affect the properties of a contrail that are important for
calculating climate impacts.
Their results have been published in Atmospheric Chemistry and Physics in 2010 and also
presented in 2009 at meetings of the American Physical Society Division of Fluid Dynamics and
the American Geophysical Union.
2.2.14
Human Computer Interactions: A Window on the Archives of the Future (PIs: Robert
Chadduck, National Archives and Record Administration; Maria Esteva, and Weijia Xu,
TACC)
With the proliferation of digital records, the task of the archivist
has grown exponentially more complex. This problem is
especially acute for the National Archives and Records
Administration (NARA), the government agency responsible for
managing and preserving the nation’s historical records.
In 2014, NARA expects to receive 8.8 Petabytes (10 15 bites) of
electronic records, bringing their accumulated volume to more
than 35 Petabytes.
To find innovative and scalable solutions to large scale and
heterogeneous electronic records collections, NARA turned to the
TeraGrid and TACC, drawing on the expertise of the center’s
digital archivists and data experts.
Using data from NARA’s Cyberinfrastructure for a Billion
Electronic Records (“CI-BER”) research collaboration, a testbed
of documents collected by NARA, the researchers created a
visualization based on the “treemap” and relational database
systems to represent the collection’s arrangement and to show the
its properties at different levels of aggregation, classification and abstraction. The NARA treemap
presents a view of the 3-million-file collection that allows one to identify the differences between
groups of records through visual clues and distinguish patterns and outliers that would be difficult
to spot otherwise. Moreover, it allows archivists to learn by visually comparing and contrasting
groups of records.
TACC’s experts are currently building a multi-touch enabled tiled display system to improve
interactivity and to enhance the collaborative aspects of visual analysis. The group is also
experimenting with cloud-computing methods, using the distributed storage of Longhorn at
TACC with the open-source Hadoop package to scale data
analysis methods.
The collaboration between NARA and the TeraGrid is leading to
the development of tools that combine the power of advanced
computing with the experience and skills of archivists and data
curators and providing a window on the archives of the future.
2.2.15
Molecular and Cellular Biosciences: Computational
Approaches to Understanding Ion Channel and
Transporter Function (PI: Michael Grabe, U. Pittsburgh)
Work by the Grabe lab at PSC and TACC has established
previously elusive mechanisms of substrate transport in an
important class of cellular membrane channels that are key targets
Figure 2.14. Treemap view of electronic
records of 125 Federal Agencies in the
Cyber testbed collection developed for the
National Archives and Records
Administration. The different shades of
yellow represent ranges of number of files,
from more (bright yellow) to less (light
yellow).
Figure 2.15. Image of the sodium-galactose
cotransporter vSGLT (gray). The image
shows several snapshots of galactose
(colored molecule) coming unbound 22 from
vSGLT. The different colors show the time
progression of unbinding (red to purple).
Membrane lipids and water are not shown.

in treating diabetes and obesity. Their results were published recently in Nature (The mechanism
of sodium and substrate release from the binding pocket of vSGLT, Nature, 468: 988–991).
Although many protein structures have been solved at different stages of substrate transport, the
atomic-level insight required to understand the dynamics of this process has been missing. To
address this issue, the Grabe group performed several long timescale molecular dynamics (MD)
simulations (up to 200 ns) of a sugar-sodium cotransporter, vSGLT, on BigBen at PSC and
Ranger at TACC. Their simulations reveal specific conformational changes that occur in the
protein upon the release of sodium from its binding site that result in the release of galactose into
the intracellular space. These results are exciting because they provide specific atomistic
information about the mechanism of transport that can be used in developing strategies for
treating diseases linked to the function of these types of channels, such as diabetes.
2.2.16
Phylogenetic Research: The CIPRES Science Gateway (PI: Mark Miller, UCSD)
For centuries malaria has mystified physicians and terrified patients, claiming more childhood
lives than any other infectious disease across large sections of the world. Though much has been
learned about the genetics of various Plasmodium parasites, which cause malaria across
vertebrate species, key aspects of the evolutionary relationships of these parasites have been
elusive.
Now, with the aid of a portal linking them to TeraGrid expertise and computational resources,
researchers led by the University of Maryland and the University of South Carolina have clarified
the evolutionary history of Plasmodium by analyzing an unprecedented 45 distinct genes from
genomes of eight recently sequenced Plasmodium species.
The results, published online in the journal Parasitology on December 1, 2010, offer the first
comprehensive dating of the divergence of these individual Plasmodium species and provide new
insights into the complex relationship between Plasmodium species and their mammalian hosts.
Figure 2.16. Alternative hypotheses for the phylogenetic relationships among Plasmodium species. Pfal = P. falciparum;
Prei=P. reichenowi; Pgal=P. gallinaceum; Pviv=P. vivax; Pkno=P. knowlesi; Pyoe=P. yoelii; Pber=P. berghei; Pcha =P.
chabaudi. (Credit: Joana C. Silva, University of Maryland, et al.)
“The results clarify the ancient association between malaria parasites and their primate hosts,
including humans,” said James B. Munro, a researcher from the University of Maryland School
of Medicine, Baltimore, Md. “Indeed, even though the data is somewhat noisy due to issues
related to nucleotide composition, the signal is still strong enough to obtain a clear answer.”
A major finding of the research is that humans are likely to serve as reservoir for P. falciparum—
that is, humans are likely to transmit this most prevalent and most dangerous of all the parasitic
infections to great apes, and not the other way around. This finding contradicts previous studies,
which suggested that humans can be infected with P. falciparum derived from apes. The results
obtained in this study argue that “if P. falciparum infections in great apes are derived from
humans, [there may be a] need to establish refuges for the great apes that are safe from human
intrusion.”
23

The research builds on the unveiling of the genome sequences of the two most widespread human
malaria parasites—P. falciparum and P. vivax—and the monkey parasite P. knowlesi, together
with the draft genomes of the chimpanzee parasite P. reichenow; three rodent parasites, P. yoelii
yoelli, P. berghei and P. chabaudi chabaudi; and one avian parasite, P. gallinaceum. To examine
the association between malaria parasites and their primate hosts, the researchers sought to
compare genetic variations found in 45 highly conserved nuclear genes for which sequences are
available from all eight Plasmodium species.
The evolutionary relationships were inferred using, among others, the software package MrBayes,
which consumed about 200,000 CPU hours on Abe NCSA. The researchers accessed this resource
via the CIPRES Science Gateway, a browser interface developed at SDSC that permits access to
TeraGrid compute resources. “Without CIPRES, this work, and the other projects I am working
on, would not go as quickly or as smoothly,” said Munro. “CIPRES is a fantastic resource.”
Other phylogenetic or divergence time analyses were conducted on the Brazos HPC cluster at
Texas A&M University, and on the cluster at the Institute for Genome Sciences at the University
of Maryland School of Medicine. Further studies are expected to be run on Trestles, a new TG
data-intensive HPC resource housed at SDSC.
2.2.17
Physical Chemistry: Gas Phase Chemistry of Molecules and Ions of Astrophysical
Interest (PI: Zhibo Yang, U. Colorado)
Space is not empty. In fact, says Zhibo Yang, a post-doctoral researcher at the University of
Colorado, Boulder, there’s an area of the universe called the
interstellar medium (ISM) that is filled with molecules, atoms,
ions, and grains of dust. And these things are conducting some
very interesting chemical activity.
Among these reactions, ion-molecule reactions are dominant.
Yang and his colleagues are studying reactions that involve
recently detected negative ions with the help of Abe at NCSA.
The team is looking at the reaction between ions and neutrons,
measuring the reaction constants of these reactions, the charge,
and the mass of the negative ions using mass spectroscopy. But
mass spectroscopy yields no information regarding the ion
structure. To get information about the structure energies and to
understand their active mass requires very large calculations.
Yang says, “the typical calculation will last for 12 hours on our
PC but we can do the same calculation on Abe in about two to
three hours.“
So far, the group has completed experimental and theoretical
studies of reactions between hydrogen atoms and carbon chain
anions, and computational studies of gas phase reactions of
carbon chain anions with nitrogen and oxygen. They are
Figure 2.17. This figure explains the
low reactivity between a nitrogen atom
and some typical molecular anions
(deprotonated acetonitrile is used as an
example here). This reaction is spinforbidden,
whereas the energy available
for the spin-conversion is very small,
such that reaction is very slow.
providing some fundamental, general rules to help scientists understand other related reactions.
There are other scientists who are doing modeling, explains Yang, and they want to use the
team’s reaction with constant in their models.
Five oral presentations have been made at international and domestic conferences and six papers
were published in 2010, in the Journal of the American Chemical Society, the Astrophysical
Journal, and in Physical Chemistry Chemical Physics.
24

2.2.18
Physics: The Geophysical High Order Suite for Turbulence (GHOST) (PI: Annick
Pouquet, NCAR)
The Geophysical High Order Suite for Turbulence (GHOST) is a framework for studying
fundamental turbulence via direct numerical simulation (DNS) and modeling. Running on
TeraGrid computers in 2010, GHOST became the first known successful attempt to combine
OpenMP and MPI in a large-scale hybrid scheme for pseudo-spectral fluid dynamics. This
hybridization enables highly efficient use of computing resources both within each node and
between nodes across a network. Using a TeraGrid allocation at NICS, NCAR researchers were
able to do a 3,072 3 simulation of a rotating flow—the largest such simulation in the world—on
nearly 20,000 processor cores. The efficiency achieved with
GHOST allowed them to study flows at a higher resolution and
with greater fidelity by using all the cores on each node. Such
efficiency gains are allowing researchers to solve problems they
could not do before by significantly reducing the processing time
on large core-count computer systems. Pablo Mininni, Duane
Rosenberg, Raghu Reddy, and Annick Pouquet presented their
paper on the topic, Investigation of Performance of a Hybrid
MPI-OpenMP Turbulence Code, at TG'10, and it received the
conference award for best paper in the technology track.
2.2.19
Student Training: Use of TeraGrid Systems in the
Classroom and Research Lab (PI: Andrew Ruether,
Swarthmore College)
Swarthmore College uses the TeraGrid for both student
coursework and faculty research. When Ruether taught
Swarthmore’s Distributed and Parallel Computing course, all
students were given TeraGrid accounts, and they used TeraGrid
systems for class projects. Two of computer science Professor Tia
Newhall’s students extended the work done in the class to a
summer project and presented their results at the TeraGrid 2010
conference. The students developed and tested a novel
parallelization technique for solving the K-Nearest Neighbor
problem. The algorithm is used to classify objects from a large
set. A few examples of its use include discovering medical
images that contain tumors from a large medical database,
recognizing fingerprints from a national fingerprint database and
finding certain types of astronomical objects in a large
Figure 2.18. These four turbulence
visualizations show Taylor-Green flow at
increasing resolutions using 64 3 , 256 3 ,
1,024 3 , and 2,048 3 grid points. This
modeling advancement using a hybrid
MPI-OpenMP code is only possible on
systems with significantly more processor
cores than NCAR’s most powerful
supercomputer. GHOST code has been run
successfully on Kraken and Ranger.
GHOST code scales well on these and
other platforms for hydrodynamic runs
with up to 3,072 3 grid points. These results
demonstrate that pseudo-spectral codes
such as GHOST – using FFT algorithms
that are prevalent in many numerical
applications – can scale linearly to a large
number of processors.
astronomical database. The TeraGrid allowed the students to run large-scale experiments
necessary to demonstrate that their solution worked well for real-world problems.
Swarthmore also has three faculty members—in computer science, physics and chemistry—doing
publishable work using TeraGrid resources. Because Swarthmore does not have a highperformance
computing system, the TeraGrid has been a key resource for faculty researchers who
have reached the limits of desktop computing. In one case, this reduced the run time of a
simulation from one week on a desktop computer to one hour using Purdue's Condor pool, which
offers nearly 37,000 processors and is available as a high-throughput TeraGrid resource. This
allowed
the
research
group to
run many
Figure 2.19. The gray lines are magnetic field lines in plasma and the colored dotted lines are charged
particle orbits through the magnetic field. TeraGrid resources were used to model many particle orbits to get a
sense of the confinement in this configuration. Knowledge gained from the modeling could be useful in
designing fusion reactors based on magnetic confinement.
25

more simulations than it would otherwise have had time to complete. In addition, Swarthmore has
used high-performance TeraGrid systems at LONI, NCSA and TACC. For example, chemistry
Professor Paul Rablen has been using the TeraGrid for an investigation of rearrangements in
certain carbenes, highly reactive carbon molecules widely used as efficient catalysts, among other
places in the pharmaceutical industry. Rablen and a collaborator from Colby College have
submitted a paper describing their project to the Journal of Organic Chemistry. In addition,
Michael Brown, professor of physics, is working on ways to design fusion reactors and uses the
TeraGrid to model the behavior of the plasma in a magnetic containment field. Swarthmore
senior Dan Dandurand, one of Brown’s students, calculated the complex orbits of more than a
billion energetic protons, calculations that help shed light on magnetic confinement fusion. In
another set of calculations, Dandurand determined the fraction of energetic protons collected by a
simulated probe in the plasma. These calculations helped to calibrate and understand an actual
probe used in experiments. The researchers have submitted a paper, “Calibrated Cylindrical Mach
Probe in a Plasma,” using some TeraGrid-produced results to the Review of Scientific
Instruments.
3 Software Integration and Scheduling
The software integration area covers the operational tasks associated with keeping the TeraGrid’s
coordinated software up-to-date and consistently configured across all the HPC, HTC, and
storage resources; as well as a number of small capability improvement projects. In PY6, these
projects include improving our advanced scheduling capabilities, enhancing our information
services, supporting NSF’s software cyber-infrastructure development and maintenance programs
(SDCI, STCI), improving documentation, and formalizing our application hosting capabilities.
3.1 Highlights
TeraGrid began moving its GRAM5 services (used for remote job submission, primarily by
science gateways) to production status, replacing older GRAM4 services that had been used for
more than a year and in some cases as much as two years. These updates will enable science
gateways to begin reporting end user identities to HPC systems when jobs are submitted under
community (gateway-wide) credentials.
We also made modest improvements to our Automatic Resource Selection capabilities and added
resource descriptions (both HPC and data resources) and science gateway Web service
descriptions to our integrated information services.
3.2 Coordination and outreach
The GIG Software Integration team participated in various TeraGrid internal coordination
activities and in several significant external outreach and collaboration activities.
Notable internal coordination activities included working with the Science Gateways areas on the
rollout of Globus Toolkit Version 5 (GT5) related capabilities, and ongoing discussions with the
Common User Environment (CUE) team on the CUE implementation.
3.3 Capability Expansion and Maintenance
Capability expansion and maintenance include the coordinated delivery, deployment,
maintenance, and support of Coordinated TeraGrid Software and Service (CTSS) capabilities.
This includes Advanced Scheduling and Information Services capabilities that are presented in
their own sections below.
The most notable capability expansion this quarter was the delivery of improved Science
Gateway Support and Remote Computation capabilities with the new Globus Toolkit GRAM5
26

service. In addition to GRAM5, we are also updating other capability kits to include the latest
versions of Globus software, including the data movement capabilities (GridFTP servers) and
several client capabilities (Globus, UberFTP, GSI OpenSSH, and MyProxy client tool updates).
This activity started with GRAM5 Alpha testing in the summer of 2009 and culminates this
quarter with production deployments by RPs and testing by various Science Gateways. This
deployment activity will continue through 2011Q1.
We are also delivering updates to TeraGrid’s Resource Integration capability (to enable
publishing to the new Resource Description Repository and publishing TeraGrid storage resource
information), our Science Workflow Support capability (updating Condor to version 7.4.4), and
our Data Movement capability (adding support for TeraGrid’s long-term data archive replication
mechanism). We also released the first official TeraGrid Visualization Software and Services
(VTSS) registration capability enabling Viz resources to advertise visualization software
information.
3.4 Advanced Scheduling Capabilities
SDSC improved the reliability of TeraGrid’s Automatic Resource Selection capability by adding
persistent scheduling data to the TGCDB (central database). We also began work toward making
MCP (one of our automatic resource selection tools) available from TeraGrid resource login
nodes in addition to the TeraGrid user portal. To support MCP—and perhaps other applications—
we reviewed the practice and gave approval to issue proxy credentials with up to 11-day
lifetimes. We also developed a method for transferring MCP job files from the user portal to
remote login hosts.
3.5 Information Services Enhancements
We took the opportunity this quarter to improve the overall accuracy and consistency of the data
in our information services. TeraGrid’s information services are populated by data that is
published independently by each RP, which creates opportunities for inconsistent terminology
and uneven accuracy across systems. To measure the current quality of the data, we ran several
data consistency sweeps, checking for inconsistent naming, versions of software that were
inconsistent with documented expectations, and data that simply didn’t match our understanding
of reality. The Software WG dedicated one meeting to reviewing these inconsistencies and
assigning fixes to individual publishers, and we later checked that fixes had been made.
We are now using TeraGrid’s information services to publish information from the TeraGrid
Resource Description Repository (RDR). Each RP includes resource description information in
RDR, and information services pulls that information and publishes it for unified presentation to
users and staff.
Led by personnel at RENCI, the Science Gateway team is beginning to publish data in our
information services that enumerates and describes the Web services offered by science
gateways. This is an important example of how non-TeraGrid organizations (remember that
science gateways are not operated by TeraGrid itself, but by science teams with independent
funding) can use the TeraGrid infrastructure to disseminate useful scientific tools.
3.6 Operational Issues
The SDSC RP continued preparing the new Gordon resource for TeraGrid service, including
preparing CTSS capabilities and software. The Gordon team reported that they would like to see
more comprehensive and easier-to-use installation documentation for CTSS capabilities. The GIG
software integration team is working to improve this documentation per this feedback.
The TACC RP prepared Globus GRAM5 software for use on multiple resources at TACC (see
“Capability Expansion and Maintenance” above), and they reported that it has been difficult to
27

get GRAM5 from Globus 5.0.2 (the most recent update) to work uniformly on all of their
resources, particularly in regard to parallel (MPI) jobs that use multiple processes. The Globus
development team (funded separately from TeraGrid) has been working with TACC personnel to
resolve these issues.
CTSS’s remote login, data movement, application and parallel application development and
runtime support, and TeraGrid core integration capabilities are all used on most TeraGrid
systems. Use of the CTSS science gateways capability (which allows remote job submission by
science gateways) and the remote compute capability (which allows remote job submission by all
other users) varies widely at three sites, are used frequently at three sites, and are used
infrequently at two sites. Only two resource providers offer the data visualization capability but
both report infrequent use. The data management capability is used relatively infrequently at the
two sites where it is available. The science workflow support capability is available on most
resources and is used relatively infrequently, but each use results in many job submissions,
making it difficult to compare to the other capabilities.
4 Science Gateways
4.1 Highlights
2.2M CPU hours were used via community accounts this quarter, serving 942 gateway users via
12 gateways. 32% of all users charging jobs to the TeraGrid this quarter did so using community
accounts via a gateway. These figures are achieved primarily through the explosion of user
interest in the CIPRES portal, with 751 users. The portal sees use throughout the US, including
17 EPSCOR states and throughout the world. It is used in many classes, but also used in research
resulting in publications in major journals such as Nature, Cell and the Proceedings of the
National Academy of Sciences.
Figure 4-1.Q4 Science Gateway usage by machine.
28

Figure 4-2 Q4 usage by gateway.
The Indiana University Science Gateway team worked with Prof. Borries Demeler and Dr. Emre
Brookes of the University of Texas Health Science Center to transition their UltraScan science
gateway infrastructure from TACC’s Lonestar to Ranger systems. During the current reporting
quarter, UltraScan’s Ranger allocation usage went from 0 to 57,747 SUs, using the GFAC service
the Open Gateway Computing Environments (OGCE) to manage its ultracentrifugation
experimental data analysis and mask differences between Globus GRAM versions. This work
also involved significant testing of the new GRAM5 services on Ranger that greatly accelerates
TeraGrid’s deployment schedule.
4.2 Targeted Support
CIPRES.
The CIPRES gateway can now support individual user allocations. Heavy users of the gateway
can apply for their own allocation and access it directly through the same familiar gateway
interface. With the high demand for the gateway, this is a method for extending the allocation, but
will also to enable individual researchers to accomplish their own research goals unfettered by
limited computational time..
Tools were moved from Lonestar to Abe due to decommissioning of the temporary
decommissioning of Lonestar during its upgrade. Improvements to gateway user interface include
bulk file upload and download. Next quarter plans include porting codes raxml, mrbayes and
mafft to the new Trestles system.
Cactus.
The basic configuration and simulation commands for SimFactory now work with Adcirc, a
coastal circulation and storm surge model. SimFactory (http://simfactory.org) unifies middleware
for remote access and job management. The additional commands --adcprep-schedule, --enableadcprep-submission,
--adcprep-walltime were added to address Adcirc's specific needs.
29

Several additional capabilities have been developed for SimFactory. Instead of submitting
directly to a queue, it is now possible to request an interactive job using Run/Interactive submit
feature of SimFactory, and then to manually run the job inside the interactive shell. This
facilitates running on private clusters which don't need a queuing system. It is also now possible
launch Totalview on TeraGrid machines Queenbee, Abe, Kraken, and local machine Eric using
SimFactory. This relies on the run/Interactive submit feature. DDT debugging support will be
available next quarter.
Advance Reservation will be available soon on Queenbee, or other machines that use Moab.
GridAMP.
This gateway has been taking a leadership role working with sites that are using the community
shell commsh. Commsh restricts the directories where commands can be accessed, thus adding a
level of security. GridAMP is working with Kraken on their commsh implementation.
Maintenance work next quarter includes rewriting a daemon to use GRAM2/5 instead of WS-
GRAM. The team is also considering a ssh/gsissh based method like the CCSM Portal to utilize
the PBS scheduler’s “job chaining” ability.
GridAMP staff have also co-authored a paper with security staff entitled “Securing TeraGrid
Gateways”. The paper includes a survey of TeraGrid resource providers on their approaches to
securing community accounts and lays the groundwork for a more standardized approach
throughout the program. This work is described in more detail in the Gateway Infrastructure and
Services section below.
The gateway has automated TeraGrid central database (TGCDB) accounting feedback using
REST-based services developed by gateway accounting staff (section 4.3). Future plans include
updates to use new (January 2010) versions of the interface. The accounting data from TGCDB
now feeds back all the way to the gateway for examination by the individual researchers and the
PIs. Job lifecycle is tracked from submission through execution and completion, and now all the
way through the accounting record. Every job is tracked from start to finish, and every SU is
accounted for.
PET.
Work begins on a gateway prototype, leveraging efforts of a new NCAR-funded software
engineer for science gateway development. First, the project will focus on automation of the
execution portion of the workflow, allowing execution on grid-based and remote resources.
Community members will be able to upload full experiment definitions, and the system will
automate the execution of the models and return results to the researcher. Once this works, it will
be used as the back-end for the new gateway work. The gateway web development is separately
funded by NCAR. This will take the grid-automated workflow and overlay an interface that
allows web-based users to run selected prototype experiments but vary the input parameters
within constraints.
The PET model is a global economy model that simulates the action of various sectors of the
economy for regions of the world. Economic inputs and outputs are balanced in a region and
between regions through trade, given prespecified growth rates of things like population.
The goal of the gateway project is to make it possible for people to experiment with how
changing parameters changes the behavior of the economy. For example, how changing the
interest rate, the savings rate, the capital tax rate, or introducing carbon taxes in a certain region
affects sectors of the economy such as energy resource use and capital savings.
The desired outcome is to have a gateway that can be quickly used to help address the "but what
if" comments that always arise when people encounter results from a model such as this. Upon
30

hearing about the experiments, many people start off by saying "Well, this is interesting, but what
if (assumption) was (some slightly different value) instead of what you used" The gateway
should allow users to experiment with moderately computationally intensive experiments to
address these questions.
This work is in the early prototype stage.
Social Informatics Data Grid (SIDGrid).
The team has improved portability of existing gadgets (tunneling requests through container;
gadget-gadget communication using HTML5 facilities), and tested these in an iGoogle container.
Client-side script and server-side support have been developed to simplify the introduction of
new applications into the portal. Work began to expose multiple local and remote compute
resources through portal.
SIDGrid developer Tom Uram presented demonstrations of his work at the Argonne booth at
SC10 and also served as a committee member for the Gateway Computing Environments
workshop the conference.
Support continues for OOPS (protein simulation) users of portal, including the addition of new
scripts to the portal.
Next quarter plans include implementation of a data management front-end to allow users to
upload, manage, and process data using gadget-based applications defined in the portal.
Alternatives will be investigated for managing applications available to portal, including
integration with source code management systems used by scientists. Work will continue on
gadget portability, including testing in third-party containers such as iGoogle and OGCE.
Approaches will be evaluated for single sign-on in gadget context so that users won’t have to
authenticate with each gadget in a container page.
Support for groups using the OOPS (Open Protein System) portal will continue.
GCE Workshop.
The IU Science Gateway team organized the sixth Gateway Computing Environments (GCE)
workshop at Supercomputing 2010. This year’s workshop featured presentations of 13 peerreviewed
papers to be published in IEEE Digital Proceedings. The one day workshop was
attended by approximately 50 people. Presentations and additional information are available
from the workshop website, http://www.collab-ogce.org/gce10/index.php/Main_Page.
UltraScan.
The Indiana University Science Gateway team worked with Prof. Borries Demeler and Dr. Emre
Brookes of the University of Texas Health Science Center to transition their UltraScan science
gateway infrastructure from TACC’s Lonestar to Ranger systems. This involved removing
UltraScan dependencies on Globus GRAM4 and significant testing of Ranger’s new GRAM5 job
manager (documented in previous quarterly reports). UltraScan now uses the GFAC service
developed by the Open Gateway Computing Environments (OGCE) to manage its
ultracentrifugation experimental data analysis. GFAC’s capabilities include masking the
differences between GRAM versions. During the current reporting quarter, UltraScan’s Ranger
allocation usage went from 0 to 57,747 SUs. The IU team is also assisting the UltraScan team as
it develops its next generation software (UltraScan3, led by Dr. Bruce Dubbs). Both UltraScan2
and UltraScan3 use IU’s Gateway Hosting Service for significant testing and development. IU
work was led and performed by Suresh Marru and Raminder Singh, with assistance from
Patanachai Tangchaisin (student intern). As part of this overall effort, the Indiana University
Science Gateway team worked closely with Globus developers and TACC & LONI system
31

administrators in testing and debugging problems for Gram5 deployments and job manager
integration. The IU team, through its testing and bug fixing, has assisted the TeraGrid Software
Working Group in rolling out the new Gram job submission into the CTSS release.
In the upcoming quarter, the advanced support will focus on resource scheduling and job queue
predictions.
BioVLAB.
BioVLAB is a set of projects led by Prof. Sun Kim at IU that investigates problems in cancer
research that can be realized as bioinformatics workflows. Kim implements the workflows using
the OGCE workflow suite of tools (including the GFAC, the XBaya workflow composer, and the
Workflow Interpreter service). Prototyping work is done on Amazon EC2 cloud infrastructure,
but the size of the full research problems requires the use of the TeraGrid. Suresh Marru of the IU
Science Gateway team assisted Prof. Kim to request and obtain a TeraGrid allocation (reported
last quarter). During the upcoming quarter, Marru is working with the Dr. Kim’s graduate
student to support long running applications and conditional loops within the workflows.
Ocean Land Atmosphere Simulation (OLAS).
The IU Science Gateway Team has worked with Mike Lowe in acquiring new data storage
hardware to meet the real-time data needs for the OLAS gateway for hurricane simulation. Suresh
Marru from the gateways team has setup quarry gateway hosting node for OLAS gateway. The
Unidata LDM data ingesting tools are deployed to fetch real time data streams from National
Hurricane Center and other NOAA sources. The PI of the project Dr. Craig Mattocks is in a job
transition from University of North Carolina Chapel Hill. This project will slow down in the
upcoming quarter.
GridChem/ParamChem.
The IU gateways team including the student interns Ye Fan and Patanachai Tangchaisin has built
an OGCE-GridChem bridge services and have integrated the OGCE workflow suite with the
GridChem Middleware services. As reported in the previous year, the gateways team has
provided advanced support to the GridChem gateway to enhance the single job submission to do
coupled workflows. The newly developed OGCE-GridChem bridge has closed the loop on the
workflow integration. Now, the GridChem users will be able to monitor workflow progress
directly from the GridChem clients. Within the current quarter, the gateway team will assist
Sudhakar Pamidigantam and Jayeeta Gosh from GridChem to incorporate the new developments
into the upcoming release.
Neutron Science TeraGrid Gateway.
The ORNL Neutron Science TeraGrid Gateway (NSTG) has spent a significant amount of time
during this quarter continuing its outreach to the neutron science community, specifically at Oak
Ridge. The NSTG is working with the neutron science directorate at Oak Ridge to transition the
gateway to full community operations and support of the gateway and NSTG RP/SP on a
TeraGrid contributory basis moving forward. This quarter built upon the prior quarters successful
physical infrastructure move to the Spallation Neutron Source (SNS) sight. Labor effort was spent
on building SNS in house familiarity with TeraGrid operations and culture. This has been a
challenge but is not without progress. The effort is continuing. Additional effort has been spent
developing and presenting strategies that both explain the mutual benefit and enable the
continuation of the current activities and the realization of opportunities not yet explored. In
particular, the ability to move data at high bandwidth and low latency from the scientific
instruments to large scale cyberinfrastructure facilities is understand and appreciated by the SNS
and will, most likely, form the core of the continuing effort.
32

John Cobb presented the capstone talk at the NOBUGS 2010 conference describing the NSTG
activities and integration with SNS as a “Best practice” for the integration of national
cyberinfrastructure with an experimental user facility. Cobb was also on the conference
organizing committee.
GISolve.
This team continues to explore new applications and communities for high performance
computing. Recent interactions have been with the land use science community. Future plans
include working with a public health community. Several publications from the group are listed
in this report’s appendices.
CCSM/ESG.
The joint team is making progress as planned in all three major project areas. Next quarter,
overarching goals include completion of work on software licensing and the completion of phase
I of the prototype CESM1 gateway. This will support running CESM1 simulations on Steele,
automatic production of attributes, and automatic publishing of model output to ESG.
Task 1 is to provide interfaces for remote model run invocation. CESM1 is being tested on Steele
at Purdue. The team has partially resolved the MPI communication problem, but there are still
some problems scaling up. CESM-WS web service interfaces have been deployed to a production
server, including bug fixes and performance improvements. Incremental changes support new
requirements from the CESM1 gateway client. The CESM1 gateway directly invokes the CESM-
WS interfaces and currently supports case creation/configuration/submission and status tracking
Next quarter the team will address the CESM1 scaling issues, complete model validation and add
support for data access and publishing of model output from the prototype CESM1 gateway to
ESG.
Task 2 is to publish Purdue CCSM archive data to ESG. Memory of the Purdue ESG data node
has been upgraded. The FUSE interface to the ESG test gateway has been tested and successfully
published (manually) an iRODS CCSM3 data collection through this interface.
Next quarter the team will automate publishing of model output from the CESM1 gateway to
ESG test gateway.
Task 3 is to collect and publish CESM1 model data and metadata back to ESG. Work continues
on instrumenting CESM such that it can produce a METAFOR-compliant CIM document
(WCRP/CMIP5 metadata). Support was added for outputting ESMF attributes from the CESM
components and driver, and the attribute suite was further analyzed. Tested the system, and
provided code back to CESM team for integration. The team completed development of an initial
version of an ATOM feed for ingesting CIM documents into ESG. Integration of the feed was
tested with ESG, publishing model metadata back into it.
Next quarter the team plans to review metadata content and completeness by Purdue, add code to
CESM to automatically set attributes from within CESM code, integrate as CESM tagged release,
and test at Purdue. Packaging of the ATOM feed will be refined and released in a V1.0.0 open
source version. Basic installation and usage documentation will be completed and deployed at
Purdue.
Purdue Environmental Data Portal.
The Environmental Data Portal has integrated three new data sets, the Indiana precipitation
dataset, St. Joseph watershed water quality and stream flow data and a historical time series of
average precipitation and temperature data for Midwest.
33

Purdue has also created a data publishing web application so individual researchers can self
publish and share their datasets. Both file based and table based data are supported. Completion is
expected next quarter.
The Springboard hub extends access to HPC beyond the command line. Access to Purdue’s
Wispy cloud have been integrated and version 1 of Cloud Dashboard, a user interface hosted in
Springboard allows TeraGrid users to launch, control and run applications on Wispy,. This was
demonstrated at CloudCom in December 2010. Springboard has also been upgraded with the
latest HUBzero software.
4.3 Gateway Infrastructure and Services
Accounting Tools.
A new job monitoring portlet has been deployed in the TeraGrid user portal and is running at all
sites. The portlet includes start time predictions for queued jobs. The User Profile Service caches
job information from the TeraGrid central database. This includes an expanded search interface, a
view of real time jobs, attribute support and performance improvements, responding to user
requests.
Improved standardization of community accounts.
A survey of Teragrid RP sites regarding gateway security items has been completed. Results are
being written up in a white paper.
Security work also includes installation and testing of GRAM5 and its interoperation with
commsh. The discovery of incomplete configuration instructions has resulted in rerunning tests
for the while paper. Subsequently, improvements have been made to configuration instructions as
well as GRAM5 installation instructions.
Next quarter, several projects will be completed - GRAM4 installation on Kraken (including
integration with commsh and gramAudit and testing with the GridAMP gateway) commsh
documentation updates. and the white paper on securing TeraGrid science gateways
Work will begin to convert the GridAMP gateway to GRAM5 on Kraken.
Virtual machine support.
During this quarter, Indiana University added virtual machines to support failover for the
TeraGrid Mobile Portal and TeraGrid Forum.
SimpleGrid.
Job management and visualization Open Social gadgets have been developed for SimpleGrid this
quarter. These were demonstrated at SC10. SimpleGrid gadgets allow scientists to use TeraGrid
in iGoogle and other open social containers, contributing to broad use and easy integration of
TeraGrid capabilities through social networking.
SimpleGrid online courses are being developed in CI-Tutor, which is used extensively in
TeraGrid’s education, outreach and training activities. Previous online courses have been
extended to include more examples and better instructions.
Next quarter, the team will deploy SimpleGrid as a cloud resource for new science gateway
development support. Users will be able to request a virtual machine image with SimpleGrid
installed.
Helpdesk support.
The gateway helpdesk team continued to help new gateway developers including some from
National University through the use of the SimpleGrid tutorial.
34

Work initiated by a helpdesk ticket from the CMMAP gateway has evolved into policy
discussions which have had positive security implications. This work may to improved
standardization across Resource Providers in the treatment of community accounts. Helpdesk
staff also provided advice to CMMAP on the setup of a gateway-side GridFTP server.
A helpdesk ticket inquiring about support for a socket-based application with web services from
NCAR has resulted in an initial meeting with the security-wg to discuss the usage model and
determine whether it can be supported by any Resource Providers.
The helpdesk staff maintain a FAQ at http://gw3.quarry.iu.teragrid.org/simplegrid2/docs/faq/
which will be incorporated into the TeraGrid Knowledge Base and documentation.
4.4 RP Operations: SGW Operations
During this quarter RPs worked with developers and providers of Scientific Gateways on specific
RP resources in order to ensure that operation of each gateway is secure and that there is a means
for tracking accountability for gateway usage.
All labor funding in the extension period comes through the GIG and so those activities are all
captured in the above sections.
5 Users and User Support
User support in the TeraGrid comprises interlocking activities in several project areas:
documentation, allocations and accounting services in the User Facing Projects and Core
Services area (Section 6 of this report); the TeraGrid Operations Center in the Networking,
Operations and Security area (Section 8); user outreach and training in the Education, Outreach
and Training/External Relations area (Section 10); and the Frontline User Support and Advanced
User Support services described in the current section. In addition, specialized user support
services are provided by staff from the Science Gateways (Section 4) and Data and Visualization
(Section 7) areas. The synergy of all these aspects of user support is coordinated by the User
Interaction Council (UIC) which consists of the GIG Director of Science and the directors of the
above mentioned areas.
5.1 Highlights
The User Support (US) and Advanced User Support (AUS) staff continued their highly rated
services for TeraGrid users at every level. As needed, US and AUS staff collaborated with the
Data and Visualization, Science Gateways, EOT and UFC staff to provide comprehensive support
for TeraGrid users.
The UIC and the Extreme Scalability Working Group initiated the planning of the 5 th annual
TeraGrid/Blue Waters Workshop. This will be held in Pittsburgh, PA April 14-15, 2011 with the
topic “Data-Intensive Analysis, Analytics, and Informatics”.
Successful collaborations between various Advanced User Support staff and TeraGrid PIs
resulted in joint journal publications (Phillip Blood, PSC) and conference presentations (Wayne
Pfeiffer, SDSC; Raghu Reddy, PSC), including Gordon Bell finalists’ presentation (Yifeng Cui,
Amit Chourasia, SDSC) at SC10. One of the EOT highlights, in this quarter, was teaching of an
Introductory HPC workshop at Puerto Rico (University of Puerto Rico at Rio Piedras and
University of Puerto Rico at Mayaguez) in December. Multiple AUS staff such as Marcela
Madrid (PSC), Mahidhar Tatineni (SDSC), Amit Majumdar (SDSC), Sergiu Sanieleveici (PSC)
participated in this workshop at Puerto Rico.
35

User support activities focused on the introduction of the new systems to be fielded by several
RPs. Friendly use of Blacklight at PSC has already resulted in remarkable scientific progress,
notably by Dr. Sarah Young from the University of Arizona and the Broad Institute of Harvard
and MIT, working with PSC senior scientific specialist Dr. Phillip Blood. This group, led by Dr.
Steve Rounsley (U. Arizona), is comparing different assembly techniques, codes, and genomes on
Blacklight using simulated genome data, in order to inform the genomics community of the
optimal computational strategies to run their assemblies. As Blacklight has been built up, they
have run assemblies of up to 600 megabases (the Sorghum genome) with various data sets and
levels of coverage using the assembly codes ABySS, SOAPdenovo, and Velvet. Early access to
Blacklight has enabled them to quickly try different assembly parameters and different assembly
codes to find optimal ways of assembling these large genomes. They are currently working on
analyzing the data they have generated. They publish the results of their study for the community
at http://www.plantagora.org/ .
SDSC user services staff members installed software, ran benchmarks, and created a user guide in
preparation for putting the new Trestles resource into production. The machine was made
available for friendly use in early December. Detailed benchmarking was performed for several
applications, including AMBER, NAMD, ABAQUS, Velvet, SOAPdenovo, and ABySS and the
final results will be made available online. They also continued to enhance software and services
on Dash. A new framework was developed to enable users to run Hadoop on Dash and the
service is currently being tested by friendly users. In addition, the production vSMP software was
upgraded and tested. Early user experiences with Dash were presented at the “Grand Challenges
in Data-Intensive Discovery” conference organized and hosted by SDSC in October 2010.
We have also focused on assisting potential and new TeraGrid users, whom we recruit in
increasing numbers and who present novel challenges different from those of the traditional HPC
community. For example, Purdue support staff showed a potential TeraGrid user that it is
possible to run his Delphi code on a Windows environment via the Condor Pool. They assisted
another user to run his software on the Condor Pool under Linux instead of the initial request of
running on Windows; and helped another to explore whether he could install and use Numerical
software on TeraGrid machines.
5.2 User Engagement
User engagement is done by various means. Under the User Champions program, RP consultants
are assigned to each TRAC award right after the results of an allocations meeting become known.
The assignment takes place by discussion in the user services working group, taking into account
the distribution of an allocation across RP sites and machines, and the affinity between the group
and the consultants based on expertise, previous history, and institutional proximity. The assigned
consultant contacts the user group as their champion within the TeraGrid, and seeks to learn about
their plans and issues. We leverage the EOT area’s Campus Champions program (Section 10) to
fulfill this same contact role with respect to users on their campuses, especially for Startup and
Education grants. Campus Champions are enrolled as members of the user services working
group, and thus being trained to become “on-site consultants” extending the reach of TeraGrid
support. To help users efficiently scale their codes and ancillary data processing flows to the tens
of thousands of processes (on Ranger and Kraken), the Advanced User Support and User Support
areas jointly operate the extreme scalability working group. Led by Nick Nystrom (PSC), this
team brings together staff experts from all RP sites, external partners working to develop
petascale applications and tools, as well as TeraGrid users and awardees of the NSF PetaApps,
SDCI and STCI programs.
Some examples of issues raised by users and actions taken in response are as follows. In the Fall
2010 NICS User Survey, several respondents noted that the weekly NICS Capability (full-
36

machine) runs limit use of the system by smaller jobs. In response, NICS has decided to target the
new cores that will be added to Kraken (15% of the total) to such jobs.
At PSC, suggestions and requests made by production users of Pople and by friendly users of
Blacklight resulted in the installation of new versions of R, AMBER and LS-DYNA; making
Gaussian easier to run; issues encountered when launching Gaussian jobs via the GridChem
Science Gateway; and improving the instructions to change the PSC password for users who
prefer to log into PSC systems directly.
In response to users’ request to set up and run Python on Steele, Purdue staff created and
documented specific examples. They also documented how to run codes in the Condor pool using
static libraries, and how to run RMPI on Steele. The latter was done based on requests from the
Campus Champions group.
5.3 Frontline User Support
In this section we describe the frontline user support provided by TeraGrid user support staff. In
addition, TeraGrid provides online documentation, a knowledgebase and training support for
“self-serve” help and this is explained in detail in §6. By means of the UIC, Frontline and
Advanced support functions are operationally integrated with the TeraGrid Operations Center
(TOC) and the “self-serve” online help team.
The TOC help desk is the first tier of user support for the TeraGrid. It is staffed 7x24 at NCSA.
All TeraGrid users are asked to submit problem reports to the TOC via email to
help@teragrid.org, by web form from the TeraGrid User Portal, or via phone (866-907-2382).
The TOC creates a trouble ticket for each problem reported, and tracks its resolution until it is
closed. The user is automatically informed that a ticket has been opened and advised of the next
steps, as well as of a ticket number enabling follow-up.
If a ticket cannot be resolved within one hour at the TOC itself, it gets passed on to the second
tier of user support. It is referred to the RP where the problem has been experienced, or, if the
problem appears to be TeraGrid-wide, it is reported to the full user services working group.
There, it is assigned to a user consultant who begins by discussing the matter with the user, in
order to narrow the diagnosis and to understand if the issue lies with the user’s application, with
the RP or TeraGrid-wide infrastructure, or both. Tiger teams are formed by the user services
working group as necessary, to diagnose and solve complex problems that implicate several RP
sites or fields of expertise. The personal contacts assigned to TRAC awarded projects (via the
User Champions program), as well as the Campus Champions, proactively seek to develop
porting and optimization plans with their assigned users. They act as proxies for “their” users
with respect to managers, systems administrators and vendors at their local RP site and (via the
user services working group) at other RP sites, ensuring support for any special arrangements that
may be required for debugging, testing and benchmarking new or upgraded user codes.
As needed, consultants may request the assistance of advanced user support staff (the third tier of
user support), systems, or vendor experts. The immediate goal is to ensure that the user can
resume his or her scientific work as soon as possible, even if addressing the root cause requires a
longer-term effort. When a proposed root cause solution becomes available, we contact the
affected users again and request their participation in its testing. When a user’s problem requires
active assistance from a M.S.- or Ph.D.-level staff exceeding approximately one person-month of
effort, the issue is referred to the third tier of user support, i.e. for Advanced User Support
described in §5.5 following the guidance of the allocations process. In terms of experience and
qualifications, frontline support consultants range from the B.S. level to the Ph.D. level, with
many of the latter dividing their effort between frontline and advanced user support.
37

The framework for coordinating frontline support across the TeraGrid is the user services
working group which assembles staff from all the RP sites under the leadership of the Area
Director for User Support. It enables each site’s staff to leverage the experience and expertise of
all others, and a concerted approach to solving cross-site user problems based on sharing and
adopting best practices. The working group functions by means of an email list and weekly
conference calls. It coordinates the User Champions and Campus Champions programs that, as
we mentioned above, provide user engagement at the personal level. By involving the Champions
in resolving tickets submitted by “their” users, we can improve the promptness and quality of
ticket resolution, and often prevent problems from occurring in the first place, or from recurring
repeatedly.
5.3.1
Helping users port software into a TeraGrid environment
In table 5.1 below, we provide some instances of assistance provided by our consultants to enable
or improve users’ work on TeraGrid resources, which did not rise to the one FTE-month
involvement level that defines “advanced” user support.
Table 5.1. Examples of Frontline User Consulting Porting Software on TeraGrid Resources
PI Name
Juri Toomre, U.
Colorado
Yannick Pouquet,
NCAR
Paul Hargrove, UC
Berkeley
Noah Smith,
Carnegie Mellon U.
Paul Reynolds, U.
Virginia
Sameer Shende, U.
Orgeon
RP(s)
Involved
PSC,
NICS
PSC,
NICS
PSC
Name of
software
SLASH,
gridFTP
MHD Code
Berkeley
UPC
Nature of improvement
End-to-end transfer of 3TB from PSC archive to
NICS
Troubleshoot problem with the code and help them
to benchmark it on Kraken
Support for development of the Berkeley UPC
compiler during the build-up of Blacklight
PSC Java Investigated and solved performance problems
during the build-up of Blacklight
PSC Java Installed OpenJDK and solved performance
problems during the build-up of Blacklight
PSC TAU Tested new and advanced features during the buildup
of Blacklight
Nick Wright, UCSD PSC IPM Solved performance problems during the build-up of
Blacklight
Mark Ellison, Ursinus
College
PSC Gaussian PI estimated his Gaussian job would take 800 hours
to complete on Pople and he sought help speeding
up the calculation. PSC consultant determined the
optimal parameters for running this Gaussian
calculation, speeding up the job by ~3.5x
38

PI Name
Joseph Hargitai,
Albert Einstein
College of Medicine
Catherine Cooper,
Washington State U.
RP(s)
Involved
Name of
software
Nature of improvement
SDSC Hadoop SDSC staff worked to integrate Hadoop with the
global filesystem, local SSDs, and the scheduling
infrastructure. The PI is currently testing the
implementation.
TACC Underworld TACC staff member Yaakoub El Khamra worked
with the research team and the code developer to
develop a version of the code that works on Ranger.
This version of the code is the bleeding-edge
version and modules are still being modified and
tested. The PI is planning to submit an ASTA
request in order to received continued support.
Neutron Science
TeraGrid Gateway
ORNL Amber Consulted with user on desired Amber version.
Initiated arrangements to install Amber11 on NSTG
cluster
Given the strong interest by our user community in utilizing the potential of GPGPU systems, it is
noteworthy to mention, in particular, the assistance given by TACC staff members to three
research groups on Longhorn:
5.3.2
• Lucas Wilcox (PI Omar Ghattas) ran a GPU-accelerated discontinuous Galerkin (DG)
seismic wave propagation code on up to 478 GPUs of longhorn, achieving excellent
strong scaling and near-perfect weak scaling. His largest run on 478 GPUs used a mesh
with 6 billion nodes and 56 billion degrees of freedom.
• Tom Fogal (PI Hank Childs) ran a GPU-accelerated volume rendering code on an 8192-
cubed helium flame combustion dataset on 256 GPUs. This is among the largest data
ever to be directly visualized, and this GPU-based code achieved performance on 256
GPUs that would require 100,000s of CPU cores to match (they performed a similar run
on Jaguar for comparison).
• David LeBard ran a GPU-accelerated fast-analysis molecular dynamics code on 128
GPUs of Longhorn, achieving between 10x to 100x over CPU-based versions.
More Frequent User Support Issues/Questions
Among the system or site specific user issues referred to the RPs for resolution, the most frequent
had to do with login/access and account management issues (e.g. TeraGrid Portal password
versus resource specific passwords, password resets, security credentials, adding and removing
users, locating environment variables, allocations on machines leaving and joining the TeraGrid).
In decreasing order of frequency, issues with job queuing (e.g. which queues to use for various
types of jobs, throughput issues, scripting and execution issues); software availability and use
(finding, setting up, building, optimizing and running); system availability and performance
problems (e.g. memory limit questions, compute node failures, I/O timeouts, requests to kill
jobs); file systems (e.g. permissions and quotas, corrupted files), and wide area data transfer and
archive systems questions were also encountered.
39

5.3.3
Trouble Ticket Statistics
Table 5.2 shows how long tickets remained open during this quarter, that is, how long it took for
TeraGrid user support staff to provide a diagnostic, workaround or solution.
Table 5.2. Number of Q4, 2010 tickets that remained “open” for the indicated time bins.
Open for 1 hour, 1 day, 2 days, 1 week, 2 weeks, 4 weeks 140
5.4 RP Operations: User Services
5.4.1
Consulting Services
Ongoing daily operational user support was conducted at the RPs. Consulting is the second tier of
TG’s three-tier user support, with the first tier being the TG Operations Center and the third tier
being the TG Advanced User Support. TG RP Consulting staff provide real-time, frontline
support for their local RP’s resources in response to users’ requests to the TeraGrid Operations
Center and RP consulting systems. This support includes assistance such as allocations issues,
initial login issues for new users, debugging jobs, data transfer, and compiler support. Consulting
support typically engages a user for a relatively short period of time, and may pass a user on to
Advanced Support if longer-term assistance is required. Some examples of site-specific assistance
achievements and their impact on users’ research progress follow.
PSC has operated a friendly user program for community participation in successive stages of
building up Blacklight, its very large shared memory SGI Altix UV system scheduled to enter
TeraGrid production in January 2011. The PSC consulting staff has worked with 31 friendly user
groups, with a focus both on ensuring good performance for existing applications, and on
fostering novel and innovative research projects that depend on the unique capabilities of
Blacklight. Examples of the latter include: assembling the Rhesus macaque genome; assembling
the Sorghum genome and other projects related to the iPlant Collaborative; model fitting and
optimization for machine translation and semantic parsing; and network analysis. Special effort
has been devoted to programming environments that have not been traditionally supported on
supercomputing systems, such as Java, Python and R, as well as to the deployment of UPC and of
code performance analysis tools.
In support of Jacobo Bielak (Civil and Environmental Engineering, Carnegie Mellon University),
John Urbanic (PSC) developed analysis procedures using MATLAB to quantify differences
produced by different algorithms and their implementations, for example between 64-bit and 32-
bit floating-point versions, in the Hercules seismology code. Urbanic is now completing the
production version of the single/double precision selectable code, along with updating the rupture
code and addressing coordinate transforms that other groups are using.
There are a number of ongoing support efforts by the Purdue RP staff to help current and
potential TG users. They worked with Brian Henry (potential TeraGrid user – University of
Illinois - Chicago) to investigate which system to use for access to Fluent as well as if it would be
possible to run his Delphi code on a Windows environment via the Condor Pool. They assisted
Charles Moseley, Southwestern Oklahoma State University, looking into how to run specific
software on the Condor Pool under Linux instead of the initial request of running on Windows;
40

helped Robert Coridan (UIUC) to explore wehteher he could install and use Lumerical software
on TeraGrid machines.
SDSC staff worked on developing software stack, documentation, and user environment for the
new Trestles resource. Extensive benchmarking was performed for several applications including
AMBER, NAMD, ABAQUS, Velvet, SOAPdenovo, and ABySS. User services staff also ran
synthetic benchmarks like HPCC on the system. The high performance linpack (HPL) result was
submitted to the top 500 list (Trestles is at #111) and also used to compute the SU conversion
factors with other TG machines (TG documentation has been updated with this information).
SDSC staff worked with early users to port their codes and run jobs in the friendly user phase of
the operations. SDSC user services also assisted in the integration of the lustre parallel filesystem
(Data Oasis) and conducted extensive I/O testing and benchmarking from Trestles. SDSC staff
worked to develop a framework to run Hadoop via the PBS scheduler, and using persistent global
storage in combination with local SSD storage on Dash. This setup is currently being tested by
friendly users. SDSC hosted the “Grand Challenges in Data-Intensive Discovery” conference in
October, 2010. As part of the conference, user services staff presented and participated in a panel
session detailing early user experiences with the Dash system.
Under the User Champions program (section 5.2), TACC staff contacted 68 PIs with new or
renewal awards granted for the allocation period beginning October 1, 2010. Responses were
received from 10 PIs and TACC staff members resolved any user issues or referred them to the
appropriate site for resolution. TACC also initiated contact with 87 “startup” principal
investigators whose allocations began in September, October, and November of 2010. Responses
were received from 24 PIs and TACC staff members resolved any user issues or referred them to
the appropriate site for resolution. TACC staff continued to facilitate special user requests,
including resource usage outside normal queue limits, high priority in order to meet deadlines,
and special allocation requests to prevent user research projects from coming to a halt.
5.4.2
Helpdesk
During this quarter, TG RPs provided local helpdesk first-tier support that integrates with the
TeraGrid Operations Center. This support works to provide rapid response email and phone
responses for questions and trouble reports from the user community. Helpdesk staff direct users’
queries to Consulting or Advanced user support in cases where second-tier or third-tier expertise
is required.
5.5 Advanced User Support
Advanced User Support (AUS) staff are located at the RP sites and are coordinated by the AUS
Area Director. The highest level of long-term user support for a TeraGrid user or user group is
provided by the AUS staff. The overall advanced support efforts are categorized by three subefforts:
Advanced Support for TeraGrid Applications (ASTA), Advanced Support for Projects
(ASP), and Advanced Support for EOT (ASEOT).
Operations of the AUS area is coordinated by the AUS Area Director jointly with the AUS Point
of Contacts (POC) who are in most cases the managers or group leaders of the scientific
computing applications groups at each of the RP sites. All the management and coordination
issues such as analyzing TRAC ASTA review reports, contacting PIs for further understanding of
ASTA plans, matching of appropriate AUS staff to ASTAs, discussing/initiating new ASPs,
matching of AUS staff to ASPs, discussing ASEOT topics, preparing program plan and annual
reports, etc., are handled by this group. They hold biweekly teleconferences for the management
and coordination issues and continue discussions via email as needed.
On alternate bi-weeks a technical tele/web-conference is hosted using ReadyTalk for all the AUS
technical staff as well as the AUS POCs. At this technical tele/web-conference 1-2 technical
41

presentations are made by AUS technical staff discussing progress of ASTA projects, progress of
ASPs, and any other technical issue that is of interest to all the AUS technical staff. ReadyTalk
allows all the AUS staff, attending these bi-weekly tele/web-conferences to view the slides and
listen and participate in the technical presentations. This fosters an environment for collaboration
among the AUS staff located at the various RP sites and also allows sharing of technical insight
that may have impact on multiple ASTAs or ASPs. The presentation slides and audio recordings
are archived and made publicly available for interested staff as well as users at:
http://teragridforum.org/mediawiki/index.phptitle=Technical_presentations .
5.5.1
Advanced Support for TeraGrid Applications (ASTA)
Advanced Support for TeraGrid Applications (ASTA) efforts allow AUS staff to work with a user
for a period of a few months to a year. Activities include porting applications, implementing
algorithmic enhancements, implementing parallel programming methods, incorporating math
libraries, improving the scalability of codes to higher core counts, optimizing codes to utilize
specific resources, enhancing scientific workflows, providing help for science gateways and
performing visualization and data analysis projects. To receive ASTA support, TeraGrid users
submit a request as a part of their annual, Supplemental, or Startup resource allocation proposal.
The recommendation score provided by the reviewers is taken into account and upon discussion
with the user regarding a well-defined ASTA workplan, AUS staff provide ASTA support to the
user. This support often impacts a larger number of users who are part of the PI’s team. Since late
2008, in addition to the quarterly TRAC allocations, ASTA support is also available to those
requesting Supplements or Startup allocations. The TeraGrid-wide AUS effort improves the
ability to optimally match AUS staff to an ASTA project by taking into account the reviewers’
recommendation score, the AUS staff(s) expertise in a domain science/HPC/CI, the ASTA project
workplan, and the RP site where the user has a resource allocation. In addition to providing longterm
benefits to the user or the user team, projects are also beneficial to the TeraGrid as a whole.
Results of ASTA projects provide insights and exemplars for the general TeraGrid user
community and these are included in documentation, training and outreach activities. In the table
5.3 we list 48 ASTA projects of which 7 were completed in this quarter. (It should be noted that
many of the ASTA projects are renewed yearly based on the recommendation provided by TRAC
and hence are not noted as completed at the end of an allocation period.) This list also includes
Startup and Supplemental ASTA projects. Quarterly updates of ASTA work for most of these
ASTA project are provided after the table.
ASTA Project
DNS of Spatially
Developing
Turbulent Boundary
Layers, PI Ferrante,
U. Washington
Multiscale Analysis
of Size Dependence
of Deformation and
Fracture of Hierarchy
Protein Materials, PI
Buehler, MIT
An
System
Table 5.3. List of continuing and completed ASTA projects for the quarter.
Earthquake
Science
Site(s)
Involved
SDSC, NCSA,
TACC, NICS
Status
Continuing
03/11
SDSC Completed 12/10
SDSC,
TACC,
PSC,
NICS,
Continuing
through
through
Notes
42

Approach to
Physics-based
Seismic Hazard
Research, PI Jordan,
USC
Modeling Studies of
Nano
and
Biomolecular
Systems, PI
Roitberg, U. Florida
Efficient
Implementation of
Novel MD Simulation
Methods in
Optimized MD
Codes, PI Voth, U.
Chicago
Flow and Nutrient
Transport in 3D
Porous Scaffolds
Used for Bone
Tissue Growth, PI
Papavassiliou, U.
Oklahoma
Biomechanics of
Stem Cells, PI Finol,
CMU
Global Kinetic
Simulations of the
Magnetosphere, PI
Karimabadi, UCSD
EpiSims, PI Roberts,
RTI
International;NekTar-
G2, PI Karniadakis,
Brown U.; GENIUS,
PI Coveney, UCL
Enabling Neutron
Science Research
with the Neutron
Science TeraGrid
Gateway, PI Cobb,
ORNL
Simulation of Liquid
Fuel Combustors, PI
Mashayek, UIC
Multichannel
Scattering
Theory
NCSA 06/11
SDSC,
NICS
PSC,
TACC
NCSA,
NICS,
Continuing
09/11
Continuing
09/11
through
through
PSC Continuing through
06/11
PSC Continuing through
09/11
SDSC, NICS Continuing through
06/11
NIU,
NCSA,
TACC,
Purdue
SDSC,
ANL,
IU,
Continuing
09/11
through
ORNL Continuing through
03/11
NCSA Continuing through
06/11
TACC Completed 09/10
43

via the Modified
Faddeev Equation,
PI Hu, CSULB
Turbulence
Simulations Towards
the PetaScale:
Intermittency,
Mixing, Reaction and
Stratification, PI
Yeung, Georgia
Tech
First-Principles
Molecular Dynamics
for Petascale
Computers, PI Gygi,
UCD
UNRES Force-field
for Simulation of
Large Molecular
Systems, PI
Scheraga, Cornell
Petascale Adaptive
Computational Fluid
Dynamics, PI
Jansen, RPI
Insight into
Biomolecular
Structure, Dynamics,
Interactions and
Energetics from
Simulation, PI
Cheatham, U. Utah
Large Eddy
Simulation of
Particle-Turbulence
Interactions in
Complex Flows, PI
Apte, Oregon State
Nonlinear Evolution
of the Universe, PI
Cen, Princeton
Modeling Global
Climate Variability
with the Multi-scale
Modeling
Framework: The
Boundary-layer
Cloud Problem, PI
NICS,
TACC
SDSC,
Completed 09/10
NICS, TACC Continuing through
12/10
PSC Completed 12/10
NICS,
PSC
TACC,
Completed 12/10
SDSC Continuing through
12/11
TACC Continuing through
03/11
TACC,
NICS
SDSC,
Continuing
09/11
through
Purdue, SDSC Continuing through
06/11
44

Helly, SIO/UCSD
Lattice Gauge
Calculation of
Hadronic Physics, PI
Liu, University of
Kentucky
Leveraging
Supercomputing for
Large-scale Gametheoretic
Analysis, PI
Sandholm, CMU
Astrophysical
Applications of
Numerical Relativity:
Coalescing Binary
Systems and Stellar
Collapse, PI
Schnetter, LSU
Fractals in Elasto-
Plasticity, PI Ostoja-
Starzewski, UIUC
The CIPRES
Science Gateway, PI
Miller, SDSC/UCSD
High-Resolution
Modeling of
Hydrodynamic
Experiments, PI
Demeler, U. Texas
Health Science
Center
Center for Integrated
Space Weather
Modeling, PI Quinn,
Boston U.
Computational
Modeling of Thermal
Stripping in Nuclear
Reactors, PI Kimber,
U. Pittsburgh
Simulation of
Complex Organic
Mixture
Biotransformations in
Natural Systems, PI
VanBriesen, CMU
Villous Motility as a
Critical Mechanism
PSC, NICS Continuing through
03/11
PSC Continuing through
03/11
PSC,
SDSC
NCSA,
Continuing
03/11
through
NCSA Continuing through
01/11
SDSC Continuing through
06/11
IU,
Purdue
PSC,
Continuing
03/11
NICS, TACC Completed 12/10
PSC Completed 12/10
through
PSC Continuing through
04/11
NICS, SDSC Continuing through
Also see Science
Gateways section of the
quarterly report
(UltraScan)
45

for Efficient Nutrient
Absorption in the
Small Intestine, PI
Brasseur, Penn
State
Systematics of
Nuclear Surface
Vibrations in
Deformed Nuclei, PI
Engel, U. North
Carolina
Simulations of
Molecular Clouds
and Protoplanetary
Disks, PI Mac Low,
American Museum
of Natural History
Structural and
Functional
Characterization of
Glu-Plasminogen
and
Lys-
Plasminogen, PI
Kim, CMU
Design
of
Communicating
Colonies of
Biomimetic
Microcapsules, PI
Kolmakov, Univ. of
Pittsburgh
Computational
Modeling of Ultracold
Molecular Collisions
and Reactions
Involving Clusters
and Nanoparticles,
PI Naduvalath,
UNLV
Development of
Novel HIV Entry
Inhibitors Using the
ROCS Shaped
Based Matching
Algorithm and
Molecular Dynamics
Studies of gp120
Envelop Proteins, PI
LaLonde, Bryn Mawr
03/11
NICS, TACC Continuing through
03/11
PSC Continuing through
03/11
PSC Continuing through
03/11
TACC,
NCSA
SDSC,
Continuing
03/11
through
NICS, PSC Continuing through
03/11
PSC Continuing through
03/11
46

College
Simulation and Data
Analysis of Macro-
Molecular Systems,
PI Blaisten-Barojas,
George Mason
University
Applications of Bispeptide
Nanostructures, PI
Schafmeister,
Temple University
Finite-elements
Modeling of Singleand
Poly-Crystalline
Material Removal
Including
Crystallographic
Effects,
PI
Ozdoganlar, CMU
Modeling Geological
Sequestration of
Carbon Dioxide in a
Deep Saline
Reservoir Using the
TOUGH2 Simulator,
PI Valocchi, UIUC
Multi-Scale and
Isotropy in Rotating
Turbulence, PI
Pouquet, NCAR
Variational Approach
to Reservoir
Stimulation for
Enhanced
Geothermal
Systems, PI Bourdin,
LSU
DREAM Assessment
Using Octave Open
Source Software, PI
Surfleet, Oregon
State University
Cyberinfrastructure
to Support Science
and
Data
Management of the
Dark Energy Survey,
PSC Continuing through
03/11
NICS Continuing through
04/11
PSC Continuing through
03/11
NCSA Continuing through
03/11
PSC, NICS Continuing through
06/11
TACC Continuing through
06/11
Purdue Continuing through
06/11
NCSA, Purdue Continuing through
06/11
47

PI Mohr, UIUC
Kinetic Simulations
of Collisional
Inonospheric Waves
and Turbulence, PI
Oppenheim, Boston
Univ.
Research Allocation
for the Large
Synoptic Survey
Telescope Data
Challenge 3b, PI
Axelrod, University
of Arizona
Numerical Modeling
of Double-diffuive
Convection, PI
Radko, Naval
Postgraduate School
Determination of the
Conformation of
Polypeptides in
Solution as well as
on a Surface Using
Molecular Dynamics
Simulation, PI
Narsimhan, Purdue
University
NICS Continuing through
06/11
NCSA,
TACC
PSC,
Continuing
08/11
through
TACC Continuing through
09/11
SDSC Continuing through
10/11
PI: Ferrante (U Washington, Fluids, Particulate and Hydraulic Systems). DNS of Spatially
Developing Turbulent Boundary Layers. Continuing through 03/11. The ASTA team for this
project consists of Darren Adams (NCSA), David Bock (NCSA), John Peterson (TACC), Lonnie
Crosby (NICS) and Dmitry Pekurovsky (SDSC).
In this quarter David Bock has been integrating a new data reader, based on the the new HDF5
library module created by Darren Adams for this PI, into visualization rendering system. Jay
Alameda (PI of NSF SI2 funded project on Eclipse) has identified ways of using the Eclipse
parallel tool to benefit some of the issues the ASTA PI is interested in. A staff (Al Rossi from
NCSA), funded from the SI2 grant, is participating in this project jointly with Adams in this
regard.
PI: Buehler (MIT, Mechanics and Materials). Multiscale Analysis of Size Dependence of
Deformation and Fracture of Hierarchy Protein Materials. Completed 12/10. The project is lead
by Ross Walker (SDSC). There is no update for this quarter and the ASTA was completed.
PI: Jordan (University of Southern California, Earth Sciences). SCEC PetaScale Research: An
Earthquake System Science Approach to Physics-based Seismic Hazard Research. Continuing
through 06/11. The ASTA team consists of Yifeng Cui (SDSC), John Urbanic (PSC), Byoung-Do
Kim (TACC), Kwai Wong (NICS), Mark Vanmoer (NCSA), and Amit Chourasia (SDSC).
48

During this quarter Cui prepared and performed a two-day data-intensive 1-Hz M8 simulation
using 30K Kraken cores and this generated 120 TB volume data for analysis for the visualization
tool GlyphSea. Optimization of MPI-I/O scheme was done on upgraded Cray XT5. Cui
contributed significantly to the slides presented at the Gordon Bell finalist presentation by PI Tom
Jordan. Cui and Chourasia were co-authors of the Gordon Bell finalist SC10 paper.
Chourasia continued extensive visualization work for M8 simulations. Several animations were
created for analysis and presentation at the SC10 conference Gordon Bell and Master Works talks
by PI Tom Jordan. The visualization were performed on the Jaguar, Kraken and Nautilus
machines. Movies and images are available from:
http://visservices.sdsc.edu/projects/scec/m8/1.0/
Preliminary building response visualizations were also created in collaboration with SCEC and
CalTech researchers.
Urbanic continued substantial optimizations regarding single and double precision switches for
the Hercules code this quarter.
Figure 5.1 Instantaneous X velocity component from M8 simulations (every 100 th step)
(Visualization by Amit Chourasia SDSC).
49

Figure 5.2 Instantaneous Y velocity component from M8 simulations (every 100 th step)
(Visualization by Amit Chourasia, SDSC).
PI: Roitberg (U. Florida, Biophysics). Modeling Studies of Nano and Biomolecular Systems.
Continuing through 09/11. The ASTA team consists of Ross Walker (SDSC) and Christian
Halloy (NICS).
During this quarter Walker has worked on further improving the PMEMD GPU implementation
in AMBER to improve performance by between 5 and 10%. He removed the dependencies on the
external CUDPP library for carrying out RadixSorts which improves stability and works around
bugs found in several NVIDIA compiler versions. These updates were released as part of
bugfix.12 for AMBER and are being used directly by the PI. Specifically the problems addressed
are as follows:
1. Removes the dependence on CUDPP which could cause crashes on certain large
simulations due to bugs in the CUDPP library.
2. Fixes problems with large simulations, 400K+ atoms crashing randomly with an
allocation failure.
3. Fixes "ERROR: max pairlist cutoff must be less than unit cell max sphere radius!" bug
allowing cutoffs larger than 8.0 to be used for both small and large systems.
4. Writes final performance info the the mdout file.
5. Possible (not fully tested) workaround for NVIDIA GTX4XX and GTX5XX series cards
to avoid possible random hangs during PME simulations.
6. Tests for the use of non-power of 2 MPI tasks when running parallel GPU PME
calculations. Update so that the code now quits with a suitable error message.
50

7. Minor performance improvements, mostly due to the use of a faster radixsort.
Walker also made arrangements for Prof. Roitberg's graduate student to spend two months from
the middle of March onwards working with him directly at SDSC. Prof. Roitberg will be covering
the student’s expenses.
PI: Voth (University of Chicago, Physical Chemistry). Efficient implementation of novel MD
simulation methods in optimized MD codes. Continuing through 09/11. The ASTA team consists
of Phil Blood (PSC), John Peterson (TACC), and Lonnie Crosby (NICS).
In this quarter Blood has continued to work with the Voth group to find the best strategy for
improving parallel performance of their multistate empirical valence bond method for modeling
proton transport in classical molecular dynamics simulations. In past quarters the group
implemented their method in LAMMPS, but the implementation suffers from severe scaling
challenges due to the many all-to-all communications required each time step. In previous
quarters Blood ported their implementation to a new hybrid OpenMP/MPI LAMMPS code that
reduces some communication bottlenecks caused by the all-to-all communication.
In this quarter, the Voth group released a new version of their LAMMPS-MS-EVB code with
increased functionality, which Blood then ported to the faster hybrid LAMMPS code and ran
several benchmarks to test performance of the new code. This code matched the performance of
the previous hybrid LAMMPS-MS-EVB code. To explore other ways to improve performance,
members of the Voth group tried a new multiple-program implementation of the regular
LAMMPS code (without MS-EVB), with the calculations requiring all-to-all communication
running only on a small subset of the available processors. Blood performed in depth
benchmarking and performance analysis of this new implementation. Through these tests and
discussions with the Voth group it was found that the performance of this code could be improved
by careful placement of the two types of processes (processes with or without all-to-all) at
runtime. Benchmark tests showed that the new multiple-program code with careful process
placement outperformed the previous hybrid approach (see Figure 5.3). In addition, the
performance analysis demonstrated that the remaining scaling issues were due to load imbalance
between the two types of process groups. Through ongoing discussions with the Voth group
Blood has identified possible ways to further optimize these calculations and reduce load
imbalance. These optimizations, as well as porting MS-EVB to the multiple-program LAMMPS
code, will be pursued in subsequent quarters.
51

Figure 5.3 Speedup plot of ns/day achieved with various implementation of the LAMMPS
code.(Plot by Phil Blood, PSC).
PI: Papavassiliou (U. Oklahoma, Chemical, Thermal Systems). Investigation of Flow and
Nutrient Transport in Porous Scaffolds Used for Bone Tissue Growth. Continued through 06/11.
ASTA staff for this project is Raghu Reddy (PSC). There is no update on this project for this
quarter.
PI: Finol (CMU, Mechanical and Biomedical Engineering). Biomechanics of Stem Cells.
Continuing through 09/11. The ASTA consultant is Anirban Jana (PSC). One of Prof Finol's
PhD students, Samarth Raut, successfully presented his PhD proposal to his advisory committee
which includes Jana. All the work on Pople with ADINA was crucial to formulation of his PhD
proposal. Jana also pointed out faster ways to transfer simulation results from Pople to the group's
local machines, using multiple scp processes. This sped up transfers significantly from what some
members of his group was achieving before (e.g. transfers that took 1-2 days before, now take a
few hours).
PI: Karimabadi (UCSD, Magnetospheric Physics). Global Kinetic Simulations of the
Magnetosphere. Continuing through 06/11. The ASTA staff for this project are Mahidhar
Tatineni (SDSC) and Glenn Brook (NICS). Gleen Brook and Lonnie Crosby (NICS) continued to
discuss I/O study with the user Burlen from the PI’s team in this quarter.
PI: Roberts (RTI International, Computer Science) EpiSims; PI: Karniadakis (Brown U., CFD)
NekTar-G2; PI Coveney (University College of London, Chemistry) GENIUS. Continuing
through 09/11. The lead, regarding MPIg research and implementation, for these ASTA projects
is Nick Karonis (NIU, ANL). He has continued to work with Karniadakis and Grinberg and
have identified key portions of their application that are computational bottlenecks in the
application that might also be good candidates to introduce multithreading. Under Dr.
Gringberg's (from Karniadakis group) advice, Karonis started working on the application's preconditioner
and have made significant progress there introducing multithreading through
52

OpenMP. Plan is to continue the efforts to introduce multithreading, through OpenMP, through
the rest of the application as directed by Dr. Grinberg and computational bottleneck analysis.
PI: Cobb (ORNL, Neutron Science). Enabling Neutron Science Research with the Neutron
Science TeraGrid Gateway. Continuing through 03/11. Cobb’s team is providing their own
ASTA support for this project and continued to deploy software tools and work processes to
leverage experimental neutron science.
PI: Mashayek (Univ. of Illinois Chicago, Applied Mathematics). Simulation of Liquid Fuel
Combustors, Continuing through 06/11. ASTA staff for this project is Ahmed Taha (NCSA). The
Simulations using Ansys-Fluent are in progress. Results are being analyzed, processed and
communicated to the PI. A modified configuration will be used and simulations will be resumed.
PI: Gygi (UCD, Materials Science). First-Principles Molecular Dynamics for Petascale
Computers. Continuing through 12/11. The ASTA staff for this project are BD Kim (TACC) and
Bilel Hadri (NICS). Hadri help PI’s team to use OpenMP. As a result PI’s team did initial
assessment of performance with partial user of OpenMP in a limited set of functions. They
observed that they can achieve about the same performance using MPI only and a mixed
MPI/OpenMP approach, and in some cases they can obtain 5-10% speedup with the
MPI/OpenMP version. This is encouraging since there are many more opportunities for thread
parallelism in the rest of the code, and the team is pursuing that.
PI: Scheraga (Cornell U., Biophysics). UNRES Force-field for Simulation of Large Molecular
Systems. Completed 12/10. The ASTA team consists of Phil Blood and Mahin Mahmoodi (PSC).
Not much interaction occurred with PI’s team this quarter, although the PI’s team expressed
interest to continue work with the ASTA team. Based on the work done earlier the following
paper was accepted and published:
A. Liwo, S. OÅ^Âdziej, C. Czaplewski, D. S. Kleinerman, P. Blood and H. A. Scheraga, J.
Chem. Theory Comput., 2010, 6:890-909.
PI: Jansen (RPI, Fluids). Petascale Adaptive Computational Fluid Dynamics. Completed 12/10.
ASTA team for this project consists of Glenn Brook (NICS), David O’Neal (PSC), and John
Peterson (TACC). There is no major update for this quarter.
PI: Cheatham (U. Utah, Biochemistry). Insight into Biomolecular Structure, Dynamics,
Interaction and Energetics from Simulation. Continuing through 12/10. The ASTA team consists
of Ross Walker (SDSC). Over the last 3 months Walker has worked with Prof. Cheatham under
his ASTA project to provide finalized benchmark information to Prof. Cheatham for his use in
LRAC requests. Walker has also worked to help improve the I/O performance of the parallel
analysis package ptraj. As part of this he has worked with Prof. Cheatham and collaborators to
begin developing a C++ version of this package in order to make future improvements and
feature additions easier.
PI: Apte (Oregon State University, Fluid, Particulate, and Hydraulic Systems). Large Eddy
Simulation of Particle-Turbulence Interactions in Complex Flows. Continuing through 03/11.
The ASTA staff for this project is Carlos Rosales (TACC). No interaction with the PI’s team
happened in this quarter.
PI: Cen (Princeton University, Astrophysics and Cosmology). Nonlinear Evolution of the
Universe, Continuing through 09/11. The ASTA team consists of Yaakoub Kharma (TACC),
Bilel Hadri (NICS) and Amit Chourasia (SDSC). There is no update for this quarter. Chourasia
met with the PI at the Supercomputing10 conference and discussed future ASTA effort.
PI: Helly (UCSD, SIO, Atmospheric Sciences). Modeling Global Climate Variability with the
Multi-scale Modeling Framework New parameterizations of Cloud Micro-physics and
53

Developing Community Community Accounts Portal for Running MMF. Continuing through
06/11. Phil Cheeseman (Purdue), Adam Jundt (SDSC) and Mahidhar Tatineni (SDSC) are the
ASTA team for this project. AUS and Gateways staff from multiple TeraGrid sites are working to
get remote job submission and data movement pieces of the project working.
PI: Liu (University of Kentucky, Elementary Particle Physics). Lattice Gauge Calculation of
Hadronic Physics. Continuing through 03/11. The ASTA team consists of Raghu Reddy (PSC)
and Haihang You (NICS). You worked with the PI’s team and helped them with account issues
and in submitting jobs.
PI: Sandholm (CMU, Computer and Computation Research). Leveraging Supercomputing for
Large-scale Game-theoretic Analysis. Continuing through 03/11. Joel Welling and David O’Neal
(PSC) are working with the PI on this project. No update is available for this quarter.
PI: Schnetter (LSU, Gravitational Physics). Astrophysical Applications in Numerical Relativity:
Coalescing Binary Systems and Stellar Collapse. Continuing through 03/11. The ASTA team
consists of Mahin Mahmoodi (PSC), Rick Kufrin and Rui Liu (NCSA), and Wayne Pfeiffer
(SDSC). There is no update on this project for this quarter.
PI: Ostoja-Starzewski (UIUC, Mechanics and Materials) Fractals in Elasto-Plasticity.
Continuing through 01/11. The ASTA staff for this project is Seid Koric (NCSA). Following
updates are provided for this quarter and includes work done by the graduate student with Koric:
1. Size 80 3 v.s. size 100 3 : Done. Verified 80 3 is reliable estimation.
2. Displacement BC v.s. Traction BC: Done. Verified boundary conditions independence and
RVE.
3. Study on different randomness configurations: Done. Three aspects: Gaussian or Uniform
distribution types; Different random variants; Randomness on elastic moduli and/or yield limit.
4. Study on different material hardening configurations: Done. Comparison of six materials to
show hardening effects.
5. Study on different loading configurations: In progress. Study of yield surface (whether
associated flow rule holds) needs at least five more simulations.
PI: Miller (SDSC/UCSD, System and Population Biology) The CIPRES Science Gateway.
Continuing through 06/11. ASTA support is being provided by Wayne Pfeiffer (SDSC).
A paper titled "Creating the CIPRES Science Gateway for Inference of Large Phylogenetic
Trees" was presented at the GCE10 workshop in conjunction with SC10. The authors are Mark
A. Miller, Wayne Pfeiffer, and Terri Schwartz, all of SDSC.
Pfeiffer installed the MrBayes and RAxML phylogenetics codes on Trestles, the newest
supercomputer at SDSC. He also helped Adam Jundt of SDSC make these codes accessible via
modules.
Benchmark runs showed that, for the same core count, MrBayes and RAxML run slightly faster
on Trestles than on Abe at NCSA, where most CIPRES gateway runs currently execute.
However, the increased availability of computer time on Trestles suggests the use of more cores
for some MrBayes and RAxML runs on Trestles than were used on Abe. Such runs will execute
appreciably faster on Trestles than before, albeit at lower parallel efficiency. In addition, runs of
as long as two weeks are allowed on Trestles versus one week on Abe. Thus most gateway runs
of MrBayes and RAxML will be moved from Abe to Trestles.
To help users with special requirements, expedited phylogenetic analyses were manually run on
Dash and Trestles at SDSC. Specifically, MrBayes jobs were run on Dash for
54

• Joel Ledford of the California Academy of Sciences and UC Berkeley,
and RAxML jobs were run on Trestles for
• Joel Guenther of UC Berkeley.
The parallel version of the MAFFT multiple sequence alignment code was installed on Abe,
Dash, and Trestles. Parallelization is via Pthreads and gives a parallel efficiency that is fairly
high at 8 threads, but much lower above that.
Benchmark runs on 8 cores gave best run times that were
1.89x slower on Abe than Dash and
1.05x slower on Trestles than Dash.
The times vary appreciably from run to run because the parallel search algorithm uses random
numbers in a way that is not reproducible. Thus the number of iterations and even the final
alignment vary from run to run, though the resulting alignments seem to be of similar quality.
In light of the preceding results, it is planned to make CIPRES gateway runs of MAFFT on
Trestles. Currently such runs are made with a serial version of MAFFT on Triton, a non-
TeraGrid computer at SDSC.
PI: Demeler (U. Texas Health Science Center, Biochemistry and Molecular Structure and
Function) High-Resolution Modeling of Hydrodynamics Experiments. Continuing through 03/11.
ASTA support is provided by Suresh Marru, Raminder Singh (IU), Roberto Gomez (PSC), and
Phil Cheeseman (Purdue). The ASTA team consists of staff from both the Science Gateway area
and the Advanced User Support area since the PI’s project requires advanced support for
gateways as well as MPI optimization. Further description of activities is provided in the Science
Gateways section.
PI: Quinn (Boston University, Solar Terrestrial Research). Center for Integrated Space Weather
Modeling. Continuing through 12/10. The ASTA staff for this project are Justin Whitt, Glenn
Brook (NICS), and Carlos Rosales-Fernandez (TACC). The ASTA completed in 12/10. Glenn
Brook and Justin White have continued to advise the group via bi-weekly teleconferences. They
began examining the performance characteristics of their latest skeletal communications model
and verified the gains realized by the message buffering previously suggested by Glenn Brook.
They also examined the effects of different MPI rank orderings and the use of the MPI memory
copy optimization on the current model and saw no significant improvements. They completed
weak and strong scaling studies of the new communications model; as well as, compared the
performance on multiple architectures and determined the new communications model to be
bound by both latency and contention for network resources. This led them to suggest an
alternate communications stencil requiring less messages that were larger in size to be sent.
Work by the Quinn team continues toward this end.
PI: Kimber (University of Pittsburgh, Thermal Systems). Computational Modeling of Thermal
Striping in Nuclear Reactors. Completed 12/10. Anirban Jana from PSC is the ASTA staff for
this project. On the turbulence modeling and thermal striping of jets, the realizable k-epsilon
model was found to be a good candidate turbulence model for the jets, for isothermal conditions.
FLUENT simulations with this model provided close agreement with experimentally measured
axial velocity decay. Currently non-isothermal jet validations are being pursued.
On LBM, several critical physics related to boiling, namely liquid-vapor equilibrium, surface
tension of a vapor bubble, establishment of an exponential density gradient in a fluid due to
gravity, establishment of a horizontal interface between liquid and and vapor under gravity were
55

demonstrated (with rigorous quantitative validation using theoretical formulas). LBM runs have
been primarily using up Pople SUs for this project.
PI: VanBriesen (CMU, Chemical and Reaction Processes). Simulation of Complex Organic
Mixture Biotransformations in Natural Systems. Continuing through 04/11. Anirban Jana from
PSC is collaborating on this ASTA project. The graduate student, Amanda Hughes, was able to
defend her PhD last quarter, based on her Star-P runs on Pople. She is still working on finishing
up papers. Jana continued to answer 1-2 small queries from her last quarter.
PI: Brasseur (Penn State, BioPhysics).Villous Motility as a Critical Mechanism for Efficient
Nutrient Absorption in the Small Intestine. Continuing through 03/11. Lonnie Crosby (NICS), for
computational work, and Amit Chourasia (SDSC), for visualization work, are involved in this
ASTA project. Chourasia started preliminary work on visualization of Intestine3D. Chourasia
also met the PI in person to understand the data and things to look for in the visualization. The PI
found interesting features at first look from the visualization and this is contrary to their original
hypothesis.
PI. Engel (U. North Carolina, Physics). Systematics of Nuclear Surface Vibrations in Deformed
Nuclei. Continuing through 03/11. ASTA staff for this project are Meng-Shiou Wu (NICS) and
Victor Eijkhout (TACC). Wu’s major effort in the fourth quarter of the project was to assist the
team solved a scalability issue in their code. The original request from the team is as follow:
Test calculation using 80k cores: The purpose of this test was to investigate the efficiency of the
part of our calculation that is difficult to parallelize and time consuming: reading 0.4 GB of input
data into a single core and distributing the data to 80k cores. The test had not ended after three
hours.
After looking into their code, it was found that the performance issue came from their file
vbc1.17k_t.f90, line 650 – 782. The code segment reads a file of about 550MB and distributes the
data to 80K cores.
The code segment appeared to have been written to run on machines of smaller scale, with less
physical memory available. It reads a section of the data file, broadcast it to the rest of the nodes
(cores) sequentially, then read the next section, and repeat the ‘broadcast’. This process was
repeated 4533 times.
Once the cause of performance issue was determined, it was not difficult to come up with a
solution for this broadcast problem. While the data is of the size 550M, those are real numbers
stored in text format; thus after reading the file, the exact memory required for each core is
128MB, and this amount of memory for every single core is not a problem on Kraken. The
process that was repeated 4533 times can be finished in one step.
The original program reads a segment of data, copies it to another buffer then broadcast it. This
extra copy was removed and replaced with a larger buffer and with a single read, and the buffer is
used for broadcast with Cray’s MPI_Bcast which is optimized to run on SeaStar2.
The modified code shows significant performance improvement over the original code as shown
in Figure 5.4. The figure shows performance comparison of repeated send/receive and the result
of a single MPI_Bcast of a large buffer. No I/O was involved. The tests for performance
comparison used up to 24,000 cores for the modified code (30 seconds, not shown in Figure 5.4),
which is still better than the original code when using 2,400 cores (470 seconds).
Figure 5.5 shows performance comparison after I/O mechanisms was added. The first I/O
mechanism is the original method that reads a section of data repeatedly. The second I/O pattern
reads the whole file once into a single buffer.
56

The team also mentioned that they had trouble verifying the result of broadcast. An example
verification procedure using extra I/O was added and was shown to the team. Instructions of
using performance tools to collect performance data and conduct analyses for their code were also
given to the team.
While the solution of this performance is not complex, the team does not seem to have
experience of optimization; thus many instructions were prepared and were given to the team,
include: 1). A set of example programs (base on their code structure) to modify their code step by
step. 2). A power point file with more than 10 slides showing how the problem was solved, step
by step. 3). Performance data, and instruction of how those performance data were
collected/generated.
Figure 5.4 Performance comparison of repeated send/recv and single MPI_Bcast – time in secs
(Y axis) Vs # of cores (X axis) (Plot by Meng-Shiou Wu, NICS)
57

Figure 5.5 Performance comparison with I/O mechanisms - time in secs (Y axis) Vs # of cores (X
axis) (Plot by Meng-Shiou Wu, NICS)
PI. Mac Low (American Museum of Natural History, Planetary Astronomy). Simulations of
Molecular Clouds and Protoplanetary Disks. Continuing through 03/11. Roberto Gomez (PSC) is
working the PI’s team for this ASTA project. No update available for this quarter.
PI. Kim (CMU, Biochemistry and Molecular Structure and Function). Structural and Functional
Characterization of Glu-Plasminogen and Lys-Plasminogen. Continuing through 03/11. ASTA
effort is provided by Phil Blood (PSC). In this quarter the group started exploring some of the
accelerated sampling methods that Blood suggested to the team the previous quarter.
PI. Kolmakov (Univ. of Pittsburgh, Materials Research).Design of Communicating Colonies of
Biomimetic Microcapsules. Continuing through 03/11. Carlos Rosales-Fernandez (TACC), Jay
Alameda (NCSA), and Dongju Choi (SDSC) are providing ASTA support for this project. No
update available for this quarter.
PI. Naduvalath (UNLV, Chemistry).Computational Modeling of Ultracold Molecular Collisions
and Reactions Involving Clusters and Nanoparticles. Continuing through 03/11. ASTA effort is
provided by Bilel Hadri (NICS) and Mahin Mahmoodi (PSC). There has been no interaction from
the PI’s side this quarter.
PI. LaLonde (Bryn Mawr College, Biochemistry and Molecular Structure and Function).
Development of Novel HIV Entry Inhibitors Using the ROCS Shaped Based Matching Algorithm
and Molecular Dynamics Studies of gp120 Envelop Proteins. Continuing through 03/11. ASTA
staff of this project is Phil Blood (PSC). Currently PI is successfully continuing runs on the
warhole machine at PSC.
PI. Blaisten-Barojas (George Mason University, Physical Chemistry). Simulation and Data
Analysis of Macro-Molecular Systems. Continuing through 03/11. ASTA staff for this project is
Yang Wang (PSC). The user was ready to port the code to Lincoln. Wang provided the PI with
58

the makefile for compiling the source code and creating the executable. It is expected that the PI
would be able to run the code and produce science results.
PI. Schafmeister (Temple University, Organic and Macromolecular Chemistry). Applications of
Bis-peptide Nanostructures. Continuing through 04/11. ASTA staff for this project is Bilel Hadri
(NICS). There is no update for this quarter.
PI. Ozdoganlar (CMU, Manufacturing Processes and Equipment).Finite-Elements Modeling of
Single- and Poly-Crystalline Material Removal Including Crystallographic Effects. Continuing
through 03/11. ASTA staff for this project is Anirban Jana (PSC). There was one instance when
the MPI version of ABAQUS didn't seem to be working on Pople and Jana helped the graduate
student to resolve this.
PI. Valocchi (UIUC, Earth Sciences). Modeling Geological Sequestration of Carbon Dioxide in a
Deep Saline Reservoir Using the TOUGH2 Simulator. Continuing through 03/11. David Bock
(NCSA) is providing ASTA support for the project and there is no update for this quarter.
PI. Pouquet (NCAR, Physics).Multi-Scaling and Isotropy in Rotating Turbulence. Continuing
through 06/11. ASTA staff for this project are Raghu Reddy (PSC), Mahin Mahmoodi (PSC), and
Christian Halloy (NICS). Reddy’s work with this group on the hybrid code resulted in following
presentations/publications:
1. AGU presentation (poster): 'A hybrid MPI-OpenMP scheme for scalable parallel
pseudospectral computations for fluid turbulence', AGU 2010 (this paper also served as one of the
bases for a session that was convened called 'Large Scale Geosciences Applications Using GPU
and Multicore Architectures' sponsored by Earth and Space Science Informatics and co-sponsored
by Nonlinear Geophysics and Planetary Sciences sections of AGU; this paper was the basis of the
'multicore' part).
2. The Teragrid '10 presentation: 'Investigation of performance of a hybrid MPI-OpenMP
turbulence code', Aug 2010; winner of Best Paper, Technology Track.
PI Bourdin (LSU, Applied Mathematics). Variational Approach to Reservoir Stimulation for
Enhanced Geothermal Systems. Continuing through 06/11. ASTA staff for this project is
Yaakoub El Khamra (TACC). In this quarter Kharma helped with C and compilation questions
and setup a storage space. He also taught the group about existing profiling tools.
PI. Surfleet (Oregon State Univ., Engineering Systems). DREAM Assessment Using Octave Open
Source Software. Continuing through 06/11. ASTA staff for this project is Phil Cheeseman
(Purdue). PI’s team was provided some help regarding usage of scratch space.
PI. Mohr (UIUC, Astronomical Sciences). Cyberinfrastructure to Support Science and Data
Management for the Dark Energy Survey. Continuing through 06/11. Jay Alameda (NCSA) and
Phil Cheeseman (Purdue) are the ASTA staff for this project. There has been no significant
interaction with the PI’s team this quarter.
PI. Oppenheim (Boston University, Aeronomy). Kinetic Simulations of Collisional Ionospheric
Waves and the Turbulence. Continuing through 06/11. ASTA staff for this project is Haihang
You from NICS. In this quarter scaling studies were done using PI’s code called EPPIC, or the
Electrostatic Parallel Particle-in-Cell simulator. Below is a quick summary of the scaling tests.
Job 1 is the job done on Kraken. Job 2 is 2x larger in volume and number of particles (so
particle/volume is the same) and it scales fairly well.
59

Machine # Processors Domains Grids per domain # e- per proc. Wall Clock(sec)
Kraken 1024 128 8x512x512 262144 31441
Kraken 4096 512 4x1024x1024 524288 67115
PI. Axelrod (Univ. of Arizona, Physics). Research Allocation for the Large Synoptic Survey
Telescope Data Challenge 3b. Continuing through 08/11. Darren Adams (NCSA), Raghu Reddy
(PSC), and Chris Jordan (TACC) are providing ASTA support for this PI. There is no update this
quarter for this project.
PI.Radko (Naval Postgraduate School, Physical Oceanography) Numerical Modeling of Doublediffusive
Convection. Continuing through 09/11. ASTA staff for this project is Gabriele Jost.
The goal of the AUS is to increase the performance and scalability of Prof. Radko's 2D and 3D
double diffusion convection (DDC) implementations TILT2D and TILT3D. The codes are
currently parallelized base on OpenMP, which limits their scalability on SMP clusters to 1 node
(eg 16 threads on Ranger).
The ASTA staff and PI’s team are pursuing 3 possibilities:
1. Develop an MPI implementation of TILT3D
2. Investigate the use of GPGPUs (eg on Longhorn at TACC) for TILT2D
3. Explore the use of very large scale SMP nodes (eg. SGI Altix UV)
During the first quarter they focused the development of an MPI based implementation of
TILT3D. The work items below were performed.
Item 1: Planning and discussing a strategy with Prof. Radko
=============================================================
The application consists of 2 main programs. One program is used to generate initial input data
for numerical experiments, the other program performs the DDC calculations. It was decided to
use a 1D domain decomposition of the 3D arrays in the 3rd dimension.
The code to generate the input data currently generates a single binary file with input data. For the
distributed version they decided to keep this approach. Each MPI process will read its part of the
input data from the binary file.
The code DDC code uses FFTW to perform 3D real-to-complex FFTs. In order to perform the
FFT on distributed data the structure of the code will be changed to
1. perform 2D FFT, using FFTW real-to-complex in the xy-plane
2. use MPI_All_to_all to redistribute the data
3. perform a 1D complex-to-complex FFT in z dimension
4. use MPI_All_to_all to obtain the original distribution in z-dimension
Item 2: Start implementation
============================
Change the input generation code from sequential into an MPI code. Input generation was
changed to allow direct access to the binary file. This is to support the reading of multiple MPI
processes from different locations of the files.
Test the code for running on 1 MPI process.
Implement several test cases for performing a 3D FFTW in 2 steps: 2D real-to-complex (R2C) in
xy, 1D complex-to-complex (C2C) in z dimension.
60

Implement test cases for MPI based versions of the restructured FFTW.
Change the original TILT3D into an MPI program, currently running on only 1 MPI process.
Item 3: Miscellaneous
======================
Ported TILT3D and TILT3D to Ranger, tested execution of the OpenMP based implementation.
Plans for next quarter:
=======================
- Continue working on MPI implementation of TILT3D
- Port TILT2D code to ember (SGI Altix UV at NCSA) to explore scalability of OpenMP for a
large number of threads
PI.Narsimhan (Purdue University, Molecular Biosciences) Determination of the Conformation of
Polypeptides in Solution as well as on a Surface Using Molecular Dynamics Simulation.
Continuing through 10/11. The ASTA staff for this project is Ross Walker (SDSC) and Kimberly
Dillman (Purdue). Walker has been interacting with Hector Chang (student of the PI) and
Kimberly Dillman to initially characterize problems that Hector has been having with running
their 600,000+ atom system using PMEMD on multiple TeraGrid systems. The initial
conclusions are that it may be an issue with the input topology file rather than a bug in the
dynamics code itself. Walker is waiting on receiving copies of the input files in order to debug
this issue further. Once this is done they will proceed to optimizing performance for the PI's
specific problem set.
5.5.2
Advanced Support for Projects (ASP)
Under the ASP sub-area, AUS staff work on projects that directly benefit a large number of
TeraGrid users, as opposed to the ASTA projects for which AUS staff work, following the
guidance of the TRAC review process, with an individual user on his/her applications such that
TeraGrid resources can be used effectively by the user. The different categories of ASPs are:
• Installation and maintenance of domain science codes and software as well as HPC and
CI related software and tools on suitable HPC machines located at the RP sites
• Projects, decided in collaboration with users, that have the potential to impact large
number of users
• Creation of a software and associated database to monitor libraries being used by users
• Implementation and testing of software allowing cross-site MPIg based simulations to
run and associated interaction with cloud computing
Among these four categories, the second one involves more effort, from AUS staff, than the other
three.
5.5.2.1
Installation and maintenance of domain science codes and software as well as HPC
and CI related software and tools on all the HPC machines located at the RP sites
For this category of ASP tasks, AUS staff from all the RP sites continued to participate in this
effort. This is considered a foundation type work that is performed in an ongoing basis and is
needed for TeraGrid users who routinely use these codes and software for their science
simulations. Many of these codes and software are also needed and used as a part of various
ASTA projects by AUS staff. Examples of these codes and software include scientific
applications in domains sciences such as chemistry, biochemistry, materials science, engineering,
numerical mathematics, visualization and data, and in HPC/CI. AUS effort continued in this
quarter involving maintenance and proper installation of these codes and software, optimization
61

of the software for an architecture, debugging the software, interaction with and training for users
regarding usage of these codes and software. This is carried out by AUS staff that have Ph.D. and
M.S. level expertise in specific domain sciences and computer science.
5.5.2.2
Projects, decided in collaboration with users, that have the potential to impact large
number of users
Under the second category of ASP efforts, AUS staff work on projects that have the potential to
benefit a large number of users (at least 10 user groups) within a specific domain science or
TeraGrid users in general, utilizing the unique expertise and experience that AUS staff hold. The
projects are identified based on impact it will have on the user community, input from users, input
from AUS staff and other TeraGrid experts. Expertise is available within the AUS staff pool,
across the RP sites, to undertake such projects that have broader impact. AUS staff from multiple
RP sites jointly collaborated on these projects. AUS staff continued to pursue five such projects.
Two of these were pursued by the AUS staff that have domain science as well as HPC expertise
in chemistry/biochemistry and materials science and are users and, in some cases, developers of
molecular dynamics and materials science codes. Large numbers of TeraGrid users use the
molecular dynamics and materials science codes on various TeraGrid machines. The third project,
investigating hybrid programming model, was pursued by AUS staff that has expertise in the area
of HPC programming paradigm. The fourth project involves investigation of PGAS languages
and comparison of performance with MPI based programming models. Initially kernels for linear
system solvers are being explored and later other kernels/codes will be added. AUS staff from
other sites can join once the project progresses. The fifth project involves developing tutorials of
profiling and tracing tools for users.
The first project involved porting, optimizing (using the best compiler options, machine specific
environment variables), benchmarking, and profiling widely used molecular dynamics (MD)
codes on various TG machines. AUS staff from PSC, SDSC, and TACC are participating in this
project. AMBER and NAMD were benchmarked on Ranger and Kraken for various input cases
and utilizing various compiler and environment options. Benchmarks were also done using all the
cores within a node, and partial number of cores within a node to see the impact on performance
and scaling. The results were presented from the AUS benchmark webpage
(https://www.teragrid.org/web/user-support/aus_projects) and feedback was received from some
users of these codes. User news was posted to inform the general users about this webpage. As
newer machines, such as the DCL machines, come into production, further benchmarking of MD
codes is ongoing on these machines.
We believe that as such detailed MD benchmark data are made available to the TeraGrid MD
users, it will greatly benefit this large community. In addition to always having the optimally
installed MD codes on various TG machines, this project will enable the MD community to (1)
gain knowledge about performance and scaling characteristics of MD codes on various machines,
(2) know about the optimal compiler and libraries to use on TG machines, (3) install their own
versions of MD codes on TeraGrid machines, and (4) justify resource requests in allocation
proposals. This will also help TRAC reviewers to cross-check users’ requests for resources in
their allocation proposals. The profiling results will be made available to the MD code developers
and enable them to tune the future released versions of the MD codes for TeraGrid machines.
Feedback regarding the usefulness of this project, was sought from about ten prominent long time
TeraGrid MD users and strong support for this project was received from the users.
The second project is similar to the MD project described above and targets codes that are used
more for materials science type simulations, such as VASP, and CPMD. AUS staff, from NCSA,
LONI, PSC and NICS, that have expertise in these codes and in the field of materials science,
carried out this second project. No further activities were needed in this quarter for this project.
62

User news was posted to inform the general users about this webpage. User feedback received,
from materials science users, for this project also showed strong support for this project.
The third project involves evaluation and benchmarking of the hybrid (MPI-OpenMP/pthreads)
programming model. Due to the emergence of multi-core processors, hybrid programming, using
MPI tasks and multiple OpenMP threads or pthreads per tasks, is of interest to users. This model
may provide better performance than straight MPI programming. In addition hybrid programming
may allow some simulations that are otherwise not possible due to limited memory available per
core and hence provide a science capability not achievable by straight MPI codes on multi-core
architectures. In order to investigate these issues, AUS staff from PSC, NICS, TACC and SDSC
compared performance and usage of codes written in hybrid mode versus straight MPI mode. The
codes being targeted are an astrophysics code, a hybrid benchmarking code, phylogenetics codes,
FFTW, and a Monte Carlo code and work is ongoing. Raghu Reddy (PSC) wrote a draft
whitepaper based on his hybrid programming work. Other AUS staff, participating in this project,
also worked on developing whitepapers based on their hybrid code work. These whitepapers will
be made available for TeraGrid users via the AUS project webpage.
As mentioned earlier the fourth project involves investigating PGAS languages. No significant
update is available this quarter for this project.
The fifth project involved developing TAU tutorial and this is done by AUS staff from PSC. This
tutorial and documentation are available from AUS projects webpage. The TAU tutorial gives
users information about TAU installation on Ranger and Kraken and other TeraGrid resources,
and is a practical guide for TAU. It will allow users to learn about the commonly used features of
the TAU software tool and will enable users to successfully start using TAU on TeraGrid
resources.
5.5.2.3
For the third category of ASP, AUS staff at NICS are implementing and testing “ld” and “aprun"
as a part of the libraries database project on Kraken. This database will provide user support staff
knowledge about library usage such as which ones are used the most and how frequently. The
information obtained will be used to improve the software management, procurement and
maintenance requirements. Also, this information may prove useful in the planning and
development stages of new resources as they are brought to production. This project was
completed in this quarter and has now moved into production phase. The database is growing
based on regularly collected software usage data. The data is being used to identify software that
is used least often, and to identify software used most often. These information are used to tailor
support for software at NICS.
5.5.2.4
Creation of a software and associated database to monitor libraries being used by
users
Under the fourth category of ASP, the project being done involves both ASTA and ASP and it
has been described in §5.5.1 (PI Roberts (RTI International, Computer Science) EpiSims; PI:
Karniadakis (Brown U., CFD) NekTar-G2; PI Coveney (University College of London,
Chemistry) GENIUS). In this quarter this project has also transitioned into operational phase and
will continue as an ASTA project.
5.5.3
Implementation and testing of software allowing cross-site MPIg based simulations to
run and associated interaction with cloud computing
Advanced Support for EOT (ASEOT)
AUS staff from all the RP sites contribute to preparing and delivering advanced HPC and CI
technical contents for trainings and workshops that are offered throughout the year at all the RP
sites. This is done in coordination with the EOT area. List of workshops, which were offered are
63

listed in the EOT section of this report. One of the EOT highlights, in this quarter, was teaching
of an Introductory HPC workshop at Puerto Rico (University of Puerto Rico at Rio Piedras and
University of Puerto Rico at Mayaguez) in December. Multiple AUS staff such as Marcela
Madrid (PSC), Mahidhar Tatineni (SDSC), Amit Majumdar (SDSC), Sergiu Sanieleveici (PSC)
participated in this workshop at Puerto Rico.
Under the ASEOT effort, AUS staff continued to participate in the Extreme Scalability (XS)
working group, which specifically focuses on architecture, performance, tools and software
related to petascale-level simulation. AUS staff are participating in projects identified by the XS
working group, and in many cases these projects allow interaction with researchers/users funded
by NSF PetaApps, SDCI, and STCI grants. Interactions within the extreme scalability working
group also help AUS staff to identify ASPs that may impact large number of users. The hybrid
MPI-OpenMP programming model project and the TAU tools project, as stated in §5.5.2.2, are
joint projects between the AUS area and the XS working group.
AUS experts, from all the RP sites, regularly present technical talks at the bi-weekly AUS
technical tele/web-conferences. These technical talks, based on ASTA work or other AUS work,
are recorded and the slides and the audio recordings are available from:
http://teragridforum.org/mediawiki/index.phptitle=Technical_presentations.
Multiple AUS staff are regular members of the TRAC review committee and in addition many
AUS staff review TRAC proposals that request smaller amount of SUs and hence do not require
review by regular members of the TRAC review committee.
6 User Facing Projects and Core Services
The User-Facing Projects and Core Services (UFC) area continued its work of providing the
central access and information dissemination points for TeraGrid users and to present a single
interface to the collective efforts of the TeraGrid RPs. The GIG supports and coordinates effort in
this area from staff at IU, NCSA, PSC, SDSC, TACC, and UC/ANL.
6.1 Highlights
In a project spanning multiple RPs staff from NCSA, PSC and SDSC, coordinated as a team
through the GIG, successfully upgraded the TGCDB’s primary and secondary servers to new
versions of postgreSQL. This upgrade provides significant performance enhancements as well as
an ongoing supported version of the db for TeraGrid.
TeraGrid continues to set new records for use, in October, 1,373 individuals charged jobs on
TeraGrid resources, a new monthly high. Contributing to this increase was a rise in science
gateway use, nearly 1/3 of all TeraGrid users submitted jobs were done so through a science
gateway.
SDSC staff members installed Trestles documentation as well as integrated Trestles into AMIE in
preparation of the Jan 1 production date.
TACC staff members, Katie Cohen and Kent Milfeld coordinated the TRAC review process and
organized the allocation review meeting held within the reporting period. In addition, the TACC
team organized and chaired the weekly allocations working group meetings, and processed all
advance, startup and supplement allocation requests (275) during the reporting period.
6.2 Enhanced TeraGrid User Access
This objective supports the creation and distribution of new user logins across TeraGrid RPs for
the various resources, as well as enhancements to this process to integrate with campus
64

authentication infrastructures. Within the TeraGrid User Portal, users are also provided ways to
maintain their user profiles, customize their interaction with TeraGrid, and access TeraGrid
resources and services remotely.
Active TGUP development continued on the vetted/unvetted user portal account effort, which is
nearing release, and GridShib authentication to enable users to easily gain access to TeraGrid and
its resources
6.2.1
Operational Activities
The fourth quarter has typically seen a slight dip in TeraGrid user metrics, and Q4 2010 was no
exception. 515 new individuals joined the ranks of TeraGrid users, and in October, 1,373
individuals charged jobs on TeraGrid resources, a new monthly high. This continued elevated
workload is made possible by prior efforts to streamline the new-user process and reduce the
central staff effort.
TeraGrid users numbered 6,505 at the end of the quarter, including 5,563 individuals with current
TeraGrid accounts. Of the 5,563 individuals, 1,904 (34%) were associated with jobs recorded in
the TeraGrid central accounting system; an additional 942 users submitted jobs via science
gateways—that is, nearly 1/3 of TeraGrid users submitting jobs worked through gateways. Figure
6-1 shows the growth of the TeraGrid community since 2004. (Active Users = Users Charging
Jobs + Gateway Users; TeraGrid Users = Current Accounts + Gateway Users.)
The Knowledge base and the TGUP’s access capabilities continue to be among the most highly
used features, with users accessing the file manager portlet 12,973 times, the knowledge base
11,420 times, and allocations usage 8,980 times this quarter. Table 6-1 shows the complete listing
of the most highly visited TGUP applications for this quarter.
Figure 6-1. TeraGrid User Trends, Q4 2010
Table 6-1. Q4 2010 Most Visited TGUP Applications
Application
# Users Q4 2010 Visits
File Manager 64 12,973
Knowledge Base (via TGUP alone) 10,959 11,420
65

Allocation Usage 1,039 8,980
GSI-SSH 491 6,460
System Accounts 671 5,783
Data Collections 2,228 4,238
System Monitor 486 3,150
Science Gateways 28 1,397
Consulting 216 1,169
Add/Remove User Form 189 722
QBETS Wait Time 40 625
Karnak Prediction 46 620
My Jobs 154 510
QBETS Queue Prediction 395 395
Ticketing System 143 320
User Profile 184 280
Feedback 177 177
6.3 Allocations Process and Allocation Management
This objective encompasses the allocations process, both for Startup/Education allocations as well
as the merit-review TRAC Research request process, the POPS system for request handling and
management, mechanisms by which allocation PIs manage allocations through transfers,
extensions and so on, and interfaces by which allocation PIs manage the users who are authorized
to use their allocations. Operationally, this objective includes the TRAC review process, the
Startup allocations review and decision process, and the maintenance and operations of the POPS
system.
The allocation staff continues to receive in excess of two new/renewal startup requests each day.
TeraGrid received 206 Startup requests (168 approved) and 12 Education requests (12 approved).
Table 6-2 shows the overall allocations management activity handled by POPS and the
allocations staff. Note that for Transfers, the table shows only the positive side of the transaction
to show the total transfer level; there is a corresponding negative amount, adjusted for resource
exchange rates.
Table 6-2. POPS Requests and Awards, Q4 2010
Research Startup Education
# SU # SUs # SU # SUs # SU # SUs
Req Requested Awd Awarded Req Requested Awd Awarded Req Requested Awd Awarded
New 58 266,335,446 52 65,028,339 177 35,790,440 144 24,470,244 9 1,343,026 9 1,143,029
Prog. Report 7 10,874,082 7 8,180,451 n/a n/a
Renewal 61 343,166,885 58 209,659,993 29 6,438,085 24 5,501,386 3 193,090 3 193,090
Advance 23 28,700,768 23 11,658,507 n/a n/a
Justification 5 8,225,558 4 4,726,478 0 0 0 0 0 0 0 0
Supplement 16 42,771,601 13 13,336,601 22 1,905,034 19 1,580,004 1 60,000 1 60,000
Transfer 106 48,982,063 97 46,917,813 30 943,360 27 787,526 3 120,000 2 70,000
Extension 43 n/a 40 n/a 44 n/a 41 n/a 0 n/a 0 n/a
6.4 RP Integration and Information Sharing
The TGCDB continues to be successfully monitored via Inca. The failover system continues to
sync with the production TGCDB server. In a GIG coordinated project spanning multiple RPs,
staff from NCSA, PSC and SDSC working as a team successfully upgraded the TGCDB and
failover servers to a new version of postgreSQL. In doing so, significant performance
enhancements were made available as well as providing an ongoing supported version of the db
for TeraGrid.
66

6.5 RP Operations: Accounting/Core Services
TG RP staff provided local allocations and account management support for users including
account creation/deletion, password reset, and local system usage accounting. These staff
interacted with the TG Account Management transaction system (AMIE), which is a transactionbased
grid account management system that is central to all users having access to the TeraGrid.
6.5.1
NCSA staff continues to support and maintain AMIE and contributed to the recent upgrade of the
TGCDB.
6.5.2
NCSA
PSC continues to house, support and maintain the secondary instance of the TGCDB. The PSC
staff continues to ensure replication between the secondary and the primary continues. PSC staff
were central to the success of the recent upgrade of the TGCDB.
6.5.3
PSC
SDSC continues to house, support and maintain the primary instance of the TGCDB. The SDSC
staff continues to ensure replication between the secondary and the primary. SDSC staff were
central to the success of the recent upgrade of the TGCDB. SDSC staff also built the local
accounting infrastructure to track Trestles CPU usage and the Trestles resource was integrated
into the TeraGrid via AMIE in December.
6.5.4
SDSC
TACC
TACC staff members continued to contributed to activities related to User Facing Projects and
Core Services. Maytal Dahan led TGUP portal development activities including the development
of vetted/unvetted user portal accounts and GridShib authentication to enable users to easily gain
access to TeraGrid resources. Furthermore, this quarter the user portal team released expanded
job service features. The user portal team released an interface to the Karnak prediction service
offering start time and wait time predictions on TG resources. The team also released a ‘’My
Jobs’’ feature, which lists the status of a user’s jobs on the TeraGrid. It also integrates Karnak by
enabling prediction capabilities on queued jobs. The team continued to support the software and
systems for both the TeraGrid User Portal and the TeraGrid Web Site. Katie Cohen and Kent
Milfeld coordinated the TRAC review process and organized the allocation review meeting held
within the reporting period. In addition, the TACC team organized and chaired the weekly
allocations working group meetings, and processed all advance, startup and supplement allocation
requests (275) during the reporting period.
6.6 User Information Presentation
This objective ensures that users are provided with current, accurate information from across the
TeraGrid in a dynamic environment of resources, software and services. Operationally, this
objective ensures high reliability of the presentation mechanisms, including quarterly operational
support for the TGUP and TeraGrid web site. Support for TeraGrid internal web environments,
notably the Wiki, is included here.
Members of both the TGUP and Documentation teams received advance LifeRay training at the
LifeRay corporate offices in CA.
The TGUP and TeraGrid Web presence continued to increase its user base and support large
numbers of visitors and page hits. Table 6-2 shows hits, visitors and logins for the TGUP, Web
site and Wiki.
67

Table 6-3. TeraGrid Web traffic, Q4 2010.
Web Portal Wiki
Unique Visitors Hits Unique Visitors Unique Logins Hits Unique Visitors Pages Edits
Oct 7,616 650,744 3,519 1,191 365,191 2,745
8,136 48,419
Nov 7,741 610,412 3,165 1,173 304,410 3,054
Dec 5,831 499,191 3,035 1,058 308,510 2,470
6.7 RP Operations: User Facing Projects
TG RP staff maintained and updated local documentation for their resources. Some of this
activity resulted from user feedback and interaction with the TeraGrid Operations Center, RP help
desk support personnel, and members of the advanced user support team. Hardware and software
upgrades on HPC, visualization, and storage resources also resulted in local documentation
activities.
The SDSC Documentation Group added the entire User Guide section for the Trestles resource.
This included seven new pages of User Support information integrated with the existing site,
located at: http://www.sdsc.edu/us/resources/trestles/. Existing documentation was also updated
for the Dash resource and the Visualization Services section.
6.8 Information Production and Quality Assurance
This objective includes efforts to provide documentation for TeraGrid activities, provide a
knowledgebase of answers to user questions, and maintain and update general web site content,
all within the TeraGrid Web environment. This activity requires close interaction with RPs and
other GIG areas as subject-matter experts and sources of information. As part of UFC operations,
this objective focuses on updates and Quality Assurance (QA). These ongoing processes and
efforts ensure that existing user information remains correct, is coordinated across both the GIG
and RP subject-matter experts, is in sync with evolution of TG environment (via collaboration
with the CUE working group), and contains minimal errors.
6.8.1
IU
2010 Report for the TeraGrid Knowledge Base
Summary statistics for the TeraGrid Knowledge Base:
Total number of documents available in the TGKB at the end of 2010 = 760
The total number of TGKB documents including those archived and in draft = 851
Number of new documents created during 2010 = 100
Number of documents modified during 2010 = 646
Number of documents archived during 2010 = 34
Total hits (documents retrieved) = 1,005,645
Relevant hits (documents retrieved, minus bots, crawler, scrappers, link checkers) = 360,231
Relevant hits excluding searches on “common Inca error” and “CTSS core errors” = 217,440
68

Note that the total repository of TGKB documents is 851, of which 71 are archived, primarily
because of the retirement of resources, and 20 are in a draft state. This leaves 760 documents that
are currently in active use.
69

Most new documentation originated as a direct result of conversations within the User Services
group. The ticket system and the Forums in Life Ray have also spawned documentation. As a
general practice, the growth of the TGKB is now focused on support-driven documentation and
less on quota base contributions.
Not reflected in these numbers are 119,821 hits that originated from a utility that searches for
“common Inca error” and 29,970 hits on “CTSS core errors”.
An analysis of the “referrer” field in the web logs shows that a significant proportion of the
relevant hits (non-spam/bot/crawler) that originated from the TeraGrid server were initiated by
the Inca and CTSS support tools.
70

From an analysis of the hits for which we have referrer information, excluding those that
originate from support tools, it is clear that the search interfaces on the IU and TeraGrid servers
account for a relatively small proportion of the searches. People are getting to TGKB
documentation primarily through Google searches. This phenomenon increased as the year
progressed.
Some of the most frequently accessed documents during 2010 were:
How do I get started using the TeraGrid
Who is eligible to apply for a TeraGrid allocation
How do I apply for a new TeraGrid allocation
On the TeraGrid, what is Queen Bee
What's the quickest way for new users to access the TeraGrid
What is a TeraGrid-wide login
How do I request supplemental service units for a TeraGrid allocation
Getting started on Big Red
On a TeraGrid system, how can I use SoftEnv to customize my software environment
How do I request additional calendar time for my TeraGrid allocation
Big Red On the TeraGrid, what is Big Red
What is Lustre
About applying for a Research allocation on the TeraGrid
On the TeraGrid, what is Steele
71

How do I transfer an allocation from one TeraGrid system to another
How do I create a login for the TeraGrid's Partnerships Online Proposal System
On the TeraGrid, how are compute jobs charged
On the TeraGrid, what types of allocations are available
How can I add or remove a user from an existing TeraGrid allocation
Using TREE-PUZZLE on Big Red at IU
What is Globus, and where can I find out how to use it on the TeraGrid
TeraGrid overview
How do I access IU's MDSS/HPSS from my TeraGrid account
FASTA programs on Big Red
On the TeraGrid, what is Single Sign-On What is Single Sign-On
On the TeraGrid, what is the NSTG
What is ClustalW, and where is it installed on the TeraGrid
On the TeraGrid, what is Pople
On the TeraGrid, what is the Longhorn Visualization Portal
On the TeraGrid, what is Lonestar
How do I set up authentication with SSH keys on a TeraGrid resource
On Big Red, how do I transfer files using Grid-FTP (globus-url-copy)
On the TeraGrid, what is Spur
What is HPSS
What is MPI, and where can I find information about using it on TeraGrid resources
How do I compare queues on TeraGrid resources at different sites
How can I access TeraGrid account and resource information
What visualization and analysis resources are available on the TeraGrid
What is the TeraGrid User Portal, and how do I access it
What is Python, and where is it installed on the TeraGrid
7 Data and Visualization
7.1 Highlights
SDSC continued visualization support for the latest SCEC simulations with selected ones featured
at SC’10 Gordon Bell, Masterworks presentations, and the SC’10 plenary video presentation.
Members of the TACC Visualization group facilitated the efforts of TeraGrid user Rob Farber to
scale his code to 500 GPUs using the TeraGrid resource Longhorn. Farber successfully ran the
code during dedicated time and presented the results at SC10.
72

7.2 Data
7.2.1
Data Movement continues to be a stable, slowly evolving area of TeraGrid infrastructure. The
most recent deployment of GridFTP includes the ability to initiate connections using SSH, thus
providing more flexibility to users in how they access GridFTP servers. The “WANpage” has
now been deployed at PSC, providing information on WAN file system performance analogous to
the “Speedpage”, and new information services registration patterns have been developed to
support the Albedo Lustre-WAN file system and facilitate automated testing of client data
movement performance.
7.2.2
Data Movement
Wide-Area Filesystems
The Albedo Lustre-based WAN file system is now substantially complete, with all servers and
storage now deployed and all sites except SDSC having added the storage to the file system.
Security policies for providing access to the file system from non-TeraGrid resources have been
developed, in consultation with the Security working group. Several improvements to the identity
mapping code, included security patches, have been made by staff at PSC and Indiana, and
infrastructure to track client mounts and associated identity mapping data has been developed and
is already deployed at some sites.
Indiana University has worked with Brock Palen at the University of Michigan to mount DC-
WAN there. Palen approached IU for a DC-WAN mount to ease the pain his TeraGrid users have
been experiencing getting data local to Ann Arbor onto the TeraGrid. IU has successfully brought
all of the head nodes at Michigan online and is making plans to bring the compute nodes (via
Lustre Router) online as well.
7.2.3
Data Collections
The Data Replication Service remains the primary mechanism of support for TeraGrid-wide data
collections, and approximately one third of the total resources in the replication service have now
been allocated. Evaluations have been performed on various mechanisms for web-based and
desktop access to the DRS, such as WebDAV, CGI-based interfaces, and native graphical
interfaces. Work to facilitate staging of data into and out of large tape archives used by the DRS
is ongoing. The DRS has also been federated with several more specific data grids, including the
iPlant Collaborative data grid, the TACC center-wide data grid, and the SDSC data grid; this
facilitates the movement of data from campus- or project-level resources into national-scale
resources.
ORNL has been working in collaboration with the DataONE project to explore and define, with
the data area director and the TeraGrid Forum, ways for DataONE and TeraGrid to interoperate.
Specifically, a plan has been defined and is being executed for a pilot DataONE member node
deployment with a mounted TeraGrid storage substrate. This effort will produce a kind of “Data
Collections Gateway” and will also involve the Science Gateways group. Allocation of these
resources will be requested in the Q1CY11 TRAC process.
73

7.2.4
Archival Storage
SDSC continued HPSS to SAM-QFS migration of users with numerous small files. Repacking
effort continues at roughly 60-70 volumes per week. TACC has completed a set of upgrades to
the Ranch/SAM-QFS archive infrastructure, including a substantial increase in the available disk
cache that should improve overall data throughput and stability.
7.2.5
Data Architecture
The recommendations made in the data architecture document have now been implemented as
completely as possible within the context of the TeraGrid effort. Current and future efforts in data
architecture are focused on planning for and implementing the transition to XD in a transparent
fashion, ensuring continuous access to data for the user community through the transition, and
compiling and communicating lessons learnt for the benefit of XD and other large-scale
cyberinfrastructure efforts.
7.3 RP Operations: Data
7.3.1
Distributed Data Management
The PSC-wide, central Lustre file system, Bessemer, has been upgraded to Lustre version 1.8.
During this upgrade, storage was re-provisioned to Albedo, TeraGrid’s Wide Area Lustre project.
By upgrading to Lustre version 1.8, PSC will support a broader range of modern clients mounting
Bessemer. It has been mounted on Blacklight. Bessemer is also mounted again on Pople, Salk,
and several utility servers around the Center.
Josephine Palencia and Kathy Benninger of PSC along with Yujun Wu of the University of
Florida have made good progress on the ExTENCI Lustre WAN project and the tasks are on
schedule. Palencia and Wu installed the necessary Lustre WAN and Kerberos software on a
system at the University of Florida, and Benninger is helping with tuning the 10GigE interface.
With Kerberos authentication enabled on the file system, the work has increased its security. A
private wiki for the system administrators from the partner sites has been created by PSC. All
project members who need access to this wiki have been given PSC ID’s and can authenticate to
it using Kerberos. PSC leads weekly teleconference calls on this part of ExTENCI. The ExTENCI
Lustre WAN project was highlighted in two events at SC10. First, J. Ray Scott gave a talk at the
High Performance Cluster File System (HPCFS) Software Foundation meeting, which included a
slide on the ExTENCI file system, and second, there was a demonstration of the file system with
clients in the CalTech and PSC booths.
DC-WAN is now mounted on Steele’s login nodes, and Purdu continued to work on mounting the
DC-WAN on Steele’s compute nodes. A December maintenance downtime allowed Purdue staff
to make configuration changes. After Jan.-Feb. testing and working with IU staff, Purdue hopes
to release this to production.
SDSC Albedo servers and storage have been provisioned and should be ready for addition to the
file system by the end of 01/11.
LONI has ordered a 200TB storage upgrade for Queen Bee.
7.3.2 Data Collections
Purdue added three new datasets for access through its multidisciplinary data portal: (1) Indiana
precipitation dataset, (2) St Joseph Watershed water quality data and stream flow data, and (3)
74

historical time series of average precipitation and temperature data. New features were also
implemented that allow users to preview remote sensing and other image data before download.
Purdue also continued to develop a data publishing web application which aims at enabling
individual researchers to self publish and share their dataset. The component supports two types
of data management backend: iRODS for file based data and MySQL for table based data. It
provides templates and wizards to help guide users to upload, search for, and download data.
TACC completed the addition of a new collection of data from the University of California
Curation Center, representing a large collection of scholarly publications archived by the
University of California. This collection is being used to explore the opportunity for integrated
infrastructure supporting both traditional scholarly publishing activities and publication of large
scientific datasets.
7.4 Visualization
NCAR develops and distributes the visual data analysis package, VAPOR, which is funded in
part through GIG PY5&6 funds. Progress was made on two of the major milestones of the
PY5&6 award: 1) Development work on a lossy compression library for scientific data has been
completed and made available as part of the VAPOR 2.0 release. 2) Performance optimization of
the parallel API to the VAPOR progressive access data model was completed, and friendly users
for further testing are being gathered.
The NCAR VAPOR team has also developed a next-generation Progressive Data Access (PDA)
scheme based on their promising research results in wavelet coefficient prioritization. This new
scheme offers far greater data reduction than the current multi-resolution approach at comparable
fidelity, or improved fidelity at the same level of data reduction. The prototype PDA research
code was hardened and integrated into VAPOR, and made publicly available in version 2.0 of the
package, released in December, 2010. VAPOR 2.0 provides command line tools and a serial C++
API for transforming raw binary data into and out of the VDC2 format. Moreover, VAPOR’s
visual data analysis tool, vaporgui, is capable of reading and displaying VDC2 format data.
Considerable progress was also made on both the development and support for a public, parallel
API for writing Cartesian gridded data into the next-generation PDA format (now known as
VDC2) and the evaluation of the functionality, scalability and performance of the parallel API for
writing Cartesian gridded data into the VDC2 format. A prototype, Fortran-callable parallel
interface for writing VDC2 data was developed, and specifications for a public API completed.
The public API will be offered as an extension to NCAR’s own Parallel I/O (PIO) library
(http://web.ncar.teragrid.org/~dennis/pio_doc/html/). Existing Fortran PIO codes will be able to
output VDC2 data with minimal modification, calling only two or three configuration functions.
New codes will benefit from PIO’s abstraction of high-performance parallel IO, and the ability to
easily toggle between output to one of PIO’s native, uncompressed formats (currently netCDF
and raw MPI-IO), and compressed VDC2 data.
The prototype parallel interface has been tuned for scalability and high performance. The plot
below demonstrates the scalability of the library on NCAR’s IBM Power6 system with synthetic
data sets with 2048^3 and 4096^3 degrees of freedom.
Work on the public API to VDC2 should be wrapped up early in the next quarter. Once
completed, the NCAR VAPOR team will evaluate the new capabilities by working with
numerical modelers to integrate the API into an actual simulation code. Discussions with two
candidate friendly user groups have already begun: one group at the National Renewable Energy
Lab (NREL); and an internal group working at NCAR, but computing at NICS.
75

Figure 2: Throughput in Megabytes per second (MBps) vs. number of MPI tasks for
2048^3 and 4096^3 synthetic data sets.
7.5 RP Operations: Visualization
During this quarter, visualization services at the TG RPs included the maintenance,
administration, and operation of visualization resources, including visualization-specific systems.
The UChicago/Argonne RP continued its collaboration with PI Michael Norman on his Projects
in Astrophysical and Cosmological Structure Formation. A recent simulation computed by the
Enzo team, done on Kraken at NICS, uses a flux-limited diffusion solver to explore the radiation
hydrodynamics (rad-hydro) of early galaxies, in particular, the ionizing radiation created by
Population III stars. The simulation volume is 11.2 co-moving megaparsecs, and has a uniform
grid of 1024 3 cells, with over 1 billion dark matter and star particles, the largest calculation of this
type known to date. The UChicago/Argonne RP team produced an animation of multiple
variables of this new data using vl3, a hardware-accelerated volume rendering application
developed at UChicago/Argonne, visualizing these previously unseen results. This animation
was showcased in both the Argonne and SDSC booths at SC10. The Enzo team has since been
using this animation to promote their work, posting it in various Web outlets, including SciVee
(http://www.scivee.tv/node/26565) and Vimeo (http://vimeo.com/17771397). Enhancements were
also made to the vl3 Enzo reader to enable the calculation of derived quantities from multiple
variables as the data is read from disk. The Enzo and UChicago/Argonne teams also collaborated
76

on a demonstration that took place in the SDSC booth at SC10. It included real-time streaming of
volume rendered images from multiple instances of vl3 at Argonne to a tiled display on the
exhibit floor in New Orleans. The researchers were able to simultaneously visualize multiple
variables from the rad-hydro simulation and compare results. One invaluable improvement in this
demonstration over a similar one at SC09 is that control over the remote visualization was put in
the hands of the scientists. Using an interface largely refactored from previous work done on the
TeraGrid Visualization Gateway, the researchers were able to control the view, as well as
manipulate color and transfer functions from a standard Web browser. Another improvement
includes incorporating the Celeritas data-streaming library, which is optimized for high
bandwidth, wide-area networks. This enabled interactive, uncompressed streaming of the images,
eliminating artifacts due to compression, while also reducing latency. In addition, a Python client
was developed, which integrates remote streaming to the desktop with the visualization controls
in a single client. Beyond just a one-time demo, the tools developed are now in the hands of the
Enzo scientists, who are able to use them to conduct their research after returning home to SDSC.
Feedback from the Enzo team will help guide further development of the tools.
The UChicago/Argonne RP continues to work with the Parallel Simulations of Blood Flow in the
Human Cranial Tree project (PI: George Karniadakis), and in collaboration with a PetaApps
award, on the visualization of their data, using a combination of ParaView and custom-built code.
The RP team continues development of their custom ParaView reader plug-in, which loads data
in native NekTar format, computes derived quantities and curved surfaces at higher resolutions,
where appropriate, and pushes it into the ParaView pipeline. Support for time-variant data has
now been integrated into the reader, enabling researchers to animate through time steps of the
simulation results. Data caching has also been implemented in the reader, alleviating the need to
continually re-read data from disk. The team will next investigate techniques for pre-fetching
data in order to further reduce delays incurred during animation while waiting for data to be read
from disk. The NekTar team has integrated LAMMPS code with their NekTar simulations,
producing multi-scale data sets. The RP team continues to develop utilities for converting the
resulting red blood cell (RBC) and velocity continuum and various types of particle data to
formats suitable for visualizing. The latest calculations simulate a single diseased red blood cell,
and the flow around it, as it is pushed along and adheres to the boundary wall. Several
animations of both the RBC data and the NekTar continuum data were produced, and showcased
in the Argonne booth at SC10, as well as included in the SC10 Plenary Session Video loop. The
two teams also collaborated on an electronic poster submission for SC10 highlighting the various
aspects of this work, and presented it at the conference.
Mike Norman describes the radiation-hydrodynamics
simulation data during streaming visualization
demonstration at SC10.
Still frame from an animation of four variables of the
radiation-hydrodynamics simulation data.
77

NCAR operates the dual-node DAV cluster named Twister. NCAR also supports the visual data
analysis package, VAPOR. Version 2.0 (previously called version 1.6) of VAPOR was released
late this quarter. Nearly 300 copies of the software have been downloaded in the first month since
its release.
SDSC continued visualization support for the latest SCEC simulations. A few animations of M8
visualizations were featured at SC10 Gordon Bell, Masterworks presentations and SC 10 plenary
video. SDSC continued development of GlyphSea software and presented a poster at 2010 AGU
Fall meeting showcasing volumetric visualization of seismic data. A new collaboration to
visualize intestine flow in small intestine is in progress with PI James Brasseur of Pennsylvania
State University.
Purdue RP released its 3D visualization gadget for super resolution NEXRAD level II radar data
in December 2010. The gadget is available from the RP web site and the Google gadget
directory. The figure below is a screen shot of the gadget showing Oct. 26, 2010 3D storm data
from multiple radar stations.
The TACC RP continues to operate and support both Spur and Longhorn. Spur is completely
allocated out and the TACC RP has been working to ensure that visualization and data analysis
on Longhorn continues to increase.
8 Network Operations and Security
8.1 Highlights
In November, 2010 IU along with four research partners announced that they had succeeded in
using the Lustre file system over a wide area network (WAN), saturating the world's first
commercial 100 gigabit link. Using the full duplex capability of the 100 gigabit link, the team
achieved an aggregate average transfer rate of 21.904 gigabytes per second. This is over 87% of
the theoretical maximum allowed by the link. The longer-term goal is to explore the possibility of
having one or more wide area file systems that provide Dresden and Freiberg with massive data
transfer capabilities, enabling faster exchange of shared data and insights.
78

NCSA's SGI Altix UV system, Ember, went into production on November 1 st , replacing the SGSI
Altix system Cobalt.
SDSC installed Trestles, a DCL awarded 100TFlop system. SDSC began friendly user testing on
the Trestles system in early December and began TeraGrid production on Jan 1, 2011.
8.2 Networking
The TeraGrid continued operation of the backbone network consisting of core routers in Chicago,
Denver and Los Angeles, connected with dedicated 10 Gigabit/sec links and two levels of
redundancy in the form of alternative paths via NLR. TeraGrid resource providers maintain their
respective site border routers and dedicated links to one of the three core routers. The dedicated
links consist of at least one 10Gb/s link .
SDSC has been continuing to work with CENIC engineering on planning the re-routing of
TeraGrid waves in preparation of the removal of the Los Angeles TeraGrid core router. In mid-
January 2011 circuits that are currently terminating in the Los Angeles router will be re-homed to
the San Diego router.
TACC installed a new border router. The Juniper EX8208 enables the use of routing instances
(in lieu of having multiple physical routers) to keep the end-to-end peering separate from the
public routing peering (NLR, I2, UTNET, and Internet).
The following two graphs show total usage and 20-second average peaks on the TeraGrid
backbone between Chicago and Denver.
Note: Due to a data collection anomaly during early Oct (beginning 1/8 of the graphs) lower then actual levels were
recorded.
Figure 8-1: Daily peak bandwidth on
Chicago-Denver link.
Bandwidth is measured at 20s granularity.
Figure 8-2: Total transfer (GB) per day on
Chicago-Denver link.
Total theoretical capacity is ~100,000 GB/day.
8.3 RP Operations: Networking
During this quarter, TG RP staff provided support and operation of the local RPs network
connectivity to the TeraGrid backbone. Duties include daily monitoring and maintenance of the
network, planning and/or implementing changes to RP connections to the TG backbone, direct
user support for network-specific issues, switch firmware upgrades, and HPC system
interconnects. TG RP staff also kept the user community apprised of interruptions in network
connectivity via local and TeraGrid user news.
79

The ORNL NSTG operations continue to run without difficulty at the SNS location. From this
period forward, the network move from the ORNL main campus should be considered
successfully completed.
SDSC network engineers worked to facilitate an Infiniband to Ethernet bridging service to enable
TeraGrid users to have access to the new Trestles resource.
8.4 INCA
The SDSC Inca team continued to support and maintain the Inca deployment on TeraGrid and
helped system administrators to debug failures on their machines. In response to IIS changes and
requests from the GIG-pack and QA groups, existing Inca test suites were modified. Four
existing TeraGrid reporters were also modified and one new reporter was created and deployed.
At the time of this report 2,976 pieces of test data are being collected across TeraGrid platforms.
NCSA’s Ember was added to the Inca TeraGrid deployment during this time. NCSA’s Cobalt
was removed from Inca testing and TACC's Lonestar was removed from testing until the machine
returns in 2011.
The SDSC Inca team also updated TeraGrid’s Inca deployment to the working 2.6 release. The
two most relevant features of this release are data mirroring capabilities so that monitoring data
and notifications will still available if SDSC goes down (not yet configured) and historical
graphing abilities for performance data. The new graphing capabilities will allow the Information
Services team to view the response times of their Inca monitored server.
8.5 Grid Services Usage
The Globus usage listener for GT5 (GRAM5) continues to gather Grid Service usage for the
quarter. The following graphs detail this usage.
8.5.1 Data Movement (GridFTP) Statistics
Figure 8-3: Total GridFTP transfers (GB) for the quarter
per site.
Figure 8-4: Total GridFTP transfers (GB) per day per
site.
80

8.5.2 Job Submission (GRAM) Statistics
Figure 8-5: Number of GRAM submissions per
resource per day.
Figure 8-6: Total number of GRAM submissions
per resource for the quarter.
8.6 RP Operations: HPC Operations
TG RPs continued to provide the personnel and infrastructure support for the local TG RP
resources that are made available to the user community. System administration duties include
daily monitoring of the health of the system, 24 hour on-call where provided, basic filesystem
support, machine-specific parallel filesystem support, and operating system software maintenance
and upgrades. Additionally, this activity provides for CTSS kit (grid middleware) deployment,
administration and support. RP staff also assisted local and TeraGrid user services personnel to
provide timely announcements to the user community related to the status of all TG resources.
8.6.1
LONI
5.1M SUs were delivered to 14.1K TeraGrid jobs serving 102 to 156 unique users per month. The
Queen Bee system had 100% availability without outage in Q4 2010. A planned storage upgrade
for Queen Bee had to be delayed due to some issues with the State Purchasing. The purchase
order of 200TB storage was issued in early January 2011 and we expect the new storage be
available in Q1 2011.
8.6.2
NCSA
NCSA's SGI Altix UV system (Ember) went into production on November 1st. The system has a
peak performance of 16 TFlops and has a total of 8 TB of memory and 450 TBs of GPFS file
space. Each of the four UVs contains 384 cores with 2 TBs of memory and ran the Top 500 HPL
benchmark at 87.5% efficiency for 3.5TFlops each.
Cobalt retired on November 15th. All user accounts and allocations transferred for seamless
support of shared memory intensive applications to Ember.
8.6.3
ORNL
The largest operational activity for the ORNL NSTG environment continues to be staff training of
new SNS embedded systems administration. The learning curve for new staff to understand
TeraGrid’s accumulated body of knowledge is imposing. NSTG staff is climbing this learning
curve, but its operational performance has not been as high quality as prior norms. The current
81

emphasis is on getting new staff acquainted and comfortable with this environment and as such
more emphasis has been placed on learning the environment sooner than rigid uptime goals. More
rapid integration is seen as more important, particularly as NSTG looks to update the systems
used to facilitate data movement to/from TeraGrid HPC resources and neutron science facilities.
To that end, two new system administration staff attended SC10, including many TeraGrid
relevant educational events (such as cluster construction tutorial).
8.6.4
PSC
The 512-node version of Anton, a special purpose machine provided to PSC’s National Resource
for Biomedical Supercomputing (NRBSC) by special arrangement through D.E. Shaw Research
and the National Institute of General Medical Sciences of the National Institutes of Health, went
into production on Thursday 07-Oct-2010. Biomedical researchers were awarded time on Anton
through a competitive peer review process. Designed to dramatically increase the speed of
molecular dynamics (MD) simulations compared with the previous state of the art, Anton is
working fine and without problems for the researchers who received computing time in the first
phase of the project. PSC will have two one-day training workshops at PSC, one on Monday 21-
Feb-2011 and one on Wednesday 23-Feb-2011 for the 32 PI’s who received an allocation for the
second phase of the Anton project which starts on 01-Apr-2011.
Markus Dittrich and graduate student Jun Ma of PSC’s National Resource for Biomedical
Supercomputing presented a research poster entitled Computational study of active zone structure
and function at the mammalian neuromuscular junction at the Annual Meeting of the Society for
Neuroscience in San Diego Nov 13-17. Co-authors of the poster are J.R. Stiles of PSC and T.
Tarr and S.D. Meriney of the Department of Neuroscience and the Center for Neuroscience at the
University of Pittsburgh. The abstract of the poster says, “In our poster we present a
computational model of synaptic function at the mouse neuromuscular junction using MCell. Our
model reproduces established experimental results and we predict the arrangement of presynaptic
calcium channels and investigate possible mechanism for paired pulse facilitation.”
8.6.5
SDSC installed Trestles, a DCL awarded 100TFlop system. SDSC began friendly user testing on
the Trestles system in early December and began TeraGrid production on Jan 1, 2011. SDSC
brought up the Trestles system using Rocks 5.4/CentOS 5.5. SDSC updated the Dash vSMP
software from 3.0.45.36 to 3.0.45.47 to improve performance and stability of the vSMP system.
The Dash development system was upgrade through a series of versions (vSMP-3.0.45.10,
vSMP-3.0.45.36, vSMP-3.0.45.42, vSMP-3.0.45.45, vSMP-3.0.45.47, vSMP-3.0.70.0, vSMP-
3.0.110.0, vSMP-3.5.29.0) to test performance and new features of vSMP.
8.6.6
SDSC
TACC
A new TeraGrid resource, Lonestar4, is scheduled to commence production on February 1, 2011.
In order to begin installation of the system, the “old” Lonestar was removed from production
status on December 1, 2010. TeraGrid PIs with an active Lonestar allocation balance on
December 1, 2010 (and not expiring on January 1, 2011) will be granted an allocation on the new
system. The new system will be configured with 1,888 Dell M610 blades, each with 2, 6-core,
3.33 GHz Westmere processors (12 cores per blade), 24 GB of memory, and 146 GB of storage.
Compute nodes will have access to a PB Lustre Parallel file system. The network interconnect
will be QDR InfiniBand, a fully non-blocking fat-tree topology providing a point-to-point
unidirectional bandwidth of 40 Gb/sec.
82

8.7 Security
8.7.1
Security Working Group
A new proposal to support MPC on the portal was reviewed by the Security Working Group. The
new design removes MCP from the portal and moves it to Teragrid resources. One concern was
the protection of the proxy credential while in transit between the portal and the resource. The
group recommended that the portal should delegate this to GRAM vs. GridFTP.
Securing Community Accounts Survey: Victor Hazlewood reviewed the results of a survey sent
to RP security leads in early November. Although not all results have been received, he felt there
was enough information to share with the working group. Nancy began the discussion stating the
goal for this excerise is to have a more uniform development experience across RPs for Science
Gateways building off the information gathered at the "Security Summit" of 2008. The results of
the survey and suggested policy recommendations for RPs is in draft status and available upon
request.
The security team spent a siginificant amout of time responding to serious Linux vulnerabilities.
At on point we were averaging, one per week and when announced, there were no vendor patches
available to mitigate the risk. This created an akward sitiuation where RPs had to find ways to
protect against these threats until patches were forthcoming. For each of these vulnerabilities, the
Security Working Group created a wiki page so the vulnerability status of all RPs could be
tracked. This included identifying which of the production systems were at risk, what controls
could be applied to mitigate the risks until an official patch was released.
During the time of this report there were approximately 11 compromised user accounts and one
login node compromise.
The annual TeraGrid assessment project was completed and accepted during this quarter. This
year’s effort focused on an assessment of the TeraGrid User portal (TGUP) operations and
technologies. The TeraGrid user portal has become and increasingly important piece of the
TeraGrid infrastructure including a common place for many TeraGrid users to get live
information as well as pointers to static information on the portal, POPS, or other TeraGrid
maintained web presences. In addition, a username password login into the TGUP can also be
used to generate short term proxied credentials that can be used for access to TeraGrid resources
central or at the RP’s including queried of properly authorized user records on the TGCDB,
orchestrating file transfers on the TeraGrid and even command line access to TeraGrid sites. The
assessment found that TeraGrid staff were well aware of and taking measures to prevent common
web-based application vulnerabilities. In addition, the TGUP has defined and handled issues of
properly handling and proxying user credentials while accessing the portal. However, the
assessment did identify issues, in addition to these, that warranted further vigilance including
additional requirements for third party (i.e. no TeraGrid RP) hosted systems. During this quarter,
previous drafts were reviewed and discussed in several forums including the security working and
appropriate modifications were made.
8.7.2
Expanded TeraGrid Access
An update of the GSI-SSHTerm software to use the latest jGlobus 2.0 and BouncyCastle TLS
libraries was completed. GSI-SSHTerm is one of the most popular applications in the TGUP, but
the currently deployed version relies on out-of-date security libraries that do not support current
recommended security algorithms such as SHA-256. In Q1 2011 we will be testing this new GSI-
SSHTerm version for production roll-out in the TGUP. Jim Basney assisted the TGUP team with
the necessary Kerberos integration to support the new Vetted/Unvetted account management
process, and also contributed to the TGUP team with InCommon/Shibboleth testing in
preparation for rolling out the production InCommon login capability in the TGUP.
83

8.8 RP Operations: Security
During this quarter, TG RPs provided expert staff in support of local and TG-wide collaborative
security. TG RP local security operations include intrusion detection, examining log data,
investigating unusual use patterns, and incident response for all hardware and networks on a 24-
hour on-call basis. All TG RP partners are required to participate in the TG-wide Security
Working Group which coordinates security operations and incident response across the project.
During this quarter there was an incident of a user losing control of his access credentials. This is
not too unusual. The security working group and incident response teams respond to handful of
these each year. However, during this quarter it just happened that TGCDB access was not
available when this incident surfaced. It was noted that the absence of TGCDB access made
connecting this user with his/her local username on each TeraGrid resource more difficult and
consequently took more time. This did not create a serious problem for TeraGrid as this was a
single user compromise and exploitation to TeraGrid hosts was not aggressive, it did highlight
that a TGCDB outage does have some impact on security incident handling capabilities for
TeraGrid.
8.8.1
NCAR
NCAR was not directly involved in the multiple-account compromise security incidents in this
quarter. Accounts for the affected users either had not been requested on NCAR resources or had
been closed due to inactivity well before the incidents occurred. The prompt detection and
response at other RPs prevented adversaries from compromising additional accounts that were
active at NCAR. The ability of other RPs to share network connection details from adversary
activity was crucial to ensure that other accounts at NCAR were not also compromised.
NCAR’s network monitoring system is operating at full capacity with bro and argus.
8.8.2
The NCSA security team responded to five different incidents involving TeraGrid users or
resources in the fourth quarter of 2010. All five of these incidents involved remotely
compromised user accounts and one of the five resulted in the intruders escalating to root
privileges on a TeraGrid resource. Two of the compromised accounts were reported from remote
sites and actions were taken to disable the accounts and no malicious activity was discovered for
those accounts. The other three incidents were detected with the current security monitoring done
within NCSA and then we notified other TeraGrid sites who were able to take proactive
mitigation steps. The downtime for the compromised machine was less than a day. NCSA
worked with users in each incident to get their remote systems cleaned and access to NCSA reenabled.
8.8.3
NCSA
ORNL
During the fourth quarter of calendar 2010, there were no root compromises on the NSTG nodes.
There were no users who lost control of their credentials from activities on NSTG machines.
Accounts of TeraGrid users who were the victims of stolen credentials at other TeraGrid sites did
not log on to NSTG machines during suspect periods. NSTG staff took timely action to identify
and insure deactivation of accounts of users whose credentials were stolen and users whose
credentials may have been exposed during a known incident.
During this period, the ORNL RP, along with all of the TeraGrid RP’s continued to participate in
TeraGrid-wide security activities including the security working group meetings and the weekly
incident reporting calls.
84

8.8.4
PSC
During October, the PSC security team spent a significant amount of time responding to serious
Linux vulnerabilities that had been announced. These announcements were occurring about once
per week, and when announced, there were no vendor patches available. This created an awkward
situation where we had to find ways to protect against these threats until patches were made
available. For each of these vulnerabilities PSC’s Jim Marsteller created a wiki page so the status
of all production systems could be tracked. This included identifying which of the production
systems were at risk and what controls could be applied to mitigate the risks until an official
patch would be released. This same process was duplicated for TeraGrid in order to coordinate
responses from all resource providers. To date there have been no security incidents at PSC
related to these vulnerabilities.
PSC had no other security incidents during the quarter.
After conducting an audit for accounts that have not seen any activity in the past six months, PSC
disabled a number of inactive user accounts. We now have 1,200 fewer idle accounts for hackers
to target. There has only been one call from a user whose account was incorrectly disabled
(account was only used to access a Wiki maintained by PSC). We have modified our procedures
so that access to a wiki will be considered as active use of an account.
Jim Marsteller held the annual “Security 4 Lunch” on 08-Nov-2010 for all PSC staff and students.
The one hour presentation covered general information on security best practices including
material for PSC staff traveling to SC10. This year the staff was very interactive, asking a number
of security related questions. In fact, the VPN demo ran over the one hour allotted time.
8.8.5
Purdue disabled 11 TG accounts due to compromises reported at other RP sites. Purdue also
deleted SSH keys for one account due to compromise at another RP site. Purdue patched several
Linux kernel and libc 0-day vulnerabilities.
8.8.6
Purdue
SDSC
SDSC had five minor incidents this quarter. These incidents resulted in the compromise of
multiple user accounts elsewhere. Six user accounts were proactively suspended at SDSC as a
result. No evidence of abuse of these accounts was found prior to their deactivation.
SDSC performed and completed a security audit of Trestles. Minor configuration changes were
made to the system configuration to enhance security posture and improve the quality of
information available for incident detection and response.
8.8.7
TACC
A security compromise on the previous Lonestar system occurred on the two login nodes on
October 27, 2010. A then recent Linux kernel vulnerability was used by an intruder to gain root
privileges and replace ssh to collect user passwords. This tripped automatic change detection
scripts on Lonestar and notified admins of the compromise. TACC admins immediately cut off
access to Lonestar, notified TG security working group of compromised user account and began
forensic analysis of the compromise. During the investigation, the file used by the intruder to
start collecting passwords was found and had three user accounts in it, however, the intruder may
not have had time to retrieve those passwords once remote access was disabled. Those users
were immediately notified to change their passwords just in case of possible collection. A patch
was applied to the Lonestar kernel to remove the vulnerability.
In addition, three user accounts with compromises at other locations had their accounts disabled
on TACC systems.
85

8.9 Common User Environment
The CUE working group continues to work toward releasing the CUE to the Teragrid User
Community. The group wrapped up the beta testing period and is working with TG Public
Relations team to announce the development. The CUE implements and procedures have been
finalized on all current TG architectures, and documentation has been created explaining its use.
The group also continues to work on how to improve the commonality of queuing and testing
throughout the TG.
8.10 System Performance Metrics
8.10.1
Resource Usage Summary
TeraGrid saw a slight dip in delivered NUs for Q4 2010. TeraGrid compute resources delivered
13.2 billion NUs to users, down 0.2 billion, or 1%, from Q3 ( Figure 8-7). However, the
13.2 billion NUs represent a 1.7x increase in the NUs delivered in Q4 2009. Nearly 100% of
those NUs were delivered to allocated users.
Figure 8-7. TeraGrid Computational Resource Usage
8.10.2 Summary of Usage / Users / Allocations by Discipline
Figure 8-9 shows the relative fraction of PIs, current accounts, active users (individuals charging
jobs), allocations, and NUs used according to discipline. The nine disciplines with more than 2%
of NUs used are listed in the figure. Staff project usage is excluded, and users that appear on more
than one allocated project may be counted more than once, if those projects are associated with
different fields of science. Consistent with past quarters, the same disciplines dominant the NUs
used column, though there have been some changes in the ordering. As in past quarters, more
than 20% of PIs and current accounts, as well as 15% of active users, are associated with the 19
“other” disciplines, collectively representing about 4% of total quarterly usage.
86

Figure 8-8. TeraGrid Usage Summary by Discipline
8.10.3 Summary of Top PIs / Projects, Overall and by Discipline
873 different projects charged usage on TeraGrid computational resources, down just slightly
from 882 in Q3 2010. Figure 8-11 shows the top 20 PIs by total usage in NUs. The top 20 PIs
consumed 51% of the NUs used and the remaining 853 projects consumed the other 49%. Table
8-1 shows the top PIs (>2% of the discipline’s usage) in each discipline with more than 2% of
total usage; the final section shows the top PIs for all other disciplines. The overall rank of each
PI among the 873 projects charged is also listed.
87

Figure 8-9. Top 20 TeraGrid PIs, Q4 2010.
Table 8-1. Top PIs by Discipline for Q4 2010
Physics - 4,434,534,994
PI, Institution
NUs Rank
Robert Sugar, UC Santa Barbara 1,638,923,621 1
Colin Morningstar, CMU 640,211,020 2
Martin Savage, U Washington 428,451,797 4
Annick Pouquet, NCAR 341,273,022 6
Erik Schnetter, Louisiana State U 224,922,154 10
Gregory Howes, U Iowa 163,168,914 18
Klaus Bartschat, Drake U 114,348,314 27
Keh-Fei Liu, U Kentucky 112,853,570 28
All 58 Others 838,673,711
Molecular Biosciences - 2,658,196,474
PI, Institution
NUs Rank
Klaus Schulten, UIUC 345,370,971 5
Gregg Beckham, NREL 231,860,063 8
Thomas Cheatham, U Utah 227,306,075 9
Aleksei Aksimentiev, UIUC 224,136,759 11
Michael Klein, Temple U 205,449,920 12
Wonpil Im, U Kansas 157,946,455 19
Michael Feig, Michigan State U 130,993,491 24
Adrian Roitberg, U Florida 107,003,924 29
Adrian Elcock, U Iowa 82,770,951 35
Emad Tajkhorshid, UIUC 74,511,642 39
Joan-Emma Shea, UC Santa Barbara 65,880,698 44
Shantenu Jha, Louisiana State U 59,760,050 47
Benoit Roux, U Chicago 57,868,742 48
All 146 Others 685,819,292
Astronomical Sciences - 1,685,868,532
PI, Institution
NUs Rank
Adam Burrows, Princeton U 170,416,566 16
Matthias Rempel, NCAR 148,447,510 21
Thomas Quinn, U Washington 139,205,725 23
Dean Townsley, U Alabama 115,117,819 25
John Hawley, U Virginia 89,879,233 32
Juri Toomre, U Colorado 86,428,294 33
Tiziana Di Matteo, CMU 47,615,465 56
Alexei Kritsuk, UC San Diego 45,032,479 60
Kristian Beckwith, U Colorado 38,884,415 66
Patrick Fragile, College of Charleston 38,223,664 68
All 64 Others 456,883,341
Atmospheric Sciences - 1,509,180,022
PI, Institution
NUs Rank
Homayoun Karimabadi, UC San Diego 566,654,690 3
James Kinter, COLA 204,125,064 13
Amy McGovern, U Oklahoma 194,388,010 15
Jon Linker, Predictive Science, Inc. 169,502,106 17
Cecilia Bitz, U Washington 149,774,821 20
Frank Bryan, NCAR 140,617,805 22
Fuqing Zhang, Penn State U 99,263,199 30
Ming Xue, U Oklahoma 60,618,290 46
Nikolai Pogorelov, U Alabama, Huntsville 39,230,549 65
All 42 Others 127,474,964
Chemical, Thermal Systems - 705,619,623
PI, Institution
NUs Rank
Daniel Bodony, UIUC 92,044,666 31
Stephen Pope, Cornell U 79,483,414 36
Olivier Desjardins, U Colorado 74,007,481 40
George Karniadakis, Brown U 56,358,256 49
Rajat Mittal, Johns Hopkins U 47,801,999 55
Madhusudan Pai, Stanford U 38,363,095 67
Kenneth Jansen, U Colorado 29,716,357 81
Tengfei Luo, MIT 26,382,524 87
Krishnan Mahesh, U Minn 21,218,059 104
Rigoberto Hernandez, Georgia Tech 19,726,039 109
Antonino Ferrante, U Washington 15,620,546 127
Andreas Heyden, U South Carolina 15,493,910 129
Prosenjit Bagchi, Rutgers U 14,115,376 134
All 71 Others 179,349,917
Materials Research - 701,534,288
PI, Institution
NUs Rank
Juana Moreno, Louisiana State U 84,569,902 34
Chris Van de Walle, UC Santa Barbara 48,495,399 54
Stephen Paddison, U Tennessee 47,312,719 57
Ivan Oleynik, U South Florida 37,949,397 69
James Chelikowsky, UT Austin 37,422,656 70
Stefano Curtarolo, Duke U 36,784,789 74
Lubos Mitas, NC State U 28,679,061 83
Dallas Trinkle, UIUC 26,921,164 85
Marvin L. Cohen, UC Berkeley 24,977,598 91
Jacek Jakowski, U Tennessee 24,071,540 93
Boris Yakobson, Rice U 19,850,871 107
Gerbrand Ceder, MIT 19,007,395 110
Bilge Yildiz, MIT 18,783,043 111
Peter Cummings, Vanderbilt U 16,399,990 122
John D. Joannopoulos, MIT 16,098,219 125
Dane Morgan, U Wisconsin-Madison 14,480,280 133
All 96 Others 198,965,451
Chemistry - 605,512,370
PI, Institution
NUs Rank
Gregory A. Voth, U Chicago 253,092,527 7
Carlos Simmerling, SUNY Stony Brook 70,610,828 41
Guillermina Estiu, U Notre Dame 41,112,637 64
88

Arun Yethiraj, U Wisconsin-Madison 22,711,862 97
B. Montgomery Pettitt, U Houston 21,431,386 101
Kendall N. Houk, UCLA 15,505,351 128
Jennifer Wilcox, Stanford U 14,050,308 135
All 123 Others 166,707,676
Advanced Scientific Computing - 229,391,250
PI, Institution
NUs Rank
Ali Uzun, Florida State U 115,023,030 26
Bruce Boghosian, Tufts U 50,598,758 52
Stefan Boeriu, UC Santa Barbara 21,263,524 103
Martin Berzins, U Utah 12,905,492 142
Xiaowen Wang, UCLA 8,646,245 183
Richard Loft, NCAR 8,094,723 188
All 33 Others 12,438,757
Biological and Critical Systems - 217,030,681
PI, Institution
NUs Rank
Thomas Jordan, USC 203,515,374 14
Tony Keaveny, UC Berkeley 10,012,261 160
All 3 Others 949,158
All Others - 471,977,749
8.10.4 Usage by Institution
PI, Institution (Discipline)
NUs Rank
Mark Miller, SDSC (Environmental Biology) 43,578,333 62
Clinton N. Dawson, UT Austin (Mathematical Sciences) 37,173,926 71
Narayana Aluru, UIUC
(Cross-Disciplinary Activities) 36,860,594 72
Leung Tsang, U Washington (Earth Sciences) 34,595,141 76
Omar Ghattas, UT Austin
(Earth Sciences) 32,853,585 78
Baylor Fox-Kemper, U Colorado (Ocean Sciences) 25,077,517 90
Morgan Moschetti, USGS
(Earth Sciences) 16,987,228 119
Alexa Griesel, SIO (Ocean Sciences) 15,990,957 126
Ronald E. Cohen, CIW
(Earth Sciences) 12,465,931 143
Shanhui Fan, Stanford U (Electrical and Comm. Systems) 11,473,088 151
Lynn Wright, SURA
(Ocean Sciences) 11,179,288 154
Liwen Shih, U Houston-Clear Lake (Computer and Computation Research) 9,800,482 164
All 141 Others 184,399,738
Users from 319 institutions across the U.S. and abroad were associated with the usage delivered by
TeraGrid resources. The top 47 institutions (by NUs consumed) were responsible for 80% of the total
NUs delivered. Table 8-3 lists the 319 institutions, the number of users responsible for the usage at each
site, and the NUs consumed.
Table 8-2. TeraGrid Usage by Institution, Q4 2010
Institution
Users NUs
Indiana U 14 1,209,917,563
UIUC 241 947,274,596
NCAR 18 803,008,614
U Washington 35 773,850,669
Jefferson Lab 1 583,245,165
LANL 9 575,785,646
U Utah 22 379,976,674
BNL 1 326,051,936
U Colorado 47 224,511,529
Princeton U 19 223,859,648
UC San Diego 90 205,965,605
U Chicago 43 187,560,066
U Wisconsin-Madison 41 184,085,630
SUNY Stony Brook 15 177,031,382
NREL 10 170,118,242
U Kansas 15 161,622,666
Louisiana State U 41 159,911,762
U Iowa 16 155,173,281
UC Santa Barbara 45 149,052,069
U Maryland, College Park 25 147,531,049
Atmospheric Tech Services Co. 1 142,054,709
U Arizona 9 136,287,533
Michigan State U 11 135,586,126
Georgia Tech 39 130,820,561
U Oklahoma 39 129,222,144
Temple U 14 121,647,604
CMU 23 117,688,814
Florida State U 4 115,054,042
U Virginia 19 114,394,924
MIT 40 114,175,543
Arizona State U 14 110,404,033
Caltech 28 110,398,424
Boston U 18 108,077,635
U Florida 23 107,731,519
SDSC 11 107,401,874
Cornell U 36 107,279,390
Penn State U 15 105,150,433
UC Berkeley 49 101,429,549
Predictive Science, Inc. 2 96,627,908
University College, London (United Kingdom) 10 88,686,385
Purdue U 41 83,293,637
UT Arlington 6 82,759,053
Vanderbilt U 13 82,375,139
UNC 11 75,952,289
Stanford U 26 74,903,474
Drake U 2 74,019,013
College of Wm & Mary 6 73,457,068
Brown U 17 69,772,189
Institution
Users NUs
UT Austin 31 69,228,821
MPI Gravitationsphysik (Germany) 3 68,792,486
Duke U 3 67,436,232
UC Davis 35 66,442,525
U Wyoming 1 63,923,890
Florida Atlantic U 5 61,836,421
Lock Haven U 3 57,808,300
U Tennessee 17 56,636,866
U Rochester 6 53,664,639
UCLA 42 52,849,964
George Washington U 1 52,431,067
Johns Hopkins U 37 52,181,285
U Notre Dame 9 52,088,050
Baylor College of Med 3 50,059,787
Harvard U 16 47,396,464
USC 8 46,285,740
U Pittsburgh 31 45,881,069
RIT 6 45,782,967
Albert Einstein Institute (Germany) 5 45,400,150
Weill Cornell Med College 4 44,136,642
U South Florida 9 44,067,247
UC Irvine 15 40,780,159
U Alabama, Huntsville 7 39,440,356
NYU 13 38,269,787
College of Charleston 4 38,223,664
U Minn 7 33,739,652
Columbia U 19 29,373,799
U Kentucky 5 29,092,242
COLA 2 28,831,540
U Illinois, Chicago 18 28,606,508
U New Hampshire 8 27,452,549
U Michigan 32 25,826,136
SIO 3 24,684,192
Slovak Acad of Sci (Slovakia) 2 24,591,584
Tulane University 5 23,124,481
San Diego State U 6 22,929,779
Drexel U 12 22,909,409
NC Central U 2 21,605,418
Colorado College 1 21,333,888
RPI 8 21,302,205
U Illes Balears (Spain) 1 21,013,662
Duquesne U 7 20,948,678
Texas A&M 6 20,900,934
ANL 2 20,681,907
Inst HEP (China) 1 20,652,667
USGS 2 20,588,221
Rice U 5 20,020,985
NC State U 24 19,046,770
89

Institution
Users NUs
Toyota Tech Inst, Chicago 3 17,118,826
Desert Res Inst 1 16,858,090
U Houston-Downtown 2 16,645,898
Rutgers U 16 15,970,517
UT El Paso 13 15,941,729
Missouri S&T 6 15,805,150
U South Carolina 6 15,594,044
U Babes-Bolyai (Romania) 1 15,315,566
Washington U 23 14,095,068
Perimeter Inst (Canada) 1 13,197,121
University of Sussex 1 13,072,599
Lehigh U 3 12,525,876
CIW 3 12,465,930
U Toronto (Canada) 2 11,930,190
LLNL 3 11,416,587
Iowa State U 9 11,392,361
Wesleyan U 4 10,878,061
AMNH 4 10,789,837
S-Star Alliance 2 10,332,690
U Pennsylvania 6 10,303,691
Univ of Vienna 1 10,158,154
Monash U (Australia) 5 10,147,380
U Delaware 15 10,047,796
U Texas Hlth Sci Ctr Houston 7 9,921,915
U Tenn, Oak Ridge 3 9,917,531
WPI 2 9,910,015
Rockefeller U 1 9,879,235
U Houston-Clear Lake 1 9,800,482
U Central Florida 5 9,442,235
Mt Sinai Sch of Med 4 9,345,555
CWRU 2 8,582,542
SUNY at Binghamton 4 8,063,447
Michigan Tech 33 7,922,384
U Jena (Germany) 2 7,791,485
U Heidelberg (Germany) 2 7,641,263
Yale U 5 7,246,761
UC Merced 2 7,054,908
Aerospace Corp. 1 7,026,930
UC Riverside 4 7,008,793
Colorado State U 11 7,001,508
Venter Inst 1 6,811,981
Polish Acad of Sci (Poland) 2 6,489,834
UT San Antonio 4 6,308,729
Thomas Jefferson University 1 6,141,766
Georgetown U 6 6,115,811
CSU-Fullerton 2 6,084,006
UC San Francisco 7 6,039,313
Fermilab 1 5,954,681
Northwestern U 11 5,800,669
U Mass, Amherst 11 5,609,537
Montana State U 6 5,221,324
Oregon State U 6 5,216,266
Indiana U, Kokomo 2 4,894,259
Florida A&M U 3 4,884,468
UNLV 2 4,867,971
U Maryland, Baltimore Co 13 4,775,739
Corning, Inc. 1 4,714,311
U Houston 3 4,679,787
St Louis U 7 4,678,418
University of Montana 3 4,653,138
US DoE 1 4,415,527
U New Mexico 3 4,150,006
UNC System Office 1 4,081,758
Oxford U (United Kingdom) 1 4,003,403
Washington State U 8 3,661,784
LSST Corp. 2 3,396,236
Naval Postgrad School 2 3,380,823
Chosun U (South Korea) 1 3,256,671
Central Michigan U 2 3,218,427
IIMCB (Poland) 1 3,098,208
U Mass, Dartmouth 3 2,988,924
U Zurich (Switzerland) 2 2,981,723
UT Southwestern Med Ctr 2 2,762,290
SciberQuest Inc 1 2,716,447
Institution
Users NUs
Boise State U 3 2,582,013
Old Dominion U 2 2,448,650
NIST 1 2,322,942
U Nebraska, Lincoln 1 2,320,162
U Hawaii, Honolulu 1 2,195,680
U Puerto Rico System 9 2,193,279
PSC 3 2,164,350
Brandeis U 6 2,155,126
Clark Atlanta University 2 2,076,030
Eastern Michigan U 4 2,064,236
Kansas State U 5 1,998,763
NETL 1 1,888,765
CSU-Long Beach 1 1,768,921
U Vermont 1 1,660,730
U Cincinnati 3 1,644,887
San Francisco State U 1 1,642,179
Ill. Geological Survey 1 1,639,597
U Nevada-Reno 1 1,629,641
Institute for Advanced Study 1 1,626,099
University of Wisconsin-Eau Claire 2 1,599,091
ORNL 12 1,595,000
U Georgia 8 1,481,844
Bucknell U 1 1,374,194
Emory U 7 1,364,824
Long Island U 3 1,317,116
Kingsborough Comm College 2 1,306,818
CSU-Northridge 3 1,303,083
Florida Tech 3 1,301,897
U North Dakota 8 1,272,509
Northeastern U 4 1,245,598
Lamar University-Beaumont 2 1,160,282
Virginia Tech 7 1,078,605
SUNY at Albany 3 1,073,476
Princeton Plasma Physics Lab 1 1,063,815
Virginia Commonwealth U 1 1,027,087
Albany College of Pharmacy & Health Sci 2 1,016,926
Clarkson U 1 1,016,256
La Jolla Bioeng Inst 1 927,707
Wayne State U 4 904,128
UC Santa Cruz 4 893,570
UT System 1 847,592
Southwest Res Inst 3 841,579
Computational Sci & Eng 2 828,196
New College of Fla. 1 826,954
U Michigan, Flint 1 656,518
U Oregon 6 602,263
U Paris 1 587,040
ESSC (United Kingdom) 1 576,754
U Puerto Rico, Mayaguez 2 555,170
CUNY 2 546,644
Bangor U (United Kingdom) 2 518,043
U Hawaii, Manoa 4 509,838
U North Texas 2 501,365
WHOI 1 473,179
LBNL 2 468,742
Auburn U 5 443,256
U Arkansas 1 434,564
Clark U 1 408,661
Santa Clara U 1 402,033
Colorado Sch of Mines 2 395,388
Einstein College of Med 1 374,052
NASA Langley 1 364,696
US Army 1 342,581
US FDA 2 340,654
SD State U 2 333,678
Southern Methodist U 2 331,404
Kent State U 1 300,034
Barnard Coll 1 295,275
Vienna U of Technology (Austria) 2 288,265
Washington U School of Med 1 282,344
Oregon Health & Sci U 2 281,518
Vorcat, Inc. 1 269,436
Albany State U 2 241,109
SIU Carbondale 1 236,143
90

Institution
Users NUs
BRI City of Hope 1 221,516
Ursinus College 1 220,499
Wichita State U 2 200,544
Bryn Mawr College 1 194,120
Ohio U 2 189,413
Dartmouth College 2 172,468
U Maryland School of Medicine 2 163,026
U Alaska, Fairbanks 1 161,490
Natl Space Sci & Tech Ctr 1 157,459
SUNY ESF 2 149,160
Queens U, Belfast (United Kingdom) 1 141,696
Alfred University 1 120,718
U Maine 1 113,646
Keldysh Inst of Applied Math (Russia) 1 112,734
RENCI 1 98,145
N Ga College & State U 5 84,916
Climate Central 1 73,975
Francis Marion U 1 72,880
UNC, Charlotte 1 71,942
Monell Chemical Senses Ctr 1 68,266
Jackson State U 2 65,952
TU IImenau (Germany) 1 48,414
IIT 2 48,267
KAIST (South Korea) 1 46,701
Dickinson College 3 46,669
NJIT 1 45,487
U Split (Croatia) 1 43,575
CTC 1 43,397
Bloomsburg U 1 39,959
Brigham and Women's Hospital 1 37,286
Macalester College 3 35,752
Mississippi State U 3 35,397
PNL 1 32,836
Marquette University 3 22,496
Cold Spring Harbor Lab 1 19,892
Siena College 4 15,554
SUNY at Buffalo 3 14,139
Illinois State U 1 13,891
CUNY City College 2 12,005
Marian College 1 11,893
Oklahoma State U 3 8,607
Connecticut College 1 7,747
U Hawaii Research Corp 1 7,512
JPL 1 6,993
U Minn, Duluth 1 6,953
U Provence (France) 1 5,481
U Detroit Mercy 2 5,189
Fort Hay State U 1 4,828
Kitware 1 4,699
WV State College 1 4,295
HDF Group 1 4,070
Tufts U 1 3,191
Texas State U 1 2,964
Rose-Hulman Inst of Tech 1 2,563
Howard U 1 2,313
U Mass, Lowell 1 2,185
Clemson U 2 1,569
NW Research Associates 2 564
U Pittsburgh Sch of Med 1 564
ETH Zurich (Switzerland) 1 324
SLAC 1 297
U Paul Sabatier (France) 1 283
Ohio State U 1 252
Julich Supercomputing Centre (Germany) 1 248
Grand Valley State U 1 229
Allegheny College 1 142
National U 2 76
SSSU (India) 2 69
Salk Institute 1 56
U Miami 1 40
CENTRA - Inst Sup Tecnico (Portugal) 1 39
ND State U 1 17
Widener U 1 11
Inst Bioorganic Chem, Polish Acad Sci (Poland) 1 2
Institution
Users NUs
NASA Ames 1 2
91

9 Evaluation, Monitoring, and Auditing
9.1 XD – TIS
9.1.1 Technology Identification and Tracking
Update
The initial version of the XTED database and UI was released in time for SC 2010 with the
functionality outlined below. Having achieved the base set of functionality, we are have been
working towards implementing the next batch of use cases extending the functionality of the
XTED website. In addition to various bug fixes and schema modifications resulting from
feedback of the initial release, a conversion of the UI code from a servlet to a portlet
implementation was undertaken in order to better integrate with the existing features of the
Liferay portal server. We worked with the Technology Evaluation Process (TEP) group to
develop use cases to support the functionality needed by that part of the TIS project. Finally we
have begun similar conversations with the Usage, Satisfaction, and Impact Evaluation team to
identify and document the use cases needed to support their activities, to be completed in Q1
2011.
Development
Use Cases
The following provides descriptions of the functionality released during the quarter and the
functionality planned for the next release. Additionally, we highlight the use cases developed in
conjunction with the TEP team.
Released for SC 2010
1. Search Technologies - Any user can view the list of technologies available via the XD
user portal. Ability to filter by keywords.
2. View Technology Details - Any user can view detailed information about a
technology.
3. Create Technology Record - An authenticated user adds a new technology which will
be reviewed by TIS staff.
4. Edit Technology Record - An authenticated user with role of PI or Point of Contact
(POC) for a specific technology can edit an existing technology.
Next Release (Feb 2011)
1. Approve Technology - TIS staff member must approve new technologies before they
become publicly viewable.
2. Request Technology - Allows an authenticated user to request a technology that may
or may not exist in XTED.
3. Respond to Request - TIS staff responds to a technology request, adding a new
technology entry if applicable with a follow-up email sent to requester.
92

4. Review Suggested Keywords - TIS staff cleans up user submitted keywords to create
an evolving controlled vocabulary.
5. Delete Technology - Technology record creators and TIS staff can soft/hard delete a
record.
Technology Evaluation Process Use Cases (Q1 2011 release)
1. Add Review Content – The Review Contact adds the review content causing the Tech
status to be updated from 'Under Review' to 'Reviewed'
2. Edit Review Content - Update information for a Reviewed Tech
3. Assign Reviewer - Assign or reassign a 'Review Contact' for an existing Tech record
4. Update Review Status - Set Review status from 'Not Reviewed' to 'Selected for
Review' or 'Under Review'
5. TIS Staff Create Technology Record - TIS can create a tech record that doesn't exist,
cannot edit the 'general details'.
9.1.2 Technology Evaluation Laboratory
The Technology Evaluation Laboratory is composed of a combination of reserved resources
housed at TACC, the FutureGrid systems, and resource provider (RP) systems. These systems
represent the HPC computing environment found in XD and will facilitate the Technology
Evaluation Process (TEP) conducted by the TIS team. The test cluster housed at TACC is
currently being installed and prepared to emulate as closely as possible the different environments
found in the TeraGrid today with plans to extend the environments to match those systems found
in XD as well. This system provides the TEP group unrestricted access to hardware when other
systems are either not available or are unable to perform the required testing. Underway is the
creation of documentation explaining how the TEP team will utilize this hardware. The Track 2D
grid test bed or FutureGrid brought most of their systems online in the last quarter and now have
the ability to support multiple virtual machines (VMs), some which are maintained by the
FutureGrid team and others which are supplied by users. Current work with FutureGrid has
involved testing cloud technologies including Eucalyptus and Nimbus to determine strengths and
weaknesses of each to draw conclusions on which technology the TEP team will use in their
evaluations. The team is working to document the process of how we intend to use FutureGrid
for TIS activities. In addition to documentation regarding the test cluster and FutureGrid we are
compiling a list of all RP hardware available for testing, and recording information such as how
to access, points of contact, and type of access.
9.1.3 Technology Evaluation Process
The Technology Evaluation Process team is charged with developing an evaluation process for
the various software and hardware candidates that TIS will be asked to evaluate. We are also
tasked to coordinate with other groups as necessary to achieve our goals and theirs.
Progress on the Process
The Technology Evaluation Process (TEP) team has developed a formal process for testing
candidate software. It is under final review by all members of TIS.
Consisting of 6 steps:
• Develop an initial test plan
93

o
o
A brief description of the team members involved, time frames and duties
A listing of the requirements the software needs to meet
• Perform an initial study
o
A subset of the team will perform an in depth study of the candidate
• Create initial documents
o
The same subset of the team will generate several documents that will be used in
the testing and insertion processes
• Define the final test plan
o
o
• Perform the tests
o
A formal test plan will be created by all team members that includes metrics,
scripts and features to be evaluated
Subject matter experts (SME) will be used as available so that the tests are
appropriate and accurate
The testing will be performed on multiple systems by more than one team
member for rigor
• Generate the final reports
o
o
Internal reports will be generated on the testing. External reports will be
generated as the testing of the candidate merits
Any final report will be publicly available through the XTED database
The Plan to do an evaluation of the TEP
The evaluation tests were completed and demonstrated some areas that needed improvement.
Those changes have been made to the process. We have coordinated with the TIS group on the
information they require, and have documented that. We are coordinating with the XTED group
on the changes that we need in the database.
Goals
Our goal for this quarter is to have the formal version of the process accepted by all of TIS in
January and start testing real candidates thereafter. We are also coordinating on the software
candidate identification process with the rest of the group such that we receive a preliminary set
of candidates. We will also obtain a list of available resources and personnel for testing. During
this quarter, we will need to note the rate of testing, so that we can accurately estimate the amount
of future testing that can be done with TIS resources. We will also identify any changes needed in
the process which are noted during “real-world”testing.
9.1.4 Technology Insertion Support
The Technology Insertion Support (YNT) team facilitates transition of supported and
recommended technologies from evaluation to deployment by TeraGrid and XD service providers
and users. YNT works with the evaluation teams to ensure that deployment, design, installation,
configuration, maintenance, and usage methods for supported and recommended technologies are
sufficiently documented, together with validation tests and usage examples.
94

YNT publications are intended for use by systems administration staff at TeraGrid and XD
service providers to guide deployment of technologies for use with their resources. For
technologies intended for end-user installation and use, YNT publications are intended for use by
systems administration staff supporting end-users. Documentation for usage examples will be
prepared for use by end-users directly.
The YNT team has drafted an initial definition for deployment objects, which include components
necessary to sufficiently document the deployment design, installation, configuration,
maintenance, testing and usage methods for a supported and recommended technology, along
with its deployment profiles, which document deployment instances and their current status per
service provider. Deployment profiles include defined deployment roles and associated contact
information per deployment object per service provider.
The YNT team continues to work with the evaluation teams to develop procedures and
instrumentation in the evaluation process to support documenting deployment object components
including validation tests and usage examples. Additional useful metadata from the evaluation
process, such as time, expertise and effort required to perform each evaluation will be collected
during evaluations to inform prospective implementers regarding deployment effort and expertise
required. The YNT team continues to review existing deployment models in TeraGrid and other
CI projects, for guidance in establishing appropriate YNT procedures.
To assist the overall evaluation process, the YNT team has initiated work on a metric to guide
selection and to establish priorities among technology evaluation candidates. The objective of this
metric is to establish a consistent procedure by which to vet candidate technologies for scope,
applicability, community interest and availability of required evaluation resources and expertise.
The calculated metric value for each candidate technology will rank it relative to all others under
consideration. The rank-ordered list of technologies will establish the work priorities for the
evaluation teams, and will be reviewed and updated regularly to ensure that teams are working on
the most important technology evaluations, as metric factors change over time.
9.1.5 Usage, Satisfaction, and Impact Evaluation
Led by the TIS user advocate, the Usage, Satisfaction and Impact Evaluation team is charged
with engaging the user community in the evaluation and testing process, by various methods
including focus group teleconferences aimed at finding out what functional requirements are not
being met. We have conducted additional meetings with users representing groups and interested
in topics we identified last quarter, continuing the process of learning how best to prepare,
organize and conduct such meetings, and how to feed their outcome into the TIS identification
and tracking, evaluation and testing activities described above.
As a result of the 5 meetings with groups of active users and campus champions that we have
conducted so far, we have formulated specific requirements and some candidate technologies to
be evaluated in the areas of wide area file transfers, workflow engines, and single sign-on to
Teragrid resources.
We have focused on the best ways to integrate community input into the TIS process and to
enable community input and feedback via XTED. In Figure 9.1, which was used as a basis for
discussion at the TIS meeting held at SC10, red denotes user roles that must be enabled by XTED
(green represents TIS staff roles and blue represents technology developer community roles). The
figure illustrates that XTED’s role-based access controlled tools should allow our staff to enter
the requirements obtained via our user engagement processes, as well as allowing (in a later
95

stage) users to enter requirements on their own. Furthermore, we may ask some of the users to
participate in the evaluation and testing of the technologies that, based on TIS staff evaluation,
have emerged as the leading candidates to fulfill the functional requirements these users have
identified as being critical to their work. Finally, the community of current and potential
TeraGrid/XD users (as well as the developer community) needs to be empowered to comment on
the evaluation and testing results we will publish via XTED. The process is cyclical, reflecting
the dynamic nature of the XD cyberinfrastructure, of its stakeholders’ needs and expectations,
and of the potentially relevant technologies.
Figure 9.1: Stakeholder driven TIS processes to be supported via XTED
9.2 Quality Assurance
Between October and December 2010, the QA group performed testing of the new GridFTP 5
release. GridFTP5 was deployed to virtual machines on FutureGrid’s Sierra (UCSD) and Foxtrot
(University of Florida) resources and verified that several of the new features, such as data set
synchronization and the offline mode for the server, worked as expected. No major problems
were detected in this testing, though a bug related to the new dataset synchronization feature was
reported. The results are summarized on the TeraGrid Wiki at
http://www.teragridforum.org/mediawiki/index.phptitle=GridFtp5_Testing. GridFTP 5 was also
deployed to all TACC machines without any problems.
As GRAM5 has not yet been recommended for production deployment across TeraGrid,
members of the QA group continued to debug it. Two issues related to run away globus-jobmanager
processes and a problem using jobtype=multiple were raised by TACC and are being
investigated by Globus developers. A problem with PBS.pm not allowing GRAM MPI jobs to
execute was resolved at LSU.
96

In response to feedback from the Science Gateway group, QA is also investigating what is
required to submit Globus MPI jobs on various TeraGrid machines. We’d like to use these
findings to improve documented user examples and would ideally like to have standards for MPI
jobs and an automated tool for submitting them. Also, based on correspondence with the
CIPRES Gateway project, documentation will be drafted for using GRAM clients to launch jobs.
Other QA activities included discussions with members of the CUE regarding details of their
deployments and how to best test them. A basic CUE test was written and more will be created
when CUE deployments are registered in IIS. Work also continued to create a sustainable
process for collecting and analyzing usage data in order to determine which services are most
important to users. A manual graphical analysis was made of existing process accounting reports,
which could be used as an example for future automated reports. Finally, a new test was written
for the Karnak metascheduling service.
9.3 XD – TAS
In this TAS quarterly progress report we discuss infrastructure put into place to facilitate software
development, organizational meetings and technical progress made toward initial software
releases.
9.4.1 Technical Progress
XDMoD User Interface:
Substantial progress has been made on the XDMoD interface in terms of the underlying
infrastructure, look-and-feel of the interface, and its functionality. We begin with a discussion of
the underlying infrastructure (programming constructs).
The XDMoD portal is a web-based application developed using open source tools including the
LAMP (Linux, Apache, MySQL, PHP) software stack and the ExtJS user interface toolkit.
XDMoD is developed using the Model-View-Controller software design pattern where the View
is implemented entirely using the ExtJS user interface toolkit to provide the look-and-feel and
advanced features of a desktop application within a web browser. The Controller serves as a
router where service requests are received and sent to the appropriate Model (e.g., charting, report
building, application kernels, etc.) for handling. Multiple Controllers can be created to handle
specific types of requests such as the generation of charts, creation of customized reports, and
queries for information such as available resources to feed the user interface. We have chosen to
use authenticated RESTful services as the interface to all backend data services (Controllers and
Models) and to completely de-couple the user interface (View) from the backend services. This
de-coupling allows us to immediately achieve the goal of a providing a service API that can be
utilized by 3 rd party data ingestion or display tools such as Google gadgets or other portal
infrastructures.
ExtJS is an advanced cross-browser JavaScript user interface toolkit that provides a rich set of
functionality that one would typically expect to find in a desktop application. This includes dragand-drop
functionality, event handling, callbacks, and customizable widgets. This immediately
97

provides XDMoD developers with a wide variety of components from which to choose when
developing the portal. This includes tree views for organizing hierarchical data, tabbed display
panels for organizing and displaying different types of information, a mechanism for easily
utilizing RESTful services, and components for automatically parsing data returned from those
services.
Figure 1 shows the login page for XDMoD. The plot window in Figure 2, a screen capture of the
XDMoD portal interface, shows a plot of the Total SU usage by NSF directorate for 2010, and is
available under the TG Usage tab (which is highlighted). To the left of the plot window is the
Chart Menu Window, which shows the wide range of plots available, including plots of TG
utilization broken down by Job Size, Resource Provider, NSF Directorate, Principal Investigators,
Institutions, etc. Some of the important features of the portal are described below.
Figure 1. XDMoD Portal login in page.
98

Figure 2. XDMoD Portal
The 9 Navigation Tabs along the top of the window have the following functionality.
1. My Summary: Displays an overview of the individual user’s jobs.
2. TG Summary: Displays a system-wide overview of all users jobs.
3. My Usage: Displays a large number of metrics related to the individual’s usage.
4. TG Usage: Displays a large number of metrics related to the system-wide TG
usage.
5. Allocations: Displays a summary of the status of the individual’s TG allocations.
6. App Kernels: Provides information on application kernel performance on various
resources.
7. Report Generator: Provides a mechanism for generating user specified reports.
8. User Like Me: Provides an evaluation of resources based upon user specifications.
9. Search: Allows authorized users to search for data on users, institutes, etc.
Other key functional features include: the “Add Charts” button that adds the displayed chart to
the report queue, the “Report Queue” button that aids the user in managing and exporting reports
that are a compilation of user selected charts, the “Export” button that allows plot data to be
output in a variety of formats (XLS, CSV, XML, PNG), the “Chart/Dataset” button that allows
the user to toggle between the display of a given chart or the data set used in its generation, the
Date Selection pull-down menu that allows the user to select different time ranges for the desired
plot, a search feature that helps the user in locating charts, and a series of buttons that allow the
user to resize the displayed charts and control which information fields are visible.
Based on the feedback received at SC10 for the beta version of XDMoD as well as input from the
TG Quarterly Meetings, we have substantially revised the functionality planned for the initial
release of XDMoD 1.0, which is scheduled for the summer of 2011. For example, XDMoD has
been enhanced by the addition of time-based plots similar to those included in the TG Quarterly
Reports. Plots such as Number of Jobs, Total CPU Consumption, and Total SU Charged over
99

user specified time periods are now readily available. A series of three example plots of TG
utilization are shown below as Figures 3-5.
Figure 3. Total TG CPU hours on all TG resources over a one-year period by resource. Hovering
over a given color on the plot brings up a tab that indicates which resource corresponds to that
particular color.
Figure 4. Time history of all TG resources from 2006-2010 in terms of total CPU hours delivered
by resource.
100

Figure 5. Total number of TG jobs run over a two-year period by resource.
In addition to the time history type plots, a series of new data plots have been added including
utilization plots by NSF status, by person and by institution. Examples are given below for Total
CPU usage by NFS status (Figure 6), Total CPU usage per Principal Investigator (Figure 7) and
Total CPU usage per institution (Figure 8). In addition to the Pie charts, the user will be able to
choose to view the data as a bar chart or in tabular form, thereby facilitating their inclusion in
reports. Figure 8 also shows the entire XDMoD Portal interface (as opposed to just the plots
shown in Figures 6 and 7) and is included to give a better indication of the rich feature set of
metrics being developed within the portal.
101

Figure 6. Total CPU Consumption by NSF Status obtained using the XDMoD Portal.
Figure 7. Total CPU Consumption per PI obtained using the XDMoD Portal.
102

Figure 8. Total CPU Consumption per institution. Included in the figure is a screen shot of the
entire XDMoD portal
Role-based Access to XDMoD: The various roles that will be defined in XDMoD were specified
for the initial XDMoD 1.0 release this summer at TG 11. It was decided to have 5 distinct roles:
NSF Program Officer, Center Director, Principal Investigator, User and Public. Further detail on
the access levels assigned to each role are given in the attached Appendix 1. It is anticipated that
the specific information available in each of the roles will be further refined based on discussion
with NSF and XD leadership.
XDMoD Additional Progress: Developing a client for a complex and fast-changing software API
can be challenging. To assist 3 rd party developers (as well as internal developers needing to
interface with XDMoD components), we have created a tool to make this task easier. The
XDMoD REST API Call Builder, shown in Figure 9, provides a mechanism for a developer to
iteratively and hierarchically construct a RESTful API URL to perform their desired query or
action. Using a programming language feature called Reflection, the API Call Builder inspects
the publically available API interfaces for all available Controllers and Models to guide a
developer through the construction of a valid API URL. Starting with the selection of the
appropriate controller, the developer is provided with a list of available choices for each level in
the URL hierarchy and steps through the process of constructing a valid URL for the desired API
call. The URL is initially displayed with each required component in red and changes these to
green as the developer makes the required choices. A test is included that checks for errors and
the proper completion of the call and displays the call output. This tool is currently being used
for internal diagnostic usage, and still being enhanced for eventual use by external developers,
such as the developers of the TG/XD User portal.
103

Figure 9. Prototype tool to aid developers in the construction of valid RESTful API calls.
XDMoD Data Warehouse:
The TAS development and production hardware has been installed and is backed up to an off-site
location. Virtual machines have been implemented for the various server/web components of the
TAS project.
The TGCdb has successfully been mirrored at CCR from PSC allowing for fast and efficient
ingestion of all data stored within the TGCdb into the XDMoD data warehouse and aggregation
to support the plethora of charting options available. A list of desirable improvements/data
elements to the centralized database is being prepared for submission to the XD transition team.
The XDMoD data warehouse has been redesigned to support new functionality and is defined as
a simple star schema as shown in Figure 10. A dimensional approach to data storage, a star
schema is relatively easier to understand compared to the normalized approach. Upon ingestion,
the transactional data of various data sources (i.e. TGCdb) are partitioned into “facts” or
“dimensions”. Dimensions are the reference information that gives context to facts. For example
a TG job transaction can be broken up into facts such as the job_id and total CPU time consumed,
and into dimensions such as resource provider, field of science, funding allocation, and the
principal investigator of the project. The schema makes it easy to understand and query the data
pertaining to jobs logged in the TGCdb. However, we still need to optimize for performance
when it comes to aggregating data over a custom date range. For this purpose, the star schema is
further aggregated into subsequent star schemas where aggregation over time is added as an extra
dimension as shown in Figure 11. In this phase, we aggregate data over its complete duration by
day, week, month and quarter. Keeping multiple resolutions of data allows for faster queries
when aggregating or querying data over long periods of time.
104

Figure 10: XDMoD data warehouse schema
105

Figure 11: Additional time based aggregations in the XDMoD data warehouse.
Application Kernel Development:
Application kernel work for the beta release at SC10 was initially focused on Computational
Chemistry/Biophysics. Application kernels currently under development that were discussed at
SC10 include: Amber, GAMESS, NAMD, NWChem, Intel MPI, NAS parallel benchmarks, and
the HPC Challenge benchmark suite. A “USER LIKE ME” view has been prototyped to allow
users without a detailed knowledge of TG/XD resources quickly to see recommendations and
application performance data based on their field and desired application profile. As shown in the
XDMoD window below, thirteen application kernels are presently being run continuously on
local (CCR) HPC clusters. Figure 12 below shows AMBER-based application kernels run on 8,
16 or 32 processors.
106

Figure 12. AMBER-based application kernels run on 8, 16 or 32 processors.
Work on the application kernels was focused on planning for the implementation of an initial
deployment of kernels on Future Grid and subsequently one or two sites on TG/XD to be
synchronized with the release of XDMoD 1.0. Present work has been centered on the best
scheme to track different versions of each kernel and to account for changes in how each kernel
runs on the various resources when a variety of changes are made including both hardware and
software modifications.
Planning for the implementation and development of an initial set of kernels on Future Grid was
done in collaboration with Professor von Lazewski. FutureGrid will provide a useful test-bed for
development of the application kernels. While this deployment will not offer a larger scale for
application processing over the current set of local computing resources, it will test our
development and implementation methods. An xdtas account has been obtained for FutureGrid
and a preliminary plan has been determined to run a set of application kernels on the system.
Custom Report Builder/University of Indiana Subcontract:
Professor Gregor von Lazewski (Co-PI) and Dr. Lizhe Wang from Indiana University visited
CCR on 12/14/10 to 12/15/10 for a series of meetings and conferences to discuss XDMoD,
Application Kernels, and FutureGrid TAS collaboration.
107

The initial version of the custom report builder has been completed. A user may use a togglebutton
on any chart to select it for inclusion in a custom report. The report generator will utilize
an XML configuration file that describes the format and contents of the report, which will be sent
to the user via email upon report generation. Initially, a PDF version of the report containing the
title, description and image for each selected chart will be available while later versions will
expand to other output formats.
A preliminary demonstration version of the custom report builder is shown below in Figure 13.
This demonstration version features two methods for generating a report. The first method allows
a user to select individual charts as viewed in XDMoD and add them to a report queue. From the
report queue the user can then generate a report and have it delivered via email, either as a onetime
report or a regularly scheduled report. Using the second method, the user can select a
combination of resource providers, resources, and metrics to be included in the report. The report
can be immediately down loaded or sent as an e-mail if it is a one-time report or scheduled to be
sent as an e-mail if it is a recurring report on the user selected intervals.
Figure 13. Demonstration version of the Custom Report Builder.
User Needs Assessment/University of Michigan Subcontract:
108

Dr. Ann Zimmerman of the University of Michigan traveled to CCR on 10/25/10 for a full day of
meetings and conferences. She was briefed on TAS progress to date and we discussed plans for
soliciting user input for TAS products such as XDMoD and the Application Kernel Toolkit at
SC10 and beyond.
User Needs: Team members from the University of Michigan (UM) submitted their Institutional
Review Board (IRB) application for approval. Upon acceptance of the application, they will be
able to recruit and conduct research with potential users of TAS. They also developed data
collection protocols for user interviews and observations and submitted these with the IRB
paperwork. Studies of user needs will focus initially on end users of TG/XD. The sampling
strategy will include users with different size allocations, from different domains, and with
varying levels of computational expertise and experience.
9.4.2 Meetings and Events
Weekly meetings of the TAS Working Group are held every Wednesday afternoon. UB project
members attend in person with University of Michigan and Indiana University subcontractors
attending by web and audio conference. TAS personnel were very active at SC10. A description
of the presentations made at SC10 is given below.
CCR Presentations at SC10:
11/16/10 SC10 Birds-of-a-feather XDMoD presentation at SC10: Dr. Thomas R. Furlani, Dr.
Matthew D. Jones and Mr Steven M. Gallo led a BOF presentation of XDMoD at SC10. The
TAS program in general and the prototype XDMoD portal in particular was very well received.
A number of useful comments and suggestions were made by TG users at the BOF and these
suggestions will find their way into future XDMoD releases.
11/17/10 SC10 Dell booth presentation on TAS: Dr. Thomas R. Furlani gave an invited
presentation describing the TAS program, XDMoD and other CCR projects at the Dell booth.
CCR booth at SC10 features XDMoD demonstration: The CCR booth at SC10 was well
attended. The booth featured an information sheet on XDMoD and a working demonstration of
XDMoD on a laptop computer. The TG/XD users who tried the demonstration were generally
pleased with the software and they were able to offer several useful comments that will aid the
development of XDMoD.
TAS Technical Advisory Committee Meeting at SC10:
11/15/10: Meeting of the TAS Technical Advisory Committee at SC10: The TAS Technical
Advisory Committee met for the first time at SC10. Attendees included the committee members:
109

Professor Abani Patra (Chair) (UB), Dr. P.K. Yeung (CFD, Georgia Tech), Dr. Martin Berzins
(Chair, Computer Science at Utah), Dr. Craig Stewart (Executive Director, Pervasive Technology
Institute, Indiana), Dr. Richard Brower (Lattice QCD, Boston University), Dr. Sara Graves
(Director, Information Technology and Systems Center, Univ of Alabama, Huntsville), CCR
staff: Dr. Thomas Furlani (PI), Dr. Matthew D. Jones (Co-PI), Mr. Steven Gallo, Mr. Ryan
Gentner, Mr. Amin Ghadersohi, Dr. Charng-Da Lu, and Dr. Robert L. DeLeon and
subcontractors: Dr. Ann Zimmerman from the University of Michigan and Professor Gregor von
Lazewski (Co-PI) from Indiana University. Also attending John Towns (TIS PI), Kevin
Thompson (NSF), and Dr. Richard Moore (SDSC).
After brief introductory comments by Chairman Abani Patra, Dr. Furlani, Dr. Jones, Mr. Gallo
and Dr. Zimmerman provided a briefing on TAS progress to date and a demonstration of the
XDMoD Portal. A very lively and informative discussion followed and a series of comments and
recommendations were noted that will provide helpful guidance for the TAS program. A detailed
summary of the recommendations of the committee were compiled into a report. The first TAS
TAC report from this meeting is attached as Appendix 2.
9.4.3 Coordination with other TG/XD and External Projects
On Tuesday Nov. 16, at SC10, TAS members meet with TACC technical staff to discuss
XDMoD and TG portal development efforts and explore the use of common portal technologies.
During his visit to CCR, on 12/15/10, Professor Gregor von Lazewski from Indiana University
discussed TAS collaboration with FutureGrid.
9.4.4 TAS Appendices
9.4.4.1 Appendix 1: Proposed Role based Data Access Privileges for TAS
We have identified five specific roles for the XDMoD TAS interface. The purpose of this
document is to formalize the information that will be made available through XDMoD for each of
the roles.
1. Public Role (TG account not necessary but an approved XDMoD account is required)
• Overall TG Utilization broken down by resource provider, field of science, time, etc
o No user specific data
• Performance data across RP’s – performance of the TAS application kernel suite
• User Like Me (a tool to assist users in identifying the resource most appropriate for their
computational needs)
2. User Role - extends Public role and adds the following data (requires an active TG account)
110

• Personal utilization information for the authenticated user
• Drill down to user’s individual job details and allocation information
3. PI Role - extends User role and adds the following data (requires an active TG account)
• List all users under each of the PI’s allocations
• Provide detailed utilization information for all users under a particular allocation (only
view activity specific to the PI's allocations)
4. Center Director / Resource Provider / System Admin Role
• Utilization data specific to the user's associated center/resource
• Performance data specific to the user's associated center/resource
• Ability to view detailed job-level information for users of the resource
5. Program Officer Role
• No restrictions on data access
• TG Utilization
• Access to all data for a specific user
• Performance data across all RP’s
9.4.4.2 Appendix 2. Report of TAS Technical Advisory Committee
First TAS Technical Advisory Committee (TAC) Meeting Discussion Points and
Recommendations,
New Orleans 11/15/10
Attendees included:
The committee members: Prof. Abani Patra (Chair) (UB), Prof. P.K. Yeung (Georgia Tech), Prof.
Martin Berzins (Univ. of Utah), Dr. Craig Stewart (Indiana University), Dr. Richard Brower
(Boston University), Dr. Sara Graves (Univ of Alabama, Huntsville). Member of the TAS TAC
not attending SC10 meeting: Dr. Jerzy Bernholc (NC State)
TAS staff: Dr. Thomas Furlani (PI), Dr. Matthew D. Jones (Co-PI), Mr. Steven Gallo, Mr. Ryan
Gentner, Mr. Amin Ghadersohi, Dr. Charng-Da Lu, Dr. Robert L. DeLeon, Dr. Ann Zimmerman
from the University of Michigan and Professor Gregor von Lazewski (Co-PI) from Indiana
University.
Also attending John Towns (TIS PI), Kevin Thompson (NSF), and Dr. Richard Moore (SDSC).
Meeting started with a brief presentation of the Charge to the TAC and a presentation of the TAS
project including a demo of the XDMoD tool. The discussions were wide ranging and included
structural, policy and technical issues. Grouping them in these categories we have:
111

Category 1: Structural Issues:
1) TAS and TG/xD Governance: The Technical Advisory Committee deliberated on the
relationship of TAS and XD governance. Suggested mechanisms are:
• The TAC believes that TAS should provide input to XD governance rather than be part of
it.
• The TAC emphasized that it is important that TAS reports both to XD and NSF.
• TAS should be separate from XD and maintain a distinct identity (“arm’s length”).
• XD governance can request that TAS audit various aspects of XD that are of concern
and/or in need of independent oversight.
• TAS should prepare quarterly audit reports of XD activities with specific findings and
XD should be required to respond formally to these in the subsequent quarterly meeting.
2) The TAC also felt that TAS should interact closely with the XD science advisory board and
provide them with data as needed.
Category 2: Policy Issues:
1) There was an active discussion on whether it is the best use of the TG/XD resources to run
very large jobs (ones that cannot be easily run elsewhere) for which they are uniquely suited or to
run all jobs that may have scientific merit. TAS should provide data and an analysis of the job
size distribution and associated scientific productivity over the various facilities to allow NSF to
make such policy decisions.
2) Many of the TG/XD facilities have heavy non-NSF usage. One of the functions of TAS is to
indicate the extent to which the facilities are used on NSF supported programs. While TAS
probably will not be able to directly report the non-NSF usage in detail due to the lack of access
to this data at the resource providers, TAS can and should report the overall NSF level of machine
utilization.
Category 3: Technical Issues:
1) The TGCDB (TG centralized database): In general, the TGCdb records are very good, there
are some areas where data is apparently missing and other areas where there is room for added or
improved record storage. TAS reports that their initial assessment of the TGCdb finds some
inconsistency in batch job usage reporting. Three notable examples are that job exit status is not
reported, not all computational resources report usage by processor core as well as by node, and
memory per job is not recorded. Indeed, reporting is not always consistent for jobs on an
individual resource nor from one resource provider to another. Accordingly, it is the
recommendation of the TAS Technical Advisory Committee that TAS formulate a uniform
taxonomy for TGCDB and more extensive collection of resource utilization statistics.
112

2) Given the emergence of Green technology for HPC, the TAC suggests that TAS should
recommend that energy usage be reported, thereby providing a framework to measure system
performance in terms of energy consumption. The best reporting units would be energy usage per
job and energy usage per core. While potentially costly to implement in current TG systems,
solicitations for future systems should make this a requirement.
3) The TAC feels that the POPS database containing information gathered during allocation
applications, can potentially provide a unique and useful source of data for TAS e.g. publications
made using past allocation resources. TAS should utilize this data and if necessary make
recommendations on the data that is recorded, format, and data collection procedures. For
example, should the POPS data be incorporated in the TGCDB.
4) It was pointed out that practically no existing TG reporting (annual reports etc.) can be cited in
the open literature. TAS should consider either a version of the auditor’s reports that is open to
the public and/or provide information that is citable within the context of XDMoD.
5) The TAC recommends that TAS needs to access data from the TGCDB and other sources as
needed for reporting on TG/XD.
6) It was pointed out that feedback from experienced users is a very powerful method for both
helping new users learn to use the TG/XD facilities and to help to monitor how such facilities are
operating on a daily basis. Thus a social network analogous to Facebook or at least a user list
serve should be developed. TAS should investigate the feasibility of such services and possibly
assist in setting up such mechanisms but not to take the lead or the principal responsibility for
establishing and maintaining these social networking mechanisms. As an initial step in providing
users with access to data such as this, TAS should explore the feasibility of mining help desk
tickets and their resolution from the resource providers to provide user’s with a query based
system to quickly find answers to their questions. Mining the tickets could also help monitor
system performance by looking for common problems across a given resource.
7) The interaction between computational and experimental work is important. A useful metric
of the importance of computational studies is the value that the work has to experimentalists.
This information should be actively sought after by TAS.
8) TAS should mine Advanced Support for TeraGrid Applications (ASTA) files for information
and problem specific expertise and support ASTA as indicated.
9) In general only traditional CPU usage is reported. TAS should help develop mechanisms to
monitor and audit memory, storage and visualization facilities. This could be done through
developing suitable application kernels to assess system performance and recommending
additional usage reporting for these facilities.
113

10 Education, Outreach, and Training; and External Relations
The TeraGrid collective efforts have engaged at least 9,151 people through at least 162 events as
shown in the table below. The number of participants were not recorded at all events. Details on
the events are included in Appendix B – EOT Event Details.
#
Participants
# Underrepresented
people
# events
Workshops 825 435 41
Tutorials 366 36 17
Async Tutorials 1,663 32
Forum and Tours 1,078 85 33
Conference
Presentations
1,727 278 15
Demos/Fairs/Exhibits 2,945 1,670 9
Classes 156 66 7
Seminars 391 49 8
Total 9,151 2,619 162
10.1 Joint EOT-ER Outreach Highlights
US Science and Engineering Expo, October 23-24, 2010 Washington DC
On October 23-24, 2010, over a million
people flocked to Washington D.C. to
celebrate science and engineering
innovation and inventions at the inaugural
USA Science and Engineering Expo.
Staff from Indiana University, UChicago, Blue
Waters, NICS, NCSA, Shodor, and students
and educators from the DC area, joined
together in reaching out to young people to get
them interested and engaged in science and
technology. The effort was supported by a
supplemental award from OCI. The team
extends its appreciation to Irene Lombardo
and the OCI office for their support.
Pervasive Technology Institute employees
from IU volunteered in a booth co-sponsored
by the TeraGrid and Blue Waters projects.
Thousands of educators, children and their parents filled the TeraGrid tent non-stop for two
whole days. They took the time to check out a 3D movie as well as the “LittleFe” cluster
computer. iPads loaded with animated scientific visualizations and photographs of
114

supercomputers were a huge draw for little fingers which wanted to explore the easy-to-use
screens.
“It was great, because when the kids asked ‘What is a supercomputer’ we were able to show
them pictures of the Data Center at Indiana University and the new Blue Waters facility at NCSA
at the University of Illinois and also let them see LittleFe,” said IU-TeraGrid Outreach
Coordinator Robert Ping. “LittleFe is like a baby supercomputer. It’s portable and is able to be
used for training and doing demonstrations to show the computational power of a
supercomputer,” he said.
Educators loaded up their bags with NCSA coloring books and posters about supercomputing as
well as publications like the TeraGrid EOT Highlights and Science Highlights. They were
thrilled to have tangible items they could take back to their classrooms to start discussions about
computer science, supercomputing and the many fields of study it touches.
The EXPO was conceived in response to the Obama administration’s desire to encourage more
interest in Science, Technology, Engineering, and Math (STEM) careers by exposing children
and families to new technologies that are strengthening communities, building careers, and
stimulating economic growth.
The Pervasive Technology Institute’s Advanced Visualization Lab (AVL), part of the Data to
Insight Center, created the stereoscopic video shown in the booth. It is centered on real-world
applications of computational science and features current research that utilizes the TeraGrid.
The stereoscopic video allowed kids and parents alike to don 3D glasses and get a glimmer of
what it might be like to be a scientist working in the area of climate and weather.
Animations of whirling wind turbines, swirling tornadoes and inching inch-worms combined with
real-world research to show the study of climate isn’t just about the weather. It is useful in other
areas of research and study like hydrology and agriculture as well. By using the NSF funded
Linked Environments for Atmospheric Discovery (LEAD) project as the topic the public was also
able to find out how their tax dollars are put to valuable use both as research tools and as ways to
get kids interested in science and technology.
Mike Boyles a volunteer from IU and manager of the AVL said, “It doesn’t really matter what the
subject matter is of the video. The way the kids eyes lit up, their thoughtful questions and their
expressions of “wow” and “whoa” were the real value we created by showing the 3D scientific
movie.”
This wasn’t the first time masses of people gathered on the National Mall to express their
concerns and interests, but it was the first time that so many came to share their passion for
science, engineering, and technology.
“This expo is a chance for institutions like D2I and IU to show children that science and math
aren’t the scary subjects they’re sometimes thought to be, and are actually a lot of fun,” said D2I
Director Beth Plale. “Reaching out to young people and getting them intrigued in science and
technology will be critical in the coming years in order for the U.S. to be competitive in the
global scientific and economic landscape.”
115

EU-US HPC Challenges in Computational Sciences Summer School
The first European US Summer School on HPC Challenges in Computational
Sciences organized jointly by Europe’s DEISA project and the US/NSF TeraGrid project was
taking place at Santa Tecla Palace in Acireale, Sicily from Oct 3-7, 2010.
Among the participants were sixty graduate students or post-docs, selected from more than 100
applications: 25 from US and 35 from EU universities and research institutions. Students came
from a variety of disciplines, among them astronomy, atmospheric sciences, chemistry, computer
science, engineering, mathematics and physics. Female students were represented with a fraction
of 20%.
Twenty-five high level speakers were covering major fields of computational sciences, with nine
speakers from the US and sixteen from Europe. Areas covered included Challenges by Scientific
Disciplines, Programming, Performance Analysis & Profiling, Algorithmic Approaches &
Libraries, and Data Intensive Computing and Visualization.
“The summer school HPC Challenges in Computational Sciences was an excellent opportunity
for me to get an overview of the complete spectrum of High Performance Computing and
Computational Science. Attending talks from the large number of distinguished speakers from
almost every domain of computational science and High performance computing has enabled me
to get a very clear idea of the current trends in the area”, one of the students stated afterwards.
To hold a joint EU US summer school was suggested by leading computational scientists from
both continents. “Our primary objective for the student experience was to advance computational
116

sciences by enabling and stimulating future international collaboration, innovation, and discovery
through the most effective use of HPC,” said Hermann Lederer from the DEISA coordination
team at RZG, Garching.
“We hope to continue with such events every year— alternating between EU and US
destinations,” said TeraGrid Forum Chair John Towns, National Center for Supercomputing
Applications. “The overwhelmingly positive feedback of the students - 85% rated the event as
very good or excellent – can be taken as a mandate”, he added.
Leake assisted with the promotion and facilitation of the first DEISA/TG Joint Summer School
on HPC Challenges in Computational Sciences held October 4-7, 2010 in Catania, Italy. Ferguson
(NICS) was also in attendance and provided technical support for the participants. A flier and
registration tool were developed by ER team members.
Leake established a FaceBook community (with 54 ongoing members) to facilitate collaboration
before, during, and after the event. Social networking proved to be a highly effective way to
acquaint participants (from 20 countries) before the event. It helped to break the ice among the
participants and the presenters.
One of the TeraGrid presenters, Sean Ahern (NICS/ORNL), after meeting participant Jeremy
Logan (University of Maine) in Sicily, encouraged ORNL to hire Logan as a postdoc to assist
with research in parallel I/O. Ahern recognized that Jeremy's work could be applicable to ORNL's
HPC data processing research. Logan started at ORNL a few weeks ago!!
The following are comments from a few of the participants.
Benjamin Cruz Perez said, “I used TeraGrid resources to complete my master’s thesis. Without
TeraGrid’s tools and training, I would not have been able to obtain results in time for my
defense.” “Thanks to these experiences, I have met with other researchers with similar interests
who also use supercomputing for their research. I’m now comfortable using the techniques I’ve
learned for the efficient use of high performance computing, and look forward to participating in
multidisciplinary research in the future.”
“Sometimes it doesn’t hurt to push things, the system, the methodology, yourself, to the limit.
Often, it is under such duress that great things happen,” said Terrence Neumann, Marquette
University.
“I have been writing proposals for a while now and will be submitting a new Teragrid proposal
soon. It was nice learning more about Teragrid and its EU counterpart, DEISA. I enjoyed the
friendship and conversations, and I am looking forward to implementing some of the new
techniques and libraries I learned about in my own research,” said Cengiz Sen, Turkish national
postdoc fellow, University of Cincinatti.
Giuseppa MUSCIANISI, PhD Student in Mathematics and Informatics. Department of
Mathematics, University of Messina, Italy said, “With the exciting backdrop of Riviera dei
Ciclopi, I increased my knowledge about HPC challenges thanks to the great talks by experts and
discussions with colleagues from around the world. The School was very well organized. All talks
addressed the tools, software applications, and techniques needed to conduct HPC research.“
The presentations given are available from http://www.deisa.eu/Summer-School/talks
A report of the Summer School is included in Appendix D – EU-US HPC Summer School Report
117

Outreach to University of Puerto Rico
TeraGrid conducted a one day outreach session for over 50 faculty and students at the University
of Puerto Rico at Rio Piedras on December 8 and the University of Puerto at Mayaguez on
December 9, 2010. The participants were provided with hands-on access to TeraGrid systems
(Purdue’s Steele and PSC’s Pople) to learn how to submit and run parallel jobs.
Stefano Leonardi, professor at University of Puerto Rico at Mayaguez, helped to organize the
visit. Leonardi has worked at the Sapienza University of Rome, Max Planck Institute,
Saarbreucken, Germany, and at Carnegie Mellon and Berkeley Universities in the US.
Leonardi said, “My wife and I moved to Puerto Rico about two years ago because we feel it is as
close to Paradise as one can get—unless you rely on supercomputing to do your work. If it
weren’t for TeraGrid, there is no way I could conduct my research.” Participants have requested
accounts on TeraGrid subsequent to the visit.
Supercomputing’10—November 12-18, New Orleans, LA
Members of the ER team volunteered to assist with TeraGrid’s presence in the RP booths and in
the TeraGrid booth. About 1000 Science and EOT Highlights documents and other TeraGrid
communication collateral were disseminated. We distributed about 1000 orange petaflops (flipflops)
that were left-over from the TeraGrid’10 Conference.
118

The TeraGrid booth showcased group photos from TeraGrid outreach events including the EXPO
in DC and the EU-US Summer Schools.
Leake and Hunt developed an engagement activity among the TG booths to showcase the
Campus Champion Program. The theme “Embrace the Future with a Campus Champion”
identified champions as important team members for expanding and supporting the TeraGrid
community. Nineteen “trading” cards featured photographs of the champions and information
about TeraGrid’s transition to XD. These champions were provided with copies of the cards for
their efforts in acquainting local local users with TeraGrid.
http://www.teragridforum.org/mediawiki/index.phptitle=TeraGrid_and_RP_Booth_Engagement
_Activity
119

Leake shopped locally for Mardi-Gras themed decorative elements to give the booth a regional
flavor. Feather boas, beads, and fragrant floral arrangements echoed the colors from the scientific
images in the booth petaflops, and brochures. After the conference, the floral arrangements and
decorative objects were donated to a local nursing home. The arrangements, showcasing huge
stargazer lilies, were still fresh and beautiful in the nursing home for many more days.
TeraGrid’11 Conference, Salt Lake City Utah, July 18-24, 2011
ER and EOT personnel are instrumental in the initial planning for the 2011 conference. Trish
Barker (NCSA) is communication committee chair. Bill Bell (NCSA) is involved as deputy chair.
Many other ER team members serve on various committees. Diane Baxter (SDSC) and Pallavi
Ishwad (PSC) are EOT track chair. Laura McGinnis (PSC) is the lead for the Student Program
and is coordinating with OSG on a Summer School for students. Scott Lathrop (UChicago) is the
chair for BOFs.
Proceedings from TG’10 (leadership provided by Froelich and many others on the ER team) were
published by the ACM on December 4:
http://portal.acm.org/citation.cfmid=1838574
120

10.2 EOT Impact Stories
Indiana
IU event to help Indiana Business Leaders get their heads in the clouds (IU News release, CI
Newsletter, multiple local and regional media outlets in Indiana)
Turn on any technology-related news program or radio show these days and you're likely to hear
someone mention cloud computing. "Cloud" has become a hot technology buzzword, in part due
to the substantial promise cloud applications hold for business and industry.
To help leaders from corporations and small businesses begin to realize this potential, the
Pervasive Technology Institute at Indiana University hosted an educational seminar discussing
how cloud technologies are providing access to greater computational power at a lower cost. The
seminar
http://newsinfo.iu.edu/asset/page/normal/8185.htmlThe Cloud Computing for Business and
Industry seminar was held in conjunction with the IEEE 2nd International Conference on Cloud
Computing Technology and Science (CloudCom 2010).
World cloud computing leaders convene at IU-hosted conference (HPC Wire, SC Online, IU
News release, a number of national and regional outlets) (also video from from the
conference available online)
The hot technology buzzword, "cloud," describes Internet-accessible infrastructure -- such as data
storage and computing hardware -- that is hidden from users. Top cloud computing researchers
and industry leaders from around the globe are gathering in Indianapolis this week to discuss the
latest research and the future of this powerful emerging discipline.
The IEEE 2nd International Conference on Cloud Computing Technology and Science
(CloudCom 2010) took place from Tuesday, November 30 through Friday, December 3 at
University Place Conference Center at Indiana University-Purdue University Indianapolis
(IUPUI). This year's hosts are the Pervasive Technology Institute at Indiana University and the IU
School of Informatics and Computing. The conference included a number of talks on TeraGrid
related technologies and services.
SDSC
“Grand Challenges in Data-Intensive Discovery” was the focus of a conference at SDSC on
October 26-28, 2010. An element of SDSC’s TeraGrid outreach program associated with the new
Gordon Track 2D system, the conference provided an opportunity for attendees to share their
expertise while exchanging ideas about the computational challenges and concerns common to
data-intensive problems. Presenters from a broad representation of disciplines allowed
participants to understand some of their shared computational challenges. A small sampling of
attendee responses to follow-up evaluation gives an indication of the conference’s success.
A number of collaborations resulted from this conference. It was amazing to see how common
the data challenge problems are. It was also interesting to see how different fields have come
up with similar solutions independently. This clearly points to an urgent need for multidisciplinary
efforts.
I wanted to learn about the characteristics and the needs of data intensive applications; how
they are matched by existing systems, and how future systems could be built to meet these
applications’ needs. The conference had a good balance of technical and high level talks.
I wanted to learn from other researchers in different fields - especially Social Science &
Humanities - about their work with large data sets. There are commonalities with data
searching tools in neuroscience and archaeology. This was interesting to see.
121

The mix of breadth and depth, centered around what this machine [Gordon] has to offer a
potentially wide range of areas, was very good.
Participants also provided valuable feedback on additional topics for conferences, tutorials, and
workshops related to data-intensive computing. Their candid perspectives on where they saw
expertise needed in the next 3 years provides useful workforce development program guidance.
SDSC’s Diane Baxter served as a committee chair for an NSF task force on Cyberlearning and
Workforce Development, working closely with other members of the TeraGrid community and
national STEM education colleagues to examine and analyze risks and opportunities in emerging
national priorities for workforce development associated with cyberinfrastructure and
computational STEM.
SDSC’s Jeff Sale spearheaded design of the TeraGrid Campus Champions portal, an effort
involving a great deal of community input, testing, and revision. In December, the portal was
finally deployed as an integrated component of the TeraGrid User Portal. At the time of this
report the portal is being used by the community. Jeff’s ongoing and continuing online portal
support provides a mechanism for continuing improvements and updates to the portal to best
serve the needs of the Campus Champions.
As Dash came on line and Gordon testing continued, SDSC shared its discoveries along the way
through two presentations at high profile meetings that were also included in published
proceedings of those events:
• Moneta: A High-performance Storage Array Architecture for Next-generation, Nonvolatile
Memories, Adrian M. Caulfield, Arup De, Joel Coburn, Todor Mollov, Rajesh
Gupta, and Steven Swanson, Proceedings of The 43rd Annual IEEE/ACM International
Symposium on Microarchitecture, 2010.
• Understanding the Impact of Emerging Non-Volatile Memories on High-Performance,
IO-Intensive Computing, Adrian M. Caulfield, Joel Coburn, Todor Mollov, Arup De,
Ameen Akel, Jiahua He, Arun Jagatheesan, Rajesh K. Gupta, Allan Snavely, and Steven
Swanson, Supercomputing, 2010. (Nominated for best technical paper and best student
paper).
TACC
Landmarks on the Human Genome
Vishy Iyer, The University of Texas at Austin
Genetics and Nucleic Acids
We typically think of heredity as rooted in our genes, and it is. But
scientists are finding that variants in the non-coding regions of the
genome, formerly considered junk, and even proteins that are not
part of the genome at all but that interact with genes, play a role in
our personal traits as well.
Vishy Iyer, an associate professor in the Institute for Cellular and
Molecular Biology at The University of Texas at Austin, used
TeraGrid resources to explore the expression of genes related to
regulatory proteins and to investigate the role of heredity in the
transcription binding process. Based on his findings, Iyer and
colleagues published one of the first studies that used next-gen
sequencing and HPC analysis in Science in April 2010.
Figure 1: Representation of allele-specific
and non–allele-specific SNPs across the
CTCF binding motif (17). The y axis
indicates the difference between the two as
a percentage of normalized total SNPs.
Higher bars indicate an increased
representation of allelespecific SNPs
122
relative to other positions, which tends to
occur at conserved positions.

The Ranger system at the Texas Advanced Computing Center assisted the process by taking the
short sequences read by the ChIP-Seq gene sequences and aligning them to the reference genome.
It sounds simple enough, until you imagine sequencing millions of small bits of DNA and
matching them to approximately three billion base pairs. Using several thousand cores in parallel
on Ranger, the alignment required more than 175,000 processor hours, or the equivalent of 20
years on a single processor.
Their proof of principle study determined that distinctions in transcription binding can be studied
using their method. They were also able to tell the difference in binding from the gene that you
inherited from your father and mother, showing that transcription factor binding appears to be a
heritable trait. They are now using this technology to look at those differences and apply it to
cases where a mutation from a parent predisposes one to a disease.
10.3 EOT Highlights
UChicago/ANL
Following the completion of the SC10 Conference in November, Scott Lathrop, TeraGrid Area
Director for EOT, launched his role as the SC11 Conference Chair. SC11 will be held November
12-18, 2001 in Seattle, Washington. (http://sc11.supercomputing.org). Numerous staff from
TeraGrid and Blue Waters are involved as committee members for the SC11 Conference.
Indiana
The Pervasive Technology Institute at Indiana University announced that it has joined OpenSFS,
a technical organization that promotes collaboration among institutions using open source
scalable file systems such as Lustre. Indiana University participated in the TeraGrid booth at the
USA Science & Engineering Expo on the National Mall and provided a 6 minute 3D stereo video
about the Linked Environments for Atmospheric Discovery (LEAD) project. Indiana University
along with four research partners announced that they had succeeded in using the Lustre file
system over a wide area network (WAN), saturating the world's first commercial 100 gigabit link
with an aggregate average transfer rate of 21.9 GBps or 87% of theoretical maximum.
ORNL
TeraGrid staff played an important role in the SC10 education program in New Orleans in
November, 2010. Laura McGinnis was the chair of the SC10 education program this year. The
program was an unqualified success with over 100 attendees from secondary and undergraduates
institutions. An addition this year was the opening of the sessions to general conference attendees,
many of whom attended one or more sessions. In fact in a few sessions even the large rooms
available in New Orleans were full to capacity. Other TeraGrid staff assisted as well including
Edee Wiziecki, John Cobb, and Scott Lathrop.
Purdue
Purdue hosted a CI Days event on December 8-9, 2010 on Purdue's West Lafayette campus.
Purdue CI Days 2010 aimed to broaden the use of cyberinfrastructure, with expert presentations
and seven panel discussions. Most of the panelists are researchers who already taking advantage
of CI resources ranging from cutting-edge supercomputers to classroom discussion tools. They
shared their experiences and lessons learned with the audience. Approximately 200 researchers
and students attended the event.
TACC
TACC’s education and outreach programs include the scientific computing courses (2 during this
reporting period) and tours of TACC’s facility at the J.J. Pickle Research Campus (about 10 miles
north of UT Austin’s main campus) and the ACES Visualization Laboratory at the UT Austin
123

main campus. The Austin Forum continues to attract a broad audience from the university, the
city, and the wide range of technology industries around Austin. TACC’s Scientific Computing
courses (3) were all well enrolled this semester, reaching 74 graduate and undergraduate students.
Course descriptions and instructional slides for TACC’s two introductory courses, Introduction to
Scientific Programming, and Scientific/Technical Computing, were made available online to the
public. TACC also partnered with the Division of Statistics and Scientific Computation to build a
new classroom for advanced computational instruction. The room will be completed in January
and dedicated in the first quarter of 2011. TACC has continued to use social media and social
news sites like Facebook, Twitter, Slashdot and Digg, to broaden our reach and participate in
community conversations. TACC continues to experiment with webcasting training courses, and
will expand the course availability in the coming months.
10.4 Training, Education and Outreach
NCSA
New CI-Tutor Course Released: NCSA released a new online tutorial in CI-Tutor, “Introduction
to Performance Tools.” This tutorial provides an introduction to a number of commonly-used
performance tools on the TeraGrid. The tools highlighted include two Linux utilities: strace,
which traces system calls and signals during program execution; and gprof, which can be used to
track how functions are using system resources. More complex toolsets include NCSA’s
PerfSuite, a collection of tools and supporting libraries that provides a higher-level view of
application performance; and TAU (Tuning and Analysis Utilities), a suite of tools for
performance instrumentation, analysis, and visualization of large-scale parallel computer systems
and applications. The course is organized as individual lessons that provide a general overview of
the tool, demonstrates how the tool might be run on a TeraGrid computing resource, provides
examples of subsequent data analysis, and concludes with self-assessment and pointers to more
comprehensive resources regarding each tool.
CI-Tutor Upgraded: CI-Tutor was upgraded to ATutor version 2.0.2. ATutor is the open-source
learning management software on which CI-Tutor is based. This version provides increased
security and a number of enhancements to the functionality. The domain name for the site was
also changed from ci-tutor.ncsa.illinois.edu to www.citutor.org.
NICS
Jim Ferguson (NICS) joined other TeraGrid External Relations and EOT staff in Washington,
D.C. at the USA Science & Engineering Festival. (http://www.usasciencefestival.org/) The
TeraGrid EOT team was responsible for one of fifteen booths that were being staffed by NSF
projects for the October 23-24 capstone event on the National Mall and Pennsylvania Avenue. A
conservative estimate of the number of visitors to the TeraGrid booth put it around 2000 each
day. Somewhere between 200,000 and 400,000 attendees came to D.C. during the week-long
series of events.
Bhanu Rekepalli, Jacek Jackowski, and Jim Ferguson (all of NICS) participated in a Panel
Session at SC10 in New Orleans, November 17. The panel concerned Workforce Development, a
first of its kind at an SC Conference, and the three NICS speakers highlighted some of the work
they had done in this area with the South Carolina/Tennessee EPSCoR project and their
associated TeraGrid work. Approximately 40 people attended the Panel.
Jim Ferguson (NICS) was on the conference staff for the TeraGrid/DEISA Summer School, held
in Acireale, Italy, October 4-6. The workshop was targeted at graduate level students in
computational sciences, and the attendees included 25 students from the U.S. and 35 students
from European institutions. Ferguson helped review and select students from the U.S. and
Europe to attend the conference, and was responsible for engaging the students during the
124

conference to assess their level of satisfaction with the program and garner any suggestions for
future like workshops.
ORNL
A case study of the NSTG RP at ORNL was presented to the international NOBUGS 2010
workshops in October in Gatlinburg. (http://www.nobugsconference.org/Conferences) This three
day workshop had over 50 attendees form three four different continents. The NSTG case study
was the capstone presentation officially ending the conference presentations.
Purdue
Purdue’s Kay Hunt, the TeraGrid Campus Champion Program lead, gave a number of
presentations including at Oklahoma Supercomputing Symposium, EPSCoR Cyberinfrastructure
meeting in Washington, DC, EDUCAUSE in Anaheim, CA, Internet2 in Atlanta, GA, IEEE
CloudCom in Indianapolis, IN, and CI Days events at Univ. of Michigan, University of
Wisconsin-Milwaukee, and Purdue. She planned and led campus champion BOFs at SC10 and
Internet2 meeting.
Purdue RP provided accounts on Steele and assisted with the training in Puerto Rico, in
December 2010.
10.5 RP Operations: Training
Jim Marsteller held the annual “Security 4 Lunch” on 08-Nov-2010 for all PSC staff and students.
The one hour presentation covered general information on security best practices including
material for PSC staff traveling to SC10. This year the staff was very interactive, asking a number
of security related questions. In fact, the VPN demo ran over the one hour allotted time.
SDSC
RP-funded training activities included hands-on workshops, training sessions via the Access Grid,
support of online and in-person tutorials. TeraGrid RP staff participated in both local site-specific
training as well as TG-wide training.
TACC
TACC and Cornell HPC staff members taught 6 workshops within the reporting period. Staff
members provided 36.5 hours of instruction to a total of 71 students.
VIRTUAL WORKSHOP
Staff members at the Center for Advanced Computing at Cornell University continue to maintain,
support, and develop new modules for the Ranger Virtual Workshop. Available through both the
TeraGrid User Portal and the TACC User Portal, the workshop provides users access to seventeen
training modules with seven new modules currently under development and three modules
scheduled for updates.
Available Modules
An Introduction to Linux (new)
An Introduction to C Programming (new)
Parallel Programming Concepts and High-Performance Computing
Ranger Environment
Message Passing Interface (MPI)
MPI Point-to-Point Communications (updated)
MPI Collective Communications
MPI One-Sided Communication
125

OpenMP
Hybrid Programming with OpenMP and MPI
Profiling and Debugging
Optimization and Scalability
Computational Steering
Large Data Visualization
ParaView
VisIt
Using Databases
Modules Under Development
Advanced MPI (We are planning to add content to the existing MPI modules instead)
Data Analysis with Python on Ranger
Distributed Debugging Tool (DDT)
Multi-core
Multi-node Map Reduce
Parallel I/O
Profiling with mpiP
Using Scripting Tools on Ranger
Modules Being Updated
Hybrid
MPI Collective Communications
OpenMP
10.6 RP Operations: Education
NCAR
In December, NCAR opened applications to the SIParCS program for 2011. This year, 18
summer project descriptions were solicited from local mentors. NCAR conducted training in the
VAPOR visualization tool at the American Geophysical Union 2010 meeting in San Francisco.
Approximately 25 people attended.
PSC
PSC’s Pallavi Ishwad and Jacob Czech gave a 90-minute presentation on the CMIST module
“Enzyme Structure and Function” at the education program at SC10. The presentation was wellreceived.
About 25 educators, 15 from under-represented groups, attended the presentation and
provided important feedback for further improvements to the module. CMIST (Computational
Modules in Science Teaching) is a program of the National Resource for Biomedical
Supercomputing (NRBSC) at PSC that aims to bring innovative science tutorials into secondary
school classrooms, focusing on integrative computational biology, physiology, and biophysics.
CMIST modules include high quality, biologically realistic 3-D animations produced with
cutting-edge simulation and visualization software developed at NRBSC.
Ishwad also gave a 90-minute presentation on BEST, Better Educators of Science for Tomorrow.
About 40 educators, 15 from under-represented groups, attended. BEST is a PSC outreach
program that exposes teachers to modern molecular biology concepts and evaluates inclusion of
computational biology and bioinformatics into high school curriculum. In addition, BEST
prepares teachers to refocus their teaching strategies towards exposing and encouraging students
to become aware of emerging and exciting biomedical careers.
126

Also at SC10 Rebecca Day, a teacher at Frazier High School, presented three posters about
Frazier’s participation in BEST. Frazier is in a rural school district in Western Pennsylvania with
a 75% reduced meal plan population. Two of the posters were prepared by Frazier’s students in
conjunction with PSC staff members and one by Day and other teachers, also in conjunction with
PSC staff members. All four of the teachers and two of the four students are female. At least 50
people visited the posters. Titles of the posters are the following:
Purdue
1. Bioinformatics in High School
2. High School Research Internship in Bioinformatics
3. Summer Internship at PSC
In the fall semester of 2010, the Purdue C4E4 science gateway (http://c4e4.rcac.purdue.edu) was
used in a graduate level Computational Watershed Hydrology class (CE549) taught by Professor
Venkatesh Merwade of Civil Engineering. Students used the portal to calibrate SWAT (Soil and
Water Assessment Tool) model for their research. SWAT calibration is computation intensive
and in the past the students were not able to perform in a semester long class. In one study, a
student from Department of Agronomy investigated the effects of soil patent materials on
streamflow and nutrient processes using a paired catchment study in Indiana. He used the
gateway to conduct long running calibrations on the TeraGrid. Preliminary findings indicate that
nitrate output is more sensitive to soil parent material than either streamflow or sediment load.
SDSC
TG RP Education activities included activities to communicate the relationships between science,
technology, and society, and the role that scientific computing plays. TG RP staff reached out to
the K-20 community in an effort to encourage student interest in math and science during the
education process, to members of diverse, underrepresented groups in order to broaden
participation in high performance computing, and to the general public in order to promote
awareness of role of high performance computing to advance scientific discovery. Some RP sites
contribute to the scientific computing curriculum at their institution, creating and maintaining
"course packages" that are available to higher education faculty to produce their own courses.
SDSC Education programs in the fourth quarter included ten student-focused workshops and
classes, and fourteen workshops for teachers. SDSC’s TeacherTECH program benefitted from
new and expanded collaborations with other organizations in the region that enhanced programs
in bioinformatics and astronomy. Despite unusually low teacher professional development
participation statistics in the region (due to a depressing state economy characterized by
widespread layoffs and course cancellations at all levels), 221 teachers (153 underrepresented in
STEM) participated in 14 workshops totaling more than 32 contact hours. It was truly
encouraging that teachers participated enthusiastically in a new, multi-session, space science
series without any financial incentives.
Expanded programs for students proved tremendously popular, offering computer science,
computational sciences and computing tool training that are not provided in the region’s schools.
The ten classes – several of them full-day (6 hour) programs – attracted 175 (116
underrepresented in STEM) students from middle school through undergraduates.
TACC
TACC Scientific Computing Curriculum Courses: Since the fall of 2006, over 400 students have
enrolled in our courses. A total of 74 undergraduate and graduate students enrolled in the three
Fall 2010 courses, Introduction to Scientific Programming (focused on FORTRAN and C),
Scientific and Technical Computing, and Visualization and Data Analysis. Six TACC scientists
127

are teaching these courses. In each course, enrollment is very high. Engineering students continue
to hold the greatest percentage of students, compared to the natural sciences, computer sciences,
geosciences, and liberal arts. The majority of students are involved in research. Introduction to
Scientific Programming continues to attract the greatest percentage of undergraduates per course.
TACC is evaluating the quality and effectiveness of the courses with pre- and post-course surveys
based on the content, learning objectives, and the computational science competencies from the
Ralph Regula School of Computational Science. The Spring 2010 course data indicates
statistically significant gains in students’ knowledge and skills. There are a few interesting
findings: Introduction to Scientific Programming attracts mostly undergraduates, while graduates
are the overwhelming majority in the Parallel Computing course. The majority of students in the
programming class were from engineering and physics, but two liberal arts majors enrolled in the
class as well. Student achievement in the learning objectives for the parallel computing course,
based on the competencies developed by the Ralph Regula School of Computational Science,
increased from little or no experience to proficient/very experienced over the course of the
semester. An encouraging highlight is 75% of students in the scientific programming course
indicated that they are applying their new programming capability in other courses.
TACC Scientific Computing Curriculum Package: Production of the Scientific Computing
Curriculum Package came to a close in the 4 th quarter of 2010. Instructors reviewed the revised
course materials and they were made available via the TACC website at SC’10
(http://www.tacc.utexas.edu/education/academic-courses/). The presentation slides contain the
bulk of the courses’ content that students use on a regular basis, thus the content accuracy and
quality of the slides are critical to maximize impact and sustainability. Students have commented
to our instructors about the high quality of the slides. The package content extends beyond the
course slides to ensure that faculty at other institutions may readily adopt or integrate the
materials for creating their own courses or enriching existing courses. Additional content includes
assignments, tests, projects, references, and instruction guides.
10.7 RP Operations: Outreach
Indiana
Indiana University was host to the IU Statewide IT Conference and the Data to Insight Center of
the Pervasive Technology Institute hosted a booth to highlight the work scientists are able to
accomplish given the technology provided by Indiana University and the TeraGrid. During the
Show and Tell reception, Mike Boyles, Robert Ping and several students from the Center showed
the Stereoscopic video, LEADING THE WAY and showed live demonstrations of climate
forecasting enabled by Big Red, one of the TeraGrid supercomputers.
128

NCAR
For the first time, SIParCS outreach advertisements were sent to 45 universities, including 15
minority-serving institutions.
PSC
During the last quarter PSC’s Phil Blood has been the initial TeraGrid contact for 7 new Campus
Champions. He helped two of these new Champions find answers to questions that they had
regarding TeraGrid, either for themselves or users on their campuses. During the same period he
also assisted three established Champions in finding answers to questions related to their service
as Campus Champions or TeraGrid usage scenarios important to users on their campuses. The
bulk of his duties, however, involve his work with the Campus Champion Leadership Team and
Advisory Board. Each month he attends three monthly calls (leadership, advisory, and general
Campus Champion call) to help plan and organize the Champion effort and implement strategies
for improving the program. For example, in this quarter he helped develop a Champion mentor
program in which an established Champion is assigned to work with new Champions. He also
attended Champion meetings at SC’10.
PSC had a booth and made a presentation at the Three Rivers Educational Technology
Conference on 10-Nov-2011. This was a good venue to reach out to the local K-12 community
and we had many visitors to the PSC booth. Contacts made or renewed included the technology
coordinators at Winchester-Thurston and Ellis Schools; many people at the Allegheny
Intermediate Unit, including the new Executive Director Linda Hippert; Grable Foundation
Executive Director Gregg Behr; various Pittsburgh Technology Council people; and a number of
technology coordinators and STEM teachers in the region.
PSC has decided to once again be a bronze level sponsor for the SciTech Days this year. SciTech
Days is one of the activities of SciTech that is a year-round initiative of the Carnegie Science
Center in Pittsburgh. Bronze level sponsorship allows PSC to not only have a booth (March 8 –
11, 2011) at SciTech Days but also to make two awards at the Pittsburgh Regional Science and
Engineering Fair (April 1-2) for projects with the best computational element. This also helps to
further PSC’s relationship with the Carnegie Science Center as a community partner.
On 17-Nov-2011 a group of senior computer science majors from Slippery Rock University (part
of the Pennsylvania State College system) visited PSC for a tour of the machine room that
included a talk of HPC careers. Slippery Rock is also interested in possible internships for their
students in the future.
SDSC
TG RP Outreach activities included activities to communicate the relationships between science,
technology, and society, and the role that scientific computing plays. TG RP staff also reach out
to the scientific and professional communities in an effort to encourage interest in the use of
TeraGrid resources and services. Many RP sites contribute to broadening participation in the
scientific computing community that should be covered in this section.
As mentioned in the Highlights, SDSC activities included several national-scale outreach
activities critical to shaping the future of the TeraGrid, XD, and data-intensive discovery. SDSC’s
“Grand Challenges in Data-Intensive Discovery” conference in October 26-28, 2010 provided
attendees a venue for share expertise and ideas about the computational challenges and concerns
common to data-intensive problems from many fields of inquiry. Follow-up participant surveys
provided valuable feedback on future data-intensive computing topics for training, outreach, and
workforce development programs.
129

The TeraGrid Campus Champions portal was deployed as an integrated component of the
TeraGrid User Portal in December, with strong contributions from SDSC. Sale’s dedicated
online support fueled significant site design and function improvements to serve the needs of
future Campus Champions.
SDSC shared its Dash and Gordon (Track 2D) development process discoveries through highly
reviewed presentations and published proceedings of international professional meetings.
TACC
For this quarter, TACC hosted three of its monthly Austin Forum events with invited speakers
from areas of interest focused on science and technology. The goal of “The Austin Forum on
Science, Technology & Society” is to engage and educate the Austin community about the
numerous ways in which science and technology enhance the quality of their everyday life as
well as the health, prosperity, and security of the nation. Two hours are devoted to a networking
reception, presentation, and Q&A discussion among the speaker and guests, which reinforces the
broader impact and intellectual merit of the program. Speakers and topics for this reporting period
focused on:
• Oct. 2010: Dr. Steven Weinberg, Hopes and Fears for Big Science. Dr. Steven Weinberg
holds the Josey Regental Chair in Science at The University of Texas at Austin, where he
is a member of the Physics and Astronomy Departments. His research on elementary
particles and cosmology has been honored with numerous prizes and awards, including in
1979 the Nobel Prize in Physics and in 1991 the National Medal of Science. In 2004, he
received the Benjamin Franklin Medal of the American Philosophical Society with a
citation that said he is "considered by many to be the preeminent theoretical physicist
alive in the world today."
• Nov. 2010: Daniel Lorenzetti and Juan Garcia, "Transmedia Storytelling.” Transmedia
Storytelling is a new way to use multiple media outlets, concurrently, with each element
enhancing and advancing the narrative. It has profound implications for politics,
marketing, and all forms of entertainment.
• Dec. 2010: Dr. Alfred Gilman, CPRIT: Opportunities and Challenges. The Cancer
Prevention and Research Institute of Texas (CPRIT) is a new Texas agency empowered
to invest $3 billion over 10 years to enhance research and prevention activities toward
alleviation of suffering and death from cancer. This vision poses enormous questions
about distributed clinical trials management systems, as well as means for acquisition,
storage, analysis, integration, and dissemination of vast amounts of information.
TACC openly advertises tours of our facilities to the Austin and central Texas area. Tour groups
include visitors in K-12, higher education, industry, government, and the general public. An
overview of TeraGrid and its contributions to STEM research are provided at each event.
10.8 External Relations
A number of people serve on both the EOT and ER teams. The ER teams helps develop materials
for outreach, and participate as presenters during outreach events. In Q410, Leake assisted with
two CI Days, the US Expo, the DEISA/TG Summer School, and the Puerto Rican outreach
initiative. Hunt (Purdue) serves on both ER and EOT teams and leads the Campus Champion
initiative and assists with planning for the CI Days events. Ping (IU) also serves on both teams
and is responsible for facilitating TeraGrid’s presence at conferences and events. Every ER team
member serves as a TeraGrid ambassador through their involvement with outreach via
professional organizations, campus visits, committees, and attending conferences. Through these
activities, and by sharing our stories and communication collateral with others, the ET team
130

communicates TeraGrid’s impact. Leake, and ER team members, help to recruit and build
relationships to engage multidisciplinary users from vastly diverse communities, regionally,
nationally, and internationally.
A number of joint ER-EOT outreach activities were highlighted at the start of Section 10.
10.8.1 Benchmarking Effectiveness on the WWW and identifying exploits
To benchmark the effectiveness of TeraGrid’s external relations efforts, Leake reviews Google
alerts as they happen—new web information that appears mentioning TeraGrid in a unique way.
In Q410 there were 114 legitimate alerts. In calendar year 2010 there were 657 total, exactly
twice the number that occurred in 2009 (328)! This is an important benchmark for the
effectiveness of the collective efforts of TeraGrid’s External Relations team.
The process is also a way to determine quickly if TeraGrid’s resources have been exploited by
organizations that might use our domain for malicious or inappropriate ways. Suspected exploits
are immediately sent to Jim Marsteller (PSC), leader of TeraGrid’s security working group, for
handling, such as the recent case where a TeraGrid resource had been used to sell
pharmaceuticals.
10.8.2 Involvement in the Allocations Process
Leake released the quarterly allocation request deadline announcement (TRAC) mid and near the
end of the month in anticipation of the January 15 deadline. Fourteen sources carried the story.
Reminders were sent via TeraGrid’s FaceBook community and published on the TeraGrid web
site. This year the external relations team took a more active interest in the allocation process.
Each quarter, a letter was provided to each of the TRAC reviewers (about 50), to emphasize the
importance of promoting science through the efforts of the ER team.
10.8.3 2010 and 2011 Science Highlights Selection Process
In Q410, the ER team completed production of the 2010 Science Highlights publication for initial
distribution at SC10 in New Orleans. Warren Froelich (SDSC) coordinated the production of the
publication in collaboration with Carlton Bruett Design, a minority owned company. Copies of
the publication were sent to NSF (OCI & OLPA), to Hunt (Purdue) for distribution to Campus
Champions, to Ping (IU) for use with outreach to professional conferences and meetings, and to
each Resource Provider for their outreach efforts. The Science and EOT Highlights publications
are posted on the TeraGrid web site at: https://www.teragrid.org/web/news/news#2010scihigh
All members of the ER team assist with the identification of prospective highlights for reporting
and publishing purposes. Leake and Katz attend the TRAC meetings on an alternating basis to
help identify exemplary science and highlights that meet the seven selection criteria found on
page 36 of the Science Highlights book:
https://www.teragrid.org/c/document_library/get_fileuuid=e950f0a1-abb6-4de5-a509-
46e535040ecf&groupId=14002)
Katz typically highlights about 10-15 applications via TeraGrid’s quarterly reporting process.
Between 16 and 20 are eventually selected by the ER team and advisers (Lathrop, Roskies, Katz,
Towns) for further development and inclusion in the annual science highlights book.
131

In 2011, in anticipation of the transition to XD, the Science and EOT Highlights books will be
combined. The ER and EOT teams have begun the process of identifying story prospects for this
publication.
10.8.4 RP Operations: External Relations
PSC
The movie “Career Gates: STEM” with appearances by PSC’s Joel Stiles and Jacob Czech won
the National Academy of Television Arts & Sciences Mid-Atlantic Chapter’s 2010 Emmy®
Award for Children/Youth/Teen (19 and under) - Program or Special. The movie is on you-tube
at http://youtube.com/watchv=1zGHCvkS4qw.
SDSC
TeraGrid RP local external relations staff developed local site-specific content as well as
contributed to TeraGrid-wide activities and publications. External relations staff provide web and
print media outlets detailing the successes of the TeraGrid project and its user community. They
also develop materials for Supercomputing and TeraGrid conferences.
SDSC’s Communications staff continued to provide writing and publications support to TG
External Relations activities. Specifically,
• Warren Froelich, SDSC’s communications director, served as lead editor for 2010 TG
Science Highlights, coordinating, editing and providing graphic oversight for the
publication. Following his stint as Deputy Chair for TG’10, Froelich was tapped to serve
in the same capacity for TG’11, working with event chair John Towns (NCSA) on the
planning and follow-through for the annual conference scheduled next July in Salt Lake
City.
• Jan Zverina, in his role as TG science writer, continued to identify and write stories,
including an article on TG’s science gateways being used as a model for a five-year, $4.4
million NSF award to form a collaborative software framework for analysis of
geographic data. As TG’10 Communications Co-Chair, Jan also worked with Sonya
Nayak and members of the TG’10 communications team to finalize related TG’10
expenditures.
• Ben Tolo, SDSC’s graphics designer/web manager, created the TG’11 Save the Date
card, provided slide graphics for SC’10, and helped proof TG Science Highlights. Ben
also has been asked to serve on the Communications committee for TG’11.
10.8.5 News featured in Q4 on TeraGrid.org
12/1/2010: Eight TeraGrid resources were included among the November 2010 Top500 (Leake)
11/30/2010: SIParCS - Summer Internships in Parallel Computational Science application
deadline is February
11/29/2010: Women in Science Conference 2011 registration is now open
11/22/2010: TeraGrid Partner TACC Receives 2010 HPCwire Readers' Choice Award (Singer-
Villalobos/Dubrow)
11/22/2010: UChicago/Argonne--Globus Cloud Solution Facilitates Large-Scale Data Transfer
(GridFTP-enabled sites) (Leake)
11/10/2010: SDSC-ASU's OpenTopography Facility Part of $4.4 Million NSF Award To
Develop a Geographical Information Science (GIS) Framework (Froelich/Zverina)
10/28/2010: TeraGrid and DEISA host first joint Summer School in Italy (Leake)
10/21/2010: UChicago/ANL-The Beagle Has Landed In Chicago! New Cray XE6 Supercomputer
for Biomedical Simulation and Data Analysis (Leake)
132

10/18/2010: TeraGrid Partner SDSC Marks its 25th Anniversary (Froelich/Zverina)
10/11/2010: Blacklight, the World’s Largest Coherent Shared-Memory Computing System, is Up
and Running at the Pittsburgh Supercomputing Center
10.9 Enhancement of Diversity
NCAR
NCAR: SIParCS outreach position advertisements were sent to 15 minority-serving institutions.
SDSC
SDSC participated with TeraGrid colleagues at the 2010 annual conference of the Society for the
Advancement of Chicanos and Native Americans in Science (SACNAS), staffing a booth in the
exhibit hall and encouraging young scientists to participate in TeraGrid computational science
activities at all levels – from student program participants, to users, to research and teaching
faculty, to XD services professionals. The conference drew 3,350 registered participants (1,396
undergraduates and first-time student attendees), among whom were 883 students presenting their
scientific research.
Student and teacher participant statistics from both StudentTECH and TeacherTECH programs at
SDSC showed significant involvement by women and minorities underrepresented in STEM
studies and careers.
TACC
TACC accommodates a wide range of tour requests from school programs where underrepresented
students are in attendance. Highlights include a tour of home-schooled students; a
Computer Security class from Austin Community College; and a tour of the Houston Livestock
Show & Rodeo Committee. Each of these groups was given a presentation about TACC and a
tour of either the Ranger supercomputer or the visualization laboratory. TeraGrid was described
in each presentation.
10.10 International Collaborations
GIG
The GIG worked with Hermann Lederer, Deputy Director of Garching Computing Center of the
Max Planck Society (RZG), to organize the first EU-US HPC Summer School in Catania, Italy
during October, 2010. A report of that summer school is included in Appendix C EOT EU-US
Summer School Report.
Indiana
As part of Indiana University's ongoing relationship with the Technische Universitaet Dresden,
IU was invited to use their expertise in wide area filesystems to help exercise the world's first
commercial 100 Gigabit link. Provided by T-Systems and Alcatel-Lucent in June, the link uses a
single wavelength to bridge the 60 kilometers (37 miles) between Dresden and Freiberg,
Germany.
With equipment provided by Data Direct Networks (DDN) and Hewlett Packard, and in
consultation with Whamcloud, Lustre file systems in both Dresden and Freiberg. The preliminary
goals of the project were to stress test Data Direct Network's S2A9900 and SFA10K disk
appliances and fully exercise Lustre across the 100 gigabit connection.
Using the full duplex capability of the 100 gigabit link, an average transfer rate of 21.904
gigabytes per second was achieved. This rate is over 87 percent of the theoretical maximum
allowed by the link and represents enough data to fill 285 DVDs every minute.
133

LONI
LONI announced the release of SAGA version 1.5.3 (Simple API for Grid Applications). It is
now available on LRZ at Munich and is being installed in Amsterdam – part of the TG-DEISA
project. SAGA has achieved piece-wise local execution of ensemble-MD runs on 2 DEISA
machines, and will support coupled runs in the near future.
NCAR
NCAR: The user community of NCAR’s Asteroseismic Modeling Portal (AMP) is increasing in
size with many new international participants. The gateway now has users from France,
Germany, Australia, Italy, Spain, Austria, and the United Kingdom, in addition to its community
of U.S. researchers.
ORNL
The ORNL RP participated in the conference organizing committee for the 8 th NOBUGS
conference held in October in Galtinburg, TN. Attendance was over 50 from four different
continents. NOBUGS is an ongoing collaboration among neutron, light, and muon sources. It is a
forum to discuss and share advances in data acquisition, data management, analysis, and other
user group software.
The ORNL RP continues to collaborate with the Swiss NSF funded Sinergia project focused on
the fitting of diffuse neutron and x-ray scattering patterns in order to characterize the molecular
structure of materials exhibiting diffuse scattering features such as streaks. <
http://www.oci.uzh.ch/diversa/SNFProject/index.php> As the initial project begins to approach
its end, the collaboration is concentrating on finishing publication submission for completed work
and also exploring future opportunities in this area.
PSC
During 4-Oct-2010 to 7-Oct-2010, Phil Blood of PSC gave an invited talk at the first European-
US Summer School on HPC Challenges in Computational Sciences organized in a joint effort by
TeraGrid and the Distributed European Infrastructure for Supercomputing Applications (DEISA)
in Catania, Italy. There were 60 students in attendance at the summer school: 25 from U.S.
institutions and 35 from European institutions, with about 20 nationalities represented. Together
with Bernd Mohr from Jülich Supercomputing Centre, Blood spoke on tools and procedures for
performance analysis of parallel applications, focusing on concrete examples from his experience
at PSC working with research groups to improve their codes. Blood’s presentation focused on the
U.S.-based tools involved in the POINT project (Productivity from Open, INtegrated Tools), an
NSF-funded SDCI project in which PSC is a partner, while Mohr focused on European tools
involved in the POINT project and a similar European collaboration. Comments following the
talk by both instructors and students indicated that the presentation had been helpful and wellreceived.
The talk itself was only a small part of the experience, as Blood spent the week
associating with an array of students and instructors from different countries and backgrounds.
During this time, Blood helped students directly with some of the problems and challenges that
they are facing in computational science and formed several mutually beneficial associations with
both students and instructors at the summer school.
TACC
TACC continues to support researchers and students in the UT Austin-Portugal Collaboratory.
TACC is also collaborating with the Korea Institute of Science and Technology Information
(KISTI) to create an integrated computational workflow and toolset that will be made widely
available to serve the academic life science research community.
134

10.11 Broader Impacts
LONI
As part of the OGF (Open Grid Forum), multiple standards-compliant distributed backends were
developed for the Future Grid PGI (Production Grid Infrastructure) and GIN (Grid
Interoperability Now) activities.
NICS
NICS has an ongoing relationship with the University of Tennessee-Knoxville’s chapter of the
Society of Women Engineers. NICS Assistant Director Patricia Kovatch has given talks at
campus meetings of the group, and NICS will be assisting SWE when it hosts its Regional
Conference in the Spring of 2011.
ORNL
As mentioned above in section 7, John Cobb (ORNL), Chris Jordan (TACC) and Dan Katz (U.
Chicago) continue to work to bring the DataONE datanet effort and TeraGrid efforts into closer
collaborative arrangements.
The outreach effort of the ORNL RP to the eBird team at Cornell is already having a broader
impact in several dimensions. First, the direct impact of using TeraGrid resources is leading to
more accurate and more useful analyses as the basis of land-use and conservation policy
decisions. Second, this is a pathfinder effort that is showing the way to utilize HPC in the
observational biological sciences in order to leverage the scientific utility of their collected data.
In particular, the extreme volunteer “citizen science” aspect of eBird means that it is able to
collect much larger and higher quality data samples than would ever be possible from a funded
field effort. With larger data sets comes the opportunity for much higher resolution and thus more
useful analyses, but those results must also use larger and larger computational resources (in
terms of computation, memory size, and I/O stress) in order to realize this potential. Third, this
new user community and application approach (large scale statistical analysis) broadens
TeraGrid’s view of science application areas and brings a real-science data intensive use case that
can help TeraGrid better understand and tailor its operations to support.
SDSC
An NSF task force on Cyberlearning and Workforce Development involved several TeraGrid
EOT community members, including SDSC’s Baxter who served as a committee chair for
training, K-14, and informal education. The task force’s analysis of risks and opportunities for
workforce development associated with cyberinfrastructure and computational STEM will advise
both NSF program development and XD services well into the future.
TACC
TACC is endeavoring to broaden the scope of the research and education projects that it supports,
including outreach to the Humanities, Arts & Social Sciences. In October, TACC hired Rob
Turknett as the Digital Media, Arts & Humanities Coordinator for the center. In that capacity, he
is developing collaborations with humanities researchers and educating emerging communities in
Texas about the opportunities for computing at TACC.
135

11 Project Management
11.1 Highlights
The reporting process was modified this quarter to accommodate a change in the reporting
requirements as we enter the final phase of the TeraGrid.
11.2 Project Management Working Group
11.2.1 Tracking
The IPP/WBS tracking process is continuing with project managers reviewing status with area
directors. The area directors are finding the tracking process to be a useful exercise, providing
them with a thorough overview of their area’s status against the plan. The update is included as an
appendix to this report.
11.2.2 Reporting
The reporting process was modified this quarter to accommodate a change in the reporting
requirements as we enter the final phase of the TeraGrid. Normally we would generate an annual
report for the calendar year, but later this year we will be generating a final report that includes
calendar year 2010 as well as the activity in 2011.
12 Science Advisory Board Interactions
No Science Advisory Board (SAB) meetings took place during this period. The next SAB
meeting, which may be the final one under the current TeraGrid project, will be held Jan. 20-21
2011.
13 Collaborations with Non-TeraGrid Institutions
13.1 Data Systems
DataONE: John Cobb at the ORNL-RP continues to work with the DataONE DataNet project,
serving on its leadership team.
REDDnet: The ORNL-RP continues to work with the REDDnet collaboration. Three different
TeraGrid sites now have REDDnet nodes.
13.2 Science Projects
NEES: John Cobb (ORNL) has joined the NEEScomm Governing board. Although not directly a
TeraGrid/NEES partnership, this does create mutual opportunities for TeraGrid and NEES as
NEEScomm (www.nees.org) continues to improve its data management and curation efforts as
well as it HPC strategy. Additionally, John Cobb and Daniel S. Katz (UC/ANL) are on the
steering committee for a NEES-sponsored data workshop in February 2011.
TACC continues to participate in the World Community Grid (WCG) effort. The main TACC
resources contributing to the solution of problems addressed by the WCG is the visualization
system Colt (40 compute cores). The Stampede cluster that previously contributed cycles was
decommissioned 10/1. TACC staff desktops and laptops also provide cycles to the project. Over 9
years of run time have been accumulated (19,377 results) on TACC resources in the reporting
period.
136

14 TeraGrid/Open Science Grid (OSG) Joint Activity
14.1 ExTENCI
The Extending Science Through Enhanced National Cyberinfrastructure (ExTENCI) Project is a
joint Open Science Grid (OSG) and TeraGrid project, funded by OCI. The PIs are Paul Avery
(U. Florida), Ralph Roskies (PSC), and Daniel S. Katz (U. Chicago).
The goal of ExTENCI is to develop and provide production quality enhancements to the National
CyberInfrastructure that will enable specific science applications to more easily use both OSG
and TeraGrid or broaden access to a capability to both TeraGrid and OSG users.
ExTENCI has four primary areas of work:
14.1.1 ExTENCI – Distributed File System (Lustre-WAN)
Hardware and software tasks during this period:
• The Distributed File System became operational in the EXTENCI.ORG Kerberos realm.
• Three clients clusters at FNAL are mounting the distribute file system for application
testing.
• A fix for a Lustre bug that was causing server instability has been installed for testing at
PSC. Packaging for installing this fix in the production system has been delivered to UF.
• The project is now supporting two new Linux OS distributions: Centos and Open
Scientific Linux. These Linux variants are used at client sites. Having these OS’s
supported will broaden the reach of the distributed file system.
• Documentation for client installation has been developed and posted on the project
website. The documentation was used by FNAL for their installation.
• Network testing was performed between client sites and UF.
• System monitoring tools have been installed at UF.
• The distributed file system was demonstrated at SC10. Clients in the CalTech and PSC
booths on the show floor mounted the file system at UF. Performance tests were done
from the floor and graphs of the performance have been posted to the project archive. A
presentation on the status of distributed Lustre efforts was delivered at SC10 to the
HPCFS working group. This presentation included a slide on the ExTENCI distribute
file system.
• The project has begun to explore packaged Virtual Machines (VM’s) that incorporate the
CMS and Lustre software. Work will be coordinated with the Virtual Machines project
within ExTENCI.
14.1.2 ExTENCI – Virtual Machines
Hardware and software tasks during this period:
• Purdue obtained a TeraGrid allocation to track usage of related activities of the
virtualization efforts. The allocation includes Wispy, Purdue Condor and ASQL. Users
have been added to this allocation and the glideinWMS team can now submit to Purdue
Condor and Wispy
• On the CMS-related work, Purdue’s proof-of-concept VM was unsuccessful running at
Clemson due to network issues. We are recreating a VM with different virtual network
scheme.
137

• Purdue is working with CMS GlideinWMS team:
• The project discussed SOAP on Nimbus Cloud "Wispy".
• Wispy upgrade began (to the latest release of Nimbus software to get better SOAP
support the CMS GlideinWMS). Still working to resolve issues that are encountered
during the upgrade.
• Providing starter VMs to GlideinWMS team to test Condor-C submission to Windows
lab cloud.
• Iteration and testing with GlideinWMS team is planned.
• The Purdue has successfully started VMs from RENCI to Clemson using Engage job
submission mechanisms.
• Purdue participated in the discussion of planned VM extension work with the ExTENCI
team, including the initial discussion of integrating Lustre-WAN enabled CMS VMs and
testing/running them on Purdue resources.
• Alex Younts has joined the Purdue team as a full time staff and began working on the
ExTENCI project. Alex had been working with Wispy and HPC systems as an
undergraduate student in the past two years.
14.1.3 ExTENCI – Workflow & Client Tools
Hardware and software tasks during this period:
SCEC:
• Task I1.1 (Split SCEC processing into TG and OSG parts, run complete processing endto-end
on both infrastructures), started 8/1/2010, to finish 10/31/10, 100% complete.
We have demonstrated this functionality, and it works as a demonstration, but is not
practical for this science application, as OSG available slots are generally found on
multiple sites, and large data sets cannot be stored (long-term) on most OSG sites due to
limited disk storage.
• Task I1.2 (Examine and report on alternatives for data movement in the contexts of
performance and ease of use (Performance improvement by using OSG in addition to
TeraGrid will depend on the ability to lease OSG resources, both data and
compute)), started 8/1/2010, to finish 1/31/11, about 50% complete.
We have initially explored using Globus online for the data transfers, which has been
functional. We are also exploring the data transfer mechanism within Swift. We are
working with to understand performance and usability, and to improve both. This task
has become tied to the next, and will be delayed due to that tie.
• Task I1.3 (Examine and report on alternatives for task mapping, etc., in the contexts of
performance and ease of use (Performance improvement by using OSG in addition to
TeraGrid will depend on the ability to lease OSG resources, both data and compute)), to
start 11/1/2010, to finish 6/30/11, about 40% complete.
We can use Swift to launch tasks on OSG, and can get up to 2k cores. We have a new
model for using them, but have data flow problems currently; we are working with the
OSG Engage VO and other parts of OSG to overcome this. We are now looking into
potential software bottlenecks (tied to task I1.2 above).
Protein Structure:
138

• Task I2.1 (Run RAPTOR-OOPS workflow on TeraGrid.), started 8/1/2010, to
finish 10/31/10, 20% complete.
We have run limited testing of OOPS on Ranger, and plan to add Abe and QueenBee.
• Task I2.2 (Run RAPTOR-OOPS workflow on TeraGrid and OSG. Determine effective
site configuration and measure workflow behavior on the combined resources.),
started 8/1/2010, to finish 1/31/11, 10% complete.
Ran over 100,000 "Glass" jobs (which use the OOPS framework) on OSG/Engage;
installed on ~14 sites; running well on 5. Max concurrency to date is ~750 job slots.
Scripts are now ready for combined OSG/TG execution. Need to get/renew TG
allocation.
• Task I2.3 (Run one RAPTOR-OOPS workflow segment under MPI on TeraGrid and
other segments on OSG. Measure workflow behavior and efficiency.), to be started
2/1/2011, to finish 3/31/2011, 0% completed
Task in future.
• Task I2.4 (Determine effective data management strategies for RAPTOR-OOPS and
measure workflow behavior and efficiency.), to be started 3/1/2011, to finish 7/31/2011,
100% complete.
Task in future.
• Task I2.5 (Enhance workflow packaging and usability to enable additional users.
Measure and report on workflow usage and site utilization by the OOPS user
community.), started 12/1/2010, to finish 1/31/2012, 10% complete.
Work proceeding on tools and documentation needed to engage new users. We are testing
these techniques with new user M. Parisien for protein-RNA docking at UChicago.
Parisien is using simple Condor-G scripts for application installation. Parisien has his app
installed on about 10 OSG sites now. We have discovered issues with the Swift resource
selector in these tests, and are now re-testing.
• Task I2.6 (Adjust site selection based on application characteristics.), to be
started 5/1/2011, to finish 7/31/2012, 0% complete.
Task in future.
• Task I2.7 (Engage one new user community in combined OSG-TeraGrid usage using
ExTENCI techniques. Measure and report on workflow usage and site utilization by the
new user community.), started 10/1/2010, to finish 4/28/2012, 30% complete.
We have engaged new user G. Hocky of Reichman Group at Columbia U. Informal
measurements indicate 100K+ jobs of avg 1.5 hour completed in 4Q2010 using Swift on
OSG Engage VO. Used 18 sites, about 6 primary sites; peaks of over 1K cores.
14.1.4 ExTENCI – Job Submission Paradigms
These tasks have not started yet.
139

A
Publications Listing
A.1. TeraGrid Staff Publications
A.1.1 TeraGrid Staff Publications (for 2010 Q3)
For Q3 2010, the TeraGrid sites reported the following publications authored in whole or in part
by GIG- or RP-funded staff members.
1. Wenjun Wu, Thomas D. Uram, Michael Wilde, Mark Hereld, and Michael E. Papka, A Web 2.0-Based
Scientific Application Framework, 8 th International Conference on Web Services, July 5-10, 2010,
Miami, Florida.
2. Wenjun Wu, Thomas Uram, Michael Wilde, Mark Hereld and Michael E. Papka, Accelerating Science
Gateway Development with Web 2.0 and Swift, TeraGrid 2010 Conference, Pittsburgh, Pennsylvania,
August 2-5, 2010.
3. Dooley R., Dahan M. 2010 “TeraGrid ’10”, Action Folders
4. Dooley R. 2010 “Proceedings of The International Workshop on Science Gateways 2010”, Recipes for
Success in New Science Gateway Development
5. Dooley R., Dahan M. 2010 “TeraGrid ’10”, The TeraGrid Mobile User Portal: Ticketing, Data, and
Complete Job Management On The Go
6. Dooley R., Dahan M. 2010 “TeraGrid ’10”, Comprehensive File Management in the TeraGrid
7. Daruru S., Dhandapani S., Gupta G., Iliev I., Xu W., Navratil P., Marin N., Ghosh J. 2010 “Proceedings
of International Workshop on Knowledge Discovery Using Cloud and Distributed Computing Platforms
(KDCloud, 2010) in Cooperation with IEEE ICDM 2010”, Distributed, Scalable Clustering for Detecting
Halos in Terascale Astronomy
8. Jordan C., 2010 “26 th IEEE (MSST2010) Symposium on Massive Storage Systems and Technologies”,
Data Infrastructure in the TeraGrid
9. Willmore F., Rosales C., Cazes J. 2010 “TeraGrid ‘10”, Experiences with the Distributed Debugging
Tool
10. Navratil, P. 2010 “Dissertation, The University of Texas at Austin”, Memory Efficient, Scalable Ray
Tracing
11. Jeong B., Abram G., Johnson G., Navratil P. 2010 “NVIDIA GTC Conference”, Large-Scale
Visualization Using a GPU Cluster
12. Wilkins-Diehr, N., Lawrence, K., "Opening Science Gateways to Future Success: The Challenges of
Gateway Sustainability", accepted for Gateway Computing Environments workshop, November, 2010.
13. Wilkins-Diehr, N., Baru, C., Gannon, D., Keahey, K., McGee, J., Pierce, M., Wolski, R., Wu, W.
"Science Gateways: Harnessing Clouds and Software Services for Science" in Cloud Computing and
Software Services: Theory and Techniques, edited by Syed A. Ahson and Mohammad Ilyas. [Boca
Raton: CRC Press, 2011].
14. H. Karimabadi, H. X. Vu, B. Loring, Y. Omelchenko, M. Tatineni, A.
15. Majumdar, and J. Dorelli, "Enabling Breakthrough Kinetic Simulations of the Magnetosphere Using Petascale
Computing," Asia Oceania Geosciences Society (AOGS) Hyderabad, India, July 5-9, 2010
16. H. Karimabadi, H. X. Vu, B. Loring, Y. Omelchenko, M. Tatineni, A.
17. Majumdar, and J. Dorelli, "3D Global Hybrid Simulations of the Magnetosphere and I/O Strategies for Massively
Parallel Kinetic Simulations,” TG10, Pittsburgh, August 2-5, 2010
18. Xu, D., Williamson, M.J., Walker, R.C., "Advancements in Molecular Dynamics Simulations of
Biomolecules on Graphical Processing Units.", Ann. Rep. Comp. Chem., vol 6. 2010, 2-19 (Chapter 1).
19. Goetz, A.W., Woelfe, T., Walker, R.C., "Quantum Chemistry on Graphics Processing Units", Ann. Rep.
Comp. Chem., vol 6. 2010, 21-35 (Chapter 2).
20. "Creating the CIPRES Science Gateway for Inference of Large Phylogenetic Trees" by M.A. Miller, W. Pfeiffer,
and T. Schwartz was submitted and accepted for presentation at the GCE10 workshop in conjunction with SC10.
21. Y. Cui, K. Olsen, T. Jordan, K. Lee, J. Zhou, P. Small, G. Ely, D. Roten, D. K. Panda, A. Chourasia, J.
Levesque, S. Day and P. Maechling (2010). “Scalable Earthquake Simulation on Petascale
Supercomputers”, to be presented at Supercomputing 2010, New Orleans, Nov 2010

22. Chourasia, A. 2010. GlyphSea. In ACM SIGGRAPH 2010 Computer Animation Fesitval (Los Angeles,
California, July 26 - 30, 2010). SIGGRAPH '10. ACM, New York, NY, 111-111.
23. L. Zhao, W. Lee, C. X. Song, M. Huber and A. Goldner. "Bringing High Performance Climate Modeling
into the Classroom", TeraGrid 2010 conference, Pittsburgh, PA, August 2010.
24. Adam Carlyle, Stephen Harrell and Preston Smith. "Cost-effective HPC: The Community or the Cloud",
accepted to the 2nd IEEE International Conference on Cloud Computing Technology and Science,
Indianapolis, IN, November 30 - December 3, 2010.
25. Palencia, J., Budden, R., and Sullivan, K. 2010. Kerberized Lustre 2.0 over the WAN. In Proceedings of
the 2010 TeraGrid Conference (Pittsburgh, Pennsylvania, August 02 - 05, 2010). TG’10. ACM, New
York, NY, 1-5.
26. Wei He, Matthew J. Crawford, Srinivas Rapirredy, Marcela Madrid, Roberto R. Gil, Danith H. Ly,
Catalina Achim, “The Structure of a γ-Modified Peptide Nucleic Acid Duplex”, Mol. BioSyst. 6, 1619,
1629, 2010.
27. D Katz, Scott Callaghan, Robert Harknesse, Shantenu Jha, et al “Science on the TeraGrid”, accepted
for Computational Methods in Science and Technology, in (2011)
28. S Sehgal, A Merzky, M Erdelyim, S Jha, “Application-level Cloud Interoperability”, Springer book ”Grids,
Clouds and Virtualization”, Edited by Massimo Cafaro (Fall 2010)
29. A Luckow and S. Jha, “Astractions for Loosely-Coupled and Ensemble-based Simulations on Microsoft
Azure”, Short Paper accepted for IEEE CloudCom (Dec 2010)
30. Y el-Khamra, H Kim, S Jha, and M Parashar, “Exploring the Performance Fluctuations of HPC
Workloads on Clouds”, Short Paper accepted for IEEE CloudCom (Dec 2010)
31. D. Wright, O. Kenway, P Coveney and S. Jha, “Computational Estimation of Binding Affinities for
Patient Derived HIV-1 Protease Sequences Bound to Lopinavir”, accepted First International
Conference on Virtual Physiological Human (VPH) 2010,
32. “Modelling Data-driven CO2 Sequestration Using Distributed HPC CyberInfrastructure”, Y Khamra, S
Jha and C White, accepted for TeraGrid 2010 (published by ACM)
33. “Exploring Adaptation to Support Dynamic Applications on Hybrid Grids-Clouds Infrastructure”, H. Kim,
Y. el-Khamra, S. Jha and M. Parashar, Accepted, ACM ScienceClouds Workshop as part of HPDC
2010
34. “Exploring the RNA Folding Energy Landscape Using Scalable Distributed Cyberinfrastructure”, J. Kim,
W Huang, S Maddineni, F Aboul-ela, S. Jha, Accepted, Emerging Computational Methods for Life
Sciences, in conjunction with HPDC 2010.
35. Liu, Y. and Wang, S. 2010. “Asynchronous Implementation of A Parallel Genetic Algorithm for the
Generalized Assignment Problem”. In: Proceedings of TeraGrid 2010 conference. August 1-5, 2010.
Pittsburgh, PA
36. Padmanabhan, A., Tang, W., and Wang, S. 2010. “Agent-based Modeling of Agricultural Land Use on
TeraGrid”. In: Proceedings of TeraGrid 2010 conference. August 1-5, 2010. Pittsburg, PA
37. Wang, S. and Liu, Y. 2010. “GISolve 2.0: Geospatial Problem Solving Environment Based on
Synthesizing Cyberinfrastructure and Web 2.0”. In: Proceedings of TeraGrid 2010 conference. August
1-5, 2010. Pittsburgh, PA
38. Bennett, D.A., Tang, W., and Wang, S. 2010. “Toward an Understanding of Provenance in Complex
Land Use Dynamics.” Land Use Science, forthcoming
Tang, W., Bennett D. A., and Wang, S. 2010. “A Parallel Agent-based Model of Land Use Opinions.”
Journal of Land Use Science, forthcoming
A.1.2 TeraGrid Staff Publications (for 2010 Q4)
For Q4 2010, the TeraGrid sites reported the following publications authored in whole or in part
by GIG- or RP-funded staff members.
1. Wang, S., Wilkins-Diehr, N., Shi, X., Vatsavai, R. R., Zhang, J., and Padmanabhan, A. 2010.
"Proceedings of the ACM SIGSPATIAL International Workshop on High Performance and Distributed
Geographic Information Systems (ACM SIGSPATIAL HPDGIS 2010)." ACM Digital Library
2. Liu, Y., Wu, K., Wang, S., Zhao, Y., and Huang, Q. "A MapReduce Approach to Gi*(d) Spatial Statistic".
In: Proceedings of the ACM SIGSPATIAL International Workshop on High Performance and Distributed

Geographic Information Systems (ACM HPDGIS 2010), November 2, 2010, San Jose, CA, USA
3. Padmanabhan, A., and Wang, S. "A Distributed Resource Broker for Spatial Middleware Using Adaptive
Space-Filling Curve". In: Proceedings of the ACM SIGSPATIAL International Workshop on High
Performance and Distributed Geographic Information Systems (ACM HPDGIS 2010), November 2,
2010, San Jose, CA, USA
4. D. S. Katz, S. Callaghan, R. Harkness, S. Jha, K. Kurowski, S. Manos, S. Pamidighantam, M. Pierce, B.
Plale, C. Song, J. Towns, "Science on the TeraGrid," Computational Methods in Science and
Technology (CMST), v.Special Issue 2010, pp. 81-97, 2010.
5. Cobb, J. W. “SNS and TeraGrid's NSTG: A best practice for integration of national cyberinfrastructure
with an experimental user facility” NOBUGS 2010 Oct. 10-13, 2010 Gatlinburg, TN <
http://neutrons.ornl.gov/conf/nobugs2010/>
6. Javier V. Goicochea, Marcela Madrid, and Cristina Amon, “Hierarchical Modeling of Heat Transfer in
Silicon-Based Electronic Devices”, ASME Journal of Heat Transfer 132, 102401 (2010).
7. Kh. Odbadrakh, A. Rusanu, G. M. Stocks, and G. D. Samolyuk, M. Eisenbach, Yang Wang, D.M.
Nicholson, “Calculated Electronic and Magnetic Structure of Screw Dislocations in Alpha Iron”, Journal
of Applied Physics (accepted for publication).
8. Alexander J. Ropelewski, Hugh B. Nicholas, Jr. and Ricardo R. Gonzalez Mendez, “MPI-PHYLIP:
Parallelizing Computationally Intensive Phylogenetic Analysis Routines for the Analysis of Large Protein
Families”, PLoS One-
9. Yang Wang, D.M.C. Nicholson, G.M. Stocks, Aurelian Rusanu, Markus Eisenbach, and R. E. Stoller, “A
study of radiation damage effects on the magnetic structure of bulk Iron”, Journal of Applied Physics,
(accepted for publication).
10. H. Zhang, J. Woo, L. Zhao, D. Braun, X.C. Song, M. Lakshminarayanan. "Domain-specific web services
for scientific application developers," Proceedings of the 6th Gateway Computing Environments
Workshop (GCE10) at SC10, New Orleans, LA, November 2010.
11. Understanding the Impact of Emerging Non-Volatile Memories on High-Performance, IO-Intensive
Computing, Adrian M. Caulfield, Joel Coburn, Todor Mollov, Arup De, Ameen Akel, Jiahua He, Arun
Jagatheesan, Rajesh K. Gupta, Allan Snavely, and Steven Swanson, Supercomputing, 2010.
(Nominated for best technical paper and best student paper).
12. "Creating the CIPRES Science Gateway for Inference of Large Phylogenetic Trees" by M.A. Miller, W.
Pfeiffer, and T. Schwartz was presented at the GCE10 workshop in conjunction with SC10.
13. E. McQuinn, A. Chourasia, J. H. Minster, J. Schulze (2010), GlyphSea: Interactive Exploration of
Seismic Wave Fields Using Shaded Glyphs, Abstract IN23A-1351 presented at 2010 Fall Meeting,
AGU, San Francisco, Calif., 13-17 Dec.
14. Yifeng Cui, Kim B. Olsen, Thomas H. Jordan, Kwangyoon Lee, Jun Zhou, Patrick Small, Daniel Roten,
Geoffrey Ely, Dhabaleswar K. Panda, Amit Chourasia, John Levesque, Steven M. Day, and Philip
Maechling. 2010. Scalable Earthquake Simulation on Petascale Supercomputers. In Proceedings of the
2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and
Analysis (SC '10). IEEE Computer Society, Washington, DC, USA, 1-20.
15. Burtscher M., Kim B., 2010 “Proceedings of the ACM/IEEE Supercomputing Conference 2010”
PerfExpert: An Easy-to-Use Performance Diagnosis Tool for HPC Applications
16. Navratil P. 2010 “IBERGRID 2010” Introduction to Longhorn and GPU-Based Computing
17. Navratil P., 2010 “Proceedings of the Sixth High-End Visualization Workshop Increasing Hardware
Utilization for Peta-Scale Visualization
18. Jeong B., Abram G., Johnson G., Navratil P. 2010 “NVIDIA GPU Technology Conference” Large-Scale
Visualization Using a GPU Cluster
19. Hanlon M., Mock S., Nuthulapati P., Gonzales M., Soltis P., Soltis D., Majure L., Payton A., Mishler B.,
Tremblay T., Madsen T., Olmstead R., McCourt R., Wojciechowski M., Merchant N. 2010 “IEEE
Proceedings of Gateway Computing Environments 2010” My-Plant.org: A Phylogenetically Structured
Social Network

A.2. Publications from TeraGrid Users
A.2.1 Publications from TeraGrid Users (for 2010 Q3)
The following 675 publications were gathered primarily from renewal Research submissions to
the September 2010 TRAC meeting. The extraction process was conducted by a UCSD
undergraduate student at SDSC. The publications are organized by field of science and by the
proposal with which they were associated.
Advanced Scientific Computing
MCA06N043
1. A. Uzun and M. Y. Hussaini, “Simulation of Noise Generation in Near-Nozzle Region of a Chevron
Nozzle Jet,” AIAA Journal, Vol. 47, No. 8, August 2009, pp. 1793 – 1810.
2. H. Zhao, A. Uzun, M. Y. Hussaini and S. L. Woodruff, “Computations of Flow Past a Circular
Cylinder Using a Continuous-Turbulence Model,” AIAA Journal, Vol. 47, No. 10, October 2009, pp. 2460 –
2466.
3. A. Uzun, and M. Y. Hussaini, “Simulations of Vortex Formation Around a Blunt Wing Tip,”AIAA
Journal, Vol. 48, No. 6, June 2010, pp. 1221 – 1234.
4. J. Bin, A. Uzun and M. Y. Hussaini, “Adaptive Mesh Refinement for Chevron Nozzle Jet Flows,”
Computers and Fluids Journal, Vol. 39, No. 6, June 2010, pp. 979 – 993.
5. A. Uzun and M. Y. Hussaini, “High-Fidelity Numerical Simulation of a Chevron Nozzle Jet Flow,”
Under review for publication in International Journal of Aeroacoustics, 2010.
6. J. Bin, A. Uzun and M. Y. Hussaini, “Adaptive Grid Redistribution Method for a Domain with
Complex Boundaries,” Under review for publication in Journal of Computational Physics, 2010.
7. A. Uzun and M. Y. Hussaini, “High-Fidelity Numerical Simulations of a Round Nozzle Jet Flow,” In
preparation, 2010.
8. A. Uzun and M. Y. Hussaini, “Prediction of Tandem Cylinder Noise using Delayed Detached Eddy
Simulation,” In preparation, 2010.
Astronomical Sciences
AST020001
9. “Selection Rules for the Nonlinear Interaction of Internal Gravity Waves,” Jiang, C.-H. and Marcus,
P. S., March 2009, Phys. Rev. Letters, 102, 124502-1 to 124502-4.
10. “Jupiter’s Shrinking Great Red Spot and Steady Oval BA: Velocity Measurements with the ACCIV
Automated Cloud Tracking Method,” Asay-Davis, X., Marcus, P., Wong, M. and e Pater, I., May 2009,
Icarus, 203, 164–188.
11. “Changes in Jupiter’s Zonal Velocity Profile between 1979 and 2008,” Asay-Davis, X., Wong, M.,
Marcus, P., and de Pater, I., under review (minor revision required), Icarus, 2010.
12. “Jupiter’s Zonal Winds: Are They Bands of Homogenized Potential Vorticity and Do They Form a
Monotonic Staircase” Marcus, P. and Shetty S., submitted to Philsophical Transactions Royal Society A,
2010.
13. “KECK AO and HST Images of Vortices on Jupiter,” de Pater, I., Wong, M., Luszcz-Cook, S,
Adamkovics, M., Conrad, A., Marcus, P., Asay-Davis, X., and Go, C., under review (minor revision required),
Icarus, 2010.
14. “Three-Dimensional Simulations of Kelvin-Helmholtz Instability in Settled Dust Layers in
Protoplanetary Disks,” Barranco, J.A., February 2009, The Astrophysical Journal, 691, 907–921. 8. “Forming
Planetesimals by Gravitational Instability: I. The Role of the Richardson Number in Triggering the Kelvin-
Helmholtz Instability,” Lee, A., Chiang, E., Asay-Davis, X., Barranco, J., August 2010, The Astrophysical
Journal, 718.

AST070032
15. P. Sharma, I. J. Parrish, E. Quataert, “Thermal Instability with Anisotropic Thermal Conduction and
Adiabatic Cosmic Rays: Implications for Cold Filaments in Galaxy Clusters, ” accepted, Astrophysical
Journal (2010), arXiv:1003.5546 Non-Refereed Publications
16. P. Sharma, B. D. G. Chandran, E. Quataert, I. J. Parrish, “Turbulence and Mixing in the Intracluster
Medium,” in the Proceedings of The Monster’s Fiery Breath: Feedback in galaxies, groups, and clusters,
Madison, Wisconsin, ed. S. Heinz, E. Wilcots (2009), arXiv:0909.0270 Submitted/In Progress
17. P. Sharma, P. Colella, D. F. Martin, “Numerical Implementation of Streaming Down the Gradient:
Application to Fluid Modeling of Cosmic Rays,” under review, SIAM Journal of Scientific Computing (2009),
arXiv:0909.5426
18. P. Sharma, G. W. Hammett, “A Fast Semi-implicit Method for Anisotropic Diffusion,” to be
submitted to J. Comp. Phys.
AST080005
19. “Regarding the Line-of-Sight Baryonic Acoustic Feature in the Sloan Digital Sky Survey and Baryon
Oscillation Spectroscopic Survey Luminous Red Galaxy Samples” Kazin, E. A., Blanton, M. R., Scoccimarro,
R., McBride, C. K., & Berlind, A. A., 2010, The Astrophysical Journal, in press
20. “The Baryonic Acoustic Feature and Large-Scale Clustering in the Sloan Digital Sky Survey
Luminous Red Galaxy Sample” Kazin, E. A., Blanton, M. R., Scoccimarro, R., McBride, C. K., Berlind, A. A.,
Bahcall, N. A., Brinkmann, J., Czarapata, P., Frieman, J. A., Kent, S. M., Schneider, D. P., & Szalay, A. S.,
2009, The Astrophysical Journal, 710, 1444
21. “From Finance to Cosmology: The Copula of Large-Scale Structure” Scherrer, R. J., Berlind, A. A.,
Mao, Qingqing, & McBride, C. K., 2009, The Astrophysical Journal (Letters), 708, 9 Submitted or In
Preparation:
22. “The Large Suite of Dark Matter Simulations (LasDamas) Project: Mock Galaxy Catalogs for the
Sloan Digital Sky Survey” McBride, C. K., Berlind, A. A., Scoccimarro, R., Piscionere, J. A., Busha, M.,
Gardner, J., Manera, M., Wechsler, R. H., & Van den Bosch, F., 2010, The Astrophysical Journal, in
preparation
23. “Accurate Determination of the Abundance of Massive Clusters From the LasDamas Project”
McBride, C. K., Berlind, A. A., Manera, M., Scoccimarro, R., Busha, M., Manera, M., Wechsler, R. H., & Van
den Bosch, F., 2010, The Astrophysical Journal, in preparation
24. “Density Profiles of Dark Matter Halos From the LasDamas Project” Musher, D. E., Robbins, K.,
Berlind, A. A., McBride, C. K., Busha, M., Wechsler, R. H., 2010, The Astrophysical Journal, in preparation
25. “Density Profiles of Galaxy Groups and Clusters in the Sloan Digital Sky Survey” Norris, J. M.,
Berlind, A. A., & McBride, C. K., 2010, The Astrophysical Journal, in preparation
26. “The Luminosity and Color Dependence of the Sloan Digital Sky Survey Galaxy 3-Point Correlation
Function” Marin, F, Nichol, R., Frieman, J., & McBride, C. K., 2010, The Astrophysical Journal, in
preparation
AST080029
27. Penna, R., McKinney, J. C. et al. 2010, MNRAS, Simulations of Magnetized Disks Around Black
Holes: E®ects of Black Hole Spin, Disk Thickness, and Magnetic Field Geometry, in press,
http://adsabs.harvard.edu/abs/2010arXiv1003.0966P
28. Broderick, A. & McKinney, J. C. 2010, ApJ, Parsec-scale Faraday Rotation Measures from General
Relativistic MHD Simulations of Active Galactic Nuclei Jets, submitted,
http://adsabs.harvard.edu/abs/2010arXiv1006.5015B
29. Uzdensky, A. & McKinney, J. C. 2010, Physics of Plasmas, Magnetic Reconnection with Radiative
Cooling. I. Optically-Thin Regime, submitted, http://adsabs.harvard.edu/abs/2010arXiv1007.0774U

30. Tchekhovskoy, A., Narayan, R., McKinney, J. C. 2010, New Astronomy, Magnetohydrodynamic
Simulations of Gamma-Ray Burst Jets: Beyond the Progenitor Star, in press,
http://adsabs.harvard.edu/abs/2009arXiv0909.0011T
31. McKinney, J. C., Blandford, R., Uzdensky, D. 2010, MNRAS, General Relativistic
Magnetohydrodynamical Simulations of Gamma-Ray Bursts, in progress
32. McKinney, J. C. & Kohri, K. 2010, MNRAS, Code and Tables for Nuclear Equations of State and
Neutrino Opacities for General Relativistic Magnetohydrodynamical Simulations of Gamma-Ray Bursts,
submitted
33. McKinney, J. C. 2010, Physics Reports Invited Review, Simulating Black Hole Accretion Flows and
Jets, in progress
34. McKinney, J. C. & Blandford, R.D. 2010, MNRAS, Observed GRB Relativistic Jet Variability as a
Pattern Effect Induced by Unstable Jet Modes, in progress
35. McKinney, J. C. & Tchekhovskoy, A. D. 2010, MNRAS, CPC-HARM: Conservative-Primitive-
Consistent high-accuracy General Relativistic Magnetohydrodynamics code, in progress
AST080048
36. J. Debuhr, E. Quataert, & C. P. Ma, 2010, “The Growth of Massive Black Holes in Galaxy Merger
Simulations with Feedback by Radiation Pressure," MNRAS, submitted
37. J. Debuhr, E. Quataert, C. P. Ma, & P. F. Hopkins, 2010, “Self-Regulated Black Hole Growth via
Momentum Deposition in Galaxy Merger Simulations," MNRAS Letters, 406, L55
AST080049
38. Lehe, R., Parrish, I. J., Quataert, E. 2009, ApJ 707, 404, “The Heating of Test Particles in
Numerical Simulations of Alfv´enic Turbulence”
39. McCourt, M., Parrish, I. J., Quataert, E., & Sharma, P., in preparation, “Nonlinear Saturation of
Buoyancy Instabilities in Dilute Magnetized Plasmas”
40. Parrish,I.J.,Quataert,E.,Sharma,P.2009,ApJ,703,96, “AnisotropicThermalConduction and the
Cooling Flow Problem in Galaxy Clusters”
41. Parrish, I. J., Quataert, E., Sharma, P. 2010, ApJ, 712, 194, “Turbulence in Galaxy Cluster Cores:
A Key to Cluster Bimodality”
42. Sharma,P.,Parrish,I.J.,&Quataert,E.2010,AcceptedinApJ,arXiv:1003.5546, “Thermal Instability with
Anisotropic Thermal Conduction and Adiabatic Cosmic Rays: Implications for Cold Filaments in Galaxy
Clusters”
AST090074
43. Hoeflich, P.; Krisciunas, K.; Khokhlov, etal., Secondary Parameters of Type Ia Supernova Light
Curves, ApJ, 710(2010) 444.
44. Fully-threaded-tree AMR simulations of astrophysical and terrestrial combustion, being submitted
into proceedings of the ASTRONUM2010 conference, SanDiego, CA, June 2010.
45. Two incoming presentations at the incoming conference Frontiers in Computational Astrophysics io
October 2010, Lyon, France.
46. Two papers on large-scale modeling of SNIa in preparation.
47. Paper describing 3D modeling of detonation waves with detailed nuclear kinetics, being submitted
to ApJ.
AST090086

48. “Turbulence Stagnation and the Ignition of the Deflagration in Type Ia Supernovae”
Townsley, D.M., Jackson, A.P., Calder, A.C. in preparation
49. “Evaluating the Impact of Untertainties in Turbulence-Flame Interactions on Simulations of Type Ia
Supernovae” Jackson, A.P., Townsley, D.M., Calder, A.C. in preparation
AST090087
50. “Erratum: Effects of metal enrichment and metal cooling in galaxy growth and cosmic star formation
history” J.-H. Choi & K. Nagamine 2009, MNRAS, 395, 1776
51. “Effects of metal enrichment and metal cooling in galaxy growth and cosmic star formation history”
J.-H. Choi & K. Nagamine 2009, MNRAS, 393, 1595
52. “Effects of UV background and local stellar radiation on the H I column density distribution”
53. K. Nagamine, J.-H. Choi, & H. Yajima 2010, submitted to ApJL (arXiv:1006.5345)
54. “Luminosity Distribution of Gamma-Ray Burst Host Galaxies at redshift z=1 in Cosmological
Smoothed Particle Hydrodinamic Simulations: Implications for the Metallicity Dependence of GRBs”
55. Y. Niino, J.-H Choi, M. A. R. Kobayashi, K. Nagamine, T. Totani, Tomonori & B. Zhang 2010,
submitted to ApJ (arXiv:1006.5033)
56. “Escape fraction of ionizing photons from high-redshift galaxies in cosmological SPH simulations”
57. H. Yajima, J.-H. Choi & K. Nagamine 2010, submitted to MNRAS (arXiv:1002.3346)
58. “Multiphase and Variable Velocity Galactic Outflow in Cosmological SPH Simulations” J.-H. Choi &
K. Nagamine 2010, submitted to MNRAS (arXiv:1001.3525)
59. “Effects of cosmological parameters and star formation models on the cosmic star formation
history” J.-H. Choi & K. Nagamine 2009, accepted for publication in MNRAS (arXiv:0909.5425)
60. American Astronomical Society, AAS Meeting 215, 376.02 “Effects of Star Formation Models on
The Cosmic Star Formation History”
61. SFR@50: Filling the Cosmos with Stars at ABBAZIA DI SPINETO SARTEANO, ITALY “Cosmic
star formation rate in LCDM cosmological simulation”
62. Lorentz center workshop in Leiden Netherlands : The Chemical enrichment of the Intergalactic
Medium The effect of multiphase galactic winds on galaxy formation”
63. Sejong University (Korea), April 2009 “Cosmic star formation history in LCDM cosmological
simulation”
64. American Astronomical Society, AAS Meeting 213, 344.02
“Effects of metal enrichment and metal cooling in galaxy growth and cosmic star formation history”
AST090104
65. J. M. Call, J. E. Tohline & L. Lehner 2010 “A generalized advection formalism for relativistic fluid
simulations”, CQGrav, 27, in press
66. 68. W. P. Even & J. E. Tohline 2009, ‘ ‘Constructing Synchronously Rotating Double White
Dwarf Binaries”, Ap.JS., 184, 248, arXiv:0908.2116
67. 69. W. P. Even 2010, PhD Dissertation “Mass Transfer and Mergers in Double White Dwarf
Binaries”, http://etd.lsu.edu/docs/available/etd-04152010-230541/
68. D. Marcello, P. M. Motl & J. E. Tohline 2010, “Radiation-Hydrodynamic Treatment of Binary
Accretion Flows”, ApJ, in preparation
69. P. M. Motl et al. 2010, “Double Degenerate Merger Code Comparison Project: The Importance of
Initial Conditions and the Equation of State”, ApJ, in preparation

70. P. M. Motl, J. E. Tohline & J. Frank 2010 “Convergence to stability in a simulated double white
dwarf binary”, ApJ, in preparation
AST090106
71. K. Beckwith and P. Armitage. Global, High Resolution Simulations of Thin Disks with a Higher
Order Godunov Scheme. ApJ, 2010, in prep.
72. J. B. Simon, J. F. Hawley and K. Beckwith. Accretion States in Vertically Strati fied MRI-driven
Turbulence. ApJ, 2010, in prep.
73. K. Beckwith and J. M. Stone. A Multidimensional, Higher Order Godunov Scheme for Relativistic
Magnetohydrodynamics. ApJSS, 2010, in prep.
AST100040
74. Tchekhovskoy, A., McKinney, J. C., and Narayan, R. (2009). Efficiency of Magnetic to Kinetic
Energy Conversion in a Monopole Magnetosphere. ApJ , 699, 1789–1808.
75. Tchekhovskoy, A., Narayan, R., and McKinney, J. C. (2010a). Black Hole Spin and The Radio
Loud/Quiet Dichotomy of Active Galactic Nuclei. ApJ , 711, 50–63.
76. Tchekhovskoy, A., Narayan, R., and McKinney, J. C. (2010b). Magnetohydrodynamic Simulations
of Gamma-Ray Burst Jets: Beyond the Progenitor Star. New Astronomy, 15, 749.
77. Penna, R. F., McKinney, J. C., Narayan, R., Tchekhovskoy, A., et al. (2010). Simulations of
Magnetized Disks Around Black Holes: Effects of Black Hole Spin, Disk Thickness, and Magnetic Field
Geometry. MNRAS, in press (ArXiv:1003.0966).
MCA00N020
78. Cunningham, A. J., Klein, R. I., McKee, C. F. & Krumholz, M. R. 2010, ApJ, in preperation
79. Kratter, K. M., Matzner, C. D., Krumholz, M. R., & Klein, R. I., 2010, ApJ, 708, 1585
80. Krumholz, M. R., Cunningham, A. J., Klein, R. I., & McKee, C. F. 2010, ApJ, 713, 1120
81. McKee, C. F., Li, P. S. & Klein, R. I. 2010, ApJ, accepted.
82. Li, P. S., McKee, C. F. & Klein, R. I. 2010, in preparation.
83. Offner, S. S. R. & Klein, R. I. & McKee, C. F. & Krumholz, M. R. 2009, ApJ, 703, 1310
84. Offner, S. R ., Kratter, K. M, Matzner, C. D., Krumholz, M. R., and Klein, R. I., ApJ, 2010, submitted
85. Turk, M. J., Abel, T., & O’Shea, B. 2009, Science, 325, 601
MCA04N012
86. R. Cen, The State of the Universe at z ∼ 6, ApJL, submitted, 2010
87. A. Mesinger, S. Furlanetto, R. Cen, 21cmFAST: A Fast, Semi-Numerical Simulation of the High-
Redshift 21-cm Signal, ApJ, submitted, 2010
88. O. Zhan,A.Mesinger, M. McQuinn, H. Trac,R.Cen,L., Hernquist, Comparison of Reionization
Models: Radiative Transfer Simulations And Approximate, Semi-Numeric Models, ApJ, submitted, 2010
89. Z. Zheng, R. Cen, H. Trac, J. Miralda-Escude, Radiative Modeling of Lyman Alpha Emitters. II. New
Effects in Galaxy Clustering, ApJ, submitted, 2010
90. Z. Zheng, R. Cen, H. Trac, J. Miralda-Escude, Radiative Transfer Modeling of Lyman Alpha
Emitters: I. Statistics of Spectra and Luminosity,ApJ, in press, 2010
91. R. Cen, N.E. Chisari, Star Formation Feedback and Metal Enrichment History of the Intergalactic
Medium, ApJ, submitted, 2010

92. N.A. Bond, M.A. Strauss, R. Cen, Crawling the Cosmic Network: Exploring the Morphology of
Structure in the Galaxy Distribution, MNRAS, in press, 2010
93. N.A. Bond, M.A. Strauss, R. Cen, Crawling the Cosmic Network: Identifying and Quantifying
Filamentary Structure, MNRAS, in press, 2010
94. R. Cen, P. McDonald, H. Trac, A. Loeb, Probing the Epoch of Reionization with the Lyman Alpha
Forest at z ∼ 4 − 5, ApJL, 706, L164, 2009
95. J.H. Wise, R. Cen, Ionizing Photon Escape Fractions from High Redshift Dwarf Galaxies,ApJ, 693,
984, 2009
96. M.K.R. Joung,R.Cen, G.Bryan, Galaxy Size Problem at z=3: Simulated Galaxies Are Too
Small,ApJL, 692, L1, 2009
MCA95C003
97. “Simulations of Magnetorotational Turbulence with a Higher-Order Godunov Scheme,” J. B.
Simon, J. F. Hawley, K. Beckwith, ApJ, 690, 974(2009).
98. “Direct Calculation of the Radiative Efficiency of an Accretion Disk Around a Black Hole,” S. C.
Noble, J. H. Krolik, & J. F. Hawley, ApJ, 692, 411(2009).
99. “MHD Simulations of Accretion Disks and Jets :Strengths and Limitations,” J. .F.Hawley,Astrophys.
and Space Sci., 320, 107(2009).
100. “GRMHD Prediction of Coronal Variability in Accreting Black Holes,” S. C. Noble & J. H. Krolik, J.
H., ApJ, 703, 964–975(2009).
101. “Transport of Large Scale Poloidal Flux in Black Hole Accretion,” K.Beckwith,J.F.Hawley,&J.H.
Krolik, Astrophysical Journal, 707, 428–445(2009).
102. “Viscous and Resistive Effects on the Magnet orotational Instability with aNetToroidalField,” Simon,
J. B. & Hawley, J. F., ApJ, 707, 833–843(2009).
103. “What is the Numerically Converged Amplitude of MHD Turbulence in Strati fied Shearing Boxes”
J. Shi & J. H. Krolik, ApJ, 708, 1716–1727(2010).
104. “Dependence of Inner Accretion Disk Stress on Parameters: The Schwarzschild Case,” S. C.
Noble, J. H. Krolik, J. F. Hawley, ApJ, 711, 959–973(2010).
105. “General Relativistic MHD Jets,” J. H. Krolik & J. F. Hawley, in The Jet Paradigm—From Microquasars
to Quasars, Lect. Notes Phys, 794, ed. T. Belloni (Springer-Verlag: Berlin), pp. 265–287 (2010).
106. “Numerical Simulations of MHD Accretion Disks,” J. F. Hawley, in IAU Highlights of Astronomy, vol
15, in press(2010).
107. “Local Simulations of Magnetized Accretion Disks,” J. B. Simon, PhD thesis, Department of
Astronomy, University of Virginia, July, 2010.
108. Krolik: Invited talk “Dynamics of Black Hole Accretion”, Joint Astronomy Colloquium at the Max-
Planck-Institut-f¨ur-Astrophysik and the European Southern Observatory, June, 2009, Garching, Germany.
109. Hawley: Invited talk “Numerical Simulations of MHD Accretion Disks,” for Joint Discussion 7,
Astrophysical Outflows and Associated Accretion Phenomena, IAU General Assembly, Rio de Janeiro,
Brazil, Aug. 5–7, 2009.
110. Hawley: Invited talk “Global Simulations of Black Hole Accretion,” at the Workshop on Matter and
Electromagnetic Fields in Strong Gravity, Aug. 24–28, College Park, MD.
111. Krolik: Invited talk “Radiation from Black Hole Accretion”, at Matter and Electromagnetic Fields in
Strong Gravity, August, 2009, U. of Maryland, College Park MD
112. “AngularMomentumTransportinMagnetizedAccretionDisksviatheMagnetorotational Instability,” AAS
Meeting, January, 2010, Washington, DC.
113. Hawley:Invitedtalk,“3DGRMHDSimulations:StressattheISCO,”attheworkshop “Computational
RelativisticAstrophysics: Frontiers ofMHD,”PrincetonCenterforTheoreticalScience,January13–16, 2010,
Princeton, NJ.

114. Krolik:Invited talk “ConnectingRadiationtoAccretionDynamicsthroughSimulations”,atPrinceton
Workshop on Computational Relativistic Astrophysics, Princeton Center for Theoretical Science”, January,
2010, Princeton NJ
115. Hawley: Invited talk, “GRMHD Simulations of Black Hole Accretion Disks,” 5th International
Conference on Numerical Modeling of Space Plasma Flows, June 13–18, 2010, San Diego, CA.
116. Public Lecture at the Science Museum in La Laguna, Tenerife, Spain on Friday, November 6th at
19:00. The title of the lecture was “Black Holes—The Powerhouse of the Universe.” Featured black hole
simulation movies and images of TACC Ranger.
PHY070001
117. Results obtained from previous TeraGrid allocations have been reported in the following preprint:
Marcelo Gleiser, Noah Graham, and Nikitas Stamatopoulos, “Long-Lived Time Dependent Remnants During
Cosmological Symmetry Breaking: From Ination to the Electroweak Scale," arXiv:1004.4658, an expanded
version of which is currently under consideration by Physical Review D.
Atmospheric Sciences
ATM050014
118. Lin, J.-T., D. J. Wuebbles, H.-C. Huang, Z. Tao, M. Caughey, X.-Z. Liang, J.-H. Zhu and T.
Holloway, 2010: Potential effects of climate and emissions changes on surface ozone in the Chicago area.
J. Great Lakes Res., 36, 59-64.
119. Patten, K., and D. Wuebbles, 2010: Atmospheric lifetimes and ozone depletion potentials of trans-
1-chloro-3,3,3-trifluoropropylene and trans-1,2-dichloroethylene in a three-dimensional model. Atmos.
Chem. Phys., under review.
120. Stoner, A. M., K. Hayhoe, and D. Wuebbles, 2009: Assessing General Circulation Model
Simulations of Atmospheric Teleconnection Patterns. J. Climate, 22, 4348-4372, DOI:
10.1175/2009JCLI2577.1.
121. Wuebbles, D. J., and K. O. Patten, 2009: HCFC-123 in the Atmosphere: Revisiting Its Potential
Environmental Impacts and Rationale for Continued Use, Environ. Sci. Technol., 43, 32083213,
doi:10.1021/es802308m.
122. Wuebbles, D. J., K. O. Patten, D. Wang, D. Youn, M. Martínez-Avilés, and J. S. Francisco, 2010:
Three-dimensional model evaluation of the ozone depletion potentials for n-propyl bromide, trichloroethylene
and perchloroethylene. Atmos. Chem. Phys., under review.
123. Youn, D., K. O. Patten, J.-T. Lin, and D. J. Wuebbles, 2009: Explicit calculation of Indirect Global
Warming Potentials for Halons using atmospheric models. Atmos. Chem. Phys., 9, 8719-8733.
124. Youn, D., K. O. Patten, D. J. Wuebbles, H. Lee, and C.-W. So, 2010: Potential impact of iodinated
replacement compounds CF3I and CH3I on atmospheric ozone: A three-dimensional modeling study.
Atmos. Chem. Phys., under review.
ATM090032
125. Yoshimura, K., and M. Kanamitsu (2009), Specification of External Forcing for Regional Model
Integrations, Mon. Wea. Rev., 137, 1409-1421.
ATM100048
126. Becker, M., M. S. Gilmore, J. Naylor, J. Weber, G. P. Compo, J. Whitaker, T. M. Hamill, and R.
Maddox, 2010: Simulations of the supercell outbreak of 18 March 1925. 25
Denver, CO, October 2010.
th
Conf. Severe Local Storms,

127. Burkett, L. M. S. Gilmore, R. Thompson, R. Edwards, J. M. Staka, and R. B. Wilhelmson, 2010:
Characteristics of supercells simulated with tornadic and non-tornadic RUC-2 proximity soundings: Part I:
th
Sensitivity to convective initiation mechanisms. 25 Conf. on Severe Local Storms, Denver, CO, Amer.
Meteor. Soc. To be presented October 2010.
128. ——, ——, ——, and ——, 2011: Embryo differences between simulated High and Low Plains
hailstorms. Atmos. Res. In Preparation.
129. ——, L. J. Wicker, G. S. Romine, and R. B. Wilhelmson, 2011: Initial look at dynamic processes
maintaining long-track tornadoes. Geo. Res. Lett. In preparation.
130. ——, L. Burkett, R. Thompson, and R. Edwards, 2010: Characteristics of supercells simulated with
tornadic and non-tornadic RUC-2 proximity soundings: Part II: Relationship between sounding predictors
th
and low-level rotation. 25 Conf. on Severe Local Storms, Denver, CO, Amer. Meteor. Soc. To be presented
October 2010.
131. Naylor, J., M. Askelson, and M. S. Gilmore, 2010: Influences of low-level thermodynamic structure
on convective downdraft properties. Part II: Three-dimensional simulations and trajectory analysis. Mon
Wea. Rev., To be submitted Sept. 2010.
132. Naylor, J., M. S. Gilmore, R. Thompson, and R. Edwards, 2010: Characteristics of supercells
simulated with tornadic and non-tornadic RUC-2 proximity soundings: Part III: Comparisons at tornadoresolving
gridspacing. 25 Conf. on Severe Local Storms, Denver, CO, Amer. Meteor. Soc. To be presented
th
October 2010.
133. ——, M. S. Gilmore, J. M. Straka and R. B. Wilhelmson, 2010: Forward operators for the
assimilation of polarimetric radar observations. J. Atmos. Ocean. Tech., in preparation.
134. —— and ——, 2010: A novel, multiple liquid and ice hydrometeor species, hybrid bulk/bin, threemoment
microphysics parameterization scheme. 13 Conf. on Cloud Physics, Portland, OR, Amer. Meteor.
th
Soc., Presented 28 June 2010. Also in prep for J. Appl. Meteor. Clim
ATM100050
135. Tan Bo, Kaplan L. Micheal 2010: Subtropical Jetstream Circulations during the 2004 North
American Monsoon Part I – Synoptic Analysis on the Subtropical Jet Stream . J. Climate. (not published)
MCA94P023
136. Burkett, L. M. S. Gilmore, R. Thompson, R. Edwards, J. M. Staka, and R. B. Wilhelmson, 2010:
Characteristics of supercells simulated with tornadic and non-tornadic RUC-2 proximity soundings: Part I:
Sensitivity to convective initiation mechanisms. 25th Conf. on Severe Local Storms, Denver, CO, Amer.
Meteor. Soc. To be presented October 2010.
137. ——, ——, ——, and ——, 2011: Embryo differences between simulated High and Low Plains
hailstorms. Atmos. Res. In Preparation.
138. ——, L. J. Wicker, G. S. Romine, and R. B. Wilhelmson, 2011: Initial look at dynamic processes
maintaining long-track tornadoes. Geo. Res. Lett. In preparation.
139. ——, L. Burkett, R. Thompson, and R. Edwards, 2010: Characteristics of supercells simulated with
tornadic and non-tornadic RUC-2 proximity soundings: Part II: Relationship between sounding predictors
and low-level rotation. 25th Conf. on Severe Local Storms, Denver, CO, Amer. Meteor. Soc. To be
presented October 2010.
140. Grim, J. A., R. M. Rauber, G. M. McFarquhar, B. F. Jewett, and D. P. Jorgensen, 2009:
Development and forcing of the rear inflow jet in a rapidly developing and decaying squall line during
BAMEX. Mon. Wea. Rev., 137, 1206-1229.
141. Houston, A. L. and R. B. Wilhelmson, 2010a: The impact of airmass boundaries on deep
convection: A numerical case study in a high-CAPE, low-shear environment. Part I: Storm organization and
propagation. Mon. Wea. Rev., in preparation.

142. Houston, A. L. and R. B. Wilhelmson, 2010a: The impact of airmass boundaries on deep
convection: A numerical case study in a high-CAPE, low-shear environment. Part II: Storm longevity and
the mechanism of initiaton. Mon. Wea. Rev., in preparation.
143. Houston, A. L. and R. B. Wilhelmson, 2010a: The impact of airmass boundaries on deep
convection: A numerical case study in a high-CAPE, low-shear environment. Part III: Mesocyclones. Mon.
Wea. Rev., in preparation.
144. Houston, A. L. and R. B. Wilhelmson, 2010a: The impact of airmass boundaries on deep
convection: A numerical case study in a high-CAPE, low-shear environment. Part IV: Near surface vortex
sheets. Mon. Wea. Rev., in preparation.
145. Jewett, B. F., R. M. Rauber, G. McFarquhar, J. R. French, and K. R. Knupp, 2010: Profiling of
winter storms (PLOWS): What we are learning about winter precipitation bands. Preprints, 13th Conf. on
Cloud Physics, Amer. Meteor. Soc., Portland, OR, P1.74 (28 June 2010).
146. Maliawco, R. J., G. M. McFarquhar, and B. F. Jewett, 2010: Sensitivity of the evolution of an
idealized tropical cyclone to small perturbations in initial input fields. Preprints, 29th Conf. on Hurricanes
and Tropical Meteorology, Amer. Meteor. Soc., Tucson, P2.120.
147. ——, M. S. Gilmore, J. M. Straka and R. B. Wilhelmson, 2010: Forward operators for the
assimilation of polarimetric radar observations. J. Atmos. Ocean. Tech., in preparation
148. Romine G. S., M. S. Gilmore, J. M. Straka and R. B. Wilhelmson, 2009: Forward operators for the
assimilation of polarimetric radar observations. J.
149. —— and ——, 2010: A novel, multiple liquid and ice hydrometeor species, hybrid bulk/bin, threemoment
microphysics parameterization scheme. 13th Conf. on Cloud Physics, Portland, OR, Amer. Meteor.
Soc., Presented 28 June 2010. Also in prep for J. Appl. Meteor. Clim
150. Wilhelmson, R., E. Wiziechi, A. Knecht, T. Daley, W. Davis, S. Yalda, R. D. Clark, S. States, R.
Junod, S. Cecelski, L. David, 2009: An Interactive, Integrated, Instructional Pathway to the LEAD Science
Gateway, (Download a PDF of the talk ) TeraGrid Science Presentation, June 22 - 25, 2009, Arlington, VA.
http://www.teragrid.org/tg09/
Chemical, Thermal Systems
ASC040046
151. Situ, Y., Liu, L., Martha, C. S., Louis, M. E., Li, Z., Sameh, A. H., Blaisdell, G. A., and Lyrintzis, A.
S., “Reducing Communication Overhead in Large Eddy Simulation of Jet Engine Noise,” accepted for the
IEEE Cluster 2010 Conference to be held September 20-24, 2010 in Heraklion, Crete.
152. Lyrintzis, A. S., Blaisdell, G. A., Lo, S.-C., “Large Eddy Simulations of Hot Supersonic Jets for
Aeroacoustics,” Final Report on subcontract to Craft Tech. on STTR Topic No. N09-T008 Phase I (US
Navy), March 2, 2010.
ASC090076
153. L. Duan, X. Wang, and X. Zhong, “A high-order cut-cell method for numerical simulation of
hypersonic boundary-layer instability with surface roughness,” Journal of Computational Physics, In Press,
Corrected Proof, Available online 19 June 2010.
154. P. S. Rawat and X. Zhong, “On high-order shock-fitting and front-tracking schemes for numerical
simulation of shock–disturbance interactions,” Journal of Computational Physics, In Press, Corrected Proof,
Available online 31 May 2010.
155. E. Johnsen et al. (including X. Zhong), “Assessment of high-resolution methods for numerical
simulations of compressible turbulence with shock waves,” Journal of Computational Physics, 229 (2010)
1213–1237.
156. Y. Huang and X. Zhong, “Numerical Study of Laser-Spot Effects on Boundary-Layer Receptivity for
Blunt Compression-Cones in Mach-6 Freestream,” AIAA paper 20104447, June 2010.

157. L. Duan and X. Zhong, “A High Order Cut Cell Method for Numerical Simulation of Three
Dimensional Hypersonic Boundary-Layer Transition with Finite Surface Roughness,” AIAA paper 2010-
1450, January 2010.
158. J. Lei and X. Zhong, “Linear Stability Analysis of Nose Bluntness Effects on Hypersonic Boundary
Layer Transition,” AIAA paper 2010-898, January 2010.
CTS070070
159. D.H.Lee,R.Akhavan(2009) Scaling ofpolymerdrag reductionwithpolymerand flowparametersin
turbulent channel flow. Advances in Turbulence XII, 359-362, Springer.
160. R.Akhavan,J.Liu,D.H.Lee(2009)AmixedEulerian-Lagrangianschemefordirect numerical simulation
of viscoleastic turbulent flows. J. Non-Newtonian Fluid Mech. (submitted).
161. D.H.Lee,R.Akhavan(2010)Skin-frictiondragreductionbydilutepolymersolutionsinturbulent channel
flow. Part 1. Scaling of drag reduction with polymer and flow parameters. J. Fluid Mech. (submitted).
162. D.H.Lee,R.Akhavan(2010)Skin-frictiondragreductionbydilutepolymersolutionsinturbulent channel
flow. Part 2. Mechanism of drag reduction. J. Fluid Mech. (submitted).
163. R.Akhavan(2010) Stochastic simulations ofdrag reductionby dilutepolymer solutions. J. Fluid Mech.
(in preparation).
CTS080033
164. J. B. Perot, Determination of the Decay Exponent in Mechanically Stirred Isotropic Turbulence,
Fluid Dynamics Research, 2010.
165. Khajeh-Saeed and J. B. Perot, Using GPU Supercomputers to Massively Accelerate Pattern
Matching, GPU GEMS 4, 2010.
166. T. McGuiness, A. Khajeh-Saeed, and J. B. Perot, High Performance Computing on GPU Clusters,
Journal of Computational Science, 2010.
167. J. B. Perot, Discrete Conservation Properties of Unstructured Mesh Schemes, Annual Reviews of
Fluid Mechanics, 2010.
168. Khajeh-Saeed, S. Poole and J. B. Perot, Acceleration of the Smith-Waterman Algorithm using
Single and Multiple Graphics Processors, Journal of Computational Physics, 229 (11) 4247-4258, 2010.
CTS090100
169. S. C. Ammal and A. Heyden, “Modeling the noble Metal/TiO2 (110) interface with hybrid DFT
functionals: A Periodic Electrostatic Embedded Cluster Model Study”, J. Chem. Phys. (manuscript under
revision)
170. S. C. Ammal and A. Heyden, “Theoretical investigation of the water-gas shift reaction at the threephase
boundary of the Pt/TiO2 interface”, to be submitted to J. Phys. Chem. C.
171. J. S. Ratliff, S. A. Tenney, S. C. Ammal, A. Heyden, and D. A. Chen, “CO adsorption and oxidation
on Pt, Au and Pt-Au clusters on TiO2(110)”, to be submitted to J. Phys. Chem. C.
CTS090103
172. VAN POPPEL, B., DESJARDINS, O., DAILY, J.W. (2010) A Ghost Fluid, Level Set Methodology
for Simulating Electrohydrodynamic Atomization of Liquid Fuels, manuscript submitted to J. of Comp. Phys.
173. DESJARDINS, O. and PITSCH, H. (2010) Detailed numerical investigation of turbulent atomization
of liquid jets, accepted for publication in Atomization and Sprays
174. DESJARDINS, O. and PITSCH, H. (2009) A spectrally refined interface approach for simulating
multiphase flows, J. Comp. Physics, 228 (5), 1658-1677

175. VAN POPPEL, B., DESJARDINS, O., DAILY, J.W. (2010) Simulating Electrodydrodynamic
Atomization for Fuel Injection. ILASS-Americas 22nd Annual Conference on Liquid Atomization and Spray
Systems, Cincinnati, OH.
176. CZAJKOWSKI, M., DESJARDINS, O. (2010) Quadrature-free Discontinuous Galerkin Level Set
Scheme. ILASS-Americas 22nd Annual Conference on Liquid Atomization and Spray Systems, Cincinnati,
OH.
177. VAN POPPEL, B., DESJARDINS, O., DAILY, J.W. (2010) Modeling two-phase
electrohydrodynamic flows, 48th Annual AIAA Aerospace Sciences Meeting, Orlando, FL.
178. VAN POPPEL, B., DESJARDINS, O., DAILY, J.W. (2009) Modeling electrohydrodynamic
atomization, 62nd Annual Meeting of the APS Division of Fluid Dynamics, Minneapolis, MN.
179. DESJARDINS, O. (2009) Adaptive spectral refinement for accurate simulations of turbulent multiphase
flows, International Conference on Liquid Atomization and Sprays, Vail, CO.
180. DESJARDINS, O. (2009) Detailed numerical investigation of turbulent atomization of liquid jets,
International Conference on Liquid Atomization and Sprays, Vail, CO.
181. PAI, M., DESJARDINS, O., PITSCH, H. (2009) Detailed numerical simulations of liquid jets in
cross-flow, Proceedings of the 47th AIAA Aerospace Meeting, Orlando, FL.
MCA06N038
182. Reddy, H. and Abraham, J. (2010) Ignition kernel development studies relevant to lean-burn natural
gas engines. Accepted for publication in Fuel.
183. Venugopal, R. and Abraham, J. (2010) Numerical studies of the response of flamelets to
unsteadiness in the near-field of jets under diesel conditions. Combustion Science and Technology,
182(7):717-738.
184. Owston, R. and Abraham, J. (2010) Structure of hydrogen triple flames and premixed flames
compared. Combustion and Flame, 157:1552-1565.
185. Owston, R. and Abraham, J. (2010) Numerical study of hydrogen triple flame response to mixture
stratification, ambient temperature, pressure, and water vapor concentration. International Journal of
Hydrogen Energy, 35(10):4723-4735.
186. Owston, R. and Abraham, J. (2009) Flame propagation in stratified hydrogen-air mixtures: spark
placement effects. International Journal of Hydrogen Energy, 34(15):6532-6544.
187. Venugopal, R. and Abraham, J. (2009) Unsteady flamelet response in the near field of high-
Reynolds-number jets. AIAA Journal, 47(6):1491-1506.
188. Venugopal, R. and Abraham, J. (2009) Numerical studies of vortex-induced extinction/reignition
relevant to the near-field of high-Reynolds-number jets. Physics of Fluids, 21, 055106.
189. Owston, R., Magi, V. and Abraham, J. (2010) Some numerical considerations in the simulation of
low-Ma number hydrogen/air mixing layers. Central States Section Meeting of the Combustion Institute,
Urbana-Champaign, March.
190. Mukhopadhyay, S. and Abraham, J. (2010) Ignition behavior in autoigniting stratified mixtures
under IC engine conditions. Central States Section Meeting of the Combustion Institute, Urbana-Champaign,
March.
191. Reddy, H. and Abraham, J. (2010) Influence of vortex parameters on the outcome of kernel-vortex
interactions in lean methane-air mixtures. Central States Section Meeting of the Combustion Institute,
Urbana-Champaign, March.
192. Owston, R. and Abraham, J. (2010) Numerical modeling of triple flames: exploratory studies.
Central States Section Meeting of the Combustion Institute, Urbana-Champaign, March.

Chemistry
CHE060025
193. S. R. Stoyanov, A. V. Titov and P. Kr´al, Transition metal and nitrogen doped carbon
nanostructures, Coord. Chem. Rev., 253, 2852 (2009).
194. L. Vukovi´c and P. Kr´al, Coulombically driven rolling of nanorods on water, Phys. Rev. Lett. 103,
246103 (2009).
195. Titov B. Wang, K. Sint and P. Kr´al, Controllable synthetic molecular channels: Biomimetic
ammonia switch, J. Phys. Chem. B 114, 1174 (2010).
CHE080076
196. Azaria S. Eisenberg , Vasiliy Zenamenskiy , Iya Likhtina, and Ronald L. Birke, “Electronic
Spectroscopy and Computational Studies of Glutathionylco(III)balamin ” in preparation. (2010)
197. R. L. Birke, V. Znamenskiy, and J. R. Lombardi, “ A charge-transfer surface enhanced Raman
scatter model from time-dependent density functional theory calculations on a Ag 10-pyridine complex,” J.
Chem. Phys. 132, 214707(1-15) ( 2010).
198. J. R. Lombardi and R. L. Birke , “ A Unified View of Surface-Enhanced Raman Spectroscopy” ,
Acct. Chem. Res., 42, 734-742 (2109)
CHE080083
199. E. Chagarov, A.C. Kummel, “Density Functional Theory Simulations of High-k Oxides on III-V
Semiconductors”, Book Chapter in the book “Fundamentals of III-V Semiconductor MOSFET’s”, editors S.
Oktyabrsky, P. Ye, by Springer Science+Business Media LLC (2010).
200. M. Houssa, E. Chagarov, A. Kummel, “Surface Defects and Passivation of Ge and III-V Interfaces”,
MRS Bulletin 34, 504 (2009). (Research results of Dr. Chagarov and Prof. Kummel were presented on
the journal cover page).
201. E.A. Chagarov, A.C. Kummel,” Molecular dynamics simulation comparison of atomic scale
intermixing at the amorphous Al2O3/semiconductor interface for a-Al2O3/Ge, a-Al2O3/InGaAs, and a-
Al2O3/InAlAs/InGaAs”, Surface Science 603, 3191 (2009).
202. E.J. Kim, E. Chagarov, J. Cagnon, Y. Yuan, A. C. Kummel, P. M. Asbeck, S. Stemmer, K. C.
Saraswat, and P. C. McIntyre, “Atomically abrupt and unpinned Al2O3 / In0.53Ga0.47As interfaces:
Experiment and simulation”, Journal of Applied Physics 106 (12), 124508 (2009).
203. E.A. Chagarov, A.C. Kummel, “Ab initio molecular dynamics simulations of properties of a-
Al2O3/vacuum and a-ZrO2/vacuum vs. a-Al2O3/Ge(100)(2x1) and a-ZrO2/Ge(100)(2x1) interfaces”, Journal
of Chemical Physics 130, 124717 (2009).
204. M. Rodwell, W. Frensley, S. Steiger, E. Chagarov, S. Lee, H. Ryu, Y. Tan. G. Hegde, L. Wang, J.
Law, T. Boykin, G. Klimek, P. Asbeck, A. Kummel “III-V FET Channel Designs for High Current Densities
and Thin Inversion Layers” IEEE Proceedings of the Device Research Conference 2010
205. Jian Shen, Jonathon B. Clemens, Darby L. Feldwinn, Wilhelm Melitz, Tao Song, Evgueni A.
Chagarov, Ravi Droopad and Andrew C. Kummel, “Structural and Electronic Properties of Group III Rich
In0.53Ga0.47As(001)”, Surface Science, (In press) (2010)
206. E. Chagarov, A. Kummel, “Density Functional Theory Simulations of Amorphous High-k Oxides on
A Compound Semiconductor Alloy: a-Al2O3/InGaAs(4x2)(100), a-HfO2/InGaAs(4x2)(100), a-
HfO2/OH/InGaAs(4x2)(100) and a-ZrO2/InGaAs(4x2)(100).” (in preparation).
207. E.A. Chagarov, A.C. Kummel, “Density-Functional Theory Molecular Dynamics Simulations of
Oxide, Nitride, and OxyNitride High-k/Ge(100)(2x1) interface passivation”, (research paper) (in preparation).

208. J B. Clemens, E. A. Chagarov, M. Holland, R. Droopad, J Shen, and A. C. Kummel “Atomic
imaging of the monolayer nucleation and unpinning of a compound semiconductor surface during atomic
layer deposition” J. Chem. Phys. (submitted) (2010)
209. S. R. Bishop, J. B. Clemens, E. A. Chagarov, and A. C. Kummel “Theoretical analysis of initial
adsorption high-κ dielectric oxides on group III rich InxGa1-xAs(0 0 1)-(4×2) surfaces” J. Chem. Phys.
(submitted) (2010)
210. Jian Shen, Evgueni A. Chagarov, Darby L. Feldwinn, Wilhelm Melitz, Nancy M.Santagata and
Andrew C. Kummel, Ravi Droopad, Matthias Passlack, “Scanning Tunneling Microscopy/Spectroscopy
Study of Atomic and Electronic Structures of In2O on InAs and In0.53Ga0.47As(001)-(4×2) Surfaces”, J.
Chem Phys. (submitted) (2010).
CHE090062
211. Deborah L. Heyl, Joshua M. Osborne, Sarika Pamarthy, Shyamprasad Samisetti, Andrew W. Gray,
Anitha Jayaprakash, Srikanth Konda, Dorothy J. Brown, Samuel R. Miller, Reza Eizadkhah, Maria C. Milletti
‘Liposome Damage and Modeling of Fragments of Human Islet Amyloid Polypeptide (IAPP) Support a Two-
Step Model of Membrane Destruction’ International Journal of Peptide Research and Therapeutics, 2010,
16, 43.
212. Mark A. Lukowski, Dae Jin Choi and M.C. Milletti ‘Substrate Binding and Kinetic Aspects of the
Peroxidation Reaction of Four Polyunsaturated Fatty Acids in the COX Active Site of PGHS-1’ Letters in
Drug Design and Discovery, 2010, 7, 88-97.
213. Mark Lukowski, Kathryn Jacobs, Powen Hsueh, Harriet A. Lindsay and M.C. Milletti
‘Thermodynamic and kinetic factors in the aza-Cope rearrangement of a series of iminium cations’
Tetrahedron 2009, 65, 10311-10316.
CHE090066
214. Yang, Q.; Huang, Z.; Khvostichenko, D.; Kucharski, T.; Chen, J.; Boulatov, R. A molecular force
probe. Nature Nanotech. 2009, 4, 302-306.
215. Huang, Z.; Boulatov, R. Exploring chemimechanics with molecular force probes (invited review).
Pure Appl. Chem. 2010, 86, 931-952.
216. Kucharski, T.; Yang, Q.; Huang, Z.; Tian, Y.; Rubin, N. J.; Concepcion, C. J.; Boulatov, R. Kinetics
of thiol/disulfide exchange correlates weakly with the restoring force of the disulfide moiety. Angew. Chem.
Int. Ed., 2009, 48, 7040 - 7043.
217. Huang, Z.; Yang, Q.; Kucharski, T.; Khvostichenko, D.; Wakeman, S. M.; Boulatov, R. Macrocyclic
disulfides for studies of sensitized photolysis of the S-S bond. Chem. Eur. J., 2009, 15, 5212-5214.
218. Huang, Z.; Yang, Q.; Khvostichenko, D.; Kucharski, T.J.; Chen, J.; Boulatov, R. Method to derive
restoring forces of strained molecules from kinetic measurements. J. Am. Chem. Soc. 2009, 131, 1407-1409
CHE090070
219. Xu, L.; Doubleday, C. E.; Houk, K. N. “Dynamics of 1,3-Dipolar Cycloaddition Reactions of
Diazonium Betaines to Acetylene and Ethylene: Bending Vibrations Facilitate Reaction” Angew. Chem. Int.
Ed. 2009, 48, 2746-2748.
220. Xu, L.; Doubleday, C. E.; Houk, K. N. “Dynamics of 1,3-Dipolar Cycloadditions: Energy Partitioning
of Reactants and Quantitation of Synchronicity” J. Am. Chem. Soc. 2010, 132, 3029–3037.
221. Doubleday, C.; Bohnen, M. “Statistical and Nonstatistical Dynamics in a Thermal Rearrangement
with Competing 1,3 and 3,3 Shifts”.

222. Doubleday, C.; Hakim, N. “Trajectory Dynamics of O( 3 P) Reactions with CH 2
F and CH 2
Cl
Radicals”.
223. Doubleday, C.; Goldenberg, S. “Trajectory Study of a Gas Phase Nucleophilic Addition to a
Carbonyl: H – + HCOF”.
CHE090124
224. Identification of histidine biosynthesis inhibitors using docking, ensemble rescoring and whole cell
assays S. T. Henriksen, J. Liu, G. Estiu, Z. N. Oltvai, O. Wiest Bioorg. Med. Chem 2010 In press
225. Thermal behavior, powder X-ray diffraction, DFT calculations and vibrational spectra of barium
bis(pentacyanonitrosylchromate) octahydrate, Ba(3)[Cr(CN)(5)NO](2).8H(2)O(D(2)O D. B. Soria, G.L. Estiu,
R. Carbonio, P.J. Aymonimo. Spectrochim Acta A Mol Biomol Spectrosc 2010. PubMed ID 20457005
226. Affinity of sulfamates and sulfamides to Carbonic Anhydrase II isoform: experimental and molecular
modeling approaches L. Gavernet, J. Gonzalez Funes, L.Bruno-Blanch, G. Estiu, A. Maresca, C. Supuran J.
Chem. Inf. Model. In press
227. On the Inhibition of Histone Deacetylase 8. G. Estiu, N. West, R. Mazitschek, E.Greenberg, J.
Bradner, O. Wiest. Bioorg. Med. Chem. 2010, 18(11), 4103-4110
228. Mechanism of unfolding of the helicate [Cu2(mphenpr) 2]2+ into the [Cu(mphenpr)]+ monomer
(mphenpr = 1,3-bis(9-methyl-1,10-phenanthrolin-2-yl)propane) and its coupling to the chlorocarbon
dehalogenation. On the participation of metastable copper-alkyl species. L. Lemus, J. Guerrero,
J.Costamagna, G. Estiu, G. Ferraudi, A. G. Lappin, A. Oliver and B. C. Noll, Inorg.Chem. 2010, 49, 4023.
229. Selective Inhibition of BET Bromodomains. P. Filippakopoulos1, J. Qi, S. Picaud, Y. Shen, W. B.
Smith, O. Fedorov, E. M. Morse, T. Keates, T. T. Hickman, I.Felletar, M. Philpott, Sh. Munro, N. West, M. J.
Cameron, T., D. Heightman, N. La Thangue, A. Kung, Ch. A. French, O. Wiest, S. Knapp, J. E. Bradner.
Nature, accepted for publication.
230. On the Structure and Dynamics of GNA A.T. Johnson,.M. Schlegel, E. Meggers, L.O. Essen, O.
Wiest J. Am. Chem. Soc. accepted for publication.
231. Application of Q2MM for Stereoselective Reactions S. N. Lill, P. Donoghue, A. Forbes, V.
Verdolino, O. Wiest, P. Rydberg, , P.O. Norrby Curr. Org. Chem. 2010 accepted
232. On the Mechanism of the Rhodium Catalyzed Acrylamide Hydrogenation. V. Verdolino, A. Forbes,
P. Helquist, P. O. Norrby, O. Wiest J. Mol. Cat A 2010, 324, 9-14
233. Quantum Mechanical Studies of Transition Metal Catalyzed Hydrogenations V. Verdolino, A.
Forbes, P. Helquist, O. Wiest. Submitted.
CHE100115
234. Giagou, T.; Meyer, M. P. “Mechanism of the Swern Oxidation: Significant Deviations from
Transition State Theory” J. Org. Chem.(submitted).
235. Zhu, H.; Meyer, M. P. “Cationic Intermediates in Friedel-Crafts Acylation: Structural Information
from Theory and Experiment” Chem. Comm. (Invited article for ‘Emerging Investigators’ issue: submitted).
236. Giagou, T.; Meyer, M. P. “Kinetic Isotope Effects in Asymmetric Reactions” Chem. Eur. J. (Invited
Concept article: provisionally accepted).
MCA08X021
237. J.L. Belof, C.R. Cioce, X. Xu, X.P. Zhang, B. Space, H.L. Woodcock “Characterization of Tunable
Radical Metal-Carbenes: Key Intermedia tes in Catalytic Cyclopropanation”, J. Am. Chem. Soc., (submitted)

238. M.H. Alkordi, A.C. Stern, J.L. Belof, B. Space, L. Wojtas, M.J. Zaworotko,
239. M. Eddaoudi, “Molecular Squares: Confined Space with Specif ic Geometry for Hydrogen Uptake”,
J. Am. Chem. Soc., (submitted)
240. N.R. McIntyre, E.W. Lowe, Jr., J.L. Belof, M. Ivkovic, J. Shafer, B. Space,
241. D.J. Merkler, “Evidence for Substrate Pre-organization in the Peptidylglycine alpha-Amidating
Monooxygenase (PAM) Reaction Describing the Contribution of Ground State Structure to Hydrogen
Tunneling’ ’, J. Am. Chem. Soc., (submitted)
242. M.H. Alkordi, J.L. Belof, E. Rivera, L. Wojtas, J.J. Zhang, A. Kassas, M. Eddaoudi, “Insight into the
Assembly Mechanism of Metal-Orga nic Materials”, J. Am. Chem. Soc., (in press)
243. J.L. Belof, B. Space, “Rapidly Convergent Iterative Techniques Toward the Solution of Many-body
Molecular Polarization Field Equations ”, J. Chem. Theory Comput., (in press)
244. J.L. Belof, A.C. Stern, B. Space, “A Predictive Model of Hydrogen Sorption in Metal-Organic
Materials”, J. Phys. Chem. C, 113, 9316 (2009) [DOI]
Computer and Computation Research
CCR100034
245. K. Kandalla, H. Subramoni and D. K. Panda. Designing Topology-Aware Collective Communication
Algorithms for Large Scale InfiniBand Clusters : Case Studies wih Scatter and Gather. In Proceedings of the
2010 IEEE International Symposium on Parallel and Distributed Processing, 2010.
246. S. Potluri, P. Lai, K. Tomko, S. Sur, Y. Cui, M. Tatineni,K. Schulz, W. Barth, A. Majumdar and
247. D. K. Panda. Quantifying Performance Benefits of Overlap using MPI-2 in a Seismic Modeling
Application. Technical report, 2010.
248. S. Potluri, S. Sur, P. Lai, K. Tomko, Y. Cui and D. K. Panda. Performance Analysis and
Improved Communication Overlap for a Seismic Modeling Application on Large In finiBand Clusters.
Technical Report OSU-CISRC-4/10-TR09, 2010.
249. Y. Cui, K. B. Olsen, T. H. Jordan, K. Lee, J. Zhou, P. Small, D. Roten, G. Ely, D. K. Panda,
a. Chourasia, J. Levesque, S. M. Day, P. M. Maechling. Scalable Earthquake Simulation on Petascale
Supercomputers, Finalist: ACM Gordon Bell Award, SC’10. Nov 2010.
Cross-Disciplinary Activities
CCR100034
250. R. Stephens, Ph.D. Thesis, University of Illinois at Urbana-Champaign (2008). 4 R. Stephens and
R. Alkire, “Island Dynamics Algorithm for Kinetically-Limited Electrochemical Nucleation with
Additives,” J. Electrochem. Soc., 156:1, D28-D35 (2009).
251. R.N. Methekar, K. Chen, R.D. Braatz, and V.R. Subramanian, “Kinetic Monte Carlo Simulation of
Surface Heterogeneity in Graphite Electrodes for Lithium-Ion Batteries: Passive Layer Formation and
Simulation of Capacity Fade,” journal manuscript submitted (2010).
252. D. Lubomirsky, Ph.D. Thesis in progress, University of Illinois, Urbana (2011).
253. M. Karulkar, F. Xue, R.D. Braatz, and R.C. Alkire, “Effect of Additives on Kinetically-Limited
Electrodeposition,
Part I: Simulation of Nucleation and Substrate Overgrowth,” manuscript in preparation.
254. A. Bezzola, Ph.D. Thesis in progress, University of California at Santa Barbara (2012).
255. R. Stephens, M. Willis and R. Alkire, “Additive-Assisted Nucleation and Growth by
Electrodeposition: Part II: Mathematical Model and Comparison with Experimental Data,” J. Electrochem.
Soc., 156:10, D385-D394 (2009).

256. M. Willis, R. Stephens, and R. Alkire, “Additive-Assisted Nucleation and Growth by
Electrodeposition: Part I: Experimental Studies with Copper Seed Arrays on Gold Films,” J. Electrochem.
Soc., 156:10, D377-D384 (2009).
257. R. Stephens, M. Buoni, L. Petzold, and R. Alkire, “The Effect of Solution-Phase Mass Transport on
Metal Nucleation and Growth during Electrochemical Deposition with Additives,” manuscript in preparation
(2010).
258. M. Kishida and R.D. Braatz, “Worst-case Analysis of Distributed Parameter Systems with
Application to the 2D Reaction-Diffusion Equation,” Optimal Control Applications & Methods, in press (2010).
259. L. Tierney, “Markov Chains for Exploring Posterior Distributions,” Annals of Statistics, 22, 1701-
1728 (1994). 18 V. Ramadesigan, V. Boovaragavan, M. Arabandi, K. Chen, H. Tuskamoto, R.D. Braatz, and
V. Subramanian, “Parameter Estimation and Capacity Fade Analysis of Lithium-Ion Batteries Using First-
Principles-Based Efficient Reformulated Models,” ECS Transactions, 19:16, 11-19 (2009).
260. V. Ramadesigan, V. Boovaragavan, M. Arabandi, K. Chen, H. Tuskamoto, R.D. Braatz, and V.
Subramanian, “Parameter Estimation and Capacity Fade Analysis of Lithium-Ion Batteries Using
Reformulated Models,” journal manuscript nearly completed for submission (2010).
261. R.N. Methekar, V. Boovaragavan, M. Arabandi, V. Ramadesigan, V.R. Subramanian, F. Latinwo,
and R.D. Braatz, “Optimal Spatial Distribution of Microstructure in Porous Electrodes for Li-Ion Batteries,”
Proceedings of the American Control Conference, Baltimore, Maryland, 6600-6605 (2010).
262. V.R. Subramanian, V. Boovaragavan, V. Ramadesigan, K. Chen, and R.D. Braatz, “A
Reformulated Model and Its Applications to Bayesian Parameter Estimation for Lithium-ion Batteries,”
Preprints of the Foundations of Computer-Aided Process Design, Breckenridge, CO, June 7-12 (2009).
263. V.R. Subramanian, V. Boovaragavan, V. Ramadesigan, K. Chen, and R.D. Braatz, “Model
Reformulation and Design of Lithium-Ion Batteries,” in Design for Energy and the Environment, edited by
M.M. El-Halwagi and A.A. Linninger, CRC Press, Boca Raton, FL, 987-1006 (2009).
264. V. Ramadesigan, R.N. Methekar, V.R. Subramanian, F. Latinwo, and R.D. Braatz, “Optimal
Porosity Distribution for Minimized Ohmic Drop Across a Porous Electrode,” journal manuscript submitted
(2010).
265. R.N. Methekar, V. Ramadesigan, V.R. Subramanian, and R.D. Braatz, “Optimum Charging Profile
for Lithium-Ion Batteries to Maximize Energy Storage and Utilization,” ECS Transactions, 25:35, 139-146
(2010).
266. R.N. Methekar, V. Ramadesigan, V.R. Subramanian, and R.D. Braatz, “Optimal Charging Profile
for Lithium-Ion Batteries to Maximize Energy Storage,” journal manuscript submitted (2010).
Materials Research
ASC090058
267. Akihiro Kushima, Sidney Yip, and Bilge Yildiz, “Competing Strain Effects in Reactivity of LaCoO3
with Oxygen”, submitted to Phys. Rev. B (2010).
268. Akihiro Kushima and Bilge Yildiz, “Oxygen ion diffusivity in strained yttria stabilized zirconia: where
is the fastest strain”, J. Mater. Chem., (2010), DOI: 10.1039/c000259c.
269. Akihiro Kushima and Bilge Yildiz, “Role of Lattice Strain and Defect Chemistry on the Oxygen
Vacancy Migration at the (8.3%Y2O3-ZrO2)/SrTiO3 Hetero-Interface: A First Principles Study”, ECS Trans.
25, 1599, (2009).
270. European Materials Research Society 2010 Spring Meeting, Strasbourg, France, June 7-11, 2010,
Akihiro Kushima, Jeong Woo Han, Sidney Yip, and Bilge Yildiz, “Ionic transport and surface reactions on
LaCoO3 under strain: Insights from ab initio calculations”.
271. 216th Electrochemical Society Meeting, Vienna, Austria, October 4-9, 2009, Akihiro Kushima and
Bilge Yildiz, “Role of Lattice Strain and Defect Chemistry on the Oxygen Vacancy Migration at the
(8.3%Y2O3-ZrO2)/SrTiO3 Hetero-Interface: A First Principles Study”.

DMR050002
272. Unusual dielectric response in B-site size-disordered hexagonal transition metal oxides,
D. Choudhury, A. Venimadhav, C. Kakarla, K. T. Delaney, P. S. Devi, P. Mondal,
R.Nirmala,J.Gopalakrishnan,N.A.Spaldin,U.V.Waghmare, andD.D.Sarma, Appl. Phys. Lett. 96,
162903(2010)
273. Theoretical study of Schottky-barrier formation at epitaxial rare-earthmetal/semiconductor
interfaces, K. Delaney, N. A Spaldin and C. G. van de Walle, Phys. Rev. B 81, 165312(2010).
274. Electron-lattice instabilities suppress cuprate-like electronic structures in SrFeO3/SrTiO3
superlattices, J. M. Rondinelli and N. A. Spaldin, Phys. Rev. B 81, 085109(2010).
275. Strain-induced isosymmetricphase transition in BiFeO3, A. J.Hatt, N. A.Spaldin and
C. Ederer, Phys. Rev. B 81, 054109(2010).
276. Strain-Induced Ferroelectricity in Simple Rocksalt Binary Oxides, E. Bousquet, N. A. Spaldin and
Ph. Ghosez, Phys. Rev. Lett. 104, 037601(2010).
277. Mn 3+ in Trigonal Bipyramidal Coordination: A New Blue Chromophore A. E. Smith,
H. Mizoguchi, K. Delaney, N. A. Spaldin, A. W. Sleight and M. A. Subramanian,
J. Am. Chem. Soc. 131, 17084(2009).
278. First-principles modeling of ferroelectric capacitors via constrained displacement field calculations,
M. Stengel, D. Vanderbilt and N. A. Spaldin, Phys. Rev. B 80, 224110(2009). Editor’s Choice
279. A strain-driven morphotropic phase boundary in BiFeO3, R. J. Zeches, M. D. Rossell,
J. X. Zhang, A. J. Hatt,Q. He, C.-H. Yang, A. Kumar, C. H. Wang, A. Melville,
C. Adamo, G. Sheng, Y.-H. Chu, J. F. Ihlefeld, R. Erni, C. Ederer, V. Gopalan,
L.Q.Chen,D.G.Schlom,N.A.Spaldin,L.W.Martin andR.Ramesh,Science 326,977 -980(2009).
280. Current trends of the magnetoelectric effect, M. Fiebig and N. A. Spaldin, Eur. Phys.
J. B 71, 293(2009).
281. Strain effects on the electric polarization of BiMnO3, A. J. Hatt and N. A. Spaldin, Eur. Phys. J. B
71, 435(2009).
282. First-principles study of ferroelectric domain walls in multiferroic bismuth ferrite, A, Lubk, S.
Gemming and N. A. Spaldin, Phys. Rev. B 80, 104110(2009). Editor’s Choice
283. Non-d 0 Mn-driven ferroelectricity in antiferromagnetic BaMnO3, J. M. Rondinelli, A.
S. Eidelson and N. A. Spaldin, Phys. Rev. B 79, 205119(2009). Editor’s Choice
284. Enhancement of ferroelectricity at metal/oxide interfaces, M. Stengel, D. Vanderbilt and N. A.
Spaldin, Nature Materials, 8, 392(2009).
285. Superexchange-driven magnetoelectricity in magnetic vortices, K. T. Delaney, M. Mostovoy and N.
A. Spaldin, Phys. Rev. Lett. 102, 157203(2009).
286. Role of Atomic Multiplets in the Electronic Structure of Rare-Earth Semiconductors and
Semimetals, L. V. Pourovskii, K. T. Delaney, C. G. Van de Walle, N. A. Spaldin, and A. Georges, Phys. Rev.
Lett. 102, 096401(2009).
287. Electric displacement as the fundamental variable in electronic-structure calculations,
M. Stengel, N. A. Spaldin and D. Vanderbilt, Nature Physics 5,304 -308(2009).
288. Conduction at domain walls in oxide multiferroics, J. Seidel, L. W. Martin, Q. He,
Q. Zhan, Y.-H. Chu, A. Rother, M. E. Hawkridge, P. Maksymovych, P. Yu, M. Gajek, N. Balke, S. V. Kalinin,
S. Gemming, F.Wang, G. Catalan, J. F.Scott, N.
a. Spaldin, J. Orenstein and R. Ramesh, Nature Materials 8, 229,(2009).

289. Structural effects on the spin-state transition in epitaxially strained LaCoO3 films, J.
M. Rondinelli and N. A. Spaldin, Phys. Rev. B 79, 054409(2009).
DMR070008
290. Ajmi BH. Hamouda, T. J. Stasevich, Alberto Pimpinelli, and TLE, Effects of Impurities on Surface
Morphology: Some Examples, J. Phys.: Condens. Matter 21, 084215 (2009)
291. Ajmi BH. Hamouda, Rajesh Sathiyanarayanan, A. Pimpinelli, and T. L. Einstein,
Role of Codeposited Impurities During Growth: I. Explaining Distinctive Experimental Morphology on
Cu(001), preprint being submitted to Phys. Rev. B ∗
292. Rajesh Sathiyanarayanan, Ajmi BH. Hamouda, A. Pimpinelli, and T. L. Einstein,
Role of Codeposited Impurities During Growth: II. Dependence of Morphology on Binding and Barrier
Energies, preprint being submitted to Phys. Rev. B ∗
293. Rajesh Sathiyanarayanan and T. L. Einstein, Ab-initio Calculations of Interactions between Cu
Adatoms on Cu(110): Sensitivity of Strong Multi-site Interactions to Adatom Relaxations, Surface Sci. 603,
2387 (2009)
294. T. L. Einstein, Alberto Pimpinelli, et al., “Distinctive Features in Growth on Vicinal Cu(100):
Understanding the Role of Impurities by Calculating Key Energies and Simulating Morphology,” 27th Max
Born Symposium on Multiscale Modeling of Real Materials, Wroclaw, Poland, Sept. 2010 [invited].
295. T. L. Einstein, Alberto Pimpinelli, et al., “Modeling the Role of Co-deposited Impurities in Growth:
What Causes the Distinctive Step Meandering and Pyramidal Mounds on Cu(001),” German Physical
Society (DPG), Symposium on Crystal Growth Kinetics, Regensburg, Germany, March 2010 [invited].
296. Ajmi BH. Hamouda, Rajesh Sathiyanarayanan, A. Pimpinelli, and T. L. Einstein, “Role of
Codeposited Impurities in Growth: Explaining Cu(0 0 1),” American Physical Society, Portland (OR), Mar.
2010
297. Rajesh Sathiyanarayanan, Ajmi BH. Hamouda, A. Pimpinelli, and T. L. Einstein, “Role of
Codeposited Impurities in Growth: Dependence of Morphology on Binding and Barrier Energies,” American
Physical Society, Portland (OR), Mar. 2010
298. Rajesh Sathiyanarayanan and T. L. Einstein, “Role of Adatom Relaxations in Computing Latticegas
Energies: Multisite Interactions,” American Physical Society, Pittsburgh, Mar. 2009
299. Alberto Pimpinelli, Ajmi BH. Hamouda, and T. L. Einstein, “Impurities in Vacuum Deposition: Effect
on Island Nucleation and Surface Morphologies,” American Physical Society, Pittsburgh, Mar. 2009
DMR080070
300. Simulation studies of length dependence scaling exponent of inverted frustum spring constant, K.K.
Mon.
301. Sound velocity of mass modulated nanostructured composite, K.K. Mon.
302. Numerical solution of the anisotropic diffusion equation for one hard disk in a channel with
obstruction, K.K. Mon.
303. Monte Carlo simulations of two-dimensional model for hopping times , K.K. Mon.
DMR080072
304. S.R. Wilson, C.M. Hefferen, S.F. Li, J. Lind, R.M. Suter and A.D. Rollett, “Microstructural
Characterization and Evolution in 3D,” Risoe 2010 Symposium proceedings, accepted.
305. U. Lienert, M.C. Brandes, J.V. Bernier, M.J. Mills, M.P. Miller, S.F. Li, C.M. Hefferan,

306. J. Lind, R.M. Suter, “3DXRD at the Advanced Photon Source: Orientation Mapping and
Deformation Studies,” Risoe 2010 Symposium proceedings, accepted.
307. C.M. Hefferan, S.F. Li, J. Lind, U. Lienert, A.D. Rollett, P. Wynblatt, R.M. Suter, “Statistics of
High Purity Nickel Microstructure From High Energy X-ray Diffraction Microscopy,” Computers, Materials
and Continua, 14, 209-219 (2009).
308. C.M. Hefferan, S.F. Li, J. Lind, and R.M. Suter, “Tests of Microstructure Reconstruction by Forward
Modeling of HEDM Data,” Journal of Powder Diffraction, 25, 132-137 (2010).
309. C. M. Hefferan, S. F. Li, J. Lind, U. Lienert, A. D. Rollett, R. M. Suter, “In-situ Observation of
Recovery and Grain Growth in High Purity Aluminum,” Proceedings of the 2010 International Conference on
Recrystallization and Grain Growth, Sheffield, England, July 2010.
310. U. Lienert, S.F. Li, C.M. Hefferan, J. Lind, R.M. Suter, C. Brandes, J.V. Bernier,
311. M.J. Mills, M.P. Miller, C. Wejdemann, and W. Pantleon, “High Energy Diffraction Microscopy at the
Advanced Photon Source,” Journal of Materials, in preparation (invited paper).
312. S.F. Li, C.M. Hefferan, J. Lind, U. Lienert, A.D. Rollett, R.M. Suter, “Three dimensional grain
boundary networks in defected materials,” in preparation.
313. S.F. Li, J. Lind, C.M. Hefferan, U. Lienert, A.D. Rollett, R.M. Suter, “Three dimensional
microstructure statistics from high energy x-ray diffraction microscopy data”, in preparation.
DMR090073
314. Adapted Su-Schrieffer-Heeger Hamiltonian for conducting polymers Li MH and Lin X
315. How does self-trapped soliton escape Shin YW, Li MH, Botelho AL, and Lin X
316. Computing the viscosity of supercooled liquids. III. Markov network model Li, J, Kushima A, Eapen
J, Lin X, Qian XF, Mauro JC, Diep P, and Yip S
317. Twin distortions of the Peierls instability Li MH and Lin X Physical Review B 81: Act. No. 153102
APR 8 2010
318. Commentary on the temperature-dependent viscosity of supercooled liquids: A unified activation
scenario Kushima A, Lin X, and Yip S Journal of Physics:
319. Condensed Matter 21: Act. No. 504104 NOV 11 2009
320. Computing the viscosity of supercooled liquids. II. Silica and strong-fragile crossover behavior
Kushima A, Lin X, Li J, Qian XF, Eapen J, Mauro JC, Diep P, and Yip S Journal of Chemical Physics 131:
Act. No. 164505 OCT 27 2009
321. Computing the viscosity of supercooled liquids Kushima A, Lin X, Li J, Eapen J, Mauro JC, Qian
XF, Diep P, and Yip S Journal of Chemical Physics 130: Act. No. 224504 June 11 2009
MCA08X007
322. J. W. Reiner, A. M. Kolpak, Y. Segal, K. F. Garrity, S. Ismail-Beigi, C.H. Ahn, and. F. J. Walker,
“Crystalline oxides on silicon,” Adv. Mater. (Early View) (2010).
323. A. M. Kolpak, F. J. Walker, J. W. Reiner, Y. Segal, D. Su, M. S. Sawicki, C. C. roadbridge, Z.
Zhang, Y. Zhu,
324. C. H. Ahn, and S. Ismail-Beigi, “Inhibition of ferroelectricity in Si/SrTiO3 due to intrinsic interface
effects,” under review, Phys. Rev. Lett. (2010).
325. A. M. Kolpak and S. Ismail-Beigi, “First-principles studies of epitaxial SrTiO3 on silicon, Part I:
Interface structure and film polarization,” in preparation (2010).
326. A. M. Kolpak, F. J. Walker, J. W. Reiner, C. H. Ahn, and S. Ismail-Beigi, “Ferroelectricity in atomic
layer thick MX2 films on silicon,” in preparation (2010).
327. A. M. Kolpak and S. Ismail-Beigi, “First-principles studies of epitaxial SrTiO3 on silicon, Part II:
Thermodynamic stability and growth kinetics,” in preparation (2010).

328. H. Chen, A. M. Kolpak, and S. Ismail-Beigi, “Electronic and Magnetic Properties of SrTiO3/LaAlO3
Interfaces from First Principles,” Adv. Mater. (Early View) (2010).
329. H. Chen, A. M. Kolpak, and S. Ismail-Beigi, “A first-principles study of LaAlO3/SrTiO3
heterointerfaces and their variants,” under review, Phys. Rev. B (2010).
330. K. F. Garrity, A. M. Kolpak, S. Ismail-Beigi, and E. A. Altman, “Chemistry of ferroelectric surfaces,”
Adv. Mater. (Early View) (2010).
331. K. F. Garrity, A. M. Kolpak, and S. Ismail-Beigi, “Ferroelectric surface chemistry: a first-principles
study of CO2 on PbTiO3,” in preparation (2010).
332. H. Tang and S. Ismail-Beigi, “Phase diagram of two-dimensional Mg boride nanostructures -A
genetic algorithm study,” in preparation (2010).
MCA08X029
333. T.-L. Chan, A.T. Zayak, G.M. Dalpian, and J.R. Chelikowsky, “Size effects on diffusion barriers in
semiconductor nanocrystals," Phys. Rev. Lett., 102: 025901 (2009).
334. J.R. Chelikowsky, A.T. Zayak, T.-L. Chan, M.L. Tiago, Y. Zhou, and Y. Saad, “Algorithms for the
Electronic and Vibrational Properties of Nanocrystals," J. Phys. Cond. Matter, 21: 064207 (2009).
335. J.R. Chelikowsky, Y. Saad, T.-L Chan, M.L. Tiago, A.T. Zayak, and Y. Zhou “Pseudopotentials on
Grids: Application to the Electronic, Optical, and Vibrational Properties of Silicon Nanocrystals,” Journal of
Computational and Theoretical Nanoscience, 6: 1247 (2009).
336. T.-L. Chan, C.Z. Wang, K.M. Ho, and J.R. Chelikowsky, “Efficient first-principles simulation of noncontact
atomic force microscopy for structural analysis," Phys. Rev. Lett., 102: 176101 (2009).
337. M.L. Tiago, J.C. Idrobo, S. Ogut, J. Jellinek, and J.R. Chelikowsky, “Electronic and optical
excitations in Agn clusters (n=1-8): Comparison of density-functional and many-body theories,” Phys. Rev.
B, 79: 155419 (2009).
338. K.H. Khoo and J.R. Chelikowsky, “Electron transport across carbon nanotube junctions decorated
with Au nanoparticles: Density functional calculations,” Phys. Rev. B, 79: 205422 (2009).
339. H. Kwak and J.R. Chelikowsky, “Size-dependent Spin-polarization of Carbon-doped ZnO
Nanocrystals,” Appl. Phys. Lett. 95: 263108 (2009).
340. I. Vasiliev, M. Lopez del Puerto, M. Jain, A. Lugo-Solis, and J.R. Chelikowsky, “Application of
timedependent density-functional theory to molecules and nanostructures,” Theochem (special Issue entitled
“TD-DFT for molecules and molecular solids”) 914: 115 (2009).
341. J.R. Chelikowsky, Y. Saad, T.-L. Chan, M.L. Tiago, A.T. Zayak, and Y. Zhou, “Pseudopotentials on
Grids: Application to the Electronic, Optical, and Vibrational Properties of Silicon Nanocrystals,” Journal of
Computational and Theoretical Nanoscience 6: 1247 (2009). K. Sohlburg, editor (special issue on first
principles calculations for nanostructures).
342. J.R. Chelikowsky, “Using Silicon to Understand Silicon,” in Into the Nano Era – Moores Law
Beyond Planar Silicon CMOS , H. Huff, editor, Springer Series in Materials Science (Springer-Verlag, 2009),
p.41.
343. Y. Saad, J.R. Chelikowsky, and S. Shontz, “Numerical Methods for Electronic Structure
Calculations of Materials,” SIAM Review, 52: 3 (2010).
344. T.-L. Chan and J.R. Chelikowsky, “Controlling diffusion in semiconductor nanostructures by size
and dimensionality,” Nano Lett. 10: 821 (2010).
345. J.-H. Eom, T.-L. Chan, and J.R. Chelikowsky, “The role of vacancies on B doping in Si
nanocrystals,” Solid State Commun. 150: 130 (2010).
346. M. Lopez del Puerto, M. Jain, and J.R. Chelikowsky, “Time-dependent density functional theory
calculation of the Stokes shift in hydrogenated silicon clusters,” Phys. Rev. B 81: 035309 (2010).
347. L.V. Besteiro, L. Tortajada, M.L. Tiago, L.J. Gallego, J.R. Chelikowsky, and M.M.G. Alemany:
“Efficient n-type doping of zinc-blende III-V semiconductor quantum dots,” Phys. Rev. B. 81: 121307R
(2010).

348. T.-L. Chan, J.R. Chelikowsky, K.-M. Ho, C.-Z. Wang, S. Zhang: “The effect of interface on quantum
confinement and electronic structure across the Pb/Si(111) junction,” Nature Physics (submitted).
349. T.-L. Chan, H. Kwak, J.-H. Eom, S. Zhang, and J.R. Chelikowsky: “A theory of self-purification in Si
nanocrystals,” Phys. Rev. B (submitted).
350. K.H. Khoo, M. Kim, G. Schofield and J.R. Chelikowsky: “Real space pseudopotential methods for
ab initio molecular dynamics simulations,” Phys. Rev. B (under review).
351. K.H. Khoo, A.T. Zayak, H. Kwak and J.R. Chelikowsky: “First-principles study of confinement
effects on the Rahman spectra of Si Nanocrystals,” Phys. Rev. Lett. (under review).
352. I. Vasliev and J.R. Chelikowsky: “Real-space calculations of atomic and molecular polarizabilities
using asymptotically correct exchange-correlation potentials,” Phys. Rev. A (in press).
353. G. Schofield, J.R. Chelikowsky and Y. Saad: “Using Chebyshev-Filtered Subspace Iteration and
Windowing Methods to Solve the Kohn-Sham Problem,” Comp. Phys. Comm. (submitted).
354. T.-L. Chan, J.R. Chelikowsky and S.B. Zhang: “Universality of formation energies in doping Si
nanostructures,” Phys. Rev. Lett. (submitted).
355. J. Han, T.-L. Chan and J.R. Chelikowsky: “Quantum confinement, core level shifts, and dopant
segregation in P-doped Sih110i nanowires,” Phys. Rev. B (submitted).
356. J.R. Chelikowsky, “Tool for predicting nanomaterial properties,” Handbook of Nanophysics, K.
Sattler, editor in chief. (Taylor and Francis, NY, 2009), (in press).
357. J.R. Chelikowsky, “Electrons in semiconductors: Empirical and ab initio pseudopotential theories,”
Comprehensive Semiconductor Science and Technology, Hiroshi Kamimura, editor (Elsevier), (in press).
358. J.R. Chelikowsky, “Algorithms for Predicting Electronic,Magnetic and Vibrational Properties of
Nanocrystals,” Computational Nanoscience, RSC Theoretical and Computational Chemistry Series (in
press).
359. J.R. Chelikowsky, T.-L. Chan, M.M.G Alemany and G. Dalpian, “Computational Studies of
Functionalized Nanostructures,” Reports on Progress in Physics, (submitted).
360. J.R. Chelikowsky, “Advances in Real Space Algorithms for the Kohn-Sham Problem,” Theoretical
Chemistry Accounts [feature article], (in preparation).
MCA09X001
361. Y. L. Lee, J. Kleis, J. Rossmeisl, and D. Morgan, Ab initio energetics of LaBO3(001) (B=Mn, Fe,
Co, and Ni) for solid oxide fuel cell cathodes, Physical Review B 80 (2009).
362. M. de Dompablo, Y. L. Lee, and D. Morgan, First Principles Investigation of Oxygen Vacancies in
Columbite MNb2O6 (M = Mn, Fe, Co, Ni, Cu), Chemistry of Materials 22, 906.
363. J. D. Tucker, R. Najafabadi, T. R. Allen, and D. Morgan, Ab initio-based diffusion theory and tracer
diffusion in Ni-Cr and Ni-Fe alloys, Submitted to Journal of Nuclear Materials (2010).
364. S. Choudhury, L. Barnard, J. D. Tucker, T. R. Allen, B. D. Wirth, M. Asta, and D. Morgan, Ab-initio
based modeling of diffusion in bcc Fe-Ni-Cr alloys and implications for radiation induced segregation,
Submitted to Journal of Nuclear Materials (2010).
365. David Shrader, Sarah M. Khalil, Tyler Gerczak, Todd R. Allen, Andrew J. Heim, Izabela Szlufarska,
and D. Morgan, Ag Diffusion in Cubic Silicon Carbide, Submitted to Journal of Nuclear Materials (2010).
366. W. Donner, A. J. Jacobson, C. L. Chen, Y.-L. Lee, M. Gadre, and D. Morgan, Epitaxial Strain-
Induced Chemical Ordering in La[sub 0.5]Sr[sub 0.5]CoO[sub 3-δ] Films on SrTiO[sub 3], Submitted (2010).
MCA93S030
367. Miguel A Morales,Carlo Pierleoni, Eric Schwegler, and D. M. Ceperley, Evidence for a firstorder
liquid-liquid transition in high-pressure hydrogen from ab initio simulations, Proceedings of the National
Academy of Sciences (2010).

368. M. A. Morales, C. Pierleoni, and D. M. Ceperley. ”Equation of State of liquid hydrogen at high
pressure from quantum Monte Carlo Calculations”, Phys. Rev. E 81 021202 (2010).,
369. F. Lin, M. A. Morales, K. T. Delaney, C. Pierleoni, R. M. Martin, and D. M. Ceperley,
“Quantum Monte Carlo Calculation of Liquid Hydrogen Electrical Conductivity”, Phys. Rev. Lett. 103, 256401
(2009).
370. B. Clark, M. Casula, D. M. Ceperley, ”Phase Transitions in the 2D one component plasma”, Phys.
Rev. Lett. 103, 055701 (2009).
371. M. White, M. Pasienski, D. McKay, S. Zhou, D. Ceperley, B. DeMarco, Strongly interacting
bosons in a disordered optical lattice, Phys. Rev. Letts., 102, 055301 (2009); arXiv:0807.0446v2
[physics.atom-ph]
372. Simo Huotari, J. Aleksi Soininen, Tuomas Pylkk¨anen, Keijo H¨am¨al¨ainen, Arezki Issolah, Andrey
Titov, Jeremy McMinis, Jeongnim Kim, Ken Esler, David M. Ceperley, Markus Holzmann, Valerio Olevano,
Momentum Distribution and Renormalization Factor in Sodium and the Electron Gas, submitted Phys. Rev.
Letts. June 2010.
373. K. P. Esler, R. E. Cohen, B. Militzer, Jeongnim Kim, R. J. Needs, and M. D. Towler,
Fundamental High-Pressure Calibration from All-Electron Quantum Monte Carlo Calculations, Phys. Rev.
Lett. 104, 185702 (2010)
Mathematical Sciences
DMS080028
374. B. Kouchmeshky and N. Zabaras, "The effect of multiple sources of uncertainty on convex hull of
material properties", Computational Material Science, Vol. 47, pp. 342352, 2009.
375. X. Ma and N. Zabaras, "An adaptive high-dimensional stochastic model representation technique
for the solution of stochastic partial differential equations", Journal of Computational Physics, Vol. 229, pp.
3884-3915, 2010.
376. B. Kouchmeshky and N. Zabaras, "Microstructure model reduction and uncertainty quantification in
multiscale deformation processes", Computational Materials Science, Vol. 48, pp. 213-227, 2010.
377. Z. Li, B. Wen and N. Zabaras, "Computing mechanical response variability of
polycrystalline microstructures through dimensionality reduction techniques",
Computational Materials Science, in press, 2010.
B. Wen, Z. Li and N. Zabaras, "Thermal response variability of random polycrystalline microstructures",
Communications in Computational Physics, submitted.
378. X. Ma and N. Zabaras, "A stochastic mixed finite element heterogeneous multiscale method for
flow in porous media", Journal of Computational Physics, submitted.
Molecular Biosciences
CHE090097
379. D. Svozil, P. Hobza, and J. ˇSponer. Comparison of Intrinsic Stacking Energies of Ten Unique
Dinucleotide Steps in A-RNA and B-DNA Duplexes. Can We Determine Correct Order of Stability by
Quantum-Chemical Calculations J. Phys. Chem. B, 114:1191–1203, 2010.
380. B. Heddi, C. Oguey, C. Lavelle, N. Foloppe, and B. Hartmann. Intrinsic flexibility of B-DNA: the
experimental TRX scale. Nucl. Acid. Res., 38:1034–1047, 2010.
381. A. Gil, V. Branchadell, J. Bertran, and A. Oliva. An Analysis of the Different Behavior of DNA and
RNA through the Study of the Mutual Relationship between Stacking and Hydrogen Bonding. J. Phys.
Chem. B, 113:4907–4914, 2009.
382. R. H. D. Lyngdoh and H. F. Schaefer III. Elementary Lesions in DNA Subunits: Electron, Hydrogen
Atom, Proton, and Hydride Transfers. Acc. Chem. Res., 42:563–572, 2009.

383. A. Szyperska, J. Rak, J. Leszczynski, X. Li, Y. J. Ko, H. Wang, and K. H. Bowen. Valence Anions
of 9Methylguanine–1-Methylcytosine Complexes. Computational and Photoelectron Spectroscopy Studies.
J. Am. Chem. Soc., 131:2663–2669, 2009.
384. K. E. Riley, Pitoˇn´Cern´ On the Structure and Geometry of Biomolecularak, J. ˇy, and P. Hobza.
Binding Motifs (Hydrogen-Bonding, Stacking, X—Hπ): WFT and DFT Calculations. J. Chem. Theory
Comput., 6:66–80, 2010.
385. R. M. Balabin. Communications: Is quantum chemical treatment of biopolymers acurate
Intramolecular basis set superposition error (BSSE). J. Chem. Phys., 132:231101–1–4, 2010.
MCA03S027
386. Black Pyrkosz, J. Eargle, A. Sethi, and Z. Luthey-Schulten. Exit strategies for charged tRNA from
GluRS. J. Mol. Biol., 397:1350-1371, Apr 2010.
387. Rebecca W. Alexander, John Eargle, and Zaida Luthey-Schulten. Experimental and computational
determination of tRNA dynamics. FEBS letters, 584:376-386, Jan 2010.
388. D. C. Mathew and Z. Luthey-Schulten. Inuence of montmorillonite on nucleotide oligomerization
reactions: a molecular dynamics study. Orig. Life Evol. Biosph., 40:303-317, Jun 2010.
389. Elijah Roberts, John E Stone, Leonardo Sepulveda, Wen-Mei W Hwu, and Zaida Luthey-Schulten.
Long time-scale simulations of in vivo di usion using GPU hardware. In Proceedings of the 2009 IEEE
International Symposium on Parallel & Distributed Processing, May 2009.
390. Sethi, J. Eargle, A. A. Black, and Z. Luthey-Schulten. Dynamical networks in tRNA:protein
complexes. Proc. Natl. Acad. Sci. U.S.A., 106:6620-6625, Apr 2009.
391. K. Chen, J. Eargle, K. Sarkar, M. Gruebele, and Z. A. Luthey-Schulten. The functional role of
ribosomal signatures. Submitted, 2010.
MCA04N033
392. Bolintineanu, D.S., et al., Poisson-Nernst-Planck Models of Nonequilibrium Ion Electrodiffusion
through a Protegrin Transmembrane Pore. PLoS Comput Biol, 2009. 5(1): p. e1000277.
393. Sotiropoulos, V., et al., Model Reduction of Multiscale Chemical Langevin Equations: A Numerical
Case Study. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2009. 99(2): p. 5555.
394. Sotiropoulos, V. and Y.N. Kaznessis, Analytical Derivation of Moment Equations in Stochastic
Chemical Kinetics. J. Chem. Phys., 2009. submitted.
395. Sayyed-Ahmad, A. and Y.N. Kaznessis, Determining the orientation of protegrin1 in DLPC bilayers
using an implicit solvent-membrane model. PLoS One, 2009. 4(3): p. e4799.
396. Ramalingam, K., et al., Forward engineering of synthetic bio-logical AND gates Biochemical
Engineering Journal, 2009. 47(1-3): p. 38-47.
397. Bolintineanu, D., et al., Antimicrobial mechanism of pore-forming protegrin peptides: 100 pores to
kill E. coli. Peptides, 2010. 31(1): p. 1-8.
398. Vivcharuk, V. and Y. Kaznessis, Free energy profile of the interaction between a monomer or a
dimer of protegrin-1 in a specific binding orientation and a model lipid bilayer. J Phys Chem B, 2010. 114(8):
p. 2790-7.
399. Vivcharuk, V. and Y.N. Kaznessis, Dimerization of protegrin-1 peptides in different environments.
Int. J. Mol. Sci., 2010. Submitted June 2010.
400. Biliouris, K., P. Daoutidis, and Y.N. Kaznessis, Stochastic simulations of the tetracycline operon.
BMC Syst Biol, 2010. Submitted.

MCA05S010
401. “Exchange often and properly in replica exchange molecular dynamics.” Daniel J. Sindhikara,
Daniel J. Emerson§, Adrian E. Roitberg. Journal of Chemical Theory and Computation. Submitted and under
review (2010)
402. “Electronic Spectra of the Nanostar Dendrimer: Theory and Experiment.” Julio L. Palma, Evrim
Atas, Lindsay Hardison, Todd B. Marder, Jonathan C. Collings, Andrew Beeby, Joseph S. Melinger, Jeffrey
L. Krause, Valeria D. Kleiman, and Adrian E. Roitberg. Journal of Physical Chemistry. Submitted and under
review (2010)
403. “Comparative validation of computational methods for conformationalanalysis. The Non aromatic
cases of alanine3 and glycine3”. L. Fusti-Molnar, G. Seabra, Robert Abel, A. E. Roitberg and Kenneth M.
Merz, Jr. Journal of Chemical Theory and Computation. Submitted and under review (2010)
404. “p38γ activation triggers dynamical changes in allosteric docking sites”. Ramiro G. Rodriguez
Limardo, Dardo N. Ferreiro, Adrian E. Roitberg, Marcelo A. Marti and Adrian G. Turjanski. Biochemistry.
Submitted and under review (2010)
405. “Unidirectional energy transfer in conjugated molecules: the crucial role of high frequency C(triple)C
bonds.” S. Fernandez-Alberti, Valeria D. Kleiman, S. Tretiak, and Adrian E. Roitberg. Journal of Physical
Chemistry Letters. Submitted and under review (2010) “Substrate Stereo-specificity in tryptophan
dioxygenase and indoleamine 2,3-dioxygenase” L. Capece, M. Arrar, A. E. Roitberg, Syun-Ru Yeh, M. A.
Marti, D. A. Estrin. Proteins: Structure, Function, and Bioinformatics. doi:10.1002/prot.22819 (2010)
406. “Impact of calcium on N1 influenza neuraminidase dynamics and binding free energy.” Morgan
Lawrenz, Jeff Wereszczynski, Rommie Amaro, Ross Walker, Adrian Roitberg, J. Andrew McCammon.
Proteins: Structure, Function, and Bioinformatics 78:2523–2532 (2010)
407. "Constant pH Replica Exchange Molecular Dynamics in Biomolecules Using a Discrete Protonation
Model". Meng Y, Roitberg AE, Journal of Chemical Theory and Computation, 6:1401–1412 (2010).
408. “Computational Studies of Ammonia Channel Function in Glutamine 5'-
Phosphoribosylpyrophosphate Amidotransferase”, Wang, XS, Roitberg, AE, Richards, NGJ, Biochemistry,
48:12272-12282 (2009).
409. “Apo and Nickel-Bound Forms of the Pyrococcus horikoshii Species of the Metalloregulatory
Protein: NikR Characterized by Molecular Dynamics Simulations”, Sindhikara, DJ, Roitberg, AE, Merz, KM,
Biochemistry, 48:12024-12033 (2009).
410. “AM1 Parameters for the Prediction of H-1 and C-13 NMR Chemical Shifts in Proteins”, Williams,
DE, Peters, MB, Wang, B, Roitberg, AE, Merz, KM, Journal of Physical Chemistry A, 113:11550-11559
(2009).
411. “Are Current Semiempirical Methods Better Than Force Fields A Study from the Thermodynamics
Perspective”, Seabra, GD, Walker, RC, Roitberg, AE, Journal of Physical Chemistry A, 113:11938-11948
(2009).
MCA05S027
412. Bellesia, G.; Shea, J.-E. What determines the structure and stability of KFFE monomers, dimers,
and protofibrils Biophys. J. 2009, 96, 875-886.
413. Wu, C.; Murray, M. M.; Bernstein, S. L.; Condron, M. M.; Bitan, G.; Shea, J.-E.; Bowers, M. T. The
structure of A42 C-terminal fragments probed by a combined experimental and theoretical study. J. Mol.
Biol. 2009, 387, 492-501.
414. Murray, M. M.; Krone, M. G.; Bernstein, S. L.; Baumketner, A.; Condron, M. M.; Lazo, N. D.;
Teplow, D. B.; Wyttenbach, T.; Shea, J.-E.; Bowers, M. T. Amyloid -protein: Experiment and theory on the
21-30 fragment. J. Phys. Chem. B 2009, 113, 6041-6046.
415. Bernstein, S. L.; Dupuis, N. F.; Lazo, N. D.; Wyttenbach, T.; Condron, M. M.; Bitan, G.; Teplow, D.
B.; Shea, J.-E.; Ruotolo, B. T.; Robinson, C. V.; Bowers, M. T. Amyloid- protein oligomerization and the
importance of tetramers and dodecamers in the aetiology of Alzheimer's disease. Nat. Chem. 2009, 1, 326-
331.

416. Armstrong, B. D.; Soto, P.; Shea, J.-E.; Han, S. Overhauser dynamic nuclear polarization and
molecular dynamics simulations using pyrroline and piperidine ring nitroxide radicals. J. Magn. Reson. 2009,
200, 137-141.
417. Penev, E. S.; Lampoudi, S.; Shea, J.-E. TiReX: Replica-exchange molecular dynamics using
TINKER. Comput. Phys. Commun. 2009, 180, 2013-2019.
418. Wu, C.; Biancalana, M.; Koide, S.; Shea, J.-E. Binding modes of thioflavin-T to the single-layer -
sheet of the peptide self-assembly mimics. J. Mol. Biol. 2009, 394, 627-633.
419. Dupuis, N. F.; Wu, C.; Shea, J.-E.; Bowers, M. T. Human islet amyloid polypeptide monomers form
ordered -hairpins: A possible direct amyloidogenic precursor. J. Am. Chem. Soc. 2009, 191, 18283–18292.
420. Grabenauer, M.; Wu, C.; Soto, P.; Shea, J.-E.; Bowers, M. T. Oligomers of the prion protein
fragment 106-126 are likely assembled from -hairpins in solution and methionine oxidation inhibits
assembly without altering the peptide's monomeric conformation. J. Am. Chem. Soc. 2010, 132, 532-539.
421. Wei, G. H.; Jewett, A. I.; Shea, J.-E. Structural diversity of the dimer of the Alzheimer amyloid-
(25-35) peptide and polymorphism of the resulting fibrils. Phys. Chem. Chem. Phys. 2010, 12, 3622-3629.
422. Wu, C.; Bowers, M. T.; Shea, J.-E. Molecular structures of quiescently-grown and brain-derived
polymorphic fibrils of the Alzheimer amyloid Aβ9-40 peptide: A comparison to agitated fibrils. PLoS Comp.
Biol. 2010, 6, e1000693.
423. Wu, C.; Bowers, M. T.; Shea, J.-E. The binding of thioflavin T and of PIB to the cross-β subunit of
the Alzheimer Aβ9-40 protofibril. 2010, submitted.
424. Wu, C.; Shea, J.-E. On the origins of the weak folding cooperativity of a designed ultrafast
protein. 2010, submitted.
MCA06N060
425. G. Enkavi and E. Tajkhorshid. Dynamics of Spontaneous Substrate Binding and Identi fication of the
Binding Site in GlpT. Biochemistry., 49:1105–1114, 2010.
426. J. Diao, A. J. Maniotis, R. Follberg and E. Tajkhorshid. Interplay of mechanical and binding
properties of type I fibronectin. Theoret. Chem. Acc., 125:397–405, 2010.
427. Y. Wang, S. A. Shaikh and E. Tajkhorshid. Exploring transmembrane diffusion pathways with
molecular dynamics. Physiology, 25: 142-145, 2010.
428. Y. Z. Ohkubo, J. H. Morrissey, and E. Tajkhorshid. Dynamical view of membrane binding and
complex formation of human factor VIIa and tissue factor. J. Thromb. Haemost., 8:1044, 2010
429. Y. Wang and E. Tajkhorshid. Exploring the Permeability of Aquaporin AQP4 to Signaling
Gas Molecule NO with Molecular Dynamics. PROTEINS, 78(3): 661-670, 2010. 6. J. H. Morrissey, R. L.
Davis-Harrison, N. Tavoosi, K. Ke, V. Pureza, J. M. Boettcher, M. C. Clay, C. M. Rienstra, Y. Z. Ohkubo, T.
V. Pogorelov, and Emad Tajkhorshid. Protein-Phospholipid interactions in blood clotting. Thromb. Research,
125 (Suppl. 1):S23, 2010 Saher Shaikh, Po-Chao Wen, Giray Enkavi, Zhijian Huang, and Emad
Tajkhorshid. Capturing Functional Motions of Membrane Channels and Transporters with Molecular
Dynamics Simulation. J. Comput. Theor. Nanosci., 7:1-10, 2010 8. J. Feng, E. Lucchinetti, G. Enkavi, Y.
Wang, P. Gehrig, B. Roschitzki, M. Schaub, E. Tajkhorshid, K. Zaugg and M. Zaugg. Phosphorylation of the
conserved aromatic ladder (Y194/Y190) in adenine nucleotide translocase 1 (ANT1, AAC1) is Src-family
kinase-mediated and regulates ADP/ATP exchange in mitochondria: a downstream mechanism of
cardioprotection. Am. J. Physiol. Cell Physiol.,298:C740C748, 2010.
430. F. Khalili-Araghi, V. Jogini, V. Yarov-Yarovoy, E. Tajkhorshid, B. Roux, and K. Schulten.
Calculation of the gating charge for the Kv1.2 voltageactivated potassium channel. Biophys. J., 98:2189–
2198, 2010.
431. X.-D. Yang, E. Tajkhorshid, and L.-F. Chen Functional Interplay between Acetylation and
Methylation of the RelA Subunit of NF-kappaB. Mol. Cell. Biol., 30:2170–2180, 2010.
432. Z. Huang and E. Tajkhorshid. Identification of the Third Na + Site and the Sequence of Extracellular
Binding Events in the Glutamate Transporter. Biophys. J., In press.

433. H.-C. Siebert, M. Burg-Roderfeld, Th. Eckert, S. Sttzel, U. Kirch, T. Diercks, M. J.
Humphries, M. Frank, R. Wechselberger, E. Tajkhorshid, and S. Oesser. Interaction of the alpha2A domain
of integrin with small collagen fragments. Protein and Cell, In press.
434. P. C. Wen, Z. Huang, G. Enkavi, Y. Wang, J. C. Gumbart and E. Tajkhorshid. Molecular
Mechanisms of active transport across the cellular membrane. In, Editors: Mark S.P. Sansom and Philip C.
Biggin, “Molecular Simulations and Biomembranes: From Biophysics to Function”, Royal Society of
Chemistry. ISBN:978-0-85404-189-3. In Press.
435. S. A. Shaikh and E. Tajkhorshid. Modeling and dynamics of the inward-facing state of a Na + /Cl−
dependent neurotransmitter transporter homologue. PLoS Comput. Biol., In Press.
436. J. C. Gumbart, M. C. Wiener and E. Tajkhorshid. Coupling of calcium and substrate binding
through
loop alignment in the outer membrane transporter BtuB. J. Mol. Biol., 393:1129–1142, 2009.
437. C. J. Law, G. Enkavi, D. N. Wang and E. Tajkhorshid. Structural basis of substrate selectivity in the
glycerol-3-phosphate:phosphate antiporter GlpT. Biophys. J.,97:1346–1353, 2009.
438. J. Li and E. Tajkhorshid. Ion-releasing State of A Secondary Membrane Transporter. Biophys.
J.,97:L29–
L31, 2009. 18. J. Dittmer, L. Thøgersen, J. Underhaug, K. Bertelsen, T. Vosegaard, J. M. Pedersen, B.
Schiøtt, E. Tajkhorshid, T. Skrydstrup and N. C. Nielsen Incorporation of Antimicrobial Peptides into
Membranes: A Combined Liquid-State NMR and Molecular Dynamics Study of Alamethicin in DMPC/DHPC
Bicelles. J. Phys. Chem. B, 113:6928–6937, 2009.
439. K. Bertelsen, B. Paaske, L. Thogersen, E. Tajkhorshid, B. Schiott, T. Skrydstrup, N. Chr. Nielsen,
and Th. Vosegaard Residue-Specific Information about the Dynamics of Antimicrobial Peptides from
1H-15N and 2H Solid-State NMR Spectroscopy. J. Am. Chem. Soc., 131:18335–18342, 2009.
440. J. H. Morrissey, V. Pureza, R. L. Davis-Harrison, S. G. Sligar, C. M. Rienstra, A. Z. Kijac, Y. Z.
Ohkubo, and Emad Tajkhorshid. Proteinmembrane interactions: blood clotting on nanoscale bilayer. J.
Thromb. Haemost., 7 (Suppl. 1):169, 2009
441. P. C. Wen and E. Tajkhorshid. Capturing Large-Scale Conformational Dynamics of P-glycoprotein
by
MD Simulations. Proc. Natl. Acad. Sci. U.S.A., Submitted.
442. J. Li and E. Tajkhorshid. Spontaneous Unbinding of the Substrate in Sodium-Glucose
Transporter.
Biophys. J., Submitted.
443. P. C. Wen and E. Tajkhorshid. Capturing Large-Scale Conformational Dynamics of P-glycoprotein
by
MD Simulations. (Oral presentation) FASEB Summer Research Conference 2010. Snowmass Village,
CO.
444. P. C. Wen and E. Tajkhorshid. The Origin of Nucleotide Dependence of Conformational
Changes
in ABC Transporters. (Selected oral presentation at Permeation & Transport Subgroup meeting)
Biophysical Society 54th Annual Meeting. San Francisco, CA. Biophys. J., 98:627a–628a, 2010.
445. Z. Huang and E. Tajkhorshid. Sequence of Events in the Extracellular Half of the Transport Cycle in
Glutaamte Transporter. Biophysical Society 54th Annual Meeting. San Francisco, CA. Biophys. J.,
98:686a, 2010.

446. S. A. Shaikh and E. Tajkhorshid. Modeling of the Inward-Facing State of LeuT and Dynamics of the
Outward-To-Inward Transition. Biophys. J., 98:687a, 2010.
447. G. Enkavi and E. Tajkhorshid. Molecular determinants of the Stoichiometry of Transport in
GlpT.
Biophys. J., 98:687a, 2010.
448. J. Li and E. Tajkhorshid. Structural Transition Between the Ion-Releasing and Ion-Binding States of
a Secondary Membrane Transporter. Biophys. J., 98:686a–687a, 2010.
449. Y. Z. Ohkubo and E. Tajkhorshid. Dynamical Basis of the enhancement of the enzymatic activity of
factor FVIIa by tissue factor. Biophys. J., 98:689a, 2010
450. Y. Z. Ohkubo and E. Tajkhorshid. Dynamical view of membrane binding and complex formation of
human tissue factor and factor VIIa. The 5th Symposium on Hemostasis, 2010
MCA08X025
451. Claxton DP, Quick M, Shi L, de Carvalho FD, Weinstein H, Javitch JA, Mchaourab HS.
Ion/substrate-dependent conformational dynamics of a bacterial transporter homolog of
neurotransmitter:sodium symporters. Nat Struct Mol Biol. 2010 June 20.
452. Zhao Y, Terry D, Shi L, Weinstein H, Blanchard SC, Javitch JA. Single-molecule dynamics of
gating in a neurotransmitter transporter homolog. Nature. 2010 May 13;465(7295):188-93
MCA08X026
453. Krepkiy, D., Mihailescu, M., Freites, J. A., Schow, E. V., Worcester, D. L., Gawrisch, K., Tobias, D.
J., White, S. H., Swartz, K. J.. (2009). Structure and hydration of membranes embedded with voltagesensing
domains. Nature 462:473-479
454. Schow, E. V., Freites, J. A., Gogna, K. , White, S. H. and Tobias, D. J. (2010). Down-state model of
the voltage-sensing domain of a potassium channel. Biophys. J. 98:2857-2866.
455. Jardon-Valadez, E., Bondar, A.-N., and Tobias, D.J. Coupling of retinal, water, and protein in squid
rhodopsin. (2010) Biophys. J., In Press.
456. Bondar, A.-N., Del Val, C., Freites, J.A., Tobias, D.J., and White, S.H. (2010). Dynamics of SecY
translocons with translocation-defective mutations. Structure, 18:847-857.
457. Klauda, J. B., Venable, R. M., Freites, J. A., O'Connor, J. W., Tobias, D. J., Mondragon-Ramirez,
C., Vorobyov, I., MacKerell, Jr., A. D., Pastor, R. W. (2010). Update of the CHARMM all-atom additive force
field for lipids: validation on six lipid types. J. Phys. Chem. B. 114:7830-7843.
458. Bondar, A.-N., Tobias, D.J., and White, S.H. (2010). How binding of the signal peptide unlocks the
translocon. Biophys. J. 98, Supplement 1, 435a.
459. Bondar, A.-N., and White, S.H. (2010). Lipid membrane composition has a dramatic effect on the
dynamics of the GlpG rhomboid protease from Escherichia coli. Biophys. J. 98, Supplement 1, 224a.
460. Schow, E. V., Nizkorodov, A., Freites, J. A. White, S. H. and Tobias, D. J. (2010) Down-State
Model of the KvAP Full Channel. Biophys. J. 98, Supplement 1, 315a.
461. Keer, H. S., Freites, J. A., Mihailescu, E., White, S. H., and Tobias, D. J. (2010) Structure of a
DOTAP Lipid Bilayer: A Concerted Neutron Scattering and Molecular Dynamics Study. Biophys. J. 98,
Supplement 1, 492a.
462. Jardon-Veladez, E., Freites, J. A., Mihailescu, E., Worcester, D., White, S. H.,and Tobias, D.J.
(2010) Neutron Scattering and MD Simulation Study of DOPC and DOPC/cholesterol Bilayers. Biophys. J.
98, Supplement 1, 493a.

463. Clemens, D.M., Freites, J.A., Kalman, K., Neméth-Cahalan, K,. Tobias, D.J., and Hall, J.E. A
Molecular Dynamics Study of the AQP0-CaM Interaction (talk). ARVO Meeting, May 2010, Fort Lauderdale,
FL.
464. Clemens, D.M., Freites, J.A., Kalman, K., Neméth-Cahalan, K,. Tobias, D.J., and Hall, J.E. A
Molecular Dynamics Study of the AQP0-CaM Interaction (poster). NLM Informatics Training Meeting, June
2010, Denver, CO. (Best Poster Award).
465. Schow, E. V., A. Nizkorodov, J. A. Freites, S. H. White, and D. J. Tobias Atomistic Model of a
Closed Voltage-gated Potassium Channel Derived from Experimental Data.
466. Andersson, M., Freites, J.A., Tobias, D.J., and White, S.H. Accommodating charge in the lipid
bilayer.
467. Freites, J.A., Kasho, V., Tobias, D.J., Kaback, H.R., and White, S.H. Identifying transmembrane
segments in a membrane protein of complex fold.
468. Freites, J.A., Andersson, M., Kasho, V., Smirnova, I., Tobias, D.J., Kaback, H.R., and White, S.H.
Structural insights into the inward-facing conformation of lactose permease from atomistic simulations.
469. Jardon-Valadez, E., and Tobias, D.J. Protein and lipid dynamical coupling in bacteriorhodopsin and
squid rhodopsin at room temperature.
470. Bondar, A.-N., Tobias, D.J., and White, S.H. How binding of the signal peptide opens the protein
translocon.
471. Clemens, D.M., Freites, J.A., Kalman, K., Neméth-Cahalan, K,. Tobias, D.J., and Hall, J.E.
Mechanism of AQP0 Permeability Blockade by CaM.
472. Mihailescu, M., Vaswanic,R.G., Castro-Román, F., Jardón-Valadez, E., Freites, Worcester, D.L.,
Chamberlin, A.R., Tobias, D.J., White, S.H. Acyl-chain methyl distributions of liquid-disordered and liquidordered
membranes.
MCA93S020
473. R. Acharya, V. Carnevale, G. Fiorin, B. G. Levine, A L. Polishchuk, V. Balannik, I. Samish, R. A.
Lamb, L. H. Pinto and. W. F. DeGrado, and M. L. Klein. Structure and mechanism of proton transport
through the transmembrane tetrameric m2 protein bundle of the in fluenza a virus. Proc. Natl. Acad. Sci.
U.S.A, 2010. (in press).
474. Victoria Balannik, Vincenzo Carnevale, Giacomo Fiorin, Benjamin G Levine, Robert A Lamb,
Michael L Klein, William F Degrado, and Lawrence H Pinto. Functional studies and modeling of pore-lining
residue mutants of the influenza a virus m2 ion channel. Biochemistry, 49(4):696– 708, 2010.
475. B. L. Bhargava and Michael L Klein. Aqueous solutions of imidazolium ionic liquids: molecular
dynamics studies. Soft Matter, 5:3475–3480, 2009.
476. B. L. Bhargava and Michael L Klein. Formation of micelles in aqueous solutions of a room
temperature ionic liquid: a study using coarse grained molecular dynamics. Mol Phys, 107:393–401, 2009.
477. B. L. Bhargava and Michael L Klein. Initial stages of aggregation in aqueous solutions of ionic
liquids: molecular dynamics studies. J Phys Chem B, 113(28):9499–9505, 2009.
478. G. Brannigan, D. N. LeBard, J. Hnin, R. Eckenhoff, and M. L. Klein. Multiple binding sites for the
general anesthetic isoflurane identified in the nicotinic acetylcholine receptor transmembrane domain. Proc.
Natl. Acad. Sci. USA, 2010. (in press).
479. V. Carnevale, G. Fiorin, B. G. Levine, W. F. DeGrado, and M. L. Klein. Charge localization in the
pore of a biological proton channel. 2010. (in preparation).
480. V. Carnevale, G. Fiorin, B. G. Levine, W. F. DeGrado, and M. L. Klein. Polarization of water
molecules in the m2 channel of the in fluenza a virus. 2010. (in preparation).
481. R. H. DeVane, A. Jusufi, P. Moore, and M. L. Klein. Parametrization and application of a coarse
grained forcefield for benzene/ fullerene interactions with lipids. 2010. (in preparation).
482. J. E. Donald, Y. Zhang, G. Fiorin, V. Carnevale, D. Slochower, F. Gai, M. L. Klein, and W. F.
DeGrado. Association of transmembrane helices in viral fusion peptides suggests a protein-centric
mechanism of membrane fusion. 2010. (in preparation).

483. J´erˆome H´enin, Grace Brannigan, William P Dailey, Roderic Eckenhoff, and Michael L Klein. An
atomistic model for simulations of the general anesthetic iso flurane. J. Phys. Chem. B, 114(1):604–612, Jan
2010.
484. Ming-Hsun Ho, Marco De Vivo, Matteo Dal Peraro, and Michael L. Klein. Unraveling the catalytic
pathway of metalloenzyme farnesyltransferase through qm/mm computation. J. Chem. Theory Comput.,
5:16571666, 2009.
485. Robert R. Johnson, Axel Kohlmeyer, A.T. Charlie Johnson, and Michael L. Klein. Free energy
landscape of a dna-carbon nanotube hybrid using replica exchange molecular dynamics. Nano Letters,
9:537–541, 2009.
486. A. Jusufi, R. H. DeVane, W. Shinoda, and M. L. Klein. Fullerene size effects on their interaction
with lipid membranes. 2010. (in preparation).
487. S. M. Loverde, V. Ortiz, R. D. Kamien, M. L. Klein, and D. E. Discher. Curvature-driven molecular
demixing in the budding and breakup of mixed component worm-like micelles. Soft Matter, 6:1419–1425,
2010. (cover article).
488. S. M. Loverde, K. Rajagopal, M. L. Klein, and D. E. Discher. Molecular simulation studies of
polyethylene oxide polycaprolactone diblock copolymer micelles. 2010. (in preparation).
489. Virgil Percec, Daniela A Wilson, Pawaret Leowanawat, Christopher J Wilson, Andrew D Hughes,
Mark S Kaucher, Daniel A Hammer, Dalia H Levine, Anthony J Kim, Frank S Bates, Kevin P Davis, Timothy
P Lodge, Michael L Klein, Russell H DeVane, Emad Aqad, Brad M Rosen, Andreea O Argintaru, Monika J
Sienkowska, Kari Rissanen, Sami Nummelin, and Jarmo Ropponen. Self-assembly of janus dendrimers into
uniform dendrimersomes and other complex architectures. Science, 328(5981):1009– 1014, May 2010.
490. Werner Treptow and Michael L Klein. The membrane-bound state of k2p potassium channels. J
Am Chem Soc, 132(23):8145–8151, Jun 2010.
491. L. Sangeetha Vedula, Grace Brannigan, Nicoleta J Economou, Jin Xi, Michael A Hall, Renyu Liu,
Matthew J Rossi, William P Dailey, Kimberly C Grasty, Michael L Klein, Roderic G Eckenhoff, and Patrick J
Loll. A unitary anesthetic binding site at high resolution. J. Biol. Chem., 284(36):24176–24184, Sep 2009.
492. M. Wanunu, D. Cohen-Karni, R. R. Johnson, Y. Zheng, M. L. Klein, and M. Drndic. Discrimination
of modified cytosines in dna. 2010. (in preparation).
MCB050002
493. S. Dorairaj & T. W. Allen. 2007. On the thermodynamic stability of a charged arginine side chain in
a transmembrane helix. Proc.Natl.Acad.Sci. 104:4943-4948.
494. I. Vorobyov and T. W. Allen. 2009. “Molecular Dynamics Computations for Proteins: a case study in
membrane ion permeation.” Handbook of Molecular Biophysics: methods and applications. Chapter 8. ed.
H.G.Bohr. Wiley, Weinheim.
495. A.N. Thompson, I. Kim, T. D. Panosian, T. Iverson, T. W. Allen and C. M. Nimigean. 2009.
“Mechanism of potassium channel selectivity revealed by Na+ and Li+ binding sites inside the KcsA pore.”
Nat. Struct. Mol. Biol. 16: 1317-1324.
496. I. Vorobyov, B. Bekker and T.W. Allen. 2010. “The electrostatics of deformable lipid membranes.”
Biophysical Journal. 98: 2904-2913.
497. I. Vorobyov and T. W. Allen. 2010. “Electrostatics of solvent and membrane interfaces and the role
of electronic polarizability. J. Chem. Phys. 132:185101.
498. A.H. de Jesus and T. W. Allen. “On the structural and energetic consequences of membrane
hydrophobic mismatch and protein anchoring side chains”, in preparation for Biophys. J.
499. B. Bekker and T. W. Allen. “The membrane permeation mechanism of the ionophore valinomycin”,
in preparation for Biophys. J.
500. T. W. Allen, I. Vorobyov, R. Koeppe III and Olaf. S. Andersen. “On the permeation of guanidinium
and other ions through lipid bilayers”, in preparation for Biophys. J.
501. D. Bennet, I. Vorobyov, D. P. Tieleman. T. W. Allen and S. Noskov. “Chloroform partitioning into
membranes and the role of electronic polarizability”, in preparation for J. Am. Chem. Soc.

502. L. Li and T.W. Allen. “Membrane deformations under the actions of lysine and arginine containing
peptides”, in preparation for J. Phys. Chem. B.
503. Vorobyov, I. and T. W. Allen. “The role of anionic lipids in membrane association and translocation
of charged peptides”, in preparation for J. Am. Chem. Soc.
MCB060006
504. Calculation of free energies in fluid membranes subject to heterogeneous curvature fields, N. J.
Agrawal, R. Radhakrishnan, Physical Review E, 80: 011925, 2009. Pubmed ID: 19658747.
505. Computational Delineation of the Catalytic Step of a High Fidelity DNA Polymerase, R.
Venkatramani, R. Radhakrishnan, Protein Science (A Protein Society Journal), 19(4), 815-825, 2010.
Pubmed ID: 20162624.
506. The ErbB3/HER3 Intracellular Domain is Competent to Bind ATP and Catalyze
Autophosphorylation, F. Shi, S. E. Telesco, Y. Liu, R. Radhakrishnan*, M. A. Lemmon*, Proceedings of the
National Academy of Sciences, 2010, in press. Pubmed ID: 20351256; *Co-corresponding authors.
507. Linking oncogenic signaling to molecular structure, Multiscale Cancer Modeling, J. Purvis, A. Shih,
Y. Liu, R. Radhakrishnan, Eds. T.S. Deisboeck, G. Stamatakos, Chapman & Hall-CRC Mathematical and
Computational Biology Series, 2010, in press.
508. Computational methods related to reaction chemistry, A. J. Shih*, S. T. Telesco*, Y. Liu, R.
Venkatramani, R. Radhakrishnan, Comprehensive Biomaterials, eds. P. Ducheyne, Elsevier London, 2010,
in press. *These authors contributed equally.
509. Structural analysis of the activation loop conformation of Epidermal Growth Factor Receptor
Tyrosine Kinase through molecular dynamics simulations, A. Shih, S. H. Choi, M. A. Lemmon, R.
Radhakrishnan, 2009, submitted to Biophsical J.
510. Protein-Mediated Orchestration of Vesiculation Prior to Vesicle-Scission in Clathrin-Dependent
Endocytosis, N. Agrawal, J. Nukpezah, R. Radhakrishnan, 2009, submitted to Plos Computational Biology.
511. A finite element method for simulating thermally fluctuating Brownian particles, B. Uma, T. N.
Swaminathan, R. Radhakrishnan, D. M. Eckmann, P. S. Ayyaswamy, 2010, submitted to Phys. Rev. E.
512. Design of functionalized nanocarriers for endothelial targeting based on Monte Carlo calculations of
absolute free energy of binding, J. Liu, D. M. Eckmann, P. S. Ayyaswamy, V. Muzykantov, R.
Radhakrishnan, 2010, submitted to PNAS.
513. A molecular docking study of the binding of ruthenium complex compounds to PIM1, GSK3, and
CDK2 protein kinases, Y. Liu, N. J. Agrawal, R. Radhakrishnan, 2009, submitted.
514. Mechanisms of phosphoryl transfer in ErbB3 kinase, F. Jia, S. Telesco, Y. Liu, R. Radhakrishnan,
2010, to be submitted.
515. Energy landscapes in tyrosine phosphorylation and mechanisms of phosphoryl transfer in ErbB1
kinase, Y. Liu, R. Radhakrishnan, 2010, to be submitted.
516. Binding of Erlotinib to Inactive ErbB1 tyrosine kinase domain, Y. Liu, J. Park, M. A. Lemmon, R.
Radhakrishnan, 2010, to be submitted.
517. Mechanism of clathrin-mediated vesicle budding in receptor endocytosis, N. J. Agrawal, S. Engles,
R. Toy, R. Radhakrishnan, 2010, to be submitted.
MCB060069
518. Gedeon, P.C.; Indarte, M.; Surratt, C.K.; Madura, J.D. Molecular dynamics of leucine and
dopamine transporter proteins in a model cell membrane lipid bilayer, Proteins, 2010. 78, 797.
519. Asciutto, E.K.; General, I.J.; Xiong, K.; Asher S.A.; and Madura, J.D. Sodium Perchlorate effects on
the helical stability of a mainly alanine peptide. Biophys. J, 2010, 98, 186.
520. Indarte, M; Liu, Y.; Madura, J.D. and Surratt, C.K. Receptor---based discovery of plasmalemmal
monoamine transporter inhibitor via high---throughput docking and pharmacophore modeling. ACS Chemical
Neuroscience, 2010. 1, 223.

MCB070009
521. J. Lee, S. Ham, and W. Im (2009) Beta-Hairpin Restraint Potentials for Calculations ofPotentials of
Mean Force as a Function of Beta-Hairpin Tilt, Rotation, and Distance. J. Comput. Chem. 30:1334-1343.
522. W. Im, J. Lee, T. Kim, and H. Rui (2009) Novel Free Energy Calculations to Explore Mechanisms
and Energetics of Membrane Protein Structure and Function, J. Comput. Chem. 30:1622-1633.
523. S. Jo, J.B. Lim, J.B. Klauda, and W. Im (2009) CHARMM-GUI Membrane Builder for Mixed Bilayers
and Its Application to Yeast Membranes, Biophys. J. 97:50-58.
524. H. Rui, J. Lee, and W. Im (2009) Comparative Molecular Dynamics Simulation Studies of Protegrin-
1 Monomer and Dimer in Different Lipid Bilayers, Biophys. J. 97:787-795.
525. T. Rathinavelan, L. Zhang, W.L. Picking, D.D. Weis, R.N. De Guzman, and W. Im (2010) A
Repulsive Electrostatic Mechanism for Protein Export through the Type III Secretion Apparatus, Biophys. J.
98:452-461 [cover].
526. T. Kim and W. Im (2010) Revisiting Hydrophobic Mismatch with Free Energy Calculations
ofTransmembrane Helix Tilt and Rotation. Biophys. J. (in press).
527. H. Rui and W. Im (2010) Protegrin-1 Orientation and Physicochemical Properties in Membrane
Bilayers Studied by Potential of Mean Force Calculations. J. Comput. Chem. (in press).
528. K.C. Song, P.W. Livanec, J.B. Klauda, K. Kuczera, R.C. Dunn, and W. Im (2010) Orientation of
Fluorescent Lipid Analog BODIPY-PC to Probe Lipid Membrane Properties: A Comparison of Simulation
with Experiment. submitted.
529. S. Jo, H. Rui, J.B. Lim, J.B. Klauda, and W. Im (2010) Cholesterol Flip-Flop: Insights from Free
Energy Simulation Studies, submitted.
MCB070118
530. Ruibo Wu, Shenglong Wang, Nengjie Zhou, Zexing Cao and Yingkai Zhang, A Proton-Shuttle
Reaction Mechanism for Histone Deacetylase 8 and the Catalytic Role of Metal Ions. J. Am. Chem. Soc., in
press, 2010.
531. Yanzi Zhou, Shenglong Wang and Yingkai Zhang, Catalytic Reaction Mechanism of
Acetylcholinesterase Determined by Born-Oppenheimer ab initio QM/MM Molecular Dynamics Simulations,
J. Phys. Chem. B, 114, 8817 - 8825, 2010.
532. Ruibo Wu, Po Hu, Shenglong Wang, Zexing Cao and Yingkai Zhang, Flexibility of Catalytic Zinc
Coordination in Thermolysin and HDAC8: A Born-Oppenheimer ab initio QM/MM Molecular Dynamics
Study, J. Chem. Theory Comput., 6, 337-343, 2010.
533. Kamau Fahie, Po Hu, Steven Swatkoski, Robert J. Cotter, Yingkai Zhang, and Cynthia Wolberger.
Side chain specificity of ADP-ribosylation by a sirtuin. FEBS J., 276, 7159-71767, 2009.
534. Zhenyu Lu, Jonathan Lai, Yingkai Zhang. Importance of Charge Independent Effects in Readout
the Trimethyllysine Mark by HP1 Chromodomain. J. Am. Chem. Soc., 131, 14928-14931, 2009.
MCB080015
535. “Structural Determinants of Cadherin-23 Function in Hearing and Deafness”. M. Sotomayor, W. A.
Weihofen, R. Gaudet, D. P. Corey. Neuron, 66: 85-100, 2010.
536. “Calmodulin-interaction sites are critical for the thermosensitivity of a heat-activated ion channel”.
E. Procko, S.-Y. Lau, M. Sotomayor, A. Beristain-Barajas, A. McFedries, R. Gaudet. In preparation.
537. “Characterization and Structural Studies of the Plasmodium falciparum Ubiquitin and Nedd8
Hydrolase UCHL3”. K. Artavanis-Tsakonas, W. A. Weihofen, J. M. Antos, B. I. Coleman, C. Comeaux, M. T.
Duraisingh, R. Gaudet, H. L. Ploegh. Journal of Biological Chemistry, 285: 6857-6866, 2010.

MCB080093
538. J. Spiriti and A. van der Vaart, Mechanism of the calcium-induced trans-cis isomerization of a nonprolyl
peptide bond in Clostridium histolyticum collagenase, Biochemistry, 49, 5314-5320 (2010).
539. C. Wagner, P. Jurutka, P. Marshall, T. Groy, A. van der Vaart, J. Ziller, J. Furmick, M. Graeber, E.
Matro, B. Miguel, I. Tran, J. Kwon, J. Tedeschi, S. Moosavi, A. Danishyar, J. Philp, R. Khamees, J. Jackson,
D. Grupe, S. Badshah, and J. Hart, Modeling, Synthesis and biological evaluation of potential retinoid-Xreceptor
(RXR) selective agonists: Novel analogs of 4-[1-(3,5,5,8,8-pentamethyl-5,6,7,8-tetrahydro-
2naphthyl)ethynyl]benzoic acid (bexarotene), Journal of Medicinal Chemistry, 52, 59505966 (2009).
540. H. Kamberaj and A. van der Vaart, Extracting the causality of correlated motions from molecular
dynamics simulations, Biophysical Journal, 97, 1747-1755 (2009).
541. P. Maragakis, A. van der Vaart, and M. Karplus, Gaussian-mixture umbrella sampling, Journal of
Physical Chemistry B, 113, 4664-4673 (2009).
542. H. Kamberaj and A. van der Vaart, An optimized replica exchange molecular dynamics method,
Journal of Chemical Physics, 130, 074906 (2009).
543. H. Kamberaj and A. van der Vaart, Correlated motions and interactions at the onset of the DNAinduced
partial unfolding of Ets-1, Biophysical Journal, 96, 1307-1317 (2009).
MCB090134
544. J. Wereszczynski and J. Andrew McCammon “Simulations of p97 Reveal a Structure Consistent
With Crystallographic and Small-Angle X-Ray Scattering Experiments” (In Preparation)
545. J. Wereszczynski and I. Andricioaei, “Free Energy Calculations Reveal Rotating-Ratchet
Mechanism for DNA Supercoil Relaxation by Topoisomerase IB and its Inhibition” Biophys J. (In Press)
546. M. Lawrenz, J. Wereszczynski, R. Amaro, R. Walker, A. Roitberg, and J. Andrew McCammon,
“Impact of Calcium on N1 Influenza Neuraminidase Dynamics and Binding Free Energy” Proteins (In Press)
547. J. Wereszczynski and I. Andricioaei, “Conformational and Solvent Entropy Contributions to the
Thermal Response of Nucleic Acid-Based Nanothermometers” J. Phys. Chem. B 114, 2076-2082 (2010)
548. J. Wereszczynski and I. Andricioaei, “On structural transitions, thermodynamic equilibrium and the
phase diagram of DNA and RNA duplexes under external tension and torque” Proc. Natl. Acad. Sci.
MCB090159
549. G.T. Beckham, J.F. Matthews, Y.J. Bomble, Lintao Bu, W.S. Adney, M.E. Himmel, M.R. Nimlos,
M.F. Crowley, 114, 2010, “Identification of amino acids responsible for processivity for a Family 1 CBM from
a fungal cellulase”, J. Phys. Chem. B.
550. M. Alahuhta, Q. Xu, Y.J. Bomble, W.S. Adney, S.Y. Ding, M. E. Himmel, and V. V. Lunin, “The
unique binding mode of the Cellulosomal CBM4 from Clostridium thermocellum Cellobiohydrolase A”, in
press at J. Mol. Biol.
551. G.T. Beckham, Y.J. Bomble, J.F. Matthews, M.G. Resch, J.S. Yarbrough, S.R. Decker, L. Bu, X.
Zhao, C. McCabe, J. Wohlert, M. Bergenstråhle, J.W. Brady, M.E. Himmel, M.F. Crowley, in review, “The
linker peptide from the Trichoderma reesei Family 7 cellulase is a flexible, disordered tether between the
catalytic domain and carbohydrate-binding module further extended by O-linked glycosylation”
552. Y. J. Bomble, M. R. Nimlos, G. T. Beckham, J. F. Matthews, M. E. Himmel, and M. F. Crowley
“Understanding the function of the Immunoglobulin-like domain in Family 9 enzymes for C. thermocellum”, in
preparation.
553. Y. J. Bomble, G. T. Beckham, J. F. Matthews, M. R. Nimlos , M. E. Himmel, and M. F. Crowley, “A
study of cellulosomal enzyme complex self-assembly”, to be submitted to J. Biol. Chem.
554. G.T. Beckham, B. Peters, J.F. Matthews, W.S. Adney, M.E. Himmel, M.F. Crowley, “The molecularlevel
basis for biomass recalcitrance”, in preparation.

555. Lintao Bu, G.T. Beckham, M.R. Shirts, M.R. Nimlos, W.S. Adney, M.E. Himmel, and M.F. Crowley.
Potential of mean force for cellobiose expulsion from the catalytic tunnel of Cel7A, in preparation.
556. Lintao Bu, G.T. Beckham, M.R. Shirts, M.R. Nimlos, W.S. Adney, M.E. Himmel, and M.F. Crowley.
Calculating absolute binding free energy of cellobiose to the catalytic tunnel of Cel7A, in preparation.
557. Lintao Bu, M.R. Nimlos, and M.E. Himmel. Meso-scale modeling of polysaccharides in plant cell
walls: an application to carbohydrate-binding modules translation on cellulose surface. In Computational
Modeling in Lignocellulosic Biofuel Production, in press.
558. Y. J. Bomble, Q. Xu, M. E. Himmel, and M. F. Crowley “Molecular modeling of cellulosomal
systems”. In Computational Modeling in Lignocellulosic Biofuel Production, in press.
559. A.P. Hynninen, J.F. Matthews, G.T. Beckham, M.F. Crowley, M.R. Nimlos, Coarse-grained model
for glucose, cellobiose, and cellotetraose in water”, in preparation.
560. J.F. Matthews, G.T. Beckham, M.R. Nimlos, M.E. Himmel, M.F. Crowley, A new high temperature
polymorph of cellulose predicted from molecular simuation, in preparation.
561. J.F. Matthews, M.E. Himmel, M.F. Crowley, A comprehensive comparison of cellulose models for
cellulose Ib, in preparation
MCB090161
562. Spahn, C.M., and Penczek, P.A. (2009). Exploring conformational modes of macromolecular
assemblies by multiparticle cryo-EM. Curr Opin Struct Biol 19, 623-631.
MCB090163
563. P.I.: Michael F. Hagan
564. Yu, N., Pontiggia, F., Hagan M. F.: Mechanism of HIV CA-C Dimerization, manuscript in
preparation.
MCB090168
565. A.K. Ghosh, J. Takayama, K.V. Rao, K. Ratia, R. Chaudhuri, D.C. Mulhearn, H. Lee, D.B. Nichols,
S. Baliji, S.C. Baker, M.E. Johnson, and A.D. Mesecar. 2010. “Severe Acute Respiratory Syndrome-
Coronavirus Papain-Like Novel Protease Inhibitors: Design, Synthesis, Protein-Ligand X-ray Structure and
Biological Evaluation”, Journal of Medicinal Chemistry, 53(13), 4968–4979. (Teragrid support for very late
stages of modeling.)
566. R. Chaudhuri, S. Tang, G. Zhao, H. Lu, D.A. Case and M.E. Johnson. Comparative Analysis of the
papain-like protease active sites of SARS and NL63 coronaviruses: Implications for drug design. Draft
completed, in preparation for submission.
567. R. Chaudhuri, H. Lee, J. Torres, and M.E. Johnson. Identification of novel inhibitor scaffolds for the
inhibition of the papain-like protease of SARS Coronavirus using computational models. Draft completed, in
preparation for submission.
MCB090172
568. Salis, H., Tamsir, A., and Voigt, C. (2009a). Engineering bacterial signals and sensors. Contrib
Microbiol 16, 194-225.
569. Salis, H.M., Mirsky, E.A., and Voigt, C.A. (2009b). Automated design of synthetic ribosome binding
sites to control protein expression. Nat Biotechnol 27, 946-950.
570. Tabor, J.J., Salis, H.M., Simpson, Z.B., Chevalier, A.A., Levskaya, A., Marcotte, E.M., Voigt, C.A.,
and Ellington, A.D. (2009). A synthetic genetic edge detection program. Cell 137, 1272-1281.

MCB090174
571. W. Huang, J. Kim, S. Jha, and F. Aboul-ela. A mechanism for s-adenosyl methionine assisted
formation of a riboswitch conformation: A small molecule with a strong arm. Nucleic Acid Res., 37(19):6528–
6539, 2009.
572. J. Kim, W. Huang, S. Maddineni, F. Aboul-ela, and S. Jha. Exploring the RNA folding energy
landscape using scalable distributed cyberinfrastructure. ACM HPDC 2010, Emerging Computational
Methods for the Life Sciences, 2010.
573. Wei Huanga, Vamsi Boyapatia, Sunayana Mitraa, Jooyun Kimb, Shantenu Jhab, Fareed Aboulelaa,
"Bridging the Gap between “ON” state and “OFF” State of the SAM-I riboswitch", Zing conference on
Nucleic Acids, Nov 2010
574. S.H Ko, N. Kim, J. Kim, A. Thota and S. Jha, "Efficient Runtime Environment for Coupled Multi-
Physics Simulations: Dynamic Resource Allocation and Load-Balancing", CCGrid 2010
575. S. Jha H. Kim, Y. Khamra and M. Parashar. "Exploring application and infrastructure adaptaion on
hybrid grid-cloud infrastructure". ScienceCloud: Workshop on Scientific Cloud Computing, in conjunction
with ACM HPDC-2010
576. S. Jha Y. Khamra and C. White. "Modelling data-driven co2 sequestration using distributed hpc
cyberinfrastructure: A case study". TeraGrid 2010 Conference, 2010
577. S. Jha H. Kim, Y. Khamra and M. Parashar. "Autonomic approach to integrated hpc grid and cloud
usage". IEEE Conference on eScience 2009, Oxford, 2009
578. S. Jha Y. Khamra. "Modelling data-driven co2 sequestration using distributed hpc
cyberinfrastructure". Microsoft E-Science Conference, Pittsburgh PA, 2009
579. Yaakoub Youssef El-Khamra. "Real-time reservoir characterization and beyond:
Cyberinfrastructure tools and technologies". Master’s thesis, Louisiana State University, Baton Rouge,
Louisiana, 2009
MCB090176
580. J. R. Tusell and P. R. Callis, The role of tryptophan quenching in reporting folding of the villin
headpiece., manuscript in preparation for submission to Physical Chemistry Letters.
MCB990010
581. 1. Y. Luo, C.E. Ergenekan, J.T. Fischer, M.-L. Tan, & T. Ichiye, The molecular determinants of the
increased reduction potential of the rubredoxin domain rubrerythrin relative to rubredoxin. Biophys. J., 2010.
98, 560-568.
582. Y. Luo, M.-L. Tan, & T. Ichiye, Slow time scale dynamics and the intramolecular electron transfer in
ferredoxin: A molecular dynamics study. Proteins Struc. Func. Bioinf., in preparation.
583. M.-L. Tan, Y. Luo, & T. Ichiye, Tuning the intramolecular electron transfer by mutations in
ferredoxin: A molecular dynamics study. Proteins Struc. Func. Bioinf., in preparation.
584. Y. Luo & T. Ichiye, Dynamical relaxation of Clostridium acidurici ferredoxin: A molecular dynamics
study. Proteins Struc. Func. Bioinf., in preparation.
Ocean Sciences
OCE100015
585. Ozdemir, C. E., T.-J. Hsu, S. Balachandar, 2010a, Simulation of fine sediment transport in
oscillatory boundary layer, Journal of Hydro-environment Research, 3, 247-259.
586. Ozdemir, C. E., T.-J. Hsu, S. Balachandar, 2010b, A numerical investigation of fine particle laden
flow in oscillatory channel: The role of particle induced density stratification, Journal of Fluid Mechanics,
accepted.

Physics
MCA06N025
587. “Nuclear Physics from Lattice QCD” S. R. Beane, W. Detmold, K. Orginos and M. J. Savage
arXiv:1004.2935 [hep-lat] submitted to Progress in Particle and Nuclear Physics.
588. “A method to study complex systems of mesons in Lattice QCD” W. Detmold and M. J. Savage
arXiv:1001.2768 [hep-lat], accepted for publication in Phys. Rev. D.
589. “High Statistics Analysis using Anisotropic Clover Lattices: (III) Baryon- Baryon Interactions” S. R.
Beane et al. [NPLQCD Collaboration] Phys. Rev. D 81, 054505 (2010)
590. “Meson-Baryon Scattering Lengths from Mixed-Action Lattice QCD” A. Torok et al. [NPLQCD
Collaboration] Phys. Rev. D 81, 074506 (2010)
591. “High Statistics Analysis using Anisotropic Clover Lattices: (II) Three-Baryon Systems” S. R. Beane
et al. [NPLQCD Collaboration] Phys. Rev. D 80, 074501 (2009)
MCA08X037
592. “Area Invariance of Apparent Horizons under Arbitrary Boosts" Sarp Akcay, Richard A. Matzner
and Vishnu Natchu, Journal of General Relativity and gravitation 41 387-402 (2010).
593. “The Volume Inside a Black Hole", Brandon S. DiNunno, Richard A. Matzner, Journal of General
Relativity and Gravitation 42 63 (2010) DOI: 10.1007/s10714-009-0814-x (2009).
594. “Superkicks in Hyperbolic Encounters of Binary Black Holes" (James Healy, Frank Herrmann, Ian
Hinder, Deirdre M. Shoemaker, Pablo Laguna, Richard A. Matzner)Phys. Rev. Lett. 102.041101 (2009).
595. “Status of NINJA: the Numerical INJection Analysis project" (Cadonati, Laura; Aylott, Benjamin;
Baker, John G.; Boggs, William D.; Boyle, Michael; Brady, Patrick R.; Brown, Duncan A.; Brgmann, Bernd;
Buchman, Luisa T.; Buonanno, Alessandra; Camp, Jordan; Campanelli, Manuela; Centrella, Joan; Chatterji,
Shourov; Christensen, Nelson; Chu, Tony; Diener, Peter; Dorband, Nils; Etienne, Zachariah B.; Faber,
Joshua; Fairhurst, Stephen; Farr, Benjamin; Fischetti, Sebastian; Guidi, Gianluca; Goggin, Lisa M.; Hannam,
Mark; Herrmann, Frank; Hinder, Ian; Husa, Sascha; Kalogera, Vicky; Keppel, Drew; Kidder, Lawrence E.;
Kelly, Bernard J.; Krishnan, Badri; Laguna, Pablo; Lousto, Carlos O.; Mandel, Ilya; Marronetti, Pedro;
Matzner, Richard; McWilliams, Sean T.; Matthews, Keith D.; Mercer, R. Adam; Mohapatra, Satyanarayan R.
P.; Mrou, Abdul H.; Nakano, Hiroyuki; Ochsner, Evan; Pan, Yi; Pekowsky, Larne; Pfeier, Harald P.; Pollney,
Denis; Pretorius, Frans; Raymond, Vivien; Reisswig, Christian; Rezzolla, Luciano; Rinne, Oliver; Robinson,
Craig; Rver, Christian; Santamara, Luca; Sathyaprakash, Bangalore; Scheel, Mark A.; Schnetter, Erik;
Seiler, Jennifer; Shapiro, Stuart L.; Shoemaker, Deirdre; Sperhake, Ulrich; Stroeer, Alexander; Sturani,
Riccardo; Tichy, Wolfgang; Liu, Yuk Tung; van der Sluys, Marc; van Meter, James R.; Vaulin, Ruslan;
Vecchio, Alberto; Veitch, John; Vicer, Andrea; Whelan, John T.; Zlochower, Yosef) Classical and Quantum
Gravity 26 Issue 11, pp. 114008 (June 2009) IOP DOI: 10.1088/0264-9381/26/11/114008[arXiv:0905.4227 ]
(2009).
596. “Final Mass and Spin of Merged Black Holes and the Golden Black Hole" James Healy, Pablo
Laguna, Richard A. Matzner, Deirdre M. Shoemaker, Phys. Rev D81 (Rapid Communication) (No.8)
(electronic publication 8 April, 2010) DOI: 10.1103/PhysRevD.81.081501,
http://link.aps.org/doi/10.1103/PhysRevD.81.081501
MCA09X003
597. “M. Anderson, L. Lehner, M. Megevand, D. Neilsen, \Post-merger electromag-netic emissions from
disks perturbed by binary black holes," Physical Review D81, 044004 (2010).
598. “C. Palenzuela, M. Anderson, L. Lehner, S.L. Liebling, D. Neilsen, \Stirring, not shaking: binary
black holes' effects on electromagnetic felds," Physical Review Letters 103, 081101 (2009).
599. “S.L. Liebling, L. Lehner, D. Neilsen, C. Palenzuela, \Evolutions of Magnetized and Rotating
Neutron Stars," Physical Review D 81 124023 (2010). gr-qc/1001.0575

600. “C. Palenzuela, L. Lehner, S.L. Liebling, \Dual Jets From Binary Black Holes," Accepted by
Science.
601. “C. Palenzuela, T. Garrett, L. Lehner, S.L. Liebling, \Magnetospheres of Black Hole Systems in
Force-Free Plasma," gr-qc/1007.1198
602. “S. Chawla, M. Anderson, M. Besselman, L. Lehner, S.L. Liebling, P.M. Motl, D. Neilsen, \Mergers
of Magnetized Neutron Stars with Spinning Black Holes: Disruption, Accretion and Fallback," Submitted to
PRL. gr-qc/1006.2839
603. “S.L. Liebling, \Dynamics of Rotating, Magnetized Neutron Stars," Conference Proceedings of the
Twelfth Marcel Grossman Meeting on General Relativity, 2009. gr-qc/1002.2217
MCA93S002
604. The B → D∗ lν form factor at zero recoil from three-flavor lattice QCD: A model independent
determination of |Vcb|, The Fermilab Lattice and MILC Collaborations: C. Bernard, C. DeTar, M. DiPierro,
A.X. El-Khadra, R.T. Evans, E.D. Freeland, E. Gamiz, Steven Gottlieb, U.M. Heller, J.E. Hetrick, A.S.
Kronfeld, J. Laiho, L. Levkova, P.B. Mackenzie, M. Okamoto, J. Simone, R. Sugar, D. Toussaint, R.S. Van
de Water, Phys. Rev. D79, 014506 (2009) [arXiv:0808.2519 [hep-lat].
605. The B→πlν semileptonic form factor from three-flavor lattice QCD: A model-independent
determination of |Vub|, The Fermilab Lattice and MILC Collaborations: Jon A. Bailey, C. Bernard, C. DeTar,
M. Di Pierro, A. X. El-Khadra, R. T. Evans, E. D. Freeland, E. Gamiz, Steven Gottlieb, U. M. Heller, J. E.
Hetrick, A. S. Kronfeld, J. Laiho, L. Levkova, P. B. Mackenzie, M. Okamoto, J. N. Simone, R. Sugar, D.
Toussaint, R. S. Van deWater, Phys. Rev. D79, 054507 (2009) [arXiv:0811.3640 [hep-lat]].
606. Full nonperturbative QCD simulations with 2+1 flavors of improved staggered quarks, The MILC
Collaboration: A. Bazavov, C. Bernard, C. DeTar, Steven Gottlieb, U.M. Heller, J.E. Hetrick, J. Laiho, L.
Levkova, P.B. Mackenzie, M.B. Oktay, R. Sugar, D. Toussaint, and R.S. Van de Water, Rev. Mod. Phys. 82,
1349-1417 (2010) [arXiv:0903.3598 [hep-lat]].
607. Equation of state and QCD transition at finite temperature, The HotQCD Collaboration: A. Bazavov,
T. Bhattacharya, M. Cheng, N.H. Christ, C. DeTar, S. Ejiri, Steven Gottlieb, R. Gupta, U.M. Heller, K.
Huebner, C. Jung, F. Karsch, E. Laermann, L. Levkova, C. Miao, R.D. Mawhinney, P. Petreczky, C.
Schmidt, R.A. Soltz, W. Soeldner, R. Sugar, D. Toussaint, and P. Vranas, Phys. Rev. D80, 014504 (2009)
[arXiv:0903.4379 [hep-lat]].
608. The strange quark condensate in the nucleon in 2+1 flavor QCD, D. Toussaint and W. Freeman for
the MILC Collaboration, Phys. Rev. Lett. 103, 122002 (2009) [arXiv:0905.2432 [hep-lat]].
609. Visualization of semileptonic form factors from lattice QCD, The Fermilab Lattice and MILC
Collaborations: C. Bernard, C. DeTar, M. Di Pierro, A.X. El-Khadra, R.T. Evans, E.D. Freeland, E. Gamiz,
Steven Gottlieb, U.M. Heller, J.E. Hetrick, A.S. Kronfeld, J. Laiho, L. Levkova, P.B. Mackenzie, M. Okamoto,
M.B. Oktay, J.N. Simone, R. Sugar, D. Toussaint, R.S. Van de Water, Phys. Rev. D80, 034026, (2009)
[arXiv:0906.2498 [hep-lat]].
610. QCD Thermodynamics from the Lattice, C. DeTar and U. Heller, European Physical Journal A 41,
404-437 (2009) [arXiv:0905.2949 [hep-lat]]. 764. Quarkonium mass splittings in three-flavor lattice QCD, T.
Burch, C. DeTar, M. Di Pierro, A.X. El- Khadra, E.D. Freeland, Steven Gottlieb, A.S. Kronfeld, L. Levkova,
P.B. Mackenzie, J.N. Simone, Phys. Rev. D81, 034508 (2010) [arXiv:0912.2701 [hep-lat]].
611. Tuning Fermilab Heavy Quarks in 2+1 Flavor Lattice QCD with Application to Hyper fine Splittings,
The Fermilab Lattice and MILC Collaborations: C. Bernard, C. DeTar, M. Di Pierro, A.X. El-Khadra, R.T.
Evans, E.D. Freeland, E. Gmiz, Steven Gottlieb, U.M. Heller, J.E. Hetrick, A.S. Kronfeld, J. Laiho, L.
Levkova, P.B. Mackenzie, J.N. Simone, R. Sugar, D. Toussaint, R.S. Van de Water, arXiv:1003.1937 [heplat],
submitted to Phys. Rev. D.
612. QCD thermodynamics with nonzero chemical potential at Nt = 6 and effects from heavy quarks,
The MILC Collaboration: C. DeTar, L. Levkova, Steven Gottlieb, U.M. Heller, J.E. Hetrick, R. Sugar, D.
Toussaint, Phys. Rev. D81, 114504 (2010) [arXiv:1003.5682 [hep-lat]].
613. Topological susceptibility with the asqtad action, MILC collaboration: A. Bazavov, C. Bernard, B.
Billeter, C. DeTar, Steven Gottlieb, U. M. Heller, J. E. Hetrick, J. Laiho, L. Levkova, M.B. Oktay, J. Osborn,
R. L. Sugar, D. Toussaint, R. S. Van de Water, Phys. Rev. D81 114501 (2010) [arXiv:1003.5695 [hep-lat]].

614. Scaling studies of QCD with the dynamical HISQ action, MILC collaboration: A. Bazavov, C.
Bernard, C. DeTar,W. Freeman, Steven Gottlieb, U. M. Heller, J. E. Hetrick, J. Laiho, L. Levkova, M. Oktay,
J. Osborn, R.L. Sugar, D. Toussaint, R.S. Van de Water, arXiv:1004.0342 [hep-lat], submitted to Phys. Rev.
D.
615. Status and prospect for determining fB, fBs, fBs / fB on the lattice, C. Bernard, T. Blum, T.A. De-
Grand, C. DeTar, S. Gottlieb, U.M. Heller, N. Ishizuka, L. K¨ arkk¨ainen, J. Labrenz, K. Rummukainen, A.
Soni, R.L. Sugar and D. Toussaint, Proceedings of the LISHEP ’95 Session C: Heavy Flavor Physics, page
399-407, Edited by F. Caruso, M.E. Pol, A. Santoro and R. Shellard, Editions Frontieres.
616. Light Hadron Spectrum and Heavy-light Decay Constants from the MILC Collaboration, C. Bernard,
T.A. DeGrand, C. DeTar, Steven Gottlieb, Urs M. Heller, J. E. Hetrick, N. Ishizuka, C. McNeile, K.
Rummukainen, R. Sugar, D. Toussaint, and M. Wingate, Proceedings of DFP99, Katsushi Arisaka and Zvi
Bern, eds.
617. Sample size effects inmultivariate fitting of correlated data, D. Toussaint andW. Freeman,
[arXiv:0808.2211[hep-lat]].
618. Contributions of charm anihilation to the hyper fine splitting in charmonium, L. Levkova and C.
DeTar, Proceedings of Science (Lattice 2008), 133 (2009) [arXiv:0809.5086 [hep-lat]].
619. Recent Progress in Lattice QCD Thermodynamics, C. DeTar Proceedings of Science (Lattice
2008), 001 (2009) [arXiv:0811.2429 [hep-lat]].
620. 108. Heavy baryon mass spectrum from lattice QCD with 2+1 dynamical sea quark flavors, H. Na
and S.Gottlieb, Proceedings of Science (Lattice 2008), 119 (2009) [arXiv:812.1235 [hep-lat]].
621. Electromagnetic splittings of hadrons from improved staggered quarks in full QCD, The MILC
Collaboration: S. Basak, A.Bazavov, C. Bernard, C. DeTar, W. Freeman, Steven Gottlieb, U.M. Heller, J.E.
Hetrick, J. Laiho, L. Levkova, J. Osborn, R. Sugar, D. Toussaint Proceedings of Science (Lattice 2008), 127
(2009) [arXiv:0812.4486 [hep-lat]].
622. HISQ action in dynamical simulations, The MILC Collaboration: A. Bazavov, C. Bernard, C. DeTar,
W. Freeman, Steven Gottlieb, U.M. Heller, J.E. Hetrick, J. Laiho, L. Levkova, J. Osborn, R. Sugar, D.
Toussaint, Proceedings of Science (Lattice 2008), 033 (2009) [arXiv:0903.0874 [hep-lat]].
623. B and D Meson Decay Constants, The Fermilab Lattice and MILC Collaborations: C. Bernard, C.
DeTar, M. Di Pierro, A. X. El-Khadra, R. T. Evans, E. D. Freeland, E. Gamiz, Steven Gottlieb, U. M. Heller,
J. E. Hetrick, A. S. Kronfeld, J. Laiho, L. Levkova, P. B. Mackenzie, J. Simone, R. Sugar, D. Toussaint, R. S.
Van de Water, Proceedings of Science (Lattice 2008), 278 (2009) [arXiv:0904.1895 [hep-lat]].
624. QCD equation of state at non-zero chemical potential, The MILC Collaboration: S. Basak, A. Bazavov,
C. Bernard, C. DeTar,W. Freeman, U.M. Heller, J.E. Hetrick, J. Laiho, L. Levkova, J. Osborn, R. Sugar,
D. Toussaint, Proceedings of Science (Lattice 2008), 171 (2009) [arXiv:0910.0276 [hep-lat]].
625. Cell processor implementation of a MILC lattice QCD application, Guochun Shi, Volodymyr Kindratenko,
Steven Gottlieb, Proceedings of Science (Lattice 2008), 026 (2009) [arXiv:0910.0262 [hep-lat]].
15114. MILC results for light pseudoscalars, The MILC Collaboration: A. Bazavov, C. Bernard, C. DeTar, X.
Du,W. Freeman, Steven Gottlieb, UrsM. Heller, J.E. Hetrick, J. Laiho, L. Levkova, M.B. Oktay, J. Osborn, R.
Sugar, D. Toussaint, R.S. Van de Water, Proceedings of Science (Chiral Dynamics) CD09 (2009)
[arXiv:0910.2966 [hep-lat]].
626. Results from the MILC collaboration’s SU(3) chiral perturbation theory analysis, The MILC
Collaboration: A. Bazavov, C. Bernard, C. DeTar, X. Du, W. Freeman, Steven Gottlieb, Urs M. Heller, J.E.
Hetrick, J. Laiho, L. Levkova, M.B. Oktay, J. Osborn, R. Sugar, D. Toussaint, R.S. Van de Water,
Proceedings of Science (Lattice 2009) 079 (2009) [arXiv:0910.3618 [hep-lat]].
627. Quarkonium mass splittings with Fermilab heavy quarks and 2+1 flavors of improved staggered sea
quarks, T. Burch, C.E. DeTar, M. Di Pierro, A.X. El-Khadra, Steven Gottlieb, A.S. Kronfeld, L. Levkova, P.B.
Mackenzie, J. Simone (Fermilab Lattice and MILC Collaboratins), Proceedings of Science (Lattice 2009) 115
(2009) [arXiv:0911.0361].
628. SU(2) chiral fits to light pseudoscalar masses and decay constants, The MILC Collaboration: A.
Bazavov, C. Bernard, C. DeTar, X. Du, W. Freeman, Steven Gottlieb, Urs M. Heller, J.E. Hetrick, J. Laiho, L.
Levkova, M.B. Oktay, J. Osborn, R. Sugar, D. Toussaint, R.S. Van de Water, Proceedings of Science
(Lattice 2009) 077 (2009) [arXiv:0911.0472].

629. Progress on four flavor QCD with the HISQ action, MILC Collaboration: A. Bazavov, C. Bernard, C.
DeTar, W. Freeman, Steven Gottlieb, U.M. Heller, J.E. Hetrick, J. Laiho, L. Levkova, J. Osborn, R. Sugar, D.
Toussaint, R.S. Van de Water, Proceedings of Science (Lattice 2009) 123 (2009) [arXiv:0911.0869].
630. Progress on charm semileptonic form factors from 2+1 flavor lattice QCD, The Fermilab Lattice and
MILC Collaborations: Jon A. Bailey, A. Bazavov, C. Bernard, C. Bouchard, C. DeTar, A.X. El-Khadra, E.D.
Freeland, W. Freeman, E. Gamiz, Steven Gottlieb, U.M. Heller, J.E. Hetrick, A.S. Kronfeld, J. Laiho, L.
Levkova, P.B.Mackenzie, M.B. Oktay, M. Di Pierro, J.N. Simone, R. Sugar, D. Toussaint, R.S. Van de
Water, Proceedings of Science (Lattice 2009) 250 (2009) [arXiv:0912.0214].
631. The strange quark content of the nucleon in 2+1 flavor lattice QCD,Walter Freeman, Doug
Toussaint, Proceedings of Science (Lattice 2009) 137 (2009) [arXiv:0912.1144].
632. The Ds and D+ Leptonic Decay Constants from Lattice QCD, The Fermilab Lattice andMILC
Collaborations: A. Bazavov, C. Bernard, C. DeTar, E.D. Freeland, E. Gamiz, Steven Gottlieb, U.M. Heller,
J.E. Hetrick, A.X. El-Khadra, A.S. Kronfeld, J. Laiho, L. Levkova, P.B. Mackenzie, M.B. Oktay, M. Di Pierro,
J.N. Simone, R. Sugar, D. Toussaint, R.S. Van de Water, Proceedings of Science (Lattice 2009) 249 (2009)
[arXiv:0912.5221].
633. First results on QCD thermodynamics with HISQ action, A. Bazavov and P. Petreczky, Proceedings
of Science (Lattice 2009) 163 (2009) [arXiv:0912.5421 [hep-lat]].
634. Deconfinement and chiral transition with the highly improved staggered quark (HISQ) action, A.
Bazavov and P. Petreczky, arXiv:10004.0342 [hep-lat], to be published in Journal of Physics: Conference
Series.
635. Accelerating Quantum Chromodynamics Calculations with GPUs, Guochun Shi, Steven Gottlieb,
Aaron Torok, and Volodymyr Kindratenko, Accepted by 2010 Symposium on Application Accelerators in
High Performance Computing (SAAHPC), July 13–15, 2010, Knoxville TN.
636. Parallel Zero-Copy Algorithms for Fast Fourier Transform and Conjugate Gradient usingMPI
Datatypes, Torsten Hoefler and Steven Gottlieb, Accepted by EuroMPI 2010, September, 12–15, 2010,
Suttgart, Germany.
MCA99S008
637. B. D. Farris, Y. T. Liu, and S. L. Shapiro. Binary black hole mergers in gaseous environments:
\Binary Bondi" and \binary Bondi-Hoyle-Lyttleton" accretion. Phys. Rev. D. , 81(8):084008{+, April 2010.
638. Y. T. Liu, Z. B. Etienne, and S. L. Shapiro. Evolution of near-extremal-spin black holes using the
moving puncture technique. Phys. Rev. D. , 80(12):121503{+, December 2009.
MCA99S015
639. Donald Sinclair, “New results with colour-sextet quarks”,
http://agenda.infn.it/sessionDisplay.pysessionId=19&slotId=0&confId=2128#2010- 06-15
640. W. Armour, J. B. Kogut and C. Strouthos, “Chiral symmetry breaking and monopole dynamics in
non-compact QED3 coupled to a four-fermi interaction,”arXiv:1004.3053 [hep-lat].
641. D. K. Sinclair and J. B. Kogut, “Lattice Gauge Theory and (Quasi)-Conformal Technicolor,”
arXiv:1003.0439 [hep-lat].
642. J. B. Kogut and D. K. Sinclair, “Thermodynamics of lattice QCD with 2 flavours of colour-sextet
quarks: A model of walking/conformal Technicolor,” Phys. Rev. D (to be published) arXiv:1002.2988 [heplat].
643. D. K. Sinclair and J. B. Kogut, “QCD thermodynamics with colour-sextet quarks,” arXiv:0909.2019
[hep-lat].
644. C. Strouthos and J. B. Kogut, “Chiral Symmetry breaking in Three Dimensional QED,” J. Phys.
Conf. Ser. 150, 052247 (2009) [arXiv:0808.2714 [cond-mat.supr-con]].

PHY060028
645. M. Campanelli, C. O. Lousto, B. C.Mundim, H. Nakano, Y. Zlochower and H. P. Bischof, “Advances
in Simulations of Generic Black-Hole Binaries,” Class. Quant. Grav. 27, 084034 (2010) [arXiv:1001.3834 [grqc]].
646. C. O. Lousto, H. Nakano, Y. Zlochower and M. Campanelli, “Intermediate Mass Ratio Black Hole
Binaries: Numerical Relativity meets Perturbation Theory,” Phys. Rev. Lett. 104, 211101 (2010)
[arXiv:1001.2316 [gr-qc]].
647. C. O. Lousto, H. Nakano, Y. Zlochower and M. Campanelli, “Statistical studies of Spinning Black-
Hole Binaries,” Phys. Rev. D 81, 084023 (2010) [arXiv:0910.3197 [gr-qc]].
648. C. O. Lousto, M. Campanelli and Y. Zlochower, “Remnant Masses, Spins and Recoils from the
Merger of Generic Black-Hole Binaries,” Class. Quant. Grav. 27, 114006 (2010) [arXiv:0904.3541 [gr-qc]].
649. B. J. Kelly, W. Tichy, Y. Zlochower, M. Campanelli and B. F. Whiting, “Post-Newtonian Initial Data
with Waves: Progress in Evolution,” Class. Quant. Grav. 27, 114005 (2010) [arXiv:0912.5311 [gr-qc]].
650. S. C. Noble, J .H. Krolik, and J. F. Hawley “Dependence of inner accretion disk stress on
parameters: the Schwarzschild case,” Astrophys. J, 711, 959-973 (2010) [arXiv:1001.4809 [astro-ph]].
651. Antonini, Faber, Gualandris, Merritt, “Tidal Disruption of Binaries by a Supermassive Black Hole:
Progeny of the S-stars”, submitted to Astrophys. J. (2009).
652. Cadonati et al, “Status of NINJA: the Numerical INJection Analysis project”, Class. Quant. Grav. 26
114008 (2009).
653. Faber, “Status of neutron star-black hole and binary neutron star simulations”, Class. Quant. Grav.
26 114004 (2009).
654. Aylott et al, “Testing gravitational-wave searches with numerical relativity waveforms: Results from
the first Numerical INJection Analysis (NINJA) project.”, Accepted to Class. Quant. Grav. (2009).
655. Campanelli, Lousto, Zlochower, “Algebraic Classi fication of Numerical Spacetimes and Black-Hole-
Binary Remnants”, Phys. Rev. D 79 084012 (2009).
656. Noble, Krolik, “GRMHD prediction of coronal variability in accreting black holes”, Astrophys. J 703
964-975 (2009).
657. Campanelli, Lousto, Nakano, Zlochower, “Comparison of Numerical and Post-Newtonian
Waveforms for Generic Precessing Black-Hole Binaries”, Phys. Rev. D 79 084010 (2009).
658. Lousto and Zlochower, “Modeling gravitational recoil from precessing highly-spinning unequalmass
black-hole binaries”, Phys. Rev. D 79 064018 (2009).
659. Noble, Krolik, Hawley, “Direct calculation of the radiative efficiency of an accretion disk around a
black hole”, Astrophys. J 692 411-421 (2009).
PHY090002
660. A.H. Castro Neto et al., Rev. Mod. Phys. , 109 (2009); A.K. Geim, Science , 1530 (2009).
661. R. Golizadeh-Mojarad and S. Datta, Phys. Rev. B , 085410 (2009).
662. Q. Ran et al., Appl. Phys. Lett. , 103511 (2009).
663. P. Blake et al., Solid State Commun. , 1068 (2009).
664. D. B. Farmer et al., Nano Lett. , 388 (2009).
665. D. B. Farmer et al., Appl. Phys. Lett. , 213106 (2009).
SES070004
666. Bennett, D. A., W. Tang & S. Wang (Accpeted) Toward an understanding of provenance in
complex land use dynamics. Journal of Land Use Science.

667. Liu, Y., S. Wang & A. M. Segre (In preparation) A Scalable Parallel Genetic Algorithm for the
Generalized Assignment Problem. Journal of Supercomputing.
668. Shook, E., S. Wang & W. Tang (In preparation) A parallel communication framework for spatiallyexplicit
agent-based models. International Journal of Geographical Information Science.
669. Tang, W. & D. A. Bennett (2010) Agent-based modeling of animal movement: A review. Geography
Compass, 4(7), 682-700.
670. Tang, W. & D. A. Bennett (forthcoming) The Explicit representation of context in agent-based
modeling of complex adaptive spatial systems. The Annals of the Association of American Geographers.
671. Tang, W., D. A. Bennett & S. Wang (In revision) A parallel agent-based model of land use opinions.
Journal of Land Use Science.
672. Wang, S., Y. Liu, N. Wilkins-Diehr & S. Martin (2009) SimpleGrid toolkit: Enabling geosciences
gateways to cyberinfrastructure. Comput. Geosci., 35, 2283-2294.
673. Wang, S. & Y. Liu (2009) TeraGrid GIScience Gateway: Bridging cyberinfrastructure and
GIScience. International Journal of Geographical Information Science, 23(5), 631-656.
674. Wang, S. (2010) A CyberGIS Framework for the Synthesis of Cyberinfrastructure, GIS, and Spatial
Analysis. Annals of the Association of American Geographers, 100(3), 1-23.
675. Zhu, X., Wang, S., Shook, E. & Long, S. (In revision) The grain drain. Ozone, a hidden current and
mounting cause of worldwide grain shortage. Nature.
A.2.2 Publications from TeraGrid Users (for 2010 Q4)
The following 616 publications were gathered primarily from renewal Research submissions to
the December 2010 TRAC meeting. The extraction process was conducted by a UCSD
undergraduate student at SDSC. The publications are organized by field of science and by the
proposal with which they were associated.
Advanced Scientific Computing
ASC100002
1. A. Tumin, X. Wang, and X. Zhong, “Direct numerical simulation and theoretical analysis of
perturbations in hypersonic boundary layers,” Proceedings of the seventh IUTAM symposium on
laminar-turbulent transition, Stockholm, Sweden, 2009, edited by P. Schlatter and D. S. Henningson,
2010.
2. P. S. Rawat and X. Zhong, “On high-order shock-fitting and front-tracking schemes for numerical
simulation of shock–disturbance interactions,” Journal of Computational Physics, 229 (2010) 6744-
4780.
3. E. Johnsen et al. (including P. S. Rawat), “Assessment of high-resolution methods for numerical
simulations of compressible turbulence with shock waves,” Journal of Computational Physics, 229
(2010) 1213–1237.
4. A. Tumin, X. Wang, and X. Zhong, “Direct numerical simulation and theoretical analysis of perturbations
in hypersonic boundary layers,” Proceedings of the seventh IUTAM symposium on laminar-turbulent
transition, Stockholm, Sweden, 2009, edited by P. Schlatter and D. S. Henningson, 2010.
5. P. S. Rawat and X. Zhong, “On high-order shock-fitting and front-tracking schemes for numerical
simulation of shock–disturbance interactions,” Journal of Computational Physics, 229 (2010) 6744-
4780.
6. E. Johnsen et al. (including P. S. Rawat), “Assessment of high-resolution methods for numerical
simulations of compressible turbulence with shock waves,” Journal of Computational Physics, 229
(2010) 1213–1237.

Astronomical Sciences
AST060031
7. Gravitational Collapse and Filament Formation: Comparison with the Pipe Nebula Heitsch, F.,
Ballesteros-Paredes, J., Hartmann, L. 2009, ApJ, 704, 1735 AST060031
8. The Fate of High Velocity Clouds: Warm or Cold Cosmic Rain Heitsch, F., Putman, M.E. 2009, ApJ,
698, 1485 AST060031
9. Effects of Magnetic Field Strength and Orientation on Molecular Cloud Formation Heitsch, F., Stone,
J.M., Hartmann, L.W. 2009, ApJ, 695, 248 AST060031
10. The Fate of Taurus: Simmering Star Formation or Dramatic Exit Heitsch, F., Hartmann, L., ApJ
AST060031
11. Flow-Driven Cloud Formation: The Role of Magnetic Fields and Self-Gravity Heitsch, F., Stone, J.,
Hartmann, L., ApJ AST060031
AST070022
12. Choi, J. & Nagamine, K. 2009, MNRAS, 393, 1595, Effects of metal enrichment and metal cooling in
galaxy growth and cosmic star formation history
a. 2010a,MNRAS,407,1464,Effectsof cosmologicalparametersand starformationmodels on
the cosmic star formation history in ΛCDM cosmological simulations
b. 2010b, ArXiv e-prints, Multicomponent and Variable Velocity Galactic flow Out in
Cosmological Hydrodynamic Simulations
13. Nagamine, K., Choi, J., & Yajima, H. 2010, ArXiv e-prints, Effects of UV background and local stellar
radiation on the H I column density distribution
14. Niino, Y., Choi, J., Kobayashi, M. A. R., Nagamine, K., Totani, T., & Zhang, B. 2010, ArXiv e-prints,
Luminosity Distribution of Gamma-Ray Burst Host Galaxies at redshift z=1 in Cosmological Smoothed
Particle Hydrodinamic Simulations: Implications for the Metallicity Dependence of GRBs
15. Yajima, H., Choi, J., & Nagamine, K. 2010, ArXiv e-prints, Escape fraction of ionizing photons from
high-redshift galaxies in cosmological SPH simulations
AST080040
16. E. Macaulay, Hume A. Feldman, P.G. Ferreira, M. Hudson & R. Watkins, A Slight Excess of Large
Scale Power from Moments of the Peculiar Velocity Field, MNRAS submitted (ArXiv: 1010.2651).
17. Shankar Agarwal & Hume A. Feldman, The Effect of Massive Neutrinos on Matter Power Spectrum,
(2010) MNRAS in Press (ArXiv:1006.0689)
18. Hume A. Feldman, Richard Watkins & Michael J. Hudson, Cosmic Flows on 100 Mpc/h Scales:
Standardized Minimum Variance Bulk Flow, Shear and Octupole Moments, MNRAS, 407, 2328-2338
(2010) (ArXiv:0911.5516)
19. Roman Juszkiewicz, Hume A. Feldman, J. N. Fry & Andrew H. Jaffe, Weakly nonlinear dynamics and
the σ8 parameter, JCAP 02 021 (2010) (ArXiv:0901.0697)
20. Richard Watkins, Hume A. Feldman & Michael J. Hudson, Consistently Large Cosmic Flows on Scales
of 100 h -1 Mpc: a Challenge for the Standard ΛCDM Cosmology, MNRAS, 392, 743 (2009)
(ArXiv:0809.4041)
AST090007
21. Moscibrodzka, M., Gammie, C. F., Dolence, J., Shiokawa, H., Leung, P.-K. 2010, “Numerical Models of
Sgr A*,” to appear in ”The Galactic Center: A Window on the Nuclear Environment of Disk Galaxies”,
ed. Mark Morris, Daniel Q. Wang and Feng Yuan, arXiv:1002.1261
22. Guan, X., & Gammie, C. F. 2010, “Stratified Local Models of Isothermal Disks of Large Radial Extent,”
in preparation.
23. Moscibrodzka, M., Gammie, C. F., Dolence, J., Shiokawa, H., & Leung, P.-K. 2010, “Pair Production in
Low Luminosity AGN”, in preparation.
24. Dolence, J., Gammie, C. F., Moscibrodzka, M., Shiokawa, H., & Leung, P.-K. “Self-Consistent Light
Curves from 3D GR-MHD Models of Sgr A*,” in preparation.
25. Shiokawa, H., Gammie, C. F., and Dolence, J., 2011 “Convergence of Global General Relativistic
Accretion Disk Models,” in preparation.

AST090108
26. Woodward, P. R., J. Jayaraj, P.-H. Lin, P.-C. Yew, M. Knox, J. Greensky, A. Nowatzki, and K. Stoffels,
“Boosting the performance of computational fluid dynamics codes for interactive supercomputing,” Proc.
Intntl. Conf. on Comput. Sci., ICCS 2010, Amsterdam, Netherlands, May, 2010. Preprint available at
www.lcse.umn.edu/ICCS2010.
27. Herwig, F., M. Pignatari, P. R. Woodward, D. H. Porter, G. Rockefeller, C. L. Fryer, M. Bennett, and R.
12
Hirschi, “Convective-reactive proton- C combustion in Sakurai’s object (V4334 Sagittarii) and
implications for the evolution and yields from the first generations of stars,” preprint, LA-UR 10-00630,
Feb. 10, 2010, available at http://arxiv4.library.cornell.edu/pdf/1002.2241.
28. Woodward, P. R., D. H. Porter, W. Dai, T. Fuchs, A. Nowatzki, M. Knox, G. Dimonte, F. Herwig, and C.
Freyer, “The Piecewise-Parabolic Boltzmann Advection Scheme (PPB) Applied to Multifluid
Hydrodynamics,” preprint LA-UR 10-01823, March, 2010, available at www.lcse.umn.edu/PPMplusPPB.
AST100001
29. Numerical Support for the Hydrodynamic Mechanism of Pulsar Kicks
Nordhaus, J., Brandt, T. D., Burrows, A., Livne, E., Ott, C. D. 2010 – Submitted to Physical Review D --
available from arXiv:1010.0674
30. Dimension as a Key to the Neutrino Mechanism of Core-Collapse Supernova Explosions
Nordhaus, J., Burrows, A., Almgren, A., Bell, J. B. 2010 Astrophysical Journal, 720, 694
MCA08X016
31. Vasil, G. & Brummell, N.H., “Constraints on magnetic buoyancy instabilities of a shear-generated
magnetic layer”, Astrophys. J., 690, 783-794 (2009)
32. Silvers, L.J., Vasil, G.M., Brummell, N.H., & Proctor, M.R.E., “Double-diffusive instabilities of a sheargenerated
magnetic layer”, Astrophys. J., 702, L14-L18 (2009).
33. Brummell, N.H., Tobias, S.M. & Cattaneo, F., “Dynamo efficiency in compressible dynamos with and
without penetration”, Geophys. Astrophys. Fluid Dyn., accepted, to appear (2010).
34. Byington, B., Brummell, N.H. & Tobias, S.M., “The effect of small-scale motion on an essentially
nonlinear dynamo”, in Proceedings of IAU Symposium 271 Astrophysical Dynamics: From Stars to
Galaxies”, eds. N.H. Brummell, A.S. Brun. M. Miesch, accepted, to appear (2011)
35. Silvers, L.J., Vasil, G.M., Brummell, N.H. & Proctor, M.R.E., “The evolution of a double diffusive
magnetic buoyancy instability”, in Proceedings of IAU Symposium 271 Astrophysical Dynamics: From
Stars to Galaxies”, eds. N.H. Brummell, A.S. Brun. M. Miesch, accepted, to appear (2011)
36. Tobias, S.M., Cattaneo, F. & Brummell, N.H., “On the generation of organized magnetic fields”.
Astrophys. jou., submitted (2010).
37. Traxler, A., Stellmach, S., Garaud, P., Radko, T., & Brummell, N., “Dynamics of fingering convection I:
small-scale fluxes and large-scale instabilities”, Jou. Fluid Mech., submitted (2010).
38. Stellmach, S., Traxler, A., Garaud, P., Brummell, N. &Radko, T., “Dynamics of fingering convection I:
The formation of thermohaline staircases”, Jou. Fluid Mech., submitted (2010).
39. Stone, J., Byington, B., Brummell, N.H. & Gough, D.O., “Stoked dynamos” in “Proceedings of ISIMA
2010”, accepted 2010.
40. Kosuga, Y. & Brummell, N.H., “Suppression of jets in Howard-Krishnamurti convection” in “Proceedings
of ISIMA 2010”, accepted 2010.
MCA98N020
41. Arieli, Y., Rephaeli, Y., Norman, M. L. 2010. “Hydrodynamical Simulations of Galaxy Clusters with
Galcons”, ApJ, 716, 918
42. Norman, M. L. 2010. “Simulating Galaxy Clusters”, arXiv:1005.1100
43. Norman, M. L.; Reynolds, D. R.; So, G. C. 2009. “Cosmological Radiation Hydrodynamics with Enzo”,
AIP Conference Proceedings, Volume 1171, pp. 260-272
44. Norman, M. L., Paschos, P. & Harkness, R. H. 2009. “Baryon acoustic oscillations in the Lyman alpha
forest”, J. Phys. Conf. Ser., 180, 2021
45. Norman, M. L., So, G. C., Reynolds, D., Paschos, P., & Harkness, R. H. 2010. “Radiation
Hydrodynamics of Cosmic Reionization. I. The Lower Cutoff on the Dwarf Galaxy Luminosity Function”,
ApJ, submitted
46. Paschos, P., Jena, T., Tytler, D., Kirkman, D., Norman, M. L., 2009. “The Lyα forest at redshifts 0.1-1.6:
good agreement between a large hydrodynamic simulation and HST spectra”, MNRAS, 399, 1934

47. Reynolds, D. R., Hayes, J. C., Paschos, P. & Norman, M. L. 2009, “Self-consistent solution of
cosmological radiation-hydrodynamics and chemical ionization”, J. Comp. Phys., 228, 6833
48. Skory, Stephen; Turk, Matthew J.; Norman, Michael L.; Coil, Alison L., 2010. “Parallel HOP: A Scalable
Halo Finder for Massive Cosmological Data Sets”, ApJ Supp., in press, arXiv:1001.3411
49. Turk, M. J., Norman, M. L., & Abel, T. 2010. “High Entropy Polar Regions Around the First Protostars”,
ApJLett, submitted
50. Turk, M. J., Smith, B. D., Oishi, J. S., Skory, S., Skillman, S. W., Abel, T. and Norman, M. L.,2010. “yt:
An Multi-Code Analysis Toolkit for Astrophysical Simulation Data”, ApJSupp, submitted
51. Tytler, D., Paschos, P., Kirkman, D., Norman, M. L., & Jena, T. 2009. “The effect of large-scale power
on simulated spectra of the Lyα forest”, MNRAS, 393, 723
52. Wise, J. H., Norman, M. L., Abel, T., & Turk, M. J. 2010. “The Birth of a Galaxy: Population III Metal
Enrichment and Population II Stellar Populations”, ApJ, submitted
Atmospheric Sciences
EAR010001
53. Liu, S., W. Gao, M. Xu, X. Wang, and X.-Z. Liang, 2009: Regional climate model simulation of China
summer precipitation using an optimal ensemble of cumulus parameterization schemes. Frontiers of
Earth Science in China, 3(2), 248–257, DOI 10.1007/s11707-009-0022-8.
54. Beth Plale, LEAD II hybrid workflows for timely weather products, Keynote talk, 19th Pacific Rim Applications
and Grid Middleware Assembly (PRAGMA), Changchun, China, Sept 2010.
55. Beth Plale, Metadata and Provenance Collection and Representation: Antecedent to Scientific Data
Preservation, Open Data Seminar, University of Michigan, November 2010.
56. Beth Plale, LEAD II/Trident workflows for timely weather products: challenge of Vortex2, American
Chinese Cyberinfrastructure and E-science Workshop (ACCESS) on Data Intensive Sciences and
Computing (DISC), Urbana, Illinois, August 2010.
57. Beth Plale and Ashish Bhangale, LEADII: hybrid workflows in atmospheric science, DemoFest,
Microsoft Faculty Research Summit, Seattle, Washington July 2010.
58. Beth Plale, Provenance and Workflows, Computer Network Information Center of Chinese Academy of
Sciences, Beijing, China, October 2010.
59. Beth Plale, Craig Mattocks, Keith Brewster, Eran Chinthaka, Jeff Cox, Chathura Herath, Scott Jensen,
Yuan Lao, Yiming Sun, Felix Terkhorn, Ashish Bhangale, Kavitha Chandrasekar, Prashant Sabhnani,
and Robert Ping. "LEAD II : Hybrid Workflows in Atmospheric Science." Poster presented by Felix
Terkhorn at TeraGrid 2010, August 2010.
60. Eran Chinthaka Withana and Beth Plale, "Usage Patterns to Provision for Scientific Experimentation in
Clouds". 2nd IEEE International Conference on Cloud Computing and Science 2010, Indianapolis,
Indiana, Nov 30,2010. 25% acceptance.
61. Eran Chinthaka Withana and Chathura Herath, “Towards Hybrid Workflow Execution in Scientific
Workflows”, Presentation at the Supercomputing 2010 Conference, New Orleans, Louisiana, Nov 16,
2010.
62. Anderson, B.T., K. Hayhoe, and X.-Z. Liang, 2009: Anthropogenic-induced changes in the 21st Century
summertime hydroclimatology of the Northeastern US. Climatic Change, DOI 10.1007/s10584-009-
9674-3.
63. Lin, J.-T., D.J. Wuebbles, H.-C. Huang, Z.-N. Tao, M. Caughey, X.-Z. Liang, J. Zhu, and T. Holloway,
2010: Potential effects of climate and emissions changes on surface ozone in the Chicago area. J.
Great Lakes Research (in press).
64. Weaver, C.P., X.-Z. Liang, J. Zhu, P.J. Adams, P. Amar, J. Avise, M. Caughey, J. Chen, R.C. Cohen, E.
Cooter, J.P. Dawson, R. Gilliam, A. Gilliland, A.H. Goldstein, A. Grambsch, A. Guenther, W.I.
Gustafson, R.A. Harley, S. He, B. Hemming, C. Hogrefe, H.-C. Huang, S.W. Hunt, D.J. Jacob, P.L.
Kinney, K. Kunkel, J.-F. Lamarque, B. Lamb, N.K. Larkin, L.R. Leung, K.-J. Liao, J.-T. Lin, B.H. Lynn, K.
Manomaiphiboon, C. Mass, D. McKenzie, L.J. Mickley, S.M. O’Neill, C. Nolte, S.N. Pandis, P.N.
Racherla, C. Rosenzweig, A.G. Russell, E. Salathe, A.L. Steiner, E. Tagaris, Z. Tao, S. Tonse, C.
Wiedinmyer, Williams, D.A. Winner, J.-H. Woo, S. Wu, and D.J. Wuebbles, 2009: A preliminary
synthesis of modeled climate change impacts on U.S. regional ozone concentrations. Bull. Amer.
Meteorol. Soc., 90, 1843–1863.
65. Kunkel, K.E., X.-Z. Liang, and J. Zhu, 2010: Regional climate model projections of United States heat
st
waves in the 21 Century. J. Climate (accepted).
66. Wang, X., X.-Z. Liang, W. Jiang, Z. Tao, J.X.L. Wang, H. Liu, Z. Han, S. Liu, Y. Zhang, G.A. Grell, and

S.E. Peckham, 2010: WRF-Chem simulation of East Asian air quality: Sensitivity to temporal and
vertical emissions distributions. Atmos. Environ., 44, 660-669, doi:10.1016/j.atmosenv.2009.11.011.
67. Choi,H.I.,and X.-Z.Liang,2009: Improved terrestrial hydrologic representation in mesoscale land surface
models. J. Hydrometeorology (accepted).
68. Xu, M., X.-Z. Liang, W. Gao, and N. Krotkov, 2010: Comparison of TOMS retrievals and UVMRP
measurements of surface spectral UV radiation in the United States. Atmos. Chem. Phys. (submitted).
69. Liu, S., and X.-Z. Liang, 2010: Observed diurnal cycle climatology of planetary boundary layer height. J.
Climate (accepted).
70. Wang, C., X.-Z. Liang, and A.N. Samel, 2010: AMIP GCM Simulation of Precipitation Variability over the
Yangtze River Valley. J. Climate (submitted).
71. Drewry, D.T., P. Kumar, S. Long, C. Bernacchi, X.-Z. Liang, and M. Sivapalan, 2010: Ecohydrological
responses of dense canopies to environmental variability, Part 2: Role of acclimation under elevated
CO2. J. Geophys. Res.Biogeosciences (submitted).
72. Drewry, D.T., P. Kumar, S. Long, C. Bernacchi, X.-Z. Liang, and M. Sivapalan, 2010: Ecohydrological
responses of dense canopies to environmental variability, Part 1: Interplay between vertical structure
and photosynthetic pathway. J. Geophys. Res. Biogeosciences (submitted).
73. Yuan, X., and X.-Z. Liang, 2010: Evaluation of a Conjunctive Surface- Subsurface Process model
(CCSP) over the United States. J. Hydrometeorology(submitted).
74. Liang, X.-Z., M. Xu, W. Gao, K.R. Reddy, K.E. Kunkel, and D.L. Schmoldt, 2010: Physical modeling of
U.S. cotton yields and climate stresses during 1979-2005. Agronomy Journal (in revision).
75. 47. Ling, T.-J.,X.-Z. Liang, M. Xu, Z. Wang, and B. Wang, 2010: A multilevel ocean mixed-layer model
for 2-dimension applications. Acta Oceanologica Sinica (submitted).
76. 48. Hayhoe, K., C. Wake, B. Anderson, J. Bradbury, A. DeGaetano, X.-Z. Liang, J. Zhu, E. Maurer, D.
Wuebbles. 2009: Translating global change into regional trends: Climate drivers of past and future
trends in the U.S. Northeast. J. Climate (submitted).
77. 49. Markus, M., D. J. Wuebbles, X.-L. Liang, K. Hayhoe, and d. A. R. Kristovich, 2010: Diagnostic
analysis of future climate scenarios applied to urban flooding in the Chicago metropolitan area. Climatic
Change (submitted).
78. Chen, L., and X.-Z. Liang, 2010: CWRF optimized physics ensemble improving U.S. 1993 flood
prediction. In Proceedings of the 11th Annual WRF User’s Workshop, Boulder, CO, June 21-25, 4 pp.
79. Qiao, F., and X.-Z. Liang, 2010: A comparative CWRF study of the 1993 and 2008 summer U.S.
Midwest floods. In Proceedings of the 11th Annual WRF User’s Workshop, Boulder, CO, June 21-25, 4
pp.
80. Yuan, X., and X.-Z. Liang, 2010: CWRF incorporation of a conjunctive surface-subsurface process
model to improve seasonal-interannual hydroclimate forecasts. In Proceedings of the 11thAnnual WRF
User’s Workshop, Boulder, CO, June 21-25, 4 pp.
81. Liang, X.-Z., 2009: Integrated Earth System Modeling: Development and Applications. Invited seminar
at Institute of Natural Resource Sustainability, University of Illinois, February 2.
82. Liu, F., X.-Z. Liang, Z. Tao, X. Wang, J. Zhu, M. Caughey, S.M. Schindler, and S. Liu, 2009: Impact of
Low-level Jet on Regional Ozone Distributions. Invited seminar at Illinois State Water Survey, University
of Illinois, March 11.
83. Liang, X.-Z., 2009: Develop and Apply the Digital Earth System Model to Address Societal and
Environmental Issues. Invited seminar at Tongji University, Hangzhou, March 24
84. Liang, X.-Z., F. Zhang, E. Joseph and V. R. Morris, and J.X.L. Wang, 2009: CWRF Cloud-Aerosol-
Radiation Ensemble Modeling System: Validation. Oral presentation at the 10thWRF Users’ Workshop,
National Center for Atmospheric Research, Boulder, Colorado, June 23-26.
85. Liang, X.-Z., 2009: Regional Climate Model Downscaling Projection of Future Climate Changes. Invited
seminar at the Tropical Marine Science Institute, National University of Singapore, Singapore,
September 30.
86. Liang, X.-Z., X. Yuan, T. Ling, L. Chen, and J.X.L. Wang, 2009: Improving Precipitation Prediction by
the CWRF Ensemble Downscaling Approach. Oral presentation at the AGU (American Geophysical
Union) Fall Meeting, San Francisco, December 14-18.
87. Liang, X.-Z., and Feng Zhang, 2009: Development of the Cloud-Aerosol-Radiation Ensemble Modeling
System. Invited talk at the Conference for the Dynamic Earth System Model Research, Chinese
Academy of Sciences, Beijing, December 21.
88. Liang, X.-Z., 2009: Optimizing Ensemble Physics Representation to Enhance Weather Forecast Skill.
Invited talk at the Foreign Expert Forum for the GRAPES Model System Development, China
Meteorological Administration, Beijing, December 21.
89. Liang, X.-Z., 2010: Integrated Regional Earth System Modeling: Development and Application. Invited
seminar at the Desert Research Institute, Reno, January 29.
90. Liang, X.-Z., 2010: Integrated Regional Earth System Modeling: Development and Application. Invited
seminar at the Geophysical Fluid Dynamics Laboratory/NOAA, Princeton University Forrestal Campus,

February 17.
91. Liang, X.-Z., 2010: Integrated Regional Earth System Modeling: Development and Application. Invited
seminar at the Earth System Science Interdisciplinary Center jointed between the University of
Maryland and the NASA/Goddard Space Flight Center, College Park, Maryland, March 25.
92. Liang, X.-Z., 2010: Development of Improved Physics Representation for Weather Forecast and
Climate Prediction. Invited seminar at the China Meteorological Administration, Beijing, May 12.
93. Liang, X.-Z., 2010: Integrated Regional Earth System Modeling: Development and Application. Invited
seminar at the School of Resources and Environmental Sciences, East China Normal University,
Shanghai, China, May 14.
94. Liang, X.-Z., M. Xu, X. Yuan, T. Ling, H.I. Choi, F. Zhang, L. Chen, S. Liu,
95. S. Su, F. Qiao, J.X.L. Wang, K.E. Kunkel, W. Gao, E. Joseph, V. Morris, T.-W. Yu, J. Dudhia, and J.
Michalakes, 2010: Development of CWRF for regional weather and climate prediction: General model
description and basic skill evaluation—first release. Oral presentation at a planetary session of the
11thAnnual WRF User’s Workshop, Boulder, CO, June 21-25.
MCA07S033
96. Pogorelov, N.V., Borovikov, S.N., Zank, G.P., Ogino, T., Three-dimensional features of the outer
heliosphere due to coupling between the interstellar and interplanetary magnetic fields. III. The effects
of solar rotation and activity cycle, Astrophys. J., 696, 1478 (2009).
97. Florinski, V.; Balogh, A.; Jokipii, J. R.; McComas, D. J.; Opher, M.; Pogorelov, N. V.; Richardson, J. D.;
Stone, E. C.; Wood, B. E., The Dynamic Heliosphere: Outstanding Issues. Report of Working Groups 4
and 6, Space Sci. Rev., 143, 57 (2009).
98. Zank, G.P., Pogorelov, N.V., Heerikhuisen, J., et al., Space Sci. Rev., Physics of the Solar wind – local
interstellar medium interaction: Role of magnetic fields, DOI 10.1007/s11214-009-9497-6 (2009).
99. Florinski, N.V., Pogorelov, N.V., Four-dimensional transport of galactic cosmic rays in the outer
heliosphere and heliosheath, Astrophys. J., 701, 642–651 (2009).
100. Pogorelov, N.V., Heerikhuisen, J., Zank, G.P., Mitchell, J.J., Cairns, I.H., Heliospheric asymmetries due
to the action of the interstellar magnetic field, Adv. Space Res., 44, 13371344 (2009).
101. McComas, D. J., Allegrini, F., Bochsler, P.; Bzowski, M.; Christian, E. R.; Crew, G. B.; DeMajistre, R.;
Fahr, H.; Fichtner, H.; Frisch, P. C.; Funsten, H. O.; Fuselier, S. A.; Gloeckler, G.; Gruntman, M.;
Heerikhuisen, J.; Izmodenov, V.; Janzen, P.; Knappenberger, P.; Krimigis, S.; Kucharek, H.; Lee, M.;
Livadiotis, G.; Livi, S.; MacDowall, R. J.; Mitchell, D.; Möbius, E.; Moore, T.; Pogorelov, N. V.;
Reisenfeld, D.; Roelof, E.; Saul, L.; Schwadron, N. A.; Valek, P. W.; Vanderspek, R.; Wurz, P.; Zank, G.
P., Global Observations of the Interstellar Interaction from the Interstellar Boundary Explorer (IBEX),
Science, 326, Issue 5955, pp. 959962 (2009).
102. Schwadron, N. A.; Bzowski, M.; Crew, G. B.; Gruntman, M.; Fahr, H.; Fichtner, H.; Frisch, P. C.;
Funsten, H. O.; Fuselier, S.; Heerikhuisen, J.; Izmodenov, V.; Kucharek, H.; Lee, M.; Livadiotis, G.;
McComas, D. J.; Moebius, E.; Moore, T.; Mukherjee, J.; Pogorelov, N. V.; Prested, C.; Reisenfeld, D.;
Roelof, E.; Zank, G. P., Comparison of Interstellar Boundary Explorer Observations with 3D Global
Heliospheric Models, Science, 326, Issue 5955, 966968 (2009).
103. Heerikhuisen, J.; Pogorelov, N. V.; Zank, G. P.; Crew, G. B.; Frisch, P. C.; Funsten, H. O.; Janzen, P.
H.; McComas, D. J.; Reisenfeld, D. B.; Schwadron, N. A., Pick-Up Ions in the Outer Heliosheath: A
Possible Mechanism for the Interstellar Boundary EXplorer Ribbon, Astrophys. J., 708, L126-L130
(2010).
104. Zank, G. P.; Heerikhuisen, J.; Pogorelov, N. V.; Burrows, R.; McComas, D., Microstructure of the
Heliospheric Termination Shock: Implications for Energetic Neutral Atom Observations, Astrophys. J.,
708, 10921106 (2010).
105. Frisch, Priscilla C.; McComas, D. J.; Allegrini, F.; Bochsler, P.; Bzowski, M.; Christian, E. R.; Crew, G.
B.; DeMajistre, B.; Fahr, H.; Fichtner, H.; Funsten, H.; Fuselier, S. A.; Gloeckler, G.; Gruntman, M.;
Heerikhuisen, J.; Izmodenov, V.; Janzen, P.; Knappenberger, P.; Krimigis, S.; Kucharek, H.; Lee, M.;
Livadiotis, G.; Livi, S.; MacDowall, R. J.; Mitchell, D.; Moebius, E.; Moore, T.; Pogorelov, N. V.;
Reisenfeld, D.; Roelof, E.; Saul, L.; Schwadron, N. A.; Valek, P. W.; Vanderspek, R.; Wurz, P.; Zank, G.
P., First Global Observations Of The Interstellar Interaction From The Interstellar Boundary Explorer
(IBEX), American Astronomical Society, AAS Meeting #215, #415.20 (2009).
106. Borovikov, S.; Pogorelov, N. V.; Hu, Q.; Burlaga, L. F.; Zank, G. P., Numerical modeling of transient
phenomena in the distant solar wind, Eos Trans. AGU, Fall Meet. Suppl., SH21A-1499 (2009).
107. Kucharek, H.; Pogorelov, N. V.; Lee, M. A.; Moebius, E.; Funsten, H. O.; Reisenfeld, D. B.; Janzen, P.,
Pickup Ion Distributions at the Termination Shock: A Numerical Study for IBEX, Eos Trans. AGU, Fall
Meet. Suppl., SH21B-1514 (2009).

108. Frisch, P. C.; Heerikhuisen, J.; Pogorelov, N. V.; Zank, G. P.; Mueller, H.; Crew, G. B.; Demajestre, B.;
Funsten, H. O.; McComas, D. J.; Moebius, E.; Reisenfeld, D. B.; Roelof, E. C.; Schwadron, N. A.;
Slavin, J. D., Studies in the Sensitivity of IBEX to Variable Heliosphere Boundary Conditions, Eos
Trans. AGU, Fall Meet. Suppl., SH21B-1515 (2009).
109. Pogorelov, N. V.; Frisch, P. C.; Heerikhuisen, J.; Zank, G. P.; McComas, D. J.; Schwadron, N. A.;
Christian, E. R.; Crew, G. B.; Demajistre, R.; Gloeckler, G.; Gruntman, M.; Fahr, H.; Fichtner, H.;
Funsten, H. O.; Fuselier, S. A.; Krimigis, S. M.; Kucharek, H.; Lee, M. A.; Moebius, E.; Prested, C. L.;
Reisenfeld, D. B.; Roelof, E. C.; Stone, E. C.; Witte, M., Global Structure of the Heliosphere in the
Interstellar Magnetic Field, Eos Trans. AGU, Fall Meet. Suppl., SH21B-1517 (2009).
110. Pogorelov, N. V.; Suess, S. T.; Ebert, R. W.; Borovikov, S.; McComas, D. J.; Zank, G. P., Solar Cycle
Model Based on Ulysses Measurements (Invited), Eos Trans. AGU, Fall Meet. Suppl., SH24A-02
(2009).
111. Schwadron, N. A.; Bzowski, M.; Crew, G. B.; Gruntman, M.; Fahr, H.; Fichtner, H.; Frisch, P. C.;
Funsten, H. O.; Fuselier, S. A.; Heerikhuisen, J.; Izmodenov, V.; Kucharek, H.; Lee, M. A.; Livadiotis,
G.; McComas, D. J.; Moebius, E.; Moore, T. E.; Mukherjee, J.; Pogorelov, N. V.; Prested, C. L.;
Reisenfeld, D. B.; Roelof, E. C.; Zank, G. P., Comparison of Interstellar Boundary Explorer
Observations with 3-D Global Heliospheric Models, Eos Trans. AGU, Fall Meet. Suppl., SH32A-01
(2009).
112. Mueller, H.; Moebius, E.; Bzowski, M.; Pogorelov, N. V.; Heerikhuisen, J., Toward calculating
heliospheric filtration of interstellar oxygen on its way to IBEX, Eos Trans. AGU, Fall Meet. Suppl.,
SH32A-03 (2009).
113. Heerikhuisen, J.; Pogorelov, N. V.; Zank, G. P.; McComas, D. J.; Funsten, H. O.; Moebius, E.; Fuselier,
S. A.; Schwadron, N. A.; Frisch, P. C., IBEX observations in the context of a global heliospheric model,
Eos Trans. AGU, Fall Meet. Suppl., SH32A-06 (2009).
114. Zank, G. P.; Heerikhuisen, J.; Pogorelov, N. V.; Burrows, R.; McComas, D. J.; Schwadron, N. A.; Frisch,
P. C.; Funsten, H. O.; Moebius, E.; Fuselier, S. A.; Gruntman, M., Pick-up ions, the termination shock,
and energetic neutral atoms (Invited), Eos Trans. AGU, Fall Meet. Suppl., SH32A-07 (2009).
115. Borovikov, S. N., Kryukov, I. A., Pogorelov, N. V., Adaptive mesh refinement on curvilinear grids, in
Astronomical Society of the Pacific Conf. Ser. 406, Numerical Modeling of Space Plasma Flows:
ASTRONUM-2008, ed. N.V. Pogorelov, E. Audit, P. Colella, & G.P. Zank, 127 – 134, ASP, San
Francisco (2009).
116. Pogorelov, N. V., Borovikov, S. N., Florinski, V., Heerikhuisen, J., Kryukov, I. A., Zank, G. P., Multi-scale
fluid-kinetic simulation suite: A tool for efficient modeling of space plasma flows, in Astronomical Society
of the Pacific Conf. Ser. 406, Numerical Modeling of Space Plasma Flows: ASTRONUM-2008, ed. N.V.
Pogorelov, E. Audit, P. Colella, & G.P. Zank, 149 – 159, ASP, San Francisco (2009).
117. Heerikhuisen, J., Pogorelov, N. V., Florinski, V., Zank, G. P., Kharchenko, V., Kinetic modeling of
neutral atom transport in the heliosphere, in Astronomical Society of the Pacific Conf. Ser. 406,
Numerical Modeling of Space Plasma Flows: ASTRONUM-2008, ed. N.V. Pogorelov, E. Audit, P.
Colella, & G.P. Zank, 189 – 1198, ASP, San Francisco (2009).
118. Slavin, J. D.; Frisch, P. C.; Heerikhuisen, J.; Pogorelov, N. V.; Mueller, H. -R.; Reach, W. T.; Zank, G.
P.; Dasgupta, B.; Avinash, K., Exclusion of Tiny Interstellar Dust Grains from the Heliosphere,
TWELFTH INTERNATIONAL SOLAR WIND CONFERENCE. AIP Conference Proceedings, Volume
1216, pp. 497-501 (2010).
119. Pogorelov, N.V., Borovikov, S.N., Burlaga, L.F., Ebert, R.W., Heerikhuisen, J., Hu, Q., McComas, D.J.,
Suess, S.T., Zank, G.P., Transient Phenomena in the Distant SolarWind and in the Heliosheath,
TWELFTH INTERNATIONAL SOLAR WIND CONFERENCE. AIP Conference Proceedings, Volume
1216, pp. 559-562 (2010).
Biological and Critical Systems
MCA00N019
120. S. K. Easley, M. G. Jekir, A. J. Burghardt, M. Li, and T. M. Keaveny. Contribution of the intra-specimen
variations in tissue mineralization to PTH-and raloxifene-induced changes in stiffness of rat vertebrae.
Bone, 46:1162-1169, 2010.
121. A. J. Fields, G. L. Lee, X.S. Liu, M.G. Jekir, X.E. Guo, and T. M. Keaveny. Influenceof vertical
trabeculae on the compressive strength of the human vertebra. Journal of Bone and Mineral Research,
2010 Aug 16. [Epub ahead of print].
122. A. J. Fields, G. L. Lee, and T. M. Keaveny. Mechanisms of initial endplate failure in the human vertebral
body. Journal of Biomechanics, 2010 Sep 2. [Epub ahead of print].

123. J.C. Fox, A. Gupta, H.H. Bayraktar, A.L. Romens, D. Newitt, S. Majumdar and T.M.Keaveny. Role of
trabecular bone elastic anisotropy in the structural behaviour of the proximal femur. Computer Methods
in Biomechanics and Biomedical Engineering (2010) In Review.
124. S.K. Easley, M.T. Chang, D. Shindich, C.J. Hernandez and T.M. Keaveny. Biomechanical Effects of
Simulated Resorption Cavities in Trabecular Bone Across a Wide Range of Bone Volume Fraction.
Journal of Bone and Mineral Research (2010) In Review
125. A.J.Fields , G.L.Lee , X.S.Liu , M.G.Jekir, X.E.Guo and T.M.Keaveny. Vertebral compressive strength is
th
explained by the apparent density of the trabeculae that are vertically oriented. 56 Annual Meeting of
the Orthopaedic Research Society, New Orleans LA, 2010.
126. X.S.Liu, A.J.Fields, T.M.Keaveny, E.Shane and X.E.Guo. µCT/HR-pQCT Image based plate-rod
microstructural finite element model efficiently predicts the elastic moduli and yield strength of human
trabecular bone. Proceedings of the American Society of Mechanical Engineers, Naples FL, 2010.
127. S.Nawathe, A.L.Romens, A.J.Fields, B.Roberts, M.L.Bouxsein and T.M.Keaveny. Cortical-Trabecular
load sharing and high risk tissue distribution in the human proximal femur. 57
Orthopaedic Research Society, Long Beach CA, 2011.
th
Annual Meeting of the
Chemical, Thermal Systems
ASC040030
128. M. Zhou, O. Sahni, M. S. Shephard, C.D. Carothers and K.E. Jansen, “Adjacency-based Data
Reordering Algorithm for Acceleration of Finite Element Computations”, Scientific Programming,
18(2):107-123, 2010.
129. J.C. Vaccaro, Y. Elimelechi, Y. Chen, O. Sahni, M. Amitay and K.E. Jansen, “On the Interaction of
Control Jets in a Short Duct”, Proceedings of the 5th AIAA Flow Control Conference, Chicago, IL,
June/July 2010.
130. K.E. Jansen, O. Sahni, A. Ovcharenko, M.S. Shephard and M. Zhou, “Adaptive Computational Fluid
Dynamics: Petascale and Beyond”, Journal of Physics: Conference Series, (submitted July 2010).
131. M. Zhou, O. Sahni, T. Xie, M.S. Shephard and K.E. Jansen, “Unstructured Mesh Partition Improvement
for Implicit Finite Element at Extreme Scale”, Journal of Supercomputing, (submitted Feb. 2010).
132. M. Zhou, T. Xie, S. Seol, M.S. Shephard, O. Sahni and K.E. Jansen, “Tools to Support Mesh Adaptation
on Massively Parallel Computers”, Engineering with Computers, (submitted Jan. 2010).
133. O. Sahni, M. Zhou, M.S. Shephard and K.E. Jansen, “Scalable Implicit Finite Element Solver for
Massively Parallel Processing with Demonstration to 160K cores”, Proceedings of the 2009 ACM/IEEE
Conference on Supercomputing, 2009 ACM Gordon Bell Finalist, Portland, OR, Nov. 2009.
134. J.C. Vaccaro, Y. Elimelechi, Y. Chen, O. Sahni, M. Amitay and K.E. Jansen, “Experimental and
Numerical Investigation on the Flow Control of a Low Aspect Ratio Short Duct”, Journal of Fluid
Mechanics, 2010 (under preparation)
135. J.C. Vaccaro, O. Sahni, J. Olles, K.E. Jansen, and M. Amitay, “Experimental and Numerical
Investigation of Active Control of Inlet Ducts”, International Journal of Flow Control, 1(2):133-154, 2009.
136. J. Wood, O. Sahni, K.E. Jansen and M. Amitay, “Experimental and Numerical Investigation of Active
Control of 3-D Flows”, Proceedings of the 39th AIAA Fluid Dynamics Conference, San Antonio, TX,
June 2009.
137. O. Sahni, J. Olles and K.E. Jansen, “Simulation of Flow Control in a Serpentine Duct”, Proceedings of
the 47th AIAA Aerospace Sciences Meeting and Exhibit, Orlando, FL, Jan. 2009.
138. O. Sahni, J. Wood, K.E. Jansen and M. Amitay, “3-D Interactions between Finite-span Synthetic Jet and
Cross Flow at a Low Reynolds Number and Angle of Attack”, (accepted), Journal of Fluid Mechanics,
2010.
139. O. Sahni, C.D. Carothers, M.S. Shephard, and K.E. Jansen, “Strong Scaling Analysis of a Parallel,
Unstructured, Implicit Solver and the Influence of the Operating System Interference”, Scientific
Programming, 17(3):261-274, 2009.
140. M. Zhou, O. Sahni, H.J. Kim, C.A. Figueroa, C.A. Taylor, M.S. Shephard and K.E. Jansen,
“Cardiovascular Flow Simulations at Extreme Scale”, Computational Mechanics, 46(1):71-82, 2010.
141. M. Zhou, O. Sahni, K.D. Devine, M.S. Shephard and K.E. Jansen, “Controlling Unstructured Mesh
Partitions for Massively Parallel Simulations”, SIAM Journal on Scientific Computing, (accepted, under
print), 2010.
142. M. Zhou, C.A. Figueroa, H.J. Kim, C.A. Taylor, O. Sahni and K.E. Jansen, “Parallel Adaptive
Computation of Blood Flow in a 3D ‘Whole’ Body Model”, Proceedings of the 2009 ASME Summer
Bioengineering Conference, Lake Tahoe, CA, June 2009.
143. M. Rasquin, N. Mati, O. Sahni and K.E. Jansen, “Numerical Investigation of the Interaction between a

Finite-span Synthetic Jet and a Cross Flow over a Swept Wing”, 63rd Annual Meeting of the APS
Division of Fluid Dynamics, Long Beach, CA, Nov. 2010 (to be held).
144. K.E. Jansen, O. Sahni, A. Ovcharenko, M.S. Shephard and M. Zhou, “Adaptive Computational Fluid
Dynamics: Petascale and Beyond”, The SciDAC 2010 Conference, Chattanooga, TN, July 2010.
145. Onkar Sahni, Kenneth Jansen, Min Zhou and Mark S. Shephard, “Scalable Massively Parallel Implicit
Simulations of Fluid Flows to over 250,000 Processor-cores”, SIAM Conference on Parallel Processing,
Seattle, WA, Feb. 2010.
146. Min Zhou, Onkar Sahni, Karen D. Devine, Mark S. Shephard and Kenneth Jansen, “Improved
Unstructured Mesh Partitions for Parallel Simulations at Extreme Scale”, SIAM Conference on Parallel
Processing, Seattle, WA, Feb. 2010.
147. Mark S. Shephard, Aleksandr Ovcharenko, Ting Xie, Seegyoung Seol, Min Zhou, Onkar Sahni, and
Kenneth Jansen, “Unstructured Mesh Adaptation on Massively Parallel Computers”, SIAM Conference
on Parallel Processing, Seattle, WA, Feb. 2010.
148. J.C. Vaccaro, Y. Elimelechi, Y. Chen, O. Sahni, M. Amitay, and K.E. Jansen, “On the Interaction of
Control Jets in a Short Duct”, 5th AIAA Flow Control Conference, Chicago, IL, June/July 2010.
149. O. Sahni, M. Zhou, M.S. Shephard, and K.E. Jansen, “Scalable Implicit Finite Element Solver for
Massively Parallel Processing with Demonstration to 160K cores”, 2009 ACM/IEEE Conference on
Supercomputing, Gordon Bell Finalist, Portland, OR, Nov. 2009.
150. O. Sahni, K.E. Jansen and M. Amitay, “3-D Interactions of Synthetic Jets and Cross-Flows CFD”, 62nd
Annual Meeting of the APS Division of Fluid Dynamics, Minneapolis, MN, Nov. 2009.
151. K.E. Jansen, O. Sahni and M. Amitay, “Numerical Investigation of Actuators for Flow Control in Inlet
Ducts”, 62nd Annual Meeting of the APS Division of Fluid Dynamics, Minneapolis, MN, Nov. 2009.
152. O. Sahni, M. Amitay, M.S. Shephard and K.E. Jansen, “Investigation of Active Control of 3D Flows
using Adaptive Simulations”, 10th US National Congress on Computational Mechanics, Columbus, OH,
July 2009.
153. M. Zhou, O. Sahni, C.A. Figueroa, H.J. Kim, C.A. Taylor and K.E. Jansen “Parallel Adaptive
Computation of Blood Flow in a 3D ‘Whole’ Body Model”, 10th US National Congress on Computational
Mechanics, Columbus, OH, July 2009.
154. K.E. Jansen, M.S. Shephard, O. Sahni and M. Zhou, “Petascale Adaptive Implicit Computational Fluid
Dynamics”, 10th US National Congress on Computational Mechanics, Columbus, OH, July 2009.
155. M.S. Shephard, K.E. Jansen, O. Sahni, X.J. Luo, M. Zhou, A. Ovcharenko and T. Xie, “Recent
Advances and Experiences in Adaptive Mesh Generation and Control for Large-Scale Simulations”,
Keynote Talk, 10th US National Congress on Computational Mechanics, Columbus, OH, July 2009.
156. J. Wood, O. Sahni, K.E. Jansen and M. Amitay, “Experimental and Numerical Investigation of Active
Control of 3-D Flows, 39th AIAA Fluid Dynamics Conference, San Antonio, TX, June 2009.
157. K.E. Jansen, O. Sahni, M.S. Shephard and M. Zhou, “Anisotropic, Adaptive, Implicit CFD”, Science
Track Talk, TeraGrid’09 Meeting, Arlington, VA, June 2009.
158. M. Zhou, C.A. Figueroa, H.J. Kim, C.A. Taylor, O. Sahni and K.E. Jansen, “Parallel Adaptive
Computation of Blood Flow in a 3D ‘Whole’ Body Model”, 2009 ASME Summer Bioengineering
Conference, Lake Tahoe, CA, June 2009.
159. M. Zhou, O. Sahni, M.S. Shephard and K.E. Jansen, “Local Partition Modification for Improved Parallel
Finite Element Computations”, Finalist in 2nd BGCE Student Paper Prize, 2009 SIAM Conference on
Computational Science and Engineering, Miami, FL, March 2009.
160. K.E. Jansen, O. Sahni, C.D. Carothers and M.S. Shephard, “Performance Analysis of an Adaptive
Computational Fluid Dynamics Solver on Near Petascale Supercomputer Systems”, 2009 SIAM
Conference on Computational Science and Engineering, Miami, FL, March 2009.
CHE100005
161. S. Aboud, J. Wilcox, A density functional theory study of the charge state of hydrogen in metal-hydrides,
J. Phys. Chem. C, 2010.
162. B. Padak and J. Wilcox, Understanding Mercury Binding on Activated Carbon, Carbon, 2009, 47(12),
2855-2864.
163. E. Sasmaz, S. Aboud, J. Wilcox, Hg Binding on Pd Binary Alloys and Overlays, J. Phys. Chem. C,
2009, 113(18), 7813-7820.
164. J. Wilcox, A Kinetic Investigation of High-Temperature Mercury Oxidation by Chlorine, J. Phys. Chem.
A, 2009, 113, 6633-6639.
CTS030005
165. Bernard,P.S.(2009) “Vortexfilamentsimulationof theturbulentcoflowingjet,” Phys. Fluids 21, 025107.

166. Bernard,P.S.,Collins,J.P.andPotts,M.(2009) “Vortexfilamentsimulationof the turbulent boundary layer,”
AIAA Paper 2009-3547.
167. Bernard,P.S.,Collins,J.P.andPotts,M.(2010) “Vortexfilamentsimulationof the turbulent boundary layer ,”
AIAA Journal, 48, 1757 -1771.
168. Bernard,P.S.,(2010) “Vortexfurrowsinboundarylayertransitionandturbulence,” Physics of Fluids, under
review
CTS080002
169. A.Ö. Yazaydin, R.Q. Snurr, T.-H. Park, K. Koh, J. Liu, M.D. LeVan, A.I. Benin, P. Jakubczak, M.
Lanuza, D.B. Galloway, J.J. Low, R.R. Willis, “Screening of metal-organic frameworks for carbon
dioxide capture from flue gas using a combined experimental and modeling approach,” J. Am. Chem.
Soc. 131, 18198-18199 (2009).
170. X.Y. Bao, L.J. Broadbelt, R.Q. Snurr, “Elucidation of consistent enantioselectivity for a homologous
series of chiral compounds in homochiral metal-organic frameworks,” Phys. Chem. Chem. Phys. 12,
6466-6473 (2010). (Special issue on Characterization of Adsorbed Species)
171. C.E. Wilmer, R.Q. Snurr, “Towards rapid computational screening of metal-organic frameworks for
carbon dioxide capture: calculation of framework charges via charge equilibration,” Chem. Eng. J., in
press. (Special issue on CO 2 Capture)
172. G.A.E. Oxford, R.Q. Snurr, L.J. Broadbelt, “Hybrid quantum mechanics/molecular mechanics
investigation of (salen)Mn for use in metal-organic frameworks,” Ind. Eng. Chem. Res., in press
173. G.A.E. Oxford, D. Dubbeldam, L.J. Broadbelt, R.Q. Snurr, “Elucidating steric effects on
enantioselective epoxidation catalyzed by (salen)Mn in metal-organic frameworks,” J. Molec. Catal. A,
submitted.
174. X.Y. Bao, R.Q. Snurr, L.J. Broadbelt, “Insights into the complexity of chiral recognition by a threepoint
model,” Langmuir, in preparation
175. A.N. Dickey, A. Ö. Yazaydin, R.R. Willis, R.Q. Snurr, “Screening CO 2/N 2 selectivity in metal-organic
frameworks using Monte Carlo simulations,” Energy & Environmental Science, in preparation. (Invited
article)
CTS080046
176. H. Wang, S. B. Pope, D. A. Caughey. Central-difference schemes on non-uniform grids and their
applications in large-eddy simulations of turbulent jets and jet flames. Journal of Computational Physics,
(to be submitted), 2010.
177. K. A. Kemenov, H. Wang, S. B. Pope. Modeling effects of subgrid-scale mixture fraction variance in
LES of a piloted diffusion flame. Combustion Theory and Modelling, (submitted), 2010.
178. D. H. Rowinski, S. B. Pope. PDF calculations of piloted premixed flames. Combustion Theory and
Modelling, (submitted), 2010.
179. H. Wang, S. B. Pope. Large eddy simulation/probability density function modeling of a turbulent CH4
/H2 /N2 jet flame. Proceedings of the Combustion Institute (accepted), 2010.
180. K. A. Kemenov, S. B. Pope. Molecular diffusion effects in LES of a piloted methane-air flame.
Combustion and Flame (in press), doi:10.1016/j.combust flame.2010.08014, 2010.
CTS100002
181. Zheng, X., Xue. Q. Mittal R., and Bielamowicz, S, "A Coupled Sharp-Interface Immersed-
Boundary-Finite-Element Method for Flow-Structure Interaction with Application to Human
Phonation," J. Biomechanical Engineering, in press.
182. Zheng, X., Mittal, R. and Bielamowicz, S. "A Computational Study of Asymmetric Glottal Jet
Deflection During Phonation," J. Acoustical Society of America, in review.
183. Zheng, X., Mittal, R., Xue. Q. and Bielamowicz, "Direct-Numerical Simulation of the Glottal Jet
and Vocal-Fold Dynamics in a Three-Dimensional Laryngeal Model," J. Acoustical Society of
America, in review.
184. Xue, Q., Mittal, R. and Zheng, X., "A Computational Study of the Effect of Vocal-fold
Asymmetry on Phonation," J. Acoustical Society of America, Vol. 128, No. 2, 2010, pp. 818-827.
185. Seo, J. H. and Mittal, R., "A New Immersed Boundary Method for Aeroacoustic Sound

Prediction around Complex Geometries, " 40 th AIAA Fluid Dynamics Conference & Exhibit, AIAA
paper 2010-4434.
186. Seo, J. H. and Mittal, R., "A High-order Immersed Boundary Method for Acoustic Wave
Scattering and Low Mach number Flow Induced Sound in Complex Geometries, " J. of Computational
Physics, in press.
187. Zheng, L., Mittal, R., and Hedrick, T., "A Search for Optimal Wing Stokes in Flapping
Flight: Can Engineers Improve upon Nature " 28 th AIAA Applied Aerodynamics Conference, AIAA
paper 2010-4944.
CTS100003
188. Journal of Computational Physics: A 3D Parallel Adaptive Mesh Refinement Method for Fluid Structure
Interaction (in preparation)
189. Journal of Experimental Biology: Numerical investigation of the hydrodynamics of oscillatory batoid
swimming (in preparation)
190. Journal of Experimental Biology: Numerical investigation of the hydrodynamics of undulatory batoid
swimming (in preparation)
191. 63rd Annual Meeting of the APS Division of Fluid Dynamics: Numerical investigation of the 3D flow field
generated by a self-propelling manta ray
192. SICB Annual Meeting 2011: A numerical investigation of the hydrodynamic signature of batoid
swimming
CTS100012
193. Clausen, J. R., Aidun, C. K., “Capsule dynamics and rheology in shear flow: Particle pressure and
normal stress,” accepted by Physics of Fluids, Aug. 2010.
194. Aidun, C. K., Clausen, J. R., Reasor, D. A., Wu, J., “Visualizing Deformable Suspension Simulations in
ParaView,” Kitware Inc., The Source online magazine, www.kitware.com/products/thesource.html,
October 2010.
195. Reasor, D. A., Clausen, J. R, Aidun, C. K., “Coupling the lattice-Boltzmann and spectrin-link methods
for the direct numerical simulation of cellular blood flow,” under consideration for International Journal
for Numerical Methods in Fluids, June 2010.
196. Wu J., Yun B.M., Fallon A.M., Hanson S.R., Aidun C.K., Yoganathan A.P., “Numerical investigation of
the effects of channel geometry on platelet activation and blood damage,” accepted by Annals of
Biomedical Engineering, October 2010.
197. Khan, I., Aidun, C. K., “Direct numerical simulation of flow through deformable porous media using
parallel hybrid Lattice Boltzmann and Finite Element method,” under review for International Journal of
Numerical Methods in Engineering, 2010.
198. Khan, I., Aidun, C. K., “Modeling the macroscopic behavior of saturated deformable porous media using
direct numerical simulations,” under review for International Journal of Multiphase Flow, 2010.
199. Clausen, J. R., Reasor, D. A., Aidun, C. K., “The rheology and microstructure of concentrated
noncolloidal suspensions of deformable capsules,” in preparation for Journal of Fluid Mechanics, Fall
2010.
200. Wu J., Yun B.M., Simon H.A., Sotiropoulos F., Aidun C.K., Yoganathan A.P., “A numerical investigation
of blood damage in the hinge area of BMHV during mid-diastole,” in preparation for Annals of
Biomedical Engineering, 2010.
201. Yun B. M., Aidun C. K., “Lateral migration of a deformable spherical particle in Poiseuille flow,” in
preparation, 2010.
202. Reasor D. A., Clausen J. R., Aidun, C. K., “Rheological Characterization of Cellular Blood in Shear,” in
preparation, 2010.
203. Aidun, C. K., Clausen, J. R., Reasor, D. A., “Direct Numerical Simulation of Red Blood Cells in Shear
Flow,” 14 th SIAM Conference on Parallel Processing for Scientific Computing, Seattle, WA; Feb. 2010.
204. Clausen, J. R., Reasor, D. A., Wu, J., Aidun, C. K., “DNS of deformable capsules, fibers, and particles
suspended in flow,” Winner of the Outstanding Video Presentation,7 th International Conference for
Multiphase Flow, Tampa, FL, 2010.
205. Clausen, J. R., Reasor, D. A., Aidun, C. K., “The Effect of Particle Deformation on the Rheology of

Noncolloidal Suspensions,” 7 th International Conference for Multiphase Flow, Tampa, FL; May 2010.
206. Reasor, D. A., Clausen, J. R., Aidun, C. K., “Red Blood Cell Concentration Distributions in Rigid
Arteries,” 7 th International Conference for Multiphase Flow, Tampa, FL; May 2010.
207. Khan, I., Aidun, C. K., “Direct Numerical Simulation of Flow in Deformable Porous Media using a Hybrid
Lattice Boltzmann and Finite Element Method,” 7 th International Conference for Multiphase Flow,
Tampa, FL; May 2010.
208. Wu, J., Aidun, C. K., “A method for direct simulation of deformable particle suspensions using lattice–
Boltzmann equation with external boundary force,” 7 th International Conference for Multiphase Flow,
Tampa, FL; May 2010.
209. Khan, I., Aidun, C. K., “Direct Numerical Simulation of Flow in Deformable Porous Media,” 7 th
International Conference for Multiphase Flow, Tampa, FL; May 2010.
210. Wu J., Yun B. M., Fallon A. M., Simon H. A., Aidun C. K., Yoganathan A. P.: Numerical investigation of
blood damage in the hinge area of bileaflet mechanical heart valves. American Society of Mechanical
Engineers: Summer Bioengineering Conference, Naples, FL. June 2010.
211. Clausen, J. R., Reasor, D. A., Aidun, C. K., “The direct numerical simulation of dense suspensions of
deformable particles,” in preparation for the 82 nd Annual Meeting of the Society of Rheology, Santa Fe,
NM; Oct. 2010.
212. Clausen, J. R., Reasor, D. A., Aidun, C. K., “The Rheology and Microstructure of Dense Suspensions of
Elastic Capsules,” in preparation for the 63 rd Annual APS Division of Fluid Dynamics Meeting, Long
Beach, CA; Nov. 2010.
213. Reasor, D. A., Clausen, J. R., Aidun, C. K., “Rheological characterization of cellular blood via a hybrid
lattice-Boltzmann / coarse-grained spectrin-link method,” in preparation for the 63 rd Annual APS Division
of Fluid Dynamics Meeting, Long Beach, CA; Nov. 2010.
214. Yun B. M., Wu J., Simon H. A., Sotiropoulos F., Aidun C. K., Yoganathan A. P. “A numerical
investigation of blood damage in the hinge area of bilea flet mechanical heart valves,” in preparation for
the 63 rd Annual APS Division of Fluid Dynamics Meeting, Long Beach, CA. Nov. 2010.
Chemistry
CHE030089
215. Meng, L.; Wang, S. C.; Fettinger, J. C.; Kurth, M. J.; Tantillo, D. J. Eur. J. Org. Chem. 2009, 1578-1584:
"Controlling Selectivity for Cycloadditions of Nitrones and Alkenes Tethered by Benzimidazoles:
Combining Experiment and Theory"
216. Ho, G. A.; Nouri, D. H.; Tantillo, D. J. Tetrahedron Lett. 2009, 50, 1578-1581: "Carbocation
Rearrangements in Aspernomine Biosynthesis"
217. Hong, Y. J.; Tantillo, D. J. J. Am. Chem. Soc. 2009, 131, 7999-8015: "Consequences of Conformational
Preorganization in Sesquiterpene Biosynthesis. Theoretical Studies on the Formation of the Bisabolene,
Curcumene, Acoradiene, Zizaene, Cedrene, Duprezianene, and Sesquithuriferol Sesquiterpenes"
218. Lodewyk, M. W.; Kurth, M. J.; Tantillo, D. J. J. Org. Chem. 2009, 74, 4804-4811: "Mechanisms for
Formation of Diazocinones, Pyridazines, and Pyrazolines from Tetrazines - Oxyanion-Accelerated
Pericyclic Cascades"
219. Hong, Y. J.; Tantillo, D. J. Nature Chem. 2009, 1, 384-389: "A Potential Energy Surface Bifurcation in
Terpene Biosynthesis." This article was highlighted in Nature Chemical Biology (2009, 5, 614).
220. Wang, S. C.; Troast, D.; Sheridan-Conda, M.; Zuo, G.; LaGarde, D.; Louie, J.; Tantillo,
a. D. J. J. Org. Chem. 2009, 74, 7822-7833: "Mechanism of the Ni(0)-Catalyzed
Vinylcyclopropane - Cyclopentene Rearrangement"
221. Hong, Y. J.; Tantillo, D. J. Org. Biomol. Chem. 2009, 7, 4101-4109: "Modes of Inactivation of
Trichodiene Synthase by a Cyclopropane-Containing Farnesyldiphosphate Analog"
222. Wang, S. C.; Beal, P. A.; Tantillo, D. J. J. Comput. Chem. 2010, 31, 721-725: "Covalent Hydration
Energies for Purine Analogs by Quantum Chemical Methods"
223. Gribanova, T. N.; Starikov, A. G.; Minyaev, R. M.; Minkin, V. I.; Siebert, M. R.; Tantillo,
a. D. J. Chem. Eur. J. 2010, 16, 2272-2281: "Sandwich Compounds of Transition Metals with
Cyclopolyenes and Isolobal Boron Analogues"
224. Davis, R. L.; Tantillo, D. J. J. Org. Chem. 2010, 75, 1693-1700: "Dissecting a Dyotropic
Rearrangement"
225. Tantillo, D. J. Org. Lett. 2010, 12, 1164-1167: "How an Enzyme Might Accelerate an Intramolecular
Diels-Alder Reaction - Theozymes for the Formation of Salvileucalin B"

226. Hong, Y. J.; Tantillo, D. J. J. Am. Chem. Soc. 2010, 132, 5375-5386: "Formation of Beyerene, Kaurene,
Trachylobane and Atiserene Diterpenes by Rearrangements that Avoid Secondary Carbocations"
227. Tantillo, D. J. Chem. Soc. Rev. 2010, 39, 2847-2854: "The Carbocation Continuum in Terpene
Biosynthesis - Where are the Secondary Cations" invited tutorial review.
228. Siebert, M. R.; Tantillo, D. J. J. Phys. Org. Chem. 2010, in press: "Fundamental Properties of N-
Alkenylaziridines - Implications for the Design of New Reactions and Organocatalysts"
229. Hong, Y. J.; Tantillo, D. J. Chem. Sci. 2010, in press: "A Tangled Web - Interconnecting Pathways to
Amorphadiene and the Amorphene Sesquiterpenes"
230. Gutierrez, O.; Tantillo, D. J. Organometallics 2010, 29, 3541-3545: "Transition Metal Intervention for a
Classic Reaction - Assessing the Feasibility of Ni(0)-promoted [1,3] Sigmatropic Shifts of
Bicyclo[3.2.0]hept-2-enes"
231. Hong, Y. J.; Tantillo, D. J. Org. Biomol. Chem. 2010, 8, 4589-4600: "Quantum Chemical Dissection of
The Classic Terpinyl/Pinyl/Bornyl/Camphyl Cation Conundrum - The Role of Pyrophosphate in
Manipulating Pathways to Monoterpenes"
232. Siebert, M. R.; Osbourn, J. M.; Brummond, K. M.; Tantillo, D. J. J. Am. Chem. Soc. 2010, 132, 11952-
11966: "Differentiating Mechanistic Possibilities for the Thermal, Intramolecular [2 + 2] Cycloaddition of
Allene-ynes"
233. Siebert, M. R.; Yudin, A. K.; Tantillo, D. J. Eur. J. Org. Chem. 2010, in press: "On the Mechanism of the
Rh(I)-Catalyzed Rearrangement of N-Allylaziridines to (Z)-N-Alkenylaziridines"
234. Davis, R. L.; Tantillo, D. J. Curr. Org. Chem. 2010, 14, 1561-1577: "Theoretical Studies on
Pentadienyl Cation Electrocyclizations," invited review for special issue on Molecular Simulations in
Organic Chemistry.
CHE070035
235. A Computational Analysis of the Interaction between Flavin and Thiol(ate) Groups.
236. Implications for Flavoenzyme Catalysis (Submitted to Journal of Sulfur Chemistry)
237. Olga Dmitrenko* and Colin Thorpe C.W. Muller, J.J. Newby, C.P. Liu, C.P. Rodrigo and T.S.
Zwier Duschinsky mixing between four non-totally symmetric normal coordinates in the S1-S0 vibronic
structure of (E)-phenylvinylacetylene: a quantitative analysis Phys. Chem. Chem. Phys., 2010, DOI:
10.1039/b919912h
238. W.H. James III, E.E. Baquero, S.H. Choi, S.H. Gellman and T.S. Zwier Laser spectroscopy of
conformationally constrained alpha/beta-peptides : Ac-ACPC-Phe-NHMe and Ac-Phe-ACPC-NHMe J.
Phys. Chem. A., 2010, 114, 1581-1591
239. Chirantha P. Rodrigo, Christian Muller, William H. James III, Nathan R. Pillsbury and Timothy S.
Zwier Conformational Isomerization of bis-(4-HYDROXYPHENYL) Methane in a Supersonic Jet
Expansion, Part I: Low Barrier Potential Energy Surface in the S 0 State OSU International Symposium
on Molecular Spectroscopy, 2009
240. J.J. Newby, C.W. Muller, C.P. Liu and T. S. Zwier Jet-cooled vibronic spectroscopy and asymmetric
torsional potentials of phenylcyclopentene Phys. Chem. Chem. Phys., 2009, 11, 8330-8341
241. J.J. Newby, C.P. Liu, C.W. Muller and T. S. Zwier Jet-cooled vibronic spectroscopy of potential
intermediates along the pathway to PAH: phenylcyclopenta-1,3-diene Phys. Chem. Chem. Phys., 2009,
11, 8316-8329
242. John M. Bailey, Catherine E. Check and Thomas M. Gilbert Computational Studies of Pericyclic
Reactions between Aminoalanes and Ethyne, Ethene, and Dienes. A Reactive Aminoalane That Should
Prefer [2 + 2] and [4 + 2] Cyclizations to Dimerization Organometallics, 2009, 28 (3), pp 787-794
243. J. Suarez, S.C. Farantos, S. Stamatiadis and L. Lathouwers A Method for Solving the Molecular
Schrodinger Equation in Cartesian Coordinates via Angular Momentum Projection Operators Computer
Physics Communications Volume 180, Issue 11, November 2009, Pages 2025-2033
244. Nathan R. Pillsbury, Christian W. Muller and Timothy S. Zwier Conformational Isomerization and
Collisional Cooling Dynamics of Bis(2-hydroxyphenyl)methane J. Phys. Chem. A, 2009, 113 (17), pp
5013-5021
245. Nathan R. Pillsbury and Timothy S. Zwier Conformational Isomerization of 5-Phenyl-1-pentene Probed
by SEP-Population Transfer Spectroscopy J. Phys. Chem. A, 2009, 113 (1), pp 126-134
246. R. Bach Ring Strain Energy in the Cyclooctyl System. The Effect of Strain Energy on [3 + 2]
Cycloaddition Reactions with Azides J. Am. Chem. Soc., 2009, 131 (14), pp 5233-5243
247. Sylvian Cadars, Darren H. Brouwerb and Bradley F. Chmelka Probing Local Structures of Siliceous
Zeolite Frameworks by Solid-state NMR and First-principles Calculations of 29Si-O-29Si Scalar
Couplings Phys. Chem. Chem. Phys., 2009, 11, 1825-1837
248. T.A. LeGreve, W.H. James III and T.S. Zwier Solvent effects on the conformational preferences of
serotonin: serotonin-(H 2O) n, n=1,2 J. Phys. Chem. A., 2009, 113, 399-410

249. V.A. Shubert, C.W. Muller and T.S. Zwier Water's Role in Reshaping a Macrocycle's Binding Pocket:
Infrared and ultraviolet spectroscopy of benzo-15-crown-5-(H 2O) n and 4-aminobenzo-15-crown-5-
(H 2O) n, n=1, 2 J. Phys. Chem. A, 2009, 113, 8067-8079
250. V. Alvin Shubert, William H. James III and Timothy S. Zwier Jet-Cooled Electronic and Vibrational
Spectroscopy of Crown Ethers: Benzo-15-Crown-5 Ether and 4'-Amino-Benzo-15-Crown-5 Ether J.
Phys. Chem. A, 2009, 113 (28), pp 8055-8066
251. Wei Huang, Joohyun Kim, Shantenu Jha and Fareed Aboul-ela A Mechanism for S-adenosyl
Methionine Assisted Formation of a Riboswitch Conformation: a Small Molecule with a Strong
Arm Nucleic Acids Res. 2009 Oct; 37(19):6528-39
252. William H. James III, Esteban E. Baquero, V. Alvin Shubert, Soo Hyuk Choi, Samuel H. Gellman and
Timothy S. Zwier Single-Conformation and Diastereomer Specific Ultraviolet and Infrared Spectroscopy
of Model Synthetic Foldamers: α/β-Peptides J. Am. Chem. Soc., 2009, 131 (18), pp 6574-6590
253. Yong-Hui Tian and Miklos Kertesz Ladder-Type Polyenazine Based on Intramolecular S⋅⋅⋅N
Interactions: A Theoretical Study of a Small-Bandgap Polymer Macromolecules, 2009, 42 (16), pp
6123-6127
254. Yong-Hui Tian and Miklos Kertesz Molecular Actuators Designed with S∴N(sp 2 ) Hemibonds Attached to
a Conformationally Flexible Pivot Chem. Mater., 2009, 21 (10), pp 2149-2157
255. Yong-Hui Tian and Miklos Kertesz Low-Bandgap Pyrazine Polymers: Ladder-Type Connectivity by
Intramolecular S⋅⋅⋅N(sp 2 ) Interactions and Hydrogen Bonds Macromolecules, 2009, 42 (7), pp 2309-
2312
256. R. Dooley and S. Pamidighantam After the Dance: Hope for CI software
sustainability cyberinfrastructure soft sustainability and reusability Workshop, Mar 2009
257. HP-CAST meeting Krakow, Poland
CHE070060
258. “The Extraordinary Stability Imparted to Silver Monolayers by Chloride: Novel Atomically-flat Surfaces”
Erin V. Iski, Mahnaz El-Kouedi, Camilo Calderon, Feng Wang, Darin O. Bellisario, Tao Ye, E. Charles
H. Sykes, Electrochimica Acta, submitted
259. “Correcting for dispersion interaction and beyond in density functional theory through force matching”,
Yang Song, Omololu Akin-ojo and Feng Wang, J. Chem. Phys., in press
260. “Mimicking coarse-grained simulation without Coarse-graining, Enhanced Sampling through Shortrange
Damped Potential”, Dongshan Wei and Feng Wang, J. Chem. Phys, 133, 084101 (2010). [Cover
Article], [Selected for Virtual Journal of Biological Physics Research]
261. “The Quest for the Best Non-polarizable Force Field for Water from the Adaptive Force Matching
Method”, Omololu Akin-ojo, and Feng Wang, J. Comput. Chem., in press, 10.1002/jcc.21634
262. “Computational Investigation of 1-Palmitoyl-2-Oleoyl-sn-Glycero-3-Phosphocholine Lipid at Three
Hydration Levels”, Eric Pinnick, Shyamsunder Erramilli, and Feng Wang, Molecular Physics, 108, 2027
(2010)
263. “The potential of mean force of nitrous oxide in 1,2-dimyristoylphosphatidylcholine lipid bilayer” Eric
Pinnick, Shyamsunder Erramilli and Feng Wang, Chem. Phys. Lett., 489, 96 (2010)
264. “Understanding the rotational mechanism of a single molecule: STM and DFT investigations of dimethyl
sulfide molecular rotors on Au(111)”, Heather L. Tierney, Camilo E. Calderon, Ashleigh E. Baber, E.
Charles H. Sykes, and Feng Wang, J. Phys. Chem. C, 114, 3152 (2010)
CHE090047
265. “O + C 2H 4 Potential Energy Surface: Initial Excited States, Biradicals, and Lowest-lying Singlet at the
Multireference Level”, A.C. West, J.D. Lynch, B. Sellner, H. Lischka, W.L. Hase, and T.L. Windus, J.
Chem. Phys. A, submitted
266. "Fragmentation and Reactivity in Collisions of Protonated Diglycine with Chemically Modified F-SAM
Surfaces", G. L. Barnes, K. Young, and W. L. Hase, Journal of Chemical Physics, submitted.
267. "Potential Energy Surfaces of the First Three Singlet States of CH 3Cl", G. Granucci, G. Medders, and A.
M. Velasco, Chemical Physics Letters, accepted.
268. “Singlet and Triplet Potential Surfaces for the O 2 + C 2H 4 Reaction”, K. Park, A.C. West, E. Raheja, B.
Sellner, H. Lischka, T.L. Windus, and W.L. Hase, J. Chem. Phys. A, accepted
269. “O( 3 P) + C 2H 4 Potential Energy Surface: Study at the Multireference Level”, A.C. West, J.S. Kretchmer,
B. Sellner, K. Park, W.L. Hase, H. Lischka, and T.L. Windus, J. Phys. Chem. A, 2009, 113, 12663-
12674. Invited.
270. “Photodynamics simulations of thymine: relaxation into the first excited singlet state”, J.J. Szymczak, M.
Barbatti, J. Soo Hoo, J. Adkins, T.L. Windus, D. Nachtigallova, and H. Lischka, J. Phys. Chem. A, 2009,
113, 12686-12693. Invited.

CHE090095
271. Shields,* RM; Temelso, B; Archer,* KA; Morrell,* TE; Shields, GC. “Accurate Predictions of Water
Cluster Formation, (H2O)n=2-10.” J. Phys. Chem. A. (2010) In Press.
272. “Atmospheric Implications for Formation of Clusters of Ammonium and One to Ten Water Molecules”
Thomas E. Morrell* and George C. Shields, The Journal of Physical Chemistry A, 114(12), 4266–4271.
273. Temelso, B; Archer,* KA; Shields, GC. “Benchmark Quality Structures and Binding Energies of Small
Water Clusters” In preparation.
274. Morrell,* TE; Shields,* RM; Allodi,* MA; Wood,* EK; Kirschner, KN; Castonguay, TC; Temelso, B;
Archer,* KA; Shields, GC. “The Growth of Sulfuric Acid/Water Clusters in the Atmosphere”. In
preparation.
275. Temelso, B; Gauthier,* A; Barnes,* AB; Shields, GC. “Multi-level Computational Study of AFP-like
Peptides and Their Anti-breast Cancer Properties.” In preparation.
CHE100010
276. “Bonding and Isomerism in SF n-1Cl (n = 1-6): A quantum chemical study,” J. Leiding, D.E. Woon and T.
H. Dunning, Jr., J. Phys. Chem. A., submitted.
CHE110018
277. North, M. A., Bhattacharyya, S., and Truhlar, D. G. (2010) Improved density functional description of
the electrochemistry and structure-property descriptors of substituted flavins J. Phys. Chem. B (in
press).
CTS080032
278. Hansgen, D. A.; Vlachos, D. G.; Chen, J. G. Using first principles to predict bimetallic catalysts for the
ammonia decomposition reaction. Nature Chem. 2010, 2, 484-489.
279. Hansgen, D. A.; Vlachos, D. G.; Chen, J. G. Correlating ammonia decomposition activity with nitrogen
binding energy on Co-Pt, Ni-Pt, Fe-Pt and Cu-Pt bimetallic surfaces. Surf. Sci. In preparation.
280. Chen, Y.; Vlachos, D. G. Hydrogenation of Ethylene and Dehydrogenation and Hydrogenolysis of
Ethane on Pt(111) and Pt(211): A Density Functional Theory Study. J. Phys. Chem. C 2010, 114, 4973-
4982.
281. Salciccioli, M; Chen, Y.; Vlachos, D.G. DFT derived group additivity and linear scaling methods for
prediction of oxygenate stability on metal catalysts: Adsorption of open-ring alcohol and polyol
dehydrogenation intermediates on Pt based metals. J. Phys. Chem. C Accepted for publication.
282. Salciccioli, M.; Chen, Y.; Vlachos, D. G. Microkinetic Modeling and Reduced Rate Expressions of
Ethylene Hydrogenation and Ethane Hydrogenolysis on Platinum. Ind. Eng. Chem. Res. In Press.
283. Salciccioli, M.; Chen, Y.; Vlachos, D. G. DFT studies of transition states of ethylene glycol thermal
decomposition on platinum based catalysts. J. Phys. Chem. C In preparation.
Earth Sciences
EAR090003
284. Hartzell, S., L.Ramirez-Guzman, L. Carver, and P-C Liu (2010). Short Baseline Variations in Site
Response and Wave Propagation Effects and Their Structural Causes: Four Examples in and
around the Santa Clara Valley, California, Bull Seismol. Soc. Am. Vol 100 5A.
285. Ramirez-Guzman, L., O. Boyd, S. Hartzell, and R. Williams (2010). Large Scale Earthquake
Simulations in the Central United States, SCEC Annual Meeting, Palm Springs, CA, USA,
September 11–15.
286. Ramirez-Guzman, L., O. Boyd, S. Hartzell, and R. Williams (2010). Large Scale Earthquake Ground

Motion Simulations in the Central United States, Geological Society of America Annual Meeting,
Denver, CO, USA, 31 October – 3 November.
287. Ramirez-Guzman, L., O. Boyd, S. Hartzell, and R. Williams. Central United States Velocity Model
Version 1: Description and Validation Tests, in preparation.
Materials Research
DMR010001
288. E. Cockayne and A. van de Walle. Building effective models from scarce but accurate data: Application
to an alloy cluster expansion model. Phys. Rev. B, 81:012104, 2010.
289. P. Dalach, D. E. Ellis, and A. van de Walle. First principles thermodynamic modeling of atomic ordering
in yttria-stabilized zirconia. Phys. Rev. B, 2010. Accepted.
290. B. Meredig, A. Thompson, H.A. Hansen, C. Wolverton, and A. van de Walle. A method for locating lowenergy
solutions within DFT+U. 2010. submitted.
291. G.S. Pomrehn,E. S.Toberer,G.J.Snyder, andA.van de Walle. Entropic stabilization and
retrograde solubility in Zn4Sb3. 2010. submitted.
292. C.Ravi, A. vandeWalle, B.K.Panigrahi, H.K.Sahu, andM.C.Valsakumar. Cluster expansion-monte carlo
study of phase stability of vanadium nitrides. Phys. Rev. B, 81:104111, 2010.
293. P. Tiwary, A. van de Walle, and N. Gronbech Jensen. Ab initio construction of interatomic potentials for
uranium dioxide across all interatomic distances. Phys. Rev. B, 80:174302, 2009.
294. P. Tiwary, A. van de Walle, B. Jeon, and N. Gronbech-Jensen. Interatomic potentials for mixed
oxide (MOX) nuclear fuels. 2010. submitted.
DMR050036
295. R. G. Hennig, A. Wadehra, K. P. Driver, W. D. Parker, C. J. Umrigar, and J. W. Wilkins. Phase
transformation in Si from semiconducting diamond to metallic β-Sn phase in QMC and DFT under
hydrostatic and anisotropic stress. Phys. Rev. B, 82(1):014101, Jul 2010.
296. S. Alireza Ghasemi, Maximilian Amsler, Richard G. Hennig, Shantanu Roy, Stefan Goedecker,
Thomas J. Lenosky, C. J. Umrigar, Luigi Genovese, Tetsuya Morishita, and Kengo Nishio. Energy
landscape of silicon systems and its description by force fields, tight binding schemes, density functional
methods, and quantum Monte Carlo methods. Phys. Rev. B, 81(21):214107, Jun 2010.
297. W. D. Parker, J. W. Wilkins, and R. G. Hennig. Accuracy of quantum Monte Carlo methods for point
defects in solids. Phys. Stat. Sol., 2010. (in print, http:
//onlinelibrary.wiley.com/doi/10.1002/pssb.201046149/abstract).
298. Justin L. Luria, Kathleen A. Schwarz, Michael J. Jaquith, Richard G. Hennig, and John A.
Marohn. Spectroscopic characterization of charged defects in polycrystalline pentacene by time-and
wavelength-resolved electric force microscopy. Advanced Materials, 2010. (in print).
299. D. R. Trinkle K. Thornton and R. G. Hennig. Applying for computational time on NSF’s TeraGrid–
the worlds largest cyberinfrastructure supporting open research. JOM, 62(3):17–18, 2010.
DMR070003
300. G. Ghosh, “First-principles Calculation of Phase Stability and Cohesive Properties of Ni-Sn
Intermetallics”, Metall Mater Trans A, 40A, p 4 (2009).
DMR080044
301. T. Sadowski and R. Ramprasad, "First Principles Computational Study of Wurtzite CdTe
Nanowires", J. Mater. Sci. 45, 5463 (2010).
302. S. K. Yadav, T. Sadowski and R. Ramprasad, "Density functional theory study of ZnX (X=O, S, Se,Te)
under uniaxial strain", Phys. Rev. B 81, 144120 (2010).
303. C. Tang and R. Ramprasad, "Point defect chemistry in amorphous HfO2: Density functional theory
calculations", Phys. Rev. B 81, 161201(R) (2010).
304. T. Sadowski and R. Ramprasad, "Core/Shell CdSe/CdTe Heterostructure Nanowires Under
AxialStrain", J. Phys. Chem. C 114, 1173 (2010).
305. M. E. Stournara and R. Ramprasad, "A first principles investigatioin of isotactic polypropylene", J.

Mater. Sci. 45, 443 (2010).
306. H. Zhu, M. Aindow, and R. Ramprasd, “Stability and work function of TiCxN1-x alloy surfaces:
Density functional theory calculations”, Phys. Rev. B 80, 201406(R) (2009).
307. G. Pilania, D. Q. Tan, Y. Cao, V. S. Venkataramani, Q. Chen and R. Ramprasad, "Ab initio study of
antiferroelectric PbZrO3 (001) surfaces", J. Mater. Sci. 44, 5249 (2009).
308. G. Pilania, S. P. Alpay and R. Ramprasad, “Ab initio study of ferroelectricity in BaTiO3 nanowires”,
Phys. Rev. B 80, 014113 (2009).
309. G. Pilania, T. Sadowski and R. Ramprasad, “Oxygen adsorption on CdSe surfaces: Case study of
asymmetric anisotropic growth through ab initio computations”, J. Phys. Chem. C. 113(5), 1863 (2009).
310. G. Pilania and R. Ramprasad, "Adsorption of atomic oxygen on cubic PbTiO3 and LaMnO3 (001)
surfaces: A density functional theory study", Surf. Sci. (in print).
311. C. C. Wang and R. Ramprasad, “Dielectric properties of organosilicons from first principles”, J.Mater.
Sci. (in print).
312. G. Pilania and R. Ramprasad, "Complex Polarization Ordering in PbTiO3 Nanowires: A First Principles
Computational Study ", submitted to Phys. Rev. B.
313. H. Zhu, C. Tang and R. Ramprasad, “Phase equilibria at Si-HfO2 and Pt-HfO2 interfaces from first
principles thermodynamics”, submitted to Phys. Rev. B.
314. H. Zhu and R. Ramprasad, “Determination of the effective work function of metals interfacedwith
dielectrics using first principles thermodynamics: A case study of the Pt-HfO2 interface”, in preparation.
315. H. Zhu and R. Ramprasd, “Work function of TaCxN1-x alloys”, in preparation.
316. S. Yadav and R. Ramprasad, “ZnX nanowire (X=O, S, Se, Te) under uniaxial strain”, in
preparation.
317. Y. Cardona-Quintero, H. Zhu and R. Ramprasad. "Effect of SCH3 and SCF3 molecules in the work
function of Pt metal", in preparation.
318. S. K. Yadav, G. Pilania, and R. Ramprasad, “GGA, LDA, and LDA+U phase diagram of the ZnO(10-10)
surface in equilibrium with oxygen and hydrogen.” in preparation.
319. P. Shimpi, S. K. Yadav, R. Ramprasad, P. Gao, “C Assisted ZnO Nanowire Growth From
Nanofilm Along Non-polar Direction.” in preparation.
DMR080045
320. A. Y. Liu, “Electron-phonon coupling in compressed 1T-TaS2: Stability and superconductivity from first
principles,” Phys. Rev. B 79, 220515(R) (2009).
321. J. K. Freericks, H. R. Krishnamurthy, Y. Ge, A. Y. Liu, and Th. Pruschke, “Theoretical description of
time-resolved pump/probe photoemission in TaS2: a single-band DFT+DMFT(NRG) study within the
quasiequilibrium approximation,” Phys. Status Solidi B 246, 948 (2009).
322. Y. Ge and A. Y. Liu, “First-principles investigation of the charge-density-wave instability in 1T-TaSe2,”
Phys. Rev. B, to appear.
DMR090006
323. J. A. Yasi, T. Nogaret, D. R. Trinkle, Y. Qi, L. G. Hector Jr., andW. A. Curtin. “Basal and prism
dislocation cores in magnesium: comparison of first-principles and embeddedatom-potential methods
predictions.” Modelling Simul. Mater. Sci. Eng. 17, 055012 (2009).
324. T. Nogaret,W. A. Curtin. J. A. Yasi, L. G. Hector Jr., and D. R. Trinkle. “Atomistic study of edge and
screw hc+ai dislocations in magnesium.” ActaMater. 58, 4332-4343 (2010). 3. J. A. Yasi, L. G. Hector
Jr., and D. R. Trinkle. “First-principles data for solid-solution strengthening of magnesium: From
geometry and chemistry to properties.” Acta Mat. 58, 5704–5713 (2010).
325. H.M. Lawler and D. R. Trinkle, “First-principles calculation of H vibrational excitation at a dislocation
core of Pd.” (in review), arXiv:1008.1256 (2010).
326. M. Yu and D. R. Trinkle, “Geometry and energetics of Au/TiO2 interface from firstprinciples.” (in
preparation).
327. H. H. Wu and D. R. Trinkle, “Mechanism of oxygen diffusion in alpha-titanium.” (in preparation).
DMR100005
328. Yufeng Liang, Li Yang, “Anisotropic Adsorption of Hydrogen Atoms on Strained Graphene”, in
preparing.
329. Li Yang, “First-principles study of the optical absorption spectra of electrically gated bilayer graphene”,
Phys. Rev. B 81, 155445 (2010).
330. Li Yang, “Novel Excitons in Graphene and Bilayer Graphene”, submitted to Phys. Rev. Lett.
MCA08X038

331. L.K. Wagner and J.C. Grossman. Phys. Rev. Lett. 104, 210201 (2010).
332. J.-H. Lee, J. Wu and J. C. Grossman,Phys. Rev. Lett., 104, 16602 (2010)
333. Z. Wu, M.D. Allendorf and J.C. Grossman, J. Am. Chem. Soc., 131, 13198 (2009).
MCA08X040
334. B.J. Demaske, V.V. Zhakhovsky, N.A. Inogamov, and I.I. Oleynik, “Ablation and spallation of gold films
irradiated by ultrashort laser pulse”, Physical Review B 82 064113 (2010).
335. M.M. Budzevich, A. Landerville, M.W. Conroy, I.I. Oleynik, and C.T. White, “Anisotropic constitutive
relationships in 1,3,5-triamino-2,4,6-trinitrobenzene (TATB), Journal of Applied Physics 107, 096205
(2010).
336. J. Lahiri, Y. Lin, P. Bozkurt, I.I. Oleynik, and M. Batzill, “An extended defect in graphene as a metallic
wire”, Nature Nanotechnology, 5, 326 (2010).
337. A.C. Landerville, M.W. Conroy, I.I. Oleynik, and C.T. White, “Prediction of isothermal equation of state
of an explosive nitrate ester by van der Waals density functional theory” J. Physical Chemistry Letters 1,
346, (2010).
338. L. Adamska, M.A. Kozhushner, and I.I. Oleynik, “Electron-plasmon interactions in resonant molecular
tunnel junctions”, Physical Review B 81, 035404 (2010).
339. I. Dıez-Perez, J. Hihath, Y. Lee, L. Yu, L. Adamska, M.A. Kozhushner, I.I. Oleynik, and N. Tao,
“Rectification and stability of a single molecular diode with controlled orientation”, Nature Chemistry, 1,
635 (2009).
340. M.W. Conroy, I.I. Oleynik, S.V. Zybin, and C.T. White, “Density functional theory calculations of solid
nitromethane under hydrostatic and uniaxial compressions with empirical van der Waals corrections”,
Journal of Physical Chemistry A 113, 3610 (2009).
341. A.C. Landerville, I.I. Oleynik, and C.T. White, “Reactive molecular dynamics in detonating PETN”,
Journal of Physical Chemistry A 113, 12094 (2009).
342. R. Perriot, X. Gu, and I.I. Oleynik, Computational Nanomechanics of Graphene Membranes, Mater.
Res. Soc. Symp. Proc. 1185, II05-04 (2009).
343. A. Landerville, I.I. Oleynik, and C.T. White, “First-principles investigation of reactive molecular dynamics
in detonating RDX and TATB”, AIP Conference proceedings “Shock Compression of Condensed Matter
-2009” 1195, 817 (2009).
344. M. Budzevich, M. Conroy, A. Landerville, Y. Lin , I.I. Oleynik, and C.T. White, “Hydrostatic equation of
state and anisotropic constitutive relationships in 1,3,5-triamino-2,4,6-trinitrobenzene (TATB)”, AIP
Conference proceedings “Shock Compression of Condensed Matter -2009” 1195, 545 (2009).
345. M. Conroy, M. Budzevich, A. Landerville, Y. Lin , I.I. Oleynik, and C.T. White, “Application of van der
waals density functional theory to study physical properties of energetic materials”, AIP Conference
proceedings “Shock Compression of Condensed Matter -2009” 1195, 805 (2009).
346. A. Landerville, I.I. Oleynik, and C.T. White, “Reactive molecular dynamics of detonating PETN”, AIP
Conference proceedings “Shock Compression of Condensed Matter -2009” 1195, 813 (2009).
347. M. Conroy, A. Landerville, I.I. Oleynik, and C.T. White, “First-principles studies of hydrostatic and
uniaxial compression of a new energetic material – an energetic nitrate ester”, AIP 2 Conference
proceedings “Shock Compression of Condensed Matter -2009” 1195, 482 (2009).
348. X. Gu, Y. Lin, I.I. Oleynik, and C.T. White, “Molecular dynamics simulations of shockinduced defect
healing in silicon”, AIP Conference proceedings “Shock Compression of Condensed Matter -2009”
1195, 793 (2009).
349. Y. Lin, M. Budzevich, A. Landerville, I.I. Oleynik, and C.T. White, “Physical and chemical properties of a
new energetic material Si-PETN”, AIP Conference proceedings “Shock Compression of Condensed
Matter -2009” 1195, 482 (2009).
Mathematical Sciences
CHE090098
350. Wang, Junmei.; Cieplak, P.; Jie, L.; Wang, J.; Luo, R.; Duan, D. Development and test a set of dipole
interaction models, in preparation.
351. Wang, Junmei, Hou T.; Molecular properties prediction using molecular dynamics simulations 1 –
density and heat of vaporization, in preparation.
352. Wang, Junmei, Hou T.; Molecular properties prediction using molecular dynamics simulations 2 –
diffusion coefficient, in preparation.
CTS070004
353. S. Dong & X. Zheng. Direct numerical simulation of spiral turbulence. Journal of Fluid Mechanics, in
press, 2010.

354. S. Dong & J. Shen. An unconditionally stable rotational velocity-correction scheme for incompressible
flows. Journal of Computational Physics, 229, 7013{7029, 2010.
355. S. Dong. BDF-like methods for nonlinear dynamic analysis. Journal of Computational Physics, 229,
3019{3045, 2010.
356. K. Yeo, S. Dong, E. Climent & M.R. Maxey. Modulation of homogeneous turbulence seeded with finite
size bubbles or particles. International Journal of Multiphase Flow, 36, 221{233, 2010.
357. S. Dong & X. Zheng. Spiral turbulence in a Taylor-Couette geometry. AIAA Paper 2010-1257. 48th
AIAA Aerospace Sciences Meeting, Orlando, FL, Jan. 2010.
358. S. Dong. Blended time integration algorithms for nonlinear dynamics. AIAA Paper 2010-2890. 51st
AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Orlando, FL,
Apr. 2010.
359. S. Dong. A time-stepping scheme for flow simulations that allows the use of large time step sizes. 63rd
Annual Meeting of APS Division of Fluid Dynamics, Long Beach, CA, Nov. 2010.
360. X. Zheng & S. Dong. An eigen-based high order expansion basis for spectral elements. SIAM
Conference on Computational Science and Engineering, Reno, NV, March 2011.
DMS090001
361. Sengupta, K., Direct and Large-eddy Simulation of Compressible Flows with Spectral/hp Element
Methods, Ph.D. Thesis, University of Illinois at Chicago, Chicago, IL, 2009.
362. Kanchi, H., Sengupta, K., Jacobs, G. B., and Mashayek, F., Large-eddy Simulation of Compressible
Flow over Backward-facing Step Using Chebyshev Multidomain Method, AIAA 2010-922, 2010.
363. Kanchi, H., RANS and LES Study of Fluidic Control of Shear Layer in Dump Combustors using
Microjets, Ph.D. Thesis, University of Illinois at Chicago, Chicago, IL (in preparation)
DMS100004
364. Masud A, Calderer R (2010) A variational multiscale method for incompressible turbulent flows: residual
free bubbles and the fine scale fields. Submitted for publication to Computer Methods in Applied
Mechanics and Engineering.
365. The flow around a cylinder problem will be featured on the back cover of the Fall 2010 issue of Access,
the NCSA’s quarterly magazine.
366. Ferguson Lecture 2010: "Stabilized Methods and Turbulence: Application to Fluid-Structure Interaction."
University of Texas at Austin. Department of Civil and Environmental Engineering. May 5, 2010
367. "Stabilized Finite Element Methods and Turbulence." University of Tokyo, Department of Mechanical
Engineering, Tokyo, Japan. June 2, 2010.
368. "Stabilized Finite Element Methods and Turbulence." Purdue University. Department of Computer
Science. March 11, 2010.
Mechanical and Structural Systems
MSS110004
369. Song J, Medhekar N. Novel Scaffolding Structure for Li-ion Battery anodes. Submitted
370. Song J, Medhekar N. Thermal Properties of Impurities and Defects Decorated Graphene Nanoribbons.
In preparation.
371. Song J, Curtin WA. A Nanoscale Mechanism of Hydrogen Embrittlement in Metals. Submitted. (partially
supported by TG-MSS100005)
Molecular Biosciences
CHE060063
372. Durrant, J., Hall, L., Swift, R., Landon, M., Schnaufer, A., and Amaro, R. (2010) “Novel Naphthalene-
Based Inhibitors of Trypanosoma brucei RNA Editing Ligase 1,” PLoS Neglected Tropical Diseases,
4(8): e803. (doi:10.1371/journal.pntd.0000803).
373. Durrant, J., Amaro, R. E., Xie, L., Urbaniak, M. D., Ferguson, M. A. J, Haapalainen, A., Chen, Z., Di
Guilmi, A. M., Wunder, F., Bourne, P. E., McCammon, J. A. (2010). A Multidimensional Strategy to
Detect Polypharmacological Targets in the Absence of Structural and Sequence Homology. PLoS
Computational Biology, 6(1), e1000648.
374. Amaro, R. E., Li, W. W. (2010). Emerging Ensemble-based Methods in Virtual Screening. Current
Topics in Medicinal Chemistry, 10(1), 3-13.
375. Swift, R. V., Amaro, R. E. (2009). Discovery and design of DNA and RNA ligase inhibitors in infectious
microorganisms. Expert Opinion in Drug Discovery, 4(12), 1281-1294.
376. Wong, S., Amaro, R. E., McCammon, J. A. (2009). MM-PBSA captures key role of intercalating water
molecules at a protein-protein interface. Journal of Chemical Theory and Computation, 5(2), 422-429.

377. Lawrenz, M., Wereszczynski, J., Amaro, R., Walker, R., Roitberg, A., and McCammon, J.A., “Impact of
calcium on N1 influenza neuraminidase dynamics and binding free energy,” Proteins, 78: 2523-2532
(2010).
378. Sung, J.C., Van Wynsberghe, A.W., Amaro, R. E., Li, W. W., and McCammon, J. A., “The role of
secondary sialic acid binding sites in influenza N1 neuraminidase,” Journal of the American Chemical
Society Communication, 132(9): 2883 - 2885 (2010).
379. Newhouse, E.I., Xu, D., Markwick, P.R., Amaro, R.E., Pao, H.C., Wu, K.,J., Alam, M., McCammon, J.A.,
Arzberger, P.W., and Li, W.W., “Mechanism of Glycan Receptor Recognition and Specificity Switch for
Avian, Swine, and Human Adapted Influenza Virus Hemagglutinins: A Molecular Dynamics
Perspective,” Journal of the American Society, 131(47): 17430-42 (2009).
MCA01S027
380. In Suk Joung, Ö Persil Çetinkol, NV Hud, and TE Cheatham, III. “Molecular dynamics simulations and
coupled nucleotide substitution experiments indicate the nature of A•A base pairing and a putative
structure of the coralyne-induced homo-adenine duplex.” Nuc. Acids Res. 37, 7715-7727 (2009).
PMCID: 2794157
381. R Lavery, K Zakrzewska, DL Beveridge, TC Bishop, DA Case, TE Cheatham, III, S Dixit, B Jayaram, F
Lankas, C Laughton, JH Maddocks, A Michon, R Osman, M Orozco, A Perez, T Singh, N Spackova,
and J Sponer. “A systematic molecular dynamics study of nearest-neighbor effects on base pair and
base pair step conformations and fluctuations in B-DNA”. Nuc. Acids Res. 38, 299-313 (2010). PMCID:
2800215
382. T Truong, H Freedman, L Le, TE Cheatham, III, J Tuszynski, and L Huynh. “Explicitly-solvated ligand
contribution to continuum solvation models for binding free energies: Selectivity of theophylline binding
to an RNA aptamer.” J. Phys. Chem. B 114, 2227-2237 (2010).
383. RB Paulsen, PP Seth, EE Swayze, RH Griffey, JJ Skalicky, TE Cheatham, III, and DR Davis. “Inhibitor
induced structure change in the HCV IRES domain IIa RNA.” Proc. Natl. Acad. Sci. 107, 7263-7268
(2010).
384. CD Moore, K Shahrokh, SF Sontum, TE Cheatham, III, and GS Yost. “Improved Cyp3A4 molecular
models accurately predict Phe215 requirement for raloxifene dehydrogentation selectivity.” Biochemistry
ASAP, DOI: 10.1021/bi101139q (2010).
385. R DeMille, TE Cheatham, III, and V Molinero. “A coarse-grained model of DNA with explicit solvation by
water and ions.” J. Phys. Chem. B [in press]
386. P Banas, D Hollas, M Zgarbova, P Jurecka, M Orozco, TE Cheatham, III, J Sponer, and M Otyepka.
“Performance of molecular mechanics force fields for RNA simulations. Stability of UUCG and GNRA
hairpins.” J. Chem. Theory Comp. [in press]
387. SS Pendley, and TE Cheatham, III. “Assessing model structures of small, parallel IAAL-E3/K3 dimeric
coiled-coils with the AMBER force fields.” J. Chem. Theory Comp. [in revision]
388. SS Pendley and TE Cheatham, III. “Explicit solvent effects and parameterization corrections in MM-
PBSA calculations.” J. Chem. Theory. Comp. [in revision]
389. X Cang, J Sponer, and TE Cheatham, III. “Explaining the varied glycosidic conformational, G-tract
length, and sequence preferences for anti-parallel G-quadruplexes.” Nuc. Acids Res. [in minor revision]
390. AS Dixon, SS Pendley, BJ Bruno, DW Woessner, AA Shimpi, TE Cheatham, III and CS Lim. “Disruption
of Bcr-Abl coiled-coil oligomerization by design.” J. Biol. Chem. [in revision]
391. K Shahrokh, A Orendt, G Yost, and TE Cheatham, III. “Quantum mechanically derived AMBERcompatible
heme parameters for the states on the cytochrome P450 catalytic cycle.” J. Phys. Chem. B
[submitted]
392. X Cang, J Sponer, and TE Cheatham, III. “Insight into telomere structural polymorphism and G-DNA
folding from sequence and loop connectivity through free energy analysis.” J. Amer. Chem. Soc.
[submitted]
393. M Zgarbova, M Otyepka, J Sponer, A Mladak, P Banas, TE Cheatham, III, M Orozco, F Javier Luque,
and P Jurecka. “Refinement of the AMBER nucleic acid force field based on quantum chemical
calculations of glycosidic torsion profiles. [near final revision to be submitted soon, likely to JCTC]
MCA02N028
394. A partial nudged elastic band implementation for use with large or explicitly solvated systems Campbell,
A.J., Walker, R.C., Simmerling, C.. Intl J. of Quantum Chemistry, 2009. 109, 3781-3790.
395. An Improved Reaction Coordinate for Nucleic Acid Base Flipping Studies, Song, K., Campbell, A.J.,
Bergonzo, C., de los Santos, C., Grollman, A.P., Simmerling, C. J. Chem. Theory & Computation, 2009.
396. Recent Advances in the Study of the Bioactive Conformation of Taxol, Sun, L., Simmerling, C. and
Ojima, I., ChemMedChem, 4, 719 (2009)

397. Evaluating the Performance of the FF99SB Force Field Based on NMR Scalar Coupling Data
Wickstrom, L., Okur, A. and Simmerling, C., “”, Biophys. J., 97, 853 (2009)
398. Dynamic Behavior of DNA Duplexes Suggests a Mechanism for Selective Destabilization of Lesions by
a DNA Glycosylase, Bergonzo, C., Song, K., de los Santos, C., Grollman, A., Simmerling, C. (in
revision: Nucleic Acids Research, 2010)
399. Slow onset inhibition of bacterial & β-ketoacyl-ACP synthases by thiolactomycin, Machutta, C., Gopal,
B., Luckner, S., Kapilashrami, K., Ruzsicska, B., Simmerling, C., Kisker, C., and Tonge, P., J. Biol.
Chem. 285, 6161 (2010)
400. Three-dimensional molecular theory of solvation coupled with molecular dynamics in Amber Luchko, T.,
Gusarov, S., Roe, D., Simmerling, C., Case, D., Tuszynski, J. and Kovalenko, A., J. Chem. Theory &
Comput, 6, 607 (2010).
401. Synthesis and Molecular Modeling of a Nitrogen Mustard DNA Interstrand Crosslink Guainazzi, A.,
Campbell, A., Angelov, T., Simmerling, C. and Schärer, O., “”, Chemistry: A European Journal, in press.
402. Mechanism of Glycosylase Discrimination of Oxidatively Damaged DNA, Arthur J. Campbell, Christina
Bergonzo, Lin Fu, Haoquan Li, Dmitry O. Zharkov, Carlos de los Santos, Arthur Grollman and Carlos
Simmerling, submitted
403. Improving the Description of Salt Bridge Strength and Geometry in a Generalized Born Model, Y.
Shang, H. Nguyen, L. Wickstrom, A. Okur and C. Simmerling, J. Mol. Graphics & Modelling, submitted
404. Role of the Glycosylase Wedge in Base Excision Repair of Oxidative DNA Damage, Campbell, A.,
Bergonzo, C., Fu, L., de los Santos, C., Grollman, A., Zharkov, D. and Simmerling, C., submitted
405. ff10SB: Optimization of protein backbone and sidechain dihedrail parameters from the ff99SB force
field. Martinez, C., Maier, J., Wickstrom, L. and Simmerling, C., in preparation
406. Enhanced Sampling In Multidimensional Replica Exchange Umbrella Sampling, Lin, F., Bergonzo, C.,
C., Grollman, A. and Simmerling, C., in preparation
407. Translocation of Repair Proteins along DNA during Lesion Search, Carvalho, A., Bergonzo, C. and
Simmerling, C., in preparation
408. Mapping Conformational Changes During Base Excision Repair, Bergonzo, C., Campbell, A.J., Fu, L.,
Song, K., de los Santos, C., Grollman, A., Simmerling, C. in preparation
409. Role of the Glycosylase Active Site Loop in Early Recognition of Oxidative DNA Damage. Lin, F.,
Campbell, A., Bergonzo, C., de los Santos, C., Grollman, A. and Simmerling, C., in preparation
410. Changes in Dimer Interface Dynamics Result in Modulation of Drug Sensitivity between HIV-1 Protease
Subtypes, Shang, M., Roitberg, A., Fanucci, G. and Simmerling, C., in preparation
411. Structural Basis of Slow Onset Inhibition Mechanism in Mycobacterium tuberculosis InhA. Lai, C., Li, H.,
Liu, N., Garcia-Diaz, M., Tonge, P. and Simmerling, C., in preparation
412. The Transition between the Closed and Semi-open Form of HIV-1 Protease through Swapping of the
Flap Tip Hydrophobic Cluster, F. Ding and C. Simmerling, in preparation.
413. Flap opening mechanism of HIV-PR explored by microsecond-scale MD simulations: Role of the Dimer
Interface, F. Ding and C. Simmerling, in preparation.
414. Improved Parameters for the Generalized Born Neck Model, Nguyen, H., Roe, D., Shang, Y. and
Simmerling, C. in preparation
MCA05S028
415. B. Luan, R. Carr, M. Caffrey and A. Aksimentiev. The effect of calcium on the conformation of
cobalamin transporter BtuB. Proteins: Structure, Function, and Bioinformatics 78:1153–1162 (2010).
416. B. Luan and A. Aksimentiev. Electric and electrophoretic inversion of the DNA charge in multivalent
electrolytes. Soft Matter 6, 243-246 (2010).
417. A. Aksimentiev. Deciphering ionic current signatures of DNA transport through a nanopore. Nanoscale
2, 468–483 (2010). Cover.
418. W. Timp, U. M. Mirsaidov, D. Wang, J. Comer, A. Aksimentiev, and G. Timp. Nanopore sequencing:
Electrical measurements of the code of life. IEEE Transactions on Nanotechnology 9:281-294 (2010).
Cover.
419. D. Wells, A. Aksimentiev. Mechanical properties of a complete microtubule revealed through molecular
dynamics simulation. Biophysical Journal 99:629-637 (2010).
420. B. Luan and A. Aksimentiev. Control and reversal of the electrophoretic force on DNA in a charged
nanopore. Journal of Physics: Condensed Matter, in press (2010).
421. R. Carr, J. Comer, M. Ginsberg and A. Aksimentiev. Modeling pressure-driven transport of proteins
through a nanochannel. IEEE Transactions on Nanotechnology 99, doi:10.1109/TNANO.2010.2062530
(2010).
422. U. Mirsaidov, J. Comer, V. Dimitrov, A. Aksimentiev and G. Timp. Slowing the translocation of doublestranded
DNA using a nanopore smaller than the double helix. Nanotechnology 21:395501-10 (2010).

423. C. Maffeo, R. Schopflin, H. Brutzer, R. Stehr, A. Aksimentiev, G. Wedemann and R. Seidel. A small
effective charge describes DNA-DNA interactions in tight supercoils. Physical Review Letters
105:158101 (2010).
424. C. Stavis, T. L. Clare, J. E. Butler, A. D. Radadia, R Carr, H. Zeng, W. King, J. A. Carlisle, A.
Aksimentiev, R. Bashir and R.J. Hamers. Surface functionalization of thin-film diamond for highly stable
and selective biological interfaces. Proceedings of the National Academy of Sciences USA,
doi:10.1073/pnas.1006660107 (2010).
425. A. D. Radadia, C. J. Stavis, R. Carr, H. Zeng, W. King, J. A. Carlisle, A. Aksimentiev, R. J. Hamers, R.
Bashir. Ultrananocrystalline diamond thin films as stable antibody tethering surfaces for bacterial
capture. Submitted.
426. S. Bhattacharya, J. Muzard, L. Payet, J. Mathe, U. Bockelmann, A. Aksimentiev, V. Viasnoff. Molecular
origin of cation dependent current rectification in large proteinaceous channels: the case of alphahemolysin.
Submitted.
427. M. Wanunu, S. Bhattacharya, Y. Xie, Y. Tor, A. Aksimentiev, and M. Drndic Nanopore analysis of
individual RNA/antibiotic complexes. Submitted.
428. M. Venkatesan, J. Comer, A. Aksimentiev and R. Bashir. Lipid bilayer coated Al2O3 nanopore sensors:
Towards a hydrid biological solid-state nanopore. Submitted.
429. S. W. Kowalczyk, D. B. Wells, A. Aksimentiev, and C. Dekker. Charge reduction of DNA by transient
binding of counter ions. Submitted.
430. C Maffeo, B Luan and A. Aksimentiev. End-to-end attraction of duplex DNA. Submitted.
431. R. Carr, J. Comer, M Ginzberg, and A. Aksimentiev. Transport of small solutes through a sticky
nanochannel. In preparation.
432. S. Bhattacharya, A. Ho and A. Aksimentiev. Engineering membrane channel MspA for sequencing
DNA. In preparation.
433. J. Comer and A. Aksimentiev. Sequencing double stranded DNA by measuring ionic current through
solid-state nanopores. In preparation.
434. D. Wang, J. W. Shim, J. Comer, A. Ho, W. Timp, A. Aksimentiev, G. Timp. Using Measurements of the
Ion Current through a Synthetic Nanopore to Discriminate Nucleotides in a single DNA molecule. In
preparation.
MCA07S015
435. Lambert, R.A., P. O’Shaughnessy, M. H. Tawhai, E. A. Hoffman, and C.-L. Lin, “Regional deposition of
particles in an image-based airway model: large-eddy simulation and left-right lung ventilation
asymmetry,” Aerosol Science & Technology, 45:1-15, 2011.
436. Tawhai, M. H. and C.-L. Lin, “Airway Gas Flow,” Comprehensive Physiology, Wiley-Blackwell, in press,
2010.
437. Tawhai, M. H. and C.-L. Lin, “Image-based modeling of lung structure and function,” Journal of
Magnetic Resonance Imaging, in press, 2010.
438. Choi, J., G. Xia, M.H. Tawhai, E.A. Hoffman, and C.-L. Lin, “Numerical study of high frequency
oscillatory air flow and convective mixing in a CT-based human airway model,” DOI: 10.1007/s10439-
010-0110-7, Annals of Biomedical Engineering, 2010.
439. Yin, Y., J. Choi, E.A. Hoffman, M.H. Tawhai, and C.-L. Lin, “Simulation of pulmonary air flow with a
subject-specific boundary condition,” Journal of Biomechanics, 43(11):2159-2163, 2010.
440. Xia, G., M. H. Tawhai, E. A. Hoffman, and C.-L. Lin, “Airway Wall Stiffness and Peak Wall Shear Stress:
A Fluid-Structure Interaction Study in Rigid and Compliant Airways,” Annals of Biomedical Engineering,
38(5), 1836-1853, 2010.
441. Lin, C.-L., M. H. Tawhai, G. McLennan, and E.A. Hoffman, “Multiscale Simulation of Gas Flow in
Subject-Specific Models of the Human Lung,” IEEE Eng. in Medicine and Biology, 28(3): 25-33, 2009.
442. Choi, J., M.H. Tawhai, E.A. Hoffman, and C.-L. Lin, “On intra- and inter-subject variabilities of airflow in
the human lungs,” Phys. Fluids, 21, 101901, 2009.
MCA99S007
443. Vila, JA; Serrano, P; Wuthrich, K; Scheraga, HA. Sequential nearest-neighbor effects on computed C-
13(alpha) chemical shifts. JOURNAL OF BIOMOLECULAR NMR, 48 23-30, 2010 DOI:
10.1007/s10858-010-9435-7
444. Maisuradze, GG; Liwo, A; Oldziej, S; Scheraga, HA. Evidence, from Simulations, of a Single State with
Residual Native Structure at the Thermal Denaturation Midpoint of a Small Globular Protein. JOURNAL
OF THE AMERICAN CHEMICAL SOCIETY, 132 9444-9452 2010 DOI: 10.1021/ja1031503
445. Kozlowska, U; Liwo, A; Scheraga, HA. Determination of Side-Chain-Rotamer and Side-Chain and
Backbone Virtual-Bond-Stretching Potentials of Mean Force from AM1 Energy Surfaces of Terminally-

Blocked Amino-Acid Residues, for Coarse-Grained Simulations of Protein Structure and Folding I: The
Method. JOURNAL OF COMPUTATIONAL CHEMISTRY, 31 1143-1153 2010 DOI: 10.1002/jcc.21399
446. Kozlowska, U; Maisuradze, GG; Liwo, A; Scheraga, HA. Determination of Side-Chain-Rotamer and
Side-Chain and Backbone Virtual-Bond-Stretching Potentials of Mean Force from AM1 Energy Surfaces
of Terminally-Blocked Amino-Acid Residues, for Coarse-Grained Simulations of Protein Structure and
Folding II: Results, Comparison with Statistical Potentials, and Implementation in the UNRES Force
Field. JOURNAL OF COMPUTATIONAL CHEMISTRY, 31 1154-1167 2010 DOI: 10.1002/jcc.21402
447. Martin, OA; Villegas, ME; Vila, JA; Scheraga, HA. Analysis of C-13(alpha) and C-13(beta) chemical
shifts of cysteine and cystine residues in proteins: a quantum chemical approach. JOURNAL OF
BIOMOLECULAR NMR, 46 217-225 2010 DOI: 10.1007/s10858-010-9396-x
448. Liwo, A; Oldziej, S; Czaplewski, C; Kleinerman, DS; Blood, P; Scheraga, HA. Implementation of
Molecular Dynamics and Its Extensions with the Coarse-Grained UNRES Force Field on Massively
Parallel Systems: Toward Millisecond-Scale Simulations of Protein Structure, Dynamics, and
Thermodynamics. JOURNAL OF CHEMICAL THEORY AND COMPUTATION, 6 890-909 2010 DOI:
10.1021/ct9004068
449. Maisuradze, GG; Liwo, A; Scheraga, HA. Relation between Free Energy Landscapes of Proteins and
Dynamics. JOURNAL OF CHEMICAL THEORY AND COMPUTATION, 6 583-595 2010 DOI:
10.1021/ct9005745
450. Makowski, M; Czaplewski, C; Liwo, A; Scheraga, HA. Potential of Mean Force of Association of Large
Hydrophobic Particles: Toward the Nanoscale Limit. JOURNAL OF PHYSICAL CHEMISTRY B, 114
993-1003 2010 DOI: 10.1021/jp907794h
451. Vila, JA; Scheraga, HA. Assessing the Accuracy of Protein Structures by Quantum Mechanical
Computations of C-13(alpha) Chemical Shifts. ACCOUNTS OF CHEMICAL RESEARCH, 42 1545-1553
2009 DOI: 10.1021/ar900068s
452. Vila, JA; Arnautova, YA; Martin, OA; Scheraga, HA. Quantum-mechanics-derived C-13(alpha) chemical
shift server (CheShift) for protein structure validation. PROCEEDINGS OF THE NATIONAL ACADEMY
OF SCIENCES OF THE UNITED STATES OF AMERICA, 106 16972-16977 2009 DOI:
10.1073/pnas.0908833106
453. Sobolewski, E; Makowski, M; Oldziej, S; Czaplewski, C; Liwo, A; Scheraga, HA. Towards temperaturedependent
coarse-grained potentials of side-chain interactions for protein folding simulations. I:
Molecular dynamics study of a pair of methane molecules in water at various temperatures. PROTEIN
ENGINEERING DESIGN & SELECTION, 22 547-552 2009 DOI: 10.1093/protein/gzp028
454. He, Y; Xiao, Y; Liwo, A; Scheraga, HA. Exploring the Parameter Space of the Coarse-Grained UNRES
Force Field by Random Search: Selecting a Transferable Medium-Resolution Force Field. JOURNAL
OF COMPUTATIONAL CHEMISTRY, 30 2127-2135 2009 DOI: 10.1002/jcc.21215
MCB080011
455. Sodium release triggers galactose exit from the sodium-galactose transporter, vSGLT. Watanabe, A.S.
Choe, V. Chaptal, J.M. Rosenberg, E.M. Wright, M. Grabe, and J. Abramson Nature (in press)
456. Water permeation through the sodium-dependent galactose cotransporter vSGLT. Choe, S., J.M.
Rosenberg, J. Abramson, E.M. Wright, and M. Grabe Biophys. J. 99: L56-L58.
MCB080014
457. G. Reddy, J.E. Straub and D. Thirumalai, “Dry amyloid fibril assembly in a yeast prion peptide is
mediated by long-lived structures containing water wires”, Proc. Natl. Acad. Sci. (in review).
458. M.S. Li, N.T. Co, G. Reddy, C.K. Hu and D. Thirumalai, “Determination of factors governing
fibrillogenesis using lattice models”, Phys. Rev. Lett. (in review).
MCB090005
459. Genchev, G. Z., M. Kallberg, G. Gursoy, A. Mittal, L. Dubey, O. Perisic, G. Feng, R. Langlois, and H.
Lu. 2009. Mechanical signaling on the single protein level studied using steered molecular dynamics.
Cell Biochem Biophys 55:141-152
MCB090110
460. Goetz, A.W., Woelfe, T., Walker, R.C., "Quantum Chemistry on Graphics Processing Units", Ann. Rev.
Comp. Chem., 2010, Accepted.
461. Xu, D., Williamson, M.J., Walker, R.C., "Advancements in Molecular Dynamics Simulations of
Biomolecules on Graphical Processing Units.", Ann. Rev. Comp. Chem., 2010, Accepted
462. Lawrenz, M., Wereszczynski, J., Amaro, R., Walker, R.C., Roitberg, A.E., McCammon, J.A., "Impact of
calcium on N1 influenza neuraminidase dynamics and binding free energy", Proteins, 2010, Accepted.

463. Zhong, L., Walker, R.C., Brady, J.W. et al., “Computational Simulations of the Trichoderma reesei
Cellobiohydrolase I Acting on Microcrystalline Cellulose Iβ: The Enzyme-Substrate Complex”, Carb.
Res., 2009, 344, 1984-1992.
464. Seabra, G.M., Walker, R.C., Roitberg, A.E., “Are Current Semi-Empirical Methods Better than Force
Fields A Study from the Thermodynamics Perspective”, J. Phys. Chem. A., 2009, 113, 11938-11948.
465. Bergonzo, C., Campbell, A.J., Walker, R.C., Simmerling, C., “A Partial Nudged Elastic Band
Implementation for Use with Large or Explicitly Solvated Large Systems”, Int. J. Quant. Chem., 2009,
109, 15, 3781-3790.
466. Crowley, M.F., Williamson, M.J., Walker, R.C., “CHAMBER: Comprehensive Support for CHARMM
Force Fields Within the AMBER Software”, Int. J. Quant. Chem., 2009, 109, 15, 3767-3772
467. C. David Sherrill, Bobby G. Sumpter, Mutasem O. Sinnokrot, Michael S. Marshall, Edward G.
Hohenstein, Ross C. Walker, Ian R. Gould, “Assessment of standard force field models against highquality
ab initio potential curves for prototypes of -, CH/, and SH/ interactions”, J. Comput. Chem, 2009,
30, 2187-2193.
468. Parastou Sadatmousavi, Mark J. Williamson, Dong Xu, Mark R. Nimlos, Michael F. Crowley, Michael E.
Himmel & Ross C. Walker, "Recent Advances in Understanding the Mechanism of Action of the
Catalytic Binding Module of CBH I Cellulase through Advanced MD Simulations", 239th American
Chemical Society National Meeting, San Francisco, CA, Mar 2010.
469. Michael F. Crowley, Mark J. Williamson, Ross C. Walker et al. – “Cellobiohydrolase Processivity:
Improved enzymes for bioethanol”, SciDAC 2009, San Diego, CA, Jun 2009.
470. Michael F. Crowley, Mark J. Williamson, Ross C. Walker – “CHAMBER A CHARMM Format Convertor
to Enable the use of CHARMM Force Fields in AMBER”, 49th Sanibel Symposium, St. Simon’s Island,
GA, Feb 2009.
471. Warshel, A., Levitt, M., “Theoretical Studies of Enzymic Reactions - Dielectric, Electrostatic and Steric
Stabilization of Carbonium-Ion in Reaction of Lysozyme.” J. Molec. Biol.,1976 103, 227-249.
472. Senn, H. M., Thiel, W., “QM/MM Methods for Biomolecular Systems.” Angew. Chem. Int. Ed., 2009, 48,
1198-1229.
473. Bule, R. E., Ensing, B., Sikkema, J. Vissscher, L., “Toward a Practical Method for Adaptive QM/MM
Simulations.” J. Chem. Theory Comput., 2009, 5, 2212-2221.
474. de Seabra, G. M., Walker, R. C., Roitberg, A., “Are Current Semiempirical Methods Better Than Force
Fields A Study from the Thermodynamics Perspective.” J. Phys. Chem. A, 2009, 113, 1938-11948.
475. Torrie, G. M., Valleau, J. P., “Non-Physical Sampling Distributions in Monte-Carlo Free-Energy
Estimation - Umbrella Sampling.”, J. Comp. Phys., 1977, 23, 187-199.
476. Mills, G., Jonsson, H., Schenter, G. K., “Reversible Work Transition-State Theory - Application to
Dissociative Adsorption of Hydrogen.”, Surface Science, 1995, 324, 305-337.
477. Hansmann, U. H. E., “Parallel Tempering Algorithm for Conformational Studies of Biological
Molecules.”, Chem. Phys. Lett., 1997, 281, 140-150.
MCB090160
478. Pfaendtner, J., E. Lyman, T. D. Pollard, and G. A. Voth. 2010. Structure and dynamics of the actin
filament. J. Mol. Biol. 396:252-263.
479. Pfaendtner, J., E. De La Cruz, and G. A. Voth. 2010. Actin filament remodeling by cofilin. Proc. Natl.
Acad. Sci. 107:7299-7304.
480. Pfaendtner, J., D. Branduardi, M. Parrinello, T. D. Pollard, and G. A. Voth. 2009. Nucleotide-dependent
conformational states of actin. Proc. Natl. Acad. Sci. 106:12723-12728.
MCB110020
481. J. K. Shen, “Uncovering specific electrostatic interactions in the denatured states of proteins”, Biophys.
J., 99, 924-932 (2010).
482. J. A. Wallace and J. K. Shen, “Probing strand orientation and registry alignment in the propagation of
amyloid fibrils”, Biochemistry, 49, 5290-5298 (2010).
483. Y. Wang, J. A. Wallace and J. K. Shen, “Titration simulations of lauric acid in anionic, cationic and
neutral micelles”, in preparation.
484. J. A. Wallace and J. K. Shen, “Continuous constant pH molecular dynamics simulations in explicit
solvent”, in preparation.
485. C. Shi, J. A. Wallace and J. K. Shen, “Improving pKa predictions for internal residues”, in preparation.
MCB110024
486. Mamonova T., Wang B., Kurnikova M. and Friedman PA. Binding Capacity of the PDZ2 Domain of
NHERF1 for the Target Ligands (manuscript in preparation).

487. Mamonova T., Wang B., Kurnikova M. and Friedman P.A. Modeling and Mutation of NHERF1
Dimerization Domains, Biophys.J.98 (3), Suppl. 58a (2010).
MCB110028
488. N. Haspel, D. Zanuy, R. Nussinov, T. Teesalu, E. Ruoslahti and C. Aleman, Binding of a C-end rule
peptide to neuropilin-1 receptor: A molecular modeling approach. To be submitted, 2010.
Ocean Sciences
OCE030007
489. Boe, J., A. Hall, F. Colas, J. C. McWilliams, X. Qu, and J. Kurian, What shapes mesoscale wind
anomalies in coastal upwelling zones, Clim. Dyn., submitted, 2010.
490. Buijsman, M. C., Y. Kanarska, and J. C. McWilliams, On the generation and evolution of nonlinear
internal waves in the South China Sea, J. Geophys. Res., 115, C02012, 2010a.
491. Buijsman, M. C., J. C. McWilliams, and C. R. Jackson, East-west asymmetry in nonlinear internal waves
from Luzon Strait, J. Geophys. Res., in press, 2010b.
492. Buijsman, M. C., Y. Uchiyama, J. C. McWilliams, and C. Hill-Lindsay, Internal tides in the Southern
California Bight, J. Geophys. Res., submitted, 2010c.
493. Colas, F., X. Capet, and J. C. McWilliams, Eddy fluxes and induced circulation in Eastern Boundary
Systems, in preparation, 2010.
494. Colas, F., J. C. McWilliams, X. Capet, and J. Kurian, Heat balance and eddies in the Peru-Chile Current
System, submitted, 2010.
495. Colas, F., M. J. Molemaker, J. C. McWilliams, and X. Capet, Regimes of near-surface submesoscale
dynamics in Eastern Boundary Systems, in preparation, 2010.
496. Dong, C., E. Y. Idica, and J. C. McWilliams, Circulation and multiple-scale variability in the Southern
California Bight, J. Phys. Oceanogr., 82, doi:10.1016/j.pocean.2009.07.005, 2009.
497. Dong, C., and J. C. McWilliams, Island wakes in shallow water, J. Phys. Oceanogr., in preparation,
2010.
498. Dong, C., J. C. McWilliams, A. Hall, and M. Hughes, Numerical simulation of a synoptic event in the
Southern California Bight, J. Geophys. Res., submitted, 2010a.
499. Dong, C., P. Sangra, Y. Liu, J. C. McWilliams, and M. Hughes, Island effects on the Canary Current, J.
Geophys. Res., in preparation, 2010b.
500. Gruber, N., Z. Lachkar, H. Frenzel, P. Marchesiello, M. M¨unnich, and J. C. McWilliams, Mesoscale
eddyinduced reduction of biological production in eastern boundary upwelling systems, Nature, 2010.
501. Idica, E., 2010, Contaminant Transport in the Southern California Bight, Ph.D. thesis, Dept. of Civil and
Environmental Engineering, University of California, Los Angeles.
502. Jin, X., C. Dong, J. Kurian, J. C. McWilliams, D. B. Chelton, and Z. Li, SST-wind interaction in coastal
upwelling: Oceanic simulation with empirical coupling, J. Phys. Oceanogr., 39, 2957–2970, 2009.
503. Jin, X., N. Gruber, C. Deutsch, H. Frenzel, Z. Lachkar, D. Loher, J. C. McWilliams, and T. Nagai, The
impact of the coastal ocean on global biogeochemical cycles from both global and regional model
simulations, Global Biogeochem. Cycles, in preparation, 2010.
504. Kurian, J., F. Colas, X. Capet, J. C. McWilliams, and D. B. Chelton, Eddy properties in the California
Current System, in preparation, 2010.
505. LeMari´e, F., J. C. McWilliams, A. F. Shchepetkin, L. Debreu, and M. J. Molemaker, Minimizing spurious
diapycnal mixing associated with tracer advection, Ocean Modell., in preparation, 2010.
506. Li, Z., K. Ide, Y. Chao, and J. C. McWilliams, A multi-scale three-dimensional variational data
assimilation system for coastal ocean forecasting systems, in preparation, 2010.
507. Mason, E., 2009, High-resolution modelling of the Canary Basin oceanic circulation, Ph.D. thesis,
Universidad de Las Palmas de Gran Canaria.
508. Mason, E., F. Colas, J. Molemaker, A. F. Shchepetkin, C. Troupin, J. C. McWilliams, and P. Sangra,
Seasonal variability in the Canary Basin: a numerical study, J. Geophys. Res., submitted, 2010a.
509. Mason, E., M. J. Molemaker, A. Shchepetkin, F. Colas, J. C. McWilliams, and P. Sangra, Procedures
for offline grid nesting in regional ocean models, Ocean Modell., 35, 1–15, 2010b.
510. McWilliams, J. C., A perspective on submesoscale geophysical turbulence, in Proceedings of Newton
Institute Conference on The Nature of High Reynolds Number Turbulence, Oct. 30, 2008 to Dec. 12,
2008, in press, 2010.
511. McWilliams, J. C., and F. Colas, Climate heat balance off western south america: regional oceanic
circulation and eddies, CLIVAR Exchanges Newsletter, 53, 14–16, 2010.
512. McWilliams, J. C., F. Colas, and M. J. Molemaker, Cold filamentary intensification and oceanic surface
convergence lines, Geophys. Res. Lett., 36, L18602, doi:10.1029/2009GL039402, 2009.

513. McWilliams, J. C., and M. J. Molemaker, Baroclinic frontal arrest: a sequel to unstable frontogenesis, J.
Phys. Oceanogr., submitted, 2010.
514. McWilliams, J. C., M. J. Molemaker, and E. I. Olafsdottir, Linear fluctuation growth during frontogenesis,
J. Phys. Oceanogr., 39, 3111–3129, 2009.
515. Mitarai, S., D. A. Siegel, J. R. Watson, C. Dong, and J. C. McWilliams, Quantifying connectivity in the
coastal ocean with application to the Southern California Bight, J. Geophys. Res., 114, C10026, 2009.
516. Molemaker, M. J., J. C. McWilliams, and W. K. Dewar, Submesoscale generation of mesoscale
anticyclones in the California Undercurrent, in preparation, 2010.
517. Nencioli, F., C. Dong, T. Dickey, L. Washburn, and J. C. McWilliams, A vector geometry based eddy
detection algorithm and its application to a high-resolution numerical model product and high-frequency
radar surface velocities in the Southern California Bight, J. Atmos. Oceanic Technology, 27, 564–579,
2010.
518. Roullet, G., J. C. McWilliams, X. Capet, and M. J. Molemaker, Impact of surface pv on the energetics of
the forced-dissipated baroclinic instability, J. Phys. Oceanogr., submitted, 2010.
519. Sangra, P., A. Pascual, A. Rodriguez-Santana, F. Machin, E. Mason, J. C. McWilliams, J.-L. Pelegri, C.
Dong, A. Rubio, J. Aristegui, A. Marrero-Diaz, A. Hernandez-Guerra, A. Martinez-Marrero, and M.
Auladell, The Canary eddies corridor: A major pathway for long-lived eddies in the subtropical North
Atlantic, Deep-Sea Res. I, 56, 2100–2114, 2009.
520. Shchepetkin, A. F., and J. C. McWilliams, An accurate Boussinesq oceanic model with a practical,
“stiffened” equation of state, Ocean Modell., submitted, 2010.
521. Uchiyama, Y., and J. C. McWilliams, Three-dimensional unstable rip currents, J. Geophys. Res., in
preparation, 2010.
522. Uchiyama, Y., J. C. McWilliams, and J. M. Restrepo, Wave-current interaction in nearshore shear
instability analyzed with a vortex-force formalism, J. Geophys. Res., 114, C06021,
doi:10.1029/2008JC005135, 2009.
523. Uchiyama, Y., J. C. McWilliams, and A. F. Shchepetkin, Wave-current interaction in a three-dimensional
circulation model with a vortex force formalism: Application to the surf zone, Ocean Modell., 34, 16–35,
2010.
524. Wang, X., Y. Chao, C. Dong, J. Farrara, Z. Li, J. McWilliams, J. D. Paduan, and L. K. Rosenfeld,
Modeling tides in Monterey Bay, California, Deep-Sea Res. II, 56, 219–231, 2009.
525. Watson, J. R., C. G. Hays, P. T. Raimondi, S. Mitarai, D. A. Siegel, C. Dong, J. C. McWilliams, and C.
A. Blanchette, Currents connecting communities: The decay of nearshore community similarity with
ocean circulation, Ecology, submitted, 2010.
526. Watson, J. R., S. Mitarai, D. A. Siegel, J. Caselle, C. Dong, and J. C. McWilliams, Simulating larval
connectivity in the Southern California Bight, Mar. Ecol. Progr. Ser., submitted, 2009.
527. Weir, B., Y. Uchiyama, E. M. Lane, J. M. Restrepo, and J. C. McWilliams, A vortex force analysis of the
interaction of rip currents and gravity waves, J. Geophys. Res., submitted, 2010.
OCE100001
528. A. Griesel, S.T. Gille, J. Sprintall, J.L. McClean, M. Maltrud : Eulerian and Lagrangian diffusivities in the
Southern Ocean inferred from an eddying model, in preparation. 2011.
529. A. Griesel, S.T. Gille, J. Sprintall, J.L. McClean, J.H. La-Casce, M. Maltrud: Isopycnal diffusivities in the
Antarctic Circumpolar Current inferred from Lagrangian floats in an eddying model, J. Geophys. Res.,
115, doi:10.1029/2009JC005821. 2010.
Physics
ASC090004
530. Y. Nakatsukasa, Z. Bai and F. Gygi, “Optimizing Halley’s Iteration for Computing the Matrix Polar
Decomposition”, SIAM J. Matrix Anal. & Appl., 31, 2700 (2010).
531. D. Donadio, L. Spanu, I. Duchemin, F. Gygi, G. Galli, “Ab initio investigation of the melting line of
nitrogen at high pressure”, Phys. Rev. B82, 020102(R) (2010).
532. Duchemin and F. Gygi, “A scalable and accurate algorithm for the computation of Hartree–Fock
exchange”. Comput. Phys. Comm. 181, 855 (2010).
533. T.A. Pham, T.S. Li, S. Shankar, F. Gygi, and G. Galli, First-principles investigations of the dielectric
properties of crystalline and amorphous Si3N4 thin films, Applied Physics Letters 96 (2010).
534. C. Zhang, D. Donadio and G. Galli, First Principle Analysis of the IR Stretching Band of Liquid Water", J.
Phys. Chem. Lett.1, 1398 (2010).
DMR080007

535. Combining optical transparency with electrical conductivity: challenges and prospects, J.E. Medvedeva,
in “Transparent Electronics: From Synthesis to Applications”, Editors: A. Facchetti and T. Marks,
Publisher: John Wiley & Sons. April, 2010
536. Complex transparent conducting oxides with tunalbe properties: role of crystal symmetry, chemical
composition and carrier generation, J.E. Medvedeva and C.L. Hettiarachchi Physical Review B, 81,
125116 (2010)
MCA08X034
537. An Arnoldi-Lanczos Program to Propagate the Time-Dependent Schr¨odinger equation X. Guan, C.J.
Noble, O. Zatsarinny, K. Bartschat, B. I. Schneider, Comp. Phys. Commun. 180, 2401 (2009).
538. Probing Electron Correlation via Attosecond XUV Pulses in the Two-Photon Double Ionization of Helium
J. Feist, S. Nagele, R. Pazourek, E. Persson, B. I. Schneider, L. A. Collins, and J. Burgd¨orfer, Phys.
Rev. Lett. 103, 063002 (2009).
539. Electron Correlation in Two Photon Double Ionization of Helium from Attosecond to XFEL Pulses J.
Feist, R. Pazourek, S. Nagele, E. Persson, B. I. Schneider, L. A. Collins, and J. Burgd¨orfer, J. Phys. B.
42, 134014 (2009).
540. Spin-asymmetry function for elastic electron scattering from lead atoms in the energy range 11−14 eV
V. Hamelbeck, G.F. Hanne, O. Zatsarinny, K. Bartschat, R.K. Gangwar, and R. Srivastava, Phys. Rev.
A 80, 062711 (2009).
541. Complete Breakup of the Helium Atom by Proton and Antiproton Impact X. Guan and K. Bartschat,
Phys. Rev. Lett. 103, 213201 (2009).
542. Time-dependent B-spline R-matrix Approach to Double Ionization of Atoms by XUV Laser Pulses X.
Guan, O. Zatsarinny, C.J. Noble, K. Bartschat, and B.I. Schneider, J. Phys. B 42, 134015 (2009).
543. Absolute angle-differential cross sections for electron-impact excitation of neon within the first 3.5 eV
above threshold M. Allan, K. Franz, H. Hotop, O. Zatsarinny, and K. Bartschat, J. Phys. B 42, 044009
(2009).
544. Ab Initio Calculation of Two-Electron Emission by Attosecond Pulses J. Feist, R. Pazourek, S. Nagele,
E. Persson, B. I. Schneider, L. A. Collins, and J. Burgd¨orfer, J. Phys.: Conf. Ser. 194, 012010 (2009).
545. Spin-resolved electron-impact excitation of the 6s6p (J = 1) states in mercury F. J¨uttemann, G.F.
Hanne, O. Zatsarinny, and K. Bartschat, Phys. Rev. A 79, 042712 (2009).
546. Fully Relativistic R-Matrix Calculations for Electron Collisions with Mercury O. Zatsarinny and K.
Bartschat, Phys. Rev. A 79, 042713 (2009).
547. Benchmark Calculations for Near-Threshold Electron-Impact Excitation of Krypton and Xenon Atoms O.
Zatsarinny and K. Bartschat, J. Phys. B 43, 074031 (2010).
548. Few-cycle Intense Laser Interactions with Complex Atoms X. Guan and K. Bartschat, J. Phys.: Conf.
Ser. 212, 012023 (2010).
549. Ionization of Atomic Hydrogen in Strong Infrared Laser Fields A. N. Grum-Grzhimailo, B. Abeln, K.
Bartschat, D. Weflen, and T. Urness, Phys. Rev. A 81, 043408 (2010).
550. Differential Cross Sections for Non-Sequential Double Ionization of He by 52 eV Photons from FLASH
M. Kurka, J. Feist, D. A. Horner, A. Rudenko, Y. H. Jiang, K. U. K¨uhnel, L. Foucar, T. N. Rescigno, C.
W. McCurdy, R. Pazourek, S. Nagele, M. Schulz, O. Herrwerth, M. Lezius, M. F. Kling, M. Sch¨offler, A.
Belkacem, S. D¨usterer, R. Treusch, B. I. Schneider, L. A. Collins, J. Burgd¨orfer, C. D. Schr¨oter, R.
Moshammer, and J. Ullrich, New J. Phys. 12, 073035 (2010).
551. Delay in Photoemission M. Schultze, M. Fiess, N. Karpowicz, J. Gagnon, M. Korbman, M. Hofstetter, S.
Neppl, A. L. Cavalieri, Y. Komninos, Th. Mercouris, C. A. Nicolaides, R. Pazourek, S. Nagele, J. Feist,
J. Burgd¨orfer¨orfer, A. M. Azzeer, R. Ernstorfer, R. Kienberger, U. Kleineberg, E. Goulielmakis, F.
Krausz, and V. S. Yakovlev, Science 328, 1658 (2010).
552. Angle-differential Stokes parameters for spin-polarized electron-impact excitation of the Hg (6s6p)3P1
state at 25 eV scattering energy F. J¨uttemann, G.F. Hanne, O. Zatsarinny, K. Bartschat, R. Srivastava,
R.K. Gangwar, and A.D. Stauffer, Phys. Rev. A 81, 012705 (2010).
553. New light on the Kr (4p55s2) Feshbach resonances: high-resolution electron scattering experiments
and B-spline R-matrix calculations T.H. Hoffmann, M.-W. Ruf, H. Hotop, O. Zatsarinny, K. Bartschat,
and M. Allan, J. Phys. B 43, 085206 (2010).
554. Benchmark calculations for near-threshold electron-impact excitation of krypton and xenon atoms O.
Zatsarinny and K. Bartschat, J. Phys. B 43, 074031 (2010).
555. Electron impact excitation of the (3d104s) 2S1/2 ! (3d94s2) 2D5/2,3/2 transitions in copper atoms O.
Zatsarinny, K. Bartschat, V. Suvorov, P.J.O. Teubner, and M.J. Brunger, Phys. Rev. A 81, 062705
(2010).
556. Two-Photon Double Ionization of H2 in Intense Femtosecond Laser Pulses X. Guan, K. Bartschat, and
B.I. Schneider, accepted as Rapid Communication in Phys. Rev. A (2010); preprint available from
http://arxiv.org/abs/1009.4866

557. Strong-Field Ionization of Lithium in a MOTREMI M. Schuricke, J. Steinmann, G. Zhu, I. Ivanov, A.S.
Kheifets, A.N. Grum-Grzhimailo, K. Bartschat, A. Dorn, and J. Ullrich, submitted to Phys. Rev. A (2010).
558. Near-threshold electron impact excitation of the argon 3p54s configuration – new and revised
normalized differential cross sections using recent time-of-flight measurements for normalization. M.A.
Khakoo, O. Zatsarinny, and K. Bartschat submitted to J. Phys. B (2010).
559. Unexpected effects in spin-polarized electron impact excitation of the (3d104s5s) 3S1 state in zinc L.
Pravica, J.F. Williams, D. Cvejanovi´c, S. Samarin, K. Bartschat, O. Zatsarinny, A.D. Stauffer, and R.
Srivastava. submitted to Phys. Rev. Lett. (2010).
560. Benchmark Calculations of Atomic Data for Plasma and Lighting Applications K. Bartschat and O.
Zatsarinny, submitted to Plasma Sources Sci. and Technol. (2010).
561. Universal Features in Sequential and Nonsequential Two-Photon Double Ionization of Helium R.
Pazourek, J. Feist, S. Nagele, E. Persson, B. I. Schneider, L. A. Collins, and J. Burgd¨orfer
562. Using Neutrons in Atomic Physics: Neutron Impact Ionization of Helium M. Liertzer, J. Feist, S. Nagele,
and J. Burgd¨orfer
563. Apparent Time Delays in Classical and Quantum Mechanical Atomic Attosecond Streaking S. Nagele,
R. Pazourek, J. Feist, K. Tokesi, C. Lemell, and J. Burgd¨orfer
564. Two-Photon Double Ionization of Helium by Chirped Attosecond XUV Pulses R. Pazourek, G.
Schoissengeier, S. Nagele, J. Feist, B. I. Schneider, L. A. Collins, and J. Burgd¨orfer
565. Time-Dependent Perturbation Theory for Two-Photon Processes: Sequential vs. Nonsequential
Ionization J. Feist
566. High-Precision Study of Elastic Electron Scattering from Krypton Atoms M. Allan, O. Zatsarinny, and K.
Bartschat
567. Electron Impact Excitation of the 4p55s States in Krypton M. Allan, O. Zatsarinny, and K. Bartschat
568. Fully Relativistic Dirac B-Spline R-Matrix Calculations for Electron Scattering from Xenon M. Allan, O.
Zatsarinny, and K. Bartschat
569. Elastic electron scattering from atomic and molecular iodine O. Zatsarinny, K. Bartschat, F. Blanco, and
G. Garcia
570. Electron scattering from copper O. Zatsarinny and K. Bartschat
571. Alignment effects in two-photon double ionization of H2 X. Guan, K. Bartschat, and B.I. Schneider
572. Time-dependent treatment of one-photon double ionization of H2 X. Guan, K. Bartschat, and B.I.
Schneider
PHY080042
573. T. DeGrand, Y. Shamir and B. Svetitsky, “Running coupling and mass anomalous dimension of SU(3)
gauge theory with two flavors of symmetric-representation fermions,” Phys. Rev. D 82, 054503 (2010)
[arXiv:1006.0707 [hep-lat]].
PHY090003
574. Sperhake, U., E. Berti, V. Cardoso, F. Pretorius & N.Yunes (2010) Cross sections and maximum
radiated energy in grazing collisions of spinning and nonspinning black hole binaries in preparation
575. Berti, E., V. Cardoso & B. Kipapa (2010) Radiation from paticles with arbitrary energy falling into higherdimensional
black holes in preparation
576. Witek, H., D. Hilditch & U. Sperhake (2010) Stability of the puncture method with a generalized BSSN
formulation in preparation
577. Sperhake, U., E. Berti, V. Cardoso, F. Pretorius & N.Yunes (2010) Superkicks in Ultrarelativistic Grazing
Collisions of Spinning Black Holes in preparation
578. Brizuela, D., J. M. Mart´ın-Garcia, U. Sperhake & K. Kokkotas (2010) High-order perturbations of a
spherical collapsing star accepted for publication in Phys. Rev. D, arXiv:1009.5605 [gr-qc]
579. Barausse, E., V. Cardoso & G. Khanna (2010) Test bodies and naked singularities: is the self-force the
cosmic censor submitted to Phys. Rev. D, arXiv:1008.5159 [gr-qc]
580. Witek, H., M. Zilh˜ao, L. Gualtieri, V. Cardoso, C. Herdeiro, A. Nerozzi & U. Sperhake (2010) Numerical
relativity for D dimensional spacetimes: head-on collisions of black holes and gravitational wave
extraction accepted for publication in Phys. Rev. D, arXiv:1006.3081 [hep-th]
581. Witek, H., V. Cardoso, C. Herdeiro, A. Nerozzi, U. Sperhake & M.Zilh˜ao (2010) Black holes in a box:
towards the numerical simulation of black holes in AdS accepted for publication in Phys. Rev. D,
arXiv:1004.4633 [hep-th]
582. Molina, C., P. Pani, V. Cardoso & L. Gualtieri (2010) Gravitational signature of Schwarzschild black
holes in dynamical Chern-Simons gravity Phys. Rev. D 81, 124021
583. Kesden, M., U. Sperhake & E. Berti (2010) Relativistic suppression of black hole recoils Astrophys. J.
715, 1006-1011

584. Berti, E., V. Cardoso, T. Hinderer, M. Lemos, F. Pretorius, U. Sperhake & N. Yunes (2010)
Semianalytical estimates of scattering thresholds and gravitational radiation in ultrarelativistic black hole
encounters Phys. Rev. D 81, 104048
585. Kesden, M., U. Sperhake & E. Berti (2010) Final spins from the merger of precessing binary black holes
Phys. Rev. D 81, 084054
586. Zilh˜ao, M., H. Witek, U. Sperhake, V. Cardoso, L. Gualtieri, C. Herdeiro & A. Nerozzi (2010) Numerical
relativity for D dimensional axially symmetric space-times: formalism and code tests Phys. Rev. D 81,
084052
587. Berti, E., V. Cardoso, L. Gualtieri, F. Pretorius & U. Sperhake (2009) Comment on “Kerr Black Holes as
Particle Accelerators to Arbitrarily High Energy” Phys. Rev. Lett. 103, 239001
588. Sperhake, U., V. Cardoso, F. Pretorius, E. Berti, T. Hinderer & N. Yunes (2009) Cross section, final spin
and zoom-whirl behavior in high-energy black-hole collisions Phys. Rev. Lett. 103, 131102
589. Lovelace, G., Y. Chen, M. Cohen, J. D. Kaplan, D. Keppel, K. D. Matthews, D. A. Nichols, M. A. Scheel
& U. Sperhake (2010) Momentum flow in black-hole binaries: II. Numerical simulations of equal-mass,
headon mergers with antiparallel spins Phys. Rev. D 82, 064031
590. Gabler, M., U. Sperhake & N. Andersson (2009) Non-linear radial oscillations of neutron stars Phys.
Ref. D 80, 064012
591. Sperhake, U., V. Cardoso, F. Pretorius, E. Berti, T. Hinderer & N. Yunes (2010) Ultra-relativistic grazing
collisions of black holes Proceedings for the 12th Marcel Grossmann Meeting, arXiv:1003.0882 [gr-qc]
592. Witek, H., V. Cardoso, C. Herdeiro, A. Nerozzi, U. Sperhake & M. Zilh˜ao (2010) Black holes in a box
Proceedings for the 12th Marcel Grossmann Meeting
593. Witek, H., V. Cardoso, C. Herdeiro, A. Nerozzi, U. Sperhake & M. Zilh˜ao (2010) Black holes in a box
594. J. Phys. Conf. Ser. 229, 012072 Proceedings of the Spanish Relativity Meeting – ERE 2009
595. Zlh˜ao, M., H. Witek, V. Cardoso, C. Herdeiro, A. Nerozzi & U. Sperhake Numerical relativity in higher
dimensions J. Phys. Conf. Ser. 229, 012074 Proceedings of the Spanish Relativity Meeting – ERE 2009
596. Sperhake, U., Black hole collisions in higher-dimensional spacetimes, invited Seminar at the
Department of Mathematics, Dec 2010, University of Southampton, UK
597. Sperhake, U., Black-hole collisions and gravitational waves, Plenary Talk, Iberian Meeting Physics
Education, Sep 2010, Vila Real, Portugal
598. Cardoso, V., Quasinormal modes of black holes: recent (and past) advances, Invited Talk at the 19th
Conference on General Relativity, Jul 2010, Mexico City, Mexico
599. Sperhake, U., Relativistic suppression of black-hole superkicks, 19th Conference on General Relativity,
Jul 2010, Mexico City, Mexico
600. Berti, E., The interface between numerical relativity and data analysis: open problems, 8th International
LISA Symposium, Jul 2010, Stanford University, CA
601. Sperhake, U., 11 Orbit inspiral of unequal mass black-hole binaries, Theory Meets Data Analysis at
Comparable and Extreme Mass Ratios Workshop, Jun 2010, Perimeter Institute, Waterloo, CA
602. Pretorius, F., Black Hole Collisions, invited Talk, Taming Complexity Workshop, Apr 2010, CUNY, New
York, NY
603. Pretorius, F., Black Hole Collisions, invited Talk, APS Meeting, Feb 2010, Washington D.C.
604. Pretorius, F., Black Hole Mergers, Feb 2010, Mathematics Department Colloquium, Harvard, MA
605. Cardoso, V., High energy collisions of black holes, Invited seminar at the Department de Fisica
Fondamental, Nov 2009, Universitat de Barcelona, Spain
PHY110009
606. S.X. Hu, B. Militzer, V. N. Goncharov, S. Skupsky, “Strong Coupling and Degeneracy Effects in Inertial
Confinement Fusion Implosions”, Phys. Rev. Lett. 104, 235003 (2010).
607. S. X. Hu, “Optimizing the FEDVR-TDCC code for exploring the quantum dynamics of two-electron
systems in intense laser pulses”, Phys. Rev. E 81, 056705 (2010).
608. S. X. Hu, “Attosecond timing the ultrafast charge-transfer process in atomic collisions”, Phys. Rev. Lett.
105, xxxxxx (2010) (in press).
609. S. X. Hu and A. F. Starace, “Attosecond phase control of core-excited ionization of He atoms”, Phys.
Rev. Lett. (to be submitted).
610. S. X. Hu, L. A. Collins, and B. I. Schneider, “Attosecond soft-X-ray scanning of H2 + with circularlypolarized
pulses”, Phys. Rev. A (in preparation).
PHY990002
611. T. Chu, H. P. Pfeiffer, and M. A. Scheel. High accuracy simulations of black hole binaries: spins antialigned
with the orbital angular momentum. Phys. Rev., D80:124051, 2009, arXiv:0909.1313 [gr-qc].
612. M. D. Duez, F. Foucart, L. E. Kidder, C. D. Ott, and S. A. Teukolsky. Equation of state effects in black
hole-neutron star mergers. Class. Quant. Grav., 27:114106, 2010, arXiv:0912.3528 [astro-ph.HE].

613. F. Foucart, M. D. Duez, L. E. Kidder, and S. A. Teukolsky. Black hole-neutron star mergers: effects of
the orientation of the black hole spin. 2010, arXiv:1007.4203 [astro-ph.HE].
614. G. Lovelace et al. Momentum flow in black-hole binaries: II. Numerical simulations of equalmass, headon
mergers with antiparallel spins. Phys. Rev., D82:064031, 2010, arXiv:0907.0869 [gr-qc].
615. G. Lovelace, M. A. Scheel, and B. Szilagyi. Simulating merging binary black holes with nearly extremal
spins. 2010, arXiv:1010.2777.
616. B. Szilagyi, L. Lindblom, and M. A. Scheel. Simulations of Binary Black Hole Mergers Using Spectral
Methods. Phys. Rev., D80:124010, 2009, arXiv:0909.3557 [gr-qc].

0SB EOT
Event Details
UIndiana
Type Title Location Date(s) Hours Number of
Participant
s
Workshop MEAP – Networking IUPUI,
Indianap
olis, IN
Workshop MEAP – LEGO
Mindstorms
Workshop
MEAP – No Guts, No
Glory
Workshop Exxon/Mobil Summer
Camp
Workshop Wonderlab Summer
Camp
Demo IU Statewide IT
Conference Show and
Tell Reception
Tour Garland with
Engineering Dean and
Faculty
IUPUI,
Indianap
olis, IN
IUPUI,
Indianpoli
s, IN
Indiana
Universit
y,
Blooming
ton, IN
Indiana
Universit
y,
Blooming
ton, IN
Indiana
Universit
y,
Blooming
ton, IN
IUPUI,
Indianap
olis, IN
Tour Joan Savage & Guest IUPUI,
Indianpoli
s, IN
Tour Networking Group IUPUI,
Indinapoli
s, IN
Tour Lighting & Field
Production Class
IUPUI,
Indianap
olis, IN
14 JUN
2010
28-2
JUN/JUL
2010
19 JUL
2010
13-14
JUL 2010
23 JUL
2010
27 SEP
2010
OCT
2010
OCT
2010
OCT
2010
OCT
2010
Number
of
Underreprese
nted
people
Method
3 29 29 S
10 23 23 S
3 28 28 S
12 22 12 S
4 18 6 S
3 42 6 S
1 7 1 S
1 2 0 S
1 17 2 S
1 14 2 S

Seminar What is
Cyberinfrastructure
Fair
Research Technologies
Fair
Tour Northern Kentucky
University
Tour/Demo Palakal and India
Guests Demo/Tour
SIGUICC
S2010,
Norfolkd,
VA
Simon
Hall,
Indiana
Universit
y,
Blooming
ton, IN
IUPUI,
Indianap
olis, IN
IUPUI,
Indianpoli
s, IN
Tour Dentistry IUPUI,
Indianpoli
s, IN
Tour Tech Center IUPUI,
Indianap
olis, IN
Conference LEADING THE WAY –
Stereoscopic
Presentation at
SACNAS
Expo LEADING THE WAY –
Stereoscopic
Presentation at USA
Science & Engineering
Expo
Seminar
Presentatio
n
FutureGrid – A Testbed
for Cloud and Grid
Computing
No Guts, No Glory
Workshop Gateway Computing
Environments 2010
Demo Exposing Real-World
Applications of
Computational Science
using Stereoscopic
SACNAS
2010,
Anaheim,
CA
Washingt
on DC
Indiana
Universit
y,
Blooming
ton, IN
SC10,
New
Orleans,
LA
SC10,
New
Orleans,
LA
SC10,
New
Orleans,
LA
25 OCT
2010
25 OCT
2010
NOV
2010
NOV
2010
NOV
2010
NOV
2010
2-4 OCT
2010
24-25
OCT
2010
27 OCT
2010
13 NOV
2010
14 NOV
2010
15-18
NOV
2010
1 18 3 S
3 14 Unknow
n
1 4 0 S
1 9 1 S
1 3 0 S
1 12 1 0
12 650 620 S
14 5500 2400 S
1 32 3 S
1.5 25 3 S
8.5 50 Unknow
n
21 75 18 S
S
S

Demo
Demo
Demo
Demo
Seminar
Presentation
Supporting Real Time
Weather Predictions
with LEAD II –
Experiences with
Vortex2 Experiment
GlobalNOC Worldview
Interactive Networking
Visualization Tool
Cloud Computing and
FutureGrid
Hybrid Workflows with
LEAD II and Trident
Pushing Back on the
Data
Deluge:
Advancements in
Metadata, Archiving
and Workflows
Symposium Open Gateway
Computing
Environments
Seminar New and Emerging
Technologies and
Applications
Seminar
FutureGrid and Cloud
Computing
Panel Future of
Supercomputing
Centers
Tour Learning Community
Class
SC10,
New
Orleans,
LA
SC10,
New
Orleans,
LA
SC10,
New
Orleans,
LA
SC10,
New
Orleans,
LA
SC10,
New
Orleans,
LA
SC10,
New
Orleans,
LA
SC10,
New
Orleans,
LA
SC10,
New
Orleans,
LA
SC10,
New
Orleans,
LA
IUPUI,
Indianap
olis, IN
Classroom OSG Grid School Sao
Paulo,
Brazil
Tour/Demo CEMT IUPUI,
Indianap
olis, IN
15-18
NOV
2010
15-18
NOV
2010
15-18
NOV
2010
15-18
NOV
2010
16 NOV
2010
16 NOV
2010
17 NOV
2010
17 NOV
2010
19 NOV
2010
23 NOV,
2010
6-10
DEC
2010
DEC
2010
21 45 12 S
21 60 10 S
21 38 2 S
21 35 2 S
4 8 1 S
2 10 Unknow
n
1 18 1 S
1 5 1 S
.25 150 40 S
1 38 2 S
8 50 45 S
1 3 0 S
S

P
Tour Tech 104 IUPUI,
Indianap
olis, IN
Tour Tech 104 IUPUI,
Indianpoli
s, IN
DEC
2010
DEC
2010
1 57 2 S
1 20 2 S
ULONI
Type Title Location Date(s) Hours Number of
Participant
s
W
T
LONI HPC Workshop
at Univ. Louisiana at
Lafayetts
Intro to HPC: Account
Allocation &
Management
T Job Management w/
PBS & Loadleveler
Lafayette
, LA
11/1&2/1
0
Number
of
Underreprese
nted
people
16 12 4 S
LSU 9/8/10 2 6 0 S
LSU 9/15/10 2 5 S
T Introduction to MPI LSU 9/22/10 2 7 S
T Advanced MPI LSU 9/29/10 2 4 S
T Introduction to Cactus LSU 10/4/10 2 15 3 S
T
T
Introduction to
Gaussian
Introduction to
OpenMP
LSU 10/13/10 2 4 S
LSU 10/20/10 2 5 S
T OpenMP part 2 LSU 10/27/10 2 4 S
T Globus LSU 11/10/10 2 8 S
T
Introduction to HPC
Visualization
LSU 11/3/10 2 7 S
T Introduction SAGA LSU 11/29/10 2 12 S
Method
UNCAR
Type Title Location Date(s) Hours Number of
Participant
s
NCAR:
Recent
Advancements in
th
14P
Workshop on
Number
of
Underreprese
nted
people
1 Nov. 1.0 150 unknown S
Method

Seminar
High Resolution
Climate Modeling
the use of
HPC in
Meteorology
2010
NCAR:
Tutorial
Building Cloud
Clusters with
Amazon EC2
SC10, New
Orleans
15 Nov.
2010
1.0 100 unknown S
NCAR:
Tutorial
VAPOR: A Tool for
Interactive
Visualization of
Massive Earth-
Science Data Sets
AGU 2010,
San
Francisco
13 Dec.
2010
1.0 28 unknown S
UNCSA
Type Title Location Date(s) Hours Number of
Participants
Online
Tutorial
Online
Tutorial
Online
Tutorial
Number
of Underrepresent
ed people
Access Grid Tutorials CI-Tutor Ongoing N/A 17 Unknown A
BigSim: Simulating
PetaFLOPS
Supercomputers
Debugging Serial and
Parallel Codes
CI-Tutor Ongoing N/A 7 Unknown A
CI-Tutor Ongoing N/A 31 Unknown A
Method
Online
Tutorial
Getting Started on
theTeraGrid
CI-Tutor Ongoing N/A 13 Unknown A
Online
Tutorial
Online
Tutorial
Intermediate MPI CI-Tutor Ongoing N/A 84 Unknown A
Introduction to MPI CI-Tutor Ongoing N/A 354 Unknown A
Online
Tutorial
Introduction to Multicore
Performance
CI-Tutor Ongoing N/A 32 Unknown A
Online
Tutorial
Introduction to
OpenMP
CI-Tutor Ongoing N/A 102 Unknown A
Online
Tutorial
Online
Tutorial
Introduction to
Performance Tools
Introduction to
Visualization
CI-Tutor Ongoing N/A 12 Unknown A
CI-Tutor Ongoing N/A 18 Unknown A
Online
Tutorial
Multilevel Parallel
Programming
CI-Tutor Ongoing N/A 34 Unknown A
Online
Tutorial
Parallel Computing
Explained
CI-Tutor Ongoing N/A 45 Unknown A

Online
Tutorial
Parallel Numerical
Libraries
CI-Tutor Ongoing N/A 20 Unknown A
Online
Tutorial
Performance Tuning
for Clusters
CI-Tutor Ongoing N/A 24 Unknown A
Online
Tutorial
Tuning Applications
for High Performance
Networks
CI-Tutor Ongoing N/A 8 Unknown A
Workshop
2010 NWChem
Workshop
NCSA Dec 1-2,
2010
16 14 Unknown S
UNICS
Type Title Location Date(s) Hours Number of
Participant
s
Workshop
Workshop
College
Class
College
Class
College
Class
College
Class
College
Class
College
Class
NICS/OLCF XT Hex-
Core Workshop
Computational Thinking
for Educators
Data
U.Tennessee-
Knoxville:
Structures
U.Tennessee-
Knoxville: graduate
HPC class
Wofford College
Tufts University
Brown University
University of Colorado
Oak
Ridge
National
Lab, TN
ORAU,
Oak
Ridge,
TN
Knoxville,
TN
Knoxville,
TN
Spartanb
urg, SC
Boston,
MA
Providen
ce, RI
Boulder,
CO
May 10-
12, 2010
Jun 12-
14, 2010
2010
Spring
semester
2010
Spring
semester
2010
Spring
semester
2010
Spring
semester
2010
Spring
semester
2010
Spring
semester
Number
of
Underreprese
nted
people
24 hrs 65 20 Live
21 hrs 20 17 Live
45 16 Live
18 3 Live
6 2 Live
12 Unknown Live
11 Unknown Live
14 Unknown Live
Method

UORNL
Type Title Location Date(s) Hours Number of
Participant
s
Talk
SNS and TeraGrid's
NSTG: A best practice
for integration of
national
cyberinfrastructure with
an experimental user
facility
Galtinbue
g, TN
Workshop NOBUGS 2010 Gatlinbur
g, TN
SC’10
booth and
Sc’ed
Let me know if
someone else does not
fill this in for TG as
whole
Oct 10-
13, 2010
Oct. 10-
13, 2010
Number
of
Underreprese
nted
people
1 56 13 S
24 56 13 S
Method
UPSC
Type Title Location Date(s) Hours Number of
Participants
Focus
Group
Presentation
Conference
PSC Visit
Role of SafeNet in K-12
Cyber Safety Training
Computational
Approach to Teaching
Secondary Math and
Science
Exhibit booth at Three
Rivers Ed Tech
Conference
Careers in HPC/PSC
Overview
PSC
Pittsburgh
PA
TRETC
Pittsburgh
PA
TRETC
Pittsburgh
PA
PSC
Pittsburgh
PA
Number
of
Underrepresented
people
MethodP0F
12/15 4 9 4 S
11/10 2 6 3 S
11/10 8 30 18 S
11/17 2 13 2 S
1
1 Methodology: Synchronous (e.g., face to face in classroom or live instruction over the Internet) or asynchronous
(e.g., via the Internet using WebCT or other authoring software.) S=synchronous; A=asynchronous.

Type Title Location Date(s) Hours Number of
Participants
Poster
Session
CMIST: Enzyme –
Structure & Function
Module
BEST: Better
Educators of Science
for Tomorrow
Bioinformatics in High
School (presented by
Pittsburgh area high
school teachers and
PSC staff)
SC10
New
Orleans
LA
SC10
New
Orleans
LA
SC10
New
Orleans
LA
Number
of
Underrepresented
people
MethodP0F
11/13 1.5 25 15 S
11/13 1.5 40 15 S
11/13 1.5 50 15 S
1
Poster
Session
Poster
Session
Training
Conference
Conference
Conference
High School Research
Internship in
Bioinformatics
(presented by PSC
summer interns)
Summer Internship at
PSC (presented by
PSC summer interns)
TeraGrid New User
Training Session
SC10 Conference
Exhibitor Booth
SC10 Conference,
Education Program
SC10 Conference,
Student Field Trips
SC10
New
Orleans
LA
SC10
New
Orleans
LA
PSC/
Online
SC10
New
Orleans,
LA
SC10
New
Orleans,
LA
SC10
New
Orleans,
LA
Training Security for Lunch Pittsburgh
PA
11/13 1.5 50 15 S
11/13 1.5 50 15 S
10/29 2 30 Unknown
(6 survey
responses)
Presentation
Presentation
11/12-
11/19
11/12-
11/19
11/12-
11/19
24 app. 750 app. 100 S
45 135
supported,
54 self-paid
5 210 high
school
students +
chaperones
S
75 S
will know
later
11/8 1.5 65 15 S
S

Type Title Location Date(s) Hours Number of
Participants
Training
Performance
Engineering of Parallel
Applications
European-
US
Summer
HPC
School
Catania,
Italy.
Number
of
Underrepresented
people
MethodP0F
10/6 1.5 60 12 S
1
UPurdue
Type Title Location Date(s) Hours Number of
Participant
s
Presentatio
n
Panel
Presentatio
n
Presentatio
n
Han Zhang. "Domainspecific
web services
for scientific
application
developers," GCE10 at
SC10
Carol Song, Preston
Smith. Panel “Grid
Computing
Technologies” panel
Kay Hunt At
Cyberinfrastructure
Assessment Workshop
Kay Hunt. Fall 2010
Internet2 member
meeting.
SC10,
New
Orleans,
LA
Purdue
CI Days,
West
Lafayette
, IN
Arlington,
VA
Atlanta,
GA
Nov 14,
2010
Dec. 9,
2010
Oct.
2010
Nov.
2010
30 min 20
1 hr 40
Number
of
Underreprese
nted
people
Method
USDSC
Type Title Location Date(s) Hours # of Partic.
Tutorial
(Teachers)
Webinar: Using
Graphical Organizer
to Engage Students in
# of
Underrep.
Webbased
Tutorial
12-9-
2010
Method
1.5 6 2 S

the Classroom
Conf.
Present‟n
From Data to
Discovery: Lessons
on Using Data to
Connect Events in
Our Environment
SC-10
New
Orleans,
LA
Nov.
2010
1.5 20 14 S
Conf.
Present‟n
Understanding the
Impact of Emerging
Non-Volatile
Memories on High-
Performance, IO-
Intensive Computing
SC-10
New
Orleans,
LA
Nov.
2010
1 200 20 S
Conf.
Present‟n
Moneta: A Highperformance
Storage
Array Architecture for
Next-generation, Nonvolatile
Memories
IEEE/AC
M
MICRO;
Atlanta,
GA
Dec.
2010
1 200 20 S
Conf.
Booth
SACNAS Annual
Conference
Anaheim,
CA
October,
2010
10 3,350 3,350 S
Workshop
(Teachers)
The Basics: DNA
Extraction, Size
Exclusion,
Chromatography
SDSC,
La Jolla,
CA
9-23-
2010
2 22 17 S
Workshop
(Teachers)
SMART Team Teacher
Meeting
SDSC,
La Jolla,
CA
9-24-
2010
2 6 4 S
Workshop
(Teachers)
Graphing Calculators
for Educators
SDSC,
La Jolla,
CA
10-6-
2010
3 9 6 S
Workshop
(Students)
Fall Fest Digital
Costume Illustration
Contest
SDSC,
La Jolla,
CA
10-9-
2010
6 17 17 S
Workshop
(Students)
Introduction to
Graphing Calculators
for Middle and High
School Students
SDSC,
La Jolla,
CA
10-16-
2010
4 19 10 S
Workshop
(Students)
SMART Team:
Introduction to
Biochemistry
SDSC,
La Jolla,
CA
10-16-
2010
4 26 15 S
Workshop
(Students)
Introduction to Adobe
Illustrator – Part 1
SDSC,
La Jolla,
CA
10-16-
2010
6 7 5 S
Workshop
(Teachers)
Proteins in Action 1:
ELIZA
SDSC,
La Jolla,
10-19-
2010
2 14 10 S

4P
1P
P Step
P State
CA
Workshop
(Student)
Introduction to Adobe
Illustrator – Part 2
SDSC,
La Jolla,
CA
10-23-
2010
6 7 5 S
Workshop
(Student)
SMART Team Handson
Biotech Activities
TSRI 10-30-
2010
4 18 11 S
Workshop
(Teachers)
Analysis of Proteins:
Protein Quantification
SDSC,
La Jolla,
CA
11-2-
2010
2 12 8 S
Workshop
(Teachers)
Finding Great Web 2.0
Resources for the
Classroom
SDSC,
La Jolla,
CA
11-4-
2010
2 11 7 S
Workshop
(Teachers)
SpaceTECH: Handson,
Standards-based
Astronomy Activities
SDSC,
La Jolla,
CA
11-6-
2010
3 19 14 S
Workshop
(Student)
SMART Team RasMol
Training
SDSC,
La Jolla,
CA
11-6-
2010
3 21 13 S
Workshop
(Student)
Introduction to Digital
Comic Book Creation
– Part 1
SDSC,
La Jolla,
CA
11-13-
2010
6 15 11 S
Workshop
(Teachers)
DNA Analysis: DNA
Electrophoresis
SDSC,
La Jolla,
CA
11-16-
2010
2 18 13 S
Workshop
(Teachers)
Gaseous Plasma: The
th
of Matter, or
st
Toward
Fusion
General
Atomics
Corp.,
San
Diego,
CA
11-17-
2010
2 18 12 S
Workshop
(Teachers)
Compute This: Online
Space Science
Resources for the
Classroom
SDSC,
La Jolla,
CA
11-20-
2010
2 22 15 S
Workshop
(Teachers)
Proteins in Action: Biofuels
and Enzymes
SDSC,
La Jolla,
CA
11-30-
2010
2 16 12 S
Workshop
(Teachers)
Star Parties and More:
Astronomy Tools and
Techniques for the
Classroom
SDSC,
La Jolla,
CA
12-2-
2010
2.5 15 10 S
Workshop
(Teachers)
Understanding Mars
Space
Travelers
Emporiu
12-4-
2010
3 21 14 S

m; San
Diego,
CA
Workshop
(Teachers)
All About Space Flight:
Curriculum and
Materials for Grades
4-9
San
Diego
Air and
Space
Museum,
San
Diego,
CA
12-11-
2010
3 18 11 S
Workshop
(Student)
SMART Team Mentor
Match
The
Scripps
Research
Institute,
La Jolla,
CA
12-11-
2010
3 30 18 S
Workshop
(Student)
Intro to Digital Comic
Book Creation – 2
SDSC,
La Jolla,
CA
12-11-
2010
6 15 11 S
UTACC
Type Title Location Date(s) Hours Number of
Participant
s
Workshop
Workshop
Workshop
Workshop
Workshop
Workshop
Defensive
Programming Part 1
Defensive
Programming Part 2:
System Testing
Parallel Optimization
for Ranger
C++ Programming
Basics
Fortran90/95/2003
Programming for HPC
Data Analysis on
Ranger
TACC 10/7/201
0
TACC 10/28/20
10
TACC 10/29/20
10
TACC 12/7/201
0
TACC 12/7/201
0
Cornell 12/8-
9/2010
Number
of
Underreprese
nted
people
3 12 2 S
3 9 2 S
7.5 8 3 S
3 18 4 S
4 12 3 S
16 12 S
Method

Table: Ranger Virtual Workshop Usage
Quarter Total Logins Unique Logins
Q1 „08 74 20
Q2 „08 129 26
Q3 „08 198 31
Q4 „08 135 31
Q1 „09 184 29
Q2 „09 638 216
Q3 '09 504 403
Q4 „09 438 289
Q1 '10 425 349
Q2 '10 657 377
Total 4220 1639
Starting with Q3 '10, the Usage Table is being derived from a web tracker service. The
combination of allowing guest visitors and the rapid increase in web crawlers made the data
increasingly difficult to interpret.
Page Loads Unique Visitors First Time
Visitors
Q3 '10 2697 1026 970 56
Q4 ‘10 2648 862 658 204
Returning
Visitors
TACC openly advertises tours of our facilities to the Austin and central Texas area. Tour groups
include visitors in K-12, higher education, industry, government, and the general public. An
overview of TeraGrid and its contributions to STEM research are provided at each event.
Type Title Location Date(s) Number of
Participants
Number of
Under-
Represente
d People
Vislab Tour
SURGE: Roads to
Research / UT Austin Vislab Oct. 1 60 18
students
Vislab Tour 10THome School Group Vislab Oct. 5 9 5
The Austin
Forum on
Science,
Technology &
Society
01T“Hopes and Fears for
Big Science” with
Nobel Laureate Dr.
Steven Weinberg
AT&T
Executive
Education
and
Conference
Center
Oct. 5 361 Not tracked.
Machine Room 10TSt. Edward‟s Ranger Oct. 6 20 6

Tour
Ranger Tour
and Cluster
Management
Discussion
Ranger and
Vislab Tours
Ranger and CMS
Machine Rooms
Ranger and
Vislab Tours
Ranger Tour
Vislab Tour
Vislab Tour
Vislab Tour
Vislab Tour
Vislab tour
Ranger Machine
Room Tour
Vislab Tour
The Austin
Forum on
Science,
Technology &
Society
Vislab Tour
Vislab Tour
University / Data
Storage Class
01TChevron
01TShell
01TUT Austin
undergraduate class in
Parallel Computer
Architecture
01TDell and Bell
Helicopter
01TUS Senate Commerce
Committee & Kay
Bailey Hutchison‟s
office
01TPhysics & CS UT
Austin Freshman
Interest Group
01TUT Austin freshman
public affairs class
01TElectrical Engineering
UT Austin freshman
interest group
01TChemistry &
Biochemistry UT
Austin Freshman
01TElectrical & Computer
Engineering UT
freshman students
Austin Community
College class /
Computer Security
Houston Livestock
Show & Rodeo
Committee
01T“Transmedia
Storytelling” with
Daniel Lorenzetti
& JuanGarcia
Originality in the Arts
and Sciences
Aerospace
Engineering UT
Austin freshman year
interest group
Machine
Room
Ranger
Machine
Room and
ROC
Ranger
Machine
Room &
Vislab
Ranger &
CMS
Machine
Rooms
Ranger
Machine
Room &
Vislab
Ranger
Machine
Room
Oct. 11 7 N/A
Oct. 12 3 N/A
Oct. 18 15 Not tracked.
Oct. 19 7 N/A
Oct. 20 3 N/A
Vislab Oct. 20 18 6
Vislab Oct. 21 18 11
Vislab Oct. 27 12 Not tracked.
Vislab Nov. 2 18 6
Vislab Nov. 3 20 5
Ranger
Machine
Room
Nov. 3 17 8
Vislab Nov. 4 20 Not tracked.
AT&T
Executive
Education
and
Conference
Center
Nov. 9 158 Not tracked.
Vislab Nov. 9 16 Not tracked.
Vislab Dec. 1 19 7

The Austin
Forum on
Science,
Technology &
Society
Ranger Tour
“CPRIT:
Opportunities and
Challenges” with
Nobel Laureate Dr. Al
Gilman (Cancer
Prevention Institute
of Texas)
National Instruments
AT&T
Executive
Education
and
Conference
Center
Ranger
Machine
Room
Dec. 7 86 Not tracked.
Dec. 15 5 N/A
Totals 892 72
TACC at SC10
TACC staff members participated in the SC10 conference. 36 staff attended and contributed to
tutorials, provided demonstrations and talks in the TACC booth, assisted in the TeraGrid booth,
and presented at special sessions, BoFs, panels, and in vendor booths. TACC presented 32
posters in its booth, 12 described research projects accomplished by TeraGrid principal
investigators and another 6 described TeraGrid HPC, Vis, and data systems (i.e. Ranger,
Longhorn, Ranch) or other NSF-funded projects (FutureGrid, STCI awards). TACC booth
activities and participation in other SC10 events are summarized in the following tables.
SC10 TACC Booth Activities
Type Title Presenter
Presentation
Demo
Data-centric Cyberinfrastructure and
Applications at the Texas Advanced Computing
Center
The Longhorn Visualization Portal: A Web-
Based Interface to the Longhorn Visualization
and Data Analysis Cluster
Chris Jordan, TACC
Greg Johnson, TACC
Presentation
Parallel Optimization of Large Constrained Least
Squares Problems in a Production Environment
Emre Brooks, UT Health
Science Center at San Antonio
Presentation
Lagrangian Simulation of the Deepwater Horizon
Oil Spill: Application of the Advanced
CIRCulation Model for Oceanic, Coastal, and
Estuarine Waters (ADCIRC)
Corey Trahan, The University
of Texas at Austin
Demo Visualizing the BP Oil Spill Karla Vega, TACC
Presentation Visualizing Science at Scale with Longhorn Kelly Gaither, TACC
Presentation Large Scale Distributed GPU Isosurfacing Paul Navratil, TACC
Presentation
Presentation
NEMO and OMEN: A Journey Through
Massively Parallel Nanoelectronics Modeling
Tools
XTED: Identifying and Evaluating the Next
Generation of Cyberinfrastructure Software for
Science
Sebastien Steiger, Purdue
Maytal Dahan, Jay Boisseau
(TACC)
Presentation Supercomputing to Feed the World Dan Stanzione, TACC

Presentation
Presentation
Extreme-Scale AMR
TACC's Big Projects for Big Science in 2011 and
Beyond
Omar Ghattas, The University
of Texas at Austin
Jay Boisseau, TACC
Presentation Advanced Portals and Services at TACC Steve Mock, TACC
Demo Stereoscopic Molecular Visualization Brandt Westing, TACC
TACC Participation in SC10 Events
Type Location Title Presenter
Special Topic
Special Topic
Grand Challenges in Humanities, Arts, and
Social Sciences Computing
Recruiting and Training HPC Systems
Administrators and Engineers
Maria Esteva
Dan Stanzione
Presentation NVIDIA Booth Large Scale Distributed GPU Isosurfacing Paul Navratil
Special Topic
TACC's Scientific Computing Curriculum:
Supporting Academia and Industry
Presentation Intel Theater TACC Aids Study of Gulf Oil Spill Impact Karla Vega
Paper
Tutorial
PerfExpert: An Easy-to-Use Performance
Diagnosis Tool for HPC Applications
MO2: Hybrid MPI and OpenMP Parallel
Programming
Student Cluster Competition
Melyssa Fratkin, Bill
Barth, Dan Stanzione
B.D. Kim, John McCalpin,
Lars Koesterke
Gabriele Jost
B.D. Kim, Carlos Rosales
(Supervised student team
from UT Austin)

C
Performance on CY2010 Milestones

Work Package Dependencies Planned
QSR 2010Q4 Report Values
QSR 2010Q3 Report Values
QSR 2010Q2 Report Values
QSR 2010Q1 Report Values
Resources 12/31/10 AD Status 12/31/10 AD Notes 09/30/10 AD Status 09/30/10 AD Notes 06/30/10 AD Status 06/30/10 AD Notes 3/31/10 AD Status 03/31/10 AD Notes
QSR 2009Q4 Report Values
12/31/09 AD
12/31/09 AD Notes
Status
Project-ID
WBS
O P M
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
SI 2 Area: Software Integration
Objective 2.0 Operations: maintain/sustain
SI 2.0
current capabilities
SI.Coord 2.0.1 O SI Area Coordination
UC Ian Foster[3%], Lee
SI.Coord 2.0.1.3 O SI Area Coordination PY5 Aug-09 Jul-10 Liming[35%]; ANL JP
Navarro [35%]
SI.Coord 2.0.1.3.1 M PY 5 Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-09 Jul-10 100% 100% 75% 50% 25%
UC Ian Foster[3%], Lee
SI.Coord 2.0.1.4 O SI Area Coordination Extension Aug-10 Jul-11 Liming[35%]; ANL JP
Navarro [35%]
SI.Coord 2.0.1.4.1 M Extension Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-10 Jul-11 50% 0%
SI.Inf 2.0.2 O Operate Infrastructure Services
SI.Inf 2.0.2.3 O Operate Infrastructure Services PY5 Aug-09 Jul-10
ANL JP Navarro [2%];
Eric Blau[2%]; NCSA
Jason Brechin[5%];
TACC Warren Smith
[10%]
SI.Inf 2.0.2.3.1 M PY5 Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-09 Jul-10 100% 100% 75% 50% 25%
SI.Inf 2.0.2.4 O Operate Infrastructure Services Extension Aug-10 Jul-10
ANL JP Navarro [2%];
Eric Blau[2%]; NCSA
Jason Brechin[5%];
TACC Warren Smith
[10%]
SI.Inf 2.0.2.4.1 M Extension Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-10 Jul-11 50% 0%
SI.CTSS 2.0.3 O Maintain Current CTSS Kits
UC Eric Blau[45%], Lee
Liming[15%],
TBD[20%]; NCSA Jason
Brechin[14%]; TACC
SI.CTSS 2.0.3.3 O Maintain Selected Current CTSS Kits PY5 Aug-09 Jul-10 Warren Smith[10%];
ANL JP Navarro [10%];
UW TBD[7%], Jaime
Frey[7%], Greg
Thaine[6%]
SI.CTSS 2.0.3.3.1 M PY5 Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-09 Jul-10 100% 100% 75% 50% 25%
More RPs are in the
More RPs are in the
SI.CTSS 2.0.3.4 P Globus Tool Kit 5 - GRAM, GridFTP, and Client Oct-09 Jul-10 100%
process of installing,
100%
process of installing,
In beta, new release
70%
40% Still on original target 20% New Effort Added 2009Q4
closed and tracked under
closed and tracked under
being prepared for RPs
entension line
entension line
UC Eric Blau[45%], Lee
Liming[15%],
TBD[20%]; NCSA Jason
Brechin[14%]; TACC
SI.CTSS 2.0.3.5 O Maintain Selected Current CTSS Kits Extension Aug-10 Jul-11 Warren Smith[10%];
ANL JP Navarro [10%];
UW TBD[7%], Jaime
Frey[7%], Greg
Thaine[6%]
SI.CTSS 2.0.3.5.1 M Extension Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-10 Jul-11 50%
Talking about talking
GRAM to production,
More RPs are in the
SI.CTSS 2.0.3.4 P Globus Toolkit 5 - GRAM, GridFTP, and Client Aug-10 Jul-11 50%
GridFTP is production 10%
process of installing
and partically deployed.
Client close to prod
SI.Pkg 2.0.4 O Package Software
ANL JP Navarro
[15%];NCSA Jason
Brechin[15%]; UC TBD
[10%], Eric Blau[44%],
SI.Pkg 2.0.4.3 O Package Software PY5 Aug-09 Jul-10
Joseph Bester[35%];
UW TBD [10%], Jaime
Frey [10%], Greg
Th i [10%]
SI.Pkg 2.0.4.3.1 M PY5 Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-09 Jul-10 100% 100% 75% 50% 25%
SI.Pkg 2.0.4.4 P Local Compute Oct-09 Mar-10 Completed Completed Completed Completed 50% New Effort Added 2009Q4
SI.Pkg 2.0.4.5 P Distributed Programming Jul-09 Jun-10 Completed On target Completed On target Completed On target 75% On target 50% New Effort Added 2009Q4
ANL JP Navarro
[15%];NCSA Jason
Brechin[15%]; UC TBD
[10%], Eric Blau[44%],
SI.Pkg 2.0.4.6 O Package Software Extension Aug-10 Jul-11
Joseph Bester[35%];
UW TBD [10%], Jaime
Frey [10%], Greg
Th i [10%]
SI.Pkg 2.0.4.6.1 M Extension Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-10 Jul-11 50%
SI.Pkg 2.0.4.7 O Continue to package software Aug-10 Jul-11 50% 15%
Objective 2.1 Projects: Information Services
SI 2.1
Enhancements
ANL JP Navarro [30%];
NCSA Jason
SI.IS 2.1.4 P Information Services Enhancements PY5 Aug-09 Jul-10 Brechin[10%]; SDSC
Ken Yashimoto[15%];
UC Eric Blau[10%],
TBD[20%]
demonstrate ability for resource operators to register
SI.IS 2.1.4.1 P Core2 Aug-09 Jul-10
their own locally supported capabilities
Low priority because
SI.IS 2.1.4.1.2 P Capability documented, and registration designed Nov-09 Jan-10 55%
RPs are not requesting 55%
this capability
Cornell MATLAB
SI.IS 2.1.4.1.3 P Prototype registration(s) deployed Feb-10 Apr-10 100%
25%
registration
SI.IS 2.1.4.1.4 P Production registration(s) deployed May-10 Jul-10 100%
evaluate the feasibility of gateways publishing about
SI.IS 2.1.4.2 P SGW Aug-09 Jul-10
themselves to information services
Low priority because
RPs are not requesting
this capability
Cornell MATLAB
registration
50%
Low priority because
RPs are not requesting
this capability
50% 50% Draft doc to be circulated to RPS

Nancy working with Doc
Dependent on gateway
Dependent on gateway
in info serv teams to
team to complete their
team to complete their
SI.IS 2.1.4.2.4 P Publishing evaluation and next step decision May-10 Jul-10 85%
have a way to let 80%
80%
On-going
part before this can be
part before this can be
gateways publish
completed.
completed.
themselves
respond to data, gateways, visualization, and
From
SI.IS 2.1.4.3 P scheduling needs for expanded information service
Mar-09 Jul-10 100% 100% 75% 75% 50%
SGW, DV
content
ANL JP Navarro [30%];
NCSA Jason
Brechin[10%]; SDSC
SI.IS 2.1.5 Information Services Enhancements Extension Apr-10 Jul-11
Ken Yashimoto[15%];
UC Eric Blau[10%],
TBD[20%]
Track2D, Viz, and storage capability deployment
Working with Viz and
SI.IS 2.1.5.1 O Apr-10 Jul-11 25%
15%
registration
data working groups
SI.IS 2.1.5.2 O New TG capabilities are registered Apr-10 Jul-11 50% 15%
SI.IS 2.1.5.3 O Community capabilities are registered Apr-10 Jul-11 50% 15%
SI.IS 2.1.5.4 P IIS transition to XD Apr-10 Jul-11 10% 0%
SI 2.2
Objective 2.2 Projects: Application Service Hosting
Partnership
Kate Keahy[20%],
Lee Liming[5%]
SI.Host 2.2.2 P Application Hosting Services Partnership V1 PY4 Aug-08 Jul-09
8/3 Update: Percentage 75%, The imp
Docs
docs are no longer required. The
Application Hosting Service Description &
SI.Host 2.2.2.1 M Team, Aug-08 Jul-09 75% Lee to talk to Kate 75% Lee to talk to Kate 75% Lee to talk to Kate 75% Lee to talk to Kate 75% description document has mostly been
Implementation Documents
SGW
completed. This still needs a small
amount work to complete
IS Schema proto available, no INCA test
SI.Host 2.2.2.2 M Coordinate Inca, IS Schema for Hosting Service NOS Jul-09 Oct-09 Lee Liming[6%] 40% Lee to talk to Kate 40% Lee to talk to Kate 40% Lee to talk to Kate 40% Lee to talk to Kate 40%
yet
SI.Host 2.2.3 P Application Hosting Services Partnership V2 PY5 Aug-09 Jul-10 UC Kate Keahey[25%]
SI.Host 2.2.3.1 M Documentation V2 and transition plan Aug-09 Jul-10
SI.Host 2.2.3.1.1 M Analyze V1 experiences and produce V2 goals, plan Aug-09 Oct-09 0% Low priority 0% Low priority 0% Low priority 0% Low priority 0% Low priority
SI.Host 2.2.3.1.2 M Develope V1->V2 transition plan Feb-10 Apr-10 0% Low priority 0% Low priority 0% Low priority 0% Low priority
SI.Host 2.2.3.1.3 M Documnent V2 May-10 Jul-10 0% 0% 0%
SI.Host 2.2.3.2 M Coordinate changes and deployment Aug-09 Jul-10
To RPs
SI.Host 2.2.3.2.1 M Collect V1 RP experiences Aug-09 Oct-09 0% Low priority 0% Low priority 0% Low priority 0% Low priority 0% Low priority
SI.Host 2.2.3.2.2 M Coordinate V2 testing deployment Feb-10 Apr-10 0% 0% 0%
SI.Host 2.2.3.2.3 M Coodinate V2 production deployment May-10 Jul-10 0% 0% 0%
Objective 2.3 Projects: Public Build and Test
SI 2.3
Service
ANL JP Navarro[8%];
SI.STCI 2.3.2 P TeraGrid STCI User Support PY5 Aug-09 Jul-10
UC Eric Blau[9%]
SI.STCI 2.3.2.1 M Feasibility Evaluation for Community Usage Aug-09 Jan-10
Analyze early user experiences and summarize, assess
SI.STCI 2.3.2.1.1 M
community interest
Aug-09 Oct-09 10%
Due to staff losses and
lag in hiring
10%
replacements these task
are behind schedule.
Due to staff losses and
lag in hiring
10%
replacements these task
are behind schedule.
Due to staff losses and
lag in hiring
10%
replacements these task
are behind schedule.
Initial discussions have
taken place
10% Initial discussions have taken place
SI.STCI 2.3.2.1.2 M Produce evaluation report and conclusion Nov-09 Jan-10 0% 0% 0% 0%
SI.STCI 2.3.2.2 M Standalone Build & Test Capability Kit Feb-10 Jul-10
2.3.2.1
SI.STCI 2.3.2.2.1 M Capability kit beta testing, produce documentation Feb-10 Apr-10
SI.STCI 2.3.2.2.2 M Capability kit production rollout May-10 Jul-10
SI 2.5 Objective 2.5 Projects: Scheduling
Suman Nadella[100%],
Nicholas Trebon[100%],
SI.SPRC 2.5.2.17 P SPRUCE PY4 To RPs Aug-08 Jul-09
Pete Beckman [5%],
On-demand compute capability documented and
SI.SPRC
M
67% 67% 67% 67% 67%
integrated into production operations on RP systems
2.5.2.17.2
UC Suman
Nadella[40%], Nicholas
SI.SPRC 2.5.2.18 P SPRUCE PY5 To RPs Aug-09 Jul-10 Trebon[35%], Pete
Beckman [5%]
Deploy a production service for guaranteed network
bandwidth for urgent computations.
Deploy Spruce for virtualized cloud infrastructure
(collaboration with Eucalyptus).
SI.SPRC 2.5.2.18.3 P Work with users from VT to utilize Spruce for H1N1
Feb-10 Apr-10
influenza simulations on cloud infrastructures
Work with Purdue and other cloud providers to support
Contigent on Pete's
Spruce.
100% 100% 100%
report
Lee to check with Pete
Maintain Spruce infrastructure (i.e., web service server,
database, user/admin portals).
Update Spruce plug-ins at resource-end as new
schedulers are introduced.
SI.SPRC 2.5.2.18.4 P Work to get Spruce supported at new TeraGrid
May-10 Jul-10
resources that have replaced retired machines that
supported Spruce (e.g., Frost).
67% Lee to talk to Pete B. 67% Lee to talk to Pete B. 67%
SI.Skd 2.5.3 O Scheduling Working Group
TACC Warren Smith
SI.Skd 2.5.3.3 O Scheduling Working Group Coordination PY5 Aug-09 Jul-10 100% 100% 75% 50%
[25%]
TACC Warren Smith
SI.Skd 2.5.3.4 O Scheduling Working Group Coordination Extension Aug-10 Jul-11
[25%]
SI.QBETS 2.5.9 P QBETS PY4 moved to PY5 Aug-09 Feb-10 Warren
SI.QBETS 2.5.9.2
Queue prediction capability documented and integrated
P
into TeraGrid's production operations
Per Warren: TG Portal
Lee to talk to Warren
Lee to talk to Warren
Lee to talk to Warren
SI.QBETS 2.5.9.2.1 P Queue Predicition Capability Documented Aug-09 Nov-09 60%
doc complete. In 60%
about Karnak
60%
about Karnak
60%
about Karnak
process of adding doc to
deployment
deployment
deployment
Per Warren: Karnak is
Queue Predicition Capability integrated into TG
Not formally complete
Not formally complete
Not formally complete
SI.QBETS 2.5.9.2.2 P Nov-09 Feb-10 25%
now beta with an up 25%
25%
0%
operations
but available for use
but available for use
but available for use
time of over 90%
NICS Troy Baer[6%];
SI.MTS 2.5.10 P Metascheduling PY5 Aug-09 Jul-10
SDSC Kenneth
Yashimoto[35%]; TACC
Warren Smith[55%]
SI.MTS 2.5.10.1 P Detailed PY5 metascheduling plan Aug-09 Oct-09 Completed Completed Completed Completed
8/3 Update: On-demand capability has
already been implement by a number of
RPs, documentation and awareness is
still needed for this implementation.
This will be completed by Mar 10

SI.MTS 2.5.10.2 P Metascheduling capabilities developed Nov-09 Jan-10
Condor-G/class AD
Condor-G/class AD
Condor-G/class AD
Condor-G/class AD
Completed
metascheduling Completed
metascheduling Completed
metascheduling 80%
metascheduling
SI.MTS 2.5.10.3
Testing automated and dynamic (information services
P Feb-10 Apr-10
based) metascheduling
Completed Completed Completed Completed
SI.MTS 2.5.10.4
Production automated and dynamic (information
P May-10 Jul-10
services based) metascheduling
Completed Lee to talk to Warren Lee to talk to Warren Lee to talk to Warren
NICS Troy Baer[6%];
SI.MTS 2.5.11 P Metascheduling Extension Apr-10 Jul-11
SDSC Kenneth
Yashimoto[35%]; TACC
Warren Smith[55%]
SI.MTS 2.5.11.1
Advanced scheduling tools ported to new TeraGrid
O
systems as appropriate per RP wishes 63% 38% 13%
Best practice documentation for using advanced
SI.MTS 2.5.11.2 P scheduling capabilities in conjunction with non-
TeraGrid systems
0% Not started 0% Not started 0% Not started
Improved Coordinated Capability User
SI.ICUD 2.7 P Aug-09 Jul-10
Documentation PY5
SDSC Fariba Fana[15%]
SI.ICUD 2.7.1
High-level documentation layout designed, detailed
JP has talked to the doc
JP needs to talk to doc
JP needs to talk to doc
JP needs to talk to doc
P Aug-09 Oct-09
project tasks identified, scheduled, and assigned
5%
team 0%
team 0%
team 0%
team
We are using this line
Capability kit introduction/use case documentation
item as a method to
SI.ICUD 2.7.2 P Nov-09 Jan-10
improvements
move doc off of the wiki
10%
and to doc for XD 0% 0%
SI.ICUD 2.7.3
Half of capability kit detailed documentation
P Feb-10 Apr-10
improvements including IS integration
0% 0% 0%
SI.ICUD 2.7.4
Half of capability kit detailed documentation
P May-10 Jul-10
improvements including IS integration
0% 0% 0%
0%
Contigent on LifeRay implementation
dependant
SI
END SI

Work Package Dependencies Planned
QSR 2010Q4 Report Values
QSR 2010Q3 Report Values
Resources 12/31/10 AD Status 12/31/10 AD Notes 9/30/10 AD Status 9/30/10 AD Notes
Project-ID
WBS
OPMD
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
SGW 1 Area: Science Gateways
SGW 1.0
Objective 1.0 Operations: maintain/sustain
current capabilities
SGW.Coord 1.0.1.2 O SGW Area Coordination PY4 Aug-08 Jul-09 Nancy Wilkins-Diehr[50%], Ongoing Ongoing
In Progress
Drafted revisions to TeraGrid's community In progress
SGW.Coord
D
Standardize security approach to gateways.
Among gateways with roaming allocations, usage
of multiple TeraGrid resources will increase.
Mar-11
account policy document, TG-10. Reviewing
with co-authors Victor Hazlewood and Jim
Marsteller. Expected completion March,
SGW.Coord
D
Gateway Web services registry. It will be possible
to discover services available through Gateways
programmatically
Jul-11
SGW.Coord 1.0.1.3 O SGW Area Coordination PY5 Aug-09 Jul-10
2011
RENCI Delayed Work delayed to due SDSC staff medical
leave.
Nancy Wilkins-Diehr[50%],
SGW.Coord 1.0.1.3.1 M Quarterly Reporting (9/30, 12/31, 3/31, 6/30) Aug-09 Jul-10 Stuart Martin[50%]
Ongoing Ongoing
SGW.GDoc 1.0.2 O Gateway Documentation May-07 Jul-10
SGW.GDoc 1.0.2.1.10 P Use of GRAM audit Dec-07 Mar-08 In Progress
Stuart Martin checking TeraGrid's GRAM
documentation to see what changes are
needed for GRAM5 transition
announcement
In progress
SGW.GDoc 1.0.2.2 O Gateway Documentation PY4 Aug-08 Jun-09 Diana Diehl[50%]
Provide content for documentation, formation of
Attention turned to Web services registry
SGW.GDoc 1.0.2.2.2 M ideas and solutions to common gateway problems
Aug-08 Sep-08 Effort redirected
where good progress was made, funding for Effort redirected
(RENCI);
RENCI has ended in the Extension year
SGW.GDoc 1.0.2.3 O Gateway Documentation PY5 Aug-09 Jul-10
SDSC Diana Diehl[20%];
UNC Jason Reilly [40%],
John McGee [5%]
SGW.GDoc 1.0.2.2.1 M Up to date documentation on science gateways. Aug-09 Jul-11 Ongoing Activity will continue in the extensio Ongoing
SGW.GDoc 1.0.2.2.2 M Anatomy of Gateway tutorial Aug-09 Jul-10 Completed Completed
SGW 1.1
Objective 1.1 Projects: Gateways Targeted
Support Program
SGW.HDN 1.1.2.4 O Helpdesk PY5 Aug-09 Jul-10 NCSA Yan Liu[20%]
SGW.HDN 1.1.2.5 O
Provide helpdesk support for production science
gateways by answering user questions, routing
Aug-09 Jul-11
user requests to appropriate gateway contacts,
and tracking user responses
Ongoing
Activity will continue in the extensio Ongoing
Provide the helpdesk support of the SimpleGrid
SGW.HDN 1.1.2.6 O online training and prototyping services to gateway
Aug-09 Jul-11
developers and communities
Ongoing
Activity will continue in the extensio Ongoing
SGW.HDN 1.1.2.7 P
improve gateway helpdesk services.
Gather requirements for a knowledge base to
Feb-10 Jul-11 Completed Activity will continue in the extensio Completed
SGW.CIG 1.1.6.3 P CIG support PY5 Aug-09 Jul-10
SGW.CIG 1.1.6.4 P
SGW.CIG 1.1.6.5 P
SGW.CIG 1.1.6.6 P
SGW.CIG 1.1.6.7 P
Work with NOAO to develop a TG portal that
allows users to run customized galaxy collision
simulations using the power of the TeraGrid
A scriptable interface to portal services will be
created.
Extend the CIG portal with an easy-to-use
language-agnostic RPC-styleinterface that
parallels the functionality of the web-based
interactiveportal (and extends it where
appropriate)
Provide bindings from this interface into popular
languages (driven by user demand) such as C,
Pythonand Perl.
CalTech Julian Bunn [10%],
John McCorquodale [25%],
Roy Wiliams [5%]
Aug-09 Jul-10 Effort redirected to Arroyo
gateway
Aug-09 Oct-09 Effort redirected to Arroyo
gateway, tasks forthcoming
Nov-09
SGW.Bio 1.1.10.3 P Bio web services, registry PY4 Aug-08 Jul-09
SGW.Bio 1.1.10.3.4 M
Work with the various TeraGrid teams to evolve
the registry to meet the broader TeraGrid
requirements (RENCI);
SGW.Bio 1.1.10.3.5 M
Work with other gateways to incorporate their
service offerings into the registry (RENCI);
Jan-10
Effort redirected to Arroyo
gateway, tasks forthcoming
Feb-10 Apr-10 Effort redirected to Arroyo
gateway, tasks forthcoming
Jason Reilly[50%],
Mats Rynge[25%]
John McGee[5%]
Arroyo work completed,
collaboration with Caltech has
ended in the Extension year
Arroyo work completed,
collaboration with Caltech has
ended in the Extension year
Arroyo work completed,
collaboration with Caltech has
ended in the Extension year
In progress
To IS, NOS Aug-08 Jul-09 Completed Completed
Effort redirected to Arroyo
gateway
Effort redirected to Arroyo
gateway, tasks forthcoming
Effort redirected to Arroyo
gateway, tasks forthcoming
Effort redirected to Arroyo
gateway, tasks forthcoming
Aug-08 Jul-09 Completed Completed
SGW.GC 1.1.12 P GridChem or Other New PY5 Aug-09 Jul-10 IU Suresh Marru [50%]
SGW.GC 1.1.12.1 P
Complete Gateway clients as determined by the
Targeted Support Program
Jan-10 Jul-10 Completed Completed
SGW.GC 1.1.12.2 P
Ongoing support as required for existing Gateway
projects
Jan-10 Jul-10 Completed Completed
SGW.GC 1.1.12.3 P
Assist GridChem gateway in the areas of software
access, data management, improved workflows,
visualization and scheduling
Aug-09 Dec-09
Completed
Completed
SGW.PG 1.1.13 P PolarGrid or Other New PY5 Aug-09 Jul-10 IU SGW TBD1 [100%]
Complete Gateway clients as determined by the
SGW.PG 1.1.13.1 P Jan-10 Jul-10
Targeted Support Program
Completed Completed
End date extended to Jul-10
End date extended to Jul-10
GRAM5 (an attribute dependency for many)
being installed for testing across TG
Attention turned to Web services registry
where good progress was made, funding for
RENCI has ended in the Extension year
Arroyo work completed,
collaboration with Caltech has
ended in the Extension year
Arroyo work completed,
collaboration with Caltech has
ended in the Extension year
Arroyo work completed,
collaboration with Caltech has
ended in the Extension year

Work Package Dependencies Planned
QSR 2010Q4 Report Values
QSR 2010Q3 Report Values
Resources 12/31/10 AD Status 12/31/10 AD Notes 9/30/10 AD Status 9/30/10 AD Notes
Project-ID
WBS
OPMD
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
SGW.PG 1.1.13.2 P
Ongoing support as required for existing Gateway
projects
Jan-10 Jul-10 Completed Completed
Assist PolarGrid team with TeraGrid integration.
PolarGrid's Matlab-based code
was rewritten to remove
dependencies on the matlab
PolarGrid's Matlab-based code
was rewritten to remove
dependencies on the matlab
May include realtime processing of sensor data,
SGW.PG 1.1.13.3 P
Aug-09 Jul-10
distributed toolkit. It now runs on
distributed toolkit. It now runs on
support for parallel simulations, GIS integration
and EOT components.
the Quarry cluster in a much more
the Quarry cluster in a much more
Completed
scalable and less expensive
fashion.
Completed
scalable and less expensive
fashion.
SGW.SID 1.1.14 P SIDGrid or Other New PY5 Aug-09 Jul-10 UC Wenjun Wu [100%]
SGW.SID 1.1.14.1 P
SIDGrid scheduling will be enhanced by taking
advantage of work underway in the Scheduling
Working Group to enable monitoring of and
Aug-09 Oct-09
submission to multiple TeraGrid resources through
a single interface
Completed Using Swift Completed Using Swift
SGW.SID 1.1.14.2 P
Improved security model for community accounts
utilized by the gateway
Nov-09 Jan-10 Completed Completed
SGW.SID 1.1.14.3 P
Increased number of analysis codes available
through the gateway
Feb-10 Apr-10 Completed Completed
SGW.SID 1.1.14.4 P
Isolation of execution within virtual machines or
other constrained environments, and data sharing
with collaborators, while simultaneously protecting
the users' data and execution environment from
accidental or malicious compromise.
Feb-10 Apr-10
Completed
Completed
SGW.SID 1.1.14.5 P
VDS, the current workflow solution, will be
replaced by its successor, SWIFT.
May-10 Jul-10 Completed Completed
Documentation and code that is generally
SGW.SID 1.1.14.6 P applicable to gateway efforts will be made
May-10 Jul-10
available to TeraGrid gateway partners
Completed
Completed
SGW.OSG 1.1.15 P RENCI-OSG PY5 Aug-09 Jul-10
SGW.OSG 1.1.15.1 P
SGW.OSG 1.1.15.2 P
Prototype of OSG jobs running on TeraGrid
resources via NIMBUS
Successful interactions with and assistance for
gateway developers
SGW.ES 1.1.16 P Environmental Science Gateway PY5 Aug-09 Jul-10
SGW.ES 1.1.16.1 P
Release of a demonstration version of the ESGC
Portal capable of invoking CCSM runs on the
TeraGrid
UNC Jason Reilly [15%],
John McGee [5%]
Aug-09 Oct-09
This work will not be completed,
effort redirected to Web services
Effort redirected registry
Effort redirected
Nov-09 Apr-10 Completed Completed
Aug-09
Oct-09
NCAR NCAR SGW TBD
[50%] ;Purdue PU SGW
TBD1[50%]
Effort redirected
The team has reevaluated plans
for an ESGC portal and is instead
working on a demonstration
version of the CCSM portal
capable of invoking CCSM4 runs
on the TeraGrid. We have
implemented support for basic
operations such as case creation
and configuration. A prototype
version is expected to complete in
November 2010.
Effort redirected
This work will not be completed,
effort redirected to Web services
registry
The team has reevaluated plans
for an ESGC portal and is instead
working on a demonstration
version of the CCSM portal
capable of invoking CCSM4 runs
on the TeraGrid. We have
implemented support for basic
operations such as case creation
and configuration. A prototype
version is expected to complete in
November 2010.

Work Package Dependencies Planned
QSR 2010Q4 Report Values
QSR 2010Q3 Report Values
Resources 12/31/10 AD Status 12/31/10 AD Notes 9/30/10 AD Status 9/30/10 AD Notes
Project-ID
WBS
OPMD
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
SGW.ES 1.1.16.2 P
Ability for runs invoked on the Purdue Climate
Portal or the ESGC Gateway to be published back
to the ESG data holdings
SGW.ES 1.1.16.3 P
Investigate federation of Purdue data holdings
with ESG data archives
SGW 1.2 Objective 1.2 Projects: Gateways User Count
Nov-09
SGW.UCnt 1.2.2 P Gateway User Count PY4 To NOS Aug-08 Jul-09
SGW.UCnt
D
End gateway users can be counted
programmatically, as opposed to current manual
aggregation method. By the end of PY5, all jobs
submitted via community accounts will include
attributes identifying the end user of that gateway.
SGW.UCnt 1.2.3 P Gateway User Count PY5 NOS Aug-09 Jul-10
SGW.UCnt 1.2.3.1 M
Develop internal web pages for the security
working group and the science gateway
administration page in support of attribute-based
authentication
Jul-11
In progress
See 9/30/10 notes. End Date
changed to July, 2011.
In progress
Feb-10 Jul-10 Completed Completed
Aug-09
Jul-11
Dec-09
Tom Scavo[75%],
Jim Basney[10%],
Nancy Wilkins-Diehr[5%]
NCSA Tom Scavo[50%], Jim
Basney[10%], Terry Fleury
[25%]; SDSC Nancy Wilkins-
Diehr[5%], Michael Dwyer
[10%]
Delayed
See 9/30/10 notes, end date changed to
July, 2011.
Delayed
(1) Model output can be published
to ESG either from local file system
or via iRODS fuse interface. There
is still some work going on to
automate the process and make
the system more robust/scalable.
(2) Publishing of model metadata is
delayed because of schedule
change on the CESM and ESG
sides. We are testing a beta
version of the CESM model that
has the capability to collect model
run attributes. A system design on
how to publish the metadata back
to ESG has been defined. Still
need to implement and test it.
Publishing runs into ESG has been
demonstrated.
GRAM5 now being installed across TG
Complete
Complete
SGW.UCnt 1.2.3.2 M Ubiquitous science gateway adoption. Aug-09 Dec-09 Delayed See 9/30/10 notes Delayed Waiting for GRAM5 install
SGW.UCnt 1.2.3.3 M Ubiquitous RP adoption. Feb-09 Dec-09 Delayed See 9/30/10 notes Delayed Waiting for GRAM5 install
SGW.UCnt 1.2.3.4 M User count INCA tests Jan-10 Feb-10 Complete Complete
1.2.3.5 M Post-TG architecture documentation. Feb-10 Jul-10 Deferred Waiting for XD architecture decisions
SGW 1.4
Objective 1.4 Projects: Gateways General
Services Discovery
SGW.SG 1.4.2 P
SimpleGrid, helpdesk, accounting, new
communities PY4
Reusable SijmpleGrid modules. TeraGrid science
gateway program will open an online learning
service for gateway development by integrating
SGW.SG
D
SimpleGrid documentation, progressive online
courses, and common components contributed
from other existing gateways. All the training,
coding, and prototyping exercises will be
conducted through web browser.
SGW.SG 1.4.1.1 P
SimpleGrid, helpdesk, accounting, new
communities PY5
Continue to develop and support the SimpleGrid
SGW.SG 1.4.1.1.1 M online training service for building new science
gateways
Develop a SimpleGrid prototyping service to
support virtualized access to TeraGrid by
SGW.SG 1.4.1.1.2 M efficiently converting a new community application
to a science gateway application that provides web
access for sharing within the community.
Develop a streamlined packaging service for new
SGW.SG 1.4.1.1.3 M
communities to create their science gateway
software based on the use of the SimpleGrid
prototyping service
To NOS
Aug-08
Aug-09
Aug-09
Aug-09
Feb-10
Jul-09
Shaowen Wang[8%],
Yan Liu[80%]
Jul-11 NCSA In progress
Jul-10
Jul-11
Jul-11
Jul-11
NCSA Shaowen Wang[10%],
Yan Liu[80%]
Ongoing
In progress
In progress
Using CI Tutor, an extension year activity,
end date changed to Jul-11
Delayed
Activity will continue in the extension Ongoing
Activity will continue in the extension In progress
Activity will continue in the extension In progress
Will be using CI Tutor for this in the
extension year, CI Tutor work in progress
Work will complete in the extension
year
VM deployment in th extension
year

Work Package Dependencies Planned
QSR 2010Q4 Report Values
QSR 2010Q3 Report Values
Resources 12/31/10 AD Status 12/31/10 AD Notes 9/30/10 AD Status 9/30/10 AD Notes
Project-ID
WBS
OPMD
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
SGW.SG 1.4.1.1.4 M
Develop a user-level TeraGrid usage service
within SimpleGrid based on the community
account model and attributes-based security
services.
Feb-10
SGW.SG 1.4.1.1.5 M
Work with potential new communities to improve
the usability and documentation of the proposed
gateway support services
Aug-09 Jul-10
Conduct education and outreach work using the
SimpleGrid online training service and related
SGW.SG 1.4.1.1.6 M science gateway technologies in the contexts of
Aug-09 Jul-10
undergraduate, graduate, and K-12 education and
training and document experiences
SGW.CA 1.4.5 P Community Accounts PY5 Aug-09 Jul-10
Jul-11
NICS Victor
Hazlewood[30%];SDSC
Nancy Wilkins-Diehr[15%]
Delayed
Complete
Waiting for GRAM5 install and RP
accounting scripts to send
individual user attributes to the
TGCDB, changed end date to Jul-
11. Deferred Waiting for GRAM5 install
Complete
Complete SciDAC, TG10 Complete SciDAC, TG10
SGW.CA 1.4.5.1 M Review and documentation of requirements Aug-09 Oct-09 Complete Complete
SGW.CA 1.4.5.2 M Communication of requirements Nov-09 Jan-10 Complete Complete
SGW.CA 1.4.5.3 M Specification of a standard deployment model(s) Feb-10 Apr-10 Complete Complete
SGW.CA 1.4.5.4 M Deployment of standard model on one gateway May-10 Jul-10 Complete Complete
SGW.GCD 1.4.6 P Gateways Code Discovery PY5 Aug-09 Jul-10
SDSC Nancy Wilkins-
Diehr[5%]; UNC Josh Coyle
[30%] John McGee [5%]
SGW.GCD 1.4.6.1 P
RENCI Gateway Services advertised appropriately
in TG Info Services
Aug-09 Aug-09 Complete Complete
SGW.GCD 1.4.6.2 P Schema for Gateway services and capabilities Aug-09 Sep-09 Complete Complete
SGW SGW END

TeraGrid US Extension Year Project Plan
Project-ID
WBS
O P M
Description
Work Package Dependencies Planned
Task
Cross
Area
Start
Date
End
Date
Resources
Name [%FTE]
2010Q4
Status
Update
US 6.0 Operations: Frontline User Support
US.Coord 6.0.1 O US Area Coordination PSC Sergiu Sanielevici (30%)
0 6.0.1.3 O US Area Coordination
user interface
council
AUS, NOS,
UFP
US.Eng 6.0.2 O User Engagement Aug-09 Jul-11
Aug-10 Jul-11 ongoing
PSC David O'Neal(25%),
Richard Raymond(20%),
Raghurama Reddy(45%),
Sergiu Sanielevici (30%)
US.Eng 6.0.2.1 O User Champion Coordination Aug-09 Jul-11 ongoing
US.Eng 6.0.2.2 O Campus Champion Coordination Aug-09 Jul-11 ongoing
US.Eng 6.0.2.3 O Startup and education grant support coordination Aug-09 Jul-11 ongoing
US.Eng 6.0.2.4 O Analyze and report usage patterns Aug-09 Jul-11 ongoing
US.Eng 6.0.2.5 P TG User Survey PY5 question input All Aug-09 Jul-10 external contractor DONE
PSC David O'Neal(10%), ongoing
US.Tick 6.0.3 O Share and maintain best practices for ticket resolution across all RPs PY5 Aug-09 Jul-11
Richard Raymond(30%),
Raghurama Reddy(10%),
Sergiu Sanielevici (10%)
US.Tick 6.0.3.1 O
Focus on providing users with a substantive clarification of the nature of the problem and
the way forward
Aug-09 Jul-11 ongoing
US.Tick 6.0.3.2 O Focus on the coordinated resolution of problems spanning RPs and systems Aug-09 Jul-11 ongoing
US.Tick 6.0.3.3 O Stale ticket count reaches zero Aug-09 Jul-11 ongoing
US.Tick 6.0.3.4 O At least 85% survey ratings for promptness and quality Aug-09 Jul-11 ongoing
Estabrook [100%], ongoing
US.Cnslt.RP.NCSA 6.0.5.5 O NCSA RP User Services - Consulting Operations Apr-10 Jul-11 Jackson[100%], John [100%],
US.Cnslt.RP.IU 6.0.5.6 O IU RP User Services - Consulting Operations Apr-10 Jul-11 ongoing
US.Cnslt.RP.LONI 6.0.5.7 O LONI RP User Services - Consulting Operations Apr-10 Jul-11 Jundt [50%], Xu [100%] ongoing
Loftis [50%], Lucio [100%], ongoing
US.Cnslt.RP.NICS 6.0.5.8 O NICS RP User Services - Consulting Operations Apr-10 Jul-11
Sharkey [100%], Wong [50%],
Halloy [50%], Crosby [50%],
TBD [250%]
Blood[25%], Costa[90%], ongoing
US.Cnslt.RP.PSC
US.Cnslt.RP.PU
6.0.5.10
6.0.5.11
O
O
PSC RP User Services - Consulting Operations
PU RP User Services - Consulting Operations
Apr-10
Apr-10
Jul-11
Jul-11
Gomez[45%], Madrid[35%],
Maiden[20%], Nigra[70%],
Raymond[3%],
Sanielevici[2%] Wang[5%]
P. Smith [20%], New person
[100%]
ongoing
US.Cnslt.RP.SDSC 6.0.5.12 O SDSC RP User Services - Consulting Operations Apr-10 Jul-11
Choi [5%], Greenberg [15%],
Tatineni [55%], Wolter [100%]
ongoing
ongoing
US.Cnslt.RP.TACC 6.0.5.13 O TACC RP User Services - Consulting Operations Apr-10 Jul-11
Turner [40%], Wilson [40%],
Sadahiro [50%]
ongoing

Project-ID
WBS
OPMD
TeraGrid AUS PY4-PY5 Project Plan
Work Package Dependencies Planned
Description
Task
Cross
Area
Start
Date
End
Date
Resources
Name [%FTE]
12/31/10 AD
Status
QSR 2010Q4 Report Values
12/31/10 AD Notes
9/30/10 AD
Status
QSR 2010Q3 Report Values
9/30/10 AD Notes
AUS 10 Area: Advanced User Support
10.0 AUS Area Coordination
AUS.Coord 10.0.2 O AUS Area Coordination PY5 Aug-09 Jul-10
Majumdar [75%],Alameda[0%], Sanielevici
[0%],Barth[0%], Cheesman[0%],Jundt[0%],
Crosby[0%]
Total 0.75
10.1 Advanced Support Projects
AUS.LIB 10.1.1 P Library Database PY5 Aug-09 Jul-10
NICS Mark Fahey[10%], Intern[50%], HPC Ops
Staff[10%]
AUS.LIB 10.1.1.1 P implement wrappers for ld and aprun Aug-09 Oct-09 100% done 100% done
AUS.LIB 10.1.1.2 P test wrappers; write mining scripts Nov-09 Jan-10 100% done 100% done
AUS.LIB
10.1.1.3 P
wrappers in production mode; implement internal
web interface; share beta release with other
interested sites Feb-10 Apr-10 100% done
AUS.LIB
everything in production; begin using as a tool to
10.1.1.4 O
help manage software; prepare full release;
share with other interested sites May-10 Jul-10
AUS.LIB 10.1.1.5 O Library Database PY6 Aug-10 Jun-11
AUS.MPI 10.1.2 P MPIg Support PY5 Aug-09 Jul-10
NIU Undergrad[29%], Grad[75%], Nick
Karonis[25%]
Work with ASTA apps groups to port to MPIg on
TG sites. Continue investigation of running apps
AUS.MPI 10.1.2.1 P at full scale on Ranger. Aug-09 Oct-09
AUS.MPI 10.1.2.2 P Attempt cross-site runs with MPIg on TG sites. Nov-09 Jan-10
AUS.MPI 10.1.2.3 P
100% done
(since we
postponed TG
site runs)
100% done
(since we
postponed TG
site runs)
The tracking database is in
production; the internal web interface
was dropped as a deliverable earlier,
rather a suite of scripts to provide the
same functionality.
100% done
This is being treated as an
operational activity since the project
part of this is completed.
Due to unable to use MPIg on TG
sites we are postponing the task of
running full scale)
Due to unable to use MPIg on TG
sites cross-site runs on TG sitesis
postponed for now
95% done
100% done
(since we
postponed TG
site runs)
100% done
(since we
postponed TG
site runs)
Work with largest TG RP providers to enable
cross-site runs (this is being postponed due to
unable to use MPIg on TG sites). Instead adding
new task of integraing UPC and MPI for user
apps Feb-10 Apr-10 20% done
AUS.MPI 10.1.2.4 O
Investigate intetrating clouds, possibly with
Globus, to run MPIg jobs. May-10 Jul-10
AUS.MPI 10.1.2.5 O MPIg Support PY6 Aug-10 Jun-11
10.2 Advanced Support Operations
AUS.ASTA 10.2.1 O Advanced Support TG Applications
AUS.ASTA 10.2.1.2 O Advanced Support TG Applications PY5 Aug-09 Jul-10
NCSA 0.6 FTE
Jay Alameda [20%]
Dodi Heryadi [0%]
Seid Koric [0%]
Rick Kufrin[25%]
Sudhakar Pamidighantam [0%]
Mark Straka[15%]
Ahmed Taha[0%]
Michelle Gower[0%]
David Bock [0%]
Mark Vanmoer [0%]
PSC 3.15 FTE
Mahin Mahmoodi [10%]
Raghu Reddy [30%]
John Urbanic [25%]
Joel Welling [50%]
Philip Blood [35%]
Roberto Gomez [40%]
Marcela Madrid [20%]
David O'Neal[50%]
Yang Wang [30%]
Anirban Jana [25%]
This is being treated as an
operational activity since the project
part of this is completed.
80% done
The tracking database is in
production; the internal web interface
was dropped as a deliverable earlier,
rather a suite of scripts to provide the
same functionality.
The tracking database is in
production and we are using it as a
tool to manage software
maintenance decisions. The license
is still not complete (working with
lawyers), but the code will be shared
with any interested centers.
Due to unable to use MPIg on TG
sites we are postponing the task of
running full scale)
Due to unable to use MPIg on TG
sites cross-site runs on TG sitesis
postponed for now
Continuted to learn about OpenMP.
Worked with user to identify code
where OpenMP could work.
RPs have still yet to embrace cloud
computing. Learning more about
OpenMP. Scheduled to meet with
user in November to start modifying
their code.

Project-ID
WBS
OPMD
Description
TeraGrid AUS PY4-PY5 Project Plan
Work Package Dependencies Planned
Task
Cross
Area
Start
Date
End
Date
Resources
Name [%FTE]
12/31/10 AD
Status
QSR 2010Q4 Report Values
12/31/10 AD Notes
9/30/10 AD
Status
QSR 2010Q3 Report Values
9/30/10 AD Notes
SDSC 1.75 FTE
Natasha Balac[15%]
Dong Ju Choi[15%]
Amit Chourasia[10%]
Yifeng Cui[40%]
Dmitry Pekurovsky [25%]
Wayne Pfeiffer[10%]
Mahidhar Tatineni[20%]
Ross Walker[20%]
TBD/Doc etc.[20%]
Purdue 0.45 FTE
Phil Cheeseman[45%]
LSU 0.1 FTE
Adam Jundt [10%]
Honggao Liu /POC[0%]
ORNL 0.2 FTE
John Cobb, backup POC ORNL[0%]
Mei Li Chen, backup POC[10%]
Vickie Lynch, POC ORNL[10%]
NICS 0.8 FTE
Lonnie Crosby, POC NICS[40%]
Christian Halloy[20%]
Kwai Wong[20%]
Bruce Loftis, backup POC[0%]
TACC 3.85 FTE
Bill Barth, POC TACC
John Cazes
Lars Koesterke
B. D. Kim
Hang Liu
Robert McLay
Kent Milfeld
John Peterson
Karl Schulz (backup POC)
TACC total 3.85
IU 0.0 FTE
Don Berry[0%]
Ray Shepperd POC[0%]
ANL 0.31 FTE
Joe Insley[31%]
AUS.ASP 10.2.2 O Advanced Support Projects
AUS.ASP 10.2.2.2 O Advanced Support Projects PY5 Aug-09 Jul-10
Total 11.06
NCSA 0.4 FTE
Jay Alameda [20%]
Dodi Heryadi [0%]
Seid Koric [0%]
Ahmed Taha[0%]
Michelle Gower[0%]
David Bock [0%]
Mark Vanmoer [0%]
Mark Straka[20%]
PSC 1.65 FTE
Nick Nystrom [15%]
Mahin Mahmoodi [40%]
Raghu Reddy[5%]
Joel Welling [30%]
David O'Neal[15%]
Anirban Jana[10%]
Roberto Gomez[20%]
Yang Wang [15%]
Phil Blood [15%]
SDSC 1.05 FTE
Chourasia [15%]
Pfeiffer[40%]

Project-ID
WBS
OPMD
Description
TeraGrid AUS PY4-PY5 Project Plan
Work Package Dependencies Planned
Task
Cross
Area
Start
Date
End
Date
Resources
Name [%FTE]
12/31/10 AD
Status
QSR 2010Q4 Report Values
12/31/10 AD Notes
9/30/10 AD
Status
QSR 2010Q3 Report Values
9/30/10 AD Notes
Dmitry Pekurovsky[10%]
Tatineni[5%]
Balac[10%]
Ross Walker [25%]
Purdue 0.3 FTE
TBD 30%
LSU 0.1 FTE
Adam Jundt [10%]
TACC 1.9 FTE
Bill Barth [X%]
others/TACC
TACC total 1.9
IU 0.0 FTE
Don Berry [0]%
NICS 0.4 FTE
Christian Halloy[20%]
Kwai Wong[20%]
AUS.ASEOT 10.2.3 O Advanced Support EOT
AUS.ASEOT 10.2.3.2 O Advanced Support EOT PY5 Aug-09 Jul-10
Total 5.85
NCSA 0.25 FTE
Jay Alameda [10%]
Sudhakar Pamidighantam [0%]
David Bock [0%]
Mark Vanmoer [0%]
Mark Straka[15%]
Purdue 0.2 FTE
TBD[20%]
PSC 0.7 FTE
Nick Nystrom [10%]
Philip Blood [10%]
Marcela Madrid[30%]
Shawn Brown[10%]
Mahin Mahmoodi [10%]
NICS 0.38 FTE
Lonnie Crosby[38]
SDSC 0.4 FTE
Yifeng Cui[10%]
Dmitry Pekurovsky[15%]
TBD/doc[15%]
TACC 0.85 FTE
Lars Koesterke
Kent Milfeld [X%]
Bill Barth [X%]
TACC total 0.85
LSU 0.05 FTE
Adam Jundt [5%]
IU 0.0 FTE
Don Berry [0%]
Total 2.88

Work Package Dependencies Planned
Resources
(per GIG budget 7/7/08)
12/31/10
AD Status
QSR 2010Q4 Report Values
12/31/10 AD Notes
9/30/10 AD
Status
QSR 2010Q3 Report Values
9/30/10 AD Notes
Project-ID
WBS
O P M
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
UFP 9 Area: User-facing and Core Services Projects
UFP 9
Objective: User-facing Projects:
Maintainance and Development
May-07 Jul-10
UFP.Coord 9.0 O UFP Area Coordination Aug-07 Jul-10
UFP.Coord 9.0.3 O UFC Area Coordination PY5 Aug-09 Jul-10 UC, Jeff Koerner Ongoing Ongoing
UFP.Coord 9.0.3.2 M Quarterly Reporting (9/30, 12/31, 3/31, 6/30)
UFP.Coord
9.0.3.3
Lead UFC-RP working group for coordination
O of RPs with UFC
UFP.Acct
NCSA Steve Quinn[20%],
Michael Shapiro[20%];
Ongoing
Ongoing
9.6 O Accounting Operations PY5 Aug-09 Jul-10
SDSC Leo Carson[44%],
Tiffany Duffield[17%], Henry
Jaime[7%]
UFP.Acct 9.6.1 O TGCDB and AMIE PY5 Aug-09 Jul-10
UFP.Acct
Ongoing operations and maintenance of the
Ongoing
Ongoing
9.6.1.1
central TGCDB-AMIE environment and
O servers Aug-09 Jul-10
Leo Carson, H. Jaime (SDSC)
(0.36 FTE)
UFP.Acct
Update TGCDB tables and code to support
Ongoing
Ongoing
9.6.1.2
evolving TG requirements, including addition
of new RP resources into TGCDB; changes to
O existing resources. Aug-09 Jul-10
Michael Shapiro (NCSA) (0.10
FTE), Steve Quinn (NCSA)
(0.10 FTE)
UFP.Acct
Michael Shapiro (NCSA) (0.10 Ongoing
Ongoing
9.6.1.3
Update AMIE tables and code to support
O evolving TG requirements Aug-09 Jul-10
FTE), Steve Quinn (NCSA)
(0.10 FTE)
UFP.Acct 9.6.2 O tgusage and gx-map PY5
UFP.Acct
Maintain and perform critical fixes to tgusage
Tiffany Duffield (SDSC) Ongoing
Ongoing
utility as directed by trouble tickets and in
[replacing former staff] (0.17
9.6.2.1 O support of evolving UFC needs. Aug-09 Jul-10 FTE)
UFP.Acct
Maintain and perform critical fixes to gx-map
Ongoing
Ongoing
9.6.2.2
utility as directed by trouble tickets and in
O support of evolving UFC needs Aug-09 Jul-10
Leo Carson (SDSC) (0.15
FTE)
UFP.Acct 9.6.3 O Metrics PY5
UFP.Acct
Maintain web-based TGU query and reporting
Ongoing
Ongoing
system, in current form and later as integrated
9.6.3.1 O part of TGUP. Aug-09 Jul-10
UFP.Acct
Metrics reporting for QSRs, including updated
Ongoing
Ongoing
metrics as capabilities supported in TGCDB
9.6.3.2 O and POPS (9/30, 12/31, 3/31, 6/30) Aug-09 Jul-10
UFP.UDoc 9.1 R TG Documentation
UFP.UDoc 9.1.1 R TG Docs PY4 Aug-08 Jun-09
Fariba Fana[100%],
Diana Diehl[25%]
UFP.UDoc 9.1.1.3
TG Resource Catalog integrated with MDS +
M
enhancements
Core2 Aug-08 Jul-09 In progress In progress
UFP.UDoc 9.1.1.4 M PI User Services Log integrated with TGCDB Core2 Aug-08 Jun-09 In progress In progress
UFP.UP 9.3 User Portal
UFP.UP 9.3.2 R TG User Portal PY4 Aug-08 Jun-09
UFP.UP 9.3.2.4 M
Expanded personalization and collaboration
features across the user portal, this includes
allocation management, customized resource
listing, collaboration of proposals, file space,
etc
John Boisseau[8%],
Maytal Dahan[18%],
Patrick Hurley[35%],
Praveen Nuthalapati[45%],
Steve Mock[25%],
TACC NOS[15%]
UFP.UR 9.7 O User Requests
UFP.UR 9.7.1 O User Requests PY5 Aug-09 Jul-10 NCSA V. Halberstadt[100%]
UFP.UR
9.7.1.1
Create/establish user accounts in TGCDB,
O mail new user packets, collect RP site logins. Aug-09 Jul-10
UFP.UR
Support PI requests for removal of users from
9.7.1.2 O allocations Aug-09 Jul-10
UFP.UR
Vet/review user information for new PIs and co-
9.7.1.3 O PIs Aug-09 Jul-10
UFP.UR
Serve as first point of contact for general user
9.7.1.4 O inquiries re: UFC procedures Aug-09 Jul-10
UFP.AP 9.8 O Allocations Process
Aug-08 Jun-09 Completed Completed
** This process may change in
late PY4, early PY5
This process will continue in
PY5
Ongoing
Done
Ongoing
Ongoing
Ongoing
Done
Ongoing
Ongoing

Work Package Dependencies Planned
Resources
(per GIG budget 7/7/08)
12/31/10
AD Status
QSR 2010Q4 Report Values
12/31/10 AD Notes
9/30/10 AD
Status
QSR 2010Q3 Report Values
9/30/10 AD Notes
Project-ID
WBS
O P M
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
UFP.AP
O Allocations Process PY4
UFP.AP
D POPS interaction with RDR
UFP.AP
TACC Kent Milfeld[20%],
9.8.1 O Allocations Process PY5
Marg Murray[20%], Valerie
Alvarez[25%]
UFP.AP
Quarterly TRAC Meetings (9/09, 12/09, 3/10,
Ongoing
Ongoing
9.8.1.1 M 6/10) Aug-09 Jul-10
UFP.AP
Ongoing
Ongoing
9.8.1.2
Processing of intermittent TRAC submissions
O (supplements, justifications, etc) Aug-09 Jul-10
UFP.AP
Ongoing review and processing of Startup and
Ongoing
Ongoing
9.8.1.3 O Education requests Aug-09 Jul-10
UFP.AP
Maintenance and updates to Allocations policy
Ongoing
Ongoing
9.8.1.4 O and procedures documentation Aug-09 Jul-10
UFP.AP 9.8.1.5 O Recruiting and supporting TRAC members Ongoing Ongoing
UFP.POP 9.9 O Resource Requests (POPS)
UFP.POP
NCSA Steve Quinn[10%],
9.9.1 O Resource Requests (POPS) PY5
Ester Soriano[20%]
UFP.POP
Support for Quarterly TRAC meetings (9/09,
Ongoing
Ongoing
9.9.1.1 M 12/09, 3/10, 6/10) Aug-09 Jul-10
UFP.POP
Operate, maintain and update POPS system
Ongoing
9.9.1.2 O in support of allocations process Aug-09 Jul-10
UFP.IP 9.10 O Information Presentation/Web Presence
UFP.IP
SDSC Michael Dwyer[10%];
Information Presentation/Web Presence
TACC Patrick Hurley[25%];
9.10.1 O PY5 Aug-09 Jul-10 UC Tim Dudek[50%]
UFP.IP
Maintenance and updates for deployed portal
Ongoing
9.10.1.1 O capabilities (SSO, etc.) Aug-09 Jul-10 P. Hurley (TACC)
UFP.IP
Support for RPs entering information into
Ongoing
9.10.1.2 O catalogs, monitors, and news services. Aug-09 Jul-10 M. Dwyer (SDSC) (0.05 FTE)
UFP.IP
Ongoing maintenance, updates, and
Ongoing
operations for existing catalogs: Compute and
9.10.1.3 O data resources, CTSS, Gateways, etc. Aug-09 Jul-10 M. Dwyer (SDSC) (0.05 FTE)
UFP.IP
UFP.IP
UFP.IP
UFP.IP
Maintenance and updates to TeraGrid's Batch
Queue Prediction Service, including addition
9.10.1.4
of new resources, user support and bug fixes,
O by QBETS team at UCSB. Aug-09 Jul-10
G. Obertelli (UCSB) (0.50
FTE)
Provide statistics and metrics for QSR to
assist in measuring success for TG efforts that
appear in web/portal/KB behavior (9/30,
9.10.1.5 O 12/31, 3/31, 6/30) Aug-09 Jul-10 T. Dudek (ANL) (0.25 FTE)
Administration of WebSphere servers and
9.10.1.6 O infrastructure Aug-09 Jul-10 Subcontract
9.10.1.7 O
Administration of TeraGrid wiki for
problems/spam and fix creation of secure
areas/user access etc. Aug-09 Jul-10 T. Dudek (ANL) (0.25 FTE)
UFP.QA 9.11 O User Information Quality Assurance
UFP.QA
UFP.QA
UFP.QA
UFP.QA
UFP.QA
IU Andy Orahood[10%], Julie
Thatcher[25%], Paul
Brown[50%], Mary
Hrovat[20%], Jonathon
Bolte[25%]; SDSC Diana
Diehl[55%]; UC Tim
9.11.1 O User Information Quality Assurance PY5 Aug-09 Jul-10 Dudek[50%]
Run Web Link Validator weekly and correct
9.11.1.1 O broken links Aug-09 Jul-10
Respond to user documentation correction
requests and initiate corrections to major
9.11.1.2 O errors within 1 business day. Aug-09 Jul-10 D. Diehl (SDSC)
9.11.1.3
9.11.1.4 O
D. Diehl (SDSC) (0.55 across
all items here)
D. Diehl (SDSC); IU KB team;
T. Dudek (ANL); M. Dahan
(TACC)
Monitor and compile user feedback to UFC
web presence and procedures Aug-09 Jul-10
Respond to content update requests for
www.teragrid.org within 1 business day. Aug-09 Jul-10 T. Dudek (ANL) (0.25 FTE)
Ongoing
Ongoing
Cancelled/M
oved
Ongoing
Ongoing
Ongoing
Ongoing
Ongoing
User News application ported from
SDSC's Oracle to TeraGrid postgreSQL.
Ongoing
Ongoing
Ongoing
Ongoing
Ongoing
Ongoing
Cancelled/M
oved
Ongoing
Ongoing
Ongoing
Ongoing
Ongoing
User News application ported from
SDSC's Oracle to TeraGrid postgreSQL.

Work Package Dependencies Planned
Resources
(per GIG budget 7/7/08)
12/31/10
AD Status
QSR 2010Q4 Report Values
12/31/10 AD Notes
9/30/10 AD
Status
QSR 2010Q3 Report Values
9/30/10 AD Notes
Project-ID
WBS
O P M
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
UFP.QA
Provide central coordination of TG-wide
Ongoing
Ongoing
documentation contributions and updates
submitted from across the GIG and RP sites,
using capabilities within the Liferay
9.11.1.5 O environment. Aug-09 Jul-10 D. Diehl (SDSC)
UFP.QA
Conduct ongoing series of coordinated
Ongoing Review and release of AUS performance Ongoing Review and release of AUS performance
reviews to evaluate, synchronize and update
pages into Liferay.
pages into Liferay.
documentation, KB and other web info related
9.11.1.6 O to TG-wide user info areas. Aug-09 Jul-10 D. Diehl (SDSC); IU KB team
UFP.QA
Completion of 250 new KB documents, some
Ongoing
Ongoing
in response to US and user inputs through the
9.11.1.7 O ticket system. Aug-09 Jul-10 IU KB team
UFP.QA
Ongoing updates and special requests for
Ongoing
Ongoing
web content including posting information for
EOT and ER areas, and conference/workshop
9.11.1.8 O sites. Aug-09 Jul-10 T. Dudek (ANL) (0.25 FTE)
UFP.EUA 9.12 P Enhanced TeraGrid User Access
UFP.EUA
Enhanced TeraGrid User Access PY4
UFP.EUA
Linking Shibboleth login to TGUP login in
D
TGUP test instance completed completed
Complete user account management,
Nearing completion
UFP.EUA
D including end-to-end Shibboleth authentication
in production completed In progress
UFP.EUA
Interactive capabilities for job submission in
D
TGUP Delayed Delayed
UFP.EUA D Job reservations form in TGUP Delayed Delayed
TGUP features combined into a workflow
UFP.EUA
D including file management, job submission
and remote visualization Delayed Delayed
UFP.EUA
D Co-scheduling proof of concept portlet
NCSA Ester Soriano[10%],
UFP.EUA
Steve Quinn[10%]; TACC
Praveen Nuthulapati[100%],
9.12.1 P Enhanced TeraGrid User Access PY5 Aug-09 Jul-10 Steve Mock [50%]
UFP.EUA 9.12.1.1 P Users able to create TG logins at the TGUP Aug-09 Oct-09 Ongoing In Progress Ongoing Implementation plan approved
UFP.EUA 9.12.1.2 P Job submission interface in TGUP Nov-09 Jan-10
UFP.EUA
Users can use campus credentials to
Apr-10
9.12.1.3 P authenticate to TGUP
Feb-10
UFP.EUA
Jul-10 Cancelled New file sharing feature in TGUP file Cancelled New file sharing feature in TGUP file
manager portlet provides partial
manager portlet provides partial
capability, including cross-TG data
capability, including cross-TG data
Mar-10
movement
movement
9.12.1.4 P
Lustre-WAN access via TGUP with cross-TG
data movement
UFP.RAM 9.13 P Resource Authorization and Management
UFP.RAM
NCSA Ester Soriano[60%],
Steve Quinn[25%], Mike
Shapiro[10%]; TACC Maytal
9.13.1 P
Resource Authorization and Management
PY5 Aug-09 Jul-10
Dahan[50%]; PSC Edward
Hanna[10%], Rob Light[10%]
UFP.RAM 9.13.1.1 P Improved user authorization process Aug-09 Oct-09 Ongoing Ongoing
UFP.RAM
9.13.1.2 P
Expanding user capabilities for managing
allocations and usage
Nov-09
Jan-10 Ongoing New resource request interface deployed Ongoing
in POPS. Work beginning on revised
POPS entry screens.
UFP.RAM 9.13.1.3 P POPS integrated with TGUP Feb-10 Apr-10 Cancelled Cancelled
UFP.RAM
Resource Advisor made available to users
Jul-10
On hold
9.13.1.4 P within UFP systems
Mar-10
UFP.IIS 9.14 P RP Integration and Information Sharing
UFP.IIS
RP Integration and Information Sharing
P PY4
UFP.IIS
RDR system moved into production
Delayed
Delayed
D environment
UFP.IIS
Complete In progress Nearing completion
UFP.IIS
D
9.14.1 P
Multi-audience TGCDB and POPS reporting
capabilities implemented within TGUP as UFC
development permits
RP Integration and Information Sharing
PY5 Aug-09 Jul-10
NCSA Ester Soriano[10%],
Mike Shapiro[10%]; PSC
Edward Hanna[25%], Rob
Light[25%]
New resource request interface deployed
in POPS. Work beginning on revised
POPS entry screens.

Work Package Dependencies Planned
Resources
(per GIG budget 7/7/08)
12/31/10
AD Status
QSR 2010Q4 Report Values
12/31/10 AD Notes
9/30/10 AD
Status
QSR 2010Q3 Report Values
9/30/10 AD Notes
Project-ID
WBS
O P M
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
UFP.IIS 9.14.1.1 P RDR-based descriptions in POPS Aug-09 Oct-09 In Progress Delayed
UFP.IIS 9.14.1.2 P RDR-based descriptions in TGCDB Nov-09 Jan-10 In Progress Delayed
UFP.IIS 9.14.1.3 P Hardened, updated RDR Feb-10 Apr-10 In Progress Delayed
UFP.IIS 9.14.1.4 P RDR includes non-compute resources Mar-10 Jul-10 Completed Completed
UFP.UIP 9.15 P User Information Presentation
UFP.UIP P User Information Presentation PY4
UFP.UIP
Post-migration alignment of portal and Web
Ongoing Will be an ongoing effort for duration of Ongoing
D site look, feel, and organization.
TeraGrid.
UFP.UIP
With WebSphere environment, developing
Cancelled
In progress
D scalable documentation model for TeraGrid
UFP.UIP
Migration to RDR/MDS as definitive source of
In Progress
In progress
D Resource Catalog info
UFP.UIP
UFP.UIP
UFP.UIP
UFP.UIP
UFP.UIP
UFP.UIP
UFP
D
Enhanced TGUP system monitor with
customizable list of resources and expanded
resource information
9.15.1 P User Information Presentation PY5 Aug-09 Jul-10
PSC Edward Hanna[20%],
Rob Light[20%]; SDSC
Michael Dwyer[65%]; TACC
Rion Dooley[75%]
Integration of TeraGrid user portal info
Oct-09 Delayed Delayed
9.15.1.1 P services with RDR
Aug-09
Integration of TG catalogs and User News with
Jan-10 Delayed Delayed
9.15.1.2 P RDR
Nov-09
Enhanced system monitors for HPC, storage,
Apr-10 Delayed Delayed
9.15.1.3 P and vis resources
Feb-10
Enhanced integration of user news in TGUP
Jul-10 Delayed Delayed
9.15.1.4 P and user notification
Mar-10
END
Delayed
Delayed
Will be an ongoing effort for duration of
TeraGrid.

TeraGrid DV Extension Year Project Plan
Work Package Dependencies Planned
Resources
Project-ID
WBS
O P M X
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
DV 3.0
Objective 3.0 Operations:
maintain/sustain current capabilities
DV.Coord 3.0.1 O DAV Area Coordination
DV.Coord 3.0.1.3 O DV Area Coordination Aug-09 Jul-11
TACC Kelly Gaither [50%],
Chris Jordan[25%]
ongoing
DV.Coord 3.0.1.3.1 M Quarterly Reporting Aug-09 Jul-11 ongoing
DV.DDM.RP
Aggregate RP Distributed Data
O
Management Ops
Apr-10 Jul-11
DV.DDM.RP.NCSA 3.0.1.4 O
DV.DDM.RP.IU 3.0.1.5 O
DV.DDM.RP.LONI 3.0.1.6 O
DV.DDM.RP.NICS 3.0.1.7 O
DV.DDM.RP.ORNL 3.0.1.8 O
DV.DDM.RP.PSC 3.0.1.9 O
DV.DDM.RP.PU 3.0.1.10 O
DV.DDM.RP.SDSC 3.0.1.11 O
DV.DDM.RP.TACC 3.0.1.12 O
NCSA RP Distributed Data Management
Operations
LONI RP Distributed Data Management
Operations
NICS RP Distributed Data Management
Operations
PSC RP Distributed Data Management
Operations
SDSC RP Distributed Data Management
Operations
TACC RP Distributed Data Management
Operations
Apr-10
Jul-11
Alt [40%], B.Butler [75%],
M.Butler [65%], Cai [40%],
Chen [100%], Cribbs [50%],
Glasgow [90%], Kerner [90%],
Loftus [45%] Long [20%]
ongoing
Apr-10 Jul-11 Martinez [100%] ongoing
Apr-10
Apr-10
Jul-11
Jul-11
Kovatch [50%], Hazlewood
[50%], TBD [100%], TBD
[100%], TBD [50%]
Kar[40%], Litzinger[50%],
Nowoczynski[60], Sorge[15],
Stone[80], Yanovich[80]
ongoing
ongoing
Bennett [75%], Cai [95%], ongoing
Apr-10 Jul-11
Chen, S. [40%], Chen, L.
[80%], Dinh [50%], Hom [75%],
Rivers [65%]
Apr-10 Jul-11 Jones[25%] ongoing
DV.DDM.RP.UCANL 3.0.1.13 O
DV.DC.RP O Aggregate RP Data Collections Ops Apr-10 Jul-11
DV.DC.RP.NCSA 3.0.1.14 O
DV.DC.RP.IU 3.0.1.15 O
DV.DC.RP.LONI 3.0.1.16 O
DV.DC.RP.NICS 3.0.1.17 O NICS RP Data Collections Operations Apr-10 Jul-11 Hazlewood [50%], TBD [70%] ongoing
DV.DC.RP.ORNL 3.0.1.18 O
DV.DC.RP.PSC 3.0.1.19 O
DV.DC.RP.PU 3.0.1.20 O PU RP Data Collections Operations Apr-10 Jul-11
Zhao[50%], 2 half time Grad
Students [100%]
ongoing
DV.DC.RP.SDSC 3.0.1.21 O SDSC RP Data Collections Operations Apr-10 Jul-11 Nunes [10%], Wong [25%] ongoing
2010Q4 Status Update

DV.DC.RP.TACC 3.0.1.22 O TACC RP Data Collections Operations Apr-10 Jul-11 Urban[50%] ongoing
DV.DC.RP.UCANL 3.0.1.23 O
DV.Viz.RP O Aggregate Visualization Ops Apr-10 Jul-11
DV.Viz.RP.NCSA 3.0.1.24 O
DV.Viz.RP.IU 3.0.1.25 O
DV.Viz.RP.LONI 3.0.1.26 O
DV.Viz.RP.NICS 3.0.1.27 O NICS RP Visualization Operations Apr-10 Jul-11 TBD [60%], TBD [60%] ongoing
DV.Viz.RP.ORNL 3.0.1.28 O
DV.Viz.RP.PSC 3.0.1.29 O PSC RP Visualization Operations Apr-10 Jul-11 Foss[90%], new[10%] ongoing
DV.Viz.RP.PU 3.0.1.30 O PU RP Visualization Operations Apr-10 Jul-11
New Person [100%], 2 half time ongoing
grad students [100%]
DV.Viz.RP.SDSC 3.0.1.31 O SDSC RP Visualization Operations Apr-10 Jul-11 Chourasia [25%], TBD [150%] ongoing
Gaither[10%], Burns[50%], ongoing
DV.Viz.RP.TACC 3.0.1.32 O TACC RP Visualization Operations Apr-10 Jul-11
GregSJohnson[30%],
Schneider[50%],
GregPJohnson[50%]
DV.Viz.RP.UCANL 3.0.1.33 O UCANL RP Visualization Operations Apr-10 Jul-11 Insley[11%] ongoing
DV.DPI 3.2
Objective 3.2 Projects: Data Movement
Performance
DV.DPI 3.2.5 P Data Movement Performance PY5 Aug-09 Jul-11
PSC Kathy Benninger[20%],
Bob Budden[50%], Derek
Simmel[60%]
DV.DPI 3.2.5.2
Production tools for scheduled data
P Apr-10 Jul-11 TBD ONGOING
DV 3.3
movement
Objective 3.3 Projects: Global Wide
Area File Systems
DV.GFS
DV.GFS
3.3.3
3.3.3.1
P Global Wide Area File Systems PY5
P Lustre-WAN in Production
Aug-09
Apr-10
Jul-11
Jul-11
NICS Phil Andrews[5%],
Patricia Kovatch[15%],
Nathaniel Mendoza[15%],
Victor Hazlewood[15%]; PSC
Josephine Palencia[50%];
SDSC Jeff Bennett[25%],
Thomas Guptill[25%]; TACC
GFS Engineer[20%]
ONGOING
DV.GFS 3.3.3.6 P
DV 3.4
Evaluation of pNFS readiness for
production
Objective 3.4 Projects: TeraGrid Wide
Data Architect Design
DV.DWG 3.4.4 P Data Architecture PY5 Aug-09 Jul-10
Apr-10
Jul-11
NICS Phil Andrews[10%],
Bruce Loftis[20%]; PSC J Ray
Scott[25%]; TACC Chris
Jordan[50%]
DELAYED(SOFTWARE
AVAILABILITY)
complete
DV.DWG 3.4.4.3 P Physical instantiation of the design 3.4.3.3 TBD Apr-10 Mar-11 ONGOING
DV 3.5 Objective 3.5 Projects: Visualization
DV
END

TeraGrid NOS Project Plan
Work Package Dependencies Planned
Resources
Project-ID
WBS
O P M X
Description
Task
Cross
Area
Start
Date
End
Date
Name [%FTE]
Status
NOS 4.0
Objective 4.0 Operations:
maintain/sustain current capabilities
NOS.Coord 4.0.1 O NOS Area Coordination
NOS.Coord 4.0.1.3 O NOS Area Coordination PY5 Aug-09 Jul-10 UCANL Jeff Koerner [25%] ongoing
NOS.Coord 4.0.1.3.1
Quarterly Reporting (9/30, 12/31, 3/31,
M
6/30)
Aug-09 Jul-10 ongoing
NOS.Net 4.0.2 Networking
NOS.Net 4.0.2.3 O Networking Lead PY5 Apr-10 Mar-11
SDSC Tom Hutton [10%];
ANL Linda Winkler [50%]
ongoing
NOS.Net.RP O Aggregate RP Networking Ops Apr-10 Mar-11
NOS.NET.RP.NCSA 4.0.2.4 O NCSA RP Networking Operations Apr-10 Mar-11 Wefel [10%], Shoop [10%] ongoing
NOS.NET.RP.IU 4.0.2.5 Apr-10 Mar-11 ongoing
NOS.NET.RP.LONI 4.0.2.6 O LONI RP Networking Operations Apr-10 Mar-11 ongoing
NOS.NET.RP.NICS 4.0.2.7 O NICS RP Networking Operations Apr-10 Mar-11
Mendoza [50%], TBD [100%], ongoing
Baer [50%]
NOS.NET.RP.ORNL 4.0.2.8 ongoing
Adams[25%],
ongoing
NOS.NET.RP.PSC 4.0.2.9 O PSC RP Networking Operations Apr-10 Mar-11
Benninger[40%],
Huntoon[40%], Lambert[50%],
Lappa[25%], Rapier[25%]
NOS.NET.RP.PU 4.0.2.10 O Purdue RP Networking Operations Apr-10 Mar-11 Lewis [10%] ongoing
Carlson [45%], Dombrowski
NOS.NET.RP.SDSC 4.0.2.11 O SDSC RP Networking Operations Apr-10 Mar-11 [65%], Hutton [20%], Valente
ongoing
[60%]
NOS.NET.RP.TACC 4.0.2.12 O TACC RP Networking Operations Apr-10 Mar-11 Jones [25%] ongoing
NOS.NET.RP.UCANL 4.0.2.13 O UCANL RP Networking Operations Apr-10 Mar-11 Hedden [13%] ongoing
NOS.Sec 4.0.4 O Security Services Apr-10 Mar-11
NOS.Sec 4.0.4.1 O Operational Security Team PY5 Aug-09 Jul-10 PSC James Marsteller [50%] ongoing
NOS.Sec 4.0.4.1.1 O Incident Response activities Aug-09 Jul-10 ongoing
NOS.Sec 4.0.4.1.2
Coordinating operational security issues
O
across the project
Aug-09 Jul-10 ongoing
NOS.Sec 4.0.4.1.3 O Participate in and help lead TAGPMA Aug-09 Jul-10 ongoing
NOS.Sec.RP O Aggregate RP Security Ops Apr-10 Mar-11 ongoing
NOS.Sec.RP.NCSA 4.0.4.2 O NCSA RP Security Operations Apr-10 Mar-11
Barlow [30%], Brooks [15%],
Sharma [60%]
ongoing
NOS.Sec.RP.IU 4.0.4.3 IU RP Securit Operations Apr-10 Mar-11 Cornet [75%] ongoing
NOS.Sec.RP.LONI 4.0.4.4 ongoing
NOS.Sec.RP.NICS 4.0.4.5 ongoing
NOS.Sec.RP.ORNL 4.0.4.6 ongoing

Bennett[30%],
ongoing
NOS.Sec.RP.PSC 4.0.4.7 O PSC RP Security Operations Apr-10 Mar-11
Marsteller[40%],
Shelmire[60%], Simmel[10%],
new[10]
NOS.Sec.RP.PU 4.0.4.8 ongoing
NOS.Sec.RP.SDSC 4.0.4.9 O SDSC RP Security Operations Apr-10 Mar-11 Sakai [30%], Bennett [50%] ongoing
NOS.Sec.RP.TACC 4.0.4.10 O TACC RP Security Operations Apr-10 Mar-11 Murray [25%] ongoing
NOS.Sec.RP.UCANL 4.0.4.11 O UCANL RP Security Operations Apr-10 Mar-11 Leggett [13%] ongoing
NOS.Sec 4.0.4.3
Security Services (kerb, myproxy, CA)
John Quinn [50%]
O Aug-09 Jul-10
PY5
ongoing
Quarterly Reporting (9/30, 12/31, 3/31,
NOS.Sec 4.0.4.3.1 M
6/30)
Aug-09 Jul-10 ongoing
NOS.TOC 4.0.5 O TOC Services Aug-09 Jul-10 ongoing
NOS.TOC 4.0.5.3 O TOC Services PY5 Aug-09 Jul-10 NCSA Ops Staff [200%] ongoing
NOS.TOC 4.0.5.3.1
Quarterly Reporting (9/30, 12/31, 3/31,
M
6/30)
Aug-09 Jul-10 ongoing
NOS.TOC.RP O Aggregate RP Helpdesk Ops Apr-10 Mar-11 ongoing
Cagle [50%], Nickens [20%], ongoing
NOS.TOC.RP.NCSA 4.0.5.4 NCSA RP Helpdesk Operations Apr-10 Mar-11
Pingleton [45%], Roney [75%],
Sellers [75%], Ten Have
[45%], Wells [60%], Wilson
[55%]
NOS.TOC.RP.IU 4.0.5.5 ongoing
NOS.TOC.RP.LONI 4.0.5.6 ongoing
NOS.TOC.RP.NICS 4.0.5.7 ongoing
NOS.TOC.RP.ORNL 4.0.5.8 ongoing
Hackworth[10%], Maiden[5%], ongoing
NOS.TOC.RP.PSC 4.0.5.9 O PSC RP Helpdesk Operations Apr-10 Mar-11 Raymond[2%],
Sanielevici[3%]
NOS.TOC.RP.PU 4.0.5.10 O Purdue RP Helpdesk Operations Apr-10 Mar-11 New person [50%] ongoing
NOS.TOC.RP.SDSC 4.0.5.11 ongoing
NOS.TOC.RP.TACC 4.0.5.12 ongoing
NOS.TOC.RP.UCANL 4.0.5.13 ongoing
NOS.HPCOps 4.0.8 O HPC Operations ongoing
NOS.HPCOps.RP O Aggregate RP HPC Operations Apr-10 Mar-11 ongoing
Bouvet [85%], Fernsler [50%], ongoing
Hoyenga [80%], Khin [80%],
NOS.HPCOps.RP.NCSA 4.0.8.1 O NCSA RP HPC Operations Apr-10 Mar-11
Lapine [75%], Marcusiu
[65%], Parga [75%],
Pflugmacher [10%], J.Quinn
NOS.HPCOps.RP.IU 4.0.8.2 O IU RP HPC Operations Apr-10 Mar-11
[65%] Scharf [50%]
Lowe [100%], Moore[100%],
Miller[100%]
ongoing
Martinez [100%], Leche
NOS.HPCOps.RP.LONI 4.0.8.3 O LONI RP HPC Operations Apr-10 Mar-11 [100%], Giaime [100%],
ongoing
Scheinine [50%]
Walsh [100%], Baer [50%],
NOS.HPCOps.RP.NICS 4.0.8.4 O NICS RP HPC Operations Apr-10 Mar-11 Kovatch [50%], Jones [50%],
ongoing
Ezell [50%], TBD [300%]
NOS.HPCOps.RP.ORNL 4.0.8.5 O ongoing

Albert[85%], Bennett[45%], ongoing
Budden[90%], Flaus[20%],
NOS.HPCOps.RP.PSC 4.0.8.6 O PSC RP HPC Operations Apr-10 Mar-11
Gill[90%], Graham[25%],
Johanson[80%], Kar[40%],
Kochmar[90%],
Palencia[20%] Perrone[50%]
Scott[50%],
ongoing
Sommerfield[80%],
Sorge[75%], Sullivan[50%],
NOS.HPCOps.RP.PU
4.0.8.6
4.0.8.7
O PSC RP HPC Operations Continued
O Purdue RP HPC Operations
Apr-10
Apr-10
Mar-11
Mar-11
Vargo[90%], Vizino[90%],
Webb[50%], Wozniak[80%],
Cubbison[30%], Lappa[25%],
new[55]
P. Smith [10%], Braun [50%] ongoing
Conway [75%], Diegel [50%], ongoing
NOS.HPCOps.RP.SDSC 4.0.8.8 O SDSC RP HPC Operations Apr-10 Mar-11
Fillez [30%], Furman [60%],
Guptill [20%], Hocks [50%],
Kamrath [65%], Khem [60%],
McNew[25%], Silva [25%],
Smallen [5%], Yoshimoto
[50%]
Timm [50%], Carver [50%],
NOS.HPCOps.RP.TACC 4.0.8.9 O TACC RP HPC Operations Apr-10 Mar-11 Anderson [25%], Walling
ongoing
[25%]
NOS.HPCOps.RP.UCANL 4.0.8.10 O UCANL RP HPC Operations Apr-10 Mar-11
Leggett [16%], Insley [18%],
Hedden [45%], Olson [58%]
ongoing
Objective 4.2 Projects: Expanding
NOS 4.2 P
Secure TG Access
ongoing
NOS.Acc 4.2.2 P Expanding Secure TG Access PY4
To WGs,
Jim Basney [20%],
Aug-08 Jul-09
SGW
Terry Fleury [50%]
ongoing
NOS.Acc 4.2.2.2 M Automated DN distribution compliance Aug-08 Dec-09 Delayed
NOS.Acc 4.2.2.4
Requirements/Design for cross-grid
M
Collab.
Aug-08 Jul-10 ongoing
NOS.Acc 4.2.3 P Expanding Secure TG Access PY5 Aug-09 Jul-10
NOS.Acc 4.2.3.1 P
Ubiquitous Adoption of Gateway
accounting
Aug-09
Jul-10
NCSA Jim Basney [25%], Jon
Siwek [50%], Terry Fleury
[25%]
NOS.Acc 4.2.3.2 P INCA tests for Gateway accounting Aug-09 Jul-10 done
NOS.Acc 4.2.3.3 P
TG/OSG authorization interoperability
functional
NOS.Acc 4.2.3.4 P Shibboleth access for campus to TGUP Aug-09 Jul-10 done
NOS 4.3
Objective 4.3 Projects: Operational
Intrumentation (device tracking)
NOS.Inst 4.3.3 P Operational Instrumentation PY5 Aug-09 Jul-10 NCSA Neil Gorsuch [45%]
NOS.Inst 4.3.3.1
Ongoing Monthly, Quarterly and Annual
O
reporting
Aug-09 Jul-10 ongoing
Upgrade Globus Listener to support
NOS.Inst 4.3.3.3 P
GRAM5
Feb-10 Jul-10 done
NOS.Inst 4.3.3.4 P Integration plan for TAIS May-10 Jul-10 ongoing
NOS.INCA 4.4.3 P INCA Improvements PY5 Aug-09 Jul-10
SDSC Kate Ericson [50%],
Shava Smallen[50%]
ongoing
NOS.INCA 4.4.3.1 P QA/CUE views Aug-09 Jul-10 ongoing
NOS.INCA 4.4.3.2 P Link Tickets to Inca results Nov-09 Jan-10 ongoing
Aug-09
Jul-10
pending gram5 deployment at
all sites
not started - depends on 4.2.2.4
above

NOS.INCA 4.4.3.3 P knowledgbase linkage Jan-10 Jun-10 ongoing
NOS.INCA 4.4.3.4 P integrate inca into TGUP Mar-10 Sep-10 ongoing
NOS 4.6
Objective 4.6 Projects: Annual Risk
Assessment
NOS.Risk 4.6.1 P Risk Assessment Aug-08 Jul-09 Jim Rome [10%]
NOS.Risk 4.6.1.1 M ID assessment area and write plan Aug-08 Sep-09 Complete complete
NOS.Risk 4.6.1.2 M Begin assessment of area Aug-08 Dec-09 complete
NOS.Risk 4.6.1.3 M Draft report of findings Aug-08 Mar-10 complete
NOS.Risk 4.6.1.4 M Final assessment and recommendations Aug-08 Jun-10 complete
NOS.QACUE
NOS.QA
Objective X.X Projects: QA/CUE
PY4 Availability Improvement Aug-08 Jul-09
ANL Joe Insley[17%]; IU Mike
Lowe[50%]; LONI Archit
Kulshrestha[25%]; NCSA Dan
Lapine[25%], Luke
Scharf[25%]; NICS Victor
Hazlewood[50%]; PSC
TBD[100%]; Purdue
TBD[50%]; SDSC Thomas
Guptill[50%], Jerry
Greenberg[50%] TACC Da id
NOS.QA
Organize and identify priorities and
SDSC Kate Ericson[12.5%];
Apr-10 Mar-11
timelines for software testing cycle
Shava Smallen[12.5%]
Ongoing
NOS.QA Identify new and existing tests Apr-10 Mar-11
SDSC Kate Ericson[12.5%];
Shava Smallen[12.5%]
Ongoing
NOS.QA Determine Test baselines Dec-08 Mar-09
SDSC Kate Ericson[12.5%];
Shava Smallen[12.5%]
Complete
NOS.QA TG grid services administration guide Aug-09 Jan-10
SDSC Kate Ericson[12.5%];
Shava Smallen[12.5%]
Complete
NOS.QA PY4 Reliability Improvement Aug-08 Dec-09
NOS.QA Identify common INCA errors and classify Aug-08 Mar-10
SDSC Kate Ericson[12.5%];
Shava Smallen[12.5%]
Complete
NOS.QACUE O PY5 QA/CUE Leadership Aug-09 Jul-10
SDSC Kate Ericson[12.5%];
Shava Smallen[12.5%]; PSC
Shawn Brown[20%]
NOS.QACUE P Common User interface roll out Aug-10 Jul-10 Shawn Brown[20%] Ongoing
NOS.QA O PY5 Availability Improvement Aug-09 Jul-10
NOS.Email P Maintain TeraGrid.org email services Apr-11 Mar-11 Neil Gorscuh (5%) Complete
NOS
END

QSR 2010Q4 Report Values
QSR 2010Q3 Report Values
QSR 2010Q2 Report Values
QSR 2010Q1 Report Values
QSR 2009Q4 Report Values
Project-ID
WBS
O P M
Work Package Dependencies Planned
Cross Start End
Description
Task Area Date Date
Resources
Name [%FTE]
12/31/10 AD Status 12/31/10 AD Notes 09/30/10 AD Status 09/30/10 AD Notes 6/30/10 AD Status 6/30/10 AD Notes 3/31/10 AD Status 3/31/10 AD Notes
12/31/09 AD
Status
12/31/09 AD Notes
EOT 5 Area: Education, Outreach and Training
EOT 5.0
Objective 5.0 Operations:
maintain/sustain current capabilities
EOT.Coord 5.0.1 O EOT Area Coordination
EOT.Coord 5.0.1.7 O EOT Area Coordination PY5 Aug-09 Jul-10 Scott Lathrop[50%]
working group meetings; info sharing among
100% 100% 100% 75%
EOT.Coord 5.0.1.7.1 M
Aug-09 Jul-10
50%
RP EOT staff
PY5 Quarterly Reporting (9/30, 12/31, 3/31,
100% 100% 100% 50%
EOT.Coord 5.0.1.7.2 M
Aug-09 Jul-10
25%
6/30)
EOT.COOR 5.0.1.8 O Support HPCU portal infrastructure Aug-09 Jul-10 Shodor contract 100% 100% 100% 75% 50%
EOT.Coord 5.0.1.9 O EOT Area Coordination Extension Aug-10 Jul-11 1.5 FTE for Extension
AD and project coodinator activities for Scott
50%
EOT.Coord 5.0.1.9.1 M
Aug-10 Jul-11
and Elizabeth
EOT 5.1
Objective 5.1 Projects: Training
(HPC University Workforce Development)
IU Robert Ping [20%]
Scott Tiege[20%];
NCSA Sandie
Kappes[50%]; NICS
James Ferguson[30%],
Bruce Loftis[20%]; PSC
US, AUSS,
Laura McGinnis[30%],
EOT.TS 5.1.2 P Objective 5.1 Projects: Training
Aug-09 Jul-10
SG, DIV
Robin Flaus[5%], Ryan
Omecene[10%], Tom
Maiden[3%], John
Urbanic[3%], Phil
Blood[3%]; SDSC
Edward Jeff Sale[33%];
TACC Chris
EOT.TS 5.1.2.1
Training & Seminars PY5 (training offered
To RPs Aug-09 Jul-10
at RP sites; on-line materials developed)
Development of one new training content for
US, AUSS,
Training team 100% One is complete and the other was to 95% One is complete and the other was to 90% Developing two new online courses. 75%
EOT.TS 5.1.2.5 P live, synchronous, and/or on-line delivery each
Jul-09 Jul-10
SG, DIV
be complete by the end of Sept. s/b
be complete by the end of Sept. s/b
quarter
completed mid Oct
completed mid Oct
65% Two more in development
EOT.TS.RP 5.1.2.7 O Aggregate RP Training Operations Aug-09 Mar-10
EOT.TS.RP.
Arnold [50%], File 100% 100% 100% 75%
5.1.2.7.1 O NCSA RP Training Operations Aug-09 Mar-10
NCSA
[80%], Glick [25%],
Kappes [20%]
50%
EOT.TS.RP.I
100% 100% 100% 75%
5.1.2.7.2 O IU RP Training Operations Aug-09 Mar-10
U
50%
EOT.TS.RP.
100% 100% 100% 75%
5.1.2.7.3 O LONI RP Training Operations Aug-09 Mar-10
LONI
50%
EOT.TS.RP.
Ferguson [50%], TBD 100% 100% 100% 75%
5.1.2.7.4 O NICS RP Training Operations Aug-09 Mar-10
NICS
[50%]
50%
EOT.TS.RP.
5.1.2.7.5 O ORNL RP Training Operations Aug-09 Mar-10
ORNL
McGinnis[4%], 100% 100% 100% 75%
EOT.TS.RP.
Mahmoodi[8%],
5.1.2.7.6 O PSC RP Training Operations Aug-09 Mar-10
PSC
Urbanic[15%], Brown,
S[3%], Madrid[5%],
50%
Maiden[35%] Jana[5%]
EOT.TS.RP.
100% 100% 100% 75%
5.1.7.7 O PU RP Training Operations Aug-09 Mar-10
PU
50%
EOT.TS.RP.
Sale [50%] 100% 100% 100% 75%
5.1.7.8 O SDSC RP Training Operations Aug-09 Mar-10
SDSC
50%
EOT.TS.RP.
Turner [10%], Wilson 100% 100% 100% 75%
5.1.7.9 O TACC RP Training Operations Apr-09 Mar-10
TACC
[10%]
50%
EOT.TS.RP.
5.1.7.10 O
75%
UC/ANL RP Training Operations Aug-09 Mar-10
UCANL
50%
Training & Seminars Extension (training
EOT.TS 5.1.8.1
offered at RP sites; on-line materials
To RPs Aug-10 Jul-11
developed)
Training team 25% On track, one from PSC and one for 5% early in process but on track, one from
US, AUSS,
EOT.TS 5.1.8.1.1 P Develop 3 new classes
Aug-10 Jul-11
NICS, and one from TACC
PSC and one for NICS, and one from
SG, DIV
TACC
Webcast Parallell Computing and
US, AUSS,
Training team 0% 0%
EOT.TS 5.1.8.1.2 P
Aug-10 Jul-11
Programming Class 2 times
SG DIV
US, AUSS,
Training team 50% 2 complete, 2 planned 0% Classes are scheduled. The next
EOT.TS 5.1.8.1.2 P Produce 4 webcasts
Aug-10 Jul-11
SG DIV
class is scheduled in Oct
US, AUSS,
Training team 0% Scheduled to start in January or when 0% Scheduled to start in January or when
EOT.TS 5.1.8.1.2 P XD Transition place holder
SG DIV
the XD winner is identified
the XD winner is identified
EOT.RPTRN 5.1.9
O
Extension - Aggregate RP Training
Aug-10 Jul-11
Operations
EOT.RPTRN
Arnold [50%], File 50% 25%
5.1.9.1 O Extension - NCSA RP Training Operations Aug-10 Jul-11
.NCSA
[80%], Glick [25%],
Kappes [20%]
EOT.RPTRN
50% 25%
5.1.9.2 O Extension - IU RP Training Operations Aug-10 Jul-11
IU
EOT.RPTRN
50% 25%
5.1.9.3 O Extension - LONI RP Training Operations Aug-10 Jul-11
LONI
EOT.RPTRN
Ferguson [50%], TBD 50% 25%
5.1.9.4 O Extension - NICS RP Training Operations Aug-10 Jul-11
NICS
[50%]
McGinnis[4%], 50% 25%
EOT.RPTRN
Mahmoodi[8%],
5.1.9.5 O Extension - PSC RP Training Operations Aug-10 Jul-11
.PSC
Urbanic[15%], Brown,
S[3%], Madrid[5%],
Maiden[35%] Jana[5%]
EOT.RPTRN
50% 25%
5.1.9.6 O Extension - PU RP Training Operations Aug-10 Jul-11
PU
EOT.RPTRN
Sale [50%] 50% 25%
5.1.9.7 O Extension - SDSC RP Training Operations Aug-10 Jul-11
SDSC
EOT.RPTRN
Turner [10%], Wilson 50% 25%
5.1.9.8 O Extension - TACC RP Training Operations Aug-10 Jul-11
TACC
[10%]
EOT.RPTRN
5.1.9.9 O
50% 25%
Extension - UC/ANL RP Training Operations Aug-10 Jul-11
UCANL
EOT.OTS 5.1.10 Extension - Online Training Aug-10 Jul-11 1.04 FTE
33% 1 completed, 2 started 10% 1 started, 1 going to start next quarter,
EOT.OTS 5.1.10.1 P Develop 3-4 on-line training courses Aug-10 Jul-11
the third will follow later in the year
0% Scheduled to start in January or when 0% Scheduled to start in January or when
EOT.OTS 5.1.10.2 P XD Transition place holder Aug-10 Jul-11
the XD winner is identified
the XD winner is identified
EOT.TQA 5.1.11 Training and HPC University Quality Aug-10 Jul-11 0.48 FTE
EOT.TQA 5.1.11.1 P QA of training materials Aug-10 Jul-11 33% reviewed the one that is completed 0%
Scheduled to start in January or when
Scheduled to start in January or when
EOT.TQA 5.1.11.2 P XD Transition place holder Aug-10 Jul-11
the XD winner is identified
the XD winner is identified
EOT.COMM 5.1.12 TG Community Aug-10 Jul-11 1.25 FTE
Create a TeraGrid social collaboration
25% plan is to use LifeRay environment. 25% Prototyped
EOT.COMM 5.1.12.1 P environment for learning and community
Aug-10 Jul-11
Planning to work with Tim D.
building
0% Scheduled to start in January or when 0% Scheduled to start in January or when
EOT.COMM 5.1.12.2 P XD Transition place holder Aug-10 Jul-11
the XD winner is identified
the XD winner is identified
EOT 5.2
Objective 5.2 Projects: External Relations
(Science Highlights)
EOT.SH 5.2.3 Science Highlights 2010 Jan-10 Nov-10
UC Elizabeth
Spans fiscal years
Leake[10%]
100% Distributed at SC10, to all RPs, and 75% 50%
EOT.SH 5.2.3.1 P 2010 Science Highlights Production Apr-10 Nov-10
NSF
Froelich,ER team and 100% 100% 100%
EOT.SH 5.2.3.2 P Review and select science stories to highlight
Bruett Consulting
Initiate identification of 2011 Science
ER team 100% PY5 portion of SH ended, will be 100% PY5 portion of SH ended, will be 25%
EOT.SH 5.2.3.3 P
Apr-10 Jul-10
Highlights
covered by line 59
covered by line 59
EOT.SH 5.2.3 Science Highlights and EOT 2011 Jul-10 Nov-11 Spans fiscal years
ER team 10% This has been changed to combine 5%
Initiate collection of stories for the winning XD
EOT.SH 5.2.4.1 P
Jul-10 Nov-11
the EOT and SH docs into a single
team
pub XD Transition
EOT 5.3 Objective 5.3 Projects: Outreach

QSR 2010Q4 Report Values
QSR 2010Q3 Report Values
QSR 2010Q2 Report Values
QSR 2010Q1 Report Values
QSR 2009Q4 Report Values
Project-ID
WBS
O P M
Work Package Dependencies Planned
Cross Start End
Description
Task Area Date Date
Resources
Name [%FTE]
12/31/10 AD Status 12/31/10 AD Notes 09/30/10 AD Status 09/30/10 AD Notes 6/30/10 AD Status 6/30/10 AD Notes 3/31/10 AD Status 3/31/10 AD Notes
12/31/09 AD
Status
12/31/09 AD Notes
EOT.OE 5.3.2.2.1 M Close out books on TG'09 Conference Jul-09 Dec-09 100% 100% On-going Mike - Need to check with John T. On-going Need to check with Matt. On-going Matt is tallking with Towns to get the data.
IU Robert Ping[20%];
PSC Laura
EOT.OE 5.3.5 Outreach Events PY5 ALL TG Aug-09 Jul-10
McGinnis[10%]; SDSC
Ange Mason[20%]
presentations, exhibits at conferences and
100% 100% 100% 75%
EOT.OE 5.3.5.1 M
Aug-09 Jul-10
50%
outreach events
90% 25%
Event is in August so effort increases as
EOT.OE 5.3.5.3 M TG10 Aug-09 Jul-10
20%
the event approaches
EOT.OE 5.3.5.3.1 M Close out books on TG'10 Conference Jul-10 Dec-10 100% Closed 90% Mike - Ask Richard M.
EOT.OE 5.3.5.5 M Campus Champions meeting at TG10 Jun-10 Jul-10 Hunt 100% 100%
EOT.OE 5.3.5.6 M Campus Champions meeting Spring 2010 Jan-10 Apr-10 Hunt 100% 100% 100%
Campus Champion Visits - 2 per quarter
100% 100% 100% 75%
EOT.OE 5.3.5.7 M
Aug-09 Jul-10
50%
throughout the year
EOT.OE 5.3.5.8 O Campus Champions Consulting Aug-09 Jul-10 100% 100% 100% 75% 50%
Campus Champions monthly training sessions
100% 100% 100% 75%
EOT.OE 5.3.5.9 M
Aug-09 Jul-10
50%
- conference calls and/or sync delivery tools
EOT.OE 5.3.5.10 P Professional Society Outreach Aug-09 Jul-10 Ping [20%] 100% 100% 100% 75% 50%
Collect information on professional society
Ping 100% 100% 100% 75%
EOT.OE 5.3.5.10.1 P
Aug-09 Jul-10
50%
meetings throughout the year
Review and select future outreach events
Ping 100% 100% 100% 75%
EOT.OE 5.3.5.10.2 P quarterly: e.g. AAAS, APS, AGU, ACS, NSTA,
Aug-09 Jul-10
50%
Tapia etc
PY5 - presentations, exhibits at conferences
100% 100% 100% 75%
EOT.OE 5.3.5.10.3 P
Aug-09 Jul-10
50%
and outreach events throughout the year
EOT.OE 5.3.5.11 O TeraGrid Pathways Aug-09 Jul-10 100% 100% 100% 75% 50%
EOT.OE 5.3.5.11.1 O Consulting of under-represented populations AUSS Aug-09 Jul-10 100% 100% 100% 75% 50%
EOT.OE 5.3.5.11.2 O Fellowships for under-represented populations AUSS Aug-09 Jul-10 100% 100% 100% 75% 50%
EOT.OE 5.3.5.11.3 O Mentoring of under-served populations AUSS Aug-09 Jul-10 100% 100% 100% 75% 50%
Develop student competitions and awards in
Wiziecki [20%] 100% 100% 100% 75%
EOT.OE 5.3.5.12 P
Aug-09 Jul-10
50%
conjunction with National Science Olympiad
Work with Olympiad to establish computational
Wiziecki 100% 100% 100% 75%
EOT.OE 5.3.5.12.1 P
Aug-09 Dec-09
50%
science component
Wiziecki 100% 100% 100% 1 competition completed, one 50% 1 competition completed, one
EOT.OE 5.3.5.12.2 P Develop competitions Jan-10 May-10
remaining in May
remaining in May
Wiziecki 100% 100% 100% 1 competition completed, one 50% 1 competition completed, one
EOT.OE 5.3.5.12.3 P Deliver competitions and make awards Apr-10 Jul-10
remaining in May
remaining in May
IU Robert Ping[20%];
PSC Laura
EOT.OE 5.3.6 Outreach Events Extension ALL TG Aug-10 Jul-11
McGinnis[10%]; SDSC
Ange Mason[20%] 0.5
FTE for Extension
Campus Champions monthly training sessions
50% 25%
EOT.OE 5.3.6.1 M
Aug-10 Jul-11
- conference calls and/or sync delivery tools
EOT.OE 5.3.6.2 P Professional Society Outreach Aug-10 Jul-11 Ping [20%] 50% 25%
Collect information on professional society
Ping 50% 25%
EOT.OE 5.3.6.3 P
Aug-10 Jul-11
meetings throughout the year
Review and select future outreach events
Ping 50% 25%
EOT.OE 5.3.6.4 P quarterly: e.g. AAAS, APS, AGU, ACS, NSTA,
Aug-10 Jul-11
Tapia etc
Extension - presentations, exhibits at
50% 25%
EOT.OE 5.3.6.5 P conferences and outreach events throughout
Aug-10 Jul-11
the year
Develop student competitions and awards in
Wiziecki [20%] 50% 0%
EOT.OE 5.3.6.10 P
Aug-10 Jul-11
conjunction with National Science Olympiad
Work with Olympiad to establish computational
Wiziecki 50% 0%
EOT.OE 5.3.6.11 P
Aug-10 Dec-10
science component
EOT.OE 5.3.6.12 P Develop competitions Aug-10 May-11 Laura McGinnis 50% 25%
Deliver competitions and make awards for
Laura McGinnis 100% 0% SC10 event and TG11
EOT.OE 5.3.6.13 P
Nov-10 Jul-11
SC10
Deliver competitions and make awards for
Laura McGinnis 0% 0% SC10 event and TG11
EOT.OE 5.3.6.13 P
Nov-10 Jul-11
TG11
NCSA Edee Wiziecki
[20%]; PSC Laura
McGinnis [10%], Robin
Flaus [10%], Phil Blood
EOT.Edu 5.4.3 Education PY5 ALL TG Aug-09 Jul-10 [5%], Shawn Brown
[10%]; Purdue KayHunt
[50%], TBD [25%],
SDSC Ane Mason
[20%], Diane Baxter
EOT.EDU 5.4.3.2 M Education workshops May-10 Aug-10 100% 100% 90%
EOT.EDU 5.4.3.3 M monthly newsletter of EOT resources Aug-09 Mar-10 100% 100% 100% 75% 50%
EOT.EDU 5.4.3 P EOT Highlights 2010 Apr-10 Dec-10
EOT.EDU 5.4.3.1 P Request stories from EOT team Apr-10 Jul-10 Mason [20%] 100% 100% 100%
EOT.EDU 5.4.3.2 P Select and write stories for inclusion Jul-10 Oct-10 Mason, EOT team 100% 100% 50%
Mason, EOT team, 100% 75% 25%
EOT.EDU 5.4.3.3 P Prepare layout and proof Sep-10 Dec-10
Bruett Consulting
Bruett Consulting 100% Distributed at SC10, to RPs, and to 0% will be printed in Oct 2010
EOT.EDU 5.4.3.4 P Print Highlights Sep-10 Oct-10
NSF
EOT.EDU 5.4.3.5 M PY5 - SC10 Education Program Jan-10 Nov-10 100% 75% 75%
NCSA Edee Wiziecki
[20%]; PSC Laura
McGinnin [10%], Robin
Flaus [10%], Phil Blood
[5%], Shawn Brown
EOT.Edu 5.4.4 Education Extension ALL TG Aug-10 Jul-11
[10%]; Purdue KayHunt
[50%], TBD [25%],
SDSC Ane Mason
[20%], Diane Baxter
[20%] Extension Year
EOT.EDU 5.4.4.1 M Education workshops May-11 Jul-11
EOT.EDU 5.4.4.2 M monthly newsletter of EOT resources Aug-10 Mar-11 50% 25%
n/a
This has been changed to combine
EOT.EDU 5.4.4.3 P EOT Highlights 2011 Jan-11 Mar-11
the EOT and SH docs into a single
pub. XD Transition. This will now be
tracked under 5 2 4 1
EOT.EDU 5.4.4.3.1 P Request stories from EOT team Jan-11 Mar-11 Mason [20%] n/a
EOT.EDU 5.4.4.8 M Extension - Education Program Jan-11 Jul-11 5% Planning has started
EOT.EDU.R
Aggregate RP Education & Outreach
5.4.7 O
Aug-09 Mar-10
P
Operations
EOT.EDU.R
100% 100% 100% 75%
5.4.7.1 O NCSA RP Education & Outreach Operations Aug-09 Mar-10
50%
P NCSA
EOT.EDU.R
100% 100% 100% 75%
5.4.7.2 O IU RP Education & Outreach Operations Aug-09 Mar-10
50%
P IU
EOT.EDU.R
100% 100% 100% 75%
5.4.7.3 O LONI RP Education & Outreach Operations Aug-09 Mar-10
50%
P LONI
EOT.EDU.R
Ferguson [50%], TBD 100% 100% 100% 75%
5.4.7.4 O NICS RP Education & Outreach Operations Aug-09 Mar-10
50%
P NICS
[100%] TBD [50%]
EOT.EDU.R
5.4.7.5 O ORNL RP Education & Outreach Operations Aug-09 Mar-10
P ORNL
Budden[5%],
100% 100% 100% 75%
Flaus[15%],
Graham[15%],
Hanna[5%], Light[5%],
McGinnis[25%],
EOT.EDU.R
Nystrom[10%], Brown,
5.4.7.6 O PSC RP Education & Outreach Operations Aug-09 Mar-10
50%
P.PSC
S[2%], Cubbison[10%],
Hackworth[15%],
Maiden[10%],
Domaracki[50%],
Czech[50%],
Ishwad[25%],

QSR 2010Q4 Report Values
QSR 2010Q3 Report Values
QSR 2010Q2 Report Values
QSR 2010Q1 Report Values
QSR 2009Q4 Report Values
Project-ID
WBS
O P M
Work Package Dependencies Planned
Cross Start End
Description
Task Area Date Date
Resources
Name [%FTE]
12/31/10 AD Status 12/31/10 AD Notes 09/30/10 AD Status 09/30/10 AD Notes 6/30/10 AD Status 6/30/10 AD Notes 3/31/10 AD Status 3/31/10 AD Notes
12/31/09 AD
Status
12/31/09 AD Notes
EOT.EDU.R
100% 100% 100% 75%
5.4.7.7 O PU RP Education & Outreach Operations Aug-09 Mar-10
P PU
EOT.EDU.R
Baxter [37%], Mason 100% 100% 100% 75%
5.4.7.8 O SDSC RP Education & Outreach Operations Aug-09 Mar-10
P SDSC
[20%]
EOT.EDU.R
Armosky [25%] 100% 100% 100% 75%
5.4.7.9 O TACC RP Education & Outreach Apr-09 Mar-10
P TACC
EOT.EDU.R
100% 100% 100% 75%
5.4.7.10 O UCANL RP Education & Outreach Operations Apr-09 Mar-10
P UCANL
EOT.RPOE 5.4.8 O Education &Outreach Aug-10 Mar-11 0.78 FTE for Extension
Year
EOT.RPOE.
50% 25%
5.4.8.1 O NCSA RP Education & Outreach Operations Aug-10 Mar-11
NCSA
EOT.RPOE.I
50% 25%
5.4.8.2 O IU RP Education & Outreach Operations Aug-10 Mar-11
U
EOT.RPOE.
50% 25%
5.4.8.3 O LONI RP Education & Outreach Operations Aug-10 Mar-11
LONI
EOT.RPOE.
Ferguson [50%], TBD 50% 25%
5.4.8.4 O NICS RP Education & Outreach Operations Aug-10 Mar-11
NICS
[100%] TBD [50%]
Budden[5%],
50% 25%
Flaus[15%],
Graham[15%],
Hanna[5%], Light[5%],
McGinnis[25%],
EOT.RPOE.
Nystrom[10%], Brown,
5.4.8.6 O PSC RP Education & Outreach Operations Aug-10 Mar-11
PSC
S[2%], Cubbison[10%],
Hackworth[15%],
Maiden[10%],
Domaracki[50%],
Czech[50%],
Ishwad[25%],
EOT.RPOE.
50% 25%
5.4.8.7 O PU RP Education & Outreach Operations Aug-10 Mar-11
PU
EOT.RPOE.
Baxter [37%], Mason 50% 25%
5.4.8.8 O SDSC RP Education & Outreach Operations Aug-10 Mar-11
SDSC
[20%]
EOT.RPOE.
Armosky [25%] 50% 25%
5.4.8.9 O TACC RP Education & Outreach Aug-10 Mar-11
TACC
EOT.RPOE.
50% 25%
5.4.8.10 O UCANL RP Education & Outreach Operations Aug-10 Mar-11
UCANL
EOT.OCCP 5.4.9 O Outreach: Campus Champions Program Aug-10 Mar-11 1.95 FTE
EOT 5.5 Objective 5.5 Projects: Evaluation
Abbott - external
To SI, RPs,
EOT.EVAL 5.5.2 PY5 Evaluation
Aug-09 Jul-10 evaluator on consulting
NSF
agreement
All RPs collect data at all events throughout
all RPs 100% 100% 100% 75%
EOT.EVAL 5.5.2.1 O
Aug-09 Jul-10
year and share with external evaluator
Distribute surveys to people attending events
Abbott 100% 100% 100% 10%
EOT.EVAL 5.5.2.2 M
Feb-10 Jul-10
held 6 months ago
EOT.EVAL 5.5.2.2 M Quartlery summary of data Dec-09 Jul-10 Abbott 100% 100% 75% 50%
EOT.EVAL 5.5.2.3 M Mid-year analysis of data Jan-10 Feb-10 Abbott Completed Completed Completed Completed
EOT.EVAL 5.5.2.4 M Full analysis report annually Jul-10 Jul-10 Abbott 100% 100%
EOT.EVAL.
5.5.3 PY5 - Evaluations offered by RPs Aug-09 Jul-10
RP
EOT.EVAL.
100% 100% 100% 75%
5.5.3.1 O Collect data throughout year Aug-09 Jul-10
RP IU
UC Elizabeth
EOT.ER 5.6 P Objective 5.5 External Relations
Leake[100%]
PY5 - working group meetings; info sharing
100% 100% 100% 75%
EOT.ER 5.6.2 O
Aug-09 Jul-10
among RP ER staff
PY5 - Quarterly Reporting (9/30, 12/31, 3/31,
100% 100% 100% 50%
EOT.ER 5.6.4 O
Aug-09 Jul-10
6/30)
EOT.ER.RP 5.6.5 O Aggregate RP External Relations Operations Aug-09 Mar-10
EOT.ER.RP.
Barker [40%], Bell 100% 100% 100% 75%
5.6.5.1 O NCSA RP External Relations Operations Aug-09 Mar-10
NCSA
[25%]
EOT.ER.RP.
100% 100% 100% 75%
5.6.5.2 O IU RP External Relations Operations Apr-09 Mar-10
IU
EOT.ER.RP.
100% 100% 100% 75%
5.6.5.3 O LONI RP External Relations Operations Apr-09 Mar-10
LONI
EOT.ER.RP.
100% 100% 100% 75%
5.6.5.4 O NICS RP External Relations Operations Apr-09 Mar-10
NICS
EOT.ER.RP.
5.6.5.5 O ORNL RP External Relations Operations Apr-09 Mar-10
ORNL
EOT.ER.RP.
Schneider[50%], 100% 100% 100% 75%
5.6.5.6 O PSC RP External Relations Operations Aug-09 Mar-10
PSC
Williams[50%]
EOT.ER.RP.
100% 100% 100% 75%
5.6.5.7 O PU RP External Relations Operations Apr-09 Mar-10
PU
EOT.ER.RP.
Zverina [50%] 100% 100% 100% 75%
5.6.5.8 O SDSC RP External Relations Operations Aug-09 Mar-10
SDSC
EOT.ER.RP.
Singer [25%] 100% 100% 100% 75%
5.6.5.9 O TACC RP External Relations Apr-09 Mar-10
TACC
EOT.ER.RP.
100% 100% 100% 75%
5.6.5.10 O UC/ANL RP External Relations Operations Apr-09 Mar-10
UCANL
Communica
Leake [10%], ER team 100% 100% 90% 25%
EOT.ER 5.6.7 M TG10
tions and all RPs Aug-09 Jul-10 members
PR
Booth
EOT.ER 5.6.9 M SC10
ADs
Coordinated by Leake 100% 75% 50%
May-10 Dec-10
planning
[10%] ER team
Identify new venues for disseminating
Leake [10%] 100% 100% 100% 75%
EOT.ER 5.6.10 P
information about TeraGrid among science
Aug-09 Jul-10
and engineering communities to attract new
users to TeraGrid resources and services
Work with ER team, AUS, ADs, and TG
Leake 100% 100% 100% 75%
EOT.ER 5.6.10.1 P
Aug-09 Jul-10
Forum to identify venues
EOT.RPERC 5.6.5 O Aggregate RP External Relations Operations Aug-10 Mar-11 0.75 FTE for Extension
EOT.RPERC
Barker [40%], Bell 50% 25%
5.6.5.1 O NCSA RP External Relations Operations Aug-10 Mar-11
NCSA
[25%]
EOT.RPERC
50% 25%
5.6.5.2 O IU RP External Relations Operations Aug-10 Mar-11
IU
EOT.RPERC
50% 25%
5.6.5.3 O LONI RP External Relations Operations Aug-10 Mar-11
LONI
EOT.RPERC
50% 25%
5.6.5.4 O NICS RP External Relations Operations Aug-10 Mar-11
NICS
EOT.RPERC
50% 25%
5.6.5.5 O ORNL RP External Relations Operations Aug-10 Mar-11
ORNL
EOT.RPERC
Schneider[50%], 50% 25%
5.6.5.6 O PSC RP External Relations Operations Aug-10 Mar-11
PSC
Williams[50%]
EOT.RPERC
50% 25%
5.6.5.7 O PU RP External Relations Operations Aug-10 Mar-11
PU
EOT.RPERC
Zverina [50%] 50% 25%
5.6.5.8 O SDSC RP External Relations Operations Aug-10 Mar-11
SDSC
EOT.RPERC
Singer [25%] 50% 25%
5.6.5.9 O TACC RP External Relations Aug-10 Mar-11
TACC
EOT.RPERC
50% 25%
5.6.5.10 O UC/ANL RP External Relations Operations Aug-10 Mar-11
UCANL
Write stories about TeraGrid (non RP-specific)
Leake [10%], ER team 50% 25%
EOT.ERCom
5.6.7 M
resources and services. Customize existing
Aug-10 Mar-11
members, Grad
m
stories for NSF-OCI, OLPA, and other
Student, 0.87 FTE for
purposes
Extension
Graphic Design assistance with the facilitation
50% 25%
EOT.ERCom
5.6.9 M and updating of TeraGrid’s web
Aug-10 Mar-11
m
content—specifically images and graphic files
Graduate assistant to help with research, list
0% 0%
EOT.ERCom
5.6.10 P maintenance, and ER XD transition
Aug-10 Mar-11
m
documentation
Internal relations activities. International
EOT.ERCom
50% 25%
5.6.10.1 P collaboration. Training in support of
Aug-10 Mar-11
m
international communication
50%
50%
50%
50%
50%
50%
50%
25%
50%
50%
50%
50%
50%
50%
50%
50%
50%
20%
50%
50%

QSR 2010Q4 Report Values
QSR 2010Q3 Report Values
QSR 2010Q2 Report Values
QSR 2010Q1 Report Values
QSR 2009Q4 Report Values
Project-ID
WBS
O P M
Work Package Dependencies Planned
Cross Start End
Description
Task Area Date Date
Resources
Name [%FTE]
12/31/10 AD Status 12/31/10 AD Notes 09/30/10 AD Status 09/30/10 AD Notes 6/30/10 AD Status 6/30/10 AD Notes 3/31/10 AD Status 3/31/10 AD Notes
12/31/09 AD
Status
12/31/09 AD Notes
Science Writing (stories written and
EOT.SW 5.7
Aug-09 Jul-10
distributed)
PSC Michael
100% 100% 100% 75%
EOT.SW 5.7.3 O Science Writing PY5 Aug-09 Jul-10 Schneider[20%],
50%
Shandra Williams[20%]
Review and select science stories to highlight,
ER team; coordinated 100% 100% 100% 75%
EOT.SW 5.7.3.1 O
RPs, GIG Aug-09 Jul-10
50%
write and disseminate throughout year
by Leake
EOT.SW 5.7.3.2 O Select and write stories each quarter RPs, GIG Aug-09 Jul-10 ER team 100% 100% 100% 75% 50%
Pubish stories quarterly on web; disseminate
ER team 100% 100% 100% 75%
EOT.SW 5.7.3.3 O
GIG, UFP Aug-09 Jul-10
50%
through media - iSGTW HPCWire etc
EOT.SW 5.7.4 O Science Writing Extension Aug-10 Jul-11
PSC Michael
Schneider[20%],
Shandra Williams[20%]
Review and select science stories to highlight,
ER team; coordinated 50% 25%
EOT.SW 5.7.4.1 O
RPs, GIG Aug-10 Jul-11
write and disseminate throughout year
by Leake
EOT.SW 5.7.4.2 O Select and write stories each quarter RPs, GIG Aug-10 Jul-11 ER team 50% 25%
Pubish stories quarterly on web; disseminate
ER team 50% 25%
EOT.SW 5.7.4.3 O
GIG, UFP Aug-10 Jul-11
through media - iSGTW HPCWire etc
ER team 0% 0% Scheduled to start in January or when
Scheduled to start in January or when
EOT.SW 5.7.4.4 O XD Transition place holder GIG, UFP
the XD winner is identified
the XD winner is identified
Computational Science Problem of the
EOT.CSPW 5.8
Aug-10 Jul-11
Week (EOT CSPW)
Computational Science Problem of the Week
0.68 FTE 50% 25%
EOT.CSPW 5.8.1 O
Aug-10 Jul-11
(EOT.CSPW)
EOT END

D
EU-US HPC Summer School Report
EU-US HPC Summer School
Report
December 24, 2010
The first European US Summer School on HPC Challenges in Computational
Sciences organized jointly by Europe’s DEISA project and the US/NSF TeraGrid project
was held at Santa Tecla Palace in Acireale, Sicily from Oct 3-7, 2010.
The attendees were asked about their goals for attending and the extent to which the
summer school met those goals. Overwhelmingly, the attendees said that the summer
school met their goals, and quite a few people indicated that the event fully and
completely met their goals, and a couple people said the event exceeded their
expectations. Two people indicated that there were topics that could have been covered
in more depth to aid them including fault tolerance, and more on numerical algorithms.
When asked for their overall assessment of how well the summer school met their goals,
the attendees responded as follows:
• Excellent – 16 - 33.3%
• Very good – 25 - 52.1%
• Good – 6 - 12.5%
• Fair – 1 - 2.1%
• Poor – 0

Among the participants were sixty graduate students or post-docs, selected from more
than 100 applications: 24 from US and 35 from EU universities and research institutions.
Students came from a variety of disciplines, among them astronomy, atmospheric
sciences, chemistry, computer science, engineering, mathematics and physics.
Approximately 20% of the attendees were female students. The list of attendees is
included in Appendix B.
Twenty-five high level speakers addressed major fields of computational science, with
nine speakers from the US and sixteen from Europe. Areas covered included Challenges
by Scientific Disciplines, Programming, Performance Analysis & Profiling, Algorithmic
Approaches & Libraries, and Data Intensive Computing and Visualization.
“The summer school HPC Challenges in Computational Sciences was an excellent
opportunity for me to get an overview of the complete spectrum of High Performance
Computing and Computational Science. Attending talks from the large number of
distinguished speakers from almost every domain of computational science and High
performance computing has enabled me to get a very clear idea of the current trends in
the area”, one of the students stated afterwards.
This was planned from the outset and a joint DEISA-TeraGrid effort. “Our primary
objective for the student experience was to advance computational sciences by enabling
and stimulating future international collaboration, innovation, and discovery through the
most effective use of HPC,” said Hermann Lederer from the DEISA coordination team at
RZG, Garching. “We hope to continue with such events every year— alternating
between EU and US destinations,” said TeraGrid Forum Chair John Towns, National
Center for Supercomputing Applications. “The overwhelmingly positive feedback of the
students, in which 85% rated the event as very good or excellent, can be taken as a
mandate”, he added.
The agenda for the event is included as Appendix A. The presentation slides are
available at: http://www.deisa.eu/Summer-School/talks
A survey was conducted among the attendees and presenters to capture feedback from the
event to assess the impact of this event, and to help plan for future events. A total of 50
responses were provided from among the 60 participants, and 13 responses ere collected
from the 35 presenters.
The make-up of the attendees responding to the survey included:
• Postdoc – 6 - 12.2%
• Graduate – 36 - 73.5%
• HPC Center staff – 4 - 8.4%
• Other – 3 - 6.1%
The make-up of the presenters (and staff) responding to the survey included:

• Faculty – 3 - 37.5%
• Researcher – 2 - 25%
• HPC Center staff – 2 - 25%
• Other – 1 – 12.5%
The attendees were from among the following fields:
• Physics – 14
• Computer science – 10
• Chemistry – 7
• Engineering – 6
• Astronomy – 2
• Other – atmospheric science, biology, mathematics
The survey respondents were asked to indicate what they found to be most useful from
the summer school experience. Many of the responses indicated that they felt very
positive about the broad range of topics that were covered including scientific areas and
technological areas related to HPC. The attendees felt they learned a great deal about
HPC tools, resources, and the directions that others in the community were taking that
would help to guide their own endeavors. Many of the attendees enjoyed learning about
the challenges addressed in a diverse array of disciplines, which in turn helped them to
understand they faced similar challenges and similar opportunities for advancing their
own research. A number of attendees also mentioned that it was beneficial to learn about
career opportunities.
The survey respondents were asked to indicate what they found to be least useful from
the summer school experience. A number of attendees mentioned that they would like to
have heard less about the detailed scientific results and the theory behind the science,
with a preference for more information about the computational details and
computational methodologies that were applied to advance the research. A number of
attendees would have preferred more hands-on aspects during the summer school,
although some attendees said that would only have reduced the time to learn about the
breadth of topics that were covered, that were overall considered to be very good by the
vast majority of people responding to the survey. Some respondents mentioned that they
would like to have heard more about algorithms. There was a very strong negative
response by many attendees to the situation with the Internet access, which required work
by the staff at the site to rectify lack of access and charges to use the Internet. These
problems were worked out with the hotel on-site. There were a few negative comments
about offering parallel sessions, as these people wanted to attend the competing sessions,
yet many people appreciated the parallel sessions as they weren’t interested in both
topics, and eliminating the parallel tracks would make the summer school much longer.
A couple people suggested that the presentations could be shortened.
When asked to provide advice for future, many respondents said that all of the talks
should address the HPC challenges, general computational problems and solutions, and

the talks should address less on the specific research work and specific formulas. Some
respondents would like to hear more about GPU programming, and more coverage of
other fields of science including chemistry and biology. It was suggested that young
researchers should talk about their research and seek advice from the group, including the
more senior presenters, to guide them in their research. It was mentioned that slides from
all presenters should be made available as soon after the presentations as possible. The
attendees suggested that the use of Google Docs would help to address this. A number
of people again repeated the request for hands-on sessions. A few people suggested that
the summer school could be between ½ a day and full day longer to cover more topics.
The survey respondents recommended conducting the summer school so as not to
compete with classes, and to be closer to a major airport to eliminate the travel hassles of
getting to the meeting site. The attendees liked the BOF sessions during lunch, and
suggested that they start earlier in the week to allow people to meet and share information
on topics of common interest. The attendees would like to have received a list of all
participants and presenters, and their fields of research to help people make connections
during the summer school.
When asked how the summer school would impact their own work, most of the
respondents said the summer school would not change their research plans, but many of
them planned to apply the tools they learned about to enhance their research. Many of
the attendees plan to apply the code profiling, tuning, and optimization techniques on
their own codes, plan to explore the use of numerical libraries, and utilize the
visualization tools they learned about. Some of the attendees plan to explore some of the
new frameworks they learned about (e.g. UPC, StarSs), and to share what they learned
with their colleagues.
Overall, nearly all of the respondents felt that the event was well organized, that the
location was wonderful (except for transportation issues to and from the airport), and that
the accommodations were quite good (except for early Internet problems). Some people
felt that being near a major airport and near a city to offer more off-hours opportunities
would be useful. One person mentioned that the hotel did not provide ample options for
vegetarians.
A major emphasis for the event was to facilitate interactions among the attendees and the
presenters, and to allow ample time for people to share ideas, explore challenges and
opportunities, and to establish new colleagues and collaborations. The vast majority of
attendees felt that the summer school did allow plenty of time for people to interact and
talk. A number of people mentioned that they have new colleagues to share ideas. A few
indicated that new collaborations may result from the event. A few people indicated that
there were not enough others in the same field of study to build new connections, but that
the event overall was still very useful to their learning. A few people said more could be
done to encourage presenters to spread out among the attendees, and not sit together
during meals.

In summary, the attendees and presenters strongly recommended that further summer
schools be conducted in the US and Europe.

Appendix A - Summer School Agenda
Mon Oct 4, 2010:
9:00 – 12:00 - HPC Challenges and Technology
• HPC Challenges, Barry Schneider, NSF, US
• DEISA overview, Hermann Lederer, RZG, Germany
• TeraGrid / XD overview, John Towns, NCSA, US
• PRACE overview, Giovanni Erbacci, CINECA, Italy
13:30 – 15:30 - Challenges by Scientific Disciplines I
• Parallel track 1: Materials & Life Sciences
Nano technology: Thomas Schulthess, ETHZ, Zurich, & CSCS, Switzerland
• Parallel track 2: QCD
Quantum Chromo Dynamics: Richard Brower, Boston University, US
15:50 – 16:50 - Challenges by Scientific Disciplines II
• Parallel track 1: Materials & Life Sciences
Molecular Dynamics simulation packages I
Gromacs: David van der Spoel, Univ. Upsala, Sweden
• Parallel track 2: Plasma Physics
Fusion energy research: Frank Jenko, MPI for Plasma Physics, Garching,
Germany
Tue Oct 5, 2010:
9:00 – 12:00 - Programming
• Overview on Mixed MPI/OpenMP Programming, UPC, CAF, StarSs Model
Giovanni Erbacci, CINECA, Bologna, Italy
David Henty (Lecture 1, Lecture 2, Lecture 3,Lecture 4), EPCC, University of
Edinburgh, UK
Josep M. Perez, Barcelona Supercomputer Center, Spain
13:30 – 15:20 - Challenges by Scientific Disciplines III
• Parallel track 1: Astro Sciences
Astro sciences, Cosmology: Mike Norman, UCSD, San Diego, US
• Parallel track 2: Materials & Life Sciences
Molecular Dynamics simulation packages II
Amber: Thomas Cheatham, Univ. Utah, US
CP2K: Marcella Iannuzzi, Univ. Zurich, Switzerland
15:40– 16:35 - Challenges by Scientific Disciplines IV
• Parallel track 1: Climate Research
Climate Research: Patrick Joeckel, DLR, Germany
• Parallel track 2: Materials & Life Sciences
Molecular Dynamics simulation packages III

QuantumEspresso: Paolo Gianozzi, Univ. Udine, Italy
16:35– 17:30 - Algorithmic Approaches & Libraries I
Advanced algorithms for long-range interactions: Axel Arnold, University of
Stuttgart, Germany
Wed Oct 6, 2010:
9:00 – 12:00 - Performance Analysis & Profiling
• Points package, Scalasca, etc.
Philip Blood, PSC, Pittsburgh, US
Bernd Mohr, FZJ, Juelich, Germany
13:30 – 17:00 - Algorithmic Approaches & Libraries II
• Particle-in-Cell methods in plasma physics: Roman Hatzky, IPP, Garching,
Germany
• Software environment for efficient flow simulations: Joachim Bungartz, TUM,
Munich, Germany
• Numerical Libraries: Tony Drummond, LBNL, Berkeley, US
Thu Oct 7, 2010:
9:00 – 12:00 - Data Intensive Computing and Visualization
• Data intensive computing: John R Johnson, PNL, US
• Visualization
Sean Ahern, ORNL, Knoxville, US
Uwe Woessner, HLRS, Stuttgart, Germany

Appendix B – Attendees, Staff and Presenters
Attendees
First Name Last Name Institute Country
Xavier Abellan Ecija Barcelona Supercomputing Center Spain
Shannen Adcock University of Arkansas USA
Fabio Affinito CINECA Italy
Sean Ahern University of Tennessee USA
Axel Arnold University of Stuttgart, ICP Germany
Thomas Auckenthaler TU München Germany
Mirco Bazzani Physics department, Parma Italy
University
Philip Blood Pittsburgh Supercomputing Center, USA
Carnegie Mellon University
Steven Böing Delft University of Technology Netherlands
Jonathan
Richard Brower Boston University USA
Hans- Bungartz TUM, Department of Informatics Germany
Joachim
Benjamin Byington University of California, Santa Cruz USA
Ramon Calderer University of Illinois at Urbana- USA
Champaign
Thomas Cheatham University of Utah USA
Galen Collier Clemson University USA
Benjamin Cruz Perez University of Puerto Rico at
Mayaguez
USA,
Puerto Rico
Tony Drummond Lawrence Berkeley National USA
Laboratory
Philipp Edelmann Max-Planck-Institute for
Germany
Astrophysics
Giovanni Erbacci CINECA Italy
Johannes Feist ITAMP, Harvard-Smithsonian USA
Center for Astrophysics
James Ferguson National Institute for Computational USA
Sciences
Daniel Fletcher University of Warwick (EPRSC) UK
Wolfgang Gentzsch Max-Planck-Institute Germany
Paolo Giannozzi University of Udine and IOM- Italy
Democritos
Andreas Goetz San Diego Supercomputer Center USA
Luca Graziani Max-Planck-Institute for
Germany
Astrophysics, Garching
Thomas Guillet CEA/DSM France
Roman Hatzky IPP Germany
Andreas Hauser Genzentrum, LMU Germany

Tobias Heidig MPI Magdeburg Germany
David Henty EPCC UK
Matthew Hogan University of Massachusetts USA
Lorenz Hüdepohl Max-Planck-Institut für Astrophysik Germany
Marcella Iannuzzi University of Zurich Switzerland
Sebastian Illi University of Stuttgart, IAG Germany
Sam Jacobs University of Antwerp Belgium
Frank Jenko IPP Germany
Patrick Jöckel DLR Institute for Atmospheric Germany
Physics
John Johnson Pacific Northwest National USA
Laboratory
Alan Kelly University of Glasgow UK
Hans- Klingshirn MPI für Plasmaphysik Germany
Joachim
Elizabeth Leake University of Chicago USA
Hermann Lederer Garching Computer Centre, Max- Germany
Planck-Society
Angelo Limone Max-Planck-Institut für
Germany
Plasmaphysik
Jeremy Logan University of Maine USA
Katie Maerzke Vanderbilt University USA
Chris Malone Stony Brook University USA
Bernd Mohr Jülich Supercomputing Centre Germany
Giuseppa Muscianisi University of Messina Italy
Sreeja Nag Massachusetts Institute of
USA
Technology
Stefan Nagele Vienna University of Technology Austria
Terrence Neumann Marquette University USA
Michael Norman SDSC, UCSD USA
Szilárd Páll Center For Biomembrane Research, Sweden
Stockholm
Benjamin Payne Missouri Science and Technology USA
Renate Pazourek Vienna University of Technology Austria
Christian Pelties Department of Earth &
Germany
Environmental Sciences, LMU
Munich
Josep M. Perez Barcelona Supercomputing Center Spain
Marcelo Ponce Rochester Institute of Technology USA
Johannes Reetz Garching Computer Centre, Max- Germany
Planck-Society
Martin Roderus Technische Universität München Germany
Dominic Roehm Institute for Computational Physics Germany
Anastasia Romanova Max-Planck-Institute for
Germany
Mathematics in the Science
Romelia Salomon San Diego Supercomputing Center USA

Staff and Presenters
Ferrer
Annika Schiller Jülich Supercomputing Centre Germany
Barry Schneider National Science Foundation USA
Mandes Schönherr Institute of Physical Chemistry Switzerland
Thomas Schulthess ETHZ Switzerland
Jacob Searcy University of Oregon USA
Daniele Selli Technische Univarsitat Dresden Germany
Cengiz Sen University of Tennessee USA
Andrey Sharapov University of Pittsburgh USA
Pablo Souza University of Michigan USA
Filippo Spiga IBM and University of Milano US and Italy
Bicocca
Julien Thibault University of Utah USA
Daniel Told MPI für Plasmaphysik Germany
John Towns NCSA/University of Illinois USA
Volodymy Turchenko University of Calabria Italy
r
Ikram Ullah Royal Institute of Technology Sweden
(KTH)
David van der Spoel Department of Cell & Molecular Sweden
Biology, Uppsala University
Xingyu Wang New York University USA
Susann Wangnett Garching Computer Centre, Max- Germany
Planck-Society
Stefan Wenk Munich University, Geophysics Germany
Katelyn White University of California, Santa Cruz USA
Daniel Wilde EPCC Scotland
Uwe Woessner HLRS Germany
Guoxing Xia Max-Planck-Institute for Physics Germany
Anastasia Yanchilina Lamont Doherty Earth Observatory, USA
Columbia University
Haseeb Zia Technische Universität München Germany
First Name Last Name Institute Country
Sean Ahern University of Tennessee USA
Axel Arnold University of Stuttgart, ICP Germany
Philip Blood Pittsburgh Supercomputing Center, USA
Carnegie Mellon University
Richard Brower Boston University USA
Hans- Bungartz TUM, Department of Informatics Germany
Joachim
Thomas Cheatham University of Utah USA
Tony Drummond Lawrence Berkeley National USA

Laboratory
Giovanni Erbacci CINECA Italy
James Ferguson National Institute for Computational USA
Sciences
Paolo Giannozzi University of Udine and IOM- Italy
Democritos
Roman Hatzky IPP Germany
David Henty EPCC UK
Marcella Iannuzzi University of Zurich Switzerland
Frank Jenko IPP Germany
Patrick Jöckel DLR Institute for Atmospheric Germany
Physics
John Johnson Pacific Northwest National USA
Laboratory
Elizabeth Leake University of Chicago USA
Hermann Lederer Garching Computer Centre, Max- Germany
Planck-Society
Bernd Mohr Jülich Supercomputing Centre Germany
Michael Norman SDSC, UCSD USA
Josep M. Perez Barcelona Supercomputing Center Spain
Barry Schneider National Science Foundation USA
Thomas Schulthess ETHZ Switzerland
John Towns NCSA/University of Illinois USA
David van der Spoel Department of Cell & Molecular Sweden
Biology, Uppsala University
Uwe Woessner HLRS Germany