FY2010 - Oak Ridge National Laboratory

Director’s R&D Fund—
Ultrascale Computing and Data Science
vector fields at the boundary and (2) this approach admits different numerical discretizations, such as
finite elements or finite differences, in each physical domain, enabling a wide range of applications to
make use of this technique.
This project proposed a new general optimization-based approach where we minimize the difference
between scalar and/or vector fields at the domain interfaces subject to the constraining physics in each
domain. This year, we demonstrated that this proposed approach is a generalization of the widely
accepted mortar methods and will have applicability to a much wider class of problems. Additionally,
with an appropriate choice of regularization (either explicit based on generalized cross-validation or
implicit based on a premature solver termination dictated by an L-curve), this approach is as efficient and
potentially more robust than the existing methods. To demonstrate this, we examined three different
classes of problems. First, for a diffusion-reaction problem with an arbitrary number of domains and
mismatched discretizations, we showed that the error convergence, as expected, was provably either
second (in the H 1 norm) or third (in the L 2 norm) order. Second, we demonstrated the method accurately
captured the fluid-structure coupling interactions when the fluid is governed by the nonlinear Navier-
Stokes equations and the structure was linearly elastic. Finally, we showed that this method can
efficiently represent disparate timescales for a conjugate heat transfer problem with a moderately large
Rayleigh number (O(10 4 )). Lastly, this approach requires little additional implementation effort in
existing software efforts, as was demonstrated by using the large-scale LifeV (www.lifev.org) package for
these problems.
05243
MPI-3: Programming Model Support for Ultrascale Computer Systems
Richard L. Graham, Thomas Naughton, and Chao Wang
Project Description
Upcoming generations of ultrascale computer systems promise an unprecedented level of computational
capabilities and hand in hand provide a challenge to use these systems effectively. Hardware technology
challenges are driving these systems to be many-core and multicore systems with immense component
counts, and simulation codes, middleware, and system-level software need to be able to run in the face of
errors. We will work with the application developer aiming to run on these systems to develop scalable
strategies for dealing with such failures at the application and middleware level. We will investigate how
to partition the solution between these two levels in the context of the ubiquitous communication
standard, the Message Passing Interface (MPI) standard, and present proposed changes to this standard to
the MPI Forum for inclusion in the MPI-3.0 standard. We will look at solutions that aim to avoid failures
and at scalable mechanisms to recover once such failures have occurred.
Mission Relevance
DOE makes large investments in such areas as climate change, energy science and technology, and
material science, with simulation playing an important role in the discovery process. To meet the
simulation needs, DOE develops increasingly powerful and complex computer systems. This project aims
to provide a solution to one of the pressing issues facing those trying to use these increasingly complex
systems—harnessing the full potential of these systems in the face of component failures. With MPI
being the ubiquitous parallel communications and process control library used by scientific simulation
codes, providing fault tolerance in support of MPI is a key step in developing fault-tolerant applications.
By adding fault-tolerant capabilities to the MPI 3.0 standard, adding uncoordinated checkpoint/restart
71