National Energy Research Scientific Computing Center

4
NERSC ANNUAL REPORT 2015
Director’s Note:
A Year of Transitions
2015 was a year of transition and change for NERSC on many levels.
The most pronounced change was the center’s move from the Oakland Scientific
Facility in downtown Oakland to the new Shyh Wang Hall (also known internally as
the Computational Research and Theory facility, or CRT) on the main Berkeley Lab
campus. A significant amount of staff time was spent planning and implementing the
move of computational systems, file systems and auxiliary systems while minimizing
the impact on NERSC users and their ongoing projects.
Indeed, one of NERSC’s greatest challenges in 2015 was running multiple systems
across two different facilities with a constant number of staff. Despite having to
physically move NERSC’s largest system, Edison, and all file systems in November 2015,
NERSC was able to deliver more than the committed 3 billion hours to its users—the
same amount committed to in 2014 without the move. This was accomplished by
keeping the Hopper system in production until Cori Phase 1 was stable.
Wang Hall itself brings many new changes. The building was designed to be
extraordinarily energy efficient and expandable to meet the needs of next-generation
HPC systems. It features 12.5 megawatts of power, upgradable to over 40 megawatts, a
PUE (power usage effectiveness) of less than 1.1, ambient “free” cooling and the ability
to reclaim heat from the computers to heat the building. The new machine room
currently measures 20,000 square feet and features a novel seismic isolation floor to
protect NERSC’s many valuable resources.
Another big change in 2015 was the retirement of two of NERSC’s legacy systems,
Carver and Hopper. This was done in conjunction with the deployment of the first
phase of Cori and the Edison move. As of early 2016, the transfer of systems into the
new building was substantially complete and was accomplished with a minimum of
downtime. The process was facilitated by a first-of-a-kind 400 Gbs link, provided by
ESnet between Wang Hall and the Oakland Scientific Facility, that enabled moves of
data to be effectively transparent to users.
Another major undertaking for NERSC in 2015 was the NERSC Exascale Science
Applications Program (NESAP). We kicked off the project with 20 applications teams,
plus an additional 20 teams participating in training and general optimization efforts.
Through NESAP, Cray and Intel staff are able to work directly with code teams to
optimize applications and learn about using Cray and Intel performance tools, which
was unique because in most cases no such relationship previously existed.
2015 was also the year that NERSC decided there was enough momentum around
HPC and data-intensive science that we should form a department focused on data.
The new department has four groups, two of them new: a Data Science Engagement
group that engages with users of experimental and observational facilities, and an
Infrastructure Services group that focuses on infrastructure to support NERSC and our
data initiative. The existing Storage Systems Group and Data and Analytics Group
complete the new department.