National Energy Research Scientific Computing Center

User Support and Outreach 53
A view inside the LUX detector.
Photomultiplier tubes can pick
up the tiniest bursts of lights
when a particle interacts with
xenon atoms. Image: Matthew
Kapust, Sanford Underground
Research Facility
Supporting LUX with SciDB
The Large Underground Xenon (LUX) dark matter experiment operates nearly a mile underground
at the Sanford Underground Research Facility in South Dakota. The LUX instrument was built to
directly detect the faint interactions from galactic dark matter in the form of weakly interacting
massive particles (WIMPS). In its initial data run, LUX collected 83 million events, containing 600
million pulses, and wrote 32TB of raw data. Of this data, only 160 events survived basic analysis cuts
for potential WIMP candidates. The data rate for the new dark matter run, which started in 2014,
exceeds 250TB/year.
As a case study for high energy physics, NERSC’s DAS group worked with the LUX team to load the
raw data collected from the instrument’s inaugural run in 2013 into NERSC’s SciDB testbed—an
open source database system designed to store and analyze extremely large array-structured data. In
this case, analysis of this kind of data typically requires a researcher to search through about 1 million
10MB files within a 10TB dataset. If a WIMP candidate is spotted, the researcher would save the
candidate to another file—a process that is slow and cumbersome, taking at least a day and involving
analysis steps that are difficult to share. But the DAS group found that by using the SciDB testbed,
the same search took just 1.5 minutes from start to finish. Since the SciDB testbed was implemented
at NERSC in 2013, there has been so much interest in the use cases that the center deployed a
dedicated set of nodes for this kind of analysis. These nodes will be one of the main analysis platforms
for the LUX experiment.
Burst Buffer Early User Program Moves Forward
In August 2015, NERSC put out a Burst Buffer Early User Program call for proposals, asking
NERSC’s nearly 6,000 users to describe use cases and workflows that could benefit from accelerated
I/O. NERSC received over 30 responses from the user community. These proposals spanned the
DOE Office of Science and represented a breadth of use cases. NERSC staff evaluated each proposal
using the following criteria:
• Representation among all six DOE Office of Science programs
• Ability for application to produce scientific advancements