Annual Meeting - SCEC.org

Report | SCEC Research Accomplishments
from Cholame in central California to the southern termination of the San Andreas Fault on the Salton Sea. The SCEC M8
earthquake scenario represents the outer scale required for standard California seismic hazard calculations because there are
few, if any, larger ruptures in the existing USGS Unified California Earthquake Rupture Forecast (UCERF 2.0).
Running the M8 earthquake simulation involved a two-step process. First, we ran a dynamic rupture simulation, on NICS
Kraken supercomputer, to create a physically realizable slip-time history on the fault. Second, we ran a ground motion
simulation, on NCCS Jaguar supercomputer (the world’s fastest at the time), to model the anelastic seismic wave propagation
from the fault rupture. The latter calculation represented the 3D seismic velocity structure by 436 billion mesh points and ran
for 24 hours on Jaguar at full machine scale (more than 223,000 cores), making it the largest-ever earthquake simulation. The
M8 simulation team, led by Y. Cui, was recognized as an ACM Gordon Bell Finalist in 2010.
The SCEC M8 simulations show that SCEC’s AWP-ODC deterministic 3D wave propagation software can scale to simulate
any earthquake in the standard California earthquake rupture forecast up to 2Hz. The M8 simulations show that CME
researchers can run well-validated deterministic wave propagation simulations at 2Hz at the scale need to model the “worstcase”
seismic hazard earthquake scenarios in current California Earthquake Rupture Forecasts.
The M8 simulation results have led to several significant scientific conclusions: (1) the likelihood that large ruptures on the San
Andreas fault will transition from sub-shear speeds during rupture propagation; (2) the importance of directivity and basin
effects in ground motion amplification at high frequencies; and (3) the need to model off-fault plastic yielding and non-linear
site effects at frequencies above 1 Hz.
The SCEC M8 simulations used an advanced form of earthquake description, one based on dynamic rupture simulation, as
input to the simulation. Dynamic rupture simulations model friction-based slip on a fault so dynamic rupture simulations are
consistent with known physics of fault ruptures. Less physically-accurate, but less computationally intensive, are kinematic
rupture descriptions. Kinematic ruptures are not constrained by friction-based fault slip. Our M8 dynamic rupture simulation
progressed quickly into a Supershear rupture, reduced speed, and then returned to Supershear velocity, all during the course
of a single M8 earthquake. The fact that we observe Supershear velocities in our most physics-based rupture models suggests
that Supershear ruptures are both possible and common. By comparing ground motions from dynamic ruptures and
kinematic ruptures we have begun to establish the significant impact that Supershear rupture velocities have on ground
motions.
Another important result from the SCEC M8 simulations relates to the important of non-linear site effects. SCEC researchers
have significant experience with deterministic simulations up to 1Hz. However, as our M8 simulation work shows,
deterministic wave propagation simulation results about 1Hz are significantly less mature. M8 results showed unexpectedly
high, near-fault, ground motions. Our analysis of these results point to the need to model less-rigid, more flexible, materials at
the surface when simulating seismic waves at frequencies about 1Hz. Indications are that at frequencies above 1Hz, non-linear
effects become important. Above 1Hz, deterministic ground motion simulation results begin to diverge from observations
especially for intensity measures typically associated with higher frequency motions such as peak acceleration.
These unexpectedly high peak ground motions, at frequencies above 1Hz, indicate a need to model the plastic yielding of near
surface layers at higher frequencies. Until now, SCEC deterministic simulations at lower frequencies have not needed to
model this additional non-linear behavior. Results from M8 show that SCEC simulations will need to model non-linear
behavior as our deterministic simulations reach frequencies above 1Hz.
Hercules
Hercules simulation software, with development led by J. Bielak, is a finite-element parallel code that relies on an Octreebased
mesher and solves the anelastic wave equations by approximating the spatial variability of the displacements and the
time evolution with piecewise polynomial elements and central differences, respectively. PetaShake project has produced
Hercules improvements in the areas of single-processor tuning, and optimization of the communication topology, message
sizes, and messaging techniques. Checkpointing was implemented to reinitiate computations after unexpected interruptions.
Through these improvements and algorithm improvements, the scalability of Hercules has been established on up to 100,000
compute cores on NICS Kraken, and it has been successfully used in different earthquake validation and verification exercises,
such as the ShakeOut scenario.
98 | Southern California Earthquake Center