Salman Habib - Los Alamos National Laboratory

• Use P^3M
‣ Grid lives on Opteron layer with FFT Poissonsolves
of up to 10,000^3, uses digital filtering
and Super-Lanczos differentiation to reduce
particle-grid interaction
‣ Particles live in Cell BE and interact on “subgrid
scales” by fast hand-coded routines
‣ Only simple grid info flows between Cells and
Opterons, so thin pipe problem is solved
• Avoid Particle Communication
‣ Particle communication between Cells at
every short-range step would be too slow,
avoid this using particle caching
‣ Intermittent nearest neighbor refresh (fast)
• On the Fly Analysis
‣ Avoid I/O as much as possible, analysis
routines must run as a part of the code Overload Zone (particle “cache”)
Nodal domain contains all particles that will ever cross into its reference volume (30% overhead can be reduced to few %). Particles
inside reference volume are “active”, i.e., used to compute self-consistent force, others are “passive”, i.e., used only as tracers (their
“active” self belongs to a different compute node); as they move, particles change their state based on local information. For the PM
piece, overloading is “exact”, for the N^2 part, error at the domain edge propagates inwards. But it is (i) small, and (ii) can be controlled
by increasing Katrin Heitmann, the boundary Los Alamos layer National thickness Laboratory or via periodic refreshes.
LBNL, March 12, 2009