Computing for accelerator physics Ilya Agapov, 7/12/09, DESY ...

Computing for accelerator physics
Ilya Agapov, 7/12/09, DESY Computing Seminar

Contents
Overview of accelerators and computing requirements
Electromagnetic field and thermal calculations
Accelerator optics and beam dynamics codes
Collective effects
Energy deposition calculations and machine­detector interface
Start to end accelerator simulations
Control systems, on­line accelerator modeling and diagnostics

Areas where computing plays a role
I will cover beam physics
and some controls issues

Basic accelerator types and components
Linear Accelerators (injectors, linear collider,
FEL, spallation neutron sources)
Cyclotrons
Synchrotrons (light sources, storage rings)
Advanced accelerator concepts (plasma­wakefield)

Linear accelerators
Injectorts for synchrotrons
Neutron sources
Linear colliders (SLC, ILC, CLIC)
Accelerator­driven nuclear power
Ion therapy
DESY XFEL Layout
100keV test injector at PSI Accelerating cavity (from ACCEL) Dipole (from APS Argonne)

Synchrotrons
HEP and nuclear physics
Circular colliders
Light sources

Plasma­wakefield acceleration
Use wakes created in plasma by intense electron or laser beams
Accelerating tp ~1GeV over ~ 3 cm (would take ~10­30m with RF)
Challenges: energy spread, beam emittance,
maintaining accelerating gradient, module staging
Proof of concept SLAC, LBNL
From CERN Courier June 2007

Computing in the project lifecycle

EM field calculations
LHC main quad, ROXIE (from Russenschuk) Cavity electric field from www.gdfidl.de

Heat transfer, mechanical stress
Stress and heat transfer calculations for dumps, targets, cryostats
ANSYS
AUTODYN (now part of ANSYS)

CST particle studio
Collimator wakefiels (www.cst.com) Particle gun

vectorfields.com
OPERA­2D, OPERA­3D
Magnetostatics, thermal, quench,...
Commercial codes
Pros: CAD­level graphics, powerful mesh generation and solvers
Contras: expensive, hard to extend

Wakefield code ECHO (TU Darmstadt / DESY)
moving mesh
bunch
Slides S
I. Zagorodnov DESY)
Electromagnetic
Code for
Handling
Of
Harmful
Collective
Effects
Zagorodnov I, Weiland T., TE/TM Field Solver for Particle Beam Simulations without
Numerical Cherenkov Radiation// Physical Review – STAB,8, 2005.
Zagorodnov I., Indirect Methods for Wake Potential Integration // Physical Review
­STAB, 9, 2006.

in 2.5D stand alone application
in 3D only solver, modelling and meshing in CST Microwave Studio
Model and
mesh in CST
Microwave
Studio
Wakefield code ECHO (TU Darmstadt / DESY)
Preprocessor
in Matlab
ECHO 3D
Solver
It allows for accurate calculations on conventional PC with
only 1 p rocessor. To be p aralleliz ed …
Postprocessor
in Matlab

Beam optics
Accelerator dimensions given by:
maximum accelerating E field (linear)
maximum bending magnet strength (circular)
Tasks of beam optics –
steer the beam to the experimental station meeting the constraints:
✔ geometrical layout ­­ steering with dipole magnets – e.g. Desy to Schenefeld
✔ fit in the aperture (focusing)
✔ provide necessary beam sizes at experimental stations (focusing)
✔ correct chromatic effects (focusing depending on energy)
✔ in circular accelerators – in addition provide stability
Common approach (strong focusing) – build accelerators from blocks similar to light optics e.g.
dipole magnet – bend, quadrupole – focus/defocus, sextupole ­ correct aberrations,
RF cavity ­ accelerate

Linear optics
Start with equations of single­particle motion in the EM fields
In a coordinate system going with a reference on­axis particle all building blocks can be
represented by parametrized coordinate transformations
where x,x', y, y' – particle coordinates and trajectory angles with respect to reference orbit
Each transform depends on few parameters (usually just one).
For basic optics linear approximation is sufficient. The transform is a matrix
Taken from A. Streun,
lectures at ETH

Linear optics
A typical parametrization – through so­called twiss parameters
A beam transport system (e.g. beam parameters) easily given by matrix multiplication.
Writing a program to SIMULATE linear beam optics is straightforward and can be
done in a matter of days (fortran) or hours (matlab, mathematica, python, root etc.).
Almost every computer­inclined accelerator specialist has probably done it.
Computer­aided design:
Too many free parameters. Still designed by humans from analytical principles
starting from simple building blocks (FODO cell; final doublet/triplet)
Only tuning (matching) of optics: find exact magnet settings to fit the beam size at IP
Further issues, e.g.:
Powering constraints
xs= s cos s
Cost issues (tunnel length minimization, required current minimization)
Attempts at multiobjective genetic optimization have been made

Complications
Some magnet types can be more complicated (e.g. LHC magnet with spool pieces;
fringes etc.)
Sextupoles and higher order multipoles should be included (compute chromatic
functions)
Collective effects – at least simple calculations useful (Intra­beam scattering)
Aperture and layout information (to go to a more engineering design)
Input formats (portability between codes still bad but improving)
tracking required for:
Steering algorithms for linacs (PLACET)
For strongly non­linear fields (extraction through a quad pocket) no sensible
parametrization exists
Long­tern stability in storage rings – small perturbations play role – symplectic
integrators (non­linear dynamics)
Presence of synchrotron radiation, gas scattering

Steering (PLACET code, D.Schulte, A.Latina et al.)
Slides by A.Latina (FNAL)

Long­term stability, dynamic aperture
Poincare section for linear (left) and nonlinear maps (right)
Determining stability for large­amplitude particles requires long­term tracking
Symplectic integrators to avoid large error accumulation
Long term stability under influence of small random perturbations (RF noise, scattering)
requires sofisticated compute­intensive techniques (e.g. 6D Fokker­Planck equation)
Spin dynamics in storage rings
Codes: COSY, PTC,…
Julia Set

MAD­X
Widely used beam optics code www.cern.ch/mad
LHC standard; has built­in high precision integrators (PTC), treats hight order multipoles
Comprehensive set of beam physics processes
Matching of beam parameters
Spits out optics tables, can be plotted with external tools or built­in ps driver
call file="../LHC-cell.seq";
kqf = 0.010988503297557 ;
kqd = -0.011623337240602 ;
Beam, particle = proton, sequence=lhccell, energy = 450.0,
NPART=1.05E11, sige= 4.5e-4 ;
use,period=lhccell;
select, flag=twiss, clear;
select, flag=twiss,column=s,name,betx,bety,mux,muy;
twiss, sequence=lhccell,file='twiss-output';
match,sequence=lhccell;
constraint,sequence=lhccell,range=#e,mux=0.28,muy=0.31;
vary,name=kqf,step=1.0e-6;
vary,name=kqd,step=1.0e-6;
lmdif,calls=500,tolerance=1.e-21;
endmatch;
value, kqf;
value, kqd;

Collective effects
a computational physics research area similar to plasma simulations
Wakes and space charge (gdfidl, echo)
Wakefield acceleration (PIC OSIRIS)
Beam­beam effects in colliders (GUINEA­PIG)
Electron clouds (codes: HEADTAIL, ECLOUD, FAKTOR2)

Energy deposition and machine­detector
interface
Was not a problem in the early years
With more beam energy/intensity and superconducting magnets particle losses
due to scattering, collimation system performance etc. become more critical
Similar type of calculations as with HEP detectors (showers), but need to link it
with beam dynamics in the machine
Interfacing Monte­Carlo radiation transport (FLUKA, GEANT4, MCNP, MARS) to
accelerator tracking
Examples: STRUCT/MARS at FNAL; FLUKA/SIXTRACK for LHC; BDSIM
(standalone GEANT4 based tracking for ILC and ATF2)

BDSIM
Complicated geometries are possible (e.g. extraction with electron and photon
dump lines, bottom left).
Detailed or simplified equipment models (e.g. laser­wires)
Tracking in vacuum + secondaries
All Geant4 physics + fast tracking in vacuum and variance reduction techniques
Energy deposition
based on Geant4 + Root
Detector interface – Mokka and xml
29

Energy deposition, collimation, halo and backgrounds at ILC
ILC collimation system (top left), electron and photon halo (top right)
power losses in the extraction line (bottom left), energy deposition in a
final focus quadrupole (bottom right)
30

Start to end simulations
Several codes staged to simulate the whole accelerator, typically from injector to
the experimental station
IMPACT­T (Photo­injector), ELEGANT (Accelerator), Genesis (FEL process) for LCLS
(Y Ding et al.)
ASTRA + ELEGANT for PSI injector test facility (Y. KIM et al.)
ASTRA + ELEGANT + + CSRTRACK + GENESIS for DESY XFEL
MERLIN­based for ILC (D. Krucker et al.)
PLACET + BDSIM for CLIC (codes run in parallel, PLACET compute the wakefields
and BDSIM tracks the secondaries and computes energy deposition)

Control Systems and on­line optics analysis
Control system to drive individual hardware & processes
Process examples:
Ramp, injection, extraction, orbit correction
Advanced concepts:
GAN (Global Accelerator Network)
LHC@FNAL remote operation centre for CMS and machine https://lhcatfnal.fnal.gov/
On­line models and flight simulators – virtual accelerator to plug control software
✔ optics server at PSI’s SLS, CORBA­based, for orbit correction
✔ ATF2 flight simulator
✔ LHC on­line model

For complex machines the control system should be model­based
33

LHC On­line model
Provide virtual accelerator for software testing
Virtual accelerator for safety checks during beam steering
On­line optics matching to help with beam steering
On­line optics error fitting
Very detailed aperture and machine imperfection database
Model corrections depending on operation conditions
Client­server architecture, java­based gui and control system interface, python­based
server with MAD­X as the primary computation engine
Ad hoc python scripting
Used for LHC commissioning

Operator console – virtual mode
35

User interface for interactive expert mode ­­
fully exploiting the accelerator model
36

Example: injection and dump commissioning
Full aperture model for injected and circulating (including septum alignment)
OM used for orbit steering to detect aperture bottlenecks (oscillating bumps)
Check that the beam is steering onto a collimator when kicker off
Aperture bottleneck detected based on BLM data, confirmed by radiation
survey and cured by realignment
Magnetic error fitting from orbit and dispersion measurements
37

Aperture measurements (arcs)
Free oscillations with different starting phases generated by OM
Closed bumps for bottlenecks
Looking at BLM readings
38

CMS beam crossing (from CERN logbook 03 Dec 2009)
39

General data management issues
Data rates/storage requirements less than HEP (even multiturn BPM data)
Not talking about statistical data
Data persistence not always important
Smaller communities (lab staff + some external) ­> slow
adoption of standards, frameworks etc.
e­logbooks common
Tools: MATLAB, mathematica, root common
Software development: version control and some other
management procedures in place

Conclusion
Electromagnetic codes such as cst particle studio, gdfidl in place. Lack
of open source frameworks (like OpenFoam form fluid mechanics which
provides meshing, gui, etc.)
Compute­intensive gas, solid and fluid dynamics based on commercial
tools such as ANSYS for targets, dumps etc.
Beam optics codes – standardization/convergence a question
Machine­detector interface codes present (e.g Geant4­based BDSIM)
Collective effects and other non­trivial beam physics codes developing.
Unfortunately no frameworks available
Flight simulators and on­line models emerging for high­level controls
Accelerator physics provides plenty of compute­intensive applications