Dynamical HISQ - University of Glasgow

Dynamical HISQ
Craig McNeile
c.mcneile@physics.gla.ac.uk
University of Glasgow
Dynamical HISQ – p. 1

Overview of dynamical HISQ
I report on the various efforts to generate gauge
configurations with HISQ sea quarks.
Unfortunately running dynamical simulations
involves code optimization and efficiency.
Sorry!
No results. Sorry! Sorry!
I will try be positive and pretend that the UK will
get computer time for this or (anything). (If time,
end with Iwasaki action).
Dynamical HISQ – p. 2

Why do dynamical HISQ?
From science case for computer time in UK.
Improved scaling from more improved HISQ action and
updated radiative corrections to gauge action.
Run simulations with non-degenerate u,d,s and c
quarks.
Include electromagnetism effects. (Check MILC’s
result for m u /m d from 2+1 calculations).
Study Random Matrix Theory (RMT) with HISQ given
that the unquenched Asqtad RMT project was
unsuccessful.
Dynamical HISQ – p. 3

Who is doing what?
Kit and Eduardo’s work discussed later. Who is
writing code.
Person where project
Doug Toussaint Arizona HISQ into MILC code
Alexei Bazavov Arizona HISQ into MILC code
Carleton DeTar Utah level 3, HISQ
Tommy Burch Utah level 3, HISQ
Alan Gray EPCC, UK optimization level 3
Dynamical HISQ – p. 4

HISQ optimization
Ron is a PI for a software position in EPCC The grant was
for 1/2 a person (Alan Gray) for 18 months. Alan started at
20 % level in October.
Alan has updated Kit’s code so that it can write configs
in NERSC format. Also more Fortran90.
Alan is studying implementation of Asqtad/HISQ
forces.
Plan for Alan to optimize the level 3 code for
Cambridge machine, and maybe Bluegene/P.
The grant proposal also discussed running and testing.
Dynamical HISQ – p. 5

HISQ and the MILC code
Plaquette test of MILC code versus Kit’s code.
Tested Alexei’s MILC inverter against
Eduardo’s on 4 3 8 lattice.
Doug Toussaint used HISQ inverter on 1/8 fm
configs by computing the pion spectrum on
an ensemble (m l /m s 0.010/0.050). Good
agreement with HPQCD results.
MILC will write new optimized versions of the
HISQ routines, as more runs are done.
Dynamical HISQ – p. 6

RHMC
detM α =
∫
Dφ † Dφe −φ† r 2 (M)φ
where r(x) = x −α/2 (rational approximation).
RHMC is an exact algorithm, also allows larger δt than
R algorithm ( δt ∼ 2/3m l ) so is faster (Kennedy/Clark
claim 7 times better).
MILC are now using the exact RHMC/RHMD algorithm
for generating configurations (talk at ILDG11).
Dynamical HISQ – p. 7

RHMC improvements
There are a tool kits of faster algorithms
Higher order integrators (Omelyan ). In MILC code.
n-th root trick. detM =| detM 1/n | n =
∏ nj=1
∫
Dφ † Dφe −φ† j M −1/n φ j
Mass preconditioning
Chris Richards (Liverpool) used the above algorithms
(implemented by Clark) to get a 30% to 50% speed up with
Asqtad, but needed ∼ 4 months of tuning (Lots of
parameters and tradeoffs).
Dynamical HISQ – p. 8

MILC plans
Email from Doug Toussaint. Not started production
running, but deciding this month what they want to do with
HISQ (SCIDAC deadline).
Alexei and Doug seeing some issues.
Large jumps in the force when a first-level
smeared matrix is small or when the
phase of a unitarized matrix jumps.
Note/email to follow.
Mass dependent Naik term means additional
pseudo-fermion fields are needed (Doug).
Dynamical HISQ – p. 9

What is level3?
The famous software diagram from USQCD.
The level3 library contains the inverter and force
terms optimized for specific platforms. QOPQDP
is the level3 library largely written by DeTar and
Osborn.
Dynamical HISQ – p. 10

Level3 in practise
Level3 code inside chroma Asqtad inverter.
convert_chroma_to_qdp(in,q_source) ;
convert_chroma_to_qdp(out,psi) ;
qopout = QOP_create_V_from_qdp(out);
qopin = QOP_create_V_from_qdp(in);
QOP_asqtad_invert(&info, fla, &inv_arg, &
mass, qopout,qopin);
QOP_extract_V_to_qdp(out,qopout) ;
convert_qdp_to_chroma(psi,out) ;
Dynamical HISQ – p. 11

Is level 3 useful?
Machine Library iters Time s
QCDOC Pure chroma 736 46.9
QCDOC single prec. level 3 2932 6.0
QCDOC double prec. level 3 742 2.6
Opteron Pure chroma 1851 8893
Opteron double prec. level 3 1862 4008
Table 1: Performance of Asqtad
Dynamical HISQ – p. 12

Profiling (Woloshyn and Wong)
100
(a)
cost of different components per trajectory
80
percentage
60
40
20
0
8 4 12 4 16 4 8 4 12 4 16 4
ASQTAD
HISQ
matrix
inversion f (0) f (1) f (2)
f (3) gauge comp. of
force
eff. links
others
On an Opteron Kit’s code generated a trajectory for a 16 4
lattice in 4 hours. Target of 3000 trajectories will take 1.4
years. Dynamical HISQ – p. 13

What did HPQCD ask for ??
Postdoc and running the MILC code on
Time on Cambridge cluster
Time on Bluegene/P at Darsebury lab in Cheshire.
One rack has been delivered. ETMC twisted mass
HMC code 15% efficiency (after a day’s work by
Carsten Urbach).
3 year project costing £1,000,000.
Amazon.com offer cloud computing. Two cores for 1 hour
with 7 Gbytes of memory costs $ 0.4 (20p). (The
Cambridge machine costs 6p per core hour). Any spare
book-tokens could come in useful.....
Dynamical HISQ – p. 14

Introduction to Iwasaki action
Perhaps 50% of lattice QCD computer cycles (JLQCD,
RBC, CP-PACS) use the Iwasaki (square plus
rectangle) action.
A classic book is one that everyone knows about, but
no one has read.
How to do α s n f a 2 corrections (rather than to just annoy
people).
Iwasaki’s paper (1983) “Renormalization Group Analysis of
Lattice theories and improved lattice action II” not
published (available in PDF on spires) The derivation is
perturbative Dynamical HISQ – p. 15

The basic idea
Based on RG transformation work by Wilson
e −βS′ (Φ) =
∫
dφe −β(T(Φ,φ)+S(φ))
Fixed point solution has no a m (for any m) or one loop
corrections (section 10.4 of DeGrand-DeTar book).
Iwasaki’s block spin transformation on gauge potential.
A (n)
µ (x ′ ) = 1 8
∑
µ (x)
x∈x ′ A (n−1)
Dynamical HISQ – p. 16

The calculation
Iwasaki looked at effect of blocking (n-th level) on one loop
expression for Wilson loops
F (n) (IxJ) =
W (n) (C) = 1 − g 2N2 − 1
N
F (n) (C)
∫
12 12(k)( sin1/2Ik(n) 1
sin 1/2k (n)
D (n)
1
sin1/2Jk (n)
2
sin 1/2k (n)
2
where D 12,12 related to gluon propagator and H(k) is
blocking factor. Also obtained result for ∞ blocking
) 2 H (n) (k)
Dynamical HISQ – p. 17

Blocking on 1-loop plaquette F(1x1)
Block Wilson T. Symanzik Iwasaki
0 0.5 0.36626 0.21027
1 0.28810 0.25076 0.18826
2 0.21623 0.20599 0.18431
3 0.19446 0.19173
4 0.18865 0.18794
∞ 0.18649 0.18649 0.18649
Iwasaki gauge action wins! (Tadpole Symanzik??)
Dynamical HISQ – p. 18