Automated structure refinement with phenix.refine

Automated structure refinement with phenix.refine
Pavel Afonine
Computation Crystallography Initiative
Physical Biosciences Division
Lawrence Berkeley National Laboratory, Berkeley CA, USA
PHENIX workshop
Houston, April 30, 2010

Structure refinement
Crystallographic structure determination workflow
Purified
object
Crystals
Experimental
Data
Initial
approximate
model
Model
Re-building
Refinement
Validation
And
analysis
Deposition
(publishing)
Structure refinement: modify model parameters to describe the
experimental data as good as possible
Initial model
Calculate
model structure
factors
Correct for bulk
solvent and
other scaling
Modify
model
parameters
Improved model

Structure refinement
1. Model parameters
2. Optimization goal
3. Optimization method

Model parameters
Crystal (unit cell)
Bulk solvent
- Macromolecular crystals contain ~20-80% of solvent (mostly disordered)
Atomic model
- Position (coordinates)
- Mobility (ADP; Atomic Displacement Parameters)
- Disorder (occupancies)

Model parameters
Atomic model parameterization depends on:
- quality of experimental data (resolution, completeness, …)
- quality of current model (initial with large errors, almost final, …)
- data-to-parameters ratio (restraints have to be accounted for)

Individual
atoms
Model parameterization: coordinates
Constrained rigid bodies (torsion
angle parameterization)
Rigid body
3 * Natoms
3 * Natoms / (7 …10)
6 * Ngroups
High Resolution Low
Some a priori information may be needed such as:
- Stereochemistry restraints
- NCS restraints or constraints

Model parameterization: Atomic Displacement Parameters (“B-factors”)
Nesting of atomic displacements
atom
residue
domain
molecule
crystal
Currently one can split and distinguish only a limited number of movements
Δr total = Δr crystal +Δr domain + Δr atom
Total movement (U TOTAL )
Collective movement (U TLS )
Local movement of
independent atoms (U ATOM )
=
+

Structure refinement
1. Model parameters
2. Optimization goal
3. Optimization method

Target function
Least-Squares target
E DATA
=
∑
s
Maximum-Likelihood target
⎛
E DATA € = ∑ (1− K cs s
) ⎜ − α s
⎜
s
⎝
Real space target
2 F s
MODEL
w ( s
F OBS MODEL
s
− kF ) 2
s
( ) 2
⎛ ⎛ 2α
+ ln I s
F MODEL OBS
s
F s
⎞ ⎞ ⎞
⎜ 0 ⎜
⎟ ⎟ ⎟
ε s
β s ⎝ ⎝ ε s
β s ⎠ ⎠ ⎟ +
⎠
⎛
+K cs s
− α 2 MODEL
s ( F s ) 2
⎛ ⎛
⎜
+ ln cosh α F s s
⎜ ⎜
⎜ 2ε s
β s ⎝ ⎝ ε s
β
⎝
s
MODEL F s
OBS
⎞ ⎞ ⎞
⎟ ⎟ ⎟
⎠ ⎠ ⎟
⎠
E DATA
=
∑
grid points
€
( )
ρ best
− kρ calc
2
+ wTRESTRAINTS

Structure refinement
1. Model parameters
2. Optimization goal
3. Optimization method

Refinement target optimization algorithms
Gradient-driven minimization
- Follows the local gradient.
- The target function depends on
many parameters – many local
minima.
Simulated annealing (SA)
- Good at escaping local minima.
- Increased probability of finding a
better solution because motion against
the gradient is allowed.
Local
minimum
Global minimum
Deeper local
minimum
Global minimum
Grid search
- Robust but time inefficient. Good for small number of parameters (2-3 or so),
and impractical for larger number of parameters.

Crystallographic structure refinement in PHENIX

phenix.refine
Highly-automated state-of-the-art structure refinement tool of PHENIX
Active development mainly at Lawrence Berkeley National Laboratory:
- Paul Adams
- Pavel Afonine
- Nathaniel Echols
- Ralf Grosse-Kunstleve
- Jeff Headd
- Nigel Moriarty
- Peter Zwart
+ valuable scientific support by many others (Marat Mustyakimov, Sasha
Urzhumtsev, Vladimir Lunin, …)

Automation of structure refinement
What used to be in the past … and often still the case nowadays
Acta Cryst. (2002). D58, 2009-2017, Yousef et al.
Clearly, the modern software should do all these steps automatically
PHENIX is making a good progress in achieving this goal

phenix.refine: single program for a very broad range of resolutions
Low Medium and High Subatomic
- Group ADP refinement
- Rigid body refinement
- Torsion Angle dynamics
- Restrained refinement (xyz,
ADP: isotropic, anisotropic,
mixed)
- Automatic water picking
- Bond density model
- Unrestrained refinement
- FFT or direct
- Explicit hydrogens
- Automatic NCS restraints
- Simulated Annealing
- Automatic side chain rotamer fixing
- Occupancies (individual, group, automatic
constrains for alternative conformations)
- Various targets: LS, ML, MLHL,…
- TLS refinement
- Use hydrogens at any resolution
- Refinement with twinned data
- X-ray, Neutron, joint X-ray + Neutron
- Built-in water picking and refinement

Refine any part of a model with any strategy: all in one run
+ Automatic water picking
+ Simulated Annealing
+ Add and use hydrogens

Designed to be very easy to use
Several ways of running:
- command line version:
Running phenix.refine
phenix.refine model.pdb data.hkl [parameters]
o Can be highly customized (more than 300 parameters available to change)
- can be called from a Python script allowing to run it within different
contexts
- GUI version

phenix.refine GUI

Refinement flowchart
PDB model,
Any data format
(CNS, Shelx, MTZ, …)
Input data and model processing
Refinement strategy selection
Bulk-solvent, Anisotropic scaling, Twinning
parameters refinement
Ordered solvent (add / remove)
Target weights calculation
Coordinate refinement (real- and reciprocal space)
(rigid body, individual) (minimization or Simulated
Annealing)
ADP refinement
(TLS, group, individual iso / aniso)
Occupancy refinement (individual, group)
Output: Refined model, various maps, structure
factors, complete statistics, ready for deposition PDB
file
Repeated
several times
Files for
COOT, O,
PyMol

Effect of anisotropic scaling (U CRYSTAL )
Total model structure factor used in refinement and map calculation:
Anisotropic scaling (PDB: 2mhr)
R-factor
Effect of Bulk Solvent
R-factor
No correction
Inoptimal ksol, Bsol
PHENIX
Resolution, Å
Resolution, Å
A robust bulk-solvent correction and anisotropic scaling procedure. Acta Cryst. (2005).
P.V. Afonine, R.W. Grosse-Kunstleve & P.D. Adams

Automatic Water Picking
Input data and model processing
Refinement strategy selection
Bulk-solvent, Anisotropic scaling, Twinning
parameters refinement
Ordered solvent (water picking)
Target weights calculation
Coordinate refinement
(rigid body, individual) (minimization or SA)
ADP refinement
(TLS, group, individual iso / aniso)
Occupancy refinement (individual, group)
Output: Refined model, various maps,
structure factors, complete statistics, ready for
deposition PDB file
Water picking done within refinement
- remove “bad” water:
• 2mFo-DFc (peak hight)
• Distances
• map CC (2mFo-DFc, Fc)
• B-factors and anisotropy
• occupancy
- add new:
• mFo-DFc,
• distances
- refine water’s ADP before adding to model
Water update (picking / refinement / removing) has to be a highly integrated
part of refinement process.

Rigid body refinement
Low Resolution High
?
Challenges:
Solution
Target function profile
- Use low resolution to assure better convergence to a solution
- Using too low resolution may not be good too (map too featureless)
- Using higher resolution data assures more precise solution but the target
function profile becomes too complex so the model can get caught in a
local minimum
- How to define low-high resolution border (3…4…6A)?
PHENIX MZ protocol makes all these decisions automatically!
Automatic multiple-zone rigid-body refinement with a large convergence radius.
P. V. Afonine, R. W. Grosse-Kunstleve, A. Urzhumtsev and P. D. Adams
J. Appl. Cryst. 42, 607-615 (2009)

Automated Rigid Body Refinement: PHENIX MZ protocol
Lowest Low High Highest
Resolution
Automatically define
lowest usable resolution
zone and do RBR there.
This insures quick and
reliable convergence.
Gradually add higher resolution reflections. This
supports the convergence and assures higher
precision of the solution.
Large model movements are expected during rigid body refinement which invalidate
the solvent mask, therefore it is updated at each step.
All parameters used in the protocol are optimized to achieve the highest
convergence radius at minimal runtime.
- This is done by the grid search over ~100000 trial refinements using more than
100 different structures.

Atomic Displacement Parameters (ADP or “B-factors”)
Total atomic ADP U TOTAL = U CRYSTAL + U TLS + U ATOM
U CRYSTAL - overall anisotropic scale w.r.t. cell axes (6 parameters).
U TLS - rigid body displacements of molecules, domains, secondary
structure elements. U TLS = T + ALA t + AS + S t A t (20 TLS parameters
per group).
U ATOM - vibration of individual atoms. Should obey Hirshfeld’s rigid bond
postulate.
Group
Individual or
group isotropic
Individual
isotropic
Individual isoor
anisotropic
Individual
anisotropic
Low 3.5Å 3.0Å 2.0Å 1.5Å High

Restraints in refinement of individual ADP (Atomic Displacement Parameters)
Refinement of isotropic ADP
Refinement of anisotropic ADP
Restraints
Restraints
A bond is almost rigid, therefore the ADPs of bonded atoms are similar (Hirshfeld, 1976);
ADPs of spatially close (non-bonded) atoms are similar (Schneider, 1996);
The bond rigidity, and therefore the difference between the ADPs of bonded atoms, is related to the absolute
values of ADPs. Atoms with higher ADPs can have larger differences (Ian Tickle, CCP4 BB, March 14, 2003).
E RESTRAINTS
=
N atoms
∑
i=1
⎡
⎢
⎢
⎢
⎢
⎣
M atoms
∑
j=1
1
r ij
distance_power
( U i
− U j ) 2
⎛ U i
+ U j
⎞
⎜ ⎟
⎝ 2 ⎠
average_power
sphereR
⎤
⎥
⎥
⎥
⎥
⎦

TLS refinement in PHENIX: robust and efficient
Highly optimized algorithm based on systematic re-refinement of ~350 PDB
models
In most of cases phenix.refine produces better R-factors compared to
published
Don’t crash or get “unstable”
Rwork (PHENIX)
Rfree (PHENIX)
Rwork (PDB)
Rfree (PDB)

ADP refinement : from group B and TLS to individual anisotropic
Synaptotagmin refinement at 3.2 Å
CNS
R-free = 34.%
R = 29.%
PHENIX – Isotropic restrained ADP
R-free = 27.7%
R = 24.6%
PHENIX – TLS + Isotropic ADP
R-free = 24.4%
R = 20.7%

ADP refinement: what goes to PDB
phenix.refine outputs TOTAL B-factor (iso- and anisotropic):
U TOTAL = U ATOM + U TLS + U CRYST
Isotropic equivalent
ATOM 1 CA ALA 1 37.211 30.126 28.127 1.00 26.82 C
ANISOU 1 CA ALA 1 3397 3397 3397 2634 2634 2634 C
U TOTAL = U ATOM + U TLS + U CRYST
Stored in separate
record in PDB file
header
Atom records are self-consistent:
Straightforward visualization (color by B-factors, or anisotropic ellipsoids)
Straightforward computation of other statistics (R-factors, etc.) – no need
to use external helper programs for any conversions.

Dual-space refinement: combining real and reciprocal space refinement
Why real-space refinement ?
• Can be done locally (for example, for a residue or ligand)
• Grid search can be used -> Convergence radius can be dramatically
increased compared to gradient driven-refinement or SA
• Ordered solvent update can be enabled at earlier stage
Eliminate the tedium of manual work on fixing side chains on graphics

Local real-space refinement
Compute 2mF obs -DF model , mF obs -DF model , F model
maps
for residue in residues:
compute start T- and CC-values for residue
if need_a_fix:
for rotamer in rotamers:
torsion grid search
if is_better:
residue = rotamer
real-space refine residue: residue refined
if is_better:
residue = residue refined
update structure with residue
N macro-cycles
phenix.refine protocol
Update F model and re-compute 2mF obs -DF model map
Real-space refine whole model into 2mF obs -DF model
Validate changes:
compute 2mF obs -DF model , mF obs -DF model and F model
for residue in residues:
if is_better:
restore original residue (discard change)

Real-space refinement: torsion grid search

Option for automatic side chain flips to avoid clashes
Apply side chain flips if necessary (Asn/Gln/His) (RLab, Jeff Headd).
Picture: http://molprobity.biochem.duke.edu/
Bad
Always use hydrogens in real-space optimization
Use to build alternative conformations
Improve NCS restraints
Good

Occupancy refinement
Automatic constraints for
occupancies of atoms in
alternate locations
Any user defined selections
for individual and/or group
occupancy refinement can be
added on top of the automatic
selection.
ATOM 1 N AARG A 192 -5.782 17.932 11.414 0.72 8.38 N
ATOM 2 CA AARG A 192 -6.979 17.425 10.929 0.72 10.12 C
ATOM 3 C AARG A 192 -6.762 16.088 10.271 0.72 7.90 C
ATOM 7 N BARG A 192 -11.719 17.007 9.061 0.28 9.89 N
ATOM 8 CA BARG A 192 -10.495 17.679 9.569 0.28 11.66 C
ATOM 9 C BARG A 192 -9.259 17.590 8.718 0.28 12.76 C
ATOM 549 AU A 34 -23.064 7.146 -23.942 0.78 15.44 Au
ATOM 549 HA3 ARG A 34 -23.064 7.146 -23.942 1.00 15.44 H
ATOM 550 H AARG A 34 -24.447 7.644 -21.715 0.15 8.34 H
ATOM 551 D BARG A 34 -24.447 7.644 -21.715 0.85 7.65 D
ATOM 552 N ARG A 35 -22.459 9.801 -22.791 1.00 8.54 N
ATOM 6 S SO4 1 1.302 1.419 1.560 0.70 13.00
ATOM 7 O1 SO4 1 1.497 1.295 0.118 0.70 11.00
ATOM 8 O2 SO4 1 1.098 0.095 2.140 0.70 10.00
ATOM 9 O3 SO4 1 2.481 2.037 2.159 0.70 14.00
ATOM 10 O4 SO4 1 0.131 2.251 1.823 0.70 12.00

Two steps to perform twin refinement:
Refinement with twinned data
- run phenix.xtriage to get twin operator (twin law):
% phenix.xtriage data.mtz
- run phenix.refine:
% phenix.refine model.pdb data.mtz twin_law="-h-k,k,-l"
Taking twinning into account makes difference:
Interleukin mutant (PDB code: 1l2h)
R/R-free (%)
PHENIX (no twinning): 24.9 / 27.4
PHENIX (twin refinement): 15.3 / 19.2

Hydrogen atoms in refinement
phenix.refine offers various options for handling H atoms at any resolution:
- Riding model (low-high resolution)
- Individual atoms (ultrahigh resolution or neutron data)
- Account for scattering contribution or just use to improve the geometry
Expected benefits from using the H atoms in refinement:
- Improve R-factors
- Improve model geometry (remove bad clashes)
- Model residual density at high resolution or in neutron maps
Example: automatic re-refinement of 1000 PDB models with and without H:
pdb resolution Rfree(no H) – Rfree(with H)
1akg 1.1 1.9
1byp 1.75 1.41
1dkp 2.3 0.93
1rgv 2.9 0.50

Refinement using X-ray and Neutron diffraction data
Different techniques – different information
2mFo-DFc maps
X-ray (1.8 Å) Neutron (2.2 Å)
Fo-Fc, (H-, D-omit neutron
map), 1.6 Å resolution
+2.6σ, D atoms
-2.9σ, H atoms
Neutron maps show hydrogen atoms

Refinement at subatomic resolution
~340 structures in PDB at resolution higher than 1.0 Å
Aldose Reductase (0.66 Å resolution)
Fo-Fc (orange)
2Fo-Fc (blue)
phenix.refine has unique set of tools to correctly refine such structures

Modeling at subatomic resolution: IAS model
Basics of IAS model:
Afonine et al, Acta Cryst. D60 (2004)
First practical examples of implementation and use in PHENIX:
Afonine et al, Acta Cryst. D63, 1194-1197 (2007)
IAS modeling in PHENIX
Simple Gaussian is good enough:
j a
b
a and b are pre-computed library for most bond types

IAS modeling: benefits
Improve maps: reduce noise. Before (left) and after (right) adding of IAS.
Find new features: originally wrong water (left) replaced with SO4 ion (right)
clearly suggested by improved map after adding IAS

Analyzing refinement results…

Question: “I got Rwork=18% and Rfree=23% after refinement, is it a good
result?”
- A very common question
- Answer depends on various factors
Answer:
Quick model quality evaluation
- Yes, it’s likely a good result if the data resolution is around 2.5 Å.
- No, it is very bad result, if the data resolution is 1.0 Å or higher.
One can ask similar questions about other parameters, such as bond/angles
RMSDs, average B-factors, etc…

POLYGON: Crystallographic Model Quality at a Glance
Say you are refining a structure at 1.0 Å resolution and the R-factors are:
R work = 18% and R free is 23%.
- Are these values good? Is refinement completed?
PDB statistics: histograms for R work , R free , R free -R work for all similar structures:
Rwork at 0.9-1.1Å
0.10 - 0.12: 68
0.12 - 0.14: 94
0.14 - 0.16: 73
0.16 - 0.18: 17

New tool: phenix.polygon
Likely good model
This model needs some attention
Acta Cryst. (2009). D65, 297-300

R-factors (all models in PDB at resolution < 1 Å )
R-factor, %
Resolution, Å
PDB code
(year)
published
R-work, %
phenix.refine
2ppn (2007) 20.9 11.7
1g2y (2000) 19.5 12.3
1zlb (2005) 16.8 12.0
2g6f (2006) 18.4 12.9
2elg (2007) 23.2 13.0
1aho (1997) 16.3 9.6
1zf5 (2005) 29.0 16.9
There are ~25 models out of 324 that have suspiciously high or very high R-
factors.
- For most of them the R-factors can be decreased to typical for this
resolution values (~10-15%) in one phenix.refine run.
Automated software with integrated validation would immediately flag these
models as suspicious.

PDB: 1eic
Resolution: 1.4Å
Deposition year:
2000
PUBLISHED:
Rwork = 20%
Rfree = 25%

Under-refined models or why automation is important
Structure from PDB: 1eic (resolution = 1.4Å; deposition year: 2000)
PUBLISHED: Rwork = 20% Rfree = 25%
Clear problems:
- No ‘riding’ H atoms;
- All atoms are isotropic;
Potential problems
- Inoptimal weights, refinement is not converged, incomplete solvent model
Fixing the model with PHENIX:
- Add and refine H as riding model
- Update ordered solvent
- Refine atoms as anisotropic (except H and water)
- Optimize X-ray/Restraints weights
FINAL MODEL: Rwork = 14% Rfree = 17%
All this could be done by the software automatically, preventing deposition of underrefined
models into PDB

Final notes…

www.phenix-online.org
This
presentation
(PDF file)

Reporting bugs, problems, asking questions
Something didn’t work as expected?... program crashed?... missing
feature?...
Not Good: silently give up and run away looking for alternative software.
Good: report us a problem, ask a question, request a feature (explain why
it’s good to have), ask for help.
Reporting a bug:
Not good: “Hi! phenix.refine crashed, I don’t know what to do.”
Good: “Hi! phenix.refine crashed. Here are:
1) PHENIX version;
2) Command and parameters I used;
3) Input and output files (at least logs).”
Subscribe to PHENIX bulletin board: www.phenix-online.org

PHENIX use (April 28, 2010)
Number of structures in PDB with “REMARK 3 PROGRAM PHENIX”
948
435
1 7
96
163
2005 2006 2007 2008
Year
2009 2010
User tested !

Acknowledgments
All PHENIX developers
All PHENIX users
Non-PHENIX colleagues for scientific support, fruitful collaboration and
valuable feedback:
Sacha Urzhumtsev, Vladimir Lunin, Dale Tronrud, Dusan Turk
• Funding:
- NIH/NIGMS:
P01GM063210, P50GM062412, P01GM064692, R01GM071939
- Lawrence Berkeley Laboratory
- PHENIX Industrial Consortium