From Protein Structure to Function with Bioinformatics.pdf

76 A. Fiserfrom efforts of genome sequencing, functional genomics, proteomics systemsbiology and structural biology.3.2.5 Model EvaluationAfter a model is built, it is important to check it for possible errors. The quality ofa model can be approximately predicted from the sequence similarity between thetarget and the template. Sequence identity above 30% is a relatively good predictorof the expected accuracy of a model. If the target-template sequence identity fallsbelow 30%, the sequence identity becomes significantly less reliable as a measureof the expected accuracy of a single model. It is in such cases that model evaluationmethods are most informative.Two types of evaluation can be carried out. “Internal” evaluation of self-consistencychecks whether or not a model satisfies the restraints used to calculate it, includingrestraints that originate from the template structure or obtained from statisticalobservations. “External” evaluation relies on information that was not used in thecalculation of the model.Assessment of the stereochemistry of a model (e.g., bonds, bond angles, dihedralangles, and non-bonded atom-atom distances) with programs such asPROCHECK (Laskowski et al. 1993) and WHATCHECK (Hooft et al. 1996) is anexample of internal evaluation. Although errors in stereochemistry are rare and lessinformative than errors detected by methods for external evaluation, a cluster ofstereochemical errors may indicate that the corresponding region also containsother larger errors (e.g. alignment errors).As a minimum, external evaluations test whether or not a correct templatewas used. Luckily a wrong template can be detected easily with the currentlyavailable scoring functions. A more challenging task for the scoring functionsis the prediction of unreliable regions in the model. One way to approach thisproblem is to calculate a “pseudo energy” profile of a model, such as that producedby PROSA (Sippl 1993) or Verify3D (Eisenberg et al. 1997). The profilereports the energy for each position in the model (Fig. 3.3). Peaks in the profilefrequently correspond to errors in the model. There are several pitfalls in theuse of energy profiles for local error detection. For example, a region can beidentified as unreliable only because it interacts with an incorrectly modelledregion (Fiser et al. 2000). Other recent approaches usually combine a variety ofinputs to assess the models, either as a whole (Eramian et al. 2006) or locally(Fasnacht et al. 2007). In benchmarks the best quality assessor techniques usea simple consensus approach where reliability of a model is assessed by theagreement among alternative models that are sometimes obtained from a varietyof methods (Wallner and Elofsson 2005a, 2007). Model assessment is animportant but difficult area, due to a circular argument: scoring function termsof an effective model assessment approach should be used in the first place toproduce accurate models.