The Genom of Homo sapiens.pdf

Assessing the Quality of Finished Genomic SequenceJ. SCHMUTZ, J. WHEELER, J. GRIMWOOD, M. DICKSON, AND R.M. MYERSStanford Human Genome Center, Stanford University School of Medicine, Palo Alto, California 94304This April, the Human Genome Project (HGP) announcedthe essential completion of the human genomesequence. In just a few years, from 2001 to 2003, the percentageof finished Homo sapiens sequence jumped from25% to 99%. This represented a dramatic increase in theproduction finishing capacity of genome centers worldwideand a shift from a primary focus on the productionof draft shotgun sequence (a streamlined productionpipeline) to the generation of complete and accurate finishedgenomic sequence (a difficult process involving decision-makingand consecutive rounds of experiments).By 2001, the large genome centers had proven that theycould reduce the cost of the sequencing read through increasedautomation, conservation of reagents, and 24/7production level processes, but could they do the samething for producing finished sequence? Although it is asignificant challenge to maintain a production level ofmillions of shotgun sequencing reads per month, it is arguablymore difficult to maintain a steady output of finishedsequence that meets a defined accuracy standard.Perhaps surprising ourselves, we did it, overcoming thecomplexities of the finishing process and the allelic variationin the human genome to produce an essentiallycomplete human genome sequence.Now that 2.82 billion base pairs of finished human sequencehave been generated, how can we be assured thatthe production of finished genomic sequence merited theenormous investment? Because of the cost and immensityof the project, the finished human reference sequenceis not likely to be reproduced at any time in the near future.Therefore, to assess the general quality of the finishedproduct, we must examine small portions of it andextrapolate to the rest of the completed human genome.During the project, the Stanford Human Genome Centerwas given a mandate by the National Human GenomeResearch Institute (NHGRI) to perform such an examination.In the process, we learned much about the problemof assessing the quality of the product generated by sucha complex scientific process as the HGP. In this paper, wesummarize the results of our quality assessment of thefinished human genome sequence and offer suggestionsas to how to apply these lessons to the problem of evaluatingthe quality of future genome sequencing projects.HISTORICAL MEASUREMENTS OFSEQUENCE ACCURACY IN THE HGPIn 1997, world standards for sequence accuracy wereestablished at a meeting of HGP participants in Bermuda(now known as the “Bermuda Standards”). At this meeting,it was decided that any clone from the humangenome sequence submitted as finished should have lessthan one error per 10,000 bases and that the sequenceshould be contiguous with no gaps (http://www.gene.ucl.ac.uk/hugo/bermuda2.htm). At the time, veryfew centers were submitting clone sequences that had nogaps, and laboratory and data analysis mechanisms forfinishing all clones with no gaps were not in commonpractice among the data producers, and a sufficient protocolfor measuring the sequence accuracy component wasalso unknown. Due to the prohibitively high cost of producingfinished sequence, it was impractical to independentlyresequence and refinish many clones to establish afirm error rate for the finished sequence that was beingproduced.Since this time, two principal methods for estimatingsequence accuracy have been employed by the HGPgenome centers: the use of quality scores from Phred processedby Phrap (Ewing and Green 1998; Ewing et al.1998) and examintion of potential overlapping sequencefrom different clones for errors. To better understand thelimitations, it is helpful to understand how the accuracy isgenerally estimated with these methods.In the first of these methods, Phred assigns error probabilitiesto each base pair in every sequencing trace, basedon large training data sets. Traces base-called by Phredare subsequently assembled by the assembly algorithmPhrap, which propagates these single-base-pair errorscores to the consensus sequence constructed from manyoverlapping sequence reads. Although Phred qualityscores for measuring the accuracy of single sequencingtraces have been extensively validated (Ewing et al. 1998;Richterich 1998) and are used to monitor the quality ofproduction sequencing, the cumulative Phrap score forfinished bases appearing in a consensus sequence has notbeen similarly examined. The Phrap score provides an indicationof the value of the underlying base, but becauseof the complexity of the data assembly process, one cannotsimply add up the Phrap estimated errors for all of thebases in a finished clone to determine the error rate. In ourexperience, the Phrap error rates are tenfold lower thanthe actual errors in the clone sequences. The simple reasonfor this is that Phrap error rates are assigned only tobases appearing in the consensus, and potential problembase pairs tend not to be included in the Phrap-derivedconsensus for the very reason that they are poor-qualitybases. These problem bases include compressed orstretched peaks, and errors created by the Phrap assemblyCold Spring Harbor Symposia on Quantitative Biology, Volume LXVIII. © 2003 Cold Spring Harbor Laboratory Press 0-87969-709-1/04. 31