An Integrated Data Analysis Suite and Programming ... - TOBIAS-lib

Contents
1 Introduction 1
1.1 High-Throughput DNA Sequencing . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Approaches to Parallel DNA Sequencing . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Applications of High-Throughput Sequencing . . . . . . . . . . . . . . . . . . . . 3
1.3.1 Genome and Transcriptome Sequencing . . . . . . . . . . . . . . . . . . . 4
1.3.2 ChIP-Seq and further Immunoprecipitation Protocols . . . . . . . . . . . 5
1.4 Properties of Sequencing Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 Sequencing Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5.1 Base Calling and Read Quality Assessment . . . . . . . . . . . . . . . . . 8
1.5.2 Assembly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5.3 Short Read Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5.4 Variation and Genotype Calling . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5.5 Quantication by Deep Sequencing . . . . . . . . . . . . . . . . . . . . . . 11
1.5.6 ChIP-Seq Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.6 Storage and Representation of Sequencing Data . . . . . . . . . . . . . . . . . . . 13
1.6.1 Storage Formats for Next-Generation Sequencing Data . . . . . . . . . . . 13
1.6.2 Accelerated Queries on Sequencing Data . . . . . . . . . . . . . . . . . . . 15
1.7 Contributions of this Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2 A High-Throughput DNA Sequencing Data Analysis Suite 19
2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2 Ecient Storage of High-Throughput Sequencing Data Using Text-Based File Formats
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2.2 Widely Compatible Indexed Block-Wise Compression . . . . . . . . . . . 23
2.2.3 Ecient Queries on Text Files . . . . . . . . . . . . . . . . . . . . . . . . 25
2.2.4 Improved Compression of Read Mapping Data . . . . . . . . . . . . . . . 27
2.2.5 Compression Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.2.6 Data Storage Considerations . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.3 A Non-Destructive Read Filtering and Partitioning Framework . . . . . . . . . . 33
2.4 A Flexible Sequencing Read Demultiplexing System . . . . . . . . . . . . . . . . 34
2.4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.4.2 A Format for Description of Multiplexing Setups . . . . . . . . . . . . . . 36
2.4.3 Barcode Recognition and Sample Resolution . . . . . . . . . . . . . . . . 38
2.5 Versatile Oligomer Detection and Read Clipping . . . . . . . . . . . . . . . . . . 38
2.5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.5.2 Dynamic Programming Alignment and Backtracing Pipeline . . . . . . . 41
xi