An Integrated Data Analysis Suite and Programming ... - TOBIAS-lib

3.1. OVERVIEW 63
tation and command line parsing through an auxiliary class av_parser and incorporates an
object of class uncaught_handler to provide handling and reporting of fatal error conditions
encountered by the application.
Classes istreams and ostreams of the stream package simplify handling of les or other input
or output destinations. Interpretation of special location strings or prexes facilitates specication
of special les or other sources or destinations of input and output as option arguments or
option default values, like e. g. standard input, output or Unix pipes. Input or output locations
compressed in GZIP, BAM or XZ format are delegated to appropriate helper stream classes for
automatic decoding, encoding or handling of random access requests (section 2.2.2). Both classes
in addition provide input/output error checking and further stream-related functionality. Class
ostreams furthermore implements simplied handling of temporary le destinations as well as
optional temporary le compression for improved handling of large amounts of temporary data.
Package algo comprises of dynamic programming alignment (section 2.5.2) as well as indexing,
sorting and query algorithms and associated persistent data structure implementations
(section 2.2.3). The platform for range indexing and queries is provided by a generic 2-d tree algorithm
template d2tree, which is further utilized by the class twodex that supplies a persistent
index data structure congurable for a variety of genomic data formats. Multiple algorithm templates
with the prex suffix_ implement generic sux array construction and queries [144, 145].
The sux array algorithms are employed by objects of class suffix_index, which provides persistent
disk indexes for genomic sequences congurable and capable of answering various types
of sequence match queries. To support up to 64 bit data at reasonable space eciency, persistent
suffix_index and twodex index data are encoded and decoded non-byte aligned with respective
exact required bit width utilizing class intpack of the container package.
The libshore data processing infrastructure is dened and implemented as a set of templates
forming the processing package (section 3.2). Package parallel extends on this infrastructure
to build an abstracted parallel processing interface (section 3.3).
Despite growing acceptance of data exchange specications such as SAM/BAM for short read
mapping data, data format issues can present a major impediment to application of algorithms
to input data from diverse sources. Therefore, the library includes support for a broad variety
of sequencing read, mapping and further sequencing and DNA-related input and output le formats
provided through the fmtio package. Format support is implemented as Reader and Writer
classes with a consistent interface modeled after the requirements of the data processing infrastructure.
To simplify working with various formats, for sequencing read and alignment data the
library provides multi-format readers read_reader and alignment_reader which automatically
incorporate the adequate Reader objects for each respective input le.
File format parsing and decoding requires adequate in-memory representation for the respective
elements of the various types of data set, which are dened in the datatype package. Plain
data structures read and alignment provide the standard representation of short sequencing
read data, whereas large genomic sequences are stored as sequence_record objects with more
sophisticated internal memory management. Additionally, processing modules and utilities specic
to a certain type of data set element or certain properties of data set elements are also
grouped under this package. For example, a class alignment_filter_pipe combines application
of a variety of frequently required ltering and editing modules to read mapping data. Parsing
and decoding the SHORE alignment string pair-wise alignment representation (section 2.2.4) into
CIGAR string-equivalent tokens is provided as a class alignment_tokenizer. Class nucleotide
provides decoding, encoding and ecient manipulation of IUPAC encoded DNA bases.
The additional statistics package supplies the basis for statistical hypothesis tests based on
a variety of distributions. The complementary class mtc re-implements various multiple hypothesis
test correction procedures in C++ following the R multtest package implementation [146].