An Integrated Data Analysis Suite and Programming ... - TOBIAS-lib

64 CHAPTER 3. A C++ FRAMEWORK FOR HIGH-THROUGHPUT DNA SEQUENCING
Supported procedures include false discovery rate (FDR) calculation following Benjamini and
Hochberg [143] as well as Benjamini and Yekutieli [147] and further family wise error rate
(FWER) control via Bonferroni [148], Hochberg [149], Holm [150] as well as Sidak [151] correction
algorithms.
3.2 A Modular Signal-Slot Processing Framework
3.2.1 Reader and Writer Concepts
The requirement to support large numbers of dierent input le formats emphasizes the advantage
of consistent interfaces to provide access to the data. The libshore framework therefore
denes the concept of a Reader as a class providing three methods named has_data, current
and next which allow linear iteration over the data set's elements.
The has_data method returns a boolean value indicating whether further data set elements
are available to the respective Reader object. The method current provides access to the element
of the data set at the current position of the iteration, and next indicates the current element
has been processed and may be discarded. Initialization of the following element for retrieval by
current may be handled by either of the next or has_data methods.
Conversely, we dene the concept of a Writer as a class supporting methods append and
flush. The method append is used to pass the object a data set element for processing, whereas
flush serves as a notication that no more data are available for processing, triggering nalizing
actions such as addition of footer sequences for compressed data sets.
Intermediary processing and ltering modules can thus be realized as classes conforming to
both the Writer and Reader concepts. Method flush in this case adopts the role of nishing data
processing and propagating all remaining data to the object's Reader interface. For example, in a
sliding window analysis output of data will be delayed until enough input data become available
to cover the entire width of the window. On receiving a ush request, processing of remaining
elements must however be completed without waiting for further data associated with the sliding
window.
3.2.2 Denition of Processing Network Topology
With the Reader and Writer concepts, propagation of data through a network of processing
modules might be accomplished through a series of nested processing loops (listing 3.1). The
example illustrates propagation of data read from a single source of input through two cascaded
ltering operations. Processing results are to be written to the application's standard output
stream, whereas intermediate results following the rst step of ltering are additionally recorded
as a separate output le. The rst set of loops of lines 827 constitute the actual propagation of
input data. Read mapping data provided by the reader object are appended to the rst ltering
module (line 10). As a result, the lter object may or may not produce an arbitrary amount of
data (e. g. due to sliding window based removal of PCR duplicated sequences), which are passed
to the intermediate le writer as well as the second lter object (lines 1224). Data produced by
the second lter object must be transmitted to the nal output accordingly through a further
nested loop (lines 1721). Additional sets of nested loops correspond to propagation of remaining
data following appropriately stacked ush requests to each of the modules (lines 3056).
The demonstrated mode of explicit data propagation is verbose and error prone. Our programming
framework therefore implements a set of class templates providing a signal-slot pipeline
architecture to allow more concise and comprehensible representations. A signal constitutes a
function object that may be connected to an arbitrary number of function objects supporting