An Integrated Data Analysis Suite and Programming ... - TOBIAS-lib

30 CHAPTER 2. A HIGH-THROUGHPUT DNA SEQUENCING DATA ANALYSIS SUITE
Attribute Plain Size Compressed Size Compression Ratio
read alignments (ref. comp.) 349 Mb (33 Mb) 19 Mb (3.4 Mb) 5.7% (10.3%)
read IDs (relabeled) 227 Mb (117 Mb) 39 Mb (3.2 Mb) 17.5% (2.7%)
qualities (reduced) 341 Mb (341 Mb) 76 Mb (30 Mb) 22.3% (8.8%)
other elds 163 Mb 5.4 Mb 3.3%
Table 2.1: Compressibility of Individual Read Mapping Attributes
qualities accounted for 349 Mb and 341 Mb, each corresponding to approximately 32% of the
total storage size, followed by standard read identiers with 227 Mb (21%) and the collective
size of all further attributes (163 Mb, 15%).
Column data were compressed in XZ format at 128 kB block size. Read alignments were
thereby reduced to 5.7% (19 Mb) of their original size, further elds to 3.3% (5.4 Mb), whereas
read identiers and quality strings could not be compressed as eectively with compression ratios
of 17.5% (39 Mb) and 22.3% (76 Mb), respectively.
By application of reference-based compression, storage size of read alignments was reduced to
less than ten percent (33 Mb) in plain text format. This gain was however somewhat diminished
in compressed format due to the worse compressibility of the data, resulting in a compressed size
of 3.4 Mb at a compression ratio of 10.3%.
Relabeling of the reads resulted in a storage gain of roughly fty percent relative to the
original representation owing to the shorter read identiers. Vastly improved however was the
compressibility of the identier sequence, resulting in a reduction to just 2.7% (3.2 Mb) relative
to plain text format.
Eect of reduced quality value resolution was assessed using the cramtools quality binning
conguration N40-R8 to allow direct comparison to the CRAM format, although the algorithm
implemented in SHORE achieved similar results. The N40-R8 setting preserves quality values of
all reference sequence mismatches, while assigning quality values of sequence matches to one of
eight possible bins. Application of quality resolution reduction improved compression ratio of
the quality data from 22.3% to 8.8%.
We further assessed the eect of various combinations of compression and data reduction
congurations on total data set size by application of the SHORE compress and SHORE coal utilities
to the data set in MapList format and compare the results with storage of SAM/BAM and CRAM
formatted data (table 2.2).
Eect of compression block size was evaluated using SHORE's random access granularity presets
fast, corresponding to 128 kB block size, medium, 2 MB, and slow, 64 MB. Compression
of the 1.1 GB data set in SHORE-GZIP format at 128 kB block size reduced space requirements
to 205 MB, or 19% of the uncompressed le size. Increasing block size to 2 MB further improved
space eciency by just 1.5% to 202 MB storage size, and use of 64 MB blocks did not have a
signicant advantage over 2 MB.
Selecting the XZ compression format at 128 kB block size resulted in a storage size of 163 MB,
15% of plain data set size and a 20% improvement over SHORE-GZIP at the same block size. Eect
of increased compression block size was more pronounced for XZ than for SHORE-GZIP. At 2 MB
and 64 MB block size disk usage was reduced to 140 MB and 116 MB, respectively, improvements
of 14% and 17% over the respective previous block size.
Block-wise transposition was applied using the block size parameter matching the respective
compression block size. At the smallest block size, size of the transformed data in SHORE-GZIP
format was 181 MB, a 12% improvement over untransposed storage. Block-wise transposition
also augmented the eect of increased block sizes, achieving le sizes of 160 MB at 2 MB and of