QualComp: A new lossy compressor for quality scores based on rate distortion theory

BMC Bioinformatics

Volumn 14, Issue 1, 2013, Pages

  Ochoa, Idoia ^a   Asnani, Himanshu ^a   Bharadia, Dinesh ^a   Chowdhury, Mainak ^a   Weissman, Tsachy ^a   Yona, Golan ^a  

^a Stanford University   (United States)

Author keywords

Compression; FASTQ format; Mean squared error; Next generation sequencing; Quality scores; Rate distortion

Indexed keywords

DOWNSTREAM APPLICATIONS; FASTQ FORMAT; MEAN SQUARED ERROR; NEXT-GENERATION SEQUENCING; NUCLEOTIDE SEQUENCES; NUMERICAL EXPERIMENTS; RATE DISTORTION-THEORY; RATE DISTORTIONS;

ALGORITHMS; COMPACTION; DIGITAL STORAGE; ELECTRIC DISTORTION; MEAN SQUARE ERROR; SOFTWARE ENGINEERING;

QUALITY CONTROL;

ALGORITHM; ANIMAL; ARTICLE; COMPUTER PROGRAM; GENOME; GENOMICS; HIGH THROUGHPUT SEQUENCING; INFORMATION PROCESSING; METHODOLOGY; MOUSE; SINGLE NUCLEOTIDE POLYMORPHISM; PROCEDURES;

ALGORITHMS; ANIMALS; DATA COMPRESSION; GENOME; GENOMICS; HIGH-THROUGHPUT NUCLEOTIDE SEQUENCING; MICE; POLYMORPHISM, SINGLE NUCLEOTIDE; SOFTWARE;

EID: 84878634014     PISSN: None     EISSN: 14712105     Source Type: Journal    
DOI: 10.1186/1471-2105-14-187     Document Type: Article

References (49)

1. Lander E, Linton L, Birren B, Nusbaum C, Zody M, Baldwin J, Devon K, Dewar K, Doyle M, FitzHugh W, Funke R, Gage D, Harris K, Heaford A, Howland J, Kann L, Lehoczky J, LeVine R, McEwan P, McKernan K, Meldrim J, Mesirov J, Miranda C, Morris W, Naylor J, Raymond C, Rosetti M, Santos R, Sheridan A, et al. Initial sequencing and analysis of the human genome. Nature 2001, 409(6822):860-921. 10.1038/35057062, 11237011.
2. Genome sequencing cost. http://www.genome.gov/sequencingcosts/.
3. Hess M, Sczyrba A, Egan R, Kim T, Chokhawala H, Schroth G, Luo S, Clark D, Chen F, Zhang T, Mackie R, Pennacchio L, Tringe S, Visel A, Woyke T, Wang Z, Rubin E. Metagenomic discovery of biomass-degrading genes and genomes from cow rumen. Science 2011, 331(6016):463. 10.1126/science.1200387, 21273488.
4. Qin J, Li R, Raes J, Arumugam M, Burgdorf K, Manichanh C, Nielsen T, Pons N, Levenez F, Yamada T, Mende D, Li J, Xu J, Li S, Li D, Cao J, Wang B, Liang H, Zheng H, Xie Y, Tap J, Lepage P, Bertalan M, Batto J, Hansen T, Paslier D, Linneber A, Bjorn Nielsen H, Pelletier E, Renault P EE. A human gut microbial gene catalogue established by metagenomic sequencing. Nature 2010, 464(7285):59-65. 10.1038/nature08821, 20203603.
5. Leinonen R, Sugawara H, Shumway M. The Sequence Read Archive. Nucleic Acids Res 2011, 39:19-21. 10.1093/nar/gkq768, 3017612, 20805242.
6. Cock PJA, Fields CJ, Goto N, Heuer ML, Rice PM. The Sanger FASTQ format for sequences with quality scores, and the Solexa/Illumina FASTQ variants. Nucleic Acids Res 2009, 38:1767-1771. 2847217, 20015970.
7. Lohse M, Bolger A, Nagel A, Fernie A, Lunn J, Stitt M, Usadel B. RobiNA: a user-friendly, integrated software solution for RNA-Seq-based transcriptomics. Nucleic Acids Res 2012, 40(W1):W622-627. 10.1093/nar/gks540, 3394330, 22684630.
8. Cox M, Peterson D, Biggs P. SolexaQA: At-a-glance quality assessment of Illumina second-generation sequencing data. BMC Bioinformatics 2010, 11:485. 10.1186/1471-2105-11-485, 2956736, 20875133.
9. Li H, Ruan J, Durbin R. Mapping short DNA sequencing reads and calling variants using mapping quality scores. Genome Res 2008, 18(11):1851-1858. 10.1101/gr.078212.108, 2577856, 18714091.
10. Langmead B, Trapnell C, Pop M, Salzberg S. Ultrafast and memory-efficient alignment of short DNA sequences to the human genome. Genome Biol 2009, 10(3):R25. 10.1186/gb-2009-10-3-r25, 2690996, 19261174.
11. Li H, Durbin R. Fast and accurate short read alignment with Burrows-Wheeler transform. Bioinformatics 2009, 25(14):1754-1760. 10.1093/bioinformatics/btp324, 2705234, 19451168.
12. Lunter G, Goodson M. Stampy: A statistical algorithm for sensitive and fast mapping of Illumina sequence reads. Genome Res 2011, 21(6):936-939. 10.1101/gr.111120.110, 3106326, 20980556.
13. McKenna A, Hanna M, Banks E, Sivachenko A, Cibulskis K, Kernytsky A, Garimella K, Altshuler D, Gabriel S, Daly M, DePristo M. The Genome Analysis Toolkit: a MapReduce framework for analyzing next-generation DNA sequencing data. Genome Res 2010, 20(9):1297-1303. 10.1101/gr.107524.110, 2928508, 20644199.
14. Zhang J, Wheeler D, Yakub I, Wei S, Sood R, Rowe W, Liu P, Gibbs R, Buetow K. SNPdetector: a software tool for sensitive and accurate SNP detection. PLoS Comput Biol 2005, 1(5):e53. 10.1371/journal.pcbi.0010053, 1274293, 16261194.
15. Cao M, Dix T, Allison L, Mears C. A simple statistical algorithm for biological sequence compression. Data Compression Conference, 2007. DCC'07 2007, 43-52. Snowbird, UT, USA: IEEE.
16. Chen X, Kwon S, Li M. A compression algorithm for DNA sequences and its applications in genome comparison. Proceedings of the Fourth Annual International Conference on Computational Molecular Biology 2000, 107-107. Tokyo, Japan: ACM.
17. Chen X, Li M, Ma B, Tromp J. DNACompress: Fast and effective DNA sequence compression. Bioinformatics 2002, 18:1696-1698. 10.1093/bioinformatics/18.12.1696, 12490460.
18. Pinho AJ, Ferreira P, Neves A, Bastos C. On the representability of complete genomes by multiple competing finite-context (Markov) models. PLoS ONE 2011, 6(6):e21588. 10.1371/journal.pone.0021588, 3128062, 21738720.
19. Sato H, Yoshioka T, Konagaya A, Toyoda T. DNA data compression in the post genome era. Genome Inf 2001, 12:512-514.
20. Christley S, Lu Y, Li C, Xie X. Human Genomes as email attachments. Genome Inf 2008, 25:274-275.
21. Pinho AJ, Pratas D, Garciaa SP. GReEn: a tool for efficient compression of genome resequencing data. Nucleic Acids Res 2011, 40(4):e27-27. 3287168, 22139935.
22. Kuruppu S, Puglisi SJ, Zobel J. Relative Lempel-Ziv compression of genomes for large-scale storage and retrieval. String Processing and Information Retrieval 2010, 201-206. Los Cabos, Mexico: Springer.
23. Kuruppu S, Puglisi SJ, Zobel J. Optimized relative Lempel-Ziv compression of genomes. Proceeding of ACSC 2011, Perth, Australia: Australasian Computer Science Conference (ACSC).
24. Heath LS, Hou A, Xia H, Zhang L. A genome compression algorithm supporting manipulation. Proc LSS Comput Syst Bioinform Conf 2010, 9:38-49.
25. Ma N, Ramchandran K, Tse D. A Compression Algorithm Using Mis-aligned side information. Information Theory Proceedings (ISIT), 2012 IEEE International Symposium on 2012, 16-20. Cambridge, Massachusetts, USA: IEEE.
26. Wang C, Zhang D. A novel compression tool for efficient storage of genome resequencing data. Nucleic Acids Res 2011, 39(7):e45-45. 10.1093/nar/gkr009, 3074166, 21266471.
27. Chern BG, Ochoa I, Manolakos A, No A, Venkat K, Weissman T. Reference based genome compression. IEEE Inf Theory Workshop, ITW 2012, 427-431. Lausanne, Switzerland: IEEE.
28. Timothy W, White J, Hendy MD. Compressing DNA sequence databases with coil. Bioinformatics 2008, 9(1):242. 2426707, 18489794.
29. Deorowicz S, Grabowski S. Compression of genomic sequences in FASTQ format. Bioinformatics 2011, 27(6):860-862. 10.1093/bioinformatics/btr014, 21252073.
30. Tembe W, Lowey J, Suh E. G-SQZ: compact encoding of genomic sequence and quality data. Bioinformatics 2010, 26:2192-2194. 10.1093/bioinformatics/btq346, 20605925.
31. Jones DC, Ruzzo WL, Peng X, Katze MG. Compression of next-generation sequencing reads aided by highly efficient de novo assembly. Nucleic Acids Res 2012, 40(22):e171-171. 10.1093/nar/gks754, 3526293, 22904078.
32. Kozanitis C, Saunders C, Kruglyak S, Bafna V, Varghese G. Compressing genomic sequence fragments using SlimGene. J Comput Biol J Comput Mol Cell Biol 2011, 18:401-413.
33. Fritz MH, Leinonen R, Cochrane G, Birney E. Efficient storage of high throughput sequencing data using reference-based compression. Genome Res 2011, 21:734-774. 10.1101/gr.114819.110, 3083090, 21245279.
34. fastqz. http://mattmahoney.net/dc/fastqz/.
35. Hach F, Numanagić I, Alkan C, Sahinalp SC. SCALCE: boosting sequence compression algorithms using locally consistent encoding. Bioinformatics 2012, 28(23):3051-3057. 10.1093/bioinformatics/bts593, 23047557.
36. Cramtools. https://github.com/vadimzalunin/crammer.
37. The Pistoia Alliance. http://www.sequencesqueeze.org/.
38. Cochrane G, Cook C, Birney E. The future of DNA sequence archiving. GigaScience 2012, 1:2. http://www.gigasciencejournal.com/content/1/1/2, 10.1186/2047-217X-1-2, 3617450, 23587147.
39. Wan R, Anh VN, Asai K. Transformations for the compression of FASTQ quality scores of next generation sequencing data. Bioinformatics 2011, 28(5):628-635.
40. Lapidoth A. On the role of mismatch in rate distortion theory. Inf Theory, IEEE Trans 1997, 43(1):38-47.
41. Cover T, Thomas J. Elements of Information Theory, Volume 6 1991,
42. Lloyd S. Least squares quantization in PCM. Inf Theory, IEEE Trans on 1982, 28(2):129-137.
43. MacQueen J. Some methods for classification and analysis of multivariate observations. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1 1967, 14-14. California: University of California Press.
44. SRR032209 data. http://trace.ddbj.nig.ac.jp/DRASearch/run?acc=SRR032209.
45. SRR089526 data. http://trace.ddbj.nig.ac.jp/DRASearch/run?acc=SRR089526.
46. PhiX data. http://bix.ucsd.edu/projects/singlecell/nbt\_data.html.
47. QualComp website. https://sourceforge.net/projects/qualcomp/.
48. PhiX174 Genome. http://www.ncbi.nlm.nih.gov/nuccore/NC_001422.
49. Li H, Handsaker B, Wysoker A, Fennell T, Ruan J, Homer N, Marth G, Abecasis G, Durbin R. The sequence alignment/map format and SAMtools. Bioinformatics 2009, 25(16):2078-2079. 10.1093/bioinformatics/btp352, 2723002, 19505943.