Data structures and compression algorithms for high-throughput sequencing technologies

BMC Bioinformatics

Volumn 11, Issue , 2010, Pages

  Daily, Kenny ^a,b   Rigor, Paul ^a,b   Christley, Scott ^a,c   Xie, Xiaohui ^a,b,c   Baldi, Pierre ^a,b,c  

^a Irvine   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPRESSION ALGORITHMS; COMPRESSION TECHNIQUES; ENTROPY-CODING ALGORITHMS; EXPERIMENTAL CONDITIONS; EXPERIMENTAL PROTOCOLS; GENERAL METHODOLOGIES; HIGH-THROUGHPUT SEQUENCING; STATISTICAL PROPERTIES;

BIOINFORMATICS; DATA HANDLING; DATA STRUCTURES; DIGITAL STORAGE; ENCODING (SYMBOLS); EXPERIMENTS; TECHNOLOGY;

ALGORITHMS;

ALGORITHM; ARTICLE; BIOLOGY; ENTROPY; FACTUAL DATABASE; HIGH THROUGHPUT NUCLEOTIDE SEQUENCING; INFORMATION PROCESSING; METHODOLOGY;

ALGORITHMS; COMPUTATIONAL BIOLOGY; DATA COMPRESSION; DATABASES, FACTUAL; ENTROPY; HIGH-THROUGHPUT NUCLEOTIDE SEQUENCING;

EID: 77957765256     PISSN: None     EISSN: 14712105     Source Type: Journal    
DOI: 10.1186/1471-2105-11-514     Document Type: Article

References (35)

1. International Human Genome Sequencing Consortium Initial sequencing and analysis of the human genome. Nature 2001, 409:860-921. International Human Genome Sequencing Consortium.
2. Levy S, Sutton G, Ng PC, Feuk L, Halpern AL, Walenz BP, Axelrod N, Huang J, Kirkness EF, Denisov G, Lin Y, MacDonald JR, Pang AWC, Shago M, Stockwell TB, Tsiamouri A, Bafna V, Bansal V, Kravitz SA, Busam DA, Beeson KY, McIntosh TC, Remington KA, Abril JF, Gill J, Borman J, Rogers YH, Frazier ME, Scherer SW, Strausberg RL, Venter JC. The diploid genome sequence of an individual human. PLoS Biol 2007, 5(10):e254.
3. Wheeler DA, Srinivasan M, Egholm M, Shen Y, Chen L, McGuire A, He W, Chen YJ, Makhijani V, Roth GT, Gomes X, Tartaro K, Niazi F, Turcotte CL, Irzyk GP, Lupski JR, Chinault C, Song XZ, Liu Y, Yuan Y, Nazareth L, Qin X, Muzny DM, Margulies M, Weinstock GM, Gibbs RA, Rothberg JM. The complete genome of an individual by massively parallel DNA sequencing. Nature 2008, 452(7189):872-6.
4. Bentley DR, Balasubramanian S, Swerdlow HP, Smith GP, Milton J, Brown CG, Hall KP, Evers DJ, Barnes CL, Bignell HR, Boutell JM, Bryant J, Carter RJ, Cheetham RK, Cox AJ, Ellis DJ, Flatbush MR, Gormley NA, Humphray SJ, Irving LJ, Karbelashvili MS, Kirk SM, Li H, Liu X, Maisinger KS, Murray LJ, Obradovic B, Ost T, Parkinson ML, Pratt MR, Rasolonjatovo IMJ, Reed MT, Rigatti R, Rodighiero C, Ross MT, Sabot A, Sankar SV, Scally A, Schroth GP, Smith ME, Smith VP, Spiridou A, Torrance PE, Tzonev SS, Vermaas EH, Walter K, Wu X, Zhang L, Alam MD, Anastasi C, Aniebo IC, Bailey DMD, Bancarz IR, Banerjee S, Barbour SG, Baybayan PA, Benoit VA, Benson KF, Bevis C, Black PJ, Boodhun A, Brennan JS, Bridgham JA, Brown RC, Brown AA, Buermann DH, Bundu AA, Burrows JC, Carter NP, Castillo N, Catenazzi MCE, Chang S, Cooley RN, Crake NR, Dada OO, Diakoumakos KD, Dominguez-Fernandez B, Earnshaw DJ, Egbujor UC, Elmore DW, Etchin SS, Ewan MR, Fedurco M, Fraser LJ, Fajardo KVF, Furey WS, George D, Gietzen KJ, Goddard CP, Golda GS, Granieri PA, Green DE, Gustafson DL, Hansen NF, Harnish K, Haudenschild CD, Heyer NI, Hims MM, Ho JT, Horgan AM, Hoschler K, Hurwitz S, Ivanov DV, Johnson MQ, James T, Jones TAH, Kang GD, Kerelska TH, Kersey AD, Khrebtukova I, Kindwall AP, Kingsbury Z, Kokko-Gonzales PI, Kumar A, Laurent MA, Lawley CT, Lee SE, Lee X, Liao AK, Loch JA, Lok M, Luo S, Mammen RM, Martin JW, Mccauley PG, Mcnitt P, Mehta P, Moon KW, Mullens JW, Newington T, Ning Z, Ng BL, Novo SM, O'neill MJ, Osborne MA, Osnowski A, Ostadan O, Paraschos LL, Pickering L, Pike AC, Pike AC, Pinkard DC, Pliskin DP, Podhasky J, Quijano VJ, Raczy C, Rae VH, Rawlings SR, Rodriguez AC, Roe PM, Rogers J, Bacigalupo MCR, Romanov N, Romieu A, Roth RK, Rourke NJ, Ruediger ST, Rusman E, Sanches-Kuiper RM, Schenker MR, Seoane JM, Shaw RJ, Shiver MK, Short SW, Sizto NL, Sluis JP, Smith MA, Sohna JES, Spence EJ, Stevens K, Sutton N, Szajkowski L, Tregidgo CL, Turcatti G, Vandevondele S, Verhovsky Y, Virk SM, Wakelin S, Walcott GC, Wang J, Worsley GJ, Yan J, Yau L, Zuerlein M, Rogers J, Mullikin JC, Hurles ME, Mccooke NJ, West JS, Oaks FL, Lundberg PL, Klenerman D, Durbin R, Smith AJ. Accurate whole human genome sequencing using reversible terminator chemistry. Nature 2008, 456(7218):53-59.
5. Wang J, Wang W, Li R, Li Y, Tian G, Goodman L, Fan W, Zhang J, Li J, Zhang J, Guo Y, Feng B, Li H, Lu Y, Fang X, Liang H, Du Z, Li D, Zhao Y, Hu Y, Yang Z, Zheng H, Hellmann I, Inouye M, Pool J, Yi X, Zhao J, Duan J, Zhou Y, Qin J, Ma L, Li G, Yang Z, Zhang G, Yang B, Yu C, Liang F, Li W, Li S, Li D, Ni P, Ruan J, Li Q, Zhu H, Liu D, Lu Z, Li N, Guo G, Zhang J, Ye J, Fang L, Hao Q, Chen Q, Liang Y, Su Y, San A, Ping C, Yang S, Chen F, Li L, Zhou K, Zheng H, Ren Y, Yang L, Gao Y, Yang G, Li Z, Feng X, Kristiansen K, Wong GKS, Nielsen R, Durbin R, Bolund L, Zhang X, Li S, Yang H, Wang J. The diploid genome sequence of an Asian individual. Nature 2008, 456(7218):60-5.
6. Kaiser J. A Plan to Capture Human diversity in 1000 Genomes. Science 2008, 319:395.
7. Service RF. The Race for the $1000 Genome. Science 2006, 311:1544-1546.
8. Mardis ER. ChIP-seq: welcome to the new frontier. Nature Methods 2007, 4:613-614.
9. Hall N. Advanced Sequencing Technologies and their Wider Impact in Microbiology. The Journal of Experimental Biology 2007, 209:1518-1525.
10. Li H, Ruan J, Durbin R. Mapping short DNA sequencing reads and calling variants using mapping quality scores. Genome Res 2008, 18(11):1851-8.
11. Lin H, Zhang Z, Zhang MQ, Ma B, Li M. ZOOM! Zillions of oligos mapped. Bioinformatics 2008, 24(21):2431-7.
12. Langmead B, Trapnell C, Pop M, Salzberg S. Ultrafast and memory-efficient alignment of short DNA sequences to the human genome. Genome Biology 2009, 10(3):R25., http://genomebiology.com/2009/10/3/R25
13. Grumbach S, Tahi F. A new challenge for compression algorithms: Genetic sequences. Information Processing & Management 1994, 30(6):875-886.
14. Matsumoto T, Sadakane K, Imai H. Biological sequence compression algorithms. Genome informatics 2000, 11:43-52.
15. Apostolico A, Lonardi S. Off-Line Compression by Greedy Textual Substitution. Proceedings of the IEEE 2000, 88(11):1733-1744.
16. Chen X, Li M, Ma B, Tromp J. DNACompress: fast and effective DNA sequence compression. Bioinformatics 2002, 18:1696-1698.
17. Manzini G, Rastero M. A simple and fast DNA compressor. Softw Pract Exper 2004, 34(14):1397-1411.
18. White WTJ, Hendy MD. Compressing DNA sequence databases with coil. BMC Bioinformatics 2008, 9:242.
19. Christley S, Lu Y, Li C, Xie X. Human Genomes as Email Attachments. Bioinformatics 2008, 25:274-275.
20. The gzip home page. , http://www.gzip.org
21. McEliece RJ. The Theory of Information and Coding. 1977, Reading, MA: Addison-Wesley Publishing Company.
22. Cover TM, Thomas JA. Elements of Information Theory. 1991, New York: John Wiley.
23. Golomb SW. Run-Length Encodings. IEEE Transactions on Information Theory 1965, 12(3):399-401.
24. Elias P. Universal Codeword Sets and Representations of the Integers. IEEE Transactions on Information Theory 1975, 21(2):194-203.
25. Baldi P, Benz RW, Hirschberg D, Swamidass S. Lossless Compression of Chemical Fingerprints Using Integer Entropy Codes Improves Storage and Retrieval. Journal of Chemical Information and Modeling 2007, 47(6):2098-2109.
26. Huffman D. A method for the construction of minimum redundancy codes. Proc IRE 1952, 40:1098-1101.
27. Moffat A, Stuiver L. Binary Interpolative Coding for Effective Index Compression. Inf Retr 2000, 3:25-47.
28. Moffat A, Anh V. Binary codes for locally homogeneous sequences. Information Processing Letters 2006, 99:175-180.
29. Hirschberg DS, Baldi P. Effective Compression of Monotone and Quasi-Monotone Sequences of Integers. Proceedings of the 2008 Data Compression Conference (DCC 08) 2008, Los Alamitos, CA: IEEE Computer Society Press, .
30. Rissanen JJ, Langdonr GG. Arithmetic coding. IBM Journal of Research and Development 1979, 23(2):149-162.
31. Witten IH, Neal RM, Clearly JG. Arithmetic Coding for Data Compression. Communications of the ACM 1987, 30(6):520-540.
32. Kao MY. Encyclopedia of Algorithms. 2007, Secaucus, NJ, USA: Springer-Verlag New York, Inc.
33. Witten I, Moffat A, Cell TB. Managing Gigabytes: Compressing and Indexing Documents and Images. 1999, Morgan Kauffman, Second.
34. Johnson DS, Mortazavi A, Myers RM, Wold B. Genome-wide mapping of in vivo protein-DNA interactions. Science 2007, 316:1497-1502.
35. Li G, Ma L, Song C, Yang Z, Wang X, Huang H, Li Y, Li R, Zhang X, Yang H, Wang J, Wang J. The YH database: the first Asian diploid genome database. Nucleic Acids Res 2009, 37:D1025-8.