MRCSI: Compressing and Searching String Collections with Multiple References

Proceedings of the VLDB Endowment

Volumn 8, Issue 5 5, 2015, Pages 461-472

  Wandelt, Sebastian ^a   Leser, Ulf ^a  

^a HUMBOLDT UNIVERSITY   (Germany)

Author keywords

Compression; Dissimilar strings; Indexing

Indexed keywords

COMPRESSION RATIO (MACHINERY);

CHANGE HISTORY; COMPRESSION; DISSIMILAR STRING; INDEXING; LARGE POPULATION; MULTI REFERENCE; MULTIPLE REFERENCES; SEARCH EFFICIENCY; SEARCH/INDEX; SPACE REQUIREMENTS;

STORAGE EFFICIENCY;

EID: 84955565000     PISSN: None     EISSN: 21508097     Source Type: Journal    
DOI: 10.14778/2735479.2735480     Document Type: Chapter

References (39)

1. R. A. Baeza-Yates and C. H. Perleberg. Fast and practical approximate string matching. Inf.Proc.Lett., 59(1):21-27, 1996.
2. A. Cannane and H. E. Williams. A general-purpose compression scheme for large collections. ACM Trans. Inf. Syst., 20(3):329-355, July 2002.
3. T. Cheng, X. Yan, and K. C.-C. Chang. Entityrank: Searching entities directly and holistically. In Proceedings of the 33rd International Conference on Very Large Data Bases, VLDB '07, pages 387-398. VLDB Endowment, 2007.
4. F. Claude, A. Fariña, et al. Indexes for highly repetitive document collections. In CIKM, pages 463-468, New York, NY, USA, 2011. ACM.
5. M. Cohn and R. Khazan. Parsing with prefix and suffix dictionaries. In Data Compression Conf., pages 180-189, 1996.
6. A. Danek, S. Deorowicz, and S. Grabowski. Indexing large genome collections on a PC. CoRR, abs/1403.7481, 2014.
7. S. Deorowicz and S. Grabowski. Robust Relative Compression of Genomes with Random Access. Bioinformatics (Oxford, England), Sept. 2011.
8. S. Deorowicz and S. Grabowski. Data compression for sequencing data. Algorithms for Molecular Biology, 8:25, 2013.
9. A. Döring, D. Weese, et al. Seqan an efficient, generic C++ library for sequence analysis. BMC Bioinformatics, 9, 2008.
10. H. Ferrada, T. Gagie, et al. AliBI: An Alignment-Based Index for Genomic Datasets. ArXiv e-prints, July 2013.
11. H. Ferrada, T. Gagie, et al. Hybrid indexes for repetitive datasets. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 372(2016), 2014.
12. T. Gagie, K. Karhu, et al. Document listing on repetitive collections. In J. Fischer and P. Sanders, editors, Combinatorial Pattern Matching, volume 7922 of Lecture Notes in Computer Science, pages 107-119. Springer Berlin Heidelberg, 2013.
13. S. Gerdjikov, S. Mihov, et al. Wallbreaker: Overcoming the wall effect in similarity search. In Proceedings of the Joint EDBT/ICDT 2013 Workshops, EDBT '13, pages 366-369, New York, NY, USA, 2013. ACM.
14. O. Harismendy, P. Ng, et al. Evaluation of next generation sequencing platforms for population targeted sequencing studies. Genome Biology, 10(3):R32+, 2009.
15. W. Hon, K. Sadakane, and W. Sung. Breaking a time-and-space barrier in constructing full-text indices. SIAM Journal on Computing, 38(6):2162-2178, 2009.
16. C. Hoobin, S. Puglisi, and J. Zobel. Sample selection for dictionary-based corpus compression. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '11, pages 1137-1138, New York, NY, USA, 2011. ACM.
17. C. Hoobin, S. J. Puglisi, and J. Zobel. Relative Lempel-Ziv factorization for efficient storage and retrieval of web collections. PVLDB, 5(3):265-273, 2011.
18. Y. Jiang, D. Deng, et al. Efficient parallel partition-based algorithms for similarity search and join with edit distance constraints. In Proceedings of the Joint EDBT/ICDT 2013 Workshops, EDBT '13, pages 341-348, New York, NY, USA, 2013. ACM.
19. D. E. Knuth, J. H. Morris, Jr, and V. R. Pratt. Fast pattern matching in strings. SIAM journal on computing, 6(2):323-350, 1977.
20. S. Kreft and G. Navarro. On compressing and indexing repetitive sequences. Theoretical Computer Science, 483(0):115 -133, 2013.
21. S. Kuruppu, S. J. Puglisi, and J. Zobel. Relative lempel-ziv compression of genomes for large-scale storage and retrieval. In Proceedings of SPIRE 2010, pages 201-206, Berlin, Heidelberg, 2010. Springer-Verlag.
22. N. Larsson and A. Moffat. Offline dictionary-based compression. In Data Compression Conference, 1999. Proceedings. DCC '99, pages 296-305, Mar 1999.
23. P.-R. Loh, M. Baym, and B. Berger. Compressive genomics. Nature Biotechnology, 30(7):627-630, July 2012.
24. A. Max and G. Wisniewski. Mining naturally-occurring corrections and paraphrases from wikipedias revision history. In N. Calzolari, K. Choukri, et al., editors, LREC 2010, Valletta, Malta, 2010. European Language Resources Association.
25. E. McCreight. Efficient algorithms for enumerating intersection intervals and rectangles. Technical report, Xerox Paolo Alte Research Center, 1980.
26. S. M. Mola-Velasco. Wikipedia vandalism detection. In S. Srinivasan, K. Ramamritham, et al., editors, WWW (Companion Volume), pages 391-396. ACM, 2011.
27. G. Navarro, S. Puglisi, and J. Siren. Document retrieval on repetitive collections. In A. Schulz and D. Wagner, editors, Algorithms -ESA 2014, volume 8737 of Lecture Notes in Computer Science, pages 725-736. Springer Berlin Heidelberg, 2014.
28. E. Ohlebusch, J. Fischer, and S. Gog. CST++. In SPIRE'10, pages 322-333, 2010.
29. M. Patil, S. V. Thankachan, et al. Inverted indexes for phrases and strings. In SIGIR 2011, pages 555-564, New York, NY, USA, 2011. ACM.
30. A. J. Pinho, D. Pratas, and S. P. Garcia. Green: a tool for efficient compression of genome resequencing data. Nucleic Acids Research, Dec. 2011.
31. T. Rausch, A.-K. Emde, et al. Segment-based multiple sequence alignment. Bioinformatics, 24(16):i187-i192, 2008.
32. K. Schneeberger, J. Hagmann, et al. Simultaneous alignment of short reads against multiple genomes. Genome biology, 10(9):R98+, Sept. 2009.
33. J. Siren, N. Välimäki, and V. Mäkinen. Indexing graphs for path queries with applications in genome research. IEEE/ACM Transactions on Computational Biology and Bioinformatics, (accepted).
34. J. Tong, A. Wirth, and J. Zobel. Principled dictionary pruning for low-memory corpus compression. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '14, pages 283-292, New York, NY, USA, 2014. ACM.
35. S. Wandelt, M. Bux, and U. Leser. Trends in genome compression. Current Bioinformatics, 9(3):315-326, 2014.
36. S. Wandelt and U. Leser. FRESCO: Referential compression of highly similar sequences. IEEE/ACM Trans. Comput. Biol. Bioinformatics, 10(5):1275-1288, Sept. 2013.
37. S. Wandelt, J. Starlinger, et al. RCSI: Scalable similarity search in thousand(s) of genomes. PVLDB, 6(13):1534-1545, 2013.
38. X. Yang, B. Wang, et al. Efficient direct search on compressed genomic data. In C. S. Jensen, C. M. Jermaine, and X. Zhou, editors, ICDE, pages 961-972. IEEE Computer Society, 2013.
39. M. Zukowski, S. Heman, et al. Super-scalar RAM-CPU cache compression. In ICDE, pages 59-70, Washington, DC, USA, 2006. IEEE Computer Society.