Improved index compression techniques for versioned document collections

International Conference on Information and Knowledge Management, Proceedings

Volumn , Issue , 2010, Pages 1239-1248

  He, Jinru ^a   Zeng, Junyuan ^a   Suel, Torsten ^a  

^a POLYTECHNIC UNIVERSITY   (United States)

Author keywords

Index compression; Inverted index; Versioned documents

Indexed keywords

BITWISE TECHNIQUES; COMPRESSION TECHNIQUES; DATA SETS; DOCUMENT COLLECTION; INDEX COMPRESSION; INDEX STRUCTURE; INVERTED INDEX STRUCTURES; INVERTED INDICES; LARGE SIZES; NEW INDICES; VERSIONED DOCUMENTS;

INFORMATION RETRIEVAL SYSTEMS; KNOWLEDGE MANAGEMENT; QUERY PROCESSING;

INFORMATION RETRIEVAL;

EID: 78651335930     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1871437.1871594     Document Type: Conference Paper

References (33)

1. I. Altingovde, E. Demir, F. Can, and O. Ulusoy. Incremental cluster-based retrieval using compressed cluster-skipping inverted files. ACM Trans. on Information Systems, 26(3), June 2008.
2. P. G. Anick and R. A. Flynn. Versioning a full-text information retrieval system. In Proc. of the ACM SIGIR Conf., 1992.
3. J. L. Bentley. Multidimensional binary search trees used for associative searching. Commun. ACM, 18(9), 1975.
4. K. Berberich, S. Bedathur, T. Neumann, and G. Weikum. A time machine for text search. In Proc. of the ACM SIGIR Conf., 2007.
5. B. Bhattacharjee, S. Chawathe, V. Gopalakrishnan, P. Keleher, and B. Silaghi. Efficient peer-to-peer searches using result-caching. In Proc. of the Workshop on P2P Systems, 2003.
6. R. Blanco and A. Barreiro. Document identifier reassignment through dimensionality reduction. In Proc. of the ECIR Conf., 2005.
7. D. Blandford and G. Blelloch. Index compression through document reordering. In Proc. of the DCC Conf., 2002.
8. A. Broder, N. Eiron, M. Fontoura, M. Herscovici, R. Lempel, J. McPherson, R. Qi, and E. Shekita. Indexing shared content in information retrieval systems. In Proc. of the EDBT Conf., 2006.
9. L. Cox, C. Murray, and B. Noble. Pastiche: Making backup cheap and easy. In Proc. of the 5th Symp. on Operating System Design and Implementation, December 2002.
10. A. S. Fraenkel, S. T. Klein, Y. Choueka, and E. Segal. Improved hierarchical bit-vector compression in document retrieval systems. In Proc. of the ACM SIGIR Conf., 1986.
11. J. He, H. Yan, and T. Suel. Compact full-text indexing of versioned document collections. In Proc. of the ACM CIKM Conf., 2009.
12. S. Heman. Super-scalar database compression between RAM and CPU-cache. MS Thesis, Centrum voor Wiskunde en Informatica, 2005.
13. M. Herscovici, R. Lempel, and S. Yogev. Efficient indexing of versioned document sequences. In Proc. of the ECIR Conf., 2007.
14. R. Karp and M. Rabin. Efficient randomized pattern-matching algorithms. IBM J. of Research and Development, 31(2), 1987.
15. P. Kulkarni, F. Douglis, J. LaVoie, and J. Tracey. Redundancy elimination within large collections of files. In Proc. of the USENIX Annual Technical Conf., 2004.
16. H. Leong, N. Mamoulis, K. Berberich, and S. Bedathur. Durable top-k search in document archives. In Proc. of the ACM SIGMOD Conf., 2010.
17. A. Moffat and L. Stuiver. Binary interpolative coding for effective index compression. Information Retrieval, 3, 2000.
18. A. Muthitacharoen, B. Chen, and D. Mazières. A low-bandwidth network file system. In Proc. of the ACM Symp. on Operating Systems Principles, 2001.
19. A. Orlitsky. Interactive communication of balanced distributions and of correlated files. SIAM J. of Discrete Math, 6(4), 1993.
20. V. Poosala and Y. Ioannidis. Selectivity estimation without the attribute value independence assumption. Proc. of the Int. Conf. on Very Large Data Bases, 1997.
21. S. Quinlan and S. Dorward. Venti: a new approach to archival storage. In Proc. of the First USENIX FAST Conf., 2002.
22. S. Schleimer, D. Wilkerson, and A. Aiken. Winnowing: Local algorithms for document fingerprinting. In Proc. of the ACM SIGMOD Conf., 2003.
23. T. Schwarz, R. Bowdidge, and W. Burkhard. Low cost comparison of file copies. In Proc. of the Int. Conf. on Distributed Computing Systems, 1990.
24. W. Shieh, T. Chen, J. Shann, and C. Chung. Inverted file compression through document identifier reassignment. Inf. Processing and Management, 39(1), 2003.
25. F. Silvestri. Sorting out the document identifier assignment problem. In Proc. of the ECIR Conf., 2007.
26. F. Silvestri, S. Orlando, and R. Perego. Assigning identifiers to documents to enhance the clustering property of fulltext indexes. In Proc. of the ACM SIGIR Conf., 2004.
27. D. Teodosiu, N. Bjorner, Y. Gurevich, M. Manasse, and J. Porkka. Optimizing file replication over limited bandwidth networks using remote differential compression. TR2006-157-1, Microsoft, 2006.
28. A. Tridgell and P. Mackerras. The rsync algorithm. Technical Report TR-CS-96-05, Australian National University, June 1996.
29. H. Yan, S. Ding, and T. Suel. Inverted index compression and query processing with optimized document ordering. In Proc. of 18th WWW Conf., 2009.
30. J. Zhang, X. Long, and T. Suel. Performance of compressed inverted list caching in search engines. In Proc. of 17th WWW Conf., 2008.
31. J. Zhang and T. Suel. Efficient search in large textual collection with redundancy. In Proc. of 16th WWW Conf., 2007.
32. J. Zobel and A. Moffat. Inverted files for text search engines. ACM Computing Surveys, 38(2), 2006.
33. M. Zukowski, S. Heman, N. Nes, and P. Boncz. Super-scalar RAM-CPU cache compression. In Proc. of the ICDE Conf., 2006.