Efficient online index construction for text databases

ACM Transactions on Database Systems

Volumn 33, Issue 3, 2008, Pages

  Lester, Nicholas ^a,c   Moffat, Alistair ^b   Zobel, Justin ^a  

^a RMIT UNIVERSITY   (Australia)

^b UNIVERSITY OF MELBOURNE   (Australia)

^c RMIT University ^*  (United States)

Author keywords

Index construction; Index update; Search engines; Text indexing

Indexed keywords

INDEX CONSTRUCTION; INDEX UPDATE; SEARCH ENGINES; TEXT INDEXING;

EID: 51149086157     PISSN: 03625915     EISSN: 15574644     Source Type: Journal    
DOI: 10.1145/1386118.1386125     Document Type: Article

References (36)

1. ANH, V. AND MOFFAT, A. 2006. Improved word-aligned binary compression for text indexing. IEEE Trans. Knowl. Data Engin. 18, 6, 857-861.
2. BAEZA-YATES, R. AND RIBEIRO- NETO, B. 1999. Modern Information Retrieval. Addison-Wesley Longman, Reading, MA.
3. BAILEY, P., CRASWELL, N., AND HAWKING, D. 2003. Engineering a multi-purpose test collection for web retrieval experiments. Inform. Process. Manag. 39, 6, 853-871.
4. BAYER, R. 1972. Symmetric binary B-trees: Data structure and maintenance algorithms. Acta Inf. 1, 290-306.
5. BILIRIS, A. 1992a. An efficient database storage structure for large dynamic objects. In Proceedings of the IEEE International Conference on Data Engineering. IEEE Computer Society, 301-308.
6. BILIRIS, A. 1992b. The performance of three database storage structures for managing large objects. In Proceedings of the ACM SIGMOD International Conference on the Management of Data (SIGMOD'92). ACM, New York, 276-285.
7. BROWN, E., CALLAN, J., AND CROFT, W. 1994. Fast incremental indexing for full-text information retrieval. In Proceedings of the International Conference on Very Large Databases (VLDB'94). 192-202.
8. BÜTTCHER, S. AND CLARKE, C. 2005. Indexing time vs. query time trade-offs in dynamic information retrieval systems. In Proceedings of the ACM CIKM International Conference on Information and Knowledge Management (CIKM'05). ACM, New York, 317-318.
9. BÜTTCHER, S. AND CLARKE, C. 2006. A hybrid approach to index maintenance in dynamic text retrieval systems. In Proceedings of the European Conference on Information Retrieval (ECIR'06). 229-240.
10. BÜTTCHER, S., CLARKE, C., AND LUSHMAN, B. 2006. Hybrid index maintenance for growing text collections. In Proceedings of the ACM-SIGIR International Conference on Research and Development in Information Retrieval (SIGIR'06). ACM, New York, 356-363.
11. CAREY, M., DEWITT, D., RICHARDSON, J., AND SHEKITA, E. 1986. Object and file management in the EXODUS extensible database system. In Proceedings of the International Conference on Very Large Databases (VLDB'86). Morgan Kaufmann, 91-100.
12. CAREY, M., DEWITT, D., RICHARDSON, J., AND SHEKITA, E. 1989. Storage management for objects in EXODUS. In Object-Oriented Concepts, Databases, and Applications, W. Kim and F. Lochovsky, Eds. Addison-Wesley Longman, New York, 341-369.
13. CLARKE, C. AND CORMACK, G. 1995. Dynamic inverted indexes for a distributed full-text retrieval system. MultiText Project Tech. rep. MT-95-01, Department of Computer Science, University of Waterloo, Waterloo, Canada.
14. CLARKE, C., CRASWELL, N., AND SOBOROFF, I. 2004. Overview of TREC 2004 terabyte track. In Proceedings of the 13th Text REtrieval Conference (TREC-13). National Institute of Standards and Technology Special Publication 500-261, Gaithersburg, MD.
15. COMER, D. 1979. The ubiquitous B-tree. Comput. Surv. 11, 2, 121-137.
16. CRASWELL, N. AND HAWKING, D. 2002. Overview of the TREC-2002 web track. In Proceedings of the 11th Text REtrieval Conference (TREC-2002). National Institute of Standards and Technology Special Publication 500-251, Gaithersburg, MD, 86-95.
17. CUTTING, D. AND PEDERSEN, J. 1990. Optimizations for dynamic inverted index maintenance. In Proceedings of the ACM-SIGIR International Conference on Research and Development in Information Retrieval (SIGIR'90). ACM, New York, 405-411.
18. DEN BERCKEN, J. V., SEEGER, B., AND WIDMAYER, P. 1997. A generic approach to bulk loading multidimensional index structures. In Proceedings of the Conference on Very Large Data Bases (VLDB'97). Morgan Kaufmann, San Francisco, CA, 406-415.
19. GRAEFE, G. 2006. B-tree indexes for high update rates. SIGMOD Rec. 35, 1, 39-44.
20. HAMILTON, J. R. AND NAYAK, T. K. 2001. Microsoft SQL server full-text search. IEEE Data Engin. Bull. 24, 4, 7-10.
21. HARMAN, D. 1993. Overview of the first TREC conference. In Proceedings of the ACM-SIGIR International Conference on Research and Development in Information Retrieval (SIGIR'93). ACM, New York, 36-47.
22. HEINZ, S. AND ZOBEL, J. 2003. Efficient single-pass index construction for text databases. J. Amer. Soc. Inform. Sci. Tech. 54, 8, 713-729.
23. JAGADISH, H., NARAYAN, P., SESHADRI, S., SUDARSHAN, S., AND KANNEGANTI, R. 1997. Incremental organization for data recording and warehousing. In Proceedings of the International Conference on Very Large Databases (VLDB'97). Morgan Kaufmann, San Francisco, CA, 16-25.
24. KNUTH, D. 1973. The Art of Computer Programming, Vol. 3: Sorting and Searching, 2nd Ed. Addison-Wesley, Reading, MA.
25. LEHMAN, T. AND LINDSAY, B. 1989. The Starburst long field manager. In Proceedings of the International Conference on Very Large Data Bases (VLDB'89). 375-383.
26. LESTER, N., MOFFAT, A., AND ZOBEL, J. 2005. Fast online index construction via geometric partitioning. In Proceedings of the ACM CIKM International Conference on Information and Knowledge Management (CIKM'05). ACM, New York, 776-783.
27. LESTER, N., ZOBEL, J., AND WILLIAMS, H. 2005. Efficient online index maintenance for contiguous inverted lists. Inform. Process. Manag. 42, 4, 916-933.
28. MOFFAT, A. AND BELL, T. A. H. 1995. In situ generation of compressed inverted files. J. Amer. Soc. Inform. Sci. 46, 7, 537-550.
29. RAMAKRISHNAN, R. AND GEHRKE, J. 2003. Database Management Systems, 3rd Ed. McGraw Hill, New York, NY.
30. SCHOLER, F., WILLIAMS, H., YIANNIS, J., AND ZOBEL, J. 2002. Compression of inverted indexes for fast query evaluation. In Proceedings of the ACM-SIGIR International Conference on Research and Development in Information Retrieval (SIGIR'02). ACM, New York, 222-229.
31. SHIEH, W.-Y. AND CHUNG, C.-P. 2005. A statistics-based approach to incrementally update inverted files. Inform. Process. Manag. 41, 2, 275-288.
32. SHOENS, K., TOMASIC, A., AND GARCIA-MOLINA, H. 1994. Synthetic workload performance analysis of incremental updates. In Proceedings of the ACM-SIGIR International Conference on Research and Development in Information Retrieval (SIGIR'94). ACM, New York, 329-338.
33. TOMASIC, A., GARCIA-MOLINA, H., AND SHOENS, K. 1994. Incremental updates of inverted lists for text document retrieval. In Proceedings of the ACM-SIGMOD International Conference on the Management of Data (SIGMOD'94). ACM, New York, 289-300.
34. WILLIAMS, H. AND ZOBEL, J. 2005. Searchable words on the web. Int. J. Digital Libraries 5, 2, 99-105.
35. WITTEN, I., MOFFAT, A., AND BELL, T. C. 1999. Managing Gigabytes: Compressing and Indexing Documents and Images, 2nd Ed. Morgan Kaufmann, San Francisco, CA.
36. ZOBEL, J. AND MOFFAT, A. 2006. Inverted files for text search engines. ACM Comput. Surv. 38, 2, 1-56.