Self-Indexing Inverted Files for Fast Text Retrieval

ACM Transactions on Information Systems

Volumn 14, Issue 4, 1996, Pages 349-379

  Moffat, Alistair ^a   Zobel, Justin ^b  

^a UNIVERSITY OF MELBOURNE   (Australia)

^b RMIT UNIVERSITY   (Australia)

Author keywords

E.4 Data : Coding and Information Theory data compaction and compression; H.3.1 Information Storage and Retrieval : Content Analysis and Indexing indexing methods; H.3.2 Information Storage and Retrieval : Information Storage file organization

Indexed keywords

BOOLEAN QUERIES; FULL TEXT RETRIEVAL; INDEX COMPRESSION; INVERTED FILE; QUERY PROCESSING; SELF INDEXING;

DATA COMPRESSION; DATA STORAGE EQUIPMENT; FILE ORGANIZATION; INFORMATION RETRIEVAL; INFORMATION RETRIEVAL SYSTEMS; QUERY LANGUAGES;

INDEXING (OF INFORMATION);

EID: 0030260359     PISSN: 10468188     EISSN: None     Source Type: Journal    
DOI: 10.1145/237496.237497     Document Type: Article

References (35)

1. BELL, T. C., MOFFAT, A., NEVILL-MANNING, C. G., WITTEN, I. H., AND ZOBEL, J. 1993. Data compression in full-text retrieval systems. J. Am. Soc. Inf. Sci. 44, 9 (Oct.), 508-531.
2. BOOKSTEIN, A., KLEIN, S. T., AND RAITA, T. 1992. Model based concordance compression. In Proceedings of the IEEE Data Compression Conference, J. A. Storer and M. Cohn, Eds. IEEE Computer Society Press, Los Alamitos, Calif., 82-91.
3. BUCKLEY, C. AND LEWIT, A. F. 1985. Optimization of inverted vector searches. In Proceedings of the ACM-SIGIR International Conference on Research and Development in Information Retrieval (Montreal, Canada, June). ACM Press, New York, 97-110.
4. CHOUEKA, Y., FRAENKEL, A. S., KLEIN, S. T., AND SEGAL, E. 1987. Improved techniques for processing queries in full-text systems. In Proceedings of the ACM-SIGIR International Conference on Research and Development in Information Retrieval (New Orleans, La.). ACM Press, New York, 306-315.
5. CHOUEKA, Y., FRAENKEL, A. S., AND KLEIN, S. T. 1988. Compression of concordances in full-text retrieval systems. In Proceedings of the ACM-SIGIR International Conference on Research and Development in Information Retrieval (Grenoble, France, June). ACM Press, New York, 597-612.
6. CORMEN, T. H., LEISERSON, C. E., AND RIVEST, R. L. 1990. Introduction to Algorithms. The MIT Press, Cambridge, Mass.
7. CROFT, W. B. AND SAVINO, P. 1988. Implementing ranking strategies using text signatures. ACM Trans. Off. Inf. Syst. 6, 1 (Jan.), 42-62.
8. ELIAS, P. 1975. Universal codeword sets and representations of the integers. IEEE Trans. Inf. Theory IT-21, 2 (Mar.), 194-203.
9. ELMASRE, R. AND NAVATHE, S. B. 1989. Fundamentals of Database Systems. Benjamin/ Cummins, Menlo Park, Calif.
10. FALOUTSOS, C. 1985a. Access methods for text. ACM Comput. Surv. 17, 1, 49-74.
11. FALOUTSOS, C. 1985b. Signature files: Design and performance comparison of some signature extraction methods. In Proceedings of the ACM-SIGIR International Conference on Research and Development in Information Retrieval (Montreal, Canada, June). ACM Press, New York, 63-82.
12. FALOUTSOS, C. AND CHRISTODOULAKIS, S. 1984. Signature files: An access method for documents and its analytical performance evaluation. ACM Trans. Off. Inf. Syst. 2, 4 (Oct.), 267-288.
13. FOX, E. A., HARMAN, D. K., BAEZA-YATES, R., AND LEE, W. C. 1992. Inverted files. In Information Retrieval: Data Structures and Algorithms, W. B. Frakes and R. Baeza-Yates, Eds. Prentice-Hall, Englewood Cliffs, N.J., 28-43.
14. FRAENKEL, A. S. AND KLEIN, S. T. 1985. Novel compression of sparse bit-strings - Preliminary report. In Combinatorial Algorithms on Words, A. Apostolico and Z. Galil, Eds. NATO ASI Series F, vol. 12. Springer-Verlag, Berlin, 169-183.
15. GALLAGER, R. G. AND VAN VOORHIS, D. C. 1975. Optimal source codes for geometrically distributed integer alphabets. IEEE Trans. Inf. Theory IT-21, 2 (Mar.), 228-230.
16. GOLOMB, S. W. 1966. Run-length encodings. IEEE Trans. Inf. Theory IT-12, 3 (July), 399-401.
17. HARMAN, D. K., Ed. 1992a. Proceedings of TREC Text Retrieval Conference (Washington, D.C., Nov.). National Institute of Standards Special Publication 500-207. NIST, Washington, D.C.
18. HARMAN, D. K. 1992b. Ranking algorithms. In Information Retrieval: Data Structures and Algorithms, W. B. Frakes and R. Baeza-Yates, Eds. Prentice-Hall, Englewood Cliffs, N.J., 363-392.
19. HARMAN, D. K. AND CANDELA, G. 1990. Retrieving records from a gigabyte of text on a mini-computer using statistical ranking. J. Am. Soc. Inf. Sci. 41, 8, 581-589.
20. HWANG, F. K. AND LIN, S. 1972. A simple algorithm for merging two disjoint linearly ordered sets. SIAM J. Comput. 1, 1, 31-39.
21. KENT, A. J., SACKS-DAVIS, R., AND RAMAMOHANARAO, K. 1990. A signature file scheme based on multiple organizations for indexing very large text databases. J. Am. Soc. Inf. Sci. 41, 7, 508-534.
22. KLEIN, S. T., BOOKSTEIN, A., AND DEERWESTER, S. 1989. Storing text retrieval systems on CD-ROM: Compression and encryption considerations. ACM Trans. Inf. Syst. 7, 3 (July), 230-245.
23. LINOFF, G. AND STANFILL, C. 1993. Compression of indexes with full positional information in very large text databases. In Proceedings of the ACM-SIGIR International Conference on Research and Development in Information Retrieval, R. Korfhage, E. Rasmussen, and P. Willett, Eds. ACM Press, New York, 88-97.
24. LOVINS, J. B. 1968. Development of a stemming algorithm. Mech. Transl. Comput. 11, 1-2, 22-31.
25. LUCARELLA, D. 1988. A document retrieval system based upon nearest neighbour searching. J. Inf. Sci. 14, 25-33.
26. MCDONELL, K. J. 1977. An inverted index implementation. Comput. J. 20, 1, 116-123.
27. MOFFAT, A. AND ZOBEL, J. 1992. Parameterised compression for sparse bitmaps. In Proceedings of the ACM-SIGIR International Conference on Research and Development in Information Retrieval, N. Belkin, P. Ingwersen, and A. M. Pejtersen, Eds. ACM Press, New York, 274-285.
28. MOFFAT, A., ZOBEL, J., AND KLEIN, S. T. 1995. Improved inverted file processing for large text databases. In Proceedings of the 6th Australasian Database Conference, R. Sacks-Davis and J. Zobel, Eds. 162-171.
29. MOFFAT, A., ZOBEL, J., AND SACKS-DAVIS, R. 1994. Memory efficient ranking. Inf. Process. Manag. 30, 6, 733-744.
30. SACKS-DAVIS, R., KENT, A. J., AND RAMAMOHANARAO, K. 1987. Multi-key access methods based on superimposed coding techniques. ACM Trans. Database Syst. 12, 4 (Dec.), 655-696.
31. SALTON, G. 1989. Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley, Reading, Mass.
32. SALTON, G. AND MCGILL, M. J. 1983. Introduction to Modern Information Retrieval. McGraw-Hill, New York.
33. WITTEN, I. H., MOFFAT, A., AND BELL, T. C. 1994. Managing Gigabytes: Compressing and Indexing Documents and Images. Van Nostrand Reinhold, New York.
34. ZOBEL, J., MOFFAT, A., AND SACKS-DAVIS, R. 1992. An efficient indexing technique for full-text database systems. In Proceedings of the International Conference on Very Large Databases, L.-Y. Yuan, Ed. VLDB Endowment Press, Saratoga, Calif., 352-362.
35. ZOBEL, J., MOFFAT, A., WILKINSON, R., AND SACKS-DAVIS, R. 1995. Efficient retrieval of partial documents. Inf. Process. Manag. 31, 3, 361-377.