Modeling word occurrences for the compression of concordances

ACM Transactions on Information Systems

Volumn 15, Issue 3, 1997, Pages 254-290

  Bookstein, A ^a   Klein, S T ^b   Raita, T ^c  

^a UNIVERSITY OF CHICAGO   (United States)

^b BAR ILAN UNIVERSITY   (Israel)

^c UNIVERSITY OF TURKU   (Finland)

Author keywords

E.2 Data : Data Storage Representations composite structures; E.4 Data : Coding and Information Theory data compaction and compression; F.1.2 Computation by Abstract Devices : Modes of Computation probabilistic computation; Markov models

Indexed keywords

APPROXIMATION THEORY; CLASSIFICATION (OF INFORMATION); COMPUTATIONAL METHODS; DATA COMPRESSION; DATA STORAGE EQUIPMENT; DATA STRUCTURES; GRAPH THEORY; MARKOV PROCESSES; MATHEMATICAL MODELS;

FULL TEXT RETRIEVAL SYSTEMS; HIDDEN MARKOV MODELS (HMM);

INFORMATION RETRIEVAL SYSTEMS;

EID: 0031176241     PISSN: 10468188     EISSN: None     Source Type: Journal    
DOI: 10.1145/256163.256166     Document Type: Article

References (26)

1. ASAI, K., HAYAMIZU, S., AND ONIZUKA, K. 1993. HMM with protein structure grammar. In Proceedings of the 26th Hawaii International Conference on System Sciences. IEEE, New York, 783-791.
2. BELL, T. C., WITTEN, I. H., AND CLEARY, J. G. 1989. Modeling for text compression. ACM Comput. Surv. 21, 557-591.
3. BOOKSTEIN, A., KLEIN, S. T., AND RAITA, T. 1992a. Model based concordance compression. In Proceedings of the Data Compression Conference DCC-92. IEEE Computer Society, Washington, D.C., 82-91.
4. BOOKSTEIN, A., KLEIN, S. T., AND RAITA, T. 1994. Markov models for clusters in concordance compression, In Proceedings of the Data Compression Conference DCC-94. IEEE Computer Society, Washington, D.C., 116-125.
5. BOOKSTEIN, A., KLEIN, S. T., AND ZIFF, D. A. 1992b. A systematic approach to compressing a full text retrieval system. Inf. Process. Manage. 28, 795-806.
6. CHOUEKA, Y., FRAENKEL, A. S., AND KLEIN, S. T. 1988. Compression of concordances in full-text retrieval systems. In Proceedings of the 11th ACM-SIGIR Conference. ACM, New York, 597-612.
7. CHOUEKA, Y., FRAENKEL, A. S., KLEIN, S. T., AND SEGAL, E. 1986. Improved hierarchical bit-vector compression in document retrieval systems. In Proceedings of the 9th ACM-SIGIR Conference. ACM, New York, 88-97.
8. CORMACK, G. V. AND HORSPOOL, R. N. 1987. Data compression using dynamic Markov modeling. Comput. J. 30, 541-550.
9. ELLAS, P. 1975. Universal codeword sets and representation of the integers. IEEE Trans. Inf. Theory IT-12, 194-203.
10. EVEN, S. 1973. Algorithmic Combinatorics. The Macmillan Company, New York.
11. FELLER, W. 1968. An Introduction to Probability Theory and Its Applications. Wiley, New York.
12. FIALA, E. R. AND GREENE, D. H. 1989. Data compression with finite windows. Commun. ACM 32, 490-505.
13. FRAENKEL, A. S. 1976. All about the Responsa Retrieval Project you always wanted to know but were afraid to ask, expanded summary. Jurimetrics J. 16, 149-156.
14. HAUSSLER, D., KROGH, A., MIAN, S., AND SJÖLANDER, K. 1993. Protein modeling using hidden Markov models: Analysis of globins. In Proceedings of the 26th Hawaii International Conference on System Sciences. IEEE Computer Society, Washington, D.C., 792-802.
15. KEMENY, J. G. AND SNELL, J. L. 1960. Finite Markov Chains. Van Nostrand, New York.
16. KLEIN, S. T., BOOKSTEIN, A., AND DEERWESTER, S. 1989. Storing text retrieval systems on CD-ROM: Compression and encryption considerations. ACM Trans. Inf. Syst. 7, 230-245.
17. KNUTH, D. E. 1973. The Art of Computer Programming. Vol 1, Fundamental Algorithms. Addison-Wesley, Reading, Mass.
18. LINOFF, G. AND STANFILL, C. 1993. Compression of indexes with full positional information in very large text databases. In Proceedings of the 16th ACM-SIGIR Conference. ACM New York, 88-95.
19. LLEWELLYN, J. A. 1987. Data compression for a source with Markov characteristics. Comput. J. 30, 149-156.
20. MITTENDORF, E. AND SCHÄUBLE, P. 1994. Document and passage retrieval based on Hidden Markov models. In Proceedings of the 17th ACM-SIGIR Conference. ACM, New York, 318-327.
21. MOFFAT, A. AND ZOBEL, J. 1992. Coding for compression in full-text retrieval systems. In Proceedings of Data Compression Conference DCC-92. IEEE Computer Society, Washington, D.C., 72-81.
22. RABINER, L. R. 1989. A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77, 2 (Feb.) 257-286.
23. TEUHOLA, J. 1978. A compression method for clustered bit-vectors. Inf. Proc. Lett. 7, 308-311.
24. WISNIEWSKI, J. L. 1986. Compression of index term dictionary in an inverted file oriented database: Some efficient algorithms. Inf. Process. Manage. 22, 493-501.
25. WITTEN, I. H., BELL, T. C., AND NEVILL, C. G. 1991. Models for compression in full-text retrieval systems. In Proceedings of Data Compression Conference DCC-91. IEEE Computer Society, Washington, D.C., 23-32.
26. WITTEN, I. H., MOFFAT, A., AND BELL, T. C. 1994. Managing Gigabytes, Compressing and Indexing Documents and Images. International Thomson Publishing, London.