The Zero-Frequency Problem: Estimating the Probabilities of Novel Events in Adaptive Text Compression

IEEE Transactions on Information Theory

Volumn 37, Issue 4, 1991, Pages 1085-1094

  Witten, Ian H ^a   Bell, Timothy C ^b  

^a UNIVERSITY OF CALGARY   (Canada)

^b UNIVERSITY OF CANTERBURY   (New Zealand)

Author keywords

adaptive text compression; novel events; Poisson process; probability estimation; Zero frequency problem

Indexed keywords

CODES, SYMBOLIC; CONTROL SYSTEMS, ADAPTIVE; PROBABILITY;

POISSON PROCESSES; STATISTICAL CODING; TEXT COMPRESSION; ZERO-FREQUENCY PROBLEM;

INFORMATION THEORY;

EID: 0026187945     PISSN: 00189448     EISSN: 15579654     Source Type: Journal    
DOI: 10.1109/18.87000     Document Type: Article

References (23)

1. J. J. Rissanen and G. G. Langdon, “Universal modeling and coding,” IEEE Trans. Inform. Theory, vol. IT-27, no. 1, pp. 12–23, Jan. 1981.
2. R. Pasco, “Source coding algorithms for fast data compression,” Ph.D. thesis, Dept. Elect. Eng., Stanford Univ., Stanford, CA, 1976.
3. J. J. Rissanen, “Generalized Kraft inequality and arithmetic coding,” IBM J. Res. Dev., vol. 20, pp. 198–203, May 1976.
4. F. Rubin, “Arithmetic stream coding using fixed precision registers,” IEEE Trans. Inform. Theory, vol. IT-25, no. 6, pp. 672–675, Nov. 1979.
5. M. Guazzo, “A general minimum-redundancy source-coding algorithm,” IEEE Trans. Inform. Theory, vol. IT-26, no. 1, pp. 15–25, Jan. 1980.
6. J. J. Rissanen and G. G. Langdon, “Arithmetic coding,” IBM J. Res. Dev., vol. 23, no. 2, pp. 149–162, Mar. 1979.
7. I. H. Witten, R. Neal, and J. G. Cleary, “Arithmetic coding for data compression,” Comm-ACM, vol. 30, no. 6, pp. 520–540, June 1987.
8. J. Ziv and A. Lempel, “A universal algorithm for sequential data compression,” IEEE Trans. Inform. Theory, vol. IT-23, no. 3, pp. 337–343, May 1977.
9. ——, “Compression of individual sequences via variable-rate coding,” IEEE Trans. Inform. Theory, vol. IT-24, no. 5, pp. 530–536, Sept. 1978.
10. T. C. Bell, J. G. Cleary, and I. H, Witten, Text Compression. Englewood Cliffs, NJ: Prentice Hall, 1990.
11. M. G. Roberts, “Local order estimating Markovian analysis for noiseless source coding and authorship identification,” Ph.D. thesis, Stanford Univ., Stanford, CA, 1982.
12. C. S. Pierce, “The probability of induction,” in The World of Mathematics, Vol. 2, J. R. Newman, Ed. New York: Simon and Schuster, 1956, pp. 1341–1354.
13. I. J. Good, The Estimation of Probabilities, Research Monograph 30. Cambridge, MA: MIT Press, 1965.
14. I. J. Good and G. H. Toulmin, “The number of new species, and the increase in population coverage, when a sample is increased,” Biometrika, vol. 43, Pts. 1 and 2, pp. 45–63, June 1956.
15. B. Efron and R. Thisted, “Estimating the number of unseen species: How many words did Shakespeare know?” Biometrika, vol. 63, no. 3, pp. 435–447, 1976.
16. J. L. Bentley, D. D. Sleator, R. E. Tarjan, and V. K. Wei, “A locally adaptive data compression scheme,” Comm. ACM, vol. 29, no. 4, pp. 320–330, Apr. 1986.
17. J. G. Cleary and I. H. Witten, “Data compression using adaptive coding and partial string matching,” IEEE Trans. Commun., vol. COM-32, no. 4, pp. 396–402, Apr. 1984.
18. A. Moffat, “A note on the PPM data compression algorithm,” Res. Rep. 88/7, Dept. Comput. Sci., Univ. of Melbourne, Australia, 1988.
19. N. L. Johnson and K. Katz, Urn Models and Their Application. New York: Wiley, 1977.
20. H. Jeffreys, Theory of Probability. Oxford: Clarendon, 1939, p. 117ff.
21. J. G. Cleary and I. H. Witten, “A comparison of enumerative and adaptive codes,” IEEE Trans. Inform. Theory, vol. IT-30, no. 2, pp. 306–315, Mar. 1984.
22. R. A. Fisher, A. S. Corbet, and C. B. Williams, “The relation between the number of species and the number of individuals in a random sample of an animal population,” J. Animal Ecology, vol. 12, pp. 42–58, 1943.
23. G. Kolata, “Shakespeare's new poem: An ode to statistics,” Science, vol. 231, pp. 335–336, January 24, 1986.