Mismatched codebooks and the role of entropy coding in lossy data compression

IEEE Transactions on Information Theory

Volumn 52, Issue 5, 2006, Pages 1922-1938

  Kontoyiannis, Ioannis ^a   Zamir, Ram ^b  

^a Brown Univerity   (United States)

^b TEL AVIV UNIVERSITY   (Israel)

Author keywords

Data compression; Large deviations; Mismatch; Pattern matching; Random coding; Rate distortion theory; Robustness; Universal Gaussian codebook; Universal quantization

Indexed keywords

ENCODING (SYMBOLS); PATTERN MATCHING; RANDOM PROCESSES; STATISTICAL METHODS;

ENTROPY CODING; LOSSY DATA COMPRESSION; MISMATCHED CODEBOOKS; RANDOM CODING; RATE-DISTORTION THEORY; UNIVERSAL QUANTIZATION;

DATA COMPRESSION;

EID: 33646036701     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2006.872845     Document Type: Article

References (37)

1. T. Berger, Rate Distortion Theory: A Mathematical Basis for Data Compression. Englewood Cliffs, NJ: Prentice-Hall, 1971.
2. P. Billingsley, Convergence of Probability Measures, 2nd ed. New York: Wiley, 1999.
3. J. A. Bucklew, "Two results on the asymptotic performance of quantizers," IEEE Trans. Inf. Theory, vol. IT-30, no. 2, pp. 341-348, Mar. 1984.
4. P. A. Chou, T. Lookabaugh, and R. M. Gray, "Entropy constrained vector quantization," IEEE Trans. Acoust., Speech Signal Process., vol. 37, no. 1, pp. 31-42, Jan. 1989.
5. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
6. I. Csiszár, "I-divergence geometry of probability distributions and minimization problems," Ann. Probab., vol. 3, pp. 146-158, 1975.
7. I. Csiszár, "Sanov property, generalized I-projection and a conditional limit theorem," Ann. Probab., vol. 12, no. 3, pp. 768-793, 1984.
8. A. Dembo and I. Kontoyiannis, "Source coding, large deviations, and approximate pattern matching," IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1590-1615, Jun. 2002.
9. A. Dembo and O. Zeitouni, Large Deviations Techniques and Applications, 2nd ed. New York: Springer-Verlag, 1998.
10. P. Elias, "Universal codeword sets and representations of the integers," IEEE Trans. Inf. Theory, vol. IT-21, no. 2, pp. 194-203, Mar. 1975.
11. A. Gersho, "Asymptotically optimal block quantization," IEEE Trans. Inf. Theory, vol. IT-25, no. 4, pp. 373-380, Jul. 1979.
12. H. Gish and N. J. Pierce, "Asymptotically efficient quantization," IEEE Trans. Inf. Theory, vol. IT-14, no. 5, pp. 676-683, Sep. 1968.
13. R. M. Gray and T. Linder, "Mismatch in high-rate entropy-constrained vector quantization," IEEE Trans. Inf. Theory, vol. 49, no. 5, pp. 1204-1218, May 2003.
14. M. Gutman, "On universal quantization with various distortion measures," IEEE Trans. Inf. Theory, vol. IT-33, no. 1, Jan. 1987.
15. A. Kanlis, "Compression and transmission of information at multiple resolutions," Ph.D. dissertation, Dep. Elec. Comput. Eng., Univ. Maryland, College Park, 1998.
16. A. Kanlis, P. Narayan, and B. Rimoldi, "On three topics for a course in information theory," in Statistical Methods in Imaging, Medicine, Optics, and Communication, J. A. O'Sullivan, Ed. New York: Springer-Verlag, 2001.
17. J. C. Kieffer, "Sample converses in source coding theory," IEEE Trans. Inf. Theory, vol. 37, no. 2, pp. 263-268, Mar. 1991.
18. I. Kontoyiannis, "Sphere-covering, measure concentration, and source coding," IEEE Trans. Inf. Theory, vol. 47, no. 4, pp. 1544-1552, May 2001.
19. I. Kontoyiannis and J. Zhang, "Arbitrary source models and Bayesian codebooks in rate-distortion theory," IEEE Trans. Inf. Theory, vol. 48, no. 8, pp. 2276-2290, Aug. 2002.
20. A. Lapidoth, "On the role of mismatch in rate distortion theory," IEEE Trans. Inf. Theory, vol. 43, no. 1, pp. 38-47, Jan. 1997.
21. T. Linder and R. Zamir, "On the asymptotic tightness of the Shannon lower bound," IEEE Trans. Inf. Theory, vol. 40, no. 6, pp. 2026-2031, Sep. 1994.
22. T. Lookabaugh and R. M. Gray, "High resolution quantization theory and the vector quantizer advantage," IEEE Trans. Inf. Theory, vol. 35, no. 5, pp. 1020-1033, Sep. 1989.
23. D. L. Neuhoff, "Source coding strategies: Simple quantizers vs. simple noiseless codes," in Proc. 1986 Conf. Information Sciences and Systems, vol. 1, Princeton, NJ, 1986, pp. 267-271.
24. J. T. Pinkston, "Encoding independent sample information sources," Ph.D. Dissertation, MIT, Cambridge, MA, 1967.
25. M. S. Pinsker, Information and Information Stability of Random Variables and Processes. San Francisco, CA: Holden-Day, 1964.
26. D. J. Sakrison, "The rate distortion function for a class of sources," Inf. Contr., vol. 15, pp. 165-195, 1969.
27. D. J. Sakrison, "The rate of a class of random processes," IEEE Trans. Inf. Theory, vol. IT-16, no. 1, pp. 10-16, Jan. 1970.
28. Y. Steinberg and M. Gutman, "An algorithm for source coding subject to a fidelity criterion, based upon string matching," IEEE Trans. Inf. Theory, vol. 39, no. 3, pp. 877-886, May 1993.
29. E.-h. Yang and J. C. Kieffer, "On the performance of data compression algorithms based upon string matching," IEEE Trans. Inf. Theory, vol. 44, no. 1, pp. 47-65, Jan. 1998.
30. E.-h. Yang and J. C. Kieffer, "Simple universal lossy data compression schemes derived from the Lempel-Ziv algorithm," IEEE Trans. Inf. Theory, vol. 42, no. 1, pp. 239-245, Jan. 1996.
31. R. Zamir, "The index entropy of a mismatched codebook," IEEE Trans. Inf. Theory, vol. 48, no. 2, pp. 523-528, Feb. 2002.
32. R. Zamir and M. Feder, "On universal quantization by randomized uniform/lattice quantizers," IEEE Trans. Inf. Theory, vol. 38, no. 2, pp. 428-436, Mar. 1992.
33. R. Zamir and M. Feder, "Information rates for pre/post filtered dithered quantizers," IEEE Trans. Inf. Theory, vol. 42, no. 5, pp. 1340-1353, Sep. 1996.
34. R. Zamir and K. Rose, "Natural type selection in adaptive lossy compression," IEEE Trans. Inf. Theory, vol. 47, no. 1, pp. 99-111, Jan. 2001.
35. R. Zamir, "The rate loss in the Wyner-Ziv problem," IEEE Trans. Inf. Theory, vol. 42, no. 6, pp. 2073-2084, Nov. 1996.
36. Z. Zhang and V. K. Wei, "An on-line universal lossy data compression algorithm by continuous codebook refinement - Part I: Basic results," IEEE Trans. Inf. Theory, vol. 42, no. 3, pp. 803-821, May 1996.
37. J. Ziv, "On universal quantization," IEEE Trans. Inf. Theory, vol. IT-31, no. 3, pp. 344-347, May 1985.