Mismatch in high-rate entropy-constrained vector quantization

IEEE Transactions on Information Theory

Volumn 49, Issue 5, 2003, Pages 1204-1217

  Gray, Robert M ^a   Linder, Tamás ^b  

^a Stanford University   (United States)

^b QUEEN S UNIVERSITY   (Canada)

Author keywords

Entropy constrained; High rate; Kullback Leibler divergence; Lagrangian; Mismatch; Quantization; Relative entropy; Variable rate

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; ASYMPTOTIC STABILITY; BOUNDARY CONDITIONS; CONSTRAINT THEORY; LAGRANGE MULTIPLIERS; PROBABILITY DENSITY FUNCTION; SET THEORY; SIGNAL DISTORTION; VECTOR QUANTIZATION;

GAUSSIAN DENSITIES; KULLBACK-LEIBLER DIVERGENCE; RELATIVE ENTROPY; SHANNON RATE DISTORTION;

SIGNAL ENCODING;

EID: 0037854459     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2003.810637     Document Type: Article

References (37)

1. A. Aiyer, "Robust image compression using gauss mixture models," Ph.D. dissertation, Dept. Elec. Eng., Stanford Univ., Stanford, CA, Aug. 2001.
2. J. A. Bucklew and G. L. Wise, "Multidimensional asymptotic quantization theory with rth power distortion measures," IEEE Trans. Inform. Theory, vol. IT-28, pp. 239-247, Mar. 1982.
3. J. A. Bucklew, "Two results on the asymptotic performance of quantizers," IEEE Trans. Inform. Theory, vol. IT-30, pp. 341-348, Mar. 1984.
4. P. A. Chou and B. Betts. When optimal entropy-constrained quantizers have only a finite number of codewords, presented at the IEEE Int. Symp. Information Theory, Cambridge, MA, Aug. 1998. [Online]. Available: http://research.microsoft.com/pachou/publications.htm
5. P. A. Chou, T. Lookabaugh, and R. M. Gray, "Entropy-constrained vector quantization," IEEE Trans. Acoust. Speech Signal Processing, vol. 37, pp. 31-42, Jan. 1989.
6. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
7. I. Csiszár, "Information-type measures of difference of probability distributions and indirect observations," Studia Scient. Math. Hung., vol. 2, pp. 299-318, 1967.
8. ____, "Generalized entropy and quantization problems," in Proc. 6th Prague Conf. Information Theory, Statistical Decision Functions, Random Processes, 1973, pp. 159-174.
9. ____, "On an extremum problem of information theory," Studia Scient. Math. Hung., pp. 57-70, 1974.
10. A. Dembo and I. Kontoyiannis, "Source coding, large deviations, and approximate string matching," IEEE Trans. Inform. Theory, vol. 48, pp. 1590-1615, Jun. 2002.
11. J. L. Doob, Measure Theory. New York: Springer-Verlag, 1994.
12. A. Gersho, "Asymptotically optimal block quantization," IEEE Trans. Inform. Theory., vol. IT-25, pp. 373-380, July 1979.
13. H. Gish and J. N. Pierce, "Asymptotically efficient quantizing," IEEE Trans. Inform. Theory, vol. IT-14, pp. 676-683, Sept. 1968.
14. S. Graf and H. Luschgy, Foundations of Quantization for Probability Distributions (Lecture Notes in Mathematics). Berlin, Germany: Springer-Verlag, 2000, vol. 1730.
15. R. M. Gray, Entropy and Information Theory. New York: Springer-Verlag, 1990.
16. ____, "Gauss mixture vector quantization," in Proc. 2001 IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), vol. 3, Salt Lake City, UT, May 2001, pp. 1769-1772.
17. ____, "Gauss mixture quantization: Clustering Gauss mixtures," in Nonlinear Estimation and Classification (Lecture Notes in Mathematics), D. D. Dennison, M. H. Hansen, C. Holmes, B. Mallick, and B. Yu, Eds. New York: Springer-Verlag, 2003.
18. R. M. Gray and D. L. Neuhoff, "Quantization," IEEE Trans. Inform. Theory, vol. 44, pp. 2325-2384, Oct. 1998.
19. R. M. Gray and J. Li, "On Zador's entropy-constrained quantization theorem," in Proc. Date Compression Conf. 2001. Los Alimitos, CA: IEEE Computer Soc. Press, Mar. 2001, pp. 3-12.
20. R. M. Gray, T. Linder, and J. Li, "A Lagrangian formulation of Zador's entropy-constrained quantization theorem," IEEE Trans. Inform. Theory, vol. 48, pp. 695-707, Mar. 2002.
21. R. M. Gray, A. H. Gray, Jr., G. Rebolledo, and J. E. Shore, "Rate distortion speech coding with a minimum discrimination information distortion measure," IEEE Trans. Information Theory, vol. IT-27, no. 6, pp. 708-721, Nov. 1981.
22. R. M. Gray, J. C. Young, and A. K. Aiyer, "Minimum discrimination information clustering: modeling and quantization with Gauss mixtures," in Proc. 2001 IEEE Int. Conf. Image Processing, vol. 3, Thessaloniki, Greece, Oct. 2001, pp. 14-17.
23. A. György, T. Linder, P. A. Chou, and B. J. Betts, When optimal entropy-constrained quantizers have a finite or infinite number of code-words. Preprint. [Online]. Available http://magenta.mast.queensu.ca/~linder/psfiles/GyLiChBe02.ps
24. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction (Springer Series in Statistics). New York: Springer-Verlag, 2001.
25. S. Kullback, Information Theory and Statistics. New York: Dover, 1968. Reprint of 1959 edition published by Wiley.
26. A. Lapidoth, "On the role of mismatch in rate distortion theory," IEEE Trans. Inform. Theory, vol. 43, pp. 38-47, Jan. 1997.
27. T. Linder, "Lagrangian empirical design of variable-rate vector quantizers: Consistency and convergence rates," IEEE Trans. Inform. Theory, vol. 48, pp. 2998-3003, Nov. 2002.
28. T. Linder and K. Zeger, "Asymptotic entropy constrained performance of tessellating and universal randomized lattice quantization," IEEE Trans. Inform. Theory, vol. 40, pp. 575-579, Mar. 1994.
29. M. S. Pinsker, Information and Information Stability of Random Variables and Processes. San Francisco, CA: Holden Day, 1964. (Translated by A. Feinstein from the Russian edition published in 1960 by Izd. Akad. Nauk. SSSR.).
30. D. J. Sakrison, "Worst sources and robust codes for difference distortion measures," IEEE Trans. Inform. Theory, vol. IT-21, pp. 301-309, May 1975.
31. A. N. Shiryaev, Probability, 2nd ed. New York: Springer-Verlag, 1996.
32. L. Vandenberghe, S. Boyd, and S.-P. Wu, "Determinant maximization with linear matrix inequality constraints," SIAM J. Matrix Anal. Applic., vol. 19, pp. 499-533, 1998.
33. E. H. Yang and J. Kieffer, "On the performance of data compression algorithms based upon string matching," IEEE Trans. Inform. Theory, vol. 44, pp. 47-65, Jan. 1998.
34. P. L. Zador, "Development and evaluation of procedures for quantizing multivariate distributions," Ph.D. dissertation, Stanford Univ., Stanford, CA, 1963, Also, Stanford Univ. Dept. Statistics Tech. Rep.
35. ____, "Topics in the asymptotic quantization of continuous random variables," report, Bell Labs. Tech. Memo, 1966.
36. ____, "Asymptotic quantization error of continuous signals and the quantization dimensions," IEEE Trans. Inform. Theory, vol. IT-28, pp. 139-148, Mar. 1982.
37. R. Zamir, "The index entropy of a mismatched codebook," IEEE Trans. Inform. Theory, vol. 48, pp. 523-528, Feb. 2002.