Rates of Convergence in the Source Coding Theorem, in Empirical Quantizer Design, and in Universal Lossy Source Coding

IEEE Transactions on Information Theory

Volumn 40, Issue 6, 1994, Pages 1728-1740

  Zeger, Kenneth ^b  

^a BUDAPEST UNIVERSITY OF TECHNOLOGY AND ECONOMICS   (Hungary)

^b UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

Convergence rates; Empirical vector quantizer design; Large deviation theory; Universal lossy source coding

Indexed keywords

DESCRIBING FUNCTIONS; SIGNAL DISTORTION; SIGNAL ENCODING; VECTORS;

CONVERGENCE RATES; LARGE DEVIATION THEORY; MEAN SQUARE ERROR; UNIVERSAL LOSSY SOURCE CODING;

VECTOR QUANTIZATION;

EID: 0028548396     PISSN: 00189448     EISSN: 15579654     Source Type: Journal    
DOI: 10.1109/18.340451     Document Type: Article

References (30)

1. E. A. Abaya and G. L. Wise, “Convergence of vector quantizers with applications to optimal quantization,” SIAM J. Appl. Math., vol. 44, pp. 183-189, 1984.
2. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. Boston, Kluwer, 1992.
3. D. Pollard, “Quantization and the method of k-means,” IEEE Trans. Inform. Theory, vol. IT-28, pp. 199-205, Mar. 1982.
4. M. J. Sabin and R. M. Gray, “Global convergence and empirical consistency of the generalized Lloyd algorithm,” IEEE Trans. Inform. Theory, vol. IT-32, pp. 148-155, March 1986.
5. P. C. Cosman, K. O. Perlmutter, S. M. Perlmutter, R. A. Olshen, and R. M. Gray, “Training sequence size and vector quantizer performance,” in Proc. Asilomar Conf. Signals, Syst. Comput., pp. 434-438, 1991.
6. D. Cohn, E. Riskin, and R. Ladner, “Theory and practice of vector quantizers trained on small training sets,” IEEE Trans. Pattern Anal. Machine Intell., vol. 16, pp. 54-65, Jan. 1994.
7. D. Pollard, “Strong consistency of k-means clustering,” Ann. Stat., vol. 9, no. 1, pp. 135-140, 1981.
8. M. J. Sabin, “Global convergence and empirical consistency of the generalized Lloyd algorithm,” Ph.D. thesis, Stanford Unv., 1984.
9. R. Pile, “The transmission distortion of a source as a function of the encoding block length,” Bell Syst. Tech. J., vol. 47, pp. 827-885, 1968.
10. J. Ziv, “Coding of sources with unknown statistics—Part II: Distortion relative to a fidelity criterion,” IEEE Trans. Inform. Theory, vol. IT-18, pp. 389-394, May 1972.
11. K. Zeger, A. Bist, and T. Linder, “Universal source coding with codebook transmission,” IEEE Trans. Commun., vol. 42, pp. 336-346, Feb. 1994
12. P. A. Chou and M. Effros, “Rate and distortion redundancies for universal source coding with respect to a fidelty criterion,” summary in IEEE Int. Symp. Inform. Theory, San Antonio, TX, 1992.
13. D. L. Neuhoff, R. M. Gray, and L. D. Davisson, “Fixed rate universal block source coding with a fidelity criterion,” IEEE Trans. Inform. Theory, vol. IT-21, pp. 511-523, Sept. 1975.
14. M. B. Pursley and L. D. Davisson, “Variable rate coding for nonergodic sources and classes of ergodic sources subject to a fidelity constraint,” IEEE Trans. Inform. Theory, vol. IT-22, pp. 324-337, May 1976.
15. B. Yu and T. P. Speed, “A rate of convergence result for a universal D-semifaithful code,” IEEE Trans. Inform. Theory, vol. 39, pp. 813-821, May 1993.
16. Y. Linde, A. Buzo, and R. M. Gray, “An algorithm for vector quantizer design,” IEEE Trans. Commun., vol. COM-28, pp. 84-95, Jan. 1980.
17. W. Hoeffding, Probability inequalities for sums of bounded random variables,” J. Am. Stat. Assoc., vol. 58, pp. 13-30, 1963.
18. V. N. Vapnik and A. Ya. Chervonenkis, “On the uniform convergence of relative frequencies of events to their probabilities,” Theory Probab. Appl., vol. 16, pp. 264-280, 1971.
19. L. Devroye, “The uniform convergence of the Narayada-Watson regression function estimate,” Can. J. Stat., vol. 6, pp. 179-191, 1978.
20. R. M. Dudley, “Balls in Rkdo not cut all subsets of k + 2 points,” Adv. Math., vol. 31, no. 3, pp. 306-308, 1979.
21. K. Alexander, “Probability inequalities for empirical processes and a law of the iterated logarithm,” Ann. Probab., vol. 4, pp. 1041-1067, 1984.
22. L. Devroye, “Automatic pattern recognition: A study of the probability of error,” IEEE Trans. Pattern Anal. Machine Intell., vol. 10, pp. 530-543, 1988.
23. A. Nobel and R. Olshen, personal communication.
24. D. J. Sakrison, “A geometric treatment of the source encoding of a Gaussian random variable,” IEEE Trans. Inform. Theory, vol. IT-14, pp. 481-486, May 1968.
25. A. D. Wyner, “Communication of analog data from a Gaussian source over a noisy channel,” Bell Syst. Tech. J., pp. 801-812, May-June 1968.
26. A. D. Wyner, “On the transmission of correlated Gaussian data over a noisy channel with finite encoding block length,” Inf. Contr., vol. 20, pp. 193-215, 1972.
27. R. G. Gallager, Information Theory and Reliable Communication. New York, Wiley, 1968.
28. T. Berger, Rate Distortion Theory. Englewood Cliffs, NJ. Prentice-Hall, 1971.
29. I. Csiszar, “On an extremum problem of information theory,” Stud. Sci. Math. Hung., pp. 57-70, 1974.
30. T. Linder, G. Lugosi, and K. Zeger, “Fixed rate universal lossy source coding and rate of convergence for memoryless sources,” IEEE Trans. Inform. Theory, to appear.