A vector quantization approach to universal noiseless coding and quantization

IEEE Transactions on Information Theory

Volumn 42, Issue 4, 1996, Pages 1109-1138

  Chou, Philip A ^a,b   Effros, Michelle ^a,c   Gray, Robert M ^a,d  

^a IEEE

^b PALO ALTO RESEARCH CENTER   (United States)

^c California Institute of Technology   (United States)

^d Stanford University   (United States)

Author keywords

Adaptive; Clustering; Compression; Minimum description length; Two stage

Indexed keywords

ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS; FUNCTIONS; PROBLEM SOLVING; SIGNAL DISTORTION; SIGNAL ENCODING; VECTOR QUANTIZATION;

ASYMPTOTIC QUANTIZATION THEORY; GENERALIZED LLOYD ALGORITHM; IDENTIFIED CODE; RISSONEN THEOREM;

BLOCK CODES;

EID: 0030195609     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.508836     Document Type: Article

References (58)

1. B. D. Andrews, M. Effros, P. A. Chou, and R. M. Gray, "A mean-removed variation of weighted universal vector quantisation for image coding," in Proc. Data Compression Conf., (IEEE Computer Soc., Snowbird, UT, Mar. 1993), pp. 302-309.
2. V. Balasubramauian, "A geometric formulation of Occam's razor for inference of parametric distributions," IEEE Trans. Inform. Theory, 1995. submitted for publication.
3. A. R. Barron and T. M. Cover, "Minimum complexity density estimation," IEEE Trans. Inform. Theory, vol. 3, no 4. pp. 1034-1054, July 1991.
4. T. Berger. Rate Distortion Theory: A Mathematical Basis for Data Compression. Englewood Cliffs, NJ: Prentice-Hall, 1971.
5. J. A. Bucklew, "A note on the alsolute epsilon entropy," IEEE Trans. Inform. Theory, vol. 37. no. 1, pp. 142-144. Jan. 1991.
6. J. A. Bucklew and G. L. Wise, "Multidimensional asymptotic quantization theory with rth power distortion measures," IEEE Trans. Inform. Theory, vol. IT-28, no. 2, pp. 239-247, Mar. 1982.
7. A. Buzo, A. H. Gray, Jr., R. M. Gray, and J .D. Markel," Speech coding based upon vector quantization," IEEE Trans. Acoust. Speech Signal Processing, vol. ASSP-28, pp. 562-574. Oct. 1980.
8. W.-Y. Chan and A. Gersho, "Constrained-storage quantization of multiple vector sources by codebook sharing." IEEE Trans. Commun., vol. 39, no. 1, pp. 11-13, Jan. 1991.
9. P. A. Chou, "Code clustering for weighted universal VQ and other applications," in Proc. IEEE Int. Symp. on Information Theory, (Budapest, Hungary, June 1991), p. 253.
10. _, "Optimal partitioning for classification and regression trees," IEEE Trans. Pal. Anal. Machine Intell, vol. 13, no. 4, pp. 340-354, Apr. 1991.
11. P. A. Chou and M. Effros, "Rate and distortion redundancies for source coding with respect to a fidelity criterion," in Proc. IEEE Int. Symp. on Information Theory (San Antonio. TX, Jan. 1993).
12. P. A. Chou. T. Lookahaugh, and R. M. Gray, "Entropy-constrained vector quantization," IEEE Trans. Acoust. Speech Signal Processing, vol. 37, no. 1, pp. 31-42, Jan. 1989.
13. B. S. Clarke and A. R. Barron, "Information-theoretic asymptotics of Bayes methods," IEEE Trans. Inform. Theory, vol. 36, no. 3. pp. 453-471, May 1990.
14. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
15. L. D. Davisson, "Universal noiseless coding," IEEE Trans. Inform. Theory, vol. IT-19, no. 6, pp. 783-795, Nov. 1973.
16. _, "Minimax noiseless universal coding for Markov sources." IEEE Trans. Inform. Theory, vol. IT-29, no. 2, pp. 211-215, Mar. 1983.
17. E. J. Delp and O. R. Mitchell, "Image compression using block truncation coding." IEEE Trans. Commun., vol. COM-27, no. 9, pp. 1335-1342, 1979.
18. M. Effros and P. A. Chou, "Weighted universal bit allocation," in Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (Detroit, MI, May 1995), vol. 4, pp. 2343-2346.
19. _, "Weighted universal transform coding: universal image compression with the Karhounen Loéve transform," in Proc. IEEE Int. Conf. on Image Processing (Washington, DC, Oct. 1995), to appear.
20. M. Effros, P. A. Chou, and R. M. Gray, "Rates of convergence in adaptive universal vector quantization," in Proc. IEEE Int. Symp. on Information Theory (Trondheim, Norway, June 1994).
21. _, "Variable, dimension weighted universal vector quantization and noiseless coding," in Proc. IEEE Data Compression Conf., (Snowbird, UT, Mar. 1994), pp. 2-11.
22. _, "Variable-rate source coding theorems for stationary nonergodic sources," IEEE Trans. Inform. Theory, vol. 40, no. 6, pp. 1920-1925, Nov. 1994.
23. R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.
24. A. Gersho, "Asymptotically optimal block quantization," IEEE Trans. Inform. Theory, vol. IT-25, no. 4, pp. 373-380, July 1979.
25. R. M. Gray, Entropy und Information Theory. New York: Springer-Verlag, 1990.
26. R. M. Gray and E. D. Davisson. "Source coding theorems without the ergodic assumption," IEEE Trans. Inform. Theory, vol. IT-20, pp. 502-526, July 1974.
27. E. Györfi, I. Pàli, and E. C. van der Meulen, "There is no universal source code for an infinite source alphabet," IEEE Trans. Inform. Theory, vol. 40, no. 1. pp. 267-271, Jan. 1994.
28. D. Haussler. "Decision theoretic generalizations of the PAC model for neural net and other learning applications," Inform. Comput., vol. 100, no. 1, pp. 78-150, Sept. 1992.
29. |29] K. Jacobs, "The ergodic decomposition of the, Kolmogorov-Sinai invariant," in F. B. Wright and F. B. Wright, Eds., Ergodic Theory. New York: Academic Press, 1963.
30. J. C. Kieffer, "On the optimum average distortion attainable by a fixed rate coding of a nonergodic source," IEEE Trans. Inform. Theory, vol. IT-21, no. 2, pp. 190-193, Mar. 1975.
31. _, "A unified approach to weak universal source coding," IEEE Trans. Inform. Theory, vol. IT-24, no. 6, pp. 674-682, Nov. 1978.
32. _, "On the minimum rate for strong universal block coding of a class of ergodic sources," IEEE Trans. Inform. Theory, vol. IT-26, no. 6, pp. 693-702, Nov. 1980.
33. R. F. Krichevsky and V. K. Trofimov. "The performance of universal encoding," IEEE Trans. Inform. Theory, vol. IT-27, no. 2, pp. 199-207, Mar. 1981.
34. S. Kullback, Information Theory and Statistics. New York: Wiley, 1959; republished: New York: Dover, 1968.
35. Y. Linde, A. Buzo, and K. M. Gray. "An algorithm for vector quantizer design," IEEE Trans. Commun., vol. COM-28, pp. 84-95, Jan. 1980.
36. T. Einder, G. Eugosi, and K. Zeger, "Rates of convergence in the source coding theorem, in empirical quantizer design, and in universal lossy source coding," IEEE Trans. Inform. Theory, vol. 40, no. 6, pp. 1728-1740, Nov. 1994.
37. _, "Fixed rate universal lossy source coding and rates of convergence for memory less sources," IEEE Trans. Inform. Theory, vol. 41, no. 3, pp. 665-676, May 1995.
38. T. D. Lookabaugh and R. M. Gray, "High resolution quantization theory and the vector quantization advantage," IEEE Trans. Inform. Theory, vol. 35, no. 5, pp. 1020-1033, Sept. 1989.
39. D. L. Neuhoff, R. M. Gray, and L. D. Davisson, "Fixed rate universal block source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-21, no. 5, pp. 511-523, Sept. 1975.
40. R. J. Pile, "Coding theorems for discrete source-channel pairs," Ph.D. dissertation, MIT, Cambridge, MA, 1967.
41. _, "The transmission distortion of a source as a function of the encoding block length," Bell Syst. Tech. J., vol. 47, pp. 827-885, 1968.
42. E. Posner, E. Rodemich, and H. Rumsey, "Epsilon entropy of stochastic processes," Ann. Math. Stal., vol. 38, pp. 1000-1020, 1967.
43. M. Rabbani and P. W. Jones, Digital Image Compression Techniques. Bellingham, WA: SPIF, 1991.
44. L. R. Rabiner. J. G. Wilpon, and B.-H. Juang, "A segmental K-means training procedure for connected word recognition," AT&T Tech. J., vol. 64. no. 3, pp. 21-40, May 1986.
45. R. F. Rice and J. R. Plaunt. 'The Rice Machine: Television data compression," Tech. Rep. 900-408, Jet Propulsion Lab., Pasadena, CA, Sept. 1970.
46. _, "Adaptive variable-length coding for efficient compression of spacecraft television data," IEEE Trans. Commun., vol. COM-19, no. 12, pp. 889-897, Dec. 1971.
47. J. Rissanen, "Universal coding, information, prediction, and estimation," IEEE Trans. Inform. Theory, vol. IT-30, no. 4, pp. 629-636, July 1984.
48. _, "Stochastic complexity and modeling," Ann. Stat., vol. 14, pp. 1080-1100, Sepl. 1986.
49. _, "Fisher information and stochastic complexity," IEEE Trans. Inform. Theory, vol. 42, no. 1, pp. 40-47, Jan. 1996.
50. H. L. Royden, Real Analysis, 2nd ed. New York: Macmillan, 1968.
51. R. J. Safranek and J. D. Johnston, "A perceptually tuned sub-band image coder with image dependent quantization and posl-quantization data compression," in Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (Glasgow, Scotland, May 1989), pp. 1945-1948.
52. A. V. Trushkin, "Sufficient conditions for uniqueness of a locally optimal quantizer for a class of convex error weighting functions," IEEE Trans. Inform. Theory, vol. IT-28, no. 2, pp. 187-198, Mar. 1982.
53. B. Yu and T. P. Speed, "A rate of convergence result for a universal d-semifaithful code," IEEE Trans. Inform. Theory, vol. 39, no. 3, pp. 813-820, May 1993.
54. R- Zamir and M. Feder, "On lattice quantization noise," in Proc. IEEE Int. Symp. on Information Theory (Trondheim, Norway, June 1994).
55. K. Zeger and A. Bist, "Universal adaptive vector quantization using codebook quantization," in Proc. IEEE Int. Conf. on Acoustics. Speech. and Signal Processing (San Francisco, CA, Mar. 1992), pp. III.381-384.
56. K. Zeger, A. Bist, and T. Linder, "Universal source coding with codebook transmission." IEEE Trans. Commun., vol. 42, pp. 336-346, Feb. 1994.
57. Z. Zhang and V. K. Wei, "An on-line universal lossy data compression algorithm via continuous codebook refinement - Pt. I: Basic results," IEEE Trans. Inform. Theory, vol. 42, pp. 803-821, May 1996.
58. Z. Zhang, E.-H. Yang, and V.K. Wei, "The redundancy of source coding with a fidelity criterion," IEEE Trans. Inform. Theory, 1994, submitted.