An on-line universal lossy data compression algorithm via continuous codebook refinement-Part I: Basic results

IEEE Transactions on Information Theory

Volumn 42, Issue 3, 1996, Pages 803-821

  Zhang, Zhen ^a   Wei, Victor K ^b  

^a University of Southern California ^*  (United States)

^b CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Adaptive vector quantization; Lossy data compression; Rate distortion function; Source coding with a fidelity criterion; Universal source coding

Indexed keywords

DATA COMPRESSION; DECODING; ENCODING (SYMBOLS); MATHEMATICAL MODELS; PROBABILITY; VECTOR QUANTIZATION;

CODEBOOK TRANSMISSION; HUFFMAN ALGORITHM; LEMPEL-ZIV ALGORITHM; LOSSY DATA COMPRESSION; RATE DISTORTION FUNCTION; UNIVERSAL SOURCE CODING;

ALGORITHMS;

EID: 0030151231     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.490546     Document Type: Article

References (92)

1. N. Abiainson, Information Theoiy and Coding. New York: McGrawHill, 1963, ch. 3. pp. 61-62, Note 1.
2. H. Abu, Vector Quantization. New York: IEEE Press, 1990.
3. R. Ahlswede, "Coloring hypergraphs: A new approach to multi-user source coding I." J. Comb. Inform. Syst. Sci., vol. 4, pp. 76-115, 1979.
4. _, "Coloring hypergraphs: A new approach to multi-user source coding II," J. Comb. Inform. Syst. Sci., vol. 4, pp. 76-115, 1979.
5. V, Anantharam, "A large deviation approach to error exponents in source coding and hypothesis testing." IEEE Trans. Inform. Theory, vol. 36, pp. 938-943, 1990.
6. M. Antonini, M. Barlaud, P. Mathiieu, and I. Daubediies, "linage coding using wavelet transform," 1F.F.F. Tmns. Imag? Processing, vol. 1, pp. 205-220, 1992.
7. M.F. Barnsley and L. P. Hurd, Fractal Image Compression. Wellesley, MA: Perters, 1993.
8. L.A. Bassalygo and R. L. Dobrushin, "e-entropy and random field," Probl. Pered. Inform., vol. 23, pp. 3-15, 1987.
9. T.C. Bell, J. G. Cleary, and I. II. Witten, Text Compression. Englewood Cliffs, NJ: Prentice-Hall, 1990.
10. J. Bentley, D. Sleator. R. Tarjan, and V. K Wei, "A locally adaptive data compression scheme." Commun. ACM, vol. 29, pp. 320-330, 1986.
11. T. Berger, Rate Distortion Theory: A Mathematical Basis for Data Compression. Englewood Cliffs, NJ: Prentice-Hail, 1971.
12. _, "Rate distortion theory for sources with abstract alphabet and memory," Inform. Contr., vol. 13, pp. 254-273, 1968.
13. _, private communication.
14. T. Berger and Z. Zhang. "Minimum breakdown degradation in binary source encoding," IEEE Trans. Inform. Theory, vol. IT-29, pp. 807-814, 1983.
15. O. T.-f. Chen, B. J. Sheu, and 7. Zhang. ''An adaptive quantizer based on the gold-washing method," submitted to JEEE Trans. Circuits Syst., Apr., 1994.
16. W.-S. Chcn and Z. Zhang, "An adaptive quadtree variable dimensional vector quantization scheme based on gold washing algorithm," Tech. Rep.. CSI, Dept. Elec. Eng.-Syst., LISC, I.os Angeles, 1993.
17. O. T.-C. Chen, Z. Zhang, and B. J. Sheu, "An adaptive high speed lossy data compression," in Proc., Data Compression Con]. '92, J. A. Storer and M Cohn, Eds. IEEE Computer Soc. Press, pp. 349-358.
18. K. Cheung and V. K. Wei, "A locally adaptive source coding scheme." in Communication, Control, and Signal Processing, Proc. Bilkent Conf. on New Trends in Communication, Control, and Signal Processing, 1990, pp. 1473-1482.
19. J.H. Conway and N. J. A. Sloane, Sphere Packings, Lattices, and Groups. New York: Springer-Verlag, 1988.
20. T.M. Cover and J. A. Thomas, Elements of Information Theory (Wiley Series in Telecommunications). New York: Wiley, 1991.
21. T.M. Cover, "A proof of the data compression theorem of slepian and wolf for ergodic sources," IEEE Trans. Inform. Theory, vol. IT-21, pp. 226-228, 1975.
22. I. Csizär and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Channels. New York: Academic Press, 1981.
23. E.D. Davisson, "Universal lossless coding," IEEE Trans. Inform. Theory, vol. IT-19, pp. 783-795. 1973.
24. R. Durrett, Probability: Theory and Examples. Belmont, CA: Wadsworth, 1991.
25. W.E. Equitz and T. M. Cover, "Successive refinement of information," IEEE Trans. Inform. Theory, vol. 37, pp. 269 273, 1991.
26. P. Elias, "Universal codeword sets and representations of integers," IEEE Trans. Inform. Theory vol. IT-21, no. 2, pp. 194-203 Mar. 1975.
27. _. "Interval and recency tank source coding: Two on-line adaptive variable-length scheme," IEEE Trans. Inform. Theory vol. IT-33, no. 1, pp. I 15, Jan. 1987.
28. N. Fallcr, "An adaptive system for data compression," in Rec. 7th Asilomar Conf. on Circuits, Systems, and Computers (Naval Postgraduate School. Monterey. CA, 1973), pp. 593-597.
29. R.G. Gallager, "Variation on a theme by Huffman," IEEE Trans. Inform. Theory, vol. IT-24, pp. 668-674, 1978.
30. _, Information Theory and Reliable Communication. New York: Wiley, 1968.
31. R. Garcia-Munoz and D. E. Neuhoff, "Strong universal source coding subject to a rate distortion constraint/'/fjET: Tram. Inform. Theory, vol. IT-28, pp. 285-295, 1982.
32. A. Gersho and R. Gray. Vector Quantization and Signal Compression. Norwell, MA: Kluwer, 1992.
33. A. Gersho, "Adaptive vector quantization," Ann. des Telecommu.. vol. 41, nos. 9,10, pp. 470-480, Sept.-Oct. 1986.
34. A Gersho and M. Yano, "Adaptive vector quantization by progressive code-vector replacement." in Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, 1985, pp. 133-136.
35. R. Gray, "Vector quantization," IEEE ASSP Mag., pp. 4-29, Apr. 1984.
36. _, "Source coding theorems without the ergodic assumptions," IEEE Trans. Inform. Theory, vol. IT-20, pp. 502-516, 1974.
37. R.M. Gray, D. E. Neuhoff, and P. C. Shields, "A generalization of Ornstein's d distance with applications to information theory," Ann. Probab., vol. 3, pp. 315-328, 1975.
38. R.M. Gray and J. Kieffer, "Asymptotically mean stationary measures." Ann. Probab., vol. 8, pp. 962-973, 1980.
39. R.M. Gray and F. Saadat, "Block source coding theory for asymptoti-cally mean stationary sources," IEEE Trans. Inform. Theory, vol. IT-30, pp. 54-68, 1984.
40. R.M. Gray, Source Coding Theory. Boston, MA: Kluwer, 1990.
41. _, Entropy and Information Theory. New York: Springer-Verlag, 1990.
42. D.A. Huffman, "A method for the construction of minimum redundancy codes," Proc. IRE, vol. 40, pp. 1098-1101, 1952.
43. A.H. Kaspi and T. Berger, "Rate-distortion for correlated sources and partially separated encoders,'1 IEEE Trans. Inform. Theory, vol. IT-28, pp. 828-840, 1982.
44. j.C. KJeffer, "Fixed-rate encoding of nonstationary information sources."IEEE Trans. Inform. Theory, vol. IT-33, pp. 651-655. 1987.
45. -, "Finite-state adaptive block to variable length noiseless coding of a nonstatiotiary information source," IEEE Trans. Inform. Theory. vol. 35, pp. 1259-1263, 1989.
46. _, "Sample converses in source coding theory," IEEE Trans. Inform. Theory, vol. 37, pp. 263-268, 1991.
47. _, "Strong converses in source coding relative to a fidelity criterion.'' IEEE Trans. Inform. Theory, vol. 37, pp. 257-262, 1991.
48. _, "A unified approach to weak universal source coding," IEEE Tram. Inform. Theory, vol. 24, pp. 674-682, 1978.
49. _, "A survey of the theory of source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. 39, no. 5, pp. 1473-1490, 1993.
50. D.E. Knuth, "Dynamic Huffman coding." J. Algorithms, vol. 6, pp. 163-18C, 1985.
51. A.N. Kolmogorov, "Three approaches to the quantitative definition of information." Probl Inform.. Transm., vol. 1, pp. 1-7, 1965.
52. R.E. Krichevsky and V. K. Yrofimov, "The performance of universal encoding," IEEE Trans. Inform. Theory, vol. IT-27, pp. 199-207, 1981
53. A. Lempel and J. Ziv, "On the complexity of finite sequences," IEEE Trans. Inform. Theory, vol. IT-22, pp. 75-81, 1976.
54. A. Leon-Garcia, L. D. Davisson, and D. L. Neuhoff, "New results on coding of stationary nonergodic sources," IEEE Trans. Inform. Theory, vol. IT-25, pp. 137-144, 1979.
55. Y. Linde, A. Buzo, and R. Gray. "An algorithm for vector quantizer design," IEEE Trans. Commun., vol. COM-28, no. 1, pp. 84-95, Jan. 1980.
56. K.M. Mackenthun and M. B. Pursley, " Strongly and weakly universal source coding," in Proc.. 1977 Conf. on Information Science Systems (Baltimore, MD, The Johns Hopkms Univ., 1977), pp. 286-291.
57. K. Marton, "A simple proof of the blowing-up lemma," IEEE Trans. Inform. Theory, vol. IT-32, no. 3, pp. 445-446, May 1986.
58. [581 _, "Error exponent for source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-20, pp. 197-199, 1974.
59. H. Monta and K. Kobayashi. "An extension of LZW coding algorithm to source coding subject to a fidelity criterion." in Proc. 4th Joint Swedish-Soviet Int. Workshop on Information Theory (Gotland, Sweden. 1989), pp. 105-109.
60. N.M. Nasrabadi and Y. Feng, "Compression using address-vector quantization." IEEE Trans, Commun., vol. 38, pp. 2165-2173, 1990.
61. D.L. Neuhoff, R. M. Gray, and L. D. Davisson, "Fixed rate universal block source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-21, pp. 511 523, 1975.
62. D.L. Neuhoff and P. C. Shields, "Fixed rate universal codes for Markov sources," IEEE Trans. Inform. Theory, vol. IT-24, pp. 360-367, 1978.
63. D.S. Omslein and P. Shields, "Universal almost sure data compression." Annun! Probab , vol. 18, no. 2, pp. 441-452, 1990.
64. D.B. Paul, "A 500-800 bps adaptive vector quantization vocoder using a perceptually motivated distance measure." in Giobecom Conf Rec., 1982, pp. 1079-1082.
65. J.J. Rissanen, "Generalized Kraft inequality and arithmetic coding." IBM J. Res. Devel., vol. 20, pp. 198-203, 1976.
66. J.J. Rissanen and G. G. Langdon, "Arithmetic coding." IBM J. Res. Devel, vol. 23, pp. 149-162, 1979.
67. J.J. Rissanen, "Universal coding, information, prediction, and estimation," IEEE Trans. Inform. Theory, vol. IT-30, no. 4, pp. 629-636, July 1984.
68. "Stochastic complexity and modeling," Ann. Stal., vol. 14, no. 3, pp. 1080-1100, 1986.
69. B.Y. Ryabko, "Data compression by means of a 'book stack'," Probl. Inform. Transm., vol. 16, 1980.
70. Y. Steinberg and M. Gutman, "An algorithm for source coding subject to a fidelity criterion based on string matching," IEEE Trans. Inform. Theory, vol. 39, no. 3, pp. 877-886, May 1993.
71. C. Thomborson and B. Wei, "Systolic implementations of a move-to front text compressor," in Proc. ACM Symp. on Parallel Algorithms and Architectures (Santa Fe NM, 1989).
72. J. Vaisey and A. Gersho, "Image compression with variable block size segmentation," IEEE Trans. Sig. Processing, vol. 40, 1992.
73. V.K. Wei, "Fast variants of a locally adaptive data compression algorithm." preprint.
74. T. A Welch, "A technique for high-performance data compression," IEEE Comput., vol. 17, no. 6. pp. 8-19, June, 1984.
75. A. Wyner and J. Ziv, "Some asymptotic properties of the entropy of a stationary ergodic data source with applications to data compression," IEEE Trans. Inform. Theory, vol. 35, pp. 1250-1258, Nov. 1989.
76. H. Yamamoto and B. Rimoldi, "A universal data compression scheme with distortion," unpublished manuscript.
77. E.-H. Yang and S.-Y. Shen, "Distortion program-size complexity with respect to a (idelity criterion and rate-distortion function," IEEE Trans. Inform. Theory, vol. 39, no. 1, pp. 288-291, Jan. 1993.
78. E.-H. Yang, private communication.
79. _, "Universal almost sure data compression for abstract alphabet and arbitrary fidelity enterions," Probl. of Control, and Inform. Theory, vol. 26, no. 6, pp. 397-408, 1991.
80. E.-H. Yang and J. C. Kieffer, "Simple universal lossy data compression schemes derived from Lcmpel-Ziv algorithm," to appear in IfchE Trims. Inform. Theory.
81. "On the performances of data compression algorithms based upon string matching,'1 preprint.
82. Z. Zhang and T. Berger, "New results in binary multiple descriptions," IEEE Trans. Inform. Theory, vol. IT-33, pp. 502-521, 1987.
83. Z. Zhang and E.-H. Yang, "An on-line universal lossy data compression algorithm via continuous codebook refinement. Part II-Optimality for ffi-mixing source models," this issue, pp. 822-836.
84. Z. Zhang, E.-H. Yang, and V. K. Wei, "The redundancy of source coding with a fidelity criterion," submitted to IEEE Trans. Inform. Theory.
85. J. Ziv and A. Lempel, "A universal algorithm for sequential data compression," IEEE Trans. Inform, Theory, vol. IT-23, pp. 337-343, 1977.
86. _. "Compression of individual sequences via variable-rate coding," IEF.E Trans. Inform. Theory, vol. IT-24, pp. 530-536, 1978.
87. J. Ziv, "Coding for sources with unknown .statistics: Part I. Probability of error," IEEE Trans. Inform. Theory, vol. IT-18, pp. 384-389, 1972.
88. _. "Coding for sources with unknown statistics: Part II. Distortion relative to a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-18, pp. 389-394, 1972.
89. T. Linder, G. Lugosi, and K. Zeger, "Rates of convergence in the source coding theorem, in empirical quantizer design, and in universal lossy source coding," IEEE Trans. Inform. Theory, vol. 40, pp. 1728-1740, 1994.
90. _, ''Fixed rate universal lossy source coding and rates of convergence for memoryless sources," IEEE Trans. Information Theory, to appear.
91. M. Effros, P. A. Chou, and R. M. Gray, "Variable dimension weighted universal vector quantization and noiseless coding," in Proc. Data Compression Conf. (Snowbird. UT, Mar. 1994), pp. 2-11.
92. P.A. Chou, M. Effros. and R. M. Gray, "A vector quantization approach to universal noiseless coding and quantization." IEEE Trans. Inform. Theory to appear.