Lossy source coding

IEEE Transactions on Information Theory

Volumn 44, Issue 6, 1998, Pages 2693-2723

  Berger, Toby ^a   Gibson, Jerry D ^b  

^a Cornell University   (United States)

^b Southern Methodist University   (United States)

Author keywords

Data compression; Image coding; Rate distortion theory; Signal coding; Source coding with a fidelity criterion; Speech coding; Video coding

Indexed keywords

DATA COMPRESSION; SIGNAL DISTORTION; SIGNAL ENCODING; SPEECH CODING;

LOSSY SOURCE CODING; RATE DISTORTION THEORY;

IMAGE CODING;

EID: 0032183171     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.720552     Document Type: Article

References (274)

1. C. E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J.. vol. 27, pp. 379-423; 623-656, July and Oct. 1948.
2. Claude Elwood Shannon: Collected Papers, N. J. A. Sloane and A. D. Wyner, Eds. Piscataway, NJ: IEEE Press, 1993, pp. 5-83.)
3. "Coding theorems for a discrete source with a fidelity criterion," in IRE Conv. Rec., vol. 7, 1959, pp. 142-163.
4. Information and Decision Processes, R. E. Machol, Ed. New York: McGraw-Hill, 1960, pp. 93-126,
5. Claude Elwood Shannon: Collected Papers, N. J. A. Sloane and A. D. Wyner, Eds. Piscataway, NJ: IEEE Press, 1993, pp. 325-350.)
6. A. N. Kolmogorov, "On the Shannon theory of information transmission in the case of continuous signals," IRE Trans. Inform. Theory, vol. IT-2, pp. 102-108, 1956.
7. "The theory of transmission of information, plenary session of the Academy of Sciences of the USSR on the automization of production" (Moscow, USSR, 1956), Izv. Akad. Nauk SSSR, pp. 66-99, 1957.
8. "A new metric invariant of transitive dynamic systems and automorphisms in Lebesgue spaces," Dokl. Akad. Nauk. SSSR, vol. 119, pp. 861-864, 1958.
9. J. L. Holsinger, "Digital communication over fixed time-continuous channels with memory-With special application to telephone chan-nels," Sc.D. dissertation, Dept. Elec. Eng., MIT, Cambridge, MA (Tech. Rep. TR 366, Lincoln Labs., Lexington, MA), 1968.
10. R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.
11. Y. G. Sinai, "On the concept of entropy of a dynamical system," Dokl. Akad. Nauk. SSSR, vol. 124, pp. 768-771, 1959.
12. D. S. Ornstein, "Bernoulli shifts with the same entropy are isomorphic," Adv. Math., vol. 4, pp. 337-352, 1970.
13. E. C. Posner and E. R. Rodemich, "Epsilon entropy and data compression," Ann. Math. Statist., vol. 42, pp. 2079-2125, 1971.
14. R. J. McEliece and E. C. Posner, "Hiding and covering in a compact metric space," Ann. Statist., vol. 1, pp. 729-739, 1973.
15. C. E. Shannon, "Communication in the presence of noise," Proc. IRE, vol. 37, pp. 10-21, 1949.
16. M. S. Pinsker, "Mutual information between a pair of stationary Gaussian random processes," Dokl. Akad. Nauk. USSR, vol. 99, no. 2, pp. 213-216, 1954.
17. "Computation of the message rate of a stationary random process and the capacity of a stationary channel," Dokl. Akad. Nauk. USSR, vol. 3, no. 4, pp. 753-756, 1956.
18. R. M. Gray and L. D. Davisson, "Source coding theorems without the ergodic assumption," IEEE Trans. Inform. Theory, vol. IT-20, pp. 625-636, 1974.
19. J. A. Bucklew, "The source coding theorem via Sanov's theorem," IEEE Trans. Inform. Theory, vol. IT-33, pp. 907-909, 1987.
20. J. C. Kieffer, "A survey of the theory of source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. 39, pp. 1473-1490, 1993.
21. "A unified approach to weak universal source coding," IEEE Trans. Inform. Theory, vol. IT-24, pp. 674-682, 1978.
22. "On the minimum rate for strong universal block coding of an class of ergodic sources," IEEE Trans. Inform. Theory, vol. IT-26, pp. 693-702, 1980.
23. "A method for proving multiterminal source coding theorems," IEEE Trans. Inform. Theory, vol. IT-27, pp. 565-570, 1981.
24. "Fixed-rate encoding of nonstationary information sources," IEEE Trans. Inform. Theory, vol. IT-33, pp. 651-655, 1987.
25. "Strong converses in source coding relative to a fidelity crite-rion," IEEE Trans. Inform. Theory, vol. 37, pp. 257-262, 1991.
26. "Sample converses in source coding theory," IEEE Trans. Inform. Theory, vol. 37, pp. 263-268, 1991.
27. T. J. Goblick Jr., "A coding theorem for time-discrete analog data sources," IEEE Trans. Inform. Theory, vol. IT-15, pp. 401-407, May 1969.
28. T. Berger, "Rate-distortion theory for sources with abstract alphabets and memory," Inform. Contr., vol. 13, pp. 254-273, 1968.
29. -, Rate Distortion Theory: A Mathematical Basis for Data Com-pression. Englewood Cliffs, NJ: Prentice-Hall, 1971.
30. F. Jelinek, Probabilistic Information Theory. New York: McGraw-Hill, 1968.
31. A. M. Gerrish, "Estimation of information rates," Ph.D. dissertation, Dept. Elec. Eng., Yale Univ., New Haven, CT, 1963
32. T. J. Goblick Jr., "Coding for a discrete information source with a distortion measure," Ph.D. dissertation, Dept. Elec. Eng., MIT, Cambridge, MA, 1962.
33. T. Berger and W. C. Yu, "Rate-distortion theory for context-dependent fidelity criteria," IEEE Trans. Inform. Theory, vol. IT-18, pp. 378-384, May 1972.
34. V. Erokhin, "e-entropy of a discrete random variable," Theory Probab. Its Applic., vol. 3, pp. 97-101, 1958.
35. R. Pile, "Coding theorems for discrete source-channel pairs," Ph.D. dissertation, Dept. Elec. Eng., MIT, Cambridge, MA, 1967.
36. "The transmission distortion of a discrete source as a function of the encoding block length," Bell Syst. Tech. J., vol. 47, pp. 827-885, 1968.
37. Z. Zhang, E. H. Yang, and V. K. Wei, "The redundancy of source coding with a fidelity criterion-Part I: Known statistics," IEEE Trans. Inform. Theory, vol. 43, pp. 71-91, Jan. 1997.
38. E.-h. Yang and Z. Zhang, "The redundancy of universal fixed rate source coding," presented at the 1998 IEEE Int. Symp. Information Theory, MIT, Cambridge, MA, Aug. 16-21, 1998.
39. "Abstract alphabet source coding theorem revisited: Redundancy analysis," presented at the 1998 IEEE Int. Symp. Information Theory, MIT, Cambridge, MA, Aug. 16-21, 1998.
40. T. Linder, G. Lugosi, and K. Zeger, "Rates of convergence in the source coding theorem, in empirical quantizer design, and in universal lossy source coding," IEEE Trans. Inform. Theory, vol. 40, pp. 1728-1740, 1994.
41. "Fixed-rate universal lossy source coding and rates of convergence for memoryless sources," IEEE Trans. Inform. Theory, vol. 41, pp. 665-676, 1995.
42. J. T. Pinkston, "Information rates of independent sample sources," M.S. thesis, Dept. Elec. Eng., MIT, Cambridge, MA, 1966.
43. "Encoding independent sample sources," Ph.D. dissertation, Dept. Elec. Eng., MIT, Cambridge, MA, 1967.
44. "An application of rate-distortion theory to a converse to the coding theorem," IEEE Trans. Inform. Theory, vol. IT-15, pp. 66-71, 1969.
45. R. A. McDonald, "Information rates of Gaussian signals under criteria constraining the error spectrum," D. Eng. dissertation, Yale Univ. School Elec. Eng., New Haven, CT, 1961.
46. R. A. McDonald and P. M. Schultheiss, "Information rates of Gaussian signals under criteria constraining the error spectrum," Proc. IEEE, vol. 52, pp. 415-416, 1964.
47. "Effective bandlimits of Gaussian processes under a mean square error criterion," Proc. IEEE, vol. 52, p. 517, 1964.
48. J. J. Y. Huang and P. M. Schultheiss, "Block quantization of correlated Gaussian variables," IRE Trans. Commun., vol. CS-11, pp. 289-296, 1963.
49. H. A. Spang and P. M. Schultheiss, "Reduction of quantizing noise by use of feedback," IRE Trans. Commun., vol. CS-12, pp. 373-380, 1964.
50. B. Bunin, "Rate-distortion functions for correlated Gaussian sources," Ph.D. dissertation, Dept. Elec. Eng., Polytech. Inst. Brooklyn, Brooklyn, NY, 1969.
51. J. P. M. Schalkwijk and L. I. Bluestein, "Transmission of analog waveforms through channels with feedback," IEEE Trans. Inform. Theory, vol. IT-13, pp. 617-619, 1967.
52. J. K. Omura, "Optimum linear transmission of analog data for channels with feedback," IEEE Trans. Inform. Theory, vol. IT-14, pp. 38-43, 1968.
53. S. Butman, "Rate-distortion over bandlimited feedback channels," IEEE Trans. Inform. Theory, vol. IT-17, pp. 110-112, 1971.
54. B. S. Tsybakov, "e-entropy of a vector message," presented at the IEEE Int. Symp. Information Theory, Ellenville, NY, 1969.
55. M. S. Pinsker, "Sources of messages," Probl. Pered. Inform., vol. 14, pp. 5-20, 1963
56. R. L. Dobrushin, "Individual methods for the transmission of information for discrete channels without memory and messages with independent components," Sov. Math., vol. 4, pp. 253-256, 1963.
57. "Unified methods for transmission of information: The general case, Sov. Math., vol. 4, pp. 289-292, 1963.
58. R. L. Dobrushin and B. S. Tsybakov, "Information transmission with additional noise," IRE Trans. Inform. Theory, vol. IT-8, pp. 293-304, 1962.
59. J. K. Wolf and J. Ziv, "Transmission of noisy information to a noisy receiver with minimum distortion," IEEE Trans. Inform. Theory, vol. IT-16, pp. 406-411, 1970.
60. Y. N. Lin'kov, "Evaluation of e-entropy of random variables for small e," Probl. Inform. Transm., vol. 1, pp. 12-18 (Russian pp. 12-18), 1965.
61. C. R. Davis and M. E. Hellman, "On tree coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-21, pp. 373-378, 1975.
62. F. Jelinek, "Evaluation of distortion rate functions for low distortions," Proc. IEEE, vol. 55, pp. 2067-2068, 1967.
63. "Tree encoding of memoryless time-discrete sources with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-15, pp. 584-590, 1969.
64. F. Jelinek and J. B. Anderson, "Instrumentable tree encoding of information sources," IEEE Trans. Inform. Theory, vol. IT-17, pp. 118-119, 1971.
65. J. A. Vanderhorst, "The error locator polynomial for binary and primitive triple error correcting BCH codes," M.S. thesis, School Elec. Eng., Cornell Univ., Ithaca, NY, Sept. 1971.
66. "Complete decoding of some binary BCH codes," Ph.D. dissertation, School Elec. Eng., Cornell Univ., Ithaca, NY, Aug. 1972.
67. W. E. Toms and T. Berger, "Information rates of stochastically driven dynamic systems," IEEE Trans. Inform. Theory, vol. IT-17, pp. 113-114, 1971.
68. R. J. Dick, "Tree coding for Gaussian sources," Ph.D. dissertation, School Elec. Eng., Cornell Univ., Ithaca, NY, May 1973. (See also
69. R. J. Dick, T. Berger, and F. Jelinek, IEEE Trans. Inform. Theory, vol. IT-20, pp. 332-336, 1974.)
70. T. Berger, F. Jelinek, and J. K. Wolf, "Permutation codes for sources," IEEE Trans. Inform. Theory, vol. IT-18, pp. 160-169, 1972.
71. T. Berger, "Information rates of wiener sequences," presented at the IEEE Int. Symp. Information Theory, Ellenville, NY, Jan. 28-31, 1969.
72. "Information rates of Wiener processes," IEEE Trans. Inform. Theory, vol. IT-16, pp. 134-139, 1970.
73. "The source coding game," IEEE Trans. Inform. Theory, vol. IT-17, pp. 71-76, 1971.
74. "Nyquist's problem in data transmission," Ph.D. dissertation, Harvard Univ., Div. Eng. App. Phys., Harvard Univ., Cambridge, MA, 1966.
75. H. Gish, "Optimum quantization of random sequences," Ph.D. dissertation, Harvard Univ., Div. Eng. Appl. Phys., Harvard Univ., Cambridge, MA, 1966.
76. H. Gish and J. N. Pierce, "Asymptotically efficient quantizing," IEEE Trans. Inform. Theory, vol. IT-14, pp. 676-683, 1968.
77. Private communication, 1967.
78. D. J. Sakrison, "Source encoding in the presence of a random disturbance," IEEE Trans. Inform. Theory, vol. IT-14, pp. 165-167, 1968.
79. "A geometric treatment of the source encoding of a Gaussian random variable," IEEE Trans. Inform. Theory, vol. IT-14, pp. 481-486, 1968.
80. "The rate-distortion function of a Gaussian process with a weighted square error criterion," IEEE Trans. Inform. Theory, Addendum 610-611, vol. IT-14, pp. 506-508, 1968.
81. D. J. Sakrison and V. R. Algazi, "Comparison of line-by-line and twodimensional coding of random images," IEEE Trans. Inform. Theory, vol. IT-17, pp. 386-398, 1971.
82. D. J. Sakrison, "The rate of a class of random processes," IEEE Trans. Inform. Theory, vol. IT-16, pp. 10-16, 1970.
83. B. G. Haskell, "The computation and bounding of rate-distortion functions," IEEE Trans. Inform. Theory, vol. IT-15, pp. 525-531, 1969.
84. J. F. Hayes, A. Habibi, and P. A. Wintz, "Rate-distortion function for a Gaussian source model of images," IEEE Trans. Inform. Theory, vol. IT-16, pp. 507-509, 1970.
85. R. M. Gray, "Information rates of autoregressive sources," Ph.D. dissertation, Elec. Eng., Univ. South. Calif., Los Angeles, CA, 1969.
86. "Information rates of autoregressive processes," IEEE Trans. Inform. Theory, vol. IT-16, pp. 412-421, 1970.
87. "Rate-distortion functions for finite-state finite-alphabet Markov sources," IEEE Trans. Inform. Theory, vol. IT-17, pp. 127-134, 1971.
88. "Information rates of stationary ergodic finite-alphabet sources," IEEE Trans. Inform. Theory, vol. IT-17, pp. 516-523, 1971;
89. Correction, vol. IT-19, p. 573, 1973.
90. R. E. Blahut, "Computation of channel capacity and rate-distortion functions," IEEE Trans. Inform. Theory, vol. IT-18, pp. 460-473, 1972.
91. S. Arimoto, "An algorithm for calculating the capacity of an arbitrary discrete memoryless channel," IEEE Trans. Inform. Theory, vol. IT-18, pp. 14-20, 1972.
92. I. Csiszâr and G. Tusnady, "Information geometry and alternating minimization procedures," in Statistics and Decisions/Supplement Issue, no. 1, E. J. Dudewicz, D. Plachky, and P. K. Sen, Eds. Munich, Germany: R. Oldenbourg Verlag, 1984, pp. 205-237.
93. (Formerly entitled On Alternating Minimization Procedures, preprint of the Math. Inst. Hungarian Acad. Sei., no. 35/1981, 1981.)
94. T. M. Cover, "Broadcast channels," IEEE Trans. Inform. Theory, vol. IT-18, pp. 2-14, 1972.
95. P. Bergmans, "Random coding theorem for broadcast channels with degraded components," IEEE Trans. Inform. Theory, vol. IT-19, pp. 197-207, 1973.
96. R. Ahlswede, "Multi-way communication channels," in Proc. 2nd. Int. Symp. Information Theory (Tsahkadsor, Armenian SSR), 1971, pp. 23-52.
97. H. Liao, "Multiple access channels," Ph.D. dissertation, Dept. Elec. Eng., Univ. Hawaii, Honolulu, HI, 1972.
98. R. M. Gray and A. D. Wyner, "Source coding for a simple network,"Bell Syst. Tech.J., vol. 58, pp. 1681-1721, 1974.
99. D. Slepian and J. K. Wolf, "Noiseless coding of correlated information sources," IEEE Trans. Inform. Theory, vol. IT-19, pp. 471-480, 1973.
100. A. D. Wyner and J. Ziv, "The rate-distortion function for source coding with side-information at the receiver," IEEE Trans. Inform. Theory, vol. IT-22, pp. 1-11, 1976.
101. T. Berger, "Multiterminal source coding," in The Information Theory Approach to Communications (CISM Courses and Lectures, no. 229). Wien, New York: Springer-Verlag, 1977, pp. 171-231.
102. J. Korner and K. Marton, "The comparison of two noisy channels," in Trans. Keszthely Colloq. Information Theory (Hungarian National Academy of Sciences, Keszthely, Hungary, Aug. 8-12), pp. 411-423.
103. "Images of a set via two channels and their role in multi-user communications," IEEE Trans. Inform. Theory, vol. IT-23, pp. 751-761, 1977.
104. "How to encode the modulo-two sum of binary sources," IEEE Trans. Inform. Theory, vol. IT-25, pp. 219-221, 1979.
105. T. Berger and S. Y. Tung, "Encoding of correlated analog sources," in Proc. 1975 IEEE-USSR Joint Work. Information Theory. Piscataway, NJ: IEEE Press, Dec. 1975, pp. 7-10.
106. S. Y. Tung, "Multiterminal rate-distortion theory," Ph.D. dissertation, Cornell Univ., Ithaca, NY, 1977.
107. M. U. Chang, "Rate-distortion with a fully informed decoder and a partially informed encoder," Ph.D. dissertation, Cornell Univ., Ithaca, NY, 1978.
108. A. Shohara, "Source coding theorems for information networks," Ph.D. dissertation, Univ. Calif. Los Angeles, Tech. Rep. UCLA-ENG-7445, 1974.
109. J. K. Omura and K. B. Housewright, "Source coding studies for information networks," in Proc. IEEE 1977 Int. Conf. Communications (Chicago, 111., June 13-15, 1977). New York: IEEE Press, 1997, pp. 237-240.
110. T. Berger, K. B. Housewright, J. K. Omura, S. Y. Tung, and J. Wolfowitz, "An upper bound on the rate-distortion function for source coding with partial side information at the decoder," IEEE Trans. Inform. Theory, vol. IT-25, pp. 664-666, 1979.
111. A. Sgarro, "Source coding with side information at several decoders," IEEE Trans. Inform. Theory, vol. IT-23, pp. 179-182, 1977.
112. J. Ziv and A. Lempel, "A universal algorithm for sequential data compression," IEEE Trans. Inform. Theory, vol. IT-23, pp. 337-343, 1977.
113. "Compression of individual sequences via variable-rate coding," IEEE Trans. Inform. Theory, vol. IT-24, pp. 337-343, 1978.
114. R. Pasco, "Source coding algorithms for fast data compression," Ph.D. dissertation, Stanford Univ., Stanford, CA, 1976.
115. J. Rissanen, "Generalized Kraft inequality and arithmetic coding," IBM J. Res. Develop., vol. 20, p. 198-, 1976.
116. "Universal coding, information, prediction and estimation," IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, 1984.
117. [Ill] F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "The context-tree weighting method: Basic properties," IEEE Trans. Inform. Theory, vol. 41, pp. 653-664, 1995.
118. "Context weighting for general finite-context sources," IEEE Trans. Inform. Theory, vol. 42, pp. 1514-1520, 1996.
119. Y. Steinberg and M. Gutman, "An algorithm for source coding subject to a fidelity criterion based on string matching," IEEE Trans. Inform. Theory, vol. 39, pp. 877-886, 1993.
120. Z. Zhang and V. K. Wei, "An on-line universal lossy data compression algorithm by continuous codebook refinement," IEEE Trans. Inform. Theory, vol. 42, pp. 803-821, 1996.
121. "An on-line universal lossy data compression algorithm by continuous codebook refinement, Part II: Optimality for çi-mixing source models," IEEE Trans. Inform. Theory, vol. 42, pp. 822-836, 1996.
122. I. Sadeh, "Universal compression algorithms based on approximate string matching," in Proc. 1995 IEEE Int. Symp. Information Theory (Whistler, BC, Canada, Sept. 17-22, 1995), p. 84.
123. E. H. Yang and J. Kieffer, "Simple universal lossy data compression schemes derived from the Lempel-Ziv algorithm," IEEE Trans. Inform. Theory, vol. 42, pp. 239-245, 1996.
124. E. H. Yang, Z. Zhang, and T. Berger, "Fixed-slope universal lossy data compression," IEEE Trans. Inform. Theory, vol. 43, pp. 1465-1476, 1997.
125. I. Kontoyiannis, "An implementable lossy version of the Lempel-Ziv algorithm-Part I: Optimality for memoryless sources," NSF Tech. Rep. 99, Dept. Statist., Stanford Univ., Stanford, CA, Apr. 1998.
126. "Asymptotically optimal lossy Lempel-Ziv coding," presented at IEEE Int. Symp. Information Theory, MIT, Cambridge, MA, Aug. 16-21, 1998.
127. "A decomposition theorem for binary Markov random fields," Ann. Probab., vol. 15, pp. 1112-1125, 1987.
128. L. A. Bassalygo and R. L. Dobrushin, "e-entropy of the random field," Probl. Pered. Inform., vol. 23, pp. 3-15, 1987.
129. C. M. Newman, "Decomposition of binary random fields and zeros of partition functions,"Acad. Probab., vol. 15, pp. 1126-1130, 1978.
130. C. M. Newman and G. A. Baker, "Decomposition of ising model and the mayer expansion," in Ideas and Methods in Mathematics and Physics,-In Memory of Raphael Hoegh-Krohn (1938-1988), S. Albeverio et at., Eds. Cambridge, U.K.: Cambridge Univ. Press, 1991.
131. T. Berger and Z. Ye, "e-entropy and critical distortion of random fields," IEEE Trans. Inform. Theory, vol. 36, pp. 717-725, 1990.
132. Z. Ye and T. Berger, "A new method to estimate the critical distortion of random fields," IEEE Trans. Inform. Theory, vol. 38, pp. 152-157, 1992.
133. -, Information Measures for Discrete Random Fields. Beijing, China: Chinese Acad. Sei., 1998.
134. A. H. Kaspi and T. Berger, "Rate-distortion for correlated sources with partially separated encoders," IEEE Trans. Inform. Theory, vol. IT-28, pp. 828-840, 1982.
135. T. Berger and R. W. Yueng, "Multiterminal source encoding with one distortion criterion," IEEE Trans. Inform. Theory, vol. 35, pp. 228-236, Mar. 1989.
136. "Multiterminal source encoding with encoder breakdown," IEEE Trans. Inform. Theory, vol. 35, pp. 237-244, 1989.
137. T. Berger, S. Y. Shen, and Z. Ye, "Some communication problems of random fields," Int. J. Math. Statist. Sei, vol. 1, pp. 47-77, 1992.
138. Z. Zhang and T. Berger, "Multiple description source coding with no excess marginal fate," IEEE Trans. Inform. Theory, vol. 41, pp. 349-357, 1995.
139. W. E. Equitz and T. M. Cover, "Successive refinement of information," IEEE Trans. Inform. Theory, vol. 37, pp. 269-275, 1991. (See also
140. W. E. Equitz and T. M. Cover, "Addendum to 'successive refinement of information'," IEEE Trans. Inform. Theory, vol. 39, pp. 1465-1466, 1993.)
141. T. Berger, Z. Zhang, and H. Viswanathan, "The CEO problem," IEEE Trans. Inform. Theory, vol. 42, pp. 887-903, May 1996.
142. H. Viswanathan and T. Berger, "The quadratic Gaussian CEO problem," IEEE Trans. Inform. Theory, vol. 43, pp. 1549-1561, 1997.
143. Y. Oohama, "The rate distortion function for the quadratic Gaussian CEO problem," IEEE Trans. Inform. Theory, vol. 44, pp. 1057-1070, May 1998.
144. E.-h. Yang, Z. Zhang, and T. Berger, "Fixed-slope universal lossy data compression," IEEE Trans. Inform. Theory, vol. 43, pp. 1465-1476, Sept. 1997.
145. D. L. Neuhoff, R. M. Gray, and L. D. Davisson, "Fixed rate universal block source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-21, pp. 511-523, 1975.
146. K. M. Mackenthum Jr. and M. B. Pursley, "Strongly and weakly universal source coding," in Proc. 1977 Conf. Information Science and Systems (The Johns Hopkins University, Baltimore, MD, 1977), pp. 286-291.
147. M. B. Pursley and K. M. Mackenthum Jr., "Variable-rate source coding for classes of sources with generalized alphabets," IEEE Trans. Inform. Theory, vol. IT-23, pp. 592-597, 1977.
148. K. M. Mackenthum Jr. and M. B. Pursley, "Variable-rate universal block source coding subject to a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-24, pp. 349-360, 1978.
149. H. H. Tan, "Tree coding of discrete-time abstract alphabet stationary block-ergodic sources with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-22, pp. 671-681, 1976.
150. J. Ziv, "Coding of sources with unknown statistics-Part II: Distortion relative to a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-18, pp. 389-394, 1972.
151. T. Hashimoto, "Tree coding of sources and channels," Ph.D. dissertation, Dept. Mech. Eng., Osaka Univ., Toyonaka, Osaka, Japan, 1981.
152. A. Gersho and A. D. Wyner, "The multiple descriptions problem," presented by A. D. Wyner at the IEEE Information Theory Work., Seven Springs Conf. Or., Mt. Kisco, NY, Sept. 1979.
153. H. S. Witsenhausen, "On source networks with minimal breakdown degradation," Bell Syst. Tech. J., vol. 59, pp. 1083-1087, 1980.
154. J. K. Wolf, A. D. Wyner, and J. Ziv, "Source coding for multiple descriptions," Bell Syst. Tech. J., vol. 59, pp. 1417-1426, 1980.
155. L. H. Ozarow, "On the source coding problem with two channels and three receivers," Bell Syst. Tech. J., vol. 59, pp. 1909-1922, 1980.
156. H. A. Witsenhausen and A. D. Wyner, "Source coding for multiple descriptions II: A binary source," Bell Syst. Tech. J., vol. 60, pp. 2281-2292, 1981.
157. A. E. Gamal and T. M. Cover, "Achievable rates for multiple descriptions," IEEE Trans. Inform. Theory, vol. IT-28, pp. 851-857, 1982.
158. T. Berger and Z. Zhang, "Minimum breakdown degradation in binary source encoding," IEEE Trans. Inform. Theory, vol. IT-29, pp. 807-814, 1983.
159. R. Ahlswede, "The rate-distortion region for multiple descriptions without excess rate," IEEE Trans. Inform. Theory, vol. IT-31, pp. 721-726, 1985.
160. A. D. Wyner, "The rate-distortion function for source coding with side information at the decoder-II: General sources," Inform. Contr., vol. 38, pp. 60-80, 1978
161. [ 154] H. Yamamoto, "Source coding theory for cascade and branching communication systems," IEEE Trans. Inform. Theory, vol. IT-27, pp. 299-308, 1981.
162. C. Heegard and T. Berger, "Rate distortion when side information may be absent," IEEE Trans. Inform. Theory, vol. IT-31, pp. 727-734, 1985.
163. H. Yamamoto, "Source coding theory for a triangular communication system," IEEE Trans. Inform. Theory, vol. 42, pp. 848-853, 1996.
164. Z. Zhang and T. Berger, "New results in binary multiple descriptions," IEEE Trans. Inform. Theory, vol. IT-33, pp. 502-521, July 1987.
165. K. Rose, "A mapping approach to rate-distortion computation and analysis," IEEE Trans. Inform. Theory, vol. 42, pp. 1939-1952, 1996.
166. S. L. Fix, "Rate-distortion functions for squared error distortion measures," in Proc. 16th Annu. Allerton Conf. Communication, Control and Computers (Monticello, IL, 1978).
167. R. M. Gray, "A new class of lower bounds to information rates of stationary sources via conditional rate-distortion functions," IEEE Trans. Inform. Theory, vol. IT-19, pp. 480-489, 1973.
168. B. M. Leiner, "Rate-distortion theory for sources with side information," Ph.D. dissertation, Stanford Univ., Stanford, CA, Aug. 1973.
169. B. M. Leiner and R. M. Gray, "Rate-distortion for ergodic sources with side information," IEEE Trans. Inform. Theory, vol. IT-20, pp. 672-675, 1974.
170. R. M. Gray, D. L. Neuhoff, and J. K. Omura, "Process definitions of distortion rate functions and source coding theorems," IEEE Trans. Inform. Theory, vol. IT-21, pp. 524-532, 1975.
171. [ 164] R. M. Gray, D. L. Neuhoff, and D. S. Ornstein, "Nonblock source coding with a fidelity criterion," Ann. Probab., vol. 3, pp. 478-491, 1975.
172. R. M. Gray, D. S. Ornstein, and R. L. Dobrushin, "Block synchroniza-tion, sliding-block coding, invulnerable sources and zero error codes for discrete noisy channels," Ann. Probab., vol. 8, pp. 639-674, 1975.
173. N. Ahmed and K. R. Rao, Orthogonal Transforms for Digital Signal Processing. New York: Springer-Verlag, 1975.
174. T. C. Ancheta Jr., "Joint source channel coding," Ph.D. dissertation, Dept. Elec. Eng., Univ. Notre Dame, Notre Dame, IN, Aug. 1977.
175. [ 168] J. B. Anderson and J. B. Bodie, "Tree encoding of speech," IEEE Trans. Inform. Theory, vol. IT-21, pp. 379-387, July 1975.
176. J. B. Anderson, "Recent advances in sequential encoding of analog waveforms," in Conf. Rec., IEEE Nat. Telecommunications Conf., 1978, pp. 19.4.1-19.4.5.
177. B. S. Atal and M. R. Schroeder, "Predictive coding of speech signals," in WESCONTech. Papers, 1968, paper 8/2.
178. "Adaptive predictive coding of speech signals," Bell Syst. Tech. J., vol. 49, pp. 1973-1986, Oct. 1970.
179. B. S. Atal and S. L. Hanauer, "Speech analysis and synthesis by linear prediction of the speech wave," J. Acoust. Soc. Amer., vol. 50, pp. 637-655, 1971.
180. B. S. Atal and J. R. Remde, "A new model of LPC excitation for producing natural sounding speech at low bit rates," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, 1982, pp. 614-617.
181. B. S. Atal and M. R. Schroeder, "Stochastic coding of speech at very low bit rates," in Proc. IEEE Int. Conf. Communications, 1984, pp. 1610-1613.
182. D. W. Becker and A. J. Viterbi, "Speech digitization and compression by adaptive predictive coding with delayed decision," in Conf. Rec. Nat. Telecommunications Conf., 1975, pp. 46-18-16-23.
183. W. Chen, C. H. Smith, and S. Fralick, "A fast computational algorithm for the discrete cosine transform," IEEE Trans.Commun., vol. COM-25, pp. 1004-1009, Sept. 1977.
184. W.-H. Chen and W. K. Pratt, "Scene adaptive coder," IEEE Trans. Commun., vol. COM-32, pp. 225-232, Mar. 1984.
185. L. Chiariglione, "MPEG and multimedia communications," IEEE Trans. Circuits Syst. Video Technol., vol. 7, pp. 5-18, Feb. 1997.
186. R. V. Cox, "Three new speech coders from the ITU cover a range of applications," IEEE Commun. Mag., vol. 35, pp. 40-47, Sept. 1997.
187. C. C. Cutler, "Differential quantization of communications," U.S. Patent 2605361, July 29, 1952.
188. "Delayed encoding: Stabilizer for adaptive coders," IEEE Trans. Commun., vol. COM-19, pp. 898-907, Dec. 1971.
189. L. D. Davisson, "Rate-distortion theory and application," P roc. IEEE , vol. 60, pp. 800-808, July 1972.
190. L. D. Davisson and R. M. Gray, Data Compression. Dowden: Hutchinson & Ross, 1976.
191. T. Ebrahimi and M. Kunt, "Visual data compression for multimedia applications," Proc. IEEE , vol. 86, pp. 1109-1125, June 1998.
192. S. N. Efstratiadis and M. G. Strintizis, "Hierarchical prioritized predictive image coding," in Proc. Int. Conf. Image Processing (ICIP '96) (Switzerland, Sept. 1996), pp. 189-192.
193. P. Elias, "Predictive coding-Parts I and II," IRE Trans. Inform. Theory, vol. IT-1, pp. 16-33, Mar. 1955.
194. N. Farvardin and J. W. Modestino, "Optimal quantizer performance for a class of non-Gaussian memoryless sources," IEEE Trans. Inform. Theory, vol. 30, pp. 485-497, 1984.
195. T. R. Fischer, J. D. Gibson, and B. Koo, "Estimation and noisy source coding," IEEE Trans. Acoust., Speech, Signal Processing, vol. 38, pp. 23-34, Jan. 1990.
196. R. D. LeGall, "MPEG: A video compression standard for multimedia applications," Commun. Assoc. Comput. Mach., vol. 34, pp. 60-64, Apr. 1991.
197. R. G. Gallager, "Tree encoding for symmetric sources with a distortion measure," IEEE Trans. Inform. Theory, vol. IT-20, pp. 65-76, Jan. 1974.
198. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. Boston, MA: Kluwer, 1992.
199. I. A. Gerson and M. A. Jasiuk, "Vector sum excited linear prediction (VSELP) speech coding at 8 kbps," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, Apr. 1994, pp. 461-464.
200. J. D. Gibson, B. Koo, and S. D. Gray, "Filtering of colored noise for speech enhancement and coding," IEEE Trans. Signal Processing, vol. 39, pp. 1732-1742, Aug. 1991.
201. J. D. Gibson, T. Berger, T. Lookabaugh, D. Lindbergh, and R. L. Baker, Digital Compression for Multimedia: Principles and Standards. San Francisco, CA: Morgan-Kaufmann, 1998.
202. R. M. Gray, Source Coding Theory. Norwell, MA: Kluwer, 1990.
203. R. M. Gray and D. L. Neuhoff, "Quantization," IEEE Trans. Inform. Theory, this issue, pp. 2325-2383.
204. Y. Huang, N. P. Galatsanos, and H. M. Dreizen, "Priority DCT coding for image sequences," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (Toronto, Ont., Canada, 1991), pp. 2629-2632.
205. Y. Huang, H. M. Dreizen, and N. P. Galatsanos, "Prioritized DCT for compression and progressive transmission of images," IEEE Trans. Image Processing, vol. 1, pp. 477-487, Oct. 1992.
206. J. D. Irwin and J. B. O'Neal, "The design of optimum DPCM (Differential Pulse Code Modulation) encoding systems via the Kaiman predictor," 1968 JACC Preprints, pp. 130-136, 1968.
207. N. S. Jayant and P. Noll, Digital Coding of Waveforms. Englewood Cliffs, NJ: Prentice-Hall, 1984.
208. N. S. Jayant, Ed., Waveform Quantization and Coding. New York: IEEE Press, 1976.
209. A. M. Kondoz, Digital Speech: Coding for Low Bit Rate Communications Systems. Chichester, U.K.: Wiley, 1994.
210. J. Li and C.-C. J. Kuo, "An embedded DCT approach to progressive image compression," in Proc. Int. Conf. Image Processing (ICIP '96) (Switzerland, Sept. 1996), pp. 201-204.
211. Y. Linde, A. Buzo, and R. M. Gray, "An algorithm for vector quantizer design," IEEE Trans. Commun., vol. COM-28, pp. 84-95, Jan. 1980.
212. S. P. Lloyd, "Least squares quantization in PCM," unpublished, Bell Lab. Tech. Note, 1957.
213. D. T. Magill, "Adaptive speech compression for packet communication systems," in Conf. Rec. Nat. Telecommunications Conf., Nov. 1973, pp. 29D-1-29D-5.
214. J. L. Massey, "Joint source channel coding," in Communication Systems and Random Process Theory, J. K. Skwirzynski, Ed. Amsterdam, The Netherlands: Sijthoff and Nordhoff, 1978, pp. 279-293.
215. B. Mazor and W. A. Pearlman, "A trellis code construction and coding theorem for stationary Gaussian sources," IEEE Trans. Inform. Theory, vol. IT-29, pp. 924-930, Nov. 1983.
216. "A tree coding theorem for stationary Gaussian sources and the squared-error distortion measure," IEEE Trans. Inform. Theory, vol. IT-32, pp. 156-165, Mar. 1986.
217. S. McClellan and J. D. Gibson, "Variable-rate CELP based on subband flatness," IEEE Trans. Speech Audio Processing, vol. 5, pp. 120-130, Mar. 1997.
218. J. W. Modestino and V. Bhaskaran, "Robust two-dimensional tree encoding of images," IEEE Trans. Commun., vol. COM-29, pp. 1786-1798, Dec. 1981.
219. J. W. Modestino, V. Bhaskaran, and J. B. Anderson, "Tree encoding of images in the presence of channel errors," IEEE Trans. Inform. Theory, vol. IT-27, pp. 667-697, Nov. 1981.
220. P. Noll, "MPEG digital audio coding," IEEE Signal Processing Mag., vol. 14, pp. 59-81, Sept. 1997.
221. L. V. Oliveira and A. Alcaim, "Identification of dominant coefficients in DCT image coders using weighted vector quantization," in Proc. 1st Int. Conf. Image Processing (ICIP '94) (Austin, TX, Nov. 1994), vol. 1, pp. 110-113.
222. B. M. Oliver, "Efficient coding," Bell Syst. Tech. J., vol. 31, pp. 724-750, July 1952.
223. J. B. O'Neal Jr., "A bound on signal-to-quantizing noise ratios for digital encoding systems," Proc. IEEE , vol. 55, pp. 287-292, Mar. 1967.
224. W. A. Pearlman and P. Jakatdar, "A transform tree code for stationary Gaussian sources," IEEE Trans. Inform. Theory, vol. IT-21, pp. 761-768, 1985.
225. D. E. Pearson, "A realistic model for visual communication systems," Proc. IEEE, vol. 55, pp. 380-389, Mar. 1967
226. W. B. Pennebaker and J. L. Mitchell, JPEG Still Image Data Compression Standard. New York: Van Nostrand Reinhold, 1988.
227. N. Phamdo and N. Farvardin, "Optimal detection of discrete Markov sources over discrete memoryless channels-Applications to combined source-channel coding," IEEE Trans. Inform. Theory, vol. 40, pp. 186-193, Jan. 1994.
228. [221 ] J. R. Pierce, "The early days of information theory," IEEE Trans. Inform. Theory, vol. IT-19, pp. 3-8, Jan. 1973
229. W. K. Pratt, Digital Image Processing. New York: Wiley, 1978.
230. K. Rao and P. Yip, Discrete Cosine Transform. Boston, MA: Academic, 1990.
231. K. Ramchandran and M. Vetterli, "Rate-distortion optimal fast thresholding with complete JPEG/MPEG decoder compatibility," IEEE Trans. Image Processing, vol. 3, pp. 700-704, Sept. 1994.
232. "Best wavelet packet bases in a rate-distortion sense," IEEE
233. A. Said and W. A. Pearlman, "A new, fast, and efficient image codec based on set partitioning in hierarchical trees," IEEE Trans. Circuits Syst. Video Technol, vol. 6, pp. 243-250, June 1996.
234. D. J. Sakrison, "A geometric treatment of the source encoding of a Gaussian random variable," IEEE Trans. Inform. Theory, vol. IT-14, pp. 481-486, 1968
235. "Image coding applications of vision models," in Image Transmission Techniques, W. K. Pratt, Ed. New York: Academic, 1979, pp. 21-51.
236. R. Salami et al., "Design and description of CS-ACELP: A toll quality 8 kb/s speech coder," IEEE Trans. Speech Audio Processing, vol. 6, pp. 116-130, Mar. 1998.
237. R. Salami et al., "ITU-T G.729 Annex A: Reduced complexity 8 kb/s CS-ACELP codec for digital simultaneous voice and data," IEEE Commun. Mag., vol. 35, pp. 56-63, Sept. 1997.
238. K. Sayood et al., "A constrained joint source/channel coder design," IEEE J. Select. Areas Commun., vol. 12, pp. 1584-1593, Dec. 1994.
239. K. Sayood, Introduction to Data Compression. San Francisco, CA: Morgan-Kaufmann, 1996.
240. J. M. Shapiro, "Embedded image coding using zerotrees of wavelet coefficients," IEEE Trans. Signal Processing, vol. 41, pp. 3445-3462, Dec. 1993.
241. Y. Shoham and A. Gersho, "Efficient bit allocation for an arbitrary set of quantizers," IEEE Trans. Acoust., Speech, Signal Processing, vol. 36, pp. 1445-1453, Sept. 1988.
242. A. S. Spanias, "Speech coding: A tutorial review," Proc. IEEE , vol. 82, pp. 1541-1582, Oct. 1994.
243. L. C. Stewart, "Trellis data compression," Ph.D. dissertation, Dept. Elec. Eng., Stanford Univ., Stanford, CA, June 1981.
244. L. C. Stewart, R. M. Gray, and Y. Linde, "The design of trellis waveform coders," IEEE Trans. Commun., vol. COM-30, pp. 702-710, Apr. 1982.
245. D. Taubman and A. Zakhor, "Multirate 3-D subband coding of video," IEEE Trans. Image Processing, vol. 3, pp. 572-588, Sept. 1994.
246. A. G. Tescher and R. V. Cox, "Image coding: Variable rate differential pulse modulation through fixed rate channel," Proc. Soc. Photo-Optical Instr. Eng., vol. 119, pp. 147-154, Aug. 1977.
247. A. G. Tescher, "Transform image coding," in Advances in Electronics and Electron Physics. New York: Academic, 1979, pp. 113-155.
248. V. R. Viswanathan etal., "Variable frame rate transmission: A review of methodology and application to narrowband LPC speech coding," IEEE Trans. Commun., vol. COM-30, pp. 674-686, Apr. 1982.
249. A. Viterbi and J. Omura, "Trellis encoding of memoryless discretetime sources with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-20, pp. 325-332, May 1974.
250. G. K. Wallace, "The JPEG still picture compression standard," Commun. Assoc. Comput. Mach., vol. 34, pp. 30-44, Apr. 1991.
251. Z. Wang, "Reconsideration of a fast computational algorithm for the discrete cosine transform," IEEE Trans. Commun., vol. COM-31, pp. 121-123, Jan. 1983.
252. S. G. Wilson and S. Husain, "Adaptive tree encoding of speech at 8000 bps with a frequency-weighted error criterion," IEEE Trans. Commun., vol. COM-27, pp. 165-170, Jan. 1979.
253. P. A. Wintz and A. J. Kurtenbach, "Waveform error control in PCM telemetry," IEEE Trans. Inform. Theory, vol. IT-14, pp. 640-661, Sept. 1968
254. J. W. Woods, Ed., Subband Image Coding. New York: Kluwer, 1991.
255. J. M. Wozencraft and I. M. Jacobs, Principles of Communication Engineering. New York: Wiley, 1965.
256. Z. Xiong, O. G. Guleryuz, and M. T. Orchard, "A DCT-based embedded image coder," IEEE Signal Processing Lett., vol. 3, pp. 289-290, Nov. 1996.
257. R. Zelinski and P. Noll, "Adaptive transform coding of speech signals," IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-25, pp. 299-309, Aug. 1977.
258. "Approaches to adaptive transform speech coding at low bit rate," IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-27, pp. 89-95, Feb. 1979.
259. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
260. M. W. Marcellin, T. R. Fischer, and J. D. Gibson, "Predictive trellis coded quantization of speech," IEEE Trans. Acoust., Speech Signal Processing, vol. 38, pp. 46-55, Jan. 1990.
261. P. J. Sementilli, A. Bilgin, J. H. Kasner, and M. W. Marcellin, "Wavelet TCQ: Submission to JPEG-2000 (keynote address)," in Proc. SPIE (San Diego, CA, July 1998), vol. 3460.
262. P. Noll, "MPEG digital audio coding," IEEE Signal Processing Mag., vol. 14, pp. 59-81, Sept. 1997
263. T. Sikora, "MPEG digital video-coding standards," IEEE Signal Pro-cessing Mag., vol. 14, pp. 82-100, Sept. 1997.
264. R. V. Cox, B. G. Haskell, Y. LeCun, B. Shahraray, and L. Rabiner, "On the applications of multimedia processing to communications," Proc. IEEE, vol. 86, pp. 755-824, May 1998.
265. M. Vetterli and J. Kovacevic, Wavelets and Subband Coding. Englewood Cliffs, NJ: Prentice-Hall, 1995.
266. B. M. Oliver, J. R. Pierce, and C. E. Shannon, "The philosophy of PCM," Proc. IRE , vol. 36, pp. 1324-1331, 1948.
267. H. Dudley, "The vocoder," Bell Lab. Rec., vol. 17, pp. 1221-1226, 1939.
268. H. C. Woo and J. D. Gibson, "Low delay tree coding of speech at 8 kbitses," IEEE Trans. Speech Audio Processing, vol. 2, pp. 361-370, July 1994.
269. V. Koshelev, "Multilevel source coding and data-transmission theorem," in Proc. VII All-Union Conf. Theory of Coding and Data Transmission (Vilnius, USSR, 1978), pt. 1, pp. 85-92.
270. "Hierarchical coding of discrete sources," Probl. Pered. Inform., vol. 16, pp. 31-49, 1980.
271. "An evaluation of the average distortion for discrete scheme of sequential approximation," Probl. Pered. Inform., vol. 17, pp. 20-33, 1981.
272. B. Rimoldi, "Successive refinement of information: Characterization of the achievable rates," IEEE Trans. Inform. Theory, vol. 40, pp. 253-259, Jan. 1994.
273. V. Vaishampayan, J.-C. Batallo, and A. R. Calderbank, "On reducing granular distortion in multiple description quantization," presented at the 1998 IEEE Int. Symp. Information Theory, MIT, Cambridge, MA, Aug. 16-21, 1998.
274. V. K. Goyal, J. Kovacevic, and M. Vetterli, "Multiple description transform coding: Robustness to erasures using tight frame expansions," presented at the 1998 IEEE Int. Symp. Information Theory, MIT, Cambridge, MA, Aug. 16-21, 1998.