The redundancy of source coding with a fidelity criterion - Part one: Known statistics

IEEE Transactions on Information Theory

Volumn 43, Issue 1, 1997, Pages 71-91

  Zhang, Zhen ^a,d   Yang, En Hui ^b   Wei, Victor K ^c,d  

^a University of Southern California ^*  (United States)

^b NANKAI UNIVERSITY   (China)

^c CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^d IEEE   (United States)

Author keywords

Lower mutual information; Random coding argument; Rate distortion (distortion rate) function; Redundancy; Source coding with a fidelity criterion; Upper joint entropy

Indexed keywords

BLOCK CODES; MATHEMATICAL MODELS; REDUNDANCY; SIGNAL DISTORTION;

DISTORTION RATE; FIDELITY CRITERION; LOWER MUTUAL INFORMATION; MEMORYLESS CODES; RANDOM CODING ARGUMENT; SOURCE CODING; UPPER JOINT ENTROPY;

ENCODING (SYMBOLS);

EID: 0030784276     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.567651     Document Type: Article

References (44)

1. R. Ahlswede, P. Gács, and J. Körner, "Bounds on conditional probabilities with applications in multi-user communication," Z. Wahrschein-Lichketistheorie u. Verw. Geb., vol. 34, pp. 157-177, 1976.
2. A. R. Barron and T. M. Cover, "Minimum complexity density estimation," IEEE Trans. Inform. Theory, vol. 37, pp. 1034-1054, July 1991.
3. T. Berger, Rate Distortion Theory. Englewood Cliffs, NJ: Prentice-Hall, 1971.
4. B. S. Clarke and A. R. Barron, "Information theoretic asymptotics of Bayes methods," IEEE Trans. Inform. Theory, vol. 36, pp. 453-471, May 1990.
5. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
6. I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic Press, 1981.
7. L. D. Davisson, "Universal noiseless coding," IEEE Trans. Inform. Theory, vol. IT-19, pp. 783-795, 1973.
8. _, "Minimax noiseless universal coding for Markov sources," IEEE Trans. Inform. Theory, vol. IT-29, pp. 211-215, 1983.
9. P. Elias, "Universal codeword sets and representations of the integers," IEEE Trans. Inform. Theory, vol. IT-21, pp. 194-203, 1975.
10. R. M. Gray, Entropy and Information Theory. New York: Springer-Verlag, 1990.
11. R. M. Gray and L. D. Davisson, "Source coding theorems without the ergodic assumption," IEEE Trans. Inform. Theory, vol. IT-20, pp. 502-516, 1974.
12. R. M. Gray, D. L. Neuhoff, and P. C. Shields, "A generalization of Ornstein's d distance with applications to information theory," Ann. Probab., vol. 3, pp. 315-328, 1975.
13. E. J. Hannan and L. Kavalieris, "Regression, autoregressive methods," J. Time Series Anal., vol. 7, pp. 27-49, 1986.
14. E. M. Hemerly and M. H. A. Davis, "Strong consistency of the predictive least squares criterion for order determination of autoregressive processes," Ann. Statist., vol. 17, pp. 941-946, 1989.
15. F. Jelinek and K. Schneider, "On variable-length to block coding," IEEE Trans. Inform. Theory, vol. IT-18, pp. 765-774, 1977.
16. G. L. Khodak, "The connection between the redundancy and the delay for VB-encoding," in Proc. III Conf. on Theoretical Cybernetics (Novosibirsk, Russia, 1969), pp. 79-80.
17. _, "The estimation of redundancy for coding the messages generated by a Bernoulli source," Probl. Inform. Transm., vol. 8, pp. 28-32, 1972.
18. _, "Coding of Markov sources with low redundancy," in Proc. 2nd Int. Symp. on Information Theory (Tsahkadzor, Acad. Kiado, Budapest, Hungary, 1973), pp. 201-209.
19. J. C. Kieffer, "A unified approach to weak universal source coding," IEEE Trans. Inform. Theory, vol. IT-24, pp. 674-682, 1978.
20. R. E. Krichevskii, "The block length necessary to obtain a given redundancy," Sov. Math.-Dokl., vol. 7, pp. 1416-1419, 1968.
21. _, "The connection between the redundancy and reliability of information about the source," Probl. Inform. Transm., vol. 4, pp. 48-57, 1968.
22. R. E. Krichevskii and V. K. Trofimov, "The performance of universal encoding," IEEE Trans. Inform. Theory, vol. IT-27, pp. 199-207, 1981.
23. T. Linder, G. Lugosi, and K. Zeger, "Rates of convergence in the source coding theorem, in empirical quantizer design, and in universal lossy source coding," IEEE Trans. Inform. Theory, vol. 40, no. 6, pp. 1728-1740, Nov. 1994.
24. K. M. Mackenthun and M. B. Pursley, "Variable-rate universal block source coding subject to a fidelity constraint," IEEE Trans. Inform. Theory, vol. IT-24, pp. 349-360, 1978.
25. K. Marton, "A simple proof of the blowing-up lemma," IEEE Trans. Inform. Theory, vol. IT-32, no. 3, pp. 445-446, 1986.
26. N. Merhav, "A comment on "A rate of convergence result for a universal D-semifaithful code," IEEE Trans. Inform. Theory, vol. 41, no. 4, pp. 1200-1202, 1995.
27. D. L. Neuhoff, R. M. Gray, and L. D. Davisson, "Fixed rate universal block source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-21, pp. 511-523, 1975.
28. D. S. Ornstein and P. C. Shields, "Universal almost sure data compression," Ann. Probab., vol. 18, pp. 441-452, 1990.
29. R. J. Pile, "The transmission distortion of a source as a function of the encoding block length," Bell Syst. Tech. J., vol. 47, pp. 827-885, 1968.
30. E. Plotnik, J. Weinberger, and J. Ziv, "Upper bounds on the probability of sequences emitted by finite-state source and on the redundancy of the Lempel-Ziv algorithm," IEEE Trans. Inform. Theory, vol. 38, pp. 66-72, 1992.
31. J. Rissanen, "Stochastic complexity and modeling," Ann. Statist., vol. 14, pp. 1080-1100, 1986.
32. _, "Universal coding, information, prediction and estimation," IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, 1984.
33. _, "Complexity of strings in the class of Markov sources," IEEE Trans. Inform. Theory, vol. IT-32, pp. 526-532, 1986.
34. J. Rissanen, T. P. Speed, and B. Yu, "Density estimation by stochastic complexity," IEEE Trans. Inform. Theory, vol. 38, pt. I, pp. 315-323, 1992.
35. P. C. Shields, "Universal almost sure data compression using Markov types," Probl. Contr. Inform. Theory, vol. 19, pp. 269-277, 1990.
36. C. E. Shannon, "Mathematical theory of communication," Bell Syst. Tech. J., vol. 27, pp. 379-423, pp. 623-656, 1948.
37. Y. M. Shtarkov, "Coding of finite length messages on the output of a source with unknown statistical properties," in Proc. 5th All-Union Conf. on Information Theory (Moscow-Gorkii, USSR, 1972), pp. 147-152.
38. _, "Universal sequential coding of single messages," Probl. Inform. Transm., vol. 23, pp. 175-186, 1988.
39. V. K. Trofimov, "The redundancy of Markov source encoding," Probl. Inform. Transm., vol. 10, pp. 16-24, 1974.
40. E.-H. Yang, "Universal almost sure data compression for abstract alphabets and arbitrary fidelity enterions," Probl. Contr. Inform. Theory, vol. 20, pp. 397-408, 1991.
41. E.-H. Yang and J. C. Kieffer, "Simple universal lossy data compression schemes derived from Lempel-Ziv algorithm," IEEE Trans. Inform. Theory, vol. 42, no. 1, pp. 239-245, Jan. 1996.
42. B. Yu and T. P. Speed, "A rate of convergence result for a universal D-semifaithful code," IEEE Trans. Inform. Theory, vol. 39, pp. 813-820, 1993.
43. Z. Zhang and V. K. Wei, "An on-line universal lossy data compression algorithm by continuous codebook refinement-Part I: Basic results," IEEE Trans. Inform. Theory, vol. 42, no. 3, pp. 803-821, May 1996.
44. Z. Zhang and E.-H. Yang, "An on-line universal lossy data compression algorithm via continuous codebook refinement - Part II: Optimality for phi-mixing source models," IEEE Trans. Inform. Theory, vol. 42, no. 3, pp. 822-836, May 1996.