Universally attainable error exponents for rate-distortion coding of noisy sources

IEEE Transactions on Information Theory

Volumn 50, Issue 6, 2004, Pages 1229-1246

  Weissman, Tsachy ^a  

^a Stanford University   (United States)

Author keywords

Achievable exponents; Competitive minimax; Compound source; Denoising; Error exponents; Marton's exponent

Indexed keywords

BLOCK CODES; COMMUNICATION CHANNELS (INFORMATION THEORY); CONSTRAINT THEORY; ERRORS; SIGNAL DISTORTION; SIGNAL RECONSTRUCTION; SPURIOUS SIGNAL NOISE;

ACHIEVABLE EXPONENT; DENOISING; ERROR EXPONENT; MARTONS EXPONENT; RATE DISTORTION CODING;

SIGNAL ENCODING;

EID: 2942674914     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2004.828061     Document Type: Article

References (40)

1. A. Barron, J. Rissanen, and B. Yu, "The minimum description length principle in coding and modeling," IEEE Trans. Inform. Theory, vol. 44, pp. 2743-2760, Oct. 1998.
2. B. Baygün and A. O. Hero III, "Optimal simultaneous detection and estimation under a false alarm constraint," IEEE Trans. Inform. Theory, vol. 41, pp. 688-703, May 1995.
3. T. Berger, Rate-Distortion Theory: A Mathematical Basis for Data Compression. Englewood Cliffs, N.J: Prentice-Hall, 1971.
4. ____, "Lossy source coding," IEEE Trans. Inform. Theory, vol. 44, pp. 2693-2723, Oct. 1998.
5. R. E. Blahut, "Hypothesis testing and information theory," IEEE Trans. Inform. Theory, vol. IT-20, pp. 405-417, July 1974.
6. ____, "Information bounds of the Fano-Kullback type," IEEE Trans. Inform. Theory, vol. IT-22, pp. 410-421, July 1976.
7. ____, Principles and Practice of Information Theory. Reading, MA: Addison-Wesley, 1987.
8. P. A. Chou, M. Effros, and R. M. Gray, "A vector quantization approach to universal noiseless coding and quantization," IEEE Trans. Inform. Theory, vol. 42, pp. 1109-1138, July 1996.
9. I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.
10. A. Dembo and T. Weissman, "The minimax distortion redundancy in noisy source coding," IEEE Trans. Inform. Theory, vol. 49, pp. 3020-3030, Nov. 2003.
11. D. Donoho, "The Kolmogorov sampler," manuscript submitted for publication.
12. Y. Ephraim and R. M. Gray, "A unified approach for encoding clean and noisy sources by means of waveform and autoregressive model vector quantization," IEEE Trans. Inform. Theory, vol. 34, pp. 826-834, July 1988.
13. E. Erkip and T. M. Cover, "The efficiency of investment information," IEEE Trans. Inform. Theory, vol. 44, pp. 1026-1040, May 1998.
14. M. Feder and N. Merhav, "Universal composite hypothesis testing: A competitive minimax approach," IEEE Trans. Inform. Theory, vol. 48, pp. 1504-1518, June 2002.
15. R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.
16. P. A. Humblet, "Generalization of huffman coding to minimize the probability of buffer overflow," IEEE Trans. Inform. Theory, vol. IT-27, pp. 230-237, Mar. 1981.
17. F. Jelinek, "Buffer overflow in variable length coding of fixed rate sources," IEEE Trans. Inform. Theory, vol. IT-14, pp. 490-501, May 1968.
18. A. Kanlis and P. Narayan, "Error exponents for successive refinement by partitioning," IEEE Trans. Inform. Theory, vol. 42, pp. 275-282, Jan. 1996.
19. J. C. Kieffer, "A survey of the theory of source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. 39, pp. 1473-1490, Sept. 1993.
20. A. Lapidoth and E. Telatar, "Reliable communication under channel uncertainty," IEEE Trans. Inform. Theory, vol. 44, pp. 2148-2177, Oct. 1998.
21. E. Levitan and N. Merhav, "A competitive Neyman-Pearson approach to universal hypothesis testing with applications," IEEE Trans. Inform. Theory, vol. 48, pp. 2215-2229, Aug. 2002.
22. T. Linder, G. Lugosi, and K. Zeger, "Rates of convergence in the source coding theorem, in empirical quantizer design, and in universal lossy source coding," IEEE Trans. Inform. Theory, vol. 40, pp. 1728-1740, Nov. 1994.
23. K. Marton, "Error exponent for source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-20, pp. 197-199, Mar. 1974.
24. N. Merhav, "Universal coding with minimum probability of codeword length overflow," IEEE Trans. Inform. Theory, vol. 37, pp. 556-563, May 1991.
25. N. Merhav and I. Kontoyiannis, "Source coding exponents for zero-delay coding with finite memory," IEEE Trans. Inform. Theory, submitted for publication.
26. B. Natarajan, "Filtering random noise via data compression," in Proc. Data Compression Conference, DCC '93, 1993, pp. 60-69.
27. D. L. Neuhoff, R. M. Gray, and L. D. Davisson, "Fixed rate universal block source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-21, pp. 511-523, Sept. 1975.
28. J. Rissanen, "Universal coding, information, p.ediction, and estimation," IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, July 1984.
29. ____, "Fisher information and astochastic complexity," IEEE Trans. Inform. Theory, vol. 42, pp. 40-47, Jan. 1996.
30. ____, "MDL denoising," IEEE Trans. Inform. Theory, vol. 46, pp. 2537-2543, Nov. 2000.
31. C. E. Shannon, "Coding theorems for a discrete source with a fidelity criterion," in IRE Conv. Rec., vol. 7, 1959, pp. 142-163.
32. T. Weissman and N. Merhav, "Finite-delay lossy coding and filtering of individual sequences corrupted by noise," IEEE Trans. Inform. Theory, vol. 48, pp. 721-733, Mar. 2002.
33. ____, "Tradeoffs between the excess-code-length exponent and the excess-distortion exponent in lossy source coding," IEEE Trans. Inform. Theory, vol. 48, pp. 396-415, Feb. 2002.
34. H. S. Witsenhausen, "Indirect rate distortion problems," IEEE Trans. Inform. Theory, vol. IT-26, pp. 518-521, Sept. 1980.
35. J. K. Wolf and J. Ziv, "Transmission of noisy information to a noisy receiver with minimum distortion," IEEE Trans. Inform. Theory, vol. IT-16, pp. 406-411, July 1970.
36. A. D. Wyner, "On the probability of buffer overflow under an arbitrary bounded input-output distribution," SIAM J. Appl. Math., vol. 27, pp. 544-570, 1974.
37. E. H. Yang and J. Kieffer, "Simple universal lossy data compression schemes derived from the Lempel-Ziv algorithm," IEEE Trans. Inform. Theory, vol. 42, pp. 239-245, Jan. 1996.
38. O. Zeitouni and M. Gutman, "On universal hypotheses testing via large deviations," IEEE Trans. Inform. Theory, vol. 37, pp. 285-290, Mar. 1991.
39. J. Ziv, "Coding of sources with unknown statistics - Part II: Distortion relative to a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-18, pp. 389-394, May 1972.
40. ____, "Distortion-rate theory for individual sequences," IEEE Trans. Inform. Theory, vol. IT-26, pp. 137-143, Mar. 1980.