The Minimax Distortion Redundancy in Noisy Source Coding

IEEE Transactions on Information Theory

Volumn 49, Issue 11, 2003, Pages 3020-3030

  Dembo, Amir ^a   Weissman, Tsachy ^b,c,d  

^a STANFORD UNIVERSITY   (United States)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^c HEWLETT PACKARD LABORATORIES   (United States)

^d Stanford University   (United States)

Author keywords

Approximation estimation tradeoff; Denoising; Lossy source coding; Minimax distortion redundancy; Noisy sources

Indexed keywords

BINARY-SYMMETRIC CHANNEL (BSC); LOSSY SOURCE CODING;

APPROXIMATION THEORY; BLOCK CODES; INFORMATION THEORY; PROBABILITY; SET THEORY; THEOREM PROVING;

PROBLEM SOLVING;

EID: 0345306749     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2003.819336     Document Type: Article

References (30)

1. P. L, Bartlett, T. Linder, and G. Lugosi, "The minimax distortion redundancy in empirical quantizer design," IEEE Trans, Inform. Theory, vol. 44, pp. 1802-1813, Sept. 1998.
2. T. Berger, Rate-Distortion Theory: A Mathematical Basis for Data Compression. Englewood Cliffs, N.J.: Prentice-Hall, 1971.
3. P. A. Chou, M. Effros, and R. M. Gray, "A vector quantization approach to universal noiseless coding and quantization," IEEE Trans. Inform. Theory, vol. 42, pp. 1109-1138, July 1996.
4. F. Comets, "Large deviation estimates for a conditional probability distribution, applications to random interaction Gibbs measures," Prob. Th. Rel. Fields, vol. 80, pp. 407-432, 1989.
5. T. M. Cover and J.A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
6. I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.
7. L. D. Davisson, "Universal lossless coding," IEEE Trans. Inform. Theory, vol. IT-19, pp. 783-795, Nov. 1973.
8. A. Dembo and Y. Peres, "A topological criterion for hypothesis testing," Ann. Statist., vol. 22, pp. 106-117, 1994.
9. A. Dembo and O. Zeitouni, Large Deviations Techniques and Applications, 2nd ed. New York: Springer-Verlag, 1998.
10. R. M. Dudley, Uniform Central Limit Theorems. Cambridge, U.K.: Cambridge Univ. Press, 1999.
11. Y. Ephraim and R. M. Gray, "A unified approach for encoding clean and noisy sources by means of waveform and autoregressive model vector quantization," IEEE Trans. Inform. Theory, vol. 34, pp. 826-834, July 1988.
12. E. Erkip and T. M. Cover, "The efficiency of investment information." IEEE Trans. Inform. Theory, vol. 44, pp. 1026-1040, May 1998.
13. R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.
14. A. A. El Gamal and T. M. Cover, "Achievable rates for multiple descriptions," IEEE Trans. Inform. Theory, vol. IT-28, no. 6, pp. 851-857, Nov. 1982.
15. A. Lapidoth, "On the role of mismatch in rate distortion theory," IEEE Trans. Inform. Theory, vol. 43, pp. 38-47, Jan. 1997.
16. T. Linder, G. Lugosi, and K. Zeger, "Rates of convergence in the source coding theorem, in empirical quantizer design, and in universal lossy source coding," IEEE Trans. Inform. Theory, vol. 40, pp. 1728-1740, Nov. 1994.
17. N. Merhav, "Information theory-048733, lecture notes," Technion, Graduate Studies, Winter 1998-1999.
18. B. Natarajan, "Filtering random noise via data compression," in Proc. Data Compression Conf., DCC '93, 1993, pp. 60-69.
19. D. L. Neuhoff, R. M. Gray, and L. D. Davisson, "Fixed rate universal block source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-21, pp. 511-523, Sept. 1975.
20. D. Pollard, Convergence of Stochastic Processes. New York: Springer-Verlag, 1984.
21. J. Rissanen, "Universal coding, information, prediction, and estimation," IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, July 1984.
22. J. Stjernvall, "Dominance-A relation between distortion measures," IEEE Trans. Inform. Theory, vol. IT-29, pp. 798-807, Nov. 1983.
23. T. Weissman and N. Merhav, "Finite-delay lossy coding and filtering of individual sequences corrupted by noise," IEEE Trans. Inform. Theory, vol. 48, pp. 721-733, Mar. 2002.
24. _, "Tradeoffs between the excess-code-length exponent and the excess-distortion exponent in lossy source coding," IEEE Trans. Inform. Theory, vol. 48, pp. 396-415, Feb. 2002.
25. H. S. Witsenhausen, "Indirect rate distortion problems," IEEE Trans. Inform. Theory, vol. IT-26, pp. 518-521, Sept. 1980.
26. J. K. Wolf and J. Ziv, "Transmission of noisy information to a noisy receiver with minimum distortion." IEEE Trans. Inform. Theory, vol. IT-16, pp. 406-411, July 1970.
27. Q. Xie and A. R. Barren, "Minimax redundancy for the class of memoryless sources." IEEE Trans. Inform. Theory, vol. 43, pp. 646-657, Mar. 1997.
28. E. H. Yang and J. Kieffer, "Simple universal lossy data compression schemes derived from the Lempel-Ziv algorithm," IEEE Trans. Inform. Theory, vol. 42, pp. 239-245, Jan. 1996.
29. J. Ziv, "Coding of sources with unknown statistics - Part II: Distortion relative to a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-18, pp. 389-394, May 1972.
30. _, "Distortion-rate theory for individual sequences," IEEE Trans. Inform. Theory, vol. IT-26, pp. 137-143, Mar. 1980.