A distribution dependent refinement of Pinsker's inequality

IEEE Transactions on Information Theory

Volumn 51, Issue 5, 2005, Pages 1836-1840

  Ordentlich, Erik ^a   Weinberger, Marcelo J ^a  

^a HEWLETT PACKARD LABORATORIES   (United States)

Author keywords

Divergence; Hoeffding's inequality; L1 distance; Measure concentration; Pinsker's inequality; Sanov's theorem

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; DIFFERENTIAL EQUATIONS; POLYNOMIALS; PROBABILITY DISTRIBUTIONS; THEOREM PROVING;

DIVERGENCE; HOEFFDING INEQUALITY; MEASURE CONCENTRATION; PINSKER INEQUALITY; SANOV THEOREM;

INFORMATION THEORY;

EID: 18544370561     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2005.846407     Document Type: Article

References (11)

1. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
2. I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. Budapest, Hungary: Akadémiai Kiadó, 1981
3. A. A. Fedotov, P. Harremoös, and F. Topsøoe, "Refinements of Pinsker's inequality," IEEE Trans. Inf. Theory, vol. 49. no. 1, pp. 1491-1498, Jun. 2003.
4. W. Hoeffding, "Probability inequalities for sums of bounded random variables," Amer. Stat. Assoc. J., vol. 58, pp. 13-30, Mar. 1963.
5. K. Marton, "Bounding d̄-distance by informational divergence: A method to prove measure concentration," Ann. Probab., vol. 24, no. 2. pp. 857-866, 1996.
6. K. Marton, "Measure concentration for Euclidean distance in the case of dependent random variables," Ann. Probab., vol. 32, no. 3B, pp. 2526-2544, 2004.
7. C. McDiarmid, "On the method of bounded differences," in Surveys in Combinatories, London Mathematical Society Lecture Notes, J. Simons, Ed. London, U.K.: Cambridge Univ. Press, 1989, vol. 141, pp. 148-188.
8. M. Talagrand, "Concentration of measure and isoperimetric inequalities in product spaces," Publications Mathématiques de l'Institut des Heutes Études Scientifiques, vol. 81, pp. 73-205, 1995.
9. F. Topøoe, "Some inequalities for information divergence and related measures of discrimination," IEEE Trans. Inf. Theory, vol. 46, no. 4, pp. 1602-1609, Jul. 2000.
10. F. Topøoe, "Bounds for entropy and divergence for distributions over a two-element set," J. Inequalities in Pure and Appl. Math., no. 2, article 25, 2001.
11. T. Weissman, E. Ordentlich, G. Seroussi, S. Verdú, and M. J. Weinberger, "Universal discrete denoising: Known channel," IEEE Trans. Inf. Theory, vol. 51, no. 1, pp. 5-28, Jan. 2005.