On Pinsker's and Vajda's type inequalities for Csiszár's f-divergences

IEEE Transactions on Information Theory

Volumn 56, Issue 11, 2010, Pages 5377-5386

  Gilardoni, Gustavo L ^a  

^a UNIVERSITY OF BRASILIA   (Brazil)

Author keywords

Hellinger discrimination; information inequalities; KullbackLeibler divergence; relative entropy; relative information; Rnyi's information gain; variational or L1distance

Indexed keywords

HELLINGER DISCRIMINATION; INFORMATION GAIN; INFORMATION INEQUALITIES; KULLBACK LEIBLER DIVERGENCE; RELATIVE ENTROPY; RELATIVE INFORMATION; VARIATIONAL OR L1DISTANCE;

ENTROPY;

EID: 77958532086     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2010.2068710     Document Type: Article

References (49)

1. V. A. Volkonskij and J. A. Rozanov, "Some limit theorems for random functions - I," Theory Prob. Applicat., vol.4, pp. 178-197, 1959.
2. A. Fedotov, P. Harremoës, and F. Topsøe, "Refinements of Pinsker's Inequality," IEEE Trans. Inf. Theory, vol.49, pp. 1491-1498, Jun. 2003.
3. A. R. Barron, "Entropy and the central limit theorem," Ann. Probab., vol.14, no.1, pp. 336-342, 1986.
4. F. Topsøe, "Information theoretical optimization techniques," Kybernetika, vol.15, no.1, pp. 8-27, 1979.
5. P. Harremoës and P. Ruzankin, "Rate of convergence to Poisson law in terms of information divergence," IEEE Trans. Inf. Theory, vol.50, no.9, pp. 2145-2149, 2004.
6. M. Del Pino and J. Dolbeault, "Best constants for Gagliardo-Niremberg inequalities and applications to nonlinear difussions," J. Math. Pures Appl., vol.81, pp. 847-875, 2002.
7. I. Csiszár, "Information-type measures of difference of probability distributions and indirect observations," Studia Sci. Math. Hungary, vol.2, pp. 299-318, 1967.
8. J. H. B. Kemperman, "On the optimal rate of transmitting information," Ann. Math. Statist, vol.40, pp. 2156-2177, Dec. 1969.
9. M. S. Pinsker, Information and Information Stability of Random Variables and Processes, A. Feinstein tr., Ed. San Francisco, CA: Holden-Day, 1964.
10. H. P. McKean Jr., "Speed of approach to equilibrium for Kac's caricature of a Maxwellian gas," Arch. Rational Mech. Anal., vol.21, pp. 343-367, 1966.
11. I. Csiszár, "A note on Jensen's inequality," Studia Sci. Math. Hungary, vol.1, pp. 185-188, 1966.
12. S. Kullback, "A lower bound for discrimination information in terms of variation," IEEE Trans. Inf. Theory, vol.IT-13, pp. 126-127, Jan. 1967.
13. S. Kullback, "Correction to "A lower bound for discrimination information in terms of variation"," IEEE Trans. Inf. Theory, vol.16, p. 652, 1970.
14. I. Vajda, "Note on discrimination information and variation," IEEE Trans. Inf. Theory, vol.16, no.6, pp. 771-773, 1970.
15. F. Topsøe, "Bounds for entropy and divergence of distributions over a two-element set," J. Ineq. Pure Appl. Math., vol. 2, no. Article 25, 2001.
16. A. Fedotov, P. Harremoës, and F. Topsøe, "Best Pinsker bound equals Taylor polynomial of degree 49," Comput. Technol., vol.8, no.5, pp. 3-14, 2003.
17. I. Csiszár, "Eine informationstheoretische Ungleichung und ihre anwendung auf den Beweis der ergodizität von Markoffschen Ketten," Publ. Math. Inst. Hungary Acad., vol.8, pp. 95-108, 1963.
18. S. M. Ali and S. D. Silvey, "A general class of coefficients of divergence of one distribution from another," J. Roy. Statist. Soc. B, vol.28, pp. 131-142, 1966.
19. I. Vajda and J. Zvárová, "On generalized entropies, Bayesian decisions and statistical diversity," Kybernetika, vol.43, pp. 675-696, 2007.
20. F. Liese and I.Vajda, "On divergences and informations in statistics and information theory," IEEE Trans. Inf. Theory, vol.52, pp. 4394-4412, 2006.
21. G. L. Gilardoni, "Very accurate posterior approximations based on finite mixtures of the hyperparameters conditionals," Comput. Statist. Data Anal., vol.51, no.2, pp. 872-884, 2006.
22. D. Y. E. Lutwak and G. Y. Zhang, "Cramér-Rao and moment-entropy inequalities for Renyi entropy and generalized Fisher information," IEEE Trans. Inf. Theory, vol.51, pp. 473-478, 2005.
23. G. L. Gilardoni, "Accuracy of posterior approximations via chi-squared and harmonic divergences," J. Statist. Planning Inference, vol.128, no.2, pp. 475-487, 2005.
24. I. Vajda, Theory of Statistical Inference and Information. London, U.K.: Kluwer, 1989.
25. I. Taneja, "Generalized relative information and information inequalities," J. Ineq. Pure Appl. Math., vol.5, no.1, Article 21, 2004.
26. C. Tsallis, "Possible generalization of Boltzmann-Gibbs statistics," J. Stat. Phys., vol.52, p. 479, 1988.
27. N. Cressie and T. R. C. Read, "Multinomial goodness-of-fit tests," J. Royal Statist. Soc. B, vol.46, pp. 440-464, 1984.
28. A. Golan, "Information and entropy econometrics-Editor's view," J. Econometrics, vol.107, pp. 1-15, 2002.
29. L.Withers Jr., "Some inequalities relating different measures of divergence between two probability distributions," IEEE Trans. Inf. Theory, vol.45, no.5, pp. 1728-1735, 1999.
30. A. Rényi, "On measures of entropy and information," in Proc. 4th Berkeley Symp. Math. Statist. and Prob., Berkeley, CA, 1961, vol.1, pp. 547-561.
31. I. Csiszár, "Generalized cutoff rates and Rényi information measures," IEEE Trans. Inf. Theory, vol.41, pp. 26-34, Jan. 1995.
32. D. Dacunha-Castelle, Ecole d'Ete de Probabilites de Saint-Flour VII-1977. Berlin, Germany: Springer, 1978.
33. L. LeCam, Asymptotic Methods in Statistical Theory. New York: Springer-Verlag, 1986.
34. S. S. Dragomir, V. Gluscević, and C. E. M. Pearce, "Csiszár divergence, Ostrowski's inequality and mutual information," Nonlinear Anal., vol.47, pp. 2375-2386, 2001.
35. F. Topsøe, "Some inequalities for information divergence and related measures of discrimination," IEEE Trans. Inf. Theory, vol.46, pp. 1602-1609, 2000.
36. I. Csiszár, "On topological properties of divergences," Stud. Sci. Math. Hungary, vol.2, pp. 329-339, 1967.
37. I. Vajda, "On the divergence and singularity of probability measures," Periodica Math. Hungarica, vol.2, pp. 223-234, 1972.
38. G. L. Gilardoni, "On the minimum divergence for given total variation," C. R. Acad. Sci. Paris, Ser. I, 2006.
39. G. L. Gilardoni, Maple Script for Computations in "On Pinsker's and Vajda's Inequalities and Csiszár's Divergences" [Online]. Available: http://www.unb.br/ie/est/docentes/curriculo/gustavo-homepage/artigos/part1.txt
40. W. Rudin, Real and Complex Analysis. New York: McGraw-Hill, 1987.
41. R. L. Wheeden and A. Zygmund, Measure and Integral: An Introduction to Real Analysis. New York: Marcel Dekker, 1977.
42. S. Boyd and L. Vanderberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
43. I. Vajda, "Divergence and generalized Fischer's informations," in Trans. of Sixth Prague Conf. on Inform. Theory, Statist. Dec. Functions and Random Processes, Prague, Czech Republic, 1973, pp. 873-886.
44. R. Weiss, "An approach to Bayesian sensitivity analysis," J. R. Statist. Soc. B, vol.58, pp. 163-177, 1996.
45. C. Kraft, "Some conditions for consistency and uniform consistency of statistical procedures," Univ. California Publ. Statist., vol.1, pp. 125-142, 1955.
46. E. Ordentlich and M. J. Weinberger, "A distribution dependent refinement of Pinsker's inequality," IEEE Trans. Inf. Theory, vol.51, pp. 1836-1840, 2005.
47. F. Topsøe, "Some bounds for the logarithmic function," RGMIA Res. Rep. Collection, vol.7, no.2, Article 6, 2004.
48. F. Liese and I. Vajda, Convex Statistical Distances. Leipzig, Germany: Teubner, 1987.
49. G. L. Gilardoni, "An improvement on Vajda's inequality," in Progr. Probabl., V. Sidoravicius and M. E. Vares, Eds. Basel: Birkhäuser, 2008, vol.60.