On asymptotic properties of information-theoretic divergences

IEEE Transactions on Information Theory

Volumn 49, Issue 7, 2003, Pages 1860-1868

  Pardo, María Del Carmen ^a   Vajda, Igor ^b  

^a UNIVERSIDAD COMPLUTENSE DE MADRID   (Spain)

^b INSTITUTE OF INFORMATION THEORY AND AUTOMATION   (Czech Republic)

Author keywords

Asymptotic distributions; Asymptotic equivalence; Bregman divergences; Burbea Rao divergences; Divergence statistics; Divergences of Csisz r

Indexed keywords

ASYMPTOTIC STABILITY; EQUIVALENCE CLASSES; PROBABILITY DISTRIBUTIONS; RANDOM PROCESSES; STATISTICS; THEOREM PROVING;

ASYMPTOTIC DISTRIBUTIONS; ASYMPTOTIC EQUIVALENCE; BREGMAN DIVERGENCES; BURBEA-RAO DIVERGENCES; DIVERGENCES OF CSISZAR; INFORMATION-THEORETIC DIVERGENCES;

INFORMATION THEORY;

EID: 0038444419     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2003.813509     Document Type: Letter

References (26)

1. L. M. Bregman, "The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming," USSR Comput. Math. and Math. Phys., vol. 7, pp. 200-217, 1967.
2. J. Burbea and C. R. Rao, "On the convexity of some divergence measures based on entropy functions," Trans. IEEE Inform. Theory, vol. IT-28, pp. 489-495, May 1982.
3. T. M. Cover and J. B. Thomas, Elements of Information Theory. New York: Wiley, 1991.
4. I. Csiszár, "Eine Informationstheoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizität von Markoffschen Ketten," Publ. Math. Inst. Hungar. Acad. Sci., Ser. A, vol. 8, pp. 85-108, 1963.
5. ____, "Information-type measure of difference of probability distributions and indirect observations," Studia Sci. Math. Hungar., vol. 2, pp. 299-318, 1967.
6. ____, "Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems," Ann. Statist., vol. 19, pp. 2031-2066, 1991.
7. ____, "Maximum entropy and related methods," in Trans. 12th Prague Conf. Information Theory, Statistical Decision Functions and Random Processes. Prague, Czech Rep.: Czech Acad. Sci., 1994, pp. 58-62.
8. R. M. Gray, Entropy and Information Theory. New York: Springer-Verlag, 1990.
9. L. Györfi, G. Morvai, and I. Vajda, "Information-theoretic methods in testing the goodness-of-fit," in Proc. Int. Symp. Information Theory (ISIT 2000), Sorrento, Italy, June 25-30, 2000, p. 28.
10. L. Györfi and I. Vajda, "A class of modified Pearson and Neyman statistics," Statist. Decisions, vol. 19, pp. 239-251, 2001.
11. ____, "Asymptotic distributions for goodness-of-fit statistics in a sequence of multinomial models," Statist. Probab. Lett., vol. 56, pp. 57-67, 2002.
12. G. H. Hardy, J. E. Littlewood, and G. Pólya, Inequalities, 2nd ed. London, U.K.: Cambridge Univ. Press, 1952.
13. T. Inglot, T. Jurlewicz, and T. Ledwina, "Asymptotics for multinomial goodness of fit tests for simple hypothesis," Theory Probab. and Applications, vol. 35, pp. 771-777, 1990.
14. L. Le Cam, Asymptotic Methods in Statistical Decision Theory. New York: Springer-Verlag, 1986.
15. F. Liese and I. Vajda, Convex Statistical Distances. Leipzig, Germany: Teubner, 1987.
16. A. W. Marshall and I. Olkin, Inequalities: Theory of Majorization and Its Applications. New York: Academic, 1979.
17. M. L. Menéndez, D. Morales, L. Pardo, and I. Vajda, "Two approaches to grouping of data and related disparity statistics," Commun. Statist. Theory Methods, vol. 27, pp. 609-633, 1998.
18. M. C. Pardo, "On Burbea-Rao divergence based goodness-of-fit tests for multinomial models," J. Multivariate Anal., vol. 69, pp. 65-87, 1999.
19. h-divergence statistics in applied categorical data analysis with stratified sampling," Utilitas Math., vol. 44, 145-164, 1993.
20. M. C. Pardo and I. Vajda, "About distances of discrete distributions satisfying the data processing theorem of information theory," Trans. IEEE Inform. Theory, vol. 43, pp. 1288-1293, July 1997.
21. T. R. C. Read and N. Cressie, Goodness of Fit Statistics for Discrete Multivariate Data. New York: Springer-Verlag, 1988.
22. S. T. Rachev, "On a class of minimum functionals in a space of probability measures," Theory Probab. and Applications, vol. 29, pp. 41-48, 1984.
23. R. J. Serfling, Approximation Theorems of Mathematical Statistics. New York: Wiley, 1980.
24. S. H. Tumanyan, "On the asymptotic distribution of the chi-square criterion," Drkl. Akad. Nauk SSSR, vol. 94, pp. 1011-1012, 1954.
25. I. Vajda, Theory of Statistical Inference and Information. Boston, MA: Kluwer, 1989.
26. K. Zografos, K. Ferentinos, and T. Papaioannou, "φ-divergence statistics: Sampling properties and multinomial goodness of fit and divergence tests," Commun. Statist. Theory Methods, vol. 19, pp. 1785-1802, 1990.