α-divergence is unique, belonging to both f-divergence and bregman divergence classes

IEEE Transactions on Information Theory

Volumn 55, Issue 11, 2009, Pages 4925-4931

  Amari, Shun Ichi ^a  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

Author keywords

Bregman divergence; Canonical divergence; Dually flat structure; f divergence; Fisher information; Information geometry; Information monotonicity

Indexed keywords

BREGMAN DIVERGENCE; CANONICAL DIVERGENCE; DUALLY FLAT STRUCTURE; F-DIVERGENCE; FISHER INFORMATION; INFORMATION GEOMETRY; INFORMATION MONOTONICITY;

FISHER INFORMATION MATRIX; PROBABILITY DENSITY FUNCTION; SIGNAL PROCESSING;

PROBABILITY DISTRIBUTIONS;

EID: 70350714498     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2009.2030485     Document Type: Article

References (27)

1. I. Csiszár, "Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems," Ann. Statist. vol. 19, pp. 2032-2066, 1991.
2. S. Amari and H. Nagaoka, Methods of Information Geometry. New York: Oxford Unive. Press, 2000.
3. I. Csiszár, "Eine Informationstheoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizität von Markoffschen Ketten," Magyar Tud. Akad. Mat. Kutat Int. Kzl, vol. 8, pp. 85-108, 1963.
4. I. Csiszár, "Information measures: A critical survey," in Proc. 7th Conf. Inf. Theory, Prague, Czech Republic, 1974, pp. 83-86.
5. I. Csiszár, "Axiomatic characterizations of information measures," Entropy, vol. 10, pp. 261-273, 2008.
6. L. Bregman, "The relaxation method of finding a common point of convex sets and its application to the solution of problems in convex programming," Comput. Math. Phys. USSR, vol. 7, pp. 200-217, 1967.
7. A. Banerjee, S. Merugu, I. S. Dhillon, and J. Ghosh, "Clustering with Bregman divergences," J. Mach. Learn. Res., vol. 6, pp. 1705-1749, 2005.
8. N. Murata, T. Takenouchi, T. Kanamori, and S. Eguchi, "Information geometry of U-boost and Bregman divergence," Neural Comput. vol. 26, pp. 1651-1686, 2004.
9. A. Cichocki, R. Adunek, A. H. Phan, and S. Amari, Nonnegative Matrix and Tensor Factorizations. New York: Wiley, 2009.
10. F. Nielsen, Ed., Emerging Trends in Visual Computing, ser. Lecture Notes in Computer Science. Berlin, Germany: Springer-Verlag, 2009, vol. 6.
11. M. S. Ali and S. D. Silvey, "A general class of coefficients of divergence of one distribution from another," J. R. Statist. Soc. B no. 28, pp. 131-142, 1966.
12. I. Taneja and P. Kumar, "Relative information of type s, Csiszár's f-divergence, and information inequalities," Inf. Sci., vol. 166, pp. 105-125, 2004.
13. J. Havrda and F. Charvát, "Quantification method of classification process. Concept of structural α-entropy," Kybernetika, vol. 3, pp. 30-35, 1967.
14. H. Chernoff, "A measure of asymptotic efficiency for tests of a hypothesis based on a sum of observations," Ann. Math. Statist., vol. 23, pp. 493-507, 1952.
15. A. Rényi, "On measures of entropy and information," in Proc. 4th Symp. Math. Statist. Probl., Berkeley, CA, 1961, vol. 1, pp. 547-561.
16. C. Tsallis, "Possible generalization of Boltzmann-Gibbs statistics," J. Statist. Phys., vol. 52, pp. 479-487, 1988.
17. N. N. Chentsov, Statistical Decision Rules and Optimal Inference. Providence, RI: American Mathematical Society, 1982.
18. D. Petz, "Monotone metrics on matrix spaces," Linear Algebra Appl. vol. 244, pp. 235-249, 1996.
19. S. Amari, "Information geometry and its applications: Convex function and dually flat manifold," in Emerging Trends in Visual Computing ser. Lecture Notes in Computer Science, F. Nielsen, Ed. Berlin, Germany: Springer-Verlag, 2009, vol. 5416, pp. 75-102.
20. J. Zhang, "Divergence function, duality, and convex analysis," Neural Comput., vol. 16, pp. 159-195, 2004.
21. S. Eguchi and J. Copas, "A class of logistic type discriminant function," Biometrika, vol. 89, pp. 1-22, 2002.
22. M. Minami and S. Eguchi, "Robust blind source separation by betadivergence," Neural Comput., vol. 14, pp. 1859-1886, 2004.
23. S. Amari, "Integration of stochastic models by minimizing α-divergence," Neural Comput., vol. 19, pp. 2780-2796, 2007.
24. Y. Matsuyama, "The.-EM algorithm: Surrogate likelihood maximization using α-logarithmic information measures," IEEE Trans. Inf. Theory vol. 49, no. 3, pp. 672-706, Mar. 2002.
25. S. Amari and A. Cichocki, "Information Geometry of Divergence Functions," Bull. Polish Acad. Sci., to be published.
26. A. Cichocki, R. Zdunek, A. H. Phan, and S. Amari, Non-Negative Matrix and Tensor Factorizations: Applications to Explanatory Multi-Way Data Analysis and Blind Source Separation. New York:WIley, 2009.
27. F. Nielsen and R. Noch, "Sided and symmetrized Bregman divergence," IEEE Trans. Inf. Theory, vol. 55, no. 6, pp. 2882-2904, Jun. 2009.