The Burbea-Rao and Bhattacharyya centroids

IEEE Transactions on Information Theory

Volumn 57, Issue 8, 2011, Pages 5455-5466

  Nielsen, Frank ^a,b   Boltz, Sylvain ^b  

^a SONY COMPUTER SCIENCE LABORATORIES INC   (Japan)

^b ÉCOLE POLYTECHNIQUE   (France)

Author keywords

Bhattacharrya divergence; Bregman divergences; Burbea Rao divergence; centroid; exponential families; information geometry; Jensen Shannon divergence; Kullback Leibler divergence

Indexed keywords

CLUSTER ANALYSIS; CLUSTERING ALGORITHMS; CONVEX OPTIMIZATION; GEOMETRY; INFORMATION THEORY; ITERATIVE METHODS;

BHATTACHARRYA DIVERGENCE; BREGMAN DIVERGENCES; BURBEA-RAO DIVERGENCE; CENTROID; EXPONENTIAL FAMILY; INFORMATION GEOMETRY; JENSEN-SHANNON DIVERGENCE; KULLBACK LEIBLER DIVERGENCE;

PROBABILITY DISTRIBUTIONS;

EID: 79961015164     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2011.2159046     Document Type: Review

References (45)

1. A. N. Kolmogorov, "Sur la notion de la moyenne," Accad. Naz. Lincei Mem. Cl. Sci. Fis. Mat. Natur. Sez., vol. 12, pp. 388-391, 1933.
2. M. Nagumo, "Über eine klasse der mittelwerte," Jap. J. Mathemat., vol. 7, pp. 71-79, 1930.
3. J. D. Aczěl, "On mean values," Bull. Amer. Mathemat. Soc., vol. 54, no. 4, pp. 392-400, 1948.
4. A. Ben-Tal, A. Charnes, and M. Teboulle, "Entropic means," J. Mathemat. Anal. and Applic., vol. 139, no. 2, pp. 537-551, 1989.
5. S. M. Ali and S. D. Silvey, "A general class of coefficients of divergence of one distribution from another," J. Roy. Statist. Soc., Ser. B, vol. 28, pp. 131-142, 1966.
6. I. Csiszár, "Information-type measures of difference of probability distributions and indirect observation," Studia Scient. Mathemat. Hungarica, vol. 2, pp. 229-318, 1967.
7. F. Nielsen and R. Nock, "Sided and symmetrized Bregman centroids," IEEE Trans. Inf. Theory, vol. 55, no. 6, pp. 2048-2059, Jun. 2009.
8. Y. Censor and S. A. Zenios, Parallel Optimization: Theory, Algorithms, and Applications. Oxford, U.K.: Oxford University Press, 1997.
9. M. J. Wainwright and M. I. Jordan, "Graphical models, exponential families, and variational inference," Found. Trends in Mach. Learn., vol. 1, pp. 1-305, Jan. 2008.
10. S.-I. Amari, "r-divergence is unique, belonging to both f-divergence and bregman divergence classes," IEEE Trans. Inf. Theor., vol. 55, no. 11, pp. 4925-4931, Nov. 2009.
11. S.-I. Amari, "Integration of stochastic models by minimizing κ -divergence," Neural Comput., vol. 19, no. 10, pp. 2780-2796, 2007.
12. S. Amari and H. Nagaoka, Methods of Information Geometry, A. M. Society, Ed. Oxford, U.K.: Oxford University Press, 2000.
13. F. Nielsen and R. Nock, "The dual voronoi diagrams with respect to representational bregman divergences," in Proc. Int. Symp. Voronoi Diagrams (ISVD), Copenhagen, Denmark, Jun. 2009.
14. J.-D. Boissonnat, F. Nielsen, and R. Nock, "Bregman Voronoi diagrams," Discrete & Comput. Geom., 2010.
15. I. Csiszár, "Generalized projections for non-negative functions," Acta Mathematica Hungarica, vol. 68, no. 1-2, pp. 161-185, 1995.
16. A. Banerjee, S. Merugu, I. S. Dhillon, and J. Ghosh, "Clustering with Bregman divergences," J. Mach. Learn. Res., vol. 6, pp. 1705-1749, 2005.
17. M. Detyniecki, "Mathematical Aggregation Operators and Their Application to Video Querying," Ph. D. dissertation, Universitě Pierre et Marie Curie, Paris, France, 2000.
18. J. Burbea and C. R. Rao, "On the convexity of some divergence measures based on entropy functions," IEEE Trans. Inf. Theory, vol. IT-28, no. 3, pp. 489-495, 1982. (Pubitemid 12524012)
19. J. Burbea and C. R. Rao, "On the convexity of higher order Jensen differences based on entropy functions," IEEE Trans. Inf. Theory, vol. IT-28, no. 6, p. 961, 1982.
20. J. Lin, "Divergence measures based on the Shannon entropy," IEEE Trans. Inform. Theory, vol. 37, no. 1, pp. 145-151, Jan. 1991.
21. M. Basseville and J.-F. Cardoso, "On entropies, divergences and mean values," in Proc. IEEEWorkshop on Information Theory,Whistler, BC, Canada, 1995, p. 330.
22. L. M. Bregman, "The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming," USSR Comput. Mathemat. and Mathemat. Phys., vol. 7, pp. 200-217, 1967.
23. F. Nielsen and V. Garcia, "Statistical exponential families: A digest with flash cards," ArXiv.org:0911.4863, 2009.
24. K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed. New York: Academic Press, 1990.
25. L. Rigazio, B. Tsakam, and J. C. Junqua, "An optimal bhattacharyya centroid algorithm for gaussian clustering with applications in automatic speech recognition," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, 2000, vol. 3, pp. 1599-1602 [Online]. Available: http://ieeexplore.ieee.org/xpls/abs-all.jsp?arnumber=861998
26. V. Garcia, F. Nielsen, and R. Nock, "Hierarchical gaussian mixture model," in Proc. Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), Mar. 2010.
27. P. Chen, Y. Chen, and M. Rao, "Metrics defined by Bregman divergences: Part I," Commun. Math. Sci., vol. 6, pp. 9915-926, 2008.
28. P. Chen, Y. Chen, and M. Rao, "Metrics defined by Bregman divergences: Part II," Commun. Math. Sci., vol. 6, pp. 927-948, 2008.
29. H. Jeffreys, "An invariant form for the prior probability in estimation problems," Proc. Roy. Soc. London, vol. 186, no. 1007, pp. 453-461, Mar. 1946.
30. F. Liese and I. Vajda, "On divergences and informations in statistics and information theory," IEEE Trans. Inf. Theory, vol. 52, no. 10, pp. 4394-4412, Oct. 2006. (Pubitemid 46445281)
31. J. Zhang, "Divergence function, duality, and convex analysis," Neur. Comput., vol. 16, no. 1, pp. 159-195, 2004. (Pubitemid 37528419)
32. A. Yuille and A. Rangarajan, "The concave-convex procedure," Neur. Comput., vol. 15, no. 4, pp. 915-936, 2003. (Pubitemid 37049812)
33. B. Sriperumbudur and G. Lanckriet, "On the convergence of the concave- convex procedure," Neur. Inf. Process. Syst., 2009.
34. Y. He, A. B. Hamza, and H. Krim, "An information divergence measure for ISAR image registration," in Autom. Target Recognit. XI (SPIE), 2001, vol. 4379, pp. 199-208.
35. A. O. Hero, B. Ma, O. Michel, and J. D. Gorman, Alpha-divergence for classification, indexing and retrieval Comm. and Sig. Proc. Lab. (CSPL), Dept. EECS, University of Michigan, Ann Arbor, MI, 2001, Tech. Rep. 328.
36. A. Bhattacharyya, "On a measure of divergence between two statistical populations defined by their probability distributions," Bull. Calcutta Mathemat. Soc., vol. 35, pp. 99-110, 1943.
37. K. Matusita, "Decision rules based on the distance, for problems of fit, two samples, and estimation," Ann. Mathemat. and Statist., vol. 26, pp. 631-640, 1955.
38. T. Kailath, "The divergence and Bhattacharyya distance measures in signal selection," IEEE Trans. Commun. Technol., vol. CT-15, no. 1, pp. 52-60, Jan. 1967.
39. F. Aherne, N. Thacker, and P. Rockett, "The Bhattacharyya metric as an absolute similarity measure for frequency coded data," Kybernetika, vol. 34, no. 4, pp. 363-368, 1998. (Pubitemid 128429890)
40. E. D. Hellinger, "Die Orthogonalinvarianten Quadratischer Formen von Unendlich Vielen Variablen," , Unov. Göttingen, Göttingen, Germany, 1907.
41. S. Kakutani, "On equivalence of infinite product measures," Ann. Mathemat., vol. 49, no. 214-224, 1948.
42. L. Rigazio, B. Tsakam, and J. Junqua, "Optimal Bhattacharyya centroid algorithm for Gaussian clustering with applications in automatic speech recognition," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, 2000, vol. 3, pp. 1599-1602.
43. K. B. Petersen and M. S. Pedersen, The Matrix Cookbook Technical Univ. Denmark, oct. 2008 [Online]. Available: http://www2.imm. dtu.dk/pubdb/p.php?3274
44. A. Cichocki and S. i. Amari, "Families of alpha- beta- and gammadivergences: Flexible and robust measures of similarities," Entropy, 2010.
45. P. Vos, "Geometry of divergence," Ann. Inst. Statist. Mathematics, vol. 43, no. 3, pp. 515-537, 1991.