Information Theoretic Hierarchical Clustering

Entropy

Volumn 13, Issue 2, 2011, Pages 450-465

  Aghagolzadeh, Mehdi ^a,b   Soltanian Zadeh, Hamid ^a,c,d   Araabi, Babak Nadjar ^a,d  

^a UNIVERSITY OF TEHRAN   (Iran)

^b Michigan State University   (United States)

^c HENRY FORD HEALTH SYSTEM   (United States)

^d INSTITUTE FOR RESEARCH IN FUNDAMENTAL SCIENCES IPM   (Iran)

Author keywords

Hierarchical clustering; Information theory; Proximity measure; Quadratic mutual information; R nyi s entropy

Indexed keywords

EID: 80052277310     PISSN: None     EISSN: 10994300     Source Type: Journal    
DOI: 10.3390/e13020450     Document Type: Article

References (32)

1. Mahalanobis, P.C. On the generalized distance in statistics. Proc. Natl. Inst. Sci. India 1936, 12, 49–55.
2. Hartigan, J.A.; Wong, M.A. A k-means clustering algorithm. JR Stat. Soc. Ser. C-Appl. Stat. 1979, 28, 100–108.
3. Bezdek, J.C.; Ehrlich, R. FCM: The fuzzy c-means clustering algorithm. Comput. Geosci. 1984, 10, 191–203.
4. Johnson, S.C. Hierarchical clustering schemes. Psychometrika 1967, 32, 241–254.
5. Jain, A.K.; Mao, J.; Mohiuddin, K.M. Artificial neural networks: A tutorial. Computer 1996, 29, 31–44.
6. Cristianini, N.; Shawe-Taylor, J. An Introduction to Support Vector Machines: and Other Kernel-based Learning Methods; Cambridge University Press: Cambridge, UK, 2000.
7. Davis, J.V.; Kulis, B.; Jain, P.; Sra, S.; Dhillon, I.S. Information-theoretic metric learning. In Proceedings of the 24th international conference on Machine learning (ICML ’07), New York, NY, USA, 2007; pp. 209–216.
8. Chaudhuri, K.; McGregor, A. Finding metric structure in information theoretic clustering. In Conference on Learning Theory, COLT, Helsinki, Finland, July 2008.
9. Roweis, S.; Ghahramani, Z. A unifying review of linear Gaussian models. Neural Comput. 1999, 11, 305–345.
10. Parzen, E. On estimation of a probability density function and mode. Ann. Math. Statist. 1962, 33, 1065–1076.
11. Kraskov, A.; Grassberger, P. MIC: mutual information based hierarchical clustering. In Information Theory Statistical Learning; Springer: New York, NY, USA, 2008, pp. 101–123.
12. Zhou, X.; Wang, X.; Dougherty, E.R.; Russ, D.; Suh, E. Gene clustering based on clusterwide mutual information. J. Comput. Biol. 2004, 11, 147–161.
13. Rényi, V. On measures of information and entropy. Proceedings of the 4th Berkeley Symposium on Mathematics, Statistics and Probability, Berkeley: University of California Press, California, USA, 1961; pp. 547–561.
14. Principe, J.C.; Xu, D.; Fisher, J. Information theoretic learning. In Unsupervised Adaptive Filtering, Haykin, S., Ed.; Wiley: New York, NY, USA, 2000; pp. 265–319.
15. Gokcay, E.; Principe, J.C. Information theoretic clustering. IEEE Trans. Patt. Anal. Mach. Int. 2002, 24, 158–171.
16. Torkkola, K. Feature extraction by non parametric mutual information maximization. J. Mach. Learn. Res. 2003, 3, 1438.
17. Sullivan, G.J.; Wiegand, T. Rate-distortion optimization for video compression. Signal Process. Mag. IEEE 2002, 15, 74–90.
18. Shannon, C.E. A mathematical theory of communication. Bell Syst. Tech. J. 1948, 27, 379–423.
19. Cover, T.M.; Thomas, J.A. Elements of Information Theory; 2nd ed.; Wiley: New York, NY, USA, 2006.
20. Kapur, J.N. Measures of Information and Their Applications; Wiley: New Delhi, India, 1994.
21. Vincent, P.; Bengio, Y. Manifold parzen windows. Adv. Neural Inf. Process. Syst. 2003, 15, 849–856.
22. Davis, J.V.; Dhillon, I. Differential entropic clustering of multivariate gaussians. Adv. Neural Inf. Process. Syst. 2007, 19, 337.
23. Karypis, G.; Han, E.H.; Kumar, V. Chameleon: Hierarchical clustering using dynamic modeling. Computer 1999, 32, 68–75.
24. Guha, S.; Rastogi, R.; Shim, K. Cure: An efficient clustering algorithm for large databases. Inf. Syst. 2001, 26, 35–58.
25. Duda, R.O.; Hart, P.E. Pattern classification and scene analysis. Wiley, New York, NY, USA, 1973.
26. Silverman, B.W. Density Estimation for Statistics and Data Analysis; Chapman & Hall/CRC: London, England,UK, 1998.
27. Anderson, E. The irises of the Gaspe Peninsula. Bull. Am. IRIS Soc. 1935, 59, 2–5.
28. Blatt, M.; Wiseman, S.; Domany, E. Superparamagnetic clustering of data. Phys. Rev. Lett. 1996, 76, 3251–3254.
29. Jenssen, R.; Erdogmus, D.; Ii, K.; Principe, J.; Eltoft, T. Information force clustering using directed trees. Lect. Note. Comput. Sci. 2003, 2683, 68–72.
30. Aeberhard, S.; Coomans, D.; De Vel, O. Comparison of classifiers in high dimensional settings. Tech. Rep. 1992, 92–102.
31. Sugar, C.A.; James, G.M. Finding the number of clusters in a dataset. J. Am. Statist. Assn. 2003, 98, 750–763.
32. Jenssen, R.; Hild, K.E.; Erdogmus, D.; Principe, J.C.; Eltoft, T. Clustering using Renyi’s entropy. Neural Networks 2003, 1, 523–528.