Information-theoretic distance measures for clustering validation: Generalization and normalization

IEEE Transactions on Knowledge and Data Engineering

Volumn 21, Issue 9, 2009, Pages 1249-1262

  Luo, Ping ^a,b   Xiong, Hui ^c   Zhan, Guoxing ^d   Wu, Junjie ^e   Shi, Zhongzhi ^f  

^a INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

^b HEWLETT PACKARD LABORATORIES   (United States)

^c RUTGERS UNIVERSITY   (United States)

^d Wayne State University   (United States)

^e BEIHANG UNIVERSITY   (China)

^f INSTITUTE OF COMPUTING TECHNOLOGY   (China)

Author keywords

Clustering validation; Entropy; Information theoretic distance measures; K means clustering

Indexed keywords

CLUSTERING VALIDATION; CLUSTERINGS; CONDITIONAL ENTROPY; CRITICAL CHALLENGES; DATA SETS; DISTANCE MEASURE; INFORMATION-THEORETIC DISTANCE; INFORMATION-THEORETIC DISTANCE MEASURES; K-MEANS; K-MEANS CLUSTERING; MAXIMUM VALUES; PARTITION CLUSTERING; REAL WORLD DATA;

CLUSTER ANALYSIS; ENTROPY; INFORMATION THEORY; WATER SUPPLY SYSTEMS;

CLUSTERING ALGORITHMS;

EID: 68549094312     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2008.200     Document Type: Article

References (26)

1. J. Aczl and Z. Darczy, On Measures of Information and Their Characterizations. Academic Press, 1975.
2. D. Barbará and P. Chen, "Using Self-Similarity to Cluster Large Data Sets," Data Mining and Knowledge Discovery, vol. 7, no. 2, pp. 123-152, 2003.
3. D. Barbará, Y. Li, and J. Couto, "Coolcat: An Entropy-Based Algorithm for Categorical Clustering," Proc. 11th ACM Int'l Conf. Information and Knowledge Management (CIKM '02), pp. 582-589, 2002.
4. L. Breiman, J. Friedman, C.J. Stone, and R.A. Olshen, Classification and Regression Trees. Wadsworth Int'l Group, 1984.
5. Y. Chen, Y. Zhang, and X. Ji, "Size Regularized Cut for Data Clustering," Proc. 18th Conf. Neural Information Processing Systems (NIPS), 2005.
6. M.H. DeGroot and M.J. Schervish, Probability and Statistics, third ed. Addison-Wesley, 2001.
7. M. Halkidi, Y. Batistakis, and M. Vazirgiannis, "On Clustering Validation Techniques," J. Intelligent Information Systems, vol. 17, nos. 2/3, pp. 107-145, 2001.
8. M. Halkidi, D. Gunopulos, N. Kumar, M. Vazirgiannis, and C. Domeniconi, "A Framework for Semi-Supervised Learning Based on Subjective and Objective Clustering Criteria," Proc. Fifth IEEE Int'l Conf. Data Mining (ICDM '05), pp. 637-640, 2005.
9. A.K. Jain and R.C. Dubes, Algorithms for Clustering Data. Prentice Hall, 1998.
10. S. Jaroszewicz, D.A. Simovici, W. Kuo, and L. Ohno-Machado, "The Goodman-Kruskal Coefficient and Its Applications in the Genetic Diagnosis of Cancer," IEEE Trans. Biomedical Eng., vol. 51, no. 7, pp. 1095-1102, July 2004.
11. I. Jonyer, D.J. Cook, and L.B. Holder, "Graph-Based Hierarchical Conceptual Clustering," J. Machine Learning Research, vol. 2, pp. 19-43, 2001.
12. I. Jonyer, L.B. Holder, and D.J. Cook, "Graph-Based Hierarchical Conceptual Clustering in Structural Databases," Proc. 17th Nat'l Conf. Artificial Intelligence and 12th Conf. Innovative Applications of Artificial Intelligence (AAAI '00), p. 1078, 2000.
13. G. Karypis, http://glaros.dtc.umn.edu/gkhome/views/cluto, 2008.
14. J. Li, D. Tao, W. Hu, and X. Li, "Kernel Principle Component Analysis in Pixels Clustering," Proc. IEEE/WIC/ACM. Int'l Conf. Web Intelligence (WI '05), pp. 786-789, 2005.
15. W. Li, W.K. Ng, Y. Liu, and K.-L. Ong, "Enhancing the Effectiveness of Clustering with Spectra Analysis," IEEE Trans. Knowledge and Data Eng., vol. 19, no. 7, pp. 887-902, July 2007.
16. P. Luo, G. Zhan, Q. He, Z. Shi, and K. Lü, "On Defining Partition Entropy by Inequalities," IEEE Trans. Information Theory, vol. 53, no. 9, pp. 3233-3239, 2007.
17. R. Lopez De Mantaras, "A Distance-Based Attribute Selection Measure for Decision Tree Induction," Machine Learning, vol. 6, no. 1, pp. 81-92, 1991.
18. A.W. Marshall and I. Olkin, Inequalities: Theory of Majorization and Its Applications. Academic Press, 1979.
19. M. Meila, "Comparing Clusterings: An Axiomatic View," Proc. 22nd Int'l Conf. Machine Learning (ICML '05), pp. 577-584, 2005.
20. N.R. Pal and S.K. Pal, "Entropy: A New Definition and Its Applications," IEEE Trans. Systems Man and Cybernetics, vol. 21, no. 5, pp. 1260-1270, 1991.
21. M.H. Protter and C.B. Morrey Jr., A First Course in Real Analysis, second ed. Springer, 1991.
22. C.E. Shannon, "A Mathematical Theory of Communication," Bell System Technical J., vol. 27, pp. 379-423, 623-656, 1948.
23. D.A. Simovici and S. Jaroszewicz, "An Axiomatization of Partition Entropy," IEEE Trans. Information Theory, vol. 48, no. 7, pp. 2138-2142, 2002.
24. D.A. Simovici and S. Jaroszewicz, "A Metric Approach to Building Decision Trees Based on Goodman-Kruskal Association Index," Proc. Eighth Pacific-Asia Conf. Knowledge Discovery and Data Mining (PAKDD '04), pp. 181-190, 2004.
25. P.-N. Tan, M. Steinbach, and V. Kumar, Introduction to Data Mining. Addison-Wesley, 2005.
26. H. Xiong, J. Wu, and J. Chen, "K-Means Clustering Versus Validation Measures: A Data Distribution Perspective," Proc. 12th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining (KDD '06), pp. 779-784, 2006.