Cluster ensembles - A knowledge reuse framework for combining multiple partitions

Journal of Machine Learning Research

Volumn 3, Issue 3, 2003, Pages 583-617

  Strehl, Alexander ^a   Ghosh, Joydeep ^a  

^a The University of Texas at Austin   (United States)

Author keywords

Cluster analysis; Clustering; Consensus functions; Ensemble; Knowledge reuse; Multi learner systems; Mutual information; Partitioning; Unsupervised learning

Indexed keywords

ALGORITHMS; DATA REDUCTION; INFORMATION ANALYSIS; OPTIMIZATION; REAL TIME SYSTEMS;

CLUSTER ENSEMBLES;

LEARNING SYSTEMS;

EID: 0041965980     PISSN: 15324435     EISSN: None     Source Type: Journal    
DOI: 10.1162/153244303321897735     Document Type: Conference Paper

References (64)

1. C. J. Alpert and A. B. Kahng. Recent directions in netlist partitioning: A survey. Integration: The VLSI Journal, 19:1-18, 1995.
2. J.A. Barnett. Computational methods for a mathematical theory of evidence. In Proc. of IJCAI, pages 868-875, 1981.
3. J. P. Barthelemy, B. Laclerc, and B. Monjardet. On the use of ordered sets in problems of comparison and consensus of classifications. Journal of Classification, 3:225-256, 1986.
4. A. Blum and T. Mitchell. Combining labeled and unlabeled data with co- training. In Proceedings of the 11th Annual Conference on Computational Learning Theory (COLT-98), pages 92-100, 1998.
5. D. Boley, M. Gini, R. Gross, E. Han, K. Hastings, G. Karypis, V. Kumar, B. Mobasher, and J. Moore. Partitioning-based clustering for web document categorization. Decision Support Systems, 27:329-341, 1999.
6. Kurt D. Bollacker and Joydeep Ghosh. A supra-classifier architecture for scalable knowledge reuse. In Proc. Int'l Conf. on Machine Learning (ICML-98), pages 64-72, July 1998.
7. Kurt D. Bollacker and Joydeep Ghosh. Effective supra-classifiers for knowledge base construction. Pattern Recognition Letters, 20(11-13):1347-52, November 1999.
8. P.S. Bradley and U. M. Fayyad. Refining initial points for K-means clustering. In Proc. Int'l Conf. on Machine Learning (ICML-98), pages 91-99, July 1998.
9. Rich Caruana. Learning many related tasks at the same time with backpropagation. In Advances in Neural Information Processing Systems 7, pages 657-664, 1995.
10. S. V. Chakaravathy and J. Ghosh. Scale based clustering using a radial basis function network. IEEE Transactions on Neural Networks, 2(5):1250-61, Sept 1996.
11. Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley, 1991.
12. B. Dasarathy. Decision Fusion. IEEE CS Press, Los Alamitos, CA, 1994.
13. I. S. Dhillon and D. S. Modha. Concept decompositions for large sparse text data using clustering. Machine Learning, 42(1):143-175, January 2001.
14. T. G. Dietterich. Ensemble methods in machine learning. In J. Kittler and F. Roll, editors, Multiple Classifier Systems, pages 1-15. LNCS Vol. 1857, Springer, 2001.
15. M. Ester, H. Kriegel, J. Sander, and X. Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of 2nd International Conference on KDD, pages 226-231, 1996.
16. U. M. Fayyad, C. Reina, and P. S. Bradley. Initialization of iterative refinement clustering algorithms. In Proc. 14th Intl. Conf. on Machine Learning (ICML), pages 194-198, 1998.
17. Doug Fisher. Iterative optimization and simplification of hierarchical clusterings. Journal of Artificial Intelligence Research, 4:147-180, 1996.
18. W. Frakes. Stemming algorithms. In W. Frakes and R. Baeza-Yates, editors, Information Retrieval: Data Structures and Algorithms, pages 131-160. Prentice Hall, New Jersey, 1992.
19. A. L. N. Fred and A. K. Jain. Data clustering using evidence accumulation. In Proc. ICPR, page to appear, 2002.
20. N. Friedman, O. Mosenzon, N. Slonim, and N. Tishby. Multivariate information bottleneck. In Proc. of the Seventeenth Conf. on Uncertainty in Artificial Intelligence (UAI). AAAI Press, 2001.
21. Michael R. Garey and David S. Johnson. Computers and Intractability: A Guide to the Theory of NP-completeness. W. H. Freeman, San Francisco, CA, 1979.
22. J. Ghosh. Multiclassifier systems: Back to the future. Keynote Talk, 3rd Int'l Workshop on Multiple Classifier Systems, Cagliari, June, 2002a. Downloadable from http://www.lans.ece.utexas.edu/publications.html.
23. J. Ghosh. Multiclassifier systems: Back to the future (invited paper). In F. Roli and J. Kittler, editors, Multiple Classifier Systems, pages 1-15. LNCS Vol. 2364, Springer, 2002b.
24. J. Ghosh, A. Strehl, and S. Merugu. A consensus framework for integrating distributed clusterings under limited knowledge sharing. In Proc. NSF Workshop on Next Generation Data Mining, Baltimore, pages 99-108, Nov 2002.
25. C. W. J. Granger. Combining forecasts-twenty years later. Journal of Forecasting, 8(3): 167-173, 1989.
26. E. Han, G. Karypis, V. Kumar, and B. Mobasher. Clustering in a highdimensional space using hypergraph models. Technical Report 97-019, University of Minnesota, Department of Computer Science, 1997.
27. A. K. Jain and R. C. Dubes. Algorithms for Clustering Data. Prentice Hall, New Jersey, 1988.
28. R. A. Jarvis and E. A. Patrick. Clustering using a similarity measure based on shared nearest neighbors. IEEE Transactions on Computers, C-22, No. 11:1025-1034, 1973.
29. E. Johnson and H. Kargupta. Collective, hierarchical clustering from distributed, heterogeneous data. In M. Zaki and C. Ho, editors, Large-Scale Parallel KDD Systems, volume 1759 of Lecture Notes in Computer Science, pages 221-244. Springer-Verlag, 1999.
30. S. Kannan, T. Warnow, and S. Yooseph. Computing the local consensus of trees. In Association for Computing Machinery and the Society of Industrial Applied Mathematics, Proceedings, ACM/SIAM Symposium on Discrete Algorithms, pages 68-77, 1995.
31. H. Kargupta and P. Chan, editors. Advances in Distributed and Parallel Knowledge Discovery. AAAI/MIT Press, Cambridge, MA, 2000.
32. H. Kargupta, W. Huang, Krishnamoorthy, and E. Johnson. Distributed clustering using collective principal component analysis. Knowledge and Information Systems Journal Special Issue on Distributed and Parallel Knowledge Discovery, 3:422-448, 2001.
33. H. Kargupta, B. Park, D. Hershberger, and E. Johnson. Collective data mining: A new perspective toward distributed data mining. In Hillol Kargupta and Philip Chan, editors, Advances in Distributed and Parallel Knowledge Discovery. MIT/AAAI Press, 1999.
34. G. Karypis, E.-H. Han, and V. Kumar. Chameleon: Hierarchical clustering using dynamic modeling. IEEE Computer, 32(8):68-75, August 1999.
35. G. Karypis and V. Kumar. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on Scientific Computing, 20(1):359-392, 1998.
36. George Karypis, Rajat Aggarwal, Vipin Kumar, and Shashi Shekhar. Multilevel hypergraph partitioning: Applications in VLSI domain. In Proceedings of the Design and Automation Conference, 1997.
37. Branko Kavšek, Nada Lavrač, and Anuška Ferligoj. Consensus decision trees: Using consensus hierarchical clustering for data relabelling and reduction. In Proceedings of ECML 2001, volume 2167 of LNAI, pages 251-262. Springer, 2001.
38. B. Kernighan and S. Lin. An efficient heuristic procedure for partitioning graphs. Bell Systems Technical Journal, 49:291-307, 1970.
39. J. Kim and T. Warnow. Tutorial on phylogenetic tree estimation. In Intelligent Systems for Molecular Biology, Heidelberg, 1999.
40. J. Kittler and F. Roli, editors. Multiple Classifier Systems. LNCS Vol. 2634, Springer, 2002.
41. Teuvo Kohonen. Self-Organizing Maps. Springer, Berlin, Heidelberg, 1995. (Second Extended Edition 1997).
42. A. Krogh and J. Vedelsby. Neural network ensembles, cross validation and active learning. In D.S. Touretzky G. Tesauro and T.K. Leen, editors, Advances in Neural Information Processing Systems-7, pages 231-238, 1995.
43. Mala Mehrotra. Multi-viewpoint clustering analysis (mvp-ca) technology for mission rule set development and case-based retrieval. Technical Report AFRL-VS-TR-1999-1029, Air Force Research Laboratory, 1999.
44. Dharmendra S. Modha and W. Scott Spangler. Clustering hypertext with applications to web searching. In Proceedings of the ACM Hypertext 2000 Conference, San Antonio, TX, May 30-June 3, 2000.
45. Ion Muslea, Steve Minton, and Craig Knoblock. Selective sampling + semi-supervised learning = robust multi-view learning. In IJCAI-2001 Workshop on Text Learning Beyond Supervision, 2001.
46. D. A. Neumann and V. T. Norton. Clustering and isolation in the consensus problem for partitions. Journal of Classification, 3:281-298, 1986a.
47. D. A. Neumann and V. T. Norton. On lattice consensus methods. Journal of Classification, 3:225-256, 1986b.
48. K. Nigam and R. Ghani. Analyzing the applicability and effectiveness of co-training. In Proceedings of CIKM-00, 9th ACM International Conference on Information and Knowledge Management, pages 86-93. ACM, 2000.
49. E. Pekalska, P. Paclik, and R.P.W. Duin. A generalized kernel approach to dissimilarity-based classification. Journal of Machine Learning Research, Special Issue on Kernel Methods, 2(2):175-211, 2002.
50. Lorien Y. Pratt. Experiments on the transfer of knowledge between neural networks. In S. Hanson, G. Drastal, and R. Rivest, editors, Computational Learning Theory and Natural Learning Systems, Constraints and Prospects, chapter 19, pages 523-560. MIT Press, 1994.
51. A. Prodromidis, P. Chan, and S. Stolfo. Meta-learning in distributed data mining systems: Issues and approaches. In H. Kargupta and P. Chan, editors, Advances in Distributed and Parallel Knowledge Discovery. AAAI/MIT Press, Cambridge, MA, 2000.
52. M.D. Richard and R.P. Lippmann. Neural network classifiers estimate bayesian a posteriori probabilities. Neural Computation, 3(4):461-483, 1991.
53. A. Sharkey. On combining artificial neural networks. Connection Science, 8(3/4):299-314, 1996.
54. A. Sharkey. Combining Artificial Neural Nets. Springer-Verlag, 1999.
55. D. Silver and R. Mercer. The parallel transfer of task knowledge using dynamic learning rates based on a measure of relatedness. Connection Science Special Issue: Transfer in Inductive Systems, 1996.
56. N. Slonim and N. Tishby. Agglomerative information bottleneck. In Proc. of NIPS-12, pages 617-623. MIT Press, 2000.
57. A. Strehl and J. Ghosh. Cluster ensembles - a knowledge reuse framework for combining partitionings. In Proceedings of AAAI 2002, Edmonton, Canada, pages 93-98. AAAI, July 2002a.
58. Alexander Strehl and Joydeep Ghosh. A scalable approach to balanced, high-dimensional clustering of market-baskets. In Proc. HiPC 2000, Bangalore, volume 1970 of LNCS, pages 525-536. Springer, December 2000.
59. Alexander Strehl and Joydeep Ghosh. Relationship-based clustering and visualization for high-dimensional data mining. INFORMS Journal on Computing, 2002b. in press.
60. Alexander Strehl, Joydeep Ghosh, and Raymond Mooney. Impact of similarity measures on web-page clustering. In Proc. AAAI Workshop on AI for Web Search (AAAI 2000), Austin, pages 58-64. AAAI, July 2000.
61. S. Thrun. Is learning the n-th thing any easier than learning the first? In M.C. Mozer D.S. Touretzky and M. E. Hasselmo, editors, Advances in Neural Information Processing Systems-8, pages 640-646. MIT Press, Cambridge, MA, 1996.
62. S. Thrun and L.Y. Pratt. Learning To Learn. Kluwer Academic, Norwell, MA, 1997.
63. Sebastian Thrun and Joseph O'Sullivan. Discovering structure in multiple learning tasks: The TC alogorithm. In The 13th International Conference on Machine Learning, pages 489-497, 1996.
64. K. Turner and J. Ghosh. Linear and order statistics combiners for pattern classification. In A. Sharkey, editor, Combining Artificial Neural Nets, pages 127-162. Springer-Verlag, 1999.