A spectral clustering based ensemble pruning approach

Neurocomputing

Volumn 139, Issue , 2014, Pages 289-297

  Zhang, Huaxiang ^a,b   Cao, Linlin ^a,b  

^a SHANDONG NORMAL UNIVERSITY   (China)

^b Shandong Provincial Key Laboratory for Novel Distributed Computer Software Technology   (China)

Author keywords

Classifier similarity; Ensemble pruning; Spectral clustering

Indexed keywords

COMPUTER APPLICATIONS; NEURAL NETWORKS;

CLASSIFICATION ACCURACY; CLASSIFIER SELECTION; CLASSIFIER SIMILARITIES; ENSEMBLE CLASSIFIERS; ENSEMBLE PRUNING; HEURISTIC FUNCTIONS; PREDICTIVE ACCURACY; SPECTRAL CLUSTERING;

HEURISTIC ALGORITHMS;

ARTICLE; CLASSIFICATION ALGORITHM; CLUSTER ANALYSIS; CONTROLLED STUDY; INTERMETHOD COMPARISON; MATHEMATICAL COMPUTING; MEASUREMENT ACCURACY; PRIORITY JOURNAL; SPECTRAL CLUSTERING;

EID: 84901050845     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2014.02.030     Document Type: Article

References (55)

1. Bishop C.M. Pattern Recognition and Machine Learning 2006, Springer-Verlag New York Inc., New York, NY.
2. Hansen J.V. Combining predictors: comparison of five meta machine learning methods. Inf. Sci. 1999, 119(1-2):91-105.
3. Dietterich T. An experimental comparison of three methods for constructing ensembles of decision trees. bagging, boosting, and randomization. Mach. Learn. 2000, 40:139-158.
4. Kittler J., Hatef M., Duin R.P.W., Matas J. On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 1998, 20(3):226-239.
5. Breiman L. Bagging Predictors. Mach. Learn. 1996, 24:123-140.
6. Y. Freund, R. Schapire, Experiments with a new boosting algorithm, in: Proceedings of the 13th International Conference of Machine Learning, 1996, pp. 325-332.
7. Merz C.J. Using correspondence analysis to combine classifiers. Mach. Learn. 1999, 36(1):33-58.
8. Garca-Pedrajas N. Constructing ensembles of classifiers by means of weighted instance selection. IEEE Trans. Neural Netw. 2009, 20(2):258-277.
9. Windeatt T. Accuracy/diversity and ensemble MLP classifier design. IEEE Trans. Neural Netw. 2006, 17(5):1194-1211.
10. Bauer E., Kohavi R. An empirical comparison of voting classification algorithms. bagging, boosting, and variants. Mach. Learn. 1999, 36(1-2):105-139.
11. R. Caruana, A. Niculescu-Mizil, An empirical comparison of supervised learning algorithms, in: Proceedings of the 23rd International Conference of Machine Learning, 2006, pp. 161-168.
12. Rätsch G., Onoda T., Müller K.-R. Soft margins for adaboost. Mach. Learn. 2001, 42(3):287-320.
13. Zhou Z.-H., Wu J., Tang W. Ensembling neural networks. many could be better than all. Artif. Intell. 2002, 137(1-2):239-263.
14. Singh S., Singh M. A dynamic classifier selection and combination approach to image region labeling. Signal Process. Image Commun. 2005, 20(3):219-231.
15. Ulas A., Semerci M., Yildiz O.T., Alpaydin E. Incremental construction of classifier and discriminant ensembles. Inf. Sci. 2009, 179(9):1298-1318.
16. Martínez-Muñoz G., Hernáñdez-Lobato D., Suárez A. An analysis of ensemble pruning techniques based on ordered aggregation. IEEE Trans. Pattern Anal. Mach. Intell. 2009, 31(2):245-259.
17. Hernández-Lobato D., Martínez-Muñoz G., Suárez A. Statistical instance-based pruning in ensembles of independent classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 2009, 31(2):364-369.
18. Chen L., Kamel M.S. A generalized adaptive ensemble generation and aggregation approach for multiple classifier systems. Pattern Recognit. 2009, 42:629-644.
19. Martínez-Muñoz G., Suárez A. Using boosting to prune bagging ensembles. Pattern Recognit. Lett. 2007, 28(1):156-165.
20. Zhang Y., Burer S., Street W.N. Ensemble pruning via semi-definite programming. J. Mach. Learn. Res. 2006, 7:1315-1338.
21. Martínez-Muñoz G., Suárez A. Switching class labels to generate classification ensembles. Pattern Recognit. 2005, 38(10):1483-1494.
22. Banfield R.E., Hall L.O., Bowyer K.W., Kegelmeyer W.P. A comparison of decision tree ensemble creation techniques. IEEE Trans. Pattern Anal. Mach. Intell. 2007, 29(1):173-180.
23. Stefano C.D., Folino G., Fontanella F., Scotto di Freca A. Using Bayesian networks for selecting classifiers in GP ensembles. Inf. Sci. 2014, 258:200-216.
24. Kuncheva L. Switching between selection and fusion in combining classifiers: an experiment. IEEE Trans. Syst. Man Cybern. Part B 2002, 32(2):146-156.
25. Lysiak R., Kurzynski M., Woloszynski T. Optimal selection of ensemble classifiers using measures of competence and diversity of base classifiers. Neurocomputing 2014, 126:29-35.
26. G. Tsoumakas, I. Partalas, I. Vlahavas, A taxonomy and short review of ensemble selection, in: Proceedings of Workshop on Supervised and Unsupervised Ensemble Methods and their Applications, 2008, pp. 41-46.
27. Zhou H., Zhao X., Wang X. An effective ensemble pruning algorithm based on frequent patterns. Knowl. Based Syst. 2014, 56:79-85.
28. D.D. Margineantu, T.G. Dietterich, Pruning adaptive boosting, in: Proceedings of the 14th International Conference of Machine Learning, 1997, pp. 211-218.
29. G. Martínez-Muñoz, A. Suárez, Aggregation ordering in bagging, in: Proceedings of IASTED International Conference on Artificial Intelligence and Applications, 2004, pp. 258-263.
30. G. Martínez-Muñoz, A. Suárez, Pruning in Ordered Bagging Ensembles, in: Proceedings of the 23rd International Conference of Machine Learning, 2006, pp. 609-616.
31. Z.-H. Zhou, W. Tang, Selective ensemble of decision trees, in: Q. Liu, Y. Yao, A. Skowron (Eds.), Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing, Springer, 2003, pp. 476-483.
32. Dai Q. A novel ensemble pruning algorithm based on randomized greedy selective strategy and ballot. Neurocomputing 2013, 122:258-265.
33. Rokach L. Collective-agreement-based pruning of ensembles. Comput. Stat. Data Anal. 2009, 53:1015-1026.
34. Aksela M., Laaksonen J. Using diversity of errors for selecting members of a committee classifier. Pattern Recognit. 2006, 39:608-623.
35. Meynet J., Thiran J.-P. Information theoretic combination of pattern classifiers. Pattern Recognit. 2010, 43:3412-3421.
36. Xiao J., He C., Jiang X., Liu D. A dynamic classifier ensemble selection approach for noise data. Inf. Sci. 2010, 180:3402-3421.
37. Woods K., Kegelmeyer W.P., Bowyer K. Combination of multiple classifiers using local accuracy estimates. IEEE Trans. Pattern Anal. Mach. Intell. 1997, 19(4):405-410.
38. Giacinto G., Roli F. Dynamic classifier selection based on multiple classifier behavior. Pattern Recognit. 2001, 34(9):1879-1881.
39. Santos E.M.D., Sabourin R., Maupin P. A dynamic overproduce-and-choose strategy for the selection of classifier ensembles. Pattern Recognit. 2008, 41(10):2993-3009.
40. Zhu D. A hybrid approach for efficient ensembles. Decis. Support Syst. 2010, 48:480-487.
41. Bakker B., Heskes T. Clustering ensembles of neural network models. Neural Netw. 2003, 16:261-269.
42. Freund Y., Schapire R.E. A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. J. Comput. Syst. Sci. 1997, 55(1):119-139.
43. Verma B., Rahman A. Cluster-oriented ensemble classifier. impact of multicluster characterization on ensemble classifier learning. IEEE Trans. Knowl. Data Eng. 2012, 24(4):605-618.
44. Zouari H., Heutte L., Lecourtier Y. Controlling the diversity in classifier ensembles through a measure of agreement. Pattern Recognit. 2005, 38:2195-2199.
45. Kuncheva L.I., Whitaker C.J. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 2003, 51(2):181-207.
46. Giacinto G., Roli F. Design of effective neural network ensembles for image classification purposes. Image Vis. Comput. 2001, 19(9-10):699-707.
47. Yule G. On the association of attributes in statistics. Biometrika 1903, 2:121-134.
48. Tang E.K., Suganthan P.N., Yao X. An analysis of diversity measures. Mach. Learn. 2006, 65(1):247-271.
49. Luxburg U.V. A tutorial on spectral clustering. Stat. Comput. 2007, 17(4):395-416.
50. Golub G.H., Van Loan C.F. Matrix Computations 1996, Johns Hopkins University Press, Baltimore, Maryland, US. 3rd edition.
51. Helmke U., Fuhrmann P.A. Controllability of matrix eigenvalue algorithms: the inverse power method. Syst. Control Lett. 2000, 41(1):57-66.
52. Breiman L., Friedman J.H., Olshen R.A., Stone C.J. Classification and Regression Trees 1984, Chapman and Hall, UK.
53. A. Asuncion, D. Newman, UCI Machine Learning Repository, 〈〉. http://www.ics.uci.edu/~mlearn/MLRepository.html.
54. Demšar J. Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 2006, 7:1-30.
55. Alpaydin E. Introduction to Machine Learning 2004, The MIT Press, Cambridge, MA, USA.