Margin & diversity based ordering ensemble pruning

Neurocomputing

Volumn 275, Issue , 2018, Pages 237-246

  Guo, Huaping ^a   Liu, Hongbing ^a   Li, Ran ^a   Wu, Changan ^a   Guo, Yibo ^b   Xu, Mingliang ^b  

^a XINYANG NORMAL UNIVERSITY   (China)

^b ZHENGZHOU UNIVERSITY   (China)

Author keywords

Ensemble diversity; Ensemble pruning; Example margin

Indexed keywords

NEURAL NETWORKS;

ENSEMBLE DIVERSITY; ENSEMBLE MEMBERS; ENSEMBLE PERFORMANCE; ENSEMBLE PRUNING; EXAMPLE MARGIN; INDIVIDUAL CLASSIFIERS; OPTIMAL SUBSETS; STATE OF THE ART;

COMPUTER APPLICATIONS;

ACCURACY; ALGORITHM; ARTICLE; CLASSIFIER; ENSEMBLE PRUNING; INTERMETHOD COMPARISON; MARGIN AND DIVERSITY BASED MEASURE; MATHEMATICAL PARAMETERS; MEASUREMENT; PREDICTION; PRIORITY JOURNAL; STATISTICAL ANALYSIS;

EID: 85021887039     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2017.06.052     Document Type: Article

References (44)

1. Wu, X., Kumar, V., Quinlan, J.R., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., Ng, A., Liu, B., Yu, P.S., Zhou, Z.H., Steinbach, M., Hand, D.J., Steinberg, D., Top 10 algorithms in data mining. Knowl. Inf. Syst. 14:1 (2008), 1–37.
2. T. G. Dietterich, Ensemble methods in machine learning, in: Proceedings of the 1st International Workshop on Multiple Classifier Systems 200, pp. 1–15.
3. Breiman, L., Bagging predictors. Mach. Learn. 24:2 (1996), 123–140.
4. Freund, Y., Schapire, R.E., A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55:1 (1997), 119–139.
5. Breiman, L., Random forests. Mach. Learn. 45:1 (2001), 5–32.
6. Liu, F.T., Ting, K.M., Yu, Y., Zhou, Z.H., Spectrum of variable-random trees. J. Artif. Intell. Res. 32:1 (2008), 355–384.
7. Rodriguez, J.J., Kuncheva, L.I., Alonso, C.J., Rotation forest: a new classifier ensemble method. IEEE Trans. Pattern Anal. Mach. Intell. 28:10 (2006), 1619–1630.
8. Zhang, D., Chen, S., Zhou, Z., Yang, Q., Constraint projections for ensemble learning. Proceedings of the 23rd AAAI Conference on Artificial Intelligence, 2008, 758–763.
9. Wolpert, D.H., Stacked generalization. Neural Netw. 5:2 (1992), 241–259.
10. Kuncheva, L.I., Bezdek, J.C., Duin, R.P., Decision templates for-multiple classifier fusion: an experimental comparison. Pattern Recognit. 34:2 (2001), 299–314.
11. Liu, Y., Yao, X., Ensemble learning via negative correlation. Neural Netw. 12:10 (1999), 1399–1404.
12. Zhou, Z.H., Ensemble Methods Foundations and Algorithms. 2012, Boca Raton, Florida: CRC Press, Taylor & Francis, 67–97.
13. Guo, L., Margin framework for ensemble classifiers, Application to remote sensing data, 2011, Phd thesis, University of Bordeaux 3, France.
14. Ma, Z., Dai, Q., Liu, N., Several novel evaluation measures for rank-based ensemble pruning with applications to time series prediction. Expert Syst. Appl. 42:1 (2015), 280–292.
15. Li, P.P., Wu, X., Hu, X., Wang, H., Learning concept-drifting data streams with random ensemble decision trees. Neurocomputing 166 (2015), 68–83.
16. Zhou, Z., Wu, J., Tang, W., Ensembling neural networks: many could be better than all. Artif. Intell. 137:1–2 (2002), 239–263.
17. Martinez-Muverbnoz, G., Suarez, A., Aggregation ordering in bagging. Proceeding of the International conference on artificial intelligence and applications, 2004, 258–263.
18. Banfield, R.E., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P., Ensemble diversity measures and their application to thinning. Inf. Fusion 6:1 (2005), 49–62.
19. Partalas, I., Tsoumakas, G., Vlahavas, I.P., Focused ensemble selection: A diversity-based method for greedy ensemble selection. Proceeding of the 18th European Conference on Artificial Intelligence, 2008, 117–121.
20. Maríínez-Munoz, G., Hernández-Lobato, D., Suárez, A., An analysis of ensemble pruning techniques based on ordered aggregation. IEEE Trans. Pattern Anal. Mach. Intell. 31 (2009), 245–259.
21. Partalas, I., Tsoumakas, G., Vlahavas, I.P., An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Mach. Learn. 81:3 (2010), 257–282.
22. Lu, Z.Y., Wu, X.D., Zhu, X.Q., Bongard, J., Ensemble pruning via individual contribution ordering. Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2010, 871–880.
23. Guo, L., Boukir, S., Margin-based ordered aggregation for ensemble pruning. Pattern Recognit. Lett. 34:6 (2013), 603–609.
24. Markatopoulou, F., Tsoumakas, G., Vlahavas, I.P., Dynamic ensemble pruning based on multi-label classification. Neurocomputing 150 (2015), 501–512.
25. Qian, C., Yu, Y., Zhou, Z.H., Pareto ensemble pruning. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015, 2935–2941.
26. Kokkinos, Y., Margaritis, K.G., Confidence ratio affinity propagation in ensemble selection of neural network classifiers for distributed privacy-preserving data mining. Neurocomputing 150 (2015), 513–528.
27. Schapire, R.E., Freund, Y., Barlett, P., Lee, W.S., Boosting the margin: a new explanation for the effectiveness of voting methods. Ann. Stat. 26:5 (1998), 1651–1686.
28. Ho, T.K., The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20:8 (1998), 832–844.
29. Tamon, C., Xiang, J., On the boosting pruning problem. Proceedings of the 11th European Conference on Machine Learning, 2000, 404–412.
30. Gao, W., Zhou, Z.H., On the doubt about margin explanation of boosting. Artif. Intell. 203 (2013), 1–18.
31. Kuncheva, L.I., Whitaker, C.J., Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 51:2 (2003), 181–207.
32. Zhang, Y., Burer, S., Street, W.N., Ensemble pruning via semi-definite programming. J. Mach. Learn. Res. 7 (2006), 1315–1338.
33. Margineantu, D.D., Dietterich, T.G., Pruning adaptive boosting. Proceedings of the 14th International Conference on Machine Learning, 1997, 211–218.
34. Lazarevic, A., Obradovic, Z., Effective pruning of neural network classifier ensembles. Proceedings of the IEEE/INNS International Joint Conference on Neural Networks, 2001, 796–801.
35. Giacinto, G., Roli, F., Fumera, G., Design of effective multiple classifier systems by clustering of classifiers. Proceedings of the 15th International Conference on Pattern Recognition, 2000, 160–163.
36. Bakker, B., Heskes, T., Clustering ensembles of neural network models. Neural Netw. 16:2 (2003), 261–269.
37. Li, N., Zhou, Z.H., Selective ensemble under regularization framework. Proceedings of the 8th International Workshop Multiple Classifier Systems, 2009, 293C303.
38. Chen, H., Tino, P., Yao, X., A probabilistic ensemble pruning algorithm. Proceedings of the 6th IEEE International Conference on Data Mining-Workshops, 2006, 878–882.
39. Chen, H., Tino, P., Yao, X., Predictive ensemble pruning by expectation propagation. IEEE Trans. Knowl. Data Eng. 21:7 (2009), 999–1013.
40. Martínez-Muñoz, G., Hernández-Lobato, D., Suárez, A., An analysis of ensemble pruning techniques based on ordered aggregation. IEEE Trans. Pattern Anal. Mach. Intell. 31:2 (2009), 245–259.
41. A. Asuncion, and D. Newman, UCI machine learning repository, 2007. URL http://www.ics.uci.edu/~mlearn/MLRepository.html.
42. Quinlan, J.R., C4.5: programs for Machine Learning. 1993, Morgan Kaufmann, San Francisco, CA, USA.
43. Witten, I.H., Frank, E., Data Mining: practical Machine Learning Tools and Techniques. 2005, Morgan Kaufmann, San Francisco.
44. Demsar, J., Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 6 (2006), 1–30.