Considering diversity and accuracy simultaneously for ensemble pruning

Applied Soft Computing Journal

Volumn 58, Issue , 2017, Pages 75-91

  Dai, Qun ^a   Ye, Rui ^a   Liu, Zhuan ^a  

^a NANJING UNIVERSITY OF AERONAUTICS AND ASTRONAUTICS   (China)

Author keywords

Accuracy reinforcement (AccRein); Diversity focused two (DFTwo); Ensemble selection; Greedy ensemble pruning (GEP) algorithm; Simultaneous diversity accuracy (SDAcc)

Indexed keywords

SOFT COMPUTING; SOFTWARE ENGINEERING;

ACCURACY-REINFORCEMENT (ACCREIN); DIVERSITY-FOCUSED-TWO (DFTWO); EFFECTIVENESS AND EFFICIENCIES; ENSEMBLE PRUNING; ENSEMBLE SELECTIONS; GENERALIZATION CAPABILITY; GENERALIZATION PERFORMANCE; INDIVIDUAL CLASSIFIERS;

REINFORCEMENT;

EID: 85018324851     PISSN: 15684946     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asoc.2017.04.058     Document Type: Article

References (53)

1. [1] Banfield, R.E., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P., Ensemble diversity measures and their application to thinning. Inf. Fusion 6 (2005), 49–62.
2. [2] Martinez-Munoz, G., Suarez, A., Aggregation ordering in bagging, presented at In International Conference on Artificial Intelligence and Applications, 2004.
3. [3] Margineantu, D., Dietterich, T., Pruning adaptive boosting, presented at In Proceedings of the 14th International Conference on Machine Learning, 1997.
4. [4] Stahl, F., May, D., Bramer, M., Parallel random prism: a computationally efficient ensemble learner for classification. Research and Development in Intelligent Systems XXIX, 2012, Springer, London, 21–34.
5. [5] Dietterich, T.G., Ensemble methods in machine learning. Multiple Classifier Systems vol. 1857, Lecture Notes in Computer Science, 2000, Springer Berlin Heidelberg, 1–15.
6. [6] Chandra, A., Chen, H., Yao, X., Trade-off between diversity and accuracy in ensemble generation. Multi-Objective Machine Learning, vol. 16, Studies in Computational Intelligence, 2006, Springer Berlin Heidelberg, 429–464.
7. [7] Zhou, Z.-H., Ensemble learning. Encyclopedia of Biometrics, 2009, Springer US, 270–273.
8. [8] Webb, G.I., Zheng, Z., Multistrategy ensemble learning: reducing error by combining ensemble learning techniques. IEEE Trans. Knowl. Data Eng. 16 (2004), 980–991.
9. [9] Polikar, R., Ensemble learning. Ensemble Machine Learning, Methods and Applications, 2012, Springer US, 1–34.
10. [10] Caruana, R., Munson, A., Niculescu-Mizil, A., Getting the Most Out of Ensemble Selection, presented at Sixth International Conference on Data Mining, Hong Kong, 2006.
11. [11] Li, N., Yu, Y., Zhou, Z.-H., Diversity regularized ensemble pruning. Machine Learning and Knowledge Discovery in Databases, 2012, Springer, 330–345.
12. [12] Tsoumakas, G., Partalas, I., Vlahavas, I., An ensemble pruning primer. Applications of Supervised and Unsupervised Ensemble Methods, 2009, Springer, 1–13.
13. [13] Papanikolaou, Y., Dimitriadis, D., Tsoumakas, G., Laliotis, M., Markantonatos, N., Vlahavas, I.P., Ensemble Approaches for Large-Scale Multi-Label Classification and Question Answering in Biomedicine, CLEF (Working Notes). 2014, 1348–1360.
14. [14] Zhang, Y., Burer, S., Street, W.N., Ensemble pruning via semi-definite programming. J. Mach. Learn. Res. 7 (2006), 1315–1338.
15. [15] Partalas, I., Tsoumakas, G., Vlahavas, I., Pruning an ensemble of classifiers via reinforcement learning. Neurocomputing 72 (2009), 1900–1909.
16. [16] Zhou, Z.-H., Wu, J., Tang, W., Ensembling neural networks: many could be better than all. Artif. Intell. 137 (2002), 239–263.
17. [17] Martinez-Munoz, G., Suarez, A., Using boosting to prune bagging ensembles. Pattern Recognit. Lett. 28 (2007), 156–165.
18. [18] Tsoumakas, G., Angelis, L., Vlahavas, I., Selective fusion of heterogeneous classifiers. Intell. Data Anal. 9 (2005), 511–525.
19. [19] Dai, Q., A competitive ensemble pruning approach based on cross-validation technique. Knowledge-Based Syst. 37 (2013), 394–414.
20. [20] Dai, Q., An efficient ensemble pruning algorithm using One-Path and Two-Trips searching approach. Knowledge-Based Syst. 51 (2013), 85–92.
21. [21] Dai, Q., A novel ensemble pruning algorithm based on randomized greedy selective strategy and ballot. Neurocomputing 122 (2013), 258–265.
22. [22] Dai, Q., Liu, Z., ModEnPBT: a modified backtracking ensemble pruning algorithm. Appl. Soft Comput. 13 (2013), 4292–4302.
23. [23] Dai, Q., Zhang, T., Liu, N., A new reverse reduce-error ensemble pruning algorithm. Appl. Soft Comput. 28 (2015), 237–249.
24. [24] Caruana, R., Niculescu-Mizil, A., Crew, G., Ksikes, A., Ensemble selection from libraries of models, presented at In Proceedings of the 21st international conference on machine learning, 2004.
25. [25] Tamon, C., Xiang, J., On the boosting pruning problem, presented at in: 11th European Conference on Machine Learning. 2000, Springer Berlin.
26. [26] Partalas, I., Tsoumakas, G., Vlahavas, I., Focused ensemble selection: A diversity-based method for greedy ensemble selection, presented at Proceeding of the 2008 conference on ECAI, 2008.
27. [27] Hernandez-Lobato, D., Martinez-Munoz, G., Suarez, A., Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles. Neurocomputing 74 (2011), 2250–2264.
28. [28] Partalas, I., Tsoumakas, G., Vlahavas, I., A study on greedy algorithms for ensemble pruning, Technical Report TR-LPIS-360-12., 2012, Department of Informatics, Aristotle University of Thessaloniki, Greece.
29. [29] Martinez-Munoz, G., Hernandez-Lobato, D., Suarez, A., An analysis of ensemble pruning techniques based on ordered aggregation. IEEE Trans. Pattern Anal. Mach. Intell. 31 (2009), 245–259.
30. [30] Partalas, I., Tsoumakas, G., Vlahavas, I., An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Mach. Learn. 81 (2010), 257–282.
31. [31] Breiman, L., Random forests. Mach. Learn. 45 (2001), 5–32.
32. [32] Dietterich, T.G., Ensemble Methods in Machine Learning, presented at In Proceedings of the 1st International Workshop in Multiple Classifier Systems, Cagliari, Italy, 2000.
33. [33] Kim, K., Lin, H., Choi, J.Y., Choi, K., A design framework for hierarchical ensemble of multiple feature extractors and multiple classifiers. Pattern Recogn. 52 (2016), 1–16.
34. [34] Cruz, R.M.O., Sabourin, R., Cavalcanti, G.D.C., Ren, T.I., META-DES: A dynamic ensemble selection framework using meta-learning. Pattern Recogn. 48 (2015), 1925–1935.
35. [35] Kokkinos, Y., Margaritis, K.G., Confidence ratio affinity propagation in ensemble selection of neural network classifiers for distributed privacy-preserving data mining. Neurocomputing 150 (2015), 513–528.
36. [36] Lysiak, R., Kurzynski, M., Woloszynski, T., Optimal selection of ensemble classifiers using measures of competence and diversity of base classifiers. Neurocomputing 126 (2014), 29–35.
37. [37] Tan, C.J., Lim, C.P., Cheah, Y.-N., A multi-objective evolutionary algorithm-based ensemble optimizer for feature selection and classification with neural network models. Neurocomputing 125 (2014), 217–228.
38. [38] Lin, C., Chen, W., Qiu, C., Wu, Y., Krishnan, S., Zou, Q., LibD3C: Ensemble classifiers with a clustering and dynamic selection strategy. Neurocomputing 123 (2014), 424–435.
39. [39] Zhang, H., Zhang, X., Gao, X.-Z., Song, S., Self-organizing multiobjective optimization based on decomposition with neighborhood ensemble. Neurocomputing 173 (2016), 1868–1884.
40. [40] Yang, B., Chen, Y., Somatic mutation detection using ensemble of flexible neural tree model. Neurocomputing 179 (2016), 161–168.
41. [41] Kuncheva, L., Whitaker, C.J., Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 51 (2003), 181–207.
42. [42] Dietterich, T.G., An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Mach. Learn. 40 (2000), 139–157.
43. [43] Giacinto, G., Roli, F., Fumera, G., Design of effective multiple classifier systems by clustering of classifiers, presented at In 15th International Conference on Pattern Recognition, 2000.
44. [44] Idris, A., Khan, A., Lee, Y.S., Intelligent churn prediction in telecom: employing mRMR feature selection and RotBoost based ensemble classification. App. Intell. 39 (2013), 659–672.
45. [45] Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C., Introduction to Algorithms. second edition, 2001, The Massachusetts Institute of Technology.
46. [46] Liu, Z., Dai, Q., Liu, N., Ensemble selection by GRASP. Appl. Intell. 41 (2014), 128–144.
47. [47] Dai, Q., Li, M., Introducing randomness into greedy ensemble pruning algorithms. Appl. Intell. 42 (2015), 406–429.
48. [48] Fan, W., Chu, F., Wang, H., Yu, P.S., Pruning and dynamic scheduling of cost-sensitive ensembles, presented at In Eighteenth national conference on artificial intelligence, American association for artificial intelligence, 2002.
49. [49] Dai, Q., Han, X., An efficient ordering-based ensemble pruning algorithm via dynamic programming. Appl. Intell. 44 (2016), 816–830.
50. [50] http://www.ics.uci.edu/∼mlearn/ MLRepository.html or ftp.ics.uci.edu:pub/machine-learning-databases.
51. [51] Witten, I.H., Frank, E., Data Mining: Practical Machine Learning Tools and Techniques. 2005, Morgan Kaufmann.
52. [52] Huang, G.-B., Zhu, Q.-Y., Siew, C.-K., Extreme learning machine: theory and applications. Neurocomputing 70 (2006), 489–501.
53. [53] Zhou, D.Y., Bousquet, O., Lal, T.N., Weston, J., Scholkopf, B., Learning with local and global consistency. Adv. Neural Inf. Process. Syst. 16 (2004), 321–328.