A new reverse reduce-error ensemble pruning algorithm

Applied Soft Computing

Volumn 28, Issue , 2015, Pages 237-249

  Dai, Qun ^a   Zhang, Ting ^a   Liu, Ningzhong ^a  

^a NANJING UNIVERSITY OF AERONAUTICS AND ASTRONAUTICS   (China)

Author keywords

Classifier; Machine learning; Neural network ensemble; Pattern recognition; Reduce Error (RE) pruning; Reverse Reduce Error (RRE) pruning

Indexed keywords

CLASSIFICATION (OF INFORMATION); CLASSIFIERS; LEARNING SYSTEMS; PATTERN RECOGNITION;

BENCHMARK CLASSIFICATION; EMPIRICAL INVESTIGATION; ENSEMBLE SELECTIONS; NEAR-OPTIMAL SOLUTIONS; NEURAL NETWORK ENSEMBLES; REDUCE-ERROR (RE) PRUNING; REVERSE REDUCE-ERROR (RRE) PRUNING; SUBOPTIMAL SOLUTION;

ERRORS;

EID: 84919706710     PISSN: 15684946     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asoc.2014.10.045     Document Type: Article

References (55)

1. L.K. Hansen, P. Salamon, Neural network ensembles, IEEE Trans. Pattern Anal. Mach. Intell. 12 (1990) 993-1001.
2. A. Krogh, J. Vedelsby, Neural network ensembles, cross validation, and active learning, in: G. Tesauro, D. Touretzky, E.T. Leen (Eds.), Advances in Neural Information Processing Systems 7, MIT Press, Cambridge, MA, 1995, pp. 231-238.
3. L. Kuncheva, C.J. Whitaker, Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy, Machine Learning 51 (2003) 181-207.
4. F.J. Huang, Z.-H.Z. hou, H.-J. Zhang, T.H. Chen, Pose Invariant Face Recognition, Presented at 4th IEEE International Conference on Automatic Face and Gesture Recognition, Grenoble, France, 2000.
5. L.K. Hansen, L. Liisberg, P. Salamon, Ensemble Methods for Handwritten Digit Recognition, Presented at IEEE Workshop on Neural Networks for Signal Processing, Helsingoer, Denmark, 1992.
6. Y. Zhao, J. Gao, X. Yang, A Survey of Neural Network Ensemble, Presented at International Conference on Neural Networks and Brain, Beijing, China, 2005.
7. K.J. Cherkauer, Human Expert Level Performance on A Scientific Image Analysis Task by a System Using Combined Artificial Neural Networks, Presented at Proc. AAAI-96 Workshop on Integrating Multiple Learned Models for Improving and Scaling Machine Learning Algorithms, Portland, OR, 1996.
8. P. Cunningham, J. Carney, S. Jacob, Stability problems with artificial neural networks and the ensemble solution, Artif. Intell. Med. 20 (2000) 217-225.
9. Z.-H. Zhou, Y. Jiang, Y.-B. Yang, S.-F. Chen, Lung cancer cell identification based on artificial neural network ensembles, Artif. Intell. Med. 24 (2002) 25-36.
10. K.-P. Lin, K.-C. Hung, An efficient fuzzy weighted average algorithm for the military UAV selecting under group decision-making, Know.-Based Syst. 24 (2011) 877-889.
11. B. Kavitha, D.S. Karthikeyan, P.S. Maybell, An ensemble design of intrusion detection system for handling uncertainty using neutrosophic logic classi.er, Know.-Based Syst. 28 (2012) 88-96.
12. R.E. Banfield, L.O. Hall, K.W. Bowyer, W.P. Kegelmeyer, Ensemble diversity measures and their application to thinning, Inf. Fus. 6 (2005) 49-62.
13. R. Caruana, A. Niculescu-Mizil, G. Crew, A. Ksikes, Ensemble Selection from Libraries of Models, Presented at Proceedings of the 21st International Conference on Machine Learning, 2004.
14. D. Margineantu, T. Dietterich, Pruning Adaptive Boosting, Presented at Proceedings of the 14th International Conference on Machine Learning, 1997.
15. G. Giacinto, F. Roli, G. Fumera, Design of effective multiple classifier systems by clustering of classifiers, Presented at 15th International Conference on Pattern Recognition, 2000.
16. W. Fan, F. Chu, H. Wang, P.S. Yu, Pruning and Dynamic Scheduling of Cost-sensitive Ensembles, Presented at Eighteenth National Conference on Artificial Intelligence, American Association for Artificial Intelligence, 2002.
17. G. Martinez-Munoz, A. Suarez, Aggregation Ordering in Bagging, Presented at International Conference on Artificial Intelligence and Applications, 2004.
18. G. Martinez-Munoz, A. Suarez, Pruning in Ordered Bagging Ensembles, Presented at 23rd International Conference in Machine Learning, 2006.
19. I. Partalas, G. Tsoumakas, I. Vlahavas, Focused Ensemble Selection: A Diversity-based Method for Greedy Ensemble Selection, vol. 178, IOS Press, Patras, Greece: Amsterdam, 2008.
20. G. Tsoumakas, L. Angelis, I. Vlahavas, Selective fusion of heterogeneous classifiers, Intell. Data Anal. 9 (2005) 511-525.
21. I. Partalas, G. Tsoumakas, I. Vlahavas, Pruning an ensemble of classifiers via reinforcement learning, Neurocomputing 72 (2009) 1900-1909.
22. C. Tamon, J. Xiang, On the Boosting Pruning Problem, Presented at 11th European Conference on Machine Learning, Springer, Berlin, 2000.
23. Z.-H. Zhou, J. Wu, W. Tang, Ensembling neural networks: many could be better than all, Artif. Intell. 137 (2002) 239-263.
24. Z. Zhou, W. Tang, Selective Ensemble of Decision Trees, Presented at Proceedings of the 9th International Conference on Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing, Chongqing, China, 2003.
25. Y. Zhang, S. Burer, W.N. Street, Ensemble pruning via semi-definite programming, J. Mach. Learn. Res. 7 (2006) 1315-1338.
26. G. Martinez-Munoz, D. Hernandez-Lobato, A. Suarez, An analysis of ensemble pruning techniques based on ordered aggregation, IEEE Trans. Pattern Anal. Mach. Intell. 31 (2009) 245-259.
27. G. Martinez-Munoz, A. Suarez, Using boosting to prune bagging ensembles, Pattern Recognit. Lett. 28 (2007) 156-165.
28. I. Partalas, G. Tsoumakas, I. Vlahavas, An ensemble uncertainty aware measure for directed hill climbing ensemble pruning, Mach. Learn. 81 (2010) 257-282.
29. X. Tian, R. Wang, Soft-Voting Classification using Locally Linear Reconstruction, Presented at CIS '11 Proceedings of the 2011 Seventh International Conference on Computational Intelligence and Security, 2011.
30. Q. Dai, Z. Liu, ModEnPBT: a modified backtracking ensemble pruning algorithm, Appl. Soft Comput. 13 (2013) 4292-4302.
31. Z.-H. Zhou, Ensemble Methods: Foundations and Algorithms, Chapman and Hall/CRC, 2012.
32. G. Tsoumakas, I. Partalas, I. Vlahavas, Applications of Supervised and Unsupervised Ensemble Methods, vol. 245, Springer, Berlin, Heidelberg, 2009.
33. D. Partridge, W.B. Yates, Engineering multiversion neural-net systems, Neural Comput. 8 (1996) 869-893.
34. Y. Yang, K. Korb, K. Ting, G. Webb, Ensemble Selection for Superparent-one-dependence Estimators, Presented at AI 2005: Advances in Artificial Intelligence, 2005.
35. G. Tsoumakas, I. Partalas, I. Vlahavas, An ensemble pruning primer Applications of Supervised and Unsupervised Ensemble Methods, vol. 245, Studies in Computational Intelligence, 2009, pp. 1-13.
36. Q. Fu, S.X. Hu, S.Y. Zhao, Clustering-based selective neural network ensemble, J. Zhejiang Univ. Sci. 6A (2005) 387-392.
37. A. Lazarevic, Z. Obradovic, The Effective Pruning of Neural Network Classifiers, Presented at 2001 IEEE/INNS International Conference on Neural Networks, IJCNN, 2001.
38. B. Bakker, T. Heskes, Clustering ensembles of neural network models, Neural Netw. 16 (2003) 261-269.
39. Q. Dai, A novel ensemble pruning algorithm based on randomized greedy selective strategy and ballot, Neurocomputing 122 (2013) 258-265.
40. Q. Dai, A competitive ensemble pruning approach based on cross-validation technique, Know.-Based Syst. 37 (2013) 394-414.
41. Q. Dai, An efficient ensemble pruning algorithm using one-path and two-trips searching approach, Know.-Based Syst. 51 (2013) 85-92.
42. G. Tsoumakas, I. Katakis, I. Vlahavas, Effective Voting of Heterogeneous Classifiers, Presented at Proceedings of the 15th European Conference on Machine Learning, Pisa, Italy, 2004.
43. I. Partalas, G. Tsoumakas, I. Katakis, I. Vlahavas, Ensemble Pruning using Reinforcement Learning, Presented at 4th Hellenic Conference on Artificial Intelligence (SETN 2006), 2006.
44. C. Watkins, P. Dayan, Q-learning, Mach. Learn. 8 (1992) 279-292.
45. UCI, UCI repository of machine learning databases.
46. D. Malerba, F. Esposito, G. Semeraro, A further comparison of simplification methods for decision-tree induction, in: D. Fisher, H. Lenz (Eds.), Learning from Data: Artificial Intelligence and Statistics V, Lecture Notes in Statistics, Springer Verlag, Berlin, 1995.
47. F. Esposito, D. Malerba, G. Semeraro, Multistrategy learning for document recognition, Appl. Artif. Intell. 8 (1994) 33-84.
48. A. Weijters, The BP-SOM architecture and learning rule, Neural Process. Lett. 2 (1995) 13-16.
49. A. Weijters, V.D. Bosch, H.J. Herik, Intelligible neural networks with BP-SOM, Marcke and Daelemans, 1997, pp. 27-36.
50. J. Eggermont, Rule-extraction and learning in the BP-SOM architecture, Computer Science Department: Leiden University, 1998.
51. L. Prechelt, Proben1: A Set of Neural Network Benchmark Problems and Benchmarking Rules, Fakultat fur informatic, University Karlsruhe, Germany, 1994.
52. M. Sikora, L. Wrobel, Application of rule induction algorithms for analysis of data collected by seismic hazard monitoring systems in coal mines, Arch. Min. Sci. 55 (2010) 91-114.
53. B.P. Roe, H.-J. Yang, J. Zhu, Y. Liu, I. Stancu, G. McGregor, Boosted decision trees as an alternative to artificial neural networks for particle identification, Nuclear Instrum. Methods Phys. Res. 543 (2005) 577-584.
54. Q. Dai, N.Z. Liu, The build of n-bits binary coding ICBP ensemble system, Neurocomputing 74 (2011) 3509-3519.
55. Q. Dai, S.C. Chen, B.Z. Zhang, Improved CBP neural network model with applications in time series prediction, Neural Process. Lett. 18 (2003) 197-211.