Margin optimization based pruning for random forest

Neurocomputing

Volumn 94, Issue , 2012, Pages 54-63

  Yang, Fan ^a   Lu, Wei hang ^a   Luo, Lin kai ^a   Li, Tao ^b  

^a XIAMEN UNIVERSITY   (China)

^b The Department of Electrical and Computer Engineering   (United States)

Author keywords

Ensemble pruning; Margin optimization; Random forests

Indexed keywords

BENCHMARK DATASETS; CLASSIFICATION TREES; EMPIRICAL COMPARISON; ENSEMBLE PRUNING; GENERALIZATION ABILITY; GENERALIZATION PERFORMANCE; KEY ELEMENTS; MARGIN OPTIMIZATION; PRUNING ALGORITHMS; PRUNING METHODS; RANDOM FORESTS; TRAINING SETS;

ALGORITHMS; DECISION TREES;

FORESTRY;

ARTICLE; CLASSIFICATION ALGORITHM; CONTROLLED STUDY; DATA ANALYSIS; DATA PROCESSING; ENSEMBLE PRUNING RANDOM FOREST; INTERMETHOD COMPARISON; MATHEMATICAL ANALYSIS; MATHEMATICAL MODEL; PERFORMANCE MEASUREMENT SYSTEM; PRIORITY JOURNAL; PROCESS DESIGN; PROCESS OPTIMIZATION; PROCESS TECHNOLOGY; RANDOM FOREST; RECURSIVELY PRUNING RANDOM FOREST;

ALGORITHMS; DECISION MAKING; FORESTS; OPTIMIZATION; PRUNING;

EID: 84863500698     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2012.04.007     Document Type: Article

References (27)

1. Kuncheva L.I., Whitaker C.J. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 2003, 51(2):181-207.
2. Breiman L. Random forests. Mach. Learn. 2001, 45:5-32.
3. Martínez-Muñoz G., Suárez A. Aggregation ordering in bagging. Proceedings of International Conference on Artificial Intelligence and Applications 2004, 258-263.
4. Martínez-Muñoz G., Hernández-Lobato D., Suárez A. An analysis of ensemble pruning techniques based on ordered aggregation. IEEE Trans. Pattern Anal. Mach. Intell. 2009, 31(2):245-259.
5. Lu Z., Wu X., Zhu X., Bongard J. Ensemble pruning via individual contribution ordering. Proc. 16th ACM SIGKDD 2010, 871-880.
6. Zhou Z.H., Wu J., Tang W. Ensembling neural networks: many could be better than all. Artif. Intell. 2002, 137(1-2):239-263.
7. Zhang Y., Burer S., Street W.N. Ensemble pruning via semi-definite programming. J. Mach. Learn. Res. 2006, 7:1315-1338.
8. N. Li, Z.H. Zhou, Selective Ensemble under Regularization Framework, in: Proceedings of the 8th International Workshop on Multiple Classifier Systems, 2009, pp.293-303.
9. Zhang L., Zhou W.D. Sparse ensembles using weighted combination methods based on linear programming. Pattern Recognit. 2011, 44:97-106.
10. Tsoumakas G., Partalas I., Vlahavas I. An Ensemble Pruning Primer, Application of Supervised and Unsupervised Ensemble Methods 2009, Springer, pp.1-13.
11. G. Martínez-Muñoz, D. Hernández-Lobato, A. Suárez, Statistical Instance-based Ensemble Pruning for Multi-Class Problems, in: Proceedings of the 19th International Conference on Artificial Neural Networks, 2009, pp.90-99.
12. V. Soto, G. Martínez-Muñoz, D. Hernández-Lobato, A. Suárez, A. Double Pruning Algorithm for Classification Ensembles, in: Proceedings of the 9th International Workshop on Multiple Classifier Systems, 2010, pp. 104-113.
13. M. Robnik-Šikonja, Improving Random Forests, in: Proceedings of the 15th European Conference on Machine Learning, 2004, pp.359-370.
14. L. Reyzin, R.E. Schapire, How Boosting the Margin can also Boost Classifier Complexity, in: Proceedings of the 23rd International Conference on Machine Learning, 2006, pp. 753-760.
15. Schapire R.E., Freund Y., Bartlett P., Lee W.S. Boosting the margin: a new explanation for the effectiveness of voting methods. Annu. Stat 1998, 26(5):1651-1686.
16. Rätsch G., Warmuth M.K. Efficient margin maximizing with boosting. J. Mach. Learn. Res. 2005, 6:2131-2152.
17. M.K. Warmuth, J. Liao, G. Rätsch, Totally Corrective Boosting Algorithms that Maximize the Margin, in: Proceedings of the 23rd International Conference on Machine Learning, 2006, pp.1001-1008.
18. Breiman L. Prediction games and arcing algorithms. Neural Comput. 1999, 11(7):1493-1517.
19. L. Wang, M. Sugiyama, C. Yang, Z.H. Zhou, J. Feng, On the Margin Explanation of Boosting Algorithms, in: Proceedings of the 21st Annual Conference on Learning Theory, 2008, pp. 479-490.
20. Shen C.H., Li H.X. Boosting through optimization of margin distributions. IEEE Trans. Neural Networks 2010, 21(4):659-666.
21. Fu B., Wang Z.H., Wang Z.F. Algorithm of classifier selection for maximizing the margin. J. Comput. Sci. Front. 2011, 5(1):59-67.
22. Liaw A., Wiener M. Classification and regression by random forest. R News 2002, 2(3):18-22.
23. Frank A., Asuncion A. UCI Machine Learning Repository 2010, University of California, School of Information and Computer Science, Irvine, CA.
24. Tang E.K., Suganthan P.N., Yao X. An analysis of diversity measures. Mach. Learn. 2006, 65(1):247-271.
25. Zhang H.P., Wang M.H. Search for the smallest random forest. Stat. Interface 2009, 2:381-388.
26. H.C. Lian and B.L. Lu, An Algorithm for Pruning Redundant Modules in Min-Max Modular Network, in: Proceedings of International Joint Conference on Neural Networks, 2005,pp.1983-1988.
27. J. Li, B.L. Lu, and M. Ichikawa, An Algorithm for Pruning Redundant Modules in Min-Max Modular Network with GZC Function, in: Proceedings of the 1st International Conference on Natural Computation, 2005, pp.293-302.