Classification performance of bagging and boosting type ensemble methods with small training sets

New Generation Computing

Volumn 29, Issue 3, 2011, Pages 277-292

  Zaman, M Faisal ^a   Hirose, Hideo ^a  

^a KYUSHU INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

Bias variance decomposition; Classification performance; Small

Indexed keywords

BIAS AND VARIANCE; BIAS VARIANCE DECOMPOSITION; CLASSIFICATION ERRORS; CLASSIFICATION PERFORMANCE; DATA SETS; ENSEMBLE METHODS; OPTIMAL ENSEMBLE; OPTIMAL TRAINING; RESAMPLING; SMALL; SMALL TRAINING; TRAINING SAMPLE; TRAINING SETS; UCI MACHINE LEARNING REPOSITORY;

ADAPTIVE BOOSTING; LEARNING SYSTEMS; OPTIMIZATION; SAMPLING;

CLASSIFICATION (OF INFORMATION);

EID: 80053918498     PISSN: 02883635     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00354-011-0303-0     Document Type: Article

References (19)

1. Asuncion, A. and Newman, D. J., UCI Repository of Machine Learning, School of Information and Computer Science, University of California, Irvine. Available at http://www.ics.uci.edu/mlearn/MLRepository.html, 2007.
2. Bauer, E. and Kohavi, R., "An empirical comparison of voting classification algorithms: Bagging, boosting, and variants, " Machine Learn., 36, 1-2, pp. 105-139, 1999.
3. Breiman, L., "Bagging predictors, " Machine Learn., 24, 2, pp. 123-140, 1996.
4. Breiman, L., "Out-of-bag estimation, " Technical Report, Statistics Department, University of Berkeley CA 94708, 1996.
5. Breiman, L., "Random Forests, " Machine Learn., 45, 1, pp. 5-32, 2001.
6. Demšar, J., "Statistical comparisons of classifiers over multiple datasets, " J. Mach. Learn. Research, 7, pp. 1-30, 2006.
7. Dietterich, T. G., "Ensemble methods in machine learning, " in Proc. of 1st International Workshop on Multiple Classifier Systems 2001 (Kittler, J. and Roli, F. eds.), Springer-Verlag, Berlin/Heidelberg, 2001.
8. Freund, Y. and Schapire, R., "Experiments with a New boosting algorithm, " in Proc. of the Thirteenth International Conference on Machine Learning 1996, Morgan Kaufmann, San Francisco, pp. 148-156, 1996.
9. Hastie, T., Tibshirani, R. and Freidman, J., The Elements of Statistical Learning: Data Mining, Inference and Prediction (Second Edition), Springer-Verlag, New York, 2009.
10. Hothorn, T. and Lausen, B., "Bundling classifiers by bagging trees, " Comput. Statist. Data Anal., 49, pp. 1068-1078, 2005.
11. Kuncheva, L. I., Combining Pattern Classifiers, Methods and Algorithms, John Wiley and Sons, 2004.
12. Martínez-Munõz, G. and Suárez, A., "Out-of-bag estimation of the optimal sample size in bagging, " Pattern Recognition, 43, 1, pp. 143-152, 2010.
13. Politis, D., Romano, J. P. and Wolf, M., Subsampling, Springer Series in Statistics, Springer, Berlin, 1999.
14. Schapire, R. E "The Boosting Approach to Machine Learning : An Overview, "in Proc. of MSRI Workshop on Nonlinear Estimation and Classification 2003.
15. Denison, D. D., Hansen, M. H., Holmes, C. C., Mallick, B. and Yu, B. eds.: Lecture Notes in Statistics, 171, pp. 149-172, Springer-Verlag, 2003.
16. Webb, G. I., "MultiBoosting: A technique for combining boosting and wagging, " Machine Learn., 40, 2, pp. 159-196, 2000.
17. Zaman, F. and Hirose, H., "A Comparitive Study on the Performance of Several Ensemble Methods with Low Subsampling Ratio, " in 2nd Asian Conference on Information Intelligent Database Systems 2010.
18. Nguyen, N. T., Le, M. T. and Swiatek, J. eds.), Lecture Notes in Computer Science, 5991, Part-II, pp. 320-329, Springer-Verlag, 2010.
19. Zhang, C. X., Zhang, J. S. and Zhang, G. Y., "An efficient modified boosting method for solving classification problems, " J. Comput. Applied Mathemat., 214, pp. 381-392, 2008.