Bias-variance control via hard points shaving

International Journal of Pattern Recognition and Artificial Intelligence

Volumn 18, Issue 5, 2004, Pages 891-903

  Merler, Stefano ^a   Caprile, Bruno ^a   Furlanello, Cesare ^a  

^a ITC IRST   (Italy)

Author keywords

AdaBoost; Bias variance decomposition; Binary classification; Machine learning; Regularization

Indexed keywords

ALGORITHMS; DATA REDUCTION; ERROR ANALYSIS; MATHEMATICAL MODELS; PROBABILITY DISTRIBUTIONS; SET THEORY;

ADABOOST ALGORITHM; BIASVARIANCE DECOMPOSITION; BINARY CLASSIFICATION; REGULARIZATION;

LEARNING SYSTEMS;

EID: 8644244736     PISSN: 02180014     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0218001404003460     Document Type: Article

References (15)

1. L. Breiman, J. Friedman, R. Olshen and C. Stone, Classification and Regression Trees (Wadsworth, Belmont, CA, 1984).
2. B. Caprile, C. Furlanello and S. Merler, Highlighting hard patterns via AdaBoost weights evolution, Multiple Classifier Systems, eds. J. Kittler and F. Roli, Lecture Notes in Computer Science, Vol. 2364 (Springer, 2002) pp. 72-80.
3. T. G. Dietterich, An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization, Mach. Learn. 40(2) (2000) 139-157.
4. P. Domingos, A unified bias-variance decomposition and its applications, in Proc. 17th Int. Conf. Machine Learning (Morgan Kaufmann, San Francisco, CA, 2000) pp. 231-238.
5. B. Efron and R. Tibshirani, An Introduction to the Bootstrap, Monographs on Statistic and Applied Probability, Vol. 57 (Chapman & Hall, 1993).
6. Y. Freund and R. E. Schapire, A decision-theoretic generalization of online learning and an application to boosting, J. Comput. Syst. Sci. 55(1) (1997) 119-139.
7. J. Friedman, T. Hastie and R. Tibshirani, Additive logistic regression: a statistical view of boosting, Ann. Statist. 38(2) (2000) 337-374.
8. A. J. Grove and D. Schuurmans, Boosting in the limit: maximizing the margin of learned ensembles, in AAAI/IAAI, 1998, pp. 692-699.
9. W. Jiang, Process consistency for AdaBoost, Technical Report, Department of Statistics, Northwestern University, 2000.
10. J. R. Quinlan, Bagging, boosting, and C4.5, Thirteenth Nat. Conf. Artificial Intelligence (Cambridge, AAAI Press/MIT Press, 1996) pp. 163-175.
11. G. Rätsch, T. Onoda and K. R. Müller, Soft margins for Adaboost, Mach. Learn. 42 (2001) 287-320.
12. R. E. Schapire, The boosting approach to machine learning: an overview, in MSRI Workshop on Nonlinear Estimation and Classification, 2002.
13. R. E. Schapire, Y. Freund, P. Bartlett and W. S. Lee, Boosting the margin: a new explanation for the effectiveness of voting methods, Ann. Statist. 26(5) (1998) 1651-1686.
14. M. Sebban, R. Nock and S. Lallich, Stopping criterion for boosting-based reduction techniques: from binary to multiclass problems, J. Mach. Learn. Res. 3 (2002) 863-885.
15. V. Vapnik, The Nature of Statistical Learning Theory, Statistics for Engineering and Information Science (Springer Verlag, 2000).