Parallelizing AdaBoost by weights dynamics

Computational Statistics and Data Analysis

Volumn 51, Issue 5, 2007, Pages 2487-2498

  Merler, Stefano ^a   Caprile, Bruno ^a   Furlanello, Cesare ^a  

^a ITC IRST   (Italy)

Author keywords

AdaBoost; Boosting; Classification; Parallelization; Weights dynamics

Indexed keywords

CLASSIFICATION (OF INFORMATION); COMPUTATIONAL METHODS; DATA REDUCTION; DATABASE SYSTEMS; MATHEMATICAL MODELS; RANDOM PROCESSES;

ADABOOST; BOOSTING; PARALLELIZATION; WEIGHTS DYNAMICS;

STATISTICAL METHODS;

EID: 33751014821     PISSN: 01679473     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.csda.2006.09.001     Document Type: Article

References (23)

1. Bauer E., and Cohavi R. An empirical comparison of voting classification algorithms: bagging, boosting, and variants. Mach. Learning 36 1-2 (1999) 105-139
2. Blake, C., Merz, C., 1998. UCI repository of machine learning databases. 〈http://www.ics.uci.edu/∼mlearn/MLRepository.html〉.
3. Breiman L. Bagging predictors. Machine Learning 24 2 (1996) 123-140
4. Breiman L., Friedman J., Olshen R., and Stone C. Classification and Regression Trees (1984), Wadsworth, Belmont CA
5. Caprile B., Furlanello C., and Merler S. Highlighting hard patterns via Adaboost weights evolution. In: Kittler J., and Roli F. (Eds). Multiple Classifier Systems, Lecture Notes in Computer Science vol. 2364 (2002), Springer, Berlin 72-80
6. Collins M., Schapire R.E., and Singer Y. Logistic regression, adaboost and bregman distances. Mach. Learning 48 1-3 (2002) 253-285
7. Dietterich T. An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Mach. Learning 40 2 (2000) 139-157
8. Efron B., and Tibshirani R. An introduction to the bootstrap. Monographs on Statistic and Applied Probability vol. 57 (1993), Chapman & Hall, London
9. Freund Y., and Schapire R.E. A decision-theoretic generalization of online learning and an application to boosting. J. Comput. System Sci. 55 1 (1997) 119-139
10. Friedman, J., Hastie, T., Tibshirani, R., 2000. Additive logistic regression: a statistical view of boosting. Ann. Statist. 28 (2), 337-407.
11. Kontoghiorghes, E.J. (Ed.), 1999. Parallel processing and statistics. Special Issue of the Journal Computational Statistics & Data Analysis, vol. 31(4). Elsevier, Amsterdam.
12. Kontoghiorghes E.J. Parallel algorithms for linear models: numerical methods and estimation problems. Advances in Computational Economics vol. 15 (2000), Kluwer Academic Publishers, Boston, MA
13. Kontoghiorghes, E.J. (Ed.), 2005. Handbook of parallel computing and statistics. Statistics: Textbooks and Monograph Series, vol. 184. Marcel Dekker Inc., New York.
14. Lazarevic A., and Obradovic Z. Boosting algorithms for parallel and distributed learning. Distrib. and Parallel Databases 11 2 (2002) 203-229
15. Margineantu D., and Dietterich T. Pruning adaptive boosting. Fourteenth International Conference on Machine Learning (1997), Morgan Kaufmann, Los Altos, CA 211-218
16. Mason L., Baxter J., Bartlett P., and Frean M. Boosting algorithms as gradient descent. Advances in Neural Information Processing Systems, vol. 12 (2000), MIT Press, Cambridge, MA 512-518
17. Quinlan J. Bagging, boosting, and C4.5. Thirteenth National Conference on Artificial Intelligence (1996), AAAI Press, MIT Press, Menlo Park, CA, Cambridge, MA 163-175
18. Rätsch G., Onoda T., and Müller K. Soft margins for Adaboost. Mach. Learning 42 (2001) 287-320
19. Rudin C., Daubechies I., and Schapire R. The dynamics of AdaBoost: cyclic behavior and convergence of margins. J. Mach. Learning Res. 5 (2004) 1557-1595
20. Schapire, R.E., 2002. The boosting approach to machine learning: an overview. In: MSRI Workshop on Nonlinear Estimation and Classification.
21. Schapire R.E., Freund Y., Bartlett P., and Lee W.S. Boosting the margin: a new explanation for the effectiveness of voting methods. Ann. Statist. 26 5 (1998) 1651-1686
22. Webb G. Multiboosting: a technique for combining boosting and wagging. Mach. Learning 40 3 (2000) 159-196
23. Yu, C., Skillicorn, D.B., 2001. Parallelizing boosting and bagging. Technical Report, Department of Computing and Information Sciences, Queen's University Kingston, Canada.