A dynamic AdaBoost algorithm with adaptive changes of loss function

IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews

Volumn 42, Issue 6, 2012, Pages 1828-1841

  Gao, Yunlong ^a   Ji, Guoli ^a   Yang, Zijiang ^b   Pan, Jinyan ^c  

^a XIAMEN UNIVERSITY   (China)

^b YORK UNIVERSITY   (Canada)

^c JIMEI UNIVERSITY   (China)

Author keywords

AdaBoost; dynamic weighting scheme; overfitting; statistical analysis; variable loss function

Indexed keywords

ADABOOST ALGORITHM; ADAPTIVE FEATURES; ADAPTIVITY; BOOSTING ALGORITHM; CLASSIFICATION ACCURACY; GENERALIZATION ABILITY; GENERALIZATION PERFORMANCE; HOEFFDING INEQUALITY; IMPROVE-A; LOSS FUNCTIONS; NONCONVEX FUNCTIONS; OVERFITTING; REAL WORLD DATA; STATISTICAL TOOLS; TRAINING ERRORS; TRAINING SAMPLE; UPPER BOUND; WEIGHTING SCHEME;

ALGORITHMS; EXPERIMENTS; SAMPLING; STATISTICAL MECHANICS; STATISTICAL METHODS;

ADAPTIVE BOOSTING;

EID: 84871762030     PISSN: 10946977     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCC.2012.2227471     Document Type: Article

References (37)

1. Y. Yang and K. Chen, "Time series clustering via RPCL network ensemble with different representations," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 41, no. 2, pp. 190-199, Mar. 2011.
2. Y. W. Xu, X. B. Cao, and H. Qiao, "An efficient tree classifier ensemble-based approach for Pedestrian detection," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 1, pp. 107-117, Feb. 2011.
3. G. Martínez-Muñoz and A. Suárez, "Using all data to generate decision tree ensembles," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 34, no. 4, pp. 393-397, Nov. 2004.
4. A. L. Strehl, "Probably approximately correct (PAC) exploration in reinforcement learning," Ph.D. dissertation, Comput. Sci. Dept., The State University of New Jersey, Piscataway, NJ, 2007.
5. Y. Freund and R. E. Schapire, "A decision-theoretic generalization of online learning and an application to boosting," J. Comput. Syst. Sci., vol. 55, no. 1, pp. 119-139, 1997. (Pubitemid 127433398)
6. R. E. Schapire, "Theoretical views of boosting," in Proc. 4th Eur. Conf. Comput. Learn. Theory, Nordkirchen, Germany, 1999, pp. 1-10.
7. R. E. Schapire and Y. Singer, "Improved boosting algorithms using confidence-rated predictions," Mach. Learn., vol. 37, no. 3, pp. 297-336, 1999. (Pubitemid 32210620)
8. H.Drucker, C.Cortes, L. C.Jackl, Y.LeCun, and V.Vapnik, "Boosting and other ensemble methods," Neural Comput., vol. 6, no. 6, pp. 1289-1301, Nov. 1994.
9. R. E.Schapire, Y.Freund, P. Bartlett, and W. S. Lee, "Boosting the margin: A new explanation for the effectiveness of voting methods," Ann. Statist., vol. 26, no. 5, pp. 1651-1686, May 1998. (Pubitemid 128376902)
10. G. Rätsch, T. Onoda, and K.-R. Müller, "Soft margins for AdaBoost," Mach. Learn., vol. 42, no. 3, pp. 287-320, Mar. 2001. (Pubitemid 32188795)
11. J. J. Rodríguez and J. Maudes, "Boosting recombined weak classifiers," Pattern Recognit. Lett., vol. 29, no. 8, pp. 1049-1059, 2008. (Pubitemid 351522343)
12. Y. L. Gao, J. Y. Pan, G. L. Ji, and F. Gao, "A time-series modeling method based on the boosting gradient-descent theory," Sci. China Technol. Sci., vol. 54, no. 5, pp. 1325-1337, May 2011.
13. D. L. Shrestha and D. P. Solomatine, "Experiments with AdaBoost.RT, an improved boosting scheme for regression," Neural Comput., vol. 18, no. 7, pp. 1678-1710, Jul. 2006. (Pubitemid 44024734)
14. H. Saigo, S. Nowozin, T. Kadowaki, T. Kudo, and K. Tsuda, "gBoost: A mathematical programming approach to graph classification and regression," Mach. Learn., vol. 75, no. 1, pp. 69-89, 2009.
15. R. Busa-Fekete and B. Kégl, "Accelerating AdaBoost using UCB," J. Mach. Learn. Res.-Proc. Track, vol. 7, pp. 111-222, 2009.
16. W. M. Hu, W. Hu, and S. Maybank, "AdaBoost-based algorithm for network intrusion detection," IEEE Trans. Syst. Man, Cybern. B, Cybern., vol. 38, no. 2, pp. 577-583, Apr. 2008. (Pubitemid 351484757)
17. D. Mease and A. Wyner, "Evidence contrary to the statistical view of boosting," J. Mach. Learn. Res., vol. 9, pp. 131-156, 2008. (Pubitemid 351469030)
18. G. Rätsch, "Robust boosting via convex optimization: Theory and applications," Ph.D. dissertation, Comput. Sci. Dept., University of Potsdam, Potsdam, Germany, 2001.
19. L. Breiman, "Arcing classifiers," Ann. Statist., vol. 26, no. 3, pp. 801-824, Jun. 1998.
20. L. Nanni and A. Franco, "Reduced reward-punishment editing for building ensembles of classifiers," Expert Syst. Appl., vol. 38, no. 3, pp. 2395-2400, Mar. 2011.
21. Y. L. Gao and F. Gao, "Edited AdaBoost by weighted kNN," Neurocomputing, vol. 73, no. 16-18, pp. 3079-3088, Oct. 2010.
22. T. Kanamori, T. Takenouchi, S. Eguchi, and N. Murata, "Robust loss functions for boosting," Neural Comput., vol. 19, no. 8, pp. 2183-2244, 2007.
23. T. Zhang, "Statistical behavior and consistency of classification methods based on convex risk minimization," Ann. Statist., vol. 32, no. 1, pp. 56-134, 2003.
24. Y. Sun, S. Todorovic, and J. Li, "Reducing the overfitting of AdaBoost by controlling its data distribution skewness," Int. J. Pattern Recognit. Artif. Intell., vol. 20, no. 7, pp. 1093-1116, 2006. (Pubitemid 46031113)
25. L. Mason, J. Baxter, P. L. Bartlett, and M. R. Frean, Boosting Algorithms as Gradient Descent, NIPS. Cambridge, MA: MIT Press, 1999, pp. 512-518.
26. C. Domingo, O. Watanabe, and MAdaBoost, "A modification of Ad-aBoost," in Proc. 13th Annu. Conf. Comp. Learn. Theory, 2000, pp. 180-189.
27. A. Vezhnevets and O. Barinova, Avoiding Boosting Overfitting by Removing Confusing shamples, ECML (Lecture Notes in Computer Science Series), vol. 4701. Berlin, Germany: Springer-Verlag, 2007, pp. 430-441.
28. R. E. Schapire and Y. Singer, "Improved boosting algorithms using confidence-rated predictions," Mach. Learn., vol. 37, no. 3, pp. 297-336, 1999. (Pubitemid 32210620)
29. J. Friedman, T. Hastie, and R. Tibshirani, "Additive logistic regression: A statistical view of boosting," Ann. Statist., vol. 28, no. 2, pp. 337-407, 2000. (Pubitemid 33227445)
30. R. Jin, Y. Liu, L. Si, J. Carbonell, and A. G. Hauptmann, "A new boosting algorithm using input-dependent regularizer," in Proc. 20th Int. Conf. Mach. Learn., Washington, DC, Aug. 2003, pp. 21-24.
31. R. Meir and G. Rätsch, "An introduction to boosting and leveraging," in Advanced Lectures on Machine Learning (Lecture Notes in Computer Science Series), vol. 2600. Berlin, Germany: Springer-Verlag, 2002, pp. 118-183.
32. G. C Cawley and N. L. C. Talbot, "On over-fitting in model selection and subsequent selection bias in performance evaluation," J. Mach. Learn. Res., vol. 11, pp. 2079-2107, Jul. 2010.
33. L. Mason, P. L. Bartlett, and J. Baxter, "Improved generalization through explicit optimization of margins," Mach. Learn., vol. 38, no. 3, pp. 243-255, 2000. (Pubitemid 30572451)
34. J. Zhu, H. Zou, S. Rosset, and T. Hastie, "Multi-class AdaBoost," Statist. Interface, vol. 2, pp. 349-360, 2009.
35. Z. Wang, C. Fang, and X. Ding, "Asymmetric real AdaBoost," in Proc. IEEE 19th Int. Conf. Pattern Recognit., 2008, pp. 1-4.
36. G. Rätsch, T. Onoda, and K. R. Müller, "An improvement of AdaBoost to avoid overfitting," in Proc. Int. Conf. Neural Inf. Process., 1998, pp. 506-509.
37. S. Geman, E. Bienenstock, and R. Doursat, "Neural networks and the bias/variance dilemma," Neural Comput., vol. 4, no. 1, pp. 1-58, Jan. 1992.