Boosting through optimization of margin distributions

IEEE Transactions on Neural Networks

Volumn 21, Issue 4, 2010, Pages 659-666

  Shen, Chunhua ^a,b   Li, Hanxi ^a,b  

^a CSIRO   (Australia)

^b AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

AdaBoost; Boosting; Column generation; Margin distribution

Indexed keywords

ADABOOST; BOOSTING ALGORITHM; COLUMN GENERATION; DATA SETS; GENERALIZATION ERROR; LOSS FUNCTIONS; MACHINE LEARNING COMMUNITIES; MARGIN THEORY; OPTIMIZATION ALGORITHMS; REGRESSION PROBLEM; TRAINING DATA;

ALGORITHMS; OPTIMIZATION;

ADAPTIVE BOOSTING;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; HUMAN; INFORMATION PROCESSING; LEARNING;

ALGORITHMS; AUTOMATIC DATA PROCESSING; COMPUTER SIMULATION; GENERALIZATION (PSYCHOLOGY); HUMANS; NEURAL NETWORKS (COMPUTER);

EID: 77950861838     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2040484     Document Type: Article

References (21)

1. R. E. Schapire, Y. Freund, P. Bartlett, and W. S. Lee, "Boosting the margin: A new explanation for the effectiveness of voting methods," Ann. Stat., vol.26, no.5, pp. 1651-1686, 1998.
2. R. Meir and G. Rätsch, An Introduction to Boosting and Leveraging, ser. Advanced Lectures on Machine Learning. New York: Springer- Verlag, 2003, pp. 118-183.
3. L. Breiman, "Prediction games and arcing algorithms," Neural Comput., vol.11, no.7, pp. 1493-1517, 1999.
4. J. Friedman, T. Hastie, and R. Tibshirani, "Rejoiner for additive logistic regression: A statistical view of boosting," Ann. Stat., vol.28, pp. 400-407, 2000.
5. L. Mason, J. Baxter, P. Bartlett, and M. Frean, "Boosting algorithms as gradient descent," in Proc. Adv. Neural Inf. Process. Syst., 2000, pp. 512-518.
6. L. Wang, M. Sugiyama, C. Yang, Z.-H. Zhou, and J. Feng, "On the margin explanation of boosting algorithms," in Proc. Annu. Conf. Learn. Theory, Helsinki, Finland, 2008, pp. 479-490.
7. A. J. Grove and D. Schuurmans, "Boosting in the limit: Maximizing the margin of learned ensembles," in Proc. Nat. Conf. Artif. Intell., Madison, WI, 1998, pp. 692-699.
8. G. Rätsch and M. K. Warmuth, "Efficient margin maximizing with boosting," J. Mach. Learn. Res., vol.6, pp. 2131-2152, 2005.
9. M. K. Warmuth, J. Liao, and G. Rätsch, "Totally corrective boosting algorithms that maximize the margin," in Proc. Int. Conf. Mach. Learn., Pittsburgh, PA, 2006, pp. 1001-1008.
10. A. Demiriz, K. P. Bennett, and J. Shawe-Taylor, "Linear programming boosting via column generation," Mach. Learn., vol.46, no.1-3, pp. 225-254, 2002.
11. L. Reyzin and R. E. Schapire, "How boosting the margin can also boost classifier complexity," in Proc. Int. Conf. Mach. Learn., Pittsburgh, PA, 2006, pp. 753-760.
12. A. Garg and D. Roth, "Margin distribution and learning," in Proc. Int. Conf. Mach. Learn., Washington, DC, 2003, pp. 210-217.
13. H. Lodhi, G. J. Karakoulas, and J. Shawe-Taylor, "Boosting the margin distribution," in Proc. Int. Conf. Intell. Data Eng. Autom. Learn./Data Mining Financial Eng. Intelli. Agents, London, U.K., 2000, pp. 54-59.
14. J. Shawe-Taylor and N. Cristianini, "Further results on the margin distribution," in Proc. Annu. Conf. Learn. Theory, Santa Cruz, CA, 1999, pp. 278-285.
15. C. Shen and H. Li, "On the dual formulation of boosting algorithms," IEEE Trans. Pattern Anal. Mach. Intell., 2010 [Online]. Available: http://arxiv:0901.3590, to be published
16. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
17. G. B. Dantzig and P. Wolfe, "Decomposition principle for linear programs," Operat. Res., vol.8, no.1, pp. 101-111, 1960.
18. M. E. Lübbecke and J. Desrosiers, "Selected topics in column generation," Operat. Res., vol.53, no.6, pp. 1007-1023, 2005.
19. G. Rätsch, T. Onoda, and K.-R. Müller, "Soft margins for AdaBoost," Mach. Learn., vol.42, no.3, pp. 287-320, 2001.
20. MOSEK ApS, The MOSEK Optimization Toolbox for Matlab Manual, Version 5.0, Revision 93, 2008 [Online]. Available: http://www.mosek. com/
21. J. Demšar, "Statisitcal comparisons of classifiers over multiple data sets," J. Mach. Learn. Res., vol.7, pp. 1-30, Dec. 2006.