A scalable stagewise approach to large-margin multiclass loss-based boosting

IEEE Transactions on Neural Networks and Learning Systems

Volumn 25, Issue 5, 2014, Pages 1002-1013

  Paisitkriangkrai, Sakrapee ^a   Shen, Chunhua ^a   Van Den Hengel, Anton ^a  

^a UNIVERSITY OF ADELAIDE   (Australia)

Author keywords

Boosting; column generation; convex optimization; multiclass classification.

Indexed keywords

ADAPTIVE BOOSTING; COMPUTER VISION; CONVEX OPTIMIZATION;

BOOSTING; CLASSIFICATION ACCURACY; CLASSIFICATION MODELS; COLUMN GENERATION; COMPUTATIONALLY EFFICIENT; GENERALIZATION PERFORMANCE; MULTI-CLASS CLASSIFICATION; MULTICLASS CLASSIFICATION PROBLEMS;

CLASSIFICATION (OF INFORMATION);

EID: 84899492155     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2013.2282369     Document Type: Article

References (33)

1. C. Shen and Z. Hao,"A direct formulation for totally-corrective multiclass boosting," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 2585-2592.
2. S. Knerr, L. Personnaz, and G. Dreyfus,"Handwritten digit recognition by neural networks with single-layer training," IEEE Trans. Neural Netw., vol. 3, no. 6, pp. 962-968, Nov. 1992. (Pubitemid 23580837)
3. G. Griffin, A. D. Holub, and P. Perona,"The caltech-256," Dept. Electr. Eng. Comput. Neural Syst., California Inst. Technol., Pasadena, CA, USA, Tech. Rep. 7694, 2007.
4. S. Lazebnik, C. Schmid, and J. Ponce,"Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Nov. 2006, pp. 2169-2178. (Pubitemid 44931582)
5. R. Schapire,"Theoretical views of boosting and applications," in Proc. Int. Conf. Algorithmic Learn. Theory, London, U.K., 1999, pp. 13-25.
6. K. Crammer and Y. Singer,"On the algorithmic implementation of multiclass kernel-based vector machines," J. Mach. Learn. Res., vol. 2, pp. 265-292, Dec. 2002.
7. C. Shen and H. Li,"Boosting through optimization of margin distributions," IEEE Trans. Neural Netw., vol. 21, no. 4, pp. 659-666, Apr. 2010.
8. N. Garcia-Pedrajas,"Constructing ensembles of classifiers by means of weighted instance selection," IEEE Trans. Neural Netw., vol. 20, no. 2, pp. 258-277, Feb. 2009.
9. H. He and Y. Cao,"SSC: A classifier combination method based on signal strength," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1100-1117, Jul. 2012.
10. S. Paisitkriangkrai, C. Shen, Q. Shi, and A. van den Hengel,"Random- Boost: Simplified multi-class boosting through randomization," IEEE Trans. Neural Netw. Learn. Syst., to be published.
11. P. Sun and X. Yao,"Sparse approximation through boosting for learning large scale kernel machines," IEEE Trans. Neural Netw., vol. 21, no. 6, pp. 883-894, Jun. 2010.
12. P. Wang, C. Shen, N. Barnes, and H. Zheng,"Fast and robust object detection using asymmetric totally corrective boosting," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 1, pp. 33-46, Jan. 2012.
13. R. Schapire and Y. Singer,"Improved boosting algorithms using confidence-rated prediction," Mach. Learn., vol. 37, no. 3, pp. 297-336, 1999. (Pubitemid 32210620)
14. V. Guruswami and A. Sahai,"Multiclass learning, boosting, and error correcting codes," in Proc. Annu. Conf. Learn. Theory, 1999, pp. 145-155. (Pubitemid 32213951)
15. B. Zhang, G. Ye, Y. Wang, J. Xu, and G. Herman,"Finding shareable informative patterns and optimal coding matrix for multiclass boosting," in Proc. IEEE Int. Conf. Comp. Vis., Sep./Oct. 2009, pp. 56-63.
16. A. Torralba, K. P. Murphy, and W. T. Freeman,"Sharing visual features for multiclass and multiview object detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 5, pp. 854-869, May 2007. (Pubitemid 46534781)
17. R. Schapire,"Using output codes to boost multiclass learning problems," in Proc. 14th Int. Conf. Mach. Learn., 1997, pp. 1-9.
18. L. Li,"Multiclass boosting with repartitioning," in Proc. Int. Conf. Mach. Learn., 2006, pp. 569-576.
19. J. Duchi and Y. Singer,"Boosting with structural sparsity," in Proc. 26th Annu. Int. Conf. Mach. Learn., 2009, pp. 297-304.
20. J. Friedman, T. Hastie, and R. Tibshirani,"Additive logistic regression: A statistical view of boosting," Ann. Stat., vol. 28, no. 2, pp. 337-407, 2000. (Pubitemid 33227445)
21. A. Demiriz, K. P. Bennett, and J. Shawe-Taylor,"Linear programming boosting via column generation," Mach. Learn., vol. 46, nos. 1-3, pp. 225-254, 2002. (Pubitemid 34129970)
22. C. Shen and H. Li,"On the dual formulation of boosting algorithms," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 12, pp. 2216-2231, Dec. 2010.
23. S. Rosset, J. Zhu, and T. Hastie,"Boosting as a regularized path to a maximum margin classifier," J. Mach. Learn. Res., vol. 5, pp. 941-973, Aug. 2004.
24. M. Grant and S. Boyd. (2011, Apr.). CVX: Matlab Software for Disciplined Convex Programming, Version 1.21 [Online]. Available: http://cvxr.com/cvx
25. T. Zhang and B. Yu,"Boosting with early stopping: Convergence and consistency," Ann. Stat., vol. 33, no. 4, pp. 1538-1579, 2005. (Pubitemid 41423980)
26. P. Carbonetto,"A MATLAB interface for L-BFGS-B," Dept. Comput. Sci., Univ. Brit. Columbia, Vancouver, BC, Canada, Tech. Rep., 2007.
27. J. Demvsar,"Statistical comparisons of classifiers over multiple data sets," J. Mach. Learn. Res., vol. 7, pp. 1-30, Jan. 2006.
28. J. Zhu, H. Zou, S. Rosset, and T. Hastie,"Multi-class AdaBoost," Stat. Inter., vol. 2, no. 3, pp. 349-360, 2009.
29. A. Daniely, S. Sabato, and S. Shalev-Shwartz,"Multiclass learning approaches: A theoretical comparison with implications," in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2012.
30. J. Wu and J. M. Rehg,"CENTRIST: A visual descriptor for scene categorization," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 8, pp. 1489-1501, Aug. 2011.
31. A. Bosch, A. Zisserman, and X. Munoz,"Scene classification using a hybrid generative/discriminative approach," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 4, pp. 712-727, Apr. 2008. (Pubitemid 351559723)
32. C-C. Chang and C.-J. Lin,"LIBSVM: A library for support vector machines," ACM Trans. Intell. Syst. Tech., vol. 2, no. 3, pp. 27:1-27:27, 2011.
33. J.Wu and J. M. Rehg,"Beyond the euclidean distance: Creating effective visual codebooks using the histogram intersection kernel," in Proc. IEEE Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 630-637.