Boosting learning algorithm for pattern recognition and beyond

IEICE Transactions on Information and Systems

Volumn E94-D, Issue 10, 2011, Pages 1863-1869

  Komori, Osamu ^a   Eguchi, Shinto ^a  

^a INSTITUTE OF STATISTICAL MATHEMATICS   (Japan)

Author keywords

AUC; Boosting; Density estimation; Divergence; Entropy; ROC; U loss function

Indexed keywords

ADAPTIVE BOOSTING; ARTIFICIAL INTELLIGENCE; ENTROPY; LEARNING SYSTEMS; PATTERN RECOGNITION; PROBABILITY DENSITY FUNCTION;

AREA UNDER THE ROC CURVE; BOOSTING; DENSITY ESTIMATION; DIVERGENCE; KULLBACK LEIBLER DIVERGENCE; LOSS FUNCTIONS; PROBABILISTIC FRAMEWORK; STATISTICAL PROPERTIES;

LEARNING ALGORITHMS;

EID: 80053395623     PISSN: 09168532     EISSN: 17451361     Source Type: Journal    
DOI: 10.1587/transinf.E94.D.1863     Document Type: Review

References (32)

1. G.J. McLachlan, Discriminant Analysis and Statistical Pattern Recognition, Wiley & Sons, Hoboken, 2004.
2. C.M. Bishop, Pattern Recognition and Machine Learning, Springer, New York, 2006.
3. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction (second edition), Springer, New York, 2009.
4. J. Friedman, T. Hastie, and R. Tibshirani, "Additive logistic regression: A statistical view of boosting," The Annals of Statistics, vol.28, pp.337-407, 2000. (Pubitemid 33227445)
5. N. Murata, T. Takenouchi, T. Kanamori, and S. Eguchi, "Information geometry of U-Boost and Bregman divergence," Neural Comput., vol.16, pp.1437-1481, 2004. (Pubitemid 38740017)
6. S. Eguchi, "Information geometry and statistical pattern recognition," Sugaku Expositions, vol.19, pp.197-216, 2006.
7. T. Takenouchi and S. Eguchi, "Robustifying AdaBoost by adding the naive error rate," Neural Comput., vol.16, pp.767-787, 2004. (Pubitemid 38318131)
8. T. Zhang and B. Yu, "Boosting with early stopping: Convergence and consistency," The Annals of Statistics, vol.4, pp.1538-1579, 2005. (Pubitemid 41423980)
9. S. Eguchi and J. Copas, "A class of logistic-type discriminant functions," Biometrika, vol.89, pp.1-22, 2002. (Pubitemid 41311999)
10. D. Bamber, "The area above the ordinal dominance graph and the area below the receiver operating characteristic graph," J. Mathematical Psychology, vol.12, pp.387-415, 1975.
11. S. Ma and J. Huang, "Regularized ROC method for disease classification and biomarker selection with microarray data," Bioinformatics, vol.21, pp.4356-4362, 2005. (Pubitemid 41782925)
12. Z.Wang, Y.I. Chang, Z. Ying, L. Zhu, and Y. Yang, "A parsimonious threshold-independent pretein feature selection method through the area under receiver operating characteristic curve," Bioinformatics, vol.23, pp.2788-1794, 2007.
13. O. Komori, "A boosting method for maximization of the area under the ROC curve," Annals of the Institute of Statistical Mathematics, 2009. (online).
14. O. Komori and S. Eguchi, "A boosting method for maximizing the partial area under the ROC curve," BMC Bioinformatics, vol.11, p.314, 2010.
15. M.S. Pepe and M.L. Thompson, "Combining diagnostic test results to increase accuracy," Biostatistics, vol.1, pp.123-140, 2000. (Pubitemid 33213526)
16. Z.Wang and Y.I. Chang, "Markers selection via maximizing the partial area unber the ROC curve of linear risk scores," Biostatistics, vol.12, pp.369-385, 2011.
17. M.S. Pepe, The Statistical Evaluation of Medical Tests for Classification and Prediction, Oxford University Press, New York, 2003.
18. M.W. McIntosh and M.S. Pepe, "Combining several screening tests: Optimality of the risk score," Biometrics, vol.58, pp.657-664, 2002. (Pubitemid 35001484)
19. Y. Freund and R.E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," J. Computer and System Sciences, vol.55, pp.119-139, 1997. (Pubitemid 127433398)
20. G. Tutz and H. Binder, "Generalized additive modeling with implicit variable selection by likelihood-based boosting," Biometrics, vol.62, pp.961-971, 2006.
21. G. Ridgeway, "Looking for lumps: Boosting and bagging for density estimation," Computational Statistics & Data Analysis, vol.38, pp.379-392, 2002. (Pubitemid 34197168)
22. S. Rosset and E. Segal, "Boosting density estimation," Advances in Neural Information Processing System 15, 2003.
23. M.D. Marzio and C.C. Taylor, "On boosting kernel density methods for multivariate data: Density estimation and classification," Statistical Methods & Applications, vol.14, pp.163-178, 2005. (Pubitemid 41521318)
24. J. Klemelä, "Density estimation with stagewise optimization of the empirical risk," Mach. Learn., vol.67, pp.169-195, 2007. (Pubitemid 46616360)
25. J. Klemelä, Smoothing of Multivariate Data, Density Estimation and Visualization, John Wiley & Sons, Hoboken, New Jersey, 2009.
26. K. Naito and S. Eguchi, "Density estimation with minimization of U-divergence," submitted, 2010.
27. O. Komori, K. Naito, and S. Eguchi, "Boosting for density estimation based on U loss function," IEICE Technical Report, IBISML2010-69, 2010.
28. A. Basu, I.R. Harris, N. Hjort, and M. Jones, "Robust and efficient estimation by minimizing a density power divergence," Biometrika, vol.85, pp.549-559, 1998.
29. M. Minami and S. Eguchi, "Robust blind source separation by beta divergence," Neural Comput., vol.14, pp.1859-1886, 2002.
30. M.N.H. Mollah, M. Minami, and S. Eguchi, "Exploring latent structure of mixture ica models by the minimum beta-divergence method," Neural Comput., vol.18, pp.166-190, 2006. (Pubitemid 43543855)
31. M.N.H. Mollah, N. Sultana, M. Minami, and S. Eguchi, "Robust extraction of local structures by the minimum beta-divergence method," Neural Netw., vol.23, pp.226-238, 2010.
32. F. Emmert-Streib and M. Dehmer, Information Theory and Statistical Learning, Springer, New York, 2009