Superfast-trainable multi-class probabilistic classifier by least-squares posterior fitting

IEICE Transactions on Information and Systems

Volumn E93-D, Issue 10, 2010, Pages 2690-2701

  Sugiyama, Masashi ^a,b  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^b JAPAN SCIENCE AND TECHNOLOGY AGENCY   (Japan)

Author keywords

Classposterior probability; Kernel logistic regression; Probabilistic classification; Squared loss

Indexed keywords

MAXIMUM LIKELIHOOD; NEWTON-RAPHSON METHOD; REGRESSION ANALYSIS;

CLASSIFICATION ACCURACY; CLASSIFICATION ALGORITHM; KERNEL LOGISTIC REGRESSION; POSTERIOR PROBABILITY; PROBABILISTIC CLASSIFICATION; PROBABILISTIC CLASSIFIERS; REGULARIZED LEAST SQUARES; SYSTEM OF LINEAR EQUATIONS;

PROBABILITY;

EID: 77957851853     PISSN: 09168532     EISSN: 17451361     Source Type: Journal    
DOI: 10.1587/transinf.E93.D.2690     Document Type: Article

References (36)

1. S. Amari, "Theory of adaptive pattern classifiers," IEEE Trans. Electronic Computers, vol.EC-16, no.3, pp.299-307, 1.967.
2. B. Bakker and T. Heskes, "Task clustering and gating for Bayesian multitask learning," J. Machine Learning Research, vol.4, pp.83-99, 2003.
3. C.M. Bishop, Pattern Recognition and Machine Learning, Springer, New York, NY, USA, 2006.
4. R. Caruana, L. Pratt, and S. Thrun, "Multitask learning," Mach. Learn., vol.28, p.41, 1997.
5. C.C. Chang and C.J. Lin, "LIBSVM: A library for support vector machines," Tech. Rep., Department of Computer Science, National Taiwan University, 200.1. http://www.csie.ntu.edu.tw/~cjlin/libsvm/
6. R. Collobert and S. Bengio, "SVMTorch: Support vector machines for large-scale regression problems," J. Machine Learning Research, vol.1, pp.143-160, 2001.
7. C. Cortes and V. Vapnik, "Support vector networks," Mach. Learn., vol.20, pp.273-297, 1995.
8. R.O. Duda, P.E. Hart, and D.G. Stor, Pattern Classification, Wiley, New York, 2001.
9. B. Efron, T. Hastie, R. Tibshirani, and I. Johnstone, "Least angle regression," Annals of Statistics, vol.32, no.2, pp.407-499, 2004.
10. R.E. Fan, K.W. Chang, C.J. Hsieh, X.R. Wang, and C.J. Lin, "LIBLINEAR: A library for large linear classification," J. Machine Learning Research, vol.9, pp.1871-1874, 2008.
11. R.E. Fan, P.H. Chen, and C.J. Lin, "Working set selection using second order information for training SVM," J. Machine Learning Research, vol.6, pp. 1889-1918, 2005.
12. V. Franc and S. Sonnenburg, "Optimized cutting plane algorithm, for Large-scale risk minimization," J. Machine Learning Research, vol.10, pp.2157-2192, 2009.
13. G.M. Fung and O.L. Mangasarian, "Multicategory proximal support vector machine classifiers," Mach. Learn., vo.159, no. 1-2, pp.77-97, 2005.
14. T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu, "The entire regularization path for the support vector machine," J. Machine Learning Research, vol.5, pp.1391-1415, 2004.
15. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, New York, 2001.
16. C.W. Hsu. and C.J. Lin, "A comparison of methods for multi-class support vector machines," IEEE Trans. Neural Netw., vol.13, no.2, pp.415-425, 2002.
17. T. Joachims, "Making large-scale SVM learning practical," in Advances in Kernel. Methods-Support Vector Learning, ed. B. Scholkopf, C.J.C. Burges, and A.J. Smola, pp. 169-184, The MIT Press, Cambridge, MA, 1999.
18. T. Joachims, "Training linear SVMs in linear time," ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD2006), pp.217-226, 2006.
19. T. Kanamori, S. Hido, and M. Sugiyama, "A least-squares approach to direct importance estimation," J. Machine Learning Research, vol.10, pp.1391.-1445, July 2009.
20. T. Kanamori, T. Suzuki, and M. Sugiyama, "Condition number analysis of kernel-based density ratio estimation," Tech. Rep., arXiv, 2009.
21. T. Kanamori, T. Suzuki, and M. Sugiyama, "Theoretical analysis of density ratio estimation," IEICE Trans. Fundamentals, vol.E93-A, no.4, pp.787-798, April. 2010.
22. T. Kato, H. Kashima, M. Sugiyama, and K. Asai, "Conic programming for multi-task learning," IEEE Trans. Knowl. Data Eng., vol.22, no.7, pp.957-968, 2010.
23. 1 -regularized logistic regression," J. Machine Learning Research, vol.8, pp.1519-1555, 2007.
24. T.P. Minka, "A comparison of numerical optimizers for logistic regression," Tech. Rep., Microsoft Research, 2007.
25. A.Y. Ng and M.I. Jordan, "On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes," Advances in Neural Information Processing Systems 14, ed. TG. Dietterich, S. Becker, and Z. Ghahramani, MIT Press, Cambridge, MA, 2002.
26. J. Piatt, "Fast training of support vector machines using sequential minimal optimization," in Advances in Kernel Methods-Support Vector Learning, ed. B. Scholkopf, C.J.C. Burges, and A.J. Smola, pp.169-184, The MIT Press, Cambridge, MA, 1999.
27. J. Piatt, "Probabilities for SV machines," in Advances in Large Margin Classifiers, ed. A.J. Smola, P.L. Bartlett, B. Scholkopf, and D. Schuurmans, The MIT Press, Cambridge, MA, 2000.
28. R. Rifkin, G. Yeo, and T. Poggio, "Regularized least-squares classification," Advances in Learning Theory: Methods, Models and Applications, ed. J.A.K. Suykens, G. Horvath, S. Basu, C Micchelli, and J. Vandewalle, NATO Science Series III: Computer & Systems Sciences, vol.190, pp.13.1.-154, IOS Press, Amsterdam, The Netherlands, 2003.
29. M. Schmidt, minFunc, 2005. http://people.cs.ubc.ca/~schmidtm/Software/ minFunc.html
30. B. Scholkopf and A.J. Smola, Learning with Kernels, MIT Press, Cambridge, MA, 2002.
31. J.A.K. Suykens, TV. Gestel., J.D. Brabanter, B.D. Moor, and J. Vandewalle, Least Squares Support Vector Machines, World Scientific Pub. Co., Singapore, 2002.
32. Y. Tang and H.H. Zhang, "Multiclass proximal support vector machines," J. Computational and Graphical Statistics, vol.15, no.2, pp.339-355, 2006.
33. C.H. Teo, Q. Le, A. Smola, and S.V.N. Vishwanathan, "A scalable modular convex solver for regularized risk minimization," ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD2007), pp.727-736, 2007.
34. I. Tsang, J. Kwok, and P.M. Cheung, "Core vector machines: Fast SVM training on very large data sets," J. Machine Learning Research, vo.16, pp.363-392, 2005.
35. V.N. Vapnik, Statistical Learning Theory, Wiley, New York, 1998.
36. T.F. Wu, C.J. Lin, and R.C. Weng, "Probability estimates for multiclass classification by pairwise coupling," J. Machine Learning Research, vol.5, pp.975-1005, 2004.