Cross-validation optimization for large scale structured classification kernel methods

Journal of Machine Learning Research

Volumn 9, Issue , 2008, Pages 1147-1178

  Seeger, Matthias W ^a  

^a MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

Author keywords

Cross validation optimization; Hierarchical classification; Kernel logistic regression; Multi way classification; Newton Raphson optimization

Indexed keywords

CLASSIFICATION (OF INFORMATION); CONJUGATE GRADIENT METHOD; GRADIENT METHODS; KETONES; MATRIX ALGEBRA; OPTIMIZATION; PARAMETER ESTIMATION; SHAPE OPTIMIZATION; TEXT PROCESSING; VECTORS;

(SPM) CLASSES; APPLIED (CO); CODE OPTIMIZATION; CONJUGATE GRADIENT (CG); CROSS-VALIDATION (CV); GRADIENT-BASED; HIERARCHICAL STRUCTURING; HIGHLY EFFICIENT; IN ORDER; KERNEL MATRICES; KERNEL METHODS; KERNEL PARAMETERS; LARGE SCALES; LEARNING METHODS; LOG LIKELIHOOD; MATRIX VECTOR MULTIPLICATION (MVM); MULTI CLASS; ORDER-OF MAGNITUDES; PENALIZED LIKELIHOOD (PML); SPECIALIZATION (ORDER); TEXT CLASSIFICATION (TC); TO MANY;

STRUCTURAL OPTIMIZATION;

EID: 46249123756     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (36)

1. F. Bach and M. Jordan. Kernel independent component analysis. Journal of Machine Learning Research, 3:1-48, 2002.
2. P. Bartlett and A. Tewari. Sparseness vs estimating conditional probabilities: Some asymptotic results. In Conference on Computational Learning Theory 17, pages 564-578. Springer, 2004.
3. D. Bertsekas. Nonlinear Programming. Athena Scientific, 2nd edition, 1999.
4. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2002.
5. L. Cai and T. Hofmann. Hierarchical document categorization with support vector machines. In CIKM 13, pages 78-87, 2004.
6. K. Crammer and Y. Singer. On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research, 2:265-292, 2001.
7. P. Craven and G. Wahba. Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation. Numerische Mathematik, 31:377-403, 1979.
8. L. Csató. Gaussian Processes -Iterative Sparse Approximations. PhD thesis, Aston University, Birmingham, UK, March 2002.
9. S. Della Pietra, V. Della Pietra, and J. Lafferty. Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4):380-393, 1997.
10. A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of Roy. Stat. Soc. B, 39:1-38, 1977.
11. M. Gibbs. Bayesian Gaussian Processes for Regression and Classification. PhD thesis, University of Cambridge, 1997.
12. P.J. Green and B. Silverman. Nonparametric Regression and Generalized Linear Models. Monographs on Statistics and Probability. Chapman & Hall, 1994.
13. C.-W. Hsu and C.-J. Lin. A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks, 13:415-425, 2002.
14. S. Keerthi and D. DeCoste. A modified finite Newton method for fast solution of large scale linear SVMs. Journal of Machine Learning Research, 6:341-361, 2005.
15. S. Keerthi, V. Sindhwani, and O. Chapelle. An efficient method for gradient-based adaptation of hyperparameters in SVM models. In B. Schölkopf, J. Platt, and T. Hofmann, editors, Advances in Neural Information Processing Systems 19. MIT Press, 2007.
16. J. Lafferty, Y. Liu, and X. Zhu. Kernel conditional random fields: Representation, clique selection, and semi-supervised learning. Technical Report CMU-CS-04-115, Carnegie Mellon University, 2004.
17. Y. LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard, and L. Jackel. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1:541-551, 1989.
18. P. McCullach and J.A. Nelder. Generalized Linear Models. Number 37 in Monographs on Statistics and Applied Probability. Chapman & Hall, 1st edition, 1983.
19. M. Opper and O. Winther. Gaussian processes for classification: Mean field algorithms. Neural Computation, 12(11):2655-2684, 2000.
20. B. Pearlmutter. Fast exact multiplication by the Hessian. Neural Computation, 6(1): 147-160, 1994.
21. J. Piatt. Fast training of support vector machines using sequential minimal optimization. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods: Support Vector Learning, pages 185-208. MIT Press, 1998.
22. Y. Qi, T. Minka, R. Picard, and Z. Ghahramani. Predictive automatic relevance determination by expectation propagation. In C. Brodley, editor, International Conference on Machine Learning 21. Morgan Kaufmann, 2004.
23. C. E. Rasmussen and C. K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006.
24. Y. Saad. Iterative Methods for Sparse Linear Systems. International Thomson Publishing, 1st edition, 1996.
25. B. Schölkopf and A. Smola. Learning with Kernels. MIT Press, 1st edition, 2002.
26. M. Seeger. Bayesian Gaussian Process Models: PAC-Bayesian Generalisation Error Bounds and Sparse Approximations. PhD thesis, University of Edinburgh, July 2003. See www.kyb.tuebingen.mpg.de/bs/people/ seeger.
27. M. Seeger. Gaussian processes for machine learning. International Journal of Neural Systems, 14 (2):69-106, 2004.
28. M. Seeger. Cross-validation optimization for large scale hierarchical classification kernel methods. In B. Schölkopf, J. Platt, and T. Hofmann, editors, Advances in Neural Information Processing Systems 19, pages 1233-1240. MIT Press, 2007.
29. B. Shahbaba and R. Neal. Improving classification when a class hierarchy is available using a hierarchy-based prior. Bayesian Analysis, 2(1):221-238, 2007.
30. Y. Shen, A. Ng, and M. Seeger. Fast Gaussian process regression using KD-trees. In Y. Weiss, B. Schölkopf, and J. Piatt, editors, Advances in Neural Information Processing Systems 18. MIT Press, 2006.
31. E. Snelson and Z. Ghahramani. Sparse Gaussian processes using pseudo-inputs. In Y. Weiss, B. Schölkopf, and J. Piatt, editors, Advances in Neural Information Processing Systems 18. MIT Press, 2006.
32. Y.-W. Teh, M. Seeger, and M. I. Jordan. Semiparametric latent factor models. In Z. Ghahramani and R. Cowell, editors, Workshop on Artificial Intelligence and Statistics 10, 2005.
33. G. Wahba. Spline Models for Observational Data. CBMS-NSF Regional Conference Series. Society for Industrial and Applied Mathematics, 1990.
34. C. Williams and C. Rasmussen. Gaussian processes for regression. In D. Touretzky, M. Mozer, and M. Hasselmo, editors, Advances in Neural Information Processing Systems 8. MIT Press, 1996.
35. C. K. I. Williams and D. Barber. Bayesian classification with Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12):1342-1351, 1998.
36. C. Yang, R. Duraiswami, and L. Davis. Efficient kernel machines using the improved fast Gauss transform. In L. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems 17, pages 1561-1568. MIT Press, 2005.