Learning the kernel with hyperkernels

Journal of Machine Learning Research

Volumn 6, Issue , 2005, Pages

  Ong, Cheng Soon ^a   Smola, Alexander J ^b   Williamson, Robert C ^b  

^a MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

^b AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

Capacity control; Kernel methods; Learning the kernel; Representer theorem; Semidefinite programming; Support vector machines

Indexed keywords

AUTOMATION; CLASSIFICATION (OF INFORMATION); OPTIMIZATION; PARAMETER ESTIMATION; PROBLEM SOLVING; RISKS; VECTORS;

CAPACITY CONTROL; KERNEL METHODS; LEARNING THE KERNEL; REPRESENTER THEOREM; SEMIDEFINITE PROGRAMMING; SUPPORT VECTOR MACHINES;

LEARNING SYSTEMS;

EID: 21844468979     PISSN: 15337928     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (45)

1. A. Albert. Conditions for positive and nonnegative definiteness in terms of pseudoinverses. SIAM Journal on Applied Mathematics, 17(2):434-440, 1969.
2. N. Aronszajn. Theory of reproducing kernels. Transactions of the American Mathematical Society, 68:337-404, 1950.
3. F. R. Bach and M. I. Jordan. Kernel independent component analysis. Journal of Machine Learning Research, 3:1-48, 2002.
4. K. P. Bennett and O. L. Mangasarian. Robust linear programming discrimination of two linearly inseparable sets. Optimization Methods and Software, 1:23-34, 1992.
5. C. M. Bishop. Neural Networks for Pattern Recognition. Clarendon Press, Oxford, 1995.
6. O. Bousquet and D. Herrmann. On the complexity of learning the kernel matrix. In Advances in Neural Information Processing Systems 15, pages 399-406, 2002.
7. C. J. C. Burges. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2): 121-167, 1998.
8. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee. Choosing multiple parameters for support vector machines. Machine Learning, 46(1): 131-159, 2002.
9. C. Cortes and V. Vapnik. Support vector networks. Machine Learning, 20:273-297, 1995.
10. D. Cox and F. O'Sullivan. Asymptotic analysis of penalized likelihood and related estimators. Annals of Statistics, 18:1676-1695, 1990.
11. K. Crammer, J. Keshet, and Y. Singer. Kernel design using boosting. In Advances in Neural Information Processing Systems 15, pages 537-544, 2002.
12. N. Cristianini, J. Kandola, A. Elisseeff, and J. Shawe-Taylor. On optimizing kernel alignment. Technical report, UC Davis Department of Statistics, 2003.
13. N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. Kandola. On kernel-target alignment. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, pages 367-373, Cambridge, MA, 2002. MIT Press.
14. K. Duan, S. S. Keerthi, and A. N. Poo. Evaluation of simple performance measures for tuning svm hyperparameters. Neurocomputing, 51:41-59, 2003.
15. S. Fine and K. Scheinberg. Efficient SVM training using low-rank kernel representations. Journal of Machine Learning Research, 2:243-264, Dec 2001. http://www.jmlr.org.
16. Y. Freund and R. E. Schapire. Experiments with a new boosting algorithm. In Proceedings of the International Conference on Machine Learing, pages 148-146. Morgan Kaufmann Publishers, 1996.
17. D. Haussler. Convolutional kernels on discrete structures. Technical Report UCSC-CRL-99-10, Computer Science Department, UC Santa Cruz, 1999.
18. R. Herbrich and R. C. Williamson. Algorithmic luckiness. Journal of Machine Learning Research, 3:175-212, 2002.
19. G. S. Kimeldorf and G. Wahba. Some results on Tchebycheffian spline functions. J. Math. Anal. Applic., 33:82-95, 1971.
20. G. Lanckriet, N. Cristianini, P. Bartlett, L. El Ghaoui, and M. Jordan. Learning the kernel matrix with semidefinite programming. In Proceedings of the International Conference on Machine Learning, pages 323-330. Morgan Kaufmann, 2002.
21. G. Lanckriet, N. Cristianini, P. Bartlett, L. El Ghaoui, and M. I. Jordan. Learning the kernel matrix with semi-definite programming. Journal of Machine Learning Research, 5:27-72, 2004.
22. S. L. Lauritzen. Graphical Models. Oxford University Press, 1996.
23. J. Löfberg. YALMIP, yet another LMI parser, 2002. http://www.control.isy.liu.se/~johanl/yalmip.html.
24. A. Luntz and V. Brailovsky. On estimation of characters obtained in statistical procedure of recognition (in Russian). Technicheskaya Kibernetica, 3, 1969.
25. D. J. C. MacKay. Bayesian non-linear modelling for the energy prediction competition. ASHRAE Transcations, 4:448-472, 1994.
26. O. L. Mangasarian and D. R. Musicant. Lagrangian support vector machines. Journal of Machine Learning Research, 1:161-177, 2001.
27. D. Meyer, F. Leisch, and K. Hornik. The support vector machine under test. Neurocomputing, 55 (1-2):169-186, 2003.
28. R. Neal. Bayesian Learning in Neural Networks. Springer, 1996.
29. C. S. Ong and A. J. Smola. Machine learning using hyperkernels. In Proceedings of the International Conference on Machine Learning, pages 568-575, 2003.
30. C. S. Ong, A. J. Smola, and R. C. Williamson. Hyperkernels. In Neural Information Processing Systems, volume 15, pages 495-502. MIT Press, 2002.
31. M. Opper and O. Winther. Gaussian processes and SVM: Mean field and leave-one-out. In A. J. Smola, P. L. Bartlett, B. Schölkopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers, pages 311-326, Cambridge, MA, 2000. MIT Press.
32. G. Rätsch, T. Onoda, and K. R. Müller. Soft margins for adaboost. Machine Learning, 42(3):287-320, 2001.
33. B. Schölkopf, J. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson. Estimating the support of a high-dimensional distribution. Neural Computation, 13(7): 1443-1471, 2001.
34. B. Schölkopf and A. J. Smola. Learning with Kernels. MIT Press, 2002.
35. B. Schölkopf, A. J. Smola, R. C. Williamson, and P. L. Bartlett. New support vector algorithms. Neural Computation, 12:1207-1245, 2000.
36. M. Seeger. Bayesian methods for support vector machines and Gaussian processes. Master's thesis, University of Edinburgh, Division of Informatics, 1999.
37. A. J. Smola, B. Schölkopf, and K.-R. Müller. The connection between regularization operators and support vector kernels. Neural Networks, 11(5):637-649, 1998.
38. J. F. Sturm. Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones. Optimization Methods and Software, 11/12(1-4):625-653, 1999.
39. K. Tsuda, S. Akaho, and K. Asai. The EM algorithm for kernel matrix completion with auxiliary data. Journal of Machine Learning Research, 4:67-81, 2003.
40. L. Vandenberghe and S. Boyd. Semidefinite programming. SIAM Review., 38(1):49-95, 1996.
41. V. Vapnik. The Nature of Statistical Learning Theory. Springer, New York, 1995.
42. G. Wahba. Spline Models for Observational Data, volume 59 of CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, Philadelphia, 1990.
43. C. K. I. Williams and D. Barber. Bayesian classification with Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI, 20(12):1342-1351, 1998.
44. C. K. I. Williams and C. E. Rasmussen. Gaussian processes for regression. In D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, editors, Advances in Neural Information Processing Systems 8, pages 514-520, Cambridge, MA, 1996. MIT Press.
45. T. Zhang. Some sparse approximation bounds for regression problems. In Proc. 18th International Conf. on Machine Learning, pages 624-631. Morgan Kaufmann, San Francisco, CA, 2001.