When is there a representer theorem? Vector versus matrix regularizers

Journal of Machine Learning Research

Volumn 10, Issue , 2009, Pages 2507-2529

  Argyriou, Andreas ^a   Micchelli, Charles A ^b   Pontil, Massimiliano ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b STATE UNIVERSITY OF NEW YORK   (United States)

Author keywords

Kernel methods; Matrix learning; Minimal norm interpolation; Multi task learning; Regularization

Indexed keywords

BASIC PRINCIPLES; GENERAL CLASS; INPUT DATAS; KERNEL BASED METHODS; KERNEL METHODS; LINEAR COMBINATIONS; LINEAR MEASUREMENTS; MACHINE-LEARNING; MATRIX; MATRIX LEARNING; MATRIX THEORY; MULTITASK LEARNING; NON-DECREASING FUNCTIONS; PRACTICAL IMPORTANCE; REGULARIZATION METHODS; REGULARIZER; REPRESENTER THEOREM; SUFFICIENT CONDITIONS;

DYNAMIC LOADS; INTERPOLATION; THEOREM PROVING;

MATRIX ALGEBRA;

EID: 73549115421     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (48)

1. D. A. Aaker, V. Kumar, and G. S. Day. Marketing Research. John Wiley & Sons, 2004. 8th edition.
2. J. Abernethy, F. Bach, T. Evgeniou, and J.-P. Vert. A new approach to collaborative filtering: operator estimation with spectral regularization. Journal of Machine Learning Research, 10:803-826, 2009.
3. Y. Amit, M. Fink, N. Srebro, and S. Ullman. Uncovering shared structures in multiclass classification. In Proceedings of the Twenty-Fourth International Conference on Machine Learning, 2007.
4. R. K. Ando and T. Zhang. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6:1817-1853, 2005.
5. A. Argyriou, T. Evgeniou, and M. Pontil. Multi-task feature learning. In B. Schölkopf, J. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems 19. MIT Press, 2006.
6. A. Argyriou, C. A. Micchelli, M. Pontil, and Y. Ying. A spectral regularization framework for multi-task structure learning. In J.C. Piatt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20. MIT Press, 2007.
7. A. Argyriou, T. Evgeniou, and M. Pontii. Convex multi-task feature learning. Machine Learning, 73(3):243-272, 2008a. Special Issue on Inductive Transfer Learning; Guest Editors: D. L. Silver and K. P. Bennett.
8. A. Argyriou, A. Maurer, and M. Pontil. An algorithm for transfer learning in a heterogeneous environment. In Proceedings of the European Conference on Machine Learning, volume 5211 of Lecture Notes in Computer Science, pages 71-85. Springer, 2008b.
9. N. Aronszajn. Theory of reproducing kernels. Transactions of the American Mathematical Society, 686:337-404, 1950.
10. B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classifiers. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pages 144-152. ACM, 1992.
11. E. J. Candès and B. Recht. Exact matrix completion via convex optimization. In Foundations of Computational Mathematics, 2009. To appear.
12. G. Cavallanti, N. Cesa-Bianchi, and C. Gentile. Linear algorithms for online multitask classification. In Proceedings of the Twenty-First Annual Conference on Learning Theory, 2008.
13. F. Cucker and S. Smale. On the mathematical foundations of learning. Bulletin of the American Mathematical Society, 39(l):l-49, 2001.
14. E. De Vito, L. Rosasco, A. Caponnetto, M. Piana, and A. Verri. Some properties of regularized kernel methods. Journal of Machine Learning Research, 5:1363-1390, 2004.
15. F. Dinuzzo, M. Neve, G. De Nicolao, and U. P. Gianazza. On the representer theorem and equivalent degrees of freedom of SVR. Journal of Machine Learning Research, 8:2467-2495, 2007.
16. T. Evgeniou, M. Pontil, and T. Poggio. Regularization networks and support vector machines. Advances in Computational Mathematics, 13(l):l-50, 2000.
17. T. Evgeniou, C. A. Micchelli, and M. Pontil. Learning multiple tasks with kernel methods. Journal of Machine Learning Research, 6:615-637, 2005.
18. M. Fazel, H. Hindi, and S. P. Boyd. A rank minimization heuristic with application to minimum order system approximation. In Proceedings, American Control Conference, volume 6, pages 4734-4739, 2001.
19. F. Girosi, M. Jones, and T. Poggio. Regularization theory and neural networks architectures. Neural Computation, 7(2):219-269, 1995.
20. A. E. Hoerl and R. W. Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 42:80-86, 1970.
21. R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, 1985.
22. R. A. Horn and C. R. Johnson. Topics in Matrix Analysis. Cambridge University Press, 1991.
23. A. J. Izenman. Reduced-rank regression for the multivariate linear model. Journal of Multivariate Analysis, 5:248-264, 1975.
24. G. S. Kimeldorf and G. Wahba. A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. The Annals of Mathematical Statistics, 41(2):495-502, 1970.
25. P. J. Lenk, W. S. DeSarbo, P. E. Green, and M. R. Young. Hierarchical Bayes conjoint analysis: recovery of partworth heterogeneity from reduced experimental designs. Marketing Science, 15 (2):173-191, 1996. (Pubitemid 126110448)
26. A. Maurer. Bounds for linear multi-task learning. Journal of Machine Learning Research, 7:117-139, 2006a.
27. A. Maurer. The Rademacher complexity of linear transformation classes. In Proceedings of the Nineteenth Annual Conference on Learning Theory, volume 4005 of LNAI, pages 65-78. Springer, 2006b.
28. C. A. Micchelli and A. Pinkus. Variational problems arising from balancing several error criteria. Rendiconti di Matematica, Serie VII, 14:37-86, 1994.
29. C. A. Micchelli and M. Pontil. A function representation for learning in Banach spaces. In Proceedings of the Nineteenth Annual Conference on Learning Theory, volume 3120 of Lecture Notes in Computer Science, pages 255-269. Springer, 2004.
30. C. A. Micchelli and M. Pontil. Learning the kernel function via regularization. Journal of Machine Learning Research, 6:1099-1125, 2005a.
31. C. A. Micchelli and M. Pontii. On learning vector-valued functions. Neural Computation, 17: 177-204, 2005b.
32. C. A. Micchelli and T. J. Rivlin. Lectures on optimal recovery. In P. R. Turner, editor, Lecture Notes in Mathematics, volume 1129. Springer Verlag, 1985.
33. R. Raina, A. Y. Ng, and D. Koller. Constructing informative priors using transfer learning. In Proceedings of the Twenty-Third International Conference on Machine Learning, 2006.
34. B. Recht, M. Fazei, and P. A. Parrilo. Guaranteed minimum rank solutions to linear matrix equations via nuclear norm minimization. Preprint, 2008.
35. B. Schölkopf and A. J. Smola. Learning with Kernels. The MIT Press, 2002.
36. B. Schölkopf, R. Herbrich, and A.J. Smola. A generalized representer theorem. In Proceedings of the Fourteenth Annual Conference on Computational Learning Theory, 2001.
37. J. Shawe-Taylor and N. Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.
38. N. Srebro, J. D. M. Rennie, and T. S. Jaakkola. Maximum-margin matrix factorization. In Advances in Neural Information Processing Systems 17, pages 1329-1336. MIT Press, 2005.
39. I. Steinwart. Sparseness of support vector machines. Journal of Machine Learning Research, 4: 1071-1105, 2003.
40. A. N. Tikhonov and V. Y. Arsenin. Solutions of III Posed Problems. V. H. Winston and Sons (distributed by Wiley), 1977. F. John, Translation Editor.
41. L. Vandenberghe and S. Boyd. Semidefinite programming. SIAM Review, 38(1):49-95, 1996.
42. V. N. Vapnik. The Nature of Statistical Learning Theory. Springer, 2000.
43. G. Wahba. Spline Models for Observational Data, volume 59 of Series in Applied Mathematics. SIAM, Philadelphia, 1990.
44. G. Wahba. Multivariate function and operator estimation, based on smoothing splines and reproducing kernels. In Nonlinear Modeling and Forecasting, SFI Studies in the Sciences of Complexity, volume XII. Addison-Wesley, 1992.
45. L. Wolf, H. Jhuang, and T. Hazan. Modeling appearances with low-rank SVM. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1-6, 2007.
46. Z. Xiang and K. P. Bennett. Inductive transfer using kernel multitask latent analysis. In Inductive Transfer : 10 Years Later, NIPS Workshop, 2005.
47. T. Xiong, J. Bi, B. Rao, and V. Cherkassky. Probabilistic joint feature selection for multi-task learning. In Proceedings of SIAM International Conference on Data Mining, 2006.
48. M. Yuan, A. Ekici, Z. Lu, and R. Monteiro. Dimension reduction and coefficient estimation in multivariate linear regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69(3):329-346, 2007.