Multi-output learning via spectral filtering

Machine Learning

Volumn 87, Issue 3, 2012, Pages 259-301

  Baldassarre, Luca ^a   Rosasco, Lorenzo ^b,c   Barla, Annalisa ^d   Verri, Alessandro ^d  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b ISTITUTO ITALIANO DI TECNOLOGIA   (Italy)

^c MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^d UNIVERSITY OF GENOA   (Italy)

Author keywords

Kernel methods; Multi task; Regularization; Spectral filtering; Vector valued functions

Indexed keywords

KERNEL METHODS; MULTI-TASK; REGULARIZATION; SPECTRAL FILTERING; VECTOR-VALUED FUNCTION;

ALGORITHMS; SAMPLING;

VECTORS;

EID: 84862001051     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-012-5282-y     Document Type: Article

References (67)

1. Abernethy, J., Bartlett, P. L., & Rakhlin, A. (2007). Multitask learning with expert advice. In Proceedings of the 20th annual conference on learning theory (pp. 484-498). Berlin: Springer.
2. J. Abernethy F. Bach T. Evgeniou J.-P. Vert 2009 A new approach to collaborative filtering: Operator estimation with spectral regularization Journal of Machine Learning Research 10 803 826
3. Alvarez, M., Luengo, D., & Lawrence, N. (2009). Latent force models. In Proceedings of the twelfth international conference on artificial intelligence and statistics (Vol. 5, pp. 9-16).
4. A. Argyriou T. Evgeniou M. Pontil 2008 Convex multi-task feature learning Machine Learning 73 243 272 10.1007/s10994-007-5040-8
5. Argyriou, A., Maurer, A., & Pontil, M. (2008b). An algorithm for transfer learning in a heterogeneous environment. In The European conference on machine learning and principles and practice of knowledge discovery in databases (ECML PKDD) (pp. 71-85).
6. B. Bakker T. Heskes 2003 Task clustering and gating for Bayesian multitask learning Journal of Machine Learning Research 4 83 99 ISSN 1533-7928
7. Baldassarre, L., Rosasco, L., Barla, A., & Verri, A. (2010). Vector field learning via spectral filtering. In The European conference on machine learning and principles and practice of knowledge discovery in databases (ECML PKDD) (pp. 56-71).
8. J. L. Barron D. J. Fleet S. S. Beauchemin 1994 Performance of optical flow techniques International Journal of Computer Vision 12 1 43 77 10.1007/BF01420984
9. F. Bauer S. Pereverzev L. Rosasco 2007 On regularization algorithms in learning theory Journal of Complexity 23 1 52 72 2297015 10.1016/j.jco.2006.07. 001 ISSN 0885-064X
10. Bishop, C. M. (2006). Pattern recognition and machine learning. New York: Springer.
11. Bonilla, E. V., Chai, K. M., & Williams, C. (2007). Multi-task Gaussian process prediction. In Advances in neural information processing systems (NIPS). Rostrevor: Curran Associates, Inc.
12. O. Bousquet A. Elisseeff 2002 Stability and generalization Journal of Machine Learning Research 2 499 526 1929416 ISSN 1533-7928
13. Boyle, P., & Frean, M. (2005). Dependent Gaussian processes. In Advances in neural information processing systems (NIPS). Cambridge: MIT Press.
14. L. Breiman J. H. Friedman 1997 Predicting multivariate responses in multiple linear regression Journal of the Royal Statistical Society 59 1 3 54 1436554 10.1111/1467-9868.00054
15. Brudnak, M. (2006). Vector-valued support vector regression. In International joint conference on neural networks, 16-21 July 2006 (pp. 1562-1569).
16. 2-loss: Regression and classification Journal of the American Statistical Association 98 324 340 10.1198/016214503000125
17. Caponnetto, A. (2006). Optimal rates for regularization operators in learning theory (Technical report). CBCL paper 264/CSAIL-TR 2006-062, MIT.
18. Caponnetto, A., & De Vito, E. (2005). Risk bounds for regularized least-squares algorithm with operator-valued kernels (Technical report). CBCL paper 249/CSAIL-TR-2005-031, MIT.
19. A. Caponnetto E. De Vito 2007 Optimal rates for regularized least-squares algorithm Foundations of Computational Mathematics 7 3 331 368 2335249 10.1007/s10208-006-0196-8
20. A. Caponnetto C. A. Micchelli M. Pontil Y. Ying 2008 Universal kernels for multi-task learning Journal of Machine Learning Research 9 1615 1646 2426053 ISSN 1533-7928
21. C. Carmeli E. De Vito A. Toigo 2006 Vector valued reproducing kernel Hilbert spaces of integrable functions and Mercer theorem Analysis and Applications 4 4 377 408 2265340 10.1142/S0219530506000838
22. R. Caruana 1997 Multitask learning Machine Learning 28 41 75 10.1023/A:1007379606734
23. Chai, K. M. A., Williams, C. K. I., Klanke, S., & Vijayakumar, S. (2009). Multi-task Gaussian process learning of robot inverse dynamics. In Advances in neural information processing systems (NIPS). Rostrevor: Curran Associates, Inc.
24. E. De Vito L. Rosasco A. Caponnetto U. De Giovannini F. Odone 2005 Learning from examples as an inverse problem Journal of Machine Learning Research 6 883 904
25. De Vito, E., Pereverzev, S., & Rosasco, L. (2008). Adaptive kernel methods via the balancing principle (Technical Report). CBCL paper 275/CSAIL-TR 2008-062, MIT.
26. Devroye, L., Györfi, L., & Lugosi, G. (1996). Applications of mathematics: Vol. 31. A probabilistic theory of pattern recognition. New York: Springer.
27. T. G. Dietterich G. Bakiri 1995 Solving multiclass learning problems via error-correcting output codes Journal of Artificial Intelligence Research 2 263 286
28. B. Efron T. Hastie I. Johnstone R. Tibshirani 2004 Least angle regression Annals of Statistics 32 2 407 451 2060166 10.1214/009053604000000067
29. Engl, H. W., Hanke, M., & Neubauer, A. (1996). Mathematics and its applications: Vol. 375. Regularization of inverse problems. Dordrecht: Kluwer Academic.
30. T. Evgeniou C. A. Micchelli M. Pontil 2005 Learning multiple tasks with kernel methods Journal of Machine Learning Research 6 615 637 2249833 ISSN 1533-7928
31. Fuselier, E. J. Jr. (2006). Refined error estimates for matrix-valued radial basis functions. PhD thesis, Texas A&M University.
32. Griffin, G., Holub, A., & Perona, P. (2007). Caltech-256 object category dataset (Technical Report 7694). California Institute of Technology.
33. Gunter, L., & Zhu, J. (2006). Computing the solution path for the regularized support vector regression. In Advances in neural information processing systems (NIPS) (pp. 483-490). Cambridge: MIT Press.
34. Hastie, T., Tibshirani, R., & Friedman, J. (2001). The elements of statistical learning. New York: Springer.
35. T. Hastie R. Saharon R. Tibshirani J. Zhu 2004 The entire regularization path for the support vector machine Journal of Machine Learning Research 5 1391 1415
36. Haufe, S., Nikulin, V. V., Ziehe, A., Müller, K. R., & Nolte, G. (2009). Estimating vector fields using sparse basis field expansions. In Advances in neural information processing systems (NIPS) (pp. 617-624).
37. Hein, M., & Bousquet, O. (2004) Kernels, associated structures and generalizations (Technical Report 127). Max Planck Institute for Biological Cybernetics, July 2004.
38. A. J. Izenman 1975 Reduced-rank regression for the multivariate linear model Journal of Multivariate Analysis 5 248 264 373179 10.1016/0047-259X(75) 90042-1
39. Jacob, L., Bach, F., & Vert, J. P. (2008). Clustered multi-task learning: A convex formulation. In Advances in neural information processing systems (NIPS). Rostrevor: Curran Associates, Inc.
40. Y. Lee Y. Lin G. Wahba 2004 Multicategory support vector machines: Theory and application to the classification of microarray data and satellite radiance data Journal of the American Statistical Association 99 465 67 82 2054287 10.1198/016214504000000098
41. L. Lo Gerfo L. Rosasco F. Odone E. De Vito A. Verri 2008 Spectral algorithms for supervised learning Neural Computation 20 7 1873 1897 2417109 10.1162/neco.2008.05-07-517
42. S. Lowitzsch 2005 A density theorem for matrix-valued radial basis functions Numerical Algorithms 39 1 253 256 2137755 10.1007/s11075-004-3641-x
43. Macêdo, I., & Castro, R. (2008). Learning divergence-free and curl-free vector fields with matrix-valued kernels (Technical report). Instituto Nacional de Matematica Pura e Aplicada.
44. P. Mathé S. V. Pereverzev 2002 Moduli of continuity for operator valued functions Numerical Functional Analysis and Optimization 23 5-6 623 631 1923828 10.1081/NFA-120014755
45. McAllester, D. (2007). Generalization bounds and consistency for structured learning. In G. Bakir, T. Hofmann, B. Schölkopf, A. Smola, B. Taskar, & S. V. N. Vishwanathan (Eds.), Predicting structured data. Cambridge: MIT Press.
46. Micchelli, C. A., & Pontil, M. (2004). Kernels for multi-task learning. In Advances in neural information processing systems (NIPS). New York: MIT Press.
47. C. A. Micchelli M. Pontil 2005 On learning vector-valued functions Neural Computation 17 177 204 2175914 10.1162/0899766052530802
48. Mosci, S., Santoro, M., Verri, A., Villa, S., & Rosasco, L. (2008). Simple algorithms to solve sparsity based regularization via Fenchel duality. In OPT 2008 optimization for machine learning, NIPS 2008 workshop.
49. F. A. Mussa-Ivaldi 1992 From basis functions to basis fields: Vector field approximation from sparse data Biological Cybernetics 67 6 479 489 10.1007/BF00198755
50. F. J. Narcowich J. D. Ward 1994 Generalized Hermite interpolation via matrix-valued conditionally positive definite functions Mathematics of Computation 63 208 661 687 1254147 10.1090/S0025-5718-1994-1254147-6
51. Obozinski, G., Taskar, B., & Jordan, M. I. (2007). Multi-task feature selection (Technical report). Department of Statistics, UC Berkeley.
52. T. Poggio R. Rifkin S. Mukherjee P. Niyogi 2004 General conditions for predictivity in learning theory Nature 428 419 422 10.1038/nature02341
53. R. Rifkin A. Klautau 2004 In defense of one-vs-all classification Journal of Machine Learning Research 5 101 141 2247975 ISSN 1533-7928
54. Rifkin, R., & Lippert, R. A. (2007). Notes on regularized least squares (Technical Report). CBCL Paper 268, MIT.
55. L. Schwartz 1964 Sous-espaces hilbertiens d'espaces vectoriels topologiques et noyaux associés (noyaux reproduisants) Journal d'analyse mathématique 13 1 115 256 10.1007/BF02786620
56. Sheldon, D. (2008). Graphical multi-task learning (Technical report). Cornell University.
57. Smola, A. J., Chaussee, R., & Schölkopf, B. (1998). From regularization operators to support vector kernels. In Advances in neural information processing systems (NIPS). Cambridge: MIT Press.
58. Stein, M. L. (1999). Springer series in statistics. Interpolation of spatial data. New York: Springer.
59. I. Steinwart 2002 On the influence of the kernel on the consistency of support vector machines Journal of Machine Learning Research 2 67 93 1883281 ISSN 1533-7928
60. Szedmak, S., Shawe-Taylor, J., & Isis Group (2005). Learning via linear operators: Maximum margin regression (Technical report). In Proceedings of 2001 IEEE international conference on data mining.
61. Tewari, A., & Bartlett, P. L. (2005). On the consistency of multiclass classification methods. In Proceedings of the 18th annual conference on learning theory (Vol. 3559, pp. 143-157). Berlin: Springer.
62. I. Tsochantaridis T. Joachims T. Hofmann Y. Altun 2005 Large margin methods for structured and interdependent output variables Journal of Machine Learning Research 6 2 1453 1484 2249862
63. A. van der Merwe J. V. Zidek 1980 Multivariate regression analysis and canonical variates Canadian Journal of Statistics 8 27 39 595712 10.2307/3314667
64. Vazquez, E., & Walter, E. (2003). Multi output support vector regression. In 13th IFAC symposium on system identification, SYSID, Rotterdam, August 2003 (pp. 1820-1825). New York: IFAC.
65. G. Wang D. Y. Yeung F. H. Lochovsky 2008 A new solution path algorithm in support vector regression IEEE Transactions on Neural Networks 19 10 1753 1767 10.1109/TNN.2008.2002077
66. S. Wold H. Ruhe H. Wold W. J. Dunn III 1984 The collinearity problem in linear regression. The partial least squares (pls) approach to generalizes inverses SIAM Journal of Scientific and Statistical Computations 5 735 743 10.1137/0905052
67. T. Zhang 2004 Statistical analysis of some multi-category large margin classification methods Journal of Machine Learning Research 5 1225 1251