Learning from examples as an inverse problem

Journal of Machine Learning Research

Volumn 6, Issue , 2005, Pages

  De Vito, Ernesto ^a,b   Rosasco, Lorenzo ^c   Caponnetto, Andrea ^c   De Giovannini, Umberto ^c   Odone, Francesca ^c  

^a UNIVERSITY OF MODENA AND REGGIO EMILIA   (Italy)

^b INFN SEZIONE DI GENOVA   (Italy)

^c UNIVERSITY OF GENOA   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; LEARNING ALGORITHMS; LEARNING SYSTEMS; MATHEMATICAL OPERATORS; PROBABILITY; THEOREM PROVING;

CONSISTENCY; REGULARIZATION THEORY; STATITICAL LEARNING; SUPPORT VECTOR MACHINES;

INVERSE PROBLEMS;

EID: 21844447610     PISSN: 15337928     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (44)

1. N. Alon, S. Ben-David, N. Cesa-Bianchi, and D. Haussler. Scale sensitive dimensions, uniform convergence, and learnability. Journal of the ACM, 44:615-631, 1997.
2. A. Arbib, M. The Handbook of Brain Theory and Neural Networks. The MIT Press, Cambridge, MA, 1995.
3. N. Aronszajn. Theory of reproducing kernels. Trans. Amer. Math. Soc., 68:337-404, 1950.
4. P. L. Bartlett and S. Mendelson. Rademacher and Gaussian complexities. Journal of Machine Learning Research, 3:463-482, 2002.
5. M. Bertero, C. De Mol, and E. R. Pike. Linear inverse problems with discrete data. I. General formulation and singular system analysis. Inverse Problems, 1(4):301-330, 1985.
6. M. Bertero, C. De Mol, and E. R. Pike. Linear inverse problems with discrete data. II. Stability and regularisation. Inverse Problems, 4(3):573-594, 1988.
7. O. Bousquet and A. Elisseeff. Stability and generalization. Journal of Machine Learning Research, 2:499-526, 2002.
8. D. Chen, Q. Wu, Y. Ying, and D. Zhou. Support vector machine soft margin classifiers: Error analysis. Journal of Machine Learning research, 5:1143-1175, 2004.
9. F. Cucker and S. Smale. Best choices for regularization parameters in learning theory: on the bias-variance problem. Foundations of Computational Mathematics, 2:413-428, 2002a.
10. F. Cucker and S. Smale. On the mathematical foundations of learning. Bull. Amer. Math. Soc. (N.S.), 39(1): 1-49 (electronic), 2002b.
11. E. De Vito, A. Caponnetto, and L. Rosasco. Model selection for regularized least-squares algorithm in learning theory, to be published in Foundations of Computational Mathematics, 2004.
12. L. Devroye, L. Györfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Number 31 in Applications of mathematics. Springer, New York, 1996.
13. H. W. Engl, M. Hanke, and A. Neubauer. Regularization of inverse problems, volume 375 of Mathematics and its Applications. Kluwer Academic Publishers Group, Dordrecht, 1996.
14. T. Evgeniou, M. Pontil, and T. Poggio. Regularization networks and support vector machines. Adv. Comp. Math., 13:1-50, 2000.
15. L. Fine, T. Feedforward Neural Network Methodology. Springer-Verlag, 1999.
16. C. W. Groetsch. The theory of Tikhonov regularization for Fredholm equations of the first kind, volume 105 of Research Notes in Mathematics. Pitman (Advanced Publishing Program), Boston, MA, 1984.
17. M. Györfi, L.and Kohler, A. Krzyzak, and H. Walk. A Distribution-free Theory of Non-parametric Regression. Springer Series in Statistics, New York, 1996, 1996.
18. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer, New York, 2001.
19. J. Kaipio and E. Somersalo. Statistical and Computational Inverse Problems. Springer, 2005.
20. V. Kecman. Learning and Soft Computing. The MIT Press, Cambridge, MA, 2001.
21. V. Kurkova. Learning from data as an inverse problem. In J. Antoch, editor, COMPSTAT2004, pages 1377-1384. Springer-Verlag, 2004.
22. S. Lang. Real and Functional Analysis. Springer, New York, 1993.
23. S. Mukherjee, P. Niyogi, T. Poggio, and R. Rifkin. Statistical learning: Stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization. Technical Report CBCL Paper 223, Massachusetts Institute of Technology, January revision 2004.
24. S. Mukherjee, R. Rifkin, and T. Poggio. Regression and classification with regularization. Lectures Notes in Statistics: Nonlinear Estimation and Classification, Proceedings from MSRI Workshop, 171:107-124, 2002.
25. P. Niyogi and F. Girosi. Generalization bounds for function approximation from scattered noisy data. Adv. Comput. Math., 10:51-80, 1999.
26. C.S. Ong and S. Canu. Regularization by early stopping. Technical report, Computer Sciences Laboratory, RSISE, ANU, 2004.
27. I. F. Pinelis and A. I. Sakhanenko. Remarks on inequalities for probabilities of large deviations. Theory Probab. Appl, 30(1): 143-148, 1985. ISSN 0040-36IX.
28. T. Poggio and F. Girosi. A theory of networks for approximation and learning. In C. Lau, editor, Foundation of Neural Networks, pages 91-106. IEEE Press, Piscataway, N.J., 1992.
29. C. Rudin. A different type of convergence for statistical learning algorithms. Technical report, Program in Applied and Computational Mathematics Princeton University, 2004.
30. B. Schölkopf and A.J. Smola. Learning with Kernels. MIT Press, Cambridge, MA, 2002. URL http://www.learning-with-kernels.org.
31. L. Schwartz. Sous-espaces hilbertiens d'espaces vectoriels topologiques et noyaux associés (noyaux reproduisants). J. Analyse Math., 13:115-256, 1964.
32. C. Scovel and I. Steinwart. Fast rates support vector machines, submitted to Annals of Statistics, 2003.
33. S. Smale and Yao Y Online learning algorithms. Technical report, Toyota Technological Institute, Chicago, 2004.
34. S. Smale and D. Zhou. Estimating the approximation error in learning theory. Analysis and Applications, 1(1): 1-25, 2003.
35. S. Smale and D. Zhou. Shannon sampling and function reconstruction from point values. Bull. Amer. Math. Soc. (N.S.), 41(3):279-305 (electronic), 2004a.
36. S. Smale and D. Zhou. Shannon sampling II : Connections to learning theory, preprint, 2004b.
37. I. Steinwart. Sparseness of support vector machines. Journal of Machine Learning Research, 4: 1071-1105, 2003.
38. I. Steinwart. Consistency of support vector machines and other regularized kernel machines, accepted on IEEE Transaction on Information Theory, 2004.
39. A. N. Tikhonov, A. V. Goncharsky, V. V. Stepanov, and A. G. Yagola. Numerical methods for the solution of ill-posed problems, volume 328 of Mathematics and its Applications. Kluwer Academic Publishers Group, Dordrecht, 1995. Translated from the 1990 Russian original by R. A. M. Hoksbergen and revised by the authors.
40. A.N. Tikhonov and V.Y. Arsenin. Solutions of III Posed Problems. W. H. Winston, Washington, D.C., 1977.
41. V. N. Vapnik. Statistical learning theory. Adaptive and Learning Systems for Signal Processing, Communications, and Control. John Wiley & Sons Inc., New York, 1998. A Wiley-Interscience Publication.
42. G. Wahba. Spline models for observational data, volume 59 of CBMS-NSF Regional Conference Series in Applied Mathematics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1990.
43. V. Yurinsky. Sums and Gaussian vectors, volume 1617 of Lecture Notes in Mathematics. Springer-Verlag, Berlin, 1995.
44. T. Zhang. Leave-one-out bounds for kernel methods. Neural Computation, 13:1397-1437, 2003.