Learning with generalization capability by kernel methods of bounded complexity

Journal of Complexity

Volumn 21, Issue 3, 2005, Pages 350-367

  Kůrková, Věra ^a   Sanguineti, Marcello ^b  

^a INSTITUTE OF COMPUTER SCIENCE   (Czech Republic)

^b UNIVERSITY OF GENOA   (Italy)

Author keywords

Generalization; Kernel methods; Minimization of regularized empirical errors; Model complexity; Supervised learning; Upper bounds on rates of approximate optimization

Indexed keywords

APPROXIMATION THEORY; CONVERGENCE OF NUMERICAL METHODS; LEARNING ALGORITHMS; LEARNING SYSTEMS; LINEAR EQUATIONS; MATHEMATICAL MODELS; MATRIX ALGEBRA; SET THEORY; THEOREM PROVING; VECTORS;

APPROXIMATION OPTIMIZATION; BOUNDED COMPLEXITY; GENERALIZATIONS; KERNEL METHODS; MINIMIZATION OF REGULARIZED EMPIRICAL ERRORS; MODEL COMPLEXITY; SUPERVIZED LEARNING;

COMPUTATIONAL COMPLEXITY;

EID: 18144390163     PISSN: 0885064X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jco.2004.11.002     Document Type: Article

References (44)

1. E. Aarts J. Korst Simulated Annealing and Boltzmann Machines A Stochastic Approach to Combinatorial Optimization and Neural Computing 1989 Wiley New York
2. N. Aronszajn Theory of reproducing kernels Trans. Amer. Math. J. 68 1950 337-404
3. A.R. Barron Universal approximation bounds for superpositions of a sigmoidal function IEEE Trans. Inform. Theory 39 1993 930-945
4. R. Bellman Dynamic Programming 1957 Princeton University Press Princeton, NJ
5. C. Berg J.P.R. Christensen P. Ressel Harmonic Analysis on Semigroups 1984 Springer New York
6. M. Bertero Linear inverse and ill-posed problems Adv. Electron. Electron Phys. 75 1989 1-120
7. D.P. Bertsekas Nonlinear Programming 1999 Athena Scientific Belmont, MA
8. C. Cortes V. Vapnik Support vector networks Mach. Learning 20 1995 1-25
9. N. Cristianini J. Shawe-Taylor An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods 2000 Cambridge University Press Cambridge
10. F. Cucker S. Smale On the mathematical foundations of learning Bull. Amer. Math. Soc. 39 2001 1-49
11. F. Cucker S. Smale Best choices for regularization parameters in learning theory: On the bias-variance problem Found. Comput. Math. 2 2002 413-428
12. J.A. Cuesta-Albertos M. Wschebor Some remarks on the condition number of a real random square matrix J. Complexity 19 2003 548-554
13. J. Demmel The geometry of ill-conditioning J. Complexity 3 1987 201-229
14. A.L. Dontchev, Perturbations, Approximations, and Sensitivity Analysis of Optimal Control Systems, Lecture Notes in Control and Information Sciences, vol. 52, Springer, Berlin, Heidelberg, 1983.
15. A.L. Dontchev, T. Zolezzi, Well-Posed Optimization Problems, Lecture Notes in Mathemathics, vol. 1543, Springer, Berlin, Heidelberg, 1993.
16. F. Girosi, Regularization theory, Radial Basis Functions and networks, in: V. Cherkassky, J.H. Friedman, H. Wechsler (Eds.) From Statistics to Neural Networks, Theory and Pattern Recognition Applications - Subseries F, Computer and Systems Sciences, Springer, Berlin, 1994,, pp. 166-187.
17. F. Girosi An equivalence between sparse approximation and support vector machines Neural Comput. 10 1998 1455-1480
18. F. Girosi M. Jones T. Poggio Regularization theory and neural networks architectures Neural Comput. 7 1995 219-269
19. D.E. Goldberg Genetic Algorithms in Search, Optimization, and Machine Learning 1989 Addison-Wesley Reading, MA
20. L.K. Jones A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training Ann. Statist. 20 1992 608-613
21. G.S. Kimeldorf G. Wahba A correspondence between Bayesian estimation on stochastic processes and smoothing by splines Ann. Math. Statist. 41 1970 495-502
22. V. Kůrková Dimension-independent rates of approximation by neural networks, in: K. Warwick M. Kárný (Eds.) Computer-Intensive Methods in Control and Signal Processing: The Curse of Dimensionality. Birkhäuser Boston, 1997, pp. 261-270.
23. V. Kůrková High-dimensional approximation by neural networks, in: J. Stuykens et al. (Eds.) Advances in Learning Theory Methods, Models and Applications, IOS Press, Amsterdam, 2003, pp. 69-88.
24. V. Kůrková Learning from data as an inverse problem J. Antoch (Ed.) COMPSTAT 2004 - Proceedings in Computational Statistics, Physica-Verlag Springer Heidelberg 2004, pp. 1377-1384.
25. V. Kůrková M. Sanguineti Bounds on rates of variable-basis and neural-network approximation IEEE Trans. Inform. Theory 47 2001 2659-2665
26. V. Kůrková M. Sanguineti Comparison of worst case errors in linear and neural network approximation IEEE Trans. Inform. Theory 48 2002 264-275
27. V. Kůrková, M. Sanguineti, Error estimates for approximate optimization by the extended Ritz method, SIAM J. Optim., to appear.
28. V. Kůrková P. Savický K. Hlaváčková Representations and rates of approximation of real-valued Boolean functions by neural networks Neural Networks 11 1998 651-659
29. E.S. Levitin B.T. Polyak Convergence of minimizing sequences in conditional extremum problems Sov. Math. Dokl. 7 1996 764-767 (English translation: Dokl. Akad. Nauk SSSR 168 (1966) 997-1000).
30. J.M. Ortega Numerical Analysis A Second Course 1990 SIAM Philadelphia (reprint of the 1972 edition by Academic Press, New York)
31. E. Parzen An approach to time series analysis Ann. Math. Statist. 32 1961 951-989
32. G. Pisier, Remarques sur un résultat non publié de B. Maurey, Séminaire d'Analyse Fonctionnelle, 1980-81, Exposé no. V, École Polytechnique, Centre de Mathématiques, Palaiseau, France, pp. V.1-V.12.
33. T. Poggio F. Girosi Networks for approximation and learning Proc. IEEE 78 1990 1481-1497
34. T. Poggio F. Girosi Regularization algorithms for learning that are equivalent to multilayer networks Science 247 1990 978-982
35. T. Poggio S. Smale The mathematics of learning: Dealing with data Notices Amer. Math. Soc. 50 2003 536-544
36. B. Schölkopf, R. Herbrich, A.J. Smola, R.C. Williamson. A generalized Representer Theorem, Lecture Notes in Artificial Intelligence (Proceedings of the Annual Conference on Computational Learning Theory-COLT), Springer, London, 2001, pp. 416-426.
37. B. Schölkopf A.J. Smola Learning With Kernels-Support Vector Machines Regularization, Optimization and Beyond 2002 MIT Press Cambridge, MA
38. I.J. Schönberg Metric spaces and completely monotone functions Ann. Math. 39 1938 811-841
39. A.J. Smola B. Schölkopf Sparse greedy matrix approximation for machine learning Proceedings of the 17th International Conference on Machine Learning 2000 Morgan Kaufmann San Francisco, CA 911-918
40. A.N. Tikhonov Solutions of incorrectly formulated problems and the regularization method Soviet Math. Dokl. 4 1963 1035-1038
41. A.N. Tikhonov V.Y. Arsenin Solutions of Ill-posed Problems 1977 W.H. Winston Washington, DC
42. V. Vapnik Statistical Learning Theory 1998 Wiley Chichester, GB
43. A.A. Vladimirov Yu.E. Nesterov Yu.N. Chekanov On uniformly convex functionals Vestnik Moskov. Univ. Ser. 15 Vychisl. Mat. Kibernet. 3 1978 12-23 (English translation: Moscow Univ. Comput. Math. Cybren. (1979) 10-12).
44. G. Wahba, Splines Models for Observational Data, CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 59. SIAM, Philadelphia, PA, 1990.