On stochastic optimization and statistical learning in reproducing Kernel Hilbert spaces by support vector machines (SVM)

Informatica

Volumn 20, Issue 2, 2009, Pages 273-292

  Norkin, Vladimir ^a   Keyzer, Michiel ^b  

^a V M GLUSHKOV INSTITUTE OF CYBERNETICS   (Ukraine)

^b VU UNIVERSITY AMSTERDAM   (Netherlands)

Author keywords

Convergence; Decision rule; Empirical risk minimization; Regularization; Reproducing Kernel Hilbert Spaces (RKHS); Stochastic optimization; Support vector machine (SVM)

Indexed keywords

EID: 67651229638     PISSN: 08684952     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (34)

1. Aronszajn, N. (1950). Theory of reproducing kernels. Transactions of American Mathematical Society, 68, 337-404.
2. Bousquet, O., and A. Elisseeff (2002). Stability and generalization. Journal of Machine Learning Research, 2, 499-526.
3. Brumen B., M.B. Juric, T. Welzer, I. Rozman, H. Jaakkola and A. Papadopoulos (2007). Assessment of classification models with small amounts of data. Informatica, 18(3), 343-362.
4. Chancelier, J.-Ph., and SOWG (2006). Epi-convergence of stochastic optimization problems involving both random variables and measurability constraints approximations. Rapport de recherche du CERMICS 2006-323, 1-14 (http://cermics.enpc.fr/reports).
5. Cucker, F., and S. Smale (2001). On the mathematial foundations of learning. Bull. Amer. Math. Soc., 89(1), 1-49.
6. Cucker, F., and S. Smale (2002). Best choices for regularization parameters in learning theory: on the biasvariance problem. Found. Comput. Math., 2(4), 413-428.
7. De Vito, E., A. Caponnetto and L. Rosasco (2005). Model selection for regularized least-squares algorithm in learning theory. Found. Comput. Math., 5(1), 59-85.
8. Devroye, L., L. Gyorfi and G. Lugosi (1996). A Probabilistic Theory of Pattern Recognition. Springer-Verlag, New York.
9. Ekeland, I., and R. Temam (1976). Convex Analysis and Variational Problems. North Holland & American Elsevier, Amsterdam, Oxford, New York.
10. Ermoliev, Y.M. (1976). Methods of Stochastic Programming. Nauka, Moscow (in Russian).
11. Ermoliev, Y.M., and G. Leonardi (1982). Some proposals for stochastic facility location models. Mathematical Modelling, 3, 407-420.
12. Ermoliev, Y.M., and A.I. Yastremski (1979). Stochastic Models and Methods in Economic Planning. Nauka, Moscow (in Russian).
13. Gyorfi, L., M. Kohler, A. Krzyzak and H. Walk (2002). A Distribution Free Theory of Nonparametric Regression. Springer-Verlag, New York, Berlin, Heidelberg.
14. Keyzer, M.A. (2005). Rule-based and support vector (SV-) regression/classification algorithms for joint processing of census, map, survey and district data. Working Paper WP-05-01. Centre for World Food Studies, Amsterdam.
15. Kimeldorf, G.S., and G. Wahba (1970). A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Ann. Math. Stat., 45, 495-502.
16. Koenker, R., and G.W. Bassett (1978). Regression quantiles. Econometrica, 46, 33-50.
17. Kolmogorov, A.N., and S.V. Fomin (1981). Elements of the Theory of Functions and Functional Analysis. 5th ed. Nauka, Moscow (in Russian).
18. McDiarmid, C. (1989). On the method of bounded differences. Surveys in Combinatorics. Cambridge University Press.
19. Mukherjee, S., P. Niyogi, T. Poggio and R. Rifkin (2006). Learning theory: stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization. Adv. Comput. Math., 25(1-3), 161-193.
20. Norkin, V.I., and M.A. Keyzer (2008). On convergence of kernel learning estimators. In L. Sakalauskas, O.W. Weber and E.K. Zavadskas (Eds.), Proceedings of 20th EURO Mini Conference "Continuous Optimization and Knowledge-Based Technologies" (EUROPT-2008), Institute of Mathematics and Informatics, Vilnius. pp. 306-310.
21. Poggio, T., and S. Smale (2003). The mathematics of learning: dealing with data. Notices Amer. Math. Soc., 50(5), 537-544.
22. Raik, E. (1972). On stochastic programming problems with decision rules. In Proceedings of the Estonian Academy of Sciences. Physics. Mathematics, Vol. 21. Izvestia AN ESSR. Physika. Math., pp. 258-263.
23. Rockafellar, R.T., and R.J.-B. Wets (1998). Variational Analysis. Springer-Verlag, Berlin.
24. Ruszczynski, A., and A. Shapiro (Eds.) (2003). Stochastic programming. Handbooks in OR & MS, 10.
25. Scholkopf, B., and A.J. Smola (2002). Learning with Kernels. Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, MA.
26. Sergienko, I.V., A.M. Gupal and A.A. Vagis (2008). Bayesian approach, theory of empirical risk minimization. Comparative analysis. Cybernetics and Systems Analysis, 44(6), 822-831 (translated from Kibernetika i Sistemnyi Analiz, 2008, 6, 39-49).
27. Smale, S., and Y. Yao (2006). Online Learning Algorithms. Foundations of Computational Mathematics, 6(2), 145-170.
28. Smale, S., and D.X. Zhou (2005). Shannon sampling II: Connections to learning theory. Applied Computational Harmonic Analysis, 19(3), 285-302.
29. Takeuchi, I., Q.V. Le, T. Sears and A.J. Smola (2006). Nonparametric quantile estimation. Journal of Machine Learning Research, 7, 1231-1264.
30. Tihonov, A.N., and V.Y. Arsenin (1977). Solution of Ill-Posed Problems. WH Winston, Washington, DC.
31. Vapnik, V.N. (1998). Statistical Learning Theory. Wiley, New York.
32. Yastremskii, A.I. (1980). Optimality conditions in stochastic programming. Cybernetics and Systems Analysis, 16(1), 154-158.
33. Yudin, D.B. (1979). Problems and Methods of Stochastic Programming. Sovetskoe radio, Moscow (in Russian).
34. Yudin, D.B. (1974). Mathematical Methods of Control under Incomplete Information (Problems and Methods of Stochastic Programming). Sovetskoe radio, Moscow (in Russian).