Another look at statistical learning theory and regularization

Neural Networks

Volumn 22, Issue 7, 2009, Pages 958-969

  Cherkassky, Vladimir ^a   Ma, Yunqian ^b  

^a University of Minnesota   (United States)

^b HONEYWELL TECHNOLOGY CENTER   (United States)

Author keywords

Function approximation; Model identification; Penalization; Predictive learning; Regularization; Ridge regression; Statistical model estimation; Structural risk minimization; SVM regression; VC theory

Indexed keywords

FUNCTION APPROXIMATION; MODEL IDENTIFICATION; PENALIZATION; PREDICTIVE LEARNING; REGULARIZATION; RIDGE REGRESSION; STATISTICAL MODEL ESTIMATION; STRUCTURAL RISK MINIMIZATION; SVM REGRESSION; VC-THEORY;

ESTIMATION; EULER EQUATIONS; FUNCTION EVALUATION; MINING; MULTILAYER NEURAL NETWORKS; OPTIMIZATION; PROBABILITY DENSITY FUNCTION; REGRESSION ANALYSIS; RISK PERCEPTION; SUPPORT VECTOR MACHINES;

EDUCATION;

ARTICLE; CONTROLLED STUDY; LEARNING ALGORITHM; LEARNING THEORY; MATHEMATICAL COMPUTING; PREDICTION; PRIORITY JOURNAL; PROBLEM SOLVING; STATISTICAL MODEL; SUPPORT VECTOR MACHINE;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; HUMANS; LEARNING; MODELS, STATISTICAL; PREDICTIVE VALUE OF TESTS; SIGNAL PROCESSING, COMPUTER-ASSISTED;

EID: 69449099786     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2009.04.005     Document Type: Article

References (37)

1. Ahn, J., & Marron, J. S. (2005). The maximal data piling direction for discrimination. Biometrica (submitted for publication)
2. Barron A.R. Universal approximation bounds for superpositions of sigmoidal function. IEEE Transactions on Information Theory 39 (1993) 930-945
3. Barron A., Birge L., and P. M. Risk bounds for model selection via penalization. Probability theory and related fields (1999), Springer-Verlag 301-413
4. Breiman L., Friedman J.H., Olshen R.A., and Stone C.J. Classification and regression trees (1984), Wadsworth, Belmont, CA
5. Chambolle A., DeVore R.A., Lee N., and Lucier B. Nonlinear wavelet image processing: variational problems, compression and noise removal through wavelet shrinkage. IEEE Transactions on Image Processing 7 (1998) 319-335
6. Cherkassky V., and Ma Y. Practical Selection of SVM parameters and noise estimation for SVM regression. Neural Networks 17 (2004) 113-126
7. Cherkassky V., and Mulier F. Learning from data: Concepts, theory, and methods. 2nd ed. (2007), John Wiley & Sons
8. Donoho D., and Johnstone I. Ideal spatial adaptation by wavelet shrinkage. Biometrika 81 3 (1994) 425-455
9. DeVore R.A. Degree of nonlinear approximation. In: Chui C.K., Schumaker L.L., and Wards D.J. (Eds). Approximation Theory Vol. VI (1991), Academic Press, New York 175-201
10. DeVore R.A., and Lucier B.J. Fast wavelet techniques for near-optimal image processing. Proc. IEEE military communications conf. (1992), IEEE Press 1129-1135
11. Duda R., Hart P., and Stork D. Pattern classification. 2nd ed. (2000), Wiley, New York
12. Evgeniou T., Pontil M., and Poggio T. Regularization networks and support vector machines. Advances in Computational Mathematics 13 (2000) 1-50
13. Friedman J.H. An overview of computational learning and function approximation. In: Cherkassky, et al. (Ed). From statistics to neural networks. Theory and pattern recognition applications. NATO ASI series F (1994), Springer-Verlag 1-61
14. Girosi F. Regularization theory, radial basis functions and networks. In: Cherkassky V., et al. (Ed). From statistics to neural networks: Theory and pattern recognition applications. NATO ASI series F (1994), Springer-Verlag 166-187
15. Hand D., Mannila H., and Smyth P. Principles of data mining (2001), MIT Press
16. Hastie T., Tibshirani R., and Friedman J. The elements of statistical learning: Data mining, inference and prediction (2001), Springer
17. Haykin S. Neural networks: A comprehensive foundation. 2nd ed. (1999), MacMillan, New York
18. Hoerl A.E., and Kennard R. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12 (1970) 55-67
19. Jones L.K. A Simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Annals of Statistics 20 (1992) 608-613
20. Kirsch A. An introduction to the mathematical theory of inverse problems (1996), Springer, NY
21. Micchelli C. Interpolation of scattered data: Distance matrices and conditionally positive definite functions. Constructive Approximation 2 (1986) 11-22
22. Mika, S. (2002). Kernel fisher discriminants, Ph.D. Thesis. Technical University of Berlin
23. Poggio T., and Girosi F. Regularization algorithms for learning that are equivalent to multilayer networks. Science 247 (1990) 978-982
24. Poggio T., and Smale S. The mathematics of learning: Dealing with data. Notices of the AMS 50 5 (2003) 537-544
25. Powell M. Radial basis functions for multivariable interpolation: A review. IMA conf. on algorithms for the approximation of functions and data (1987), RMCS, Shrivenham, England 143-167
26. Ripley B.D. Pattern recognition and neural networks (1996), Cambridge Univ. Press
27. Schölkopf B., and Smola A. Learning with kernels (2002), MIT Press
28. Smola A., Schölkopf B., and Mueller K.-R. The connection between regularization operators and support vector kernels. Neural Networks 11 (1998) 637-649
29. Suykens J., and Vanderwalle J. Least squares support vector machine classifiers. Neural Processing Letters 9 3 (1998) 293-300
30. Suykens J., Van Gestel T., De Brabanter J., De Moor B., and Vandewalle J. Least squares support vector machines (2002), World Scientific Pub. Co, Singapore
31. Tikhonov N. On solving ill-posed problem and method of regularization. Doklady Akademii Nauk USSR 153 (1963) 501-504
32. Tikhonov N., and Arsenin V.Y. Solution of ill-posed problems (1977), W. H. Winston, Washington, DC
33. Vapnik V., and Chervonenkis A.Ja. Theory of pattern recognition (1974), Nauka, Moscow (in Russian)
34. Vapnik V. Estimation of dependencies based on empirical data (1982), Springer, NY
35. Vapnik V. Statistical learning theory (1998), Wiley, New York
36. Vapnik V. The nature of statistical learning theory. 2nd ed. (2000), Springer
37. Ye, J., & Xiong, T. (2007). SVM versus least squares SVM. In Proc. international conference on artificial intelligence and statistics (pp. 640-647)