Regularized kernel forms of minimum squared error methods

Zidonghua Xuebao/Acta Automatica Sinica

Volumn 30, Issue 1, 2004, Pages 26-36

  Xu, Jian Hua ^a   Zhang, Xue Gong ^a   Li, Yan Da ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

Kernel form; Nonlinear; Regularization; Squared error; Support vector machines

Indexed keywords

ALGORITHMS; ERROR ANALYSIS; FUNCTIONS; LINEAR EQUATIONS; OPTIMIZATION; PROBABILITY; REGRESSION ANALYSIS;

KERNEL FORMS; MINIMUM SQUARED ERROR; PROBABILISTIC INTERPRETATION; REGULARIZATION TERMS; SQUARED ERROR SUMMATION; SUPPORTED VECTOR MACHINES;

PATTERN RECOGNITION;

EID: 2342599779     PISSN: 02544156     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (33)

1. Duda R O, Hart P E. Pattern Classification and Scene Analysis. New York: John Wiley and Sons, 1973
2. Tou J T, Gonzadez R C. Pattern Recognition Principles. Reading: Addison-Wesley, 1974
3. Bian Zhao-Qi, Zhang Xue-Gong et al. Pattern Recognition (Second Edition). Beijing: Tsinghua University Press, 2000 (in Chinese)
4. Koford J S, Groner G F. The use of an adaptive threshold element to design a linear optimal pattern classifier. IEEE Transactions on Information Theory, 1966, 12(1): 42-50
5. Patterson J D, Womack B F. An adaptive pattern classification system. IEEE Transactions on System Science and Cybernetics, 1966, 2(1): 62-67
6. Wang Hui-Wen. Partial Least Squares Regression Method and Its Applications. Beijing: National Defence Industrial Press, 1999 (in Chinese)
7. Cortes C, Vapnik V N. Support vector networks. Machine Learning, 1995, 20(3): 273-297
8. Vapnik V N. Statistical Learning Theory. New York: Wiley, 1998
9. Vapnik V N. The Nature of Statistical Learning Theory (2nd ed). New York: Springer-Verlag, 1999
10. Zhang Xue-Gong. Introduction to statistical learning theory and support vector machines. Acta Automatica Sinica, 2000, 26(1): 32-44 (in Chinese)
11. Scholkopf B, Smola A, Muller K-R. Kernel principal component analysis. In Proceedings ICANN'97, Springer Lecture Notes in Computer Science, 1997. 583-589
12. Scholkopf B, Smola A, Muller K-R. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 1998, 10(5): 1299-1319
13. Mika S, Ratsch G, Weston J, Scholkopf B, Muller K-R. Fisher discriminant analysis with kernels. Neural Networks for Signal Processing IX, New York: IEEE Press, 1999. 41-48
14. Muller K-R, Mika S, Ratsch G, Scholkopf B. An introduction to kernel-based learning algorithm. IEEE Transactions on Neural Networks, 2001, 12(2): 181-201
15. Suykens J A K, Vandewalle J. Least squares support vector machines. Neural Processing Letters, 1999, 9(3): 293-300
16. Saunders C, Gammerman A, Vovk V. Ridge regression learning algorithm in dual variables. In: Shavlik J editor. Machine Learning Proceedings of the 15th International Conference, Morgan Kaufmann, 1998
17. Ruiz A, Lopez-de-Teruel P E. Nonlinear kernel-based statistical pattern analysis. IEEE Transactions on Neural Networks, 2001, 12(1): 16-31
18. Xu J, Zhang X, Li Y. Kernel MSE algorithm: A unified framework for KFD, LS-SVM and KRR. In: International Joint Conference on Neural Networks 2001. Washington DC, USA: IEEE Press, 2001. 1486-1491
19. Tikhonov A N, Arsenin V Y. Solution of ill-posed problem. Washington DC: Winston and Sons, 1977
20. Tikhonov A N, Goncharsky A V. Ill-posed problems in the natural sciences. Translated from Russian by Bloch M, Moscow: MIR Publishers, 1987
21. Smola A J, Scholkopf B. On a kernel-based method for pattern recognition, regression, approximation, and operator inversion. Algorithmica, 1998, 22(1-2): 211-231
22. Smola A J, Scholkopf B, Muller K-R. The connection between regularization operators and support vector kernels. Neural Networks, 1998, 11(4): 637-649
23. Saito K, Nakano R. Second order learning algorithm with squared penalty term. Neural Computation, 2000, 12(3): 709-729
24. Guyon I, Stork D G. Linear discriminant and support vector classifiers. In: Smola A J, Bartlett P, Scholkopf B, Schuurmans C, editors (2000). Advances in Large Margin Classifiers, MIT Press, 2000
25. Poggio T, Girosi F. Regularization algorithm for learning that are equivalent to multi-layer networks. Science, 1990, 247(23): 978-982
26. Bishop C M. Curvature driven smoothing: a learning algorithm for feedforward networks. IEEE Transactions on Neural Networks, 1993, 4(5): 882-884
27. Reed R. Pruning algorithms-A survey. IEEE transactions on Neural Networks, 1993, 4(5): 740-747
28. Ishikawa M. Structural learning with forgetting. Neural Networks, 1992, 9(3): 509-521
29. Mackay D J C. Bayesian interpolation. Neural Computation, 1992, 4(3): 415-447
30. Mackay D J C. A practical Bayesian framework for back-propagation networks. Neural Computation, 1992, 4(3): 448-472
31. Williams P M. Bayesian regularization and pruning using a Laplace prior. Neural Computation, 1995, 7(1): 117-143
32. Wahba G. Support vector machines, reproducing kernel Hilbert spaces and the randomized GACV. In: Scholkopf B, Burges C J C, Smola A J, editors, Advances in Kernel Methods-Support Vector Learning, Cambridge, MA: MIT Press, 1999. 69-88
33. Cristianini N, Taylor J S. An introduction to support vector machines and other kernel-based learning methods. Cambridge UK: Cambridge University Press, 2000