Nonlinear kernel forms of classical linear algorithms

Kongzhi yu Juece/Control and Decision

Volumn 21, Issue 1, 2006, Pages

  Xu, Jian Hua ^a   Zhang, Xue Gong ^b  

^a NANJING NORMAL UNIVERSITY   (China)

^b TSINGHUA UNIVERSITY   (China)

Author keywords

Kernel forms; Kernel function; Machine learning; Support vector machine

Indexed keywords

ALGORITHMS; CLASSIFICATION (OF INFORMATION); CONVERGENCE OF NUMERICAL METHODS; FUNCTIONS; ITERATIVE METHODS; OPTIMIZATION; STABILITY; VECTORS;

GENERALIZATION ABILITY; KERNEL FORMS; KERNEL FUNCTIONS; MERCER CONDITIONS; REGULARIZATION TECHNIQUE; SAMPLE VECTORS; SUPPORT VECTOR MACHINES;

LEARNING SYSTEMS;

EID: 33644943386     PISSN: 10010920     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (49)

1. Vapnik V N. The Nature of Statistical Learning Theory [M]. Second Edition. New York: Springer-Verlag, 1999.
2. Vapnik V N. Statistical Learning Theory [M]. New York: John Wiley and Sons, 1998.
3. Cristianini N, Taylor J S. An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods [M]. Cambridge UK: Cambridge University Press, 2000.
4. Scholkopf B, Smola A J. Learning with Kernels-Support Vector Machines, Regularization, Optimization and Beyond[M]. Cambridge MA: MIT Press, 2001.
5. Aizerman M A, Braverman E M, Rozonoer L I. Theoretical Foundations of the Potential Function Method in Pattern Recognition Learning [J]. Automation and Remote Control, 1964, 25(6): 821-837.
6. Tou J T, Gonzalez R C. Pattern Recognition Principles [M]. New York: Addison-Wesly, 1974.
7. Berg C, Christensen J P R, Ressel P. Harmonic Analysis on Semigroup [M]. New York: Spring-Verlag, 1984.
8. Genton M G. Classes of Kernels for Machine Learning: A Statistics Perspective [J]. J of Machine Learning Research, 2001, 2: 299-312.
9. Zein A, Raetsch G, Mika S, et al. Engineering SVM Kernels That Recognize Translation Initiation Sites [J]. Bioinformatics, 2000, 16(9): 799-807.
10. Jaakkola T, Diekhaus M, Haussler D. A Discriminative Framework for Detecting Remote Protein Homologies[J]. J of Computational Biology, 2000, 7(1-2): 95-114.
11. Tsuda K, Akaho S, Kawanabe M, et al. Asymptotic Properties of the Fisher Kernel [J]. Neural Computation, 2004, 16(1): 115-137.
12. Tsuda K, Kawanabe M, Ratsch G, et al. A New Discriminative Kernel From Probabilistic Models [J]. Neural Computation, 2002, 14(10): 2397-2414.
13. Mjolsness E, DeCoste D. Machine Learning for Science: State of the Art and Future Prospects [J]. Science, 2001, 293(5537): 2051-2055.
14. Tikhonov A N, Arsenin V Y. Solution of Ill Posed Problem [M]. Washington DC: Winston V H and Sons, 1977.
15. Bian Z Q, Zhang X G, et al. Pattern Recognition [M]. Second Edition. Beijing: Tsinghua University Press, 2000.
16. Duda R O, Hart P E. Pattern Classification and Scene Analysis[M]. New York, John Wiley and Sons, 1973.
17. Saito K, Nakano R. Second Order Learning Algorithm with Squared Penalty Term[J]. Neural Computation, 2000, 12(3): 709-729.
18. Scholkopf B, Smola A, Muller K-R. Kernel Principal Component Analysis [A]. Lecture Notes in Computer Science 1327-International Conference on Artificial Neural Networks [C]. Berlin: Springer, 1997: 583-589
19. Scholkopt B, Smora A, Muller K-R. Nonlinear Component Analysis as a Kernel Eigenvalue Problem [J]. Neural Computation, 1998, 10(5): 1299-1319.
20. Mika S, Scholkopf B, Smola A, et al. Kernel PCA and De-noising in Feature Spaces [A]. Advances in Neural Information Processing Systems 11 [C]. Cambridge MA: MIT Press, 1999: 536-542.
21. Mika S, Ratsch G, Weston J, et al. Fisher Discriminant Analysis With Kernels [A]. Neural Networks for Signal Processing 9 [C]. New York: IEEE Press, 1999: 41-48.
22. Mika S, Ratsch G, Muller K-R. A Mathematical Programming Approach to the Kernel Fisher Algorithm [A]. Advances in Neural Information Processing System 13 [C]. Cambridge MA: MIT Press, 2002: 591-597.
23. Mika S, Smola A J, Scholkopf B. An Improving Training Algorithm for Kernel Fisher Discriminants [A]. Proc of Artificial Intelligence and Statistics[C]. San Francisco CA: Morgan Kaufmann, 2001: 98-104.
24. Baudat G, Anouar F. Generalized Discriminant Analysis Using a Kernel Approach [J]. Neural Computation, 2000, 12(10): 2385-2404.
25. Ma J, Sancho-Gomez J L, Ahalt S C. Nonlinear Multiclass Discriminant Analysis [J]. IEEE Signal Processing Letters, 2003, 10(7): 196-200.
26. Wang H W. Partial Least Squares Regress Method and Its Applications [M]. Beijing: Defense Industry Press, 1999.
27. Saunders C, Gammerman A, Vovk V. Ridge Regression Learning Algorithm in Dual Variables[A]. Proc of the Fifteenth Int Conf on Machine Learning [C]. San Francisco CA: Morgan Kaufmann, 1998: 515-521.
28. Rosipal R, Trejo L J. Kernel Partial Least Squares Regression in RKHS[J]. J of Machine Learning Research, 2001, 2: 97-123.
29. Ruiz A, Lopez-de-Teruel P E. Nonlinear Kernel-based Statistical Pattern Analysis [J]. IEEE Transactions on Neural Networks, 2001, 12(1): 16-31.
30. Xu J, Zhang X, Li Y. Kernel MSE Algorithm: A Unified Framework for KFD, LS-SVM and KRR[A]. Proc of 2001 Int Joint Conf on Neural Networks[C]. New York: IEEE Press, 2001: 1486-1491.
31. Xu J H, Zhang X G, Li Y D. Regularized Kernel Forms of Minimum Squared Error Methods[J]. Acta Automatica Sinica, 2004, 30(1): 27-36.
32. Suykens J A K, Vandewalle J. Least Squares Support Vector Machines [J]. Neural Processing Letters, 1999, 9(3): 293-300.
33. Anlauf J K, Biehl M. The Adatron: An Adaptive Perception [J]. Europhysics Letter, 1989, 10(7): 687-692.
34. Friess T, Cristianini N, Campbell C. The Kernel Adatron: A Fast and Simple Learning Procedure for Support Vector Machines [A]. Proc of the Fifteenth Int Conf of Machine Learning[C]. San Francisco CA: Morgan Kaufmann, 1998: 188-196.
35. Cristinini N, Campbell C, Shawe-Taylor J. Dynamically Adapting Kernels in Support Vector Machines [A]. Advances in Neural Information Processing Systems 11 [C]. Cambridge MA: MIT Press, 1999: 204-210.
36. Guyon I, Stork D G. Linear Discriminant and Support Vector Classifiers [A]. Advances in Large Margin Classifiers [C]. Cambridge MA: MIT Press, 2000: 147-170.
37. Xu J H, Zhang X G, Li Y D. A Nonlinear Perception Algorithm Based on Kernel Functions[J]. Chinese J of Computers, 2002, 25(7): 689-695
38. Xu J H, Zhang X G, Li Y D. Nonlinear Pocket Algorithm with Kernels [J]. Acta Electronica Sinica, 2003, 31(4): 612-615.
39. Gallant S I. Optimal Linear Discriminant[A]. Proc of the Eighth Int Conf on Pattern Recognition[C]. New York: IEEE Press, 1986: 849-853.
40. Muselli M. On Convergence Properties of Pocket Algorithm [J]. IEEE Trans on Neural Networks, 1997, 8(3): 623-629.
41. Xu J, Zhang X, Li Y. Large Margin Kernel Pocket Algorithm[A]. Proc of 2001 Int Joint Conf on Neural Networks[C]. New York: IEEE Press, 2001: 1480-1485.
42. Xu J, Zhang X, Li Y. Kernel Neuron and its Training Algorithm [A]. Proc of the Eighth. Int Conf on Neural Information Processing [C]. Shanghai: Fudan University Press, 2001, 2: 861-866.
43. Ishikawa M. Structural Learning with Forgetting[J]. Neural Networks, 1996, 9(3): 509-521.
44. Xu J, Zhang X, Li Y. Sparse Training Procedure for Kernel Neuron [A]. Proc of Int Conf on Neural Networks and Signal Processing [C]. New York: IEEE Press, 2003, 1: 49-53.
45. Xu J, Zhang X. A Learning Algorithm with Gaussian Regularizer for Kernel Neuron [A]. 2004 Int Symposium on Neural Networks[C]. Berlin: Springer, 2004: 252-257.
46. Yu K, Ji L, Zhang X. Kernel Nearest Neighbor Algorithm [J]. Neural Processing Letters, 2002, 15(2): 147-156.
47. Joachims T. Making Large-scale SVM Learning Practical [A]. Advances in Kernel Methods-Support Vector Learning [C]. Cambridge MA: MIT Press, 1999: 169-184.
48. Baldi P, Brunak S. Bioinformatics: The Machine Learning Approach [M]. 2nd ed. Cambridge MA: MIT Press, 2001.
49. Hsu C W, Lin C J. A Comparison of Methods for Multiclass Support Vector Machines[J]. IEEE Trans on Neural Networks, 2002, 13(2): 415-425.