A support vector machine formulation to PCA analysis and its kernel version

IEEE Transactions on Neural Networks

Volumn 14, Issue 2, 2003, Pages 447-450

  Suykens, J A K ^a   Van Gestel, T ^a   Vandewalle, J ^a   De Moor, B ^a  

^a Katholieke Universiteit Leuven   (Belgium)

Author keywords

Kernel methods; Kernel principal component analysis (PCA); Least squares support vector machine (LS SVM); PCA analysis; SVMs

Indexed keywords

CONSTRAINT THEORY; EIGENVALUES AND EIGENFUNCTIONS; ERROR ANALYSIS; LAGRANGE MULTIPLIERS; LEAST SQUARES APPROXIMATIONS; MATHEMATICAL MODELS; NUMERICAL METHODS; OPTIMIZATION; PRINCIPAL COMPONENT ANALYSIS; PROBLEM SOLVING; REGRESSION ANALYSIS; THEOREM PROVING;

FISHER DISCRIMINANT ANALYSIS; KERNEL METHODS; MERCER THEOREM; SUPPORT VECTOR MACHINE;

NEURAL NETWORKS;

EID: 0037360871     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2003.809414     Document Type: Letter

References (26)

1. G. Baudat and F. Anouar, "Generalized discriminant analysis using a kernel approach," Neural Comput., vol. 12, pp. 2385-2404, 2000.
2. C. M. Bishop, Neural Networks for Pattern Recognition. Oxford, U.K.: Oxford Univ. Press, 1995.
3. K. I. Diamantaras and S. Y. Kung, Principal Component Neural Networks. New York: Wiley, 1996.
4. T. Evgeniou, M. Pontil, and T. Poggio, "Regularization networks and support vector machines," Adv. Comput. Math., vol. 13, no. 1, pp. 1-50, 2000.
5. K. Fukunaga, Introduction to Statistical Pattern Recognition. San Diego, CA: Academic, 1990.
6. M. Girolami, "Orthogonal series density estimation and the kernel eigenvalue problem," Neural Comput., vol. 14, no. 3, pp. 669-688, 2002.
7. G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore, MD: Johns Hopkins Univ. Press, 1989.
8. J. C. Gower, "Some distance properties of latent root and vector methods used in multivariate analysis," Biometrika, vol. 53, pp. 325-338, 1966.
9. H. Hotelling, "Simplified calculation of principal components," Psychometrica, vol. 1, pp. 27-35, 1936
10. ____, "Relations between two sets of variates," Biometrica, vol. 28, pp. 321-377, 1936.
11. I. T. Jolliffe, Principal Component Analysis, ser. Springer Series in Statistics. New York: Springer-Verlag, 1986.
12. S. Mika, G. Rätsch, J. Weston, B. Schölkopf, and K.-R. Müller, "Fisher discriminant analysis with kernels," in Neural Networks for Signal Processing IX, Y.-H. Hu, J. Larsen, E. Wilson, and S. Douglas, Eds. Piscataway, NJ: IEEE Press, 1999, pp. 41-48.
13. K. Pearson, "On lines and planes of closest fit to systems of points in space," Phil. Mag., vol. 6, no. 2, pp. 559-572, 1901.
14. T. Poggio and F. Girosi, "Networks for approximation and learning," Proc. IEEE, vol. 78, no. 9, pp. 1481-1497, 1990.
15. C. Saunders, A. Gammerman, and V. Vovk, "Ridge regression learning algorithm in dual variables," in Proc. 15th Int. Conf. Machine Learning ICML-98, Madison, WI, 1998, pp. 515-521.
16. B. Schölkopf, A. Smola, and K.-R. Müller, "Nonlinear component analysis as a kernel eigenvalue problem," Neural Comput., vol. 10, pp. 1299-1319, 1998.
17. B. Schölkopf, S. Mika, C. Burges, P. Knirsch, K.-R. Müller, G. Rätsch, and A. Smola, "Input space vs. feature space in kernel-based methods," IEEE Trans. Neural Networks, vol. 10, pp. 1000-1017, Sept. 1999.
18. B. Schölkopf and A. Smola, Learning With Kernels. Cambridge, MA: MIT Press, 2002.
19. J. A. K. Suykens and J. Vandewalle, "Least squares support vector machine classifiers," Neural Processing Lett., vol. 9, no. 3, pp. 293-300, 1999.
20. J. A. K. Suykens, J. De Brabanter, L. Lukas, and J. Vandewalle, "Weighted least squares support vector machines: Robustness and sparse approximation," Neurocomputing (Special Issue on Fundamental and Information Processing Aspects of Neurocomputing), vol. 48, no. 1-4, pp. 85-105, 2002.
21. J. A. K. Suykens, J. Vandewalle, and B. De Moor, "Optimal control by least squares support vector machines," Neural Networks, vol. 14, no. 1, pp. 23-35, 2001.
22. T. Van Gestel, A. J. K. Suykens, G. Lanckriet, A. Lambrechts, B. De Moor, and J. Vandewalle, "A Bayesian framework for least squares support vector machine classifiers, Gaussian processes and kernel Fisher discriminant analysis," Neural Comput., vol. 14, no. 5, pp. 1115-1147, 2002.
23. V. Vapnik, The Nature of Statistical Learning Theory. New-York: Springer-Verlag, 1995.
24. G. Wahba, Spline Models for Observational Data, ser. Series in Applied Mathematics. Philadelphia, PA: SIAM, 1990, vol. 59.
25. C. K. I. Williams and C. E. Rasmussen, "Gaussian processes for regression," in Advances in Neural Information Processing Systems, D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, Eds. Cambridge, MA: MIT Press, 1996, vol. 8, pp. 514-520.
26. C. K. I. Williams and M. Seeger, "Using the Nyström method to speed up kernel machines," in Advances in Neural Information Processing Systems, T. K. Leen, T. G. Dietterich, and V. Tresp, Eds. Cambridge, MA: MIT Press, 2001, vol. 13, pp. 682-688.