A general kernelization framework for learning algorithms based on kernel PCA

Neurocomputing

Volumn 73, Issue 4-6, 2010, Pages 959-967

  Zhang, Changshui ^a   Nie, Feiping ^a   Xiang, Shiming ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

Kernel method; Kernel PCA; Learning algorithm; Two stage framework

Indexed keywords

FEATURE SPACE; INNER PRODUCT; KERNEL METHODS; KERNEL PCA; KERNEL PRINCIPAL COMPONENT ANALYSIS; KERNELIZATION; NUMERICAL STABILITIES; SPEED-UPS; TWO STAGE;

CONVERGENCE OF NUMERICAL METHODS; FEATURE EXTRACTION; PRINCIPAL COMPONENT ANALYSIS;

LEARNING ALGORITHMS;

ANALYTIC METHOD; ARTICLE; CORRELATION ANALYSIS; DISCRIMINANT ANALYSIS; INFORMATION PROCESSING; KERNEL METHOD; KERNEL PRINCIPAL COMPONENT ANALYSIS; LEARNING ALGORITHM; MATHEMATICAL COMPUTING; PARTIAL LEAST SQUARES REGRESSION; PRINCIPAL COMPONENT ANALYSIS; PRIORITY JOURNAL; STATISTICAL ANALYSIS; SUPPORT VECTOR MACHINE;

EID: 75749153570     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2009.08.014     Document Type: Article

References (52)

1. Scholkopf B., and Smola A.J. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (2001), MIT Press, Cambridge, MA, USA
2. Herbrich R. Learning Kernel Classifiers: Theory and Algorithms (2001), MIT Press, Cambridge, MA, USA
3. Shawe-Taylor J., and Cristianini N. Kernel Methods for Pattern Analysis (2004), Cambridge University Press, New York, NY, USA
4. Xu D., Yan S., and Luo J. Face recognition using spatially constrained earth mover's distance. IEEE Transactions on Image Processing 17 11 (2008) 2256-2260
5. Vapnik V.N. The Nature of Statistical Learning Theory (1995), Springer, New York, NY, USA
6. Baudat G., and Anouar F. Generalized discriminant analysis using a kernel approach. Neural Computation 12 10 (2000) 2385-2404
7. Saunders G., Gammerman A., and Vovk V. Ridge regression learning algorithm in dual variables. Proceedings of the 15th International Conference on Machine Learning (1998), Morgan Kaufmann, San Francisco, CA 515-521
8. Melzer T., Reiter M., and Bischof H. Appearance models based on kernel canonical correlation analysis. Pattern Recognition 36 9 (2003) 1961-1971
9. Rosipal R., and Trejo L.J. Kernel partial least squares regression in reproducing kernel Hilbert space. Journal of Machine Learning Research 2 (2001) 97-123
10. Bach F.R., and Jordan M.I. Kernel independent component analysis. Journal of Machine Learning Research 3 (2002) 1-48
11. Girolami M. Mercer kernel-based clustering in feature space. IEEE Transactions on Neural Networks 13 (2002) 780-784
12. Yu K., Ji L., and Zhang X. Kernel nearest-neighbor algorithm. Neural Processing Letters 15 (2002) 147-156
13. J. Wang, J. Lee, C. Zhang, Kernel trick embedded Gaussian mixture model, in: ALT, 2003, pp. 159-174.
14. A. Gretton, R. Herbrich, A. Smola, The kernel mutual information, in: Proceedings of the ICASSP, 2003.
15. Schölkopf B., Smola A.J., and Müller K.-R. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10 5 (1998) 1299-1319
16. Jolliffe I.T. Principal Component Analysis. second ed. (2002), Springer, New York
17. Harmeling S., Ziehe A., Kawanabe M., and Müller K.-R. Kernel-based nonlinear blind source separation. Neural Computation 15 5 (2003) 1089-1124
18. Yang J., Frangi A.F., Yang J.-Y., Zhang D., and Jin Z. KPCA plus LDA: a complete kernel fisher discriminant framework for feature extraction and recognition. IEEE Transactions on PAMI 27 2 (2005) 230-244
19. B. Cao, D. Shen, J.-T. Sun, Q. Yang, Z. Chen, Feature selection in a kernel space, in: ICML, 2007, pp. 121-128.
20. Horn R.A., and Johnson C.R. Matrix Analysis (1990), Cambridge University Press, Cambridge
21. Courant R., and Hilbert D. Methods of Mathematical Physics (1953), Interscience Publishers
22. Rosipal R., Girolami M., Trejo L.J., and Cichocki A. Kernel PCA for feature extraction and de-noising in nonlinear regression. Neural Computing and Applications 10 3 (2001) 231-243
23. Fine S., and Scheinberg K. Efficient SVM training using low-rank kernel representations. Journal of Machine Learning Research 2 (2001) 243-264
24. F.R. Bach, M.I. Jordan, Predictive low-rank decomposition for kernel methods, in: ICML, 2005, pp. 33-40.
25. Tikhonov A.N., and Arsenin V.Y. Solutions of Ill-Posed Problems (1977), John, New York
26. Burges C.J.C. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2 2 (1998) 121-167
27. Hoerl A.E., and Kennard R. Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12 1 (1970) 55-67
28. Fisher R.A. The use of multiple measurements in taxonomic problems. Annals of Eugenics 7 (1936) 179-188
29. Fukunaga K. Introduction to Statistical Pattern Recognition. second ed. (1990), Academic Press, Boston, MA
30. Chen L., Liao H., Ko M., Lin J., and Yu G. A new LDA based face recognition system which can solve the small sample size problem. Pattern Recognition 33 10 (2000) 1713-1726
31. Yu H., and Yang J. A direct LDA algorithm for high-dimensional data with application to face recognition. Pattern Recognition 34 (2001) 2067-2070
32. H. Li, T. Jiang, K. Zhang, Efficient and robust feature extraction by maximum margin criterion, in: NIPS, 2003.
33. Li H., Jiang T., and Zhang K. Efficient and robust feature extraction by maximum margin criterion. IEEE Transactions on Neural Networks 17 (2006) 1045-9227
34. Xu D., Yan S., Lin S., and Huang T.S. Convergent 2-d subspace learning with null space analysis. IEEE Transactions on Circuits and Systems for Video Technology 18 12 (2008) 1753-1759
35. Xu D., Yan S., Zhang L., Lin S., Zhang H.-J., and Huang T.S. Reconstruction and recognition of tensor-based objects with concurrent subspaces analysis. IEEE Transactions on Circuits and Systems for Video Technology 18 1 (2008) 36-47
36. Xu D., and Yan S. Semi-supervised bilinear subspace learning. IEEE Transactions on Image Processing 18 7 (2009) 1671-1676
37. S. Mika, G. Ratsch, J. Weston, B. Scholkopf, K. Muller, Fisher discriminant analysis with kernels, in: Proceedings of IEEE Neural Networks for Signal Processing Workshop, 1999, pp. 41-48.
38. Lu J., Plataniotis K., and Venetsanopoulos A. Face recognition using kernel direct discriminant analysis algorithms. IEEE Transactions on Neural Networks 14 (2003) 117-126
39. Zheng W., Zhao L., and Zou C. Foley-Sammon optimal discriminant vectors using kernel approach. IEEE Transactions on Neural Networks 16 (2005) 1-9
40. Liu Q., Tang X., Lu H., and Ma S. Face recognition using kernel scatter-difference-based discriminant analysis. IEEE Transactions on Neural Networks 17 (2006) 1081-1085
41. Guo Y.-F., Li S.-J., Yang J.-Y., Shu T.-T., and Wu L.-D. A generalized Foley-Sammon transform based on generalized fisher discriminant criterion and its application to face recognition. Pattern Recognition Letter 24 1-3 (2003) 147-158
42. F. Nie, S. Xiang, C. Zhang, Neighborhood minmax projections, in: IJCAI, 2007, pp. 993-998.
43. H. Wang, S. Yan, D. Xu, X. Tang, T.S. Huang, Trace ratio vs. ratio trace for dimensionality reduction, in: CVPR, 2007.
44. He X.F., Yan S.C., Hu Y.X., Niyogi P., and Zhang H.J. Face recognition using Laplacian faces. IEEE Transactions on PAMI 27 3 (2005) 328-340
45. Hotelling H. Relations between two sets of variates. Biometrika 28 (1936) 312-377
46. Wold S., Ruhe A., Wold H., and Dunn W.J. The collinearity problem in linear regression. the partial least squares (PLS) approach to generalized inverses. SIAM Journal of Scientific and Statistical Computations 5 (1984) 735-743
47. Hoskuldsson A. PLS regression methods. Journal of Chemometrics 2 (1998) 211-228
48. E. Xing, A. Ng, M. Jordan, S. Russell, Distance metric learning, with application to clustering with side-information, in: NIPS, 2003.
49. A. Asuncion, D. Newman, UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences, 2007 〈http://www.ics.uci.edu/∼mlearn/MLRepository.html〉.
50. F.S. Samaria, A.C. Harter, Parameterisation of a stochastic model for human face identification, in: 2nd IEEE Workshop on Applications of Computer Vision, 1994, pp. 138-142.
51. Graham D.B., and Allinson N.M. Characterizing virtual eigensignatures for general purpose face recognition. in face recognition: from theory to applications, NATO ASI Series F. Computer and Systems Sciences 163 (1998) 446-456
52. S.A. Nene, S.K. Nayar, H. Murase, Columbia object image library (COIL-20), Technical Report CUCS-005-96, Columbia University, 1996.