Kernel component analysis using an epsilon-insensitive robust loss function

IEEE Transactions on Neural Networks

Volumn 19, Issue 9, 2008, Pages 1583-1598

  Alzate, Carlos ^a   Suykens, Johan A K ^a  

^a UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

Epsilon insensitive loss function; Kernel principal component analysis (PCA); Least squares support vector machines (LS SVM); Loss function; Robustness; Sparseness

Indexed keywords

CURVE FITTING; FEATURE EXTRACTION; FINANCIAL DATA PROCESSING; LEAST SQUARES APPROXIMATIONS; SUPPORT VECTOR MACHINES; TWO PHASE FLOW;

COMPONENT ANALYSIS; EIGEN-DECOMPOSITION; EIGENVALUES; EIGENVECTORS; EPSILON-INSENSITIVE LOSS FUNCTION; FEATURE SPACES; HIGH-DIMENSIONAL FEATURE SPACE; INPUT SPACES; KERNEL MATRICES; KERNEL PCA; KERNEL PRINCIPAL COMPONENT ANALYSIS; KERNEL PRINCIPAL COMPONENT ANALYSIS (PCA); LEAST SQUARES SUPPORT VECTOR MACHINES (LS-SVM); LEAST-SQUARES SUPPORT VECTOR MACHINE; LOSS FUNCTION; LOSS FUNCTIONS; PRINCIPAL COMPONENTS; ROBUSTNESS; SPARSENESS;

PRINCIPAL COMPONENT ANALYSIS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; METHODOLOGY; PRINCIPAL COMPONENT ANALYSIS; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; MODELS, STATISTICAL; PATTERN RECOGNITION, AUTOMATED; PRINCIPAL COMPONENT ANALYSIS;

EID: 52149083934     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2008.2000443     Document Type: Article

References (28)

1. I. T. Jolliffe, Principal Component Analysis. New York: Springer Verlag, 1986.
2. B. Schölkopf, A. J. Smola, and K.-R. Müller, "Nonlinear component analysis as a Kernel eigenvalue problem," Neural Comput., vol. 10, pp. 1299-1319, 1998.
3. S. Mika, B. Schölkopf, A. J. Smola, K.-R. Müller, M. Scholz, and G. Rätsch, "Kernel PCA and de-noising in feature spaces," in Advances in Neural Information Processing Systems 11, M. S. Kearns, S. A. Solla, and D. A. Cohn, Eds. Cambridge, MA: MIT Press, 1999.
4. B. Schölkopf, S. Mika, C. J. C. Burges, P. Knirsch, K.-R. Müller, M. Scholz, G. Rätsch, and A. J. Smola, "Input space versus feature space in kernel-based methods," IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 1000-1017, Sep. 1999.
5. J. T. Kwok and I. W. Tsang, "The pre-image problem in kernel methods," IEEE Trans. Neural Netw., vol. 15, no. 6, pp. 1517-1525, Nov. 2004.
6. T. Takahashi and T. Kurita, "Robust de-noising by kernel PCA," in Proc. Int. Conf. Artif. Neural Netw., 2002, pp. 739-744.
7. J. A. K. Suykens, T. Van Gestel, J. Vandewalle, and B. De Moor, "A support vector machine formulation to PCA analysis and its kernel version," IEEE Trans. Neural Netw., vol. 14, no. 2, pp. 447-450, Mar. 2003.
8. P. Huber, Robust Statistics. New York: Wiley, 1981.
9. P. J. Rousseeuw and A. M. Leroy, Robust Regression and Outlier Detection. New York: Wiley, 1987.
10. F. Critchley, "Influence in principal component analysis," Biometrika, vol. 72, pp. 627-636, 1985.
11. L. Xu and A. L. Yuille, "Robust principal component analysis by self-organizing rules based on statistical physics approach," IEEE Trans. Neural Netw., vol. 6, no. 1, pp. 131-143, Jan. 1995.
12. I. Higuchi and S. Eguchi, "The influence function of principal component analysis by self-organizing rule," Neural Comput., vol. 10, pp. 1435-1444, 1998.
13. A. J. Smola, O. Mangasarian, and B. Schölkopf, "Sparse kernel feature analysis," Data Mining Inst., Univ. Wisconsin, Madison, WI, Tech. Rep. 99-03, 1999.
14. J. A. K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle, Least Squares Support Vector Machines., Singapore: World Scientific, 2002.
15. C. Alzate and J. A. K. Suykens, "A weighted Kernel PCA formulation with out-of-sample extensions for spectral clustering methods," in Proc. Int. Joint Conf. Neural Netw., 2006, pp. 138-144.
16. J. A. K. Suykens, J. De Brabanter, L. Lukas, and J. Vandewalle, "Weighted least squares support vector machines: Robustness and sparse approximation," Neurocomputing, vol. 48, pp. 85-105, 2002.
17. R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge, U.K.: Cambridge Univ. Press, 1990.
18. F. De la Torre and M. Black, "Robust principal component analysis for computer vision," in Proc. Int. Conf. Comput. Vis., 2001, pp. 362-369.
19. I. Higuchi and S. Eguchi, "Robust principal component analysis with adaptive selection for tuning parameters," J. Mach. Learn. Res., vol. 5, pp. 453-471, 2004.
20. C. Croux and G. Haesbroeck, "Principal component analysis based on robust estimators of the covariance or correlation matrix: Influence functions and efficiencies," Biometrika, vol. 87, no. 3, pp. 603-618, 2000.
21. M. Hubert, P. J. Rousseeuw, and K. Vanden Branden, "ROBPCA: A new approach to robust principal component analysis," Technometrics, vol. 47, pp. 64-79, 2005.
22. F. R. Hampel, E. M. Ronchetti, P. J. Rousseeuw, and W. A. Stahel, Robust Statistics: The Approach Based on Influence Functions. New York: Wiley, 1986.
23. A. Christmann and I. Steinwart, "On robustness properties of convex risk minimization methods for pattern recognition," J. Mach. Learn. Res., vol. 5, pp. 1007-1034, Aug. 2004.
24. V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
25. J. Nocedal and S. J. Wright, Numerical Optimization. New York: Springer-Verlag, 2000.
26. G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore, MD: The Johns Hopkins University Press, 1996.
27. J. H. Wilkinson, The Algebraic Eigenvalue Problem. Oxford, U.K.: Clarendon, 1978.
28. M. J. D. Powell, "A fast algorithm for nonlinearly constrained optimization calculations," in Proc. Dundee Biennial Conf. Numer. Anal., G. A. Watson, Ed., 1978, pp. 144-157.