Noise Level Estimation for Model Selection in Kernel PCA Denoising

IEEE Transactions on Neural Networks and Learning Systems

Volumn 26, Issue 11, 2015, Pages 2650-2663

  Varon, Carolina ^a,b   Alzate, Carlos ^c   Suykens, Johan A K ^a,b  

^a STADIUS Center for Dynamical Systems Signal Processing and Data Analytics Department of Electrical Engineering (ESAT)   (Belgium)

^b UNIVERSITY OF LEUVEN   (Belgium)

^c IBM RESEARCH   (Ireland)

Author keywords

Kernel principal component analysis (kPCA); least squares support vector machines (LS SVMs); noise level estimation; unsupervised learning.

Indexed keywords

EIGENVALUES AND EIGENFUNCTIONS; RADIAL BASIS FUNCTION NETWORKS; SUPPORT VECTOR MACHINES; UNSUPERVISED LEARNING;

DISTANCE DISTRIBUTIONS; INCOMPLETE CHOLESKY FACTORIZATIONS; KERNEL PRINCIPAL COMPONENT ANALYSES (KPCA); LARGE-SCALE APPLICATIONS; LEAST SQUARES SUPPORT VECTOR MACHINES; MODEL SELECTION CRITERIA; NOISE LEVEL ESTIMATION; RADIAL BASIS FUNCTION KERNELS;

PRINCIPAL COMPONENT ANALYSIS;

EID: 85027922927     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2015.2388696     Document Type: Article

References (33)

1. I. T. Jolliffe, Principal Component Analysis. New York, NY, USA: Springer-Verlag, 1986.
2. Z. Fan et al., "Modified principal component analysis: An integration of multiple similarity subspace models," IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 8, pp. 1538-1552, Aug. 2014.
3. A. Papaioannou and S. Zafeiriou, "Principal component analysis with complex kernel: The widely linear model," IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 9, pp. 1719-1726, Sep. 2014.
4. B. Schölkopf, A. Smola, and K.-R. Müller, "Nonlinear component analysis as a kernel eigenvalue problem," Neural Comput., vol. 10, no. 5, pp. 1299-1319, Jul. 1998.
5. S. Mika, B. Schölkopf, A. Smola, K.-R. Müller, M. Scholz, and G. Rätsch, "Kernel PCA and de-noising in feature spaces," in Advances in Neural Information Processing Systems, M. J. Kearns, S. A. Solla, and D. A. Cohn, Eds. Cambridge, MA, USA: MIT Press, 1999.
6. J. T.-Y. Kwok and I. W. Tsang, "The pre-image problem in kernel methods," IEEE Trans. Neural Netw., vol. 15, no. 6, pp. 1517-1525, Nov. 2004.
7. C. M. Bishop, "Bayesian PCA," in Advances in Neural Information Processing Systems, M. J. Kearns, S. A. Solla, and D. A. Cohn, Eds. Cambridge, MA, USA: MIT Press, 1999.
8. J. Li and D. Tao, "Simple exponential family PCA," IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 3, pp. 485-497, Mar. 2013.
9. J. J. Hull, "A database for handwritten text recognition research," IEEE Trans. Pattern Anal. Mach. Intell., vol. 16, no. 5, pp. 550-554, May 1994.
10. C. Alzate and J. A. K. Suykens, "Kernel component analysis using an epsilon-insensitive robust loss function," IEEE Trans. Neural Netw., vol. 19, no. 9, pp. 1583-1598, Sep. 2008.
11. K. W. Jørgensen and L. K. Hansen, "Model selection for Gaussian kernel PCA denoising," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 1, pp. 163-168, Jan. 2012.
12. J. Rissanen, "MDL denoising," IEEE Trans. Inf. Theory, vol. 46, no. 7, pp. 2537-2543, Nov. 2000.
13. A. Hinneburg, C. C. Aggarwal, and D. A. Keim, "What is the nearest neighbor in high dimensional spaces?" in Proc. 26th Int. Conf. Very Large Data Bases, Cairo, Egypt, Sep. 2000, pp. 506-515.
14. M. Köppen, "The curse of dimensionality," in Proc. 5th Online World Conf. Soft Comput. Ind. Appl. (WSC5), Sep. 2000, pp. 4-8.
15. K. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft, "When is 'nearest neighbor' meaningful?" in Database Theory, C. Beeri and P. Buneman, Eds. Berlin, Germany: Springer-Verlag, 1999.
16. S. Lespinats, M. Verleysen, A. Giron, and B. Fertil, "DD-HDS: A method for visualization and exploration of high-dimensional data," IEEE Trans. Neural Netw., vol. 18, no. 5, pp. 1265-1279, May 2007.
17. C. K. I. Williams and M. Seeger, "Using the Nyström method to speed up kernel machines," in Advances in Neural Information Processing Systems, T. K. Leen, T. G. Dietterich, and V. Tresp, Eds. Cambridge, MA, USA: MIT Press, 2001.
18. G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore, MD, USA: The Johns Hopkins Univ. Press, 1996.
19. J. A. K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle, Least Squares Support Vector Machines. Singapore: World Scientific, 2002.
20. C. Alzate and J. A. K. Suykens, "Sparse kernel models for spectral clustering using the incomplete Cholesky decomposition," in Proc. IEEE Int. Joint Conf. Neural Netw. (IJCNN), Hong Kong, Jun. 2008, pp. 3556-3563.
21. C. Alzate and J. A. K. Suykens, "Sparse kernel spectral clustering models for large-scale data analysis," Neurocomputing, vol. 74, no. 9, pp. 1382-1390, 2011.
22. F. R. Bach and M. I. Jordan, "Kernel independent component analysis," J. Mach. Learn. Res., vol. 3, pp. 1-48, Jul. 2003.
23. M. Li, W. Bi, J. T. Kwok, and B.-L. Lu, "Large-scale Nyström kernel matrix approximation using randomized SVD," IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 1, pp. 152-164, Jan. 2015.
24. J. A. K. Suykens, T. Van Gestel, J. Vandewalle, and B. De Moor, "A support vector machine formulation to PCA analysis and its kernel version," IEEE Trans. Neural Netw., vol. 14, no. 2, pp. 447-450, Mar. 2003.
25. J. L. Horn, "A rationale and test for the number of factors in factor analysis," Psychometrika, vol. 30, no. 2, pp. 179-185, 1965.
26. R. Bellman, Adaptive Control Processes: A Guided Tour. Princeton, NJ, USA: Princeton Univ. Press, 1961.
27. M. Girolami, "Orthogonal series density estimation and the kernel eigenvalue problem," Neural Comput., vol. 14, no. 3, pp. 669-688, 2002.
28. B. W. Silverman, Density Estimation for Statistics and Data Analysis. London, U.K.: Chapman & Hall, 1986.
29. D. Widjaja, C. Varon, A. C. Dorado, J. A. K. Suykens, and S. Van Huffel, "Application of kernel principal component analysis for single-lead-ECG-derived respiration," IEEE Trans. Biomed. Eng., vol. 59, no. 4, pp. 1169-1176, Apr. 2012.
30. M. E. Tipping, "Sparse kernel principal component analysis," in Advances in Neural Information Processing Systems, T. K. Leen, T. G. Dietterich, and V. Tresp, Eds. Cambridge, MA, USA: MIT Press, 2001.
31. S. Fine and K. Scheinberg, "Efficient SVM training using low-rank kernel representations," J. Mach. Learn. Res., vol. 2, pp. 243-264, Mar. 2002.
32. C. Alzate and J. A. K. Suykens, "A regularized kernel CCA contrast function for ICA," Neural Netw., vol. 21, nos. 2-3, pp. 170-181, 2008.
33. X. Kong, K. Li, Q. Yang, L. Wenyin, and M.-H. Yang, "A new image quality metric for image auto-denoising," in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Sydney, NSW, USA, Dec. 2013, pp. 2888-2895.