Principal component analysis based on non-parametric maximum entropy

Neurocomputing

Volumn 73, Issue 10-12, 2010, Pages 1840-1852

  He, Ran ^a   Hu, Baogang ^b   Yuan, XiaoTong ^c   Zheng, Wei Shi ^d  

^a DALIAN UNIVERSITY OF TECHNOLOGY   (China)

^b INSTITUTE OF AUTOMATION   (China)

^c NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^d QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Entropy; Information theoretic learning; PCA; Subspace learning

Indexed keywords

A-LAPLACIAN; EIGENVECTORS; INFORMATION THEORETIC LEARNING; LINEAR METHODS; MAXIMUM ENTROPY; NON-PARAMETRIC; OPTIMAL SOLUTIONS; PARZEN WINDOW ESTIMATION; PCA; PROBABILITY MATRIXES; QUADRATIC ENTROPY; REAL-WORLD DATASETS; RECONSTRUCTION ERROR; ROBUST PCA; ROTATION INVARIANT; SUBSPACE LEARNING;

ENTROPY; INFORMATION THEORY; OPTIMIZATION; PROBABILITY DISTRIBUTIONS;

PRINCIPAL COMPONENT ANALYSIS;

ARTICLE; DISCRIMINANT ANALYSIS; ENTROPY; FISHER EXACT TEST; KERNEL METHOD; LEARNING ALGORITHM; NORMAL DISTRIBUTION; PRINCIPAL COMPONENT ANALYSIS; PRIORITY JOURNAL; THEORETICAL MODEL;

EID: 77952546541     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2009.12.032     Document Type: Article

References (72)

1. Guyon I., Elissee A. An introduction to variable and feature selection. Journal of Machine Learning Research 2003, 3:1157-1182.
2. J. Fan, R. Li, Statistical challenges with high dimensionality: feature selection in knowledge discovery, in: the Madrid International Congress of Mathematician, 2006, pp. 595-622.
3. Tao D., Li X., Wu X., Maybank S. Tensor rank one discriminant analysis-a convergent method for discriminative multilinear subspace selection. Neurocomputing 2008, 71:1866-1882.
4. Belhumeur P., Hespanha J., Kriegman D. Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 1997, 19(1):711-720.
5. He X., Yan S., Hu Y., Niyogi P., Zhang H. Face recognition using Laplacian faces. IEEE Transactions on Pattern Analysis and Machine Intelligence 2005, 27(3):328-340.
6. G. Hinton, S. Roweis, Stochastic neighbor embedding, in: Advances in Neural Information Processing Systems, vol. 15, MIT Press, 2002, pp. 833-840.
7. Zhang T., Li X., Yang D.T.J. Local coordinates alignment (LCA): a novel manifold learning approach. IJPRAI 2008, 22(4):667-690.
8. Hotelling H. Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology 1933, 24:417-441.
9. Li J., Li X., Tao D. KPCA for semantic object extraction in images. Pattern Recognition 2008, 41:3244-3250.
10. Pang Y., Tao D., Yuan Y., L1 X. Binary two-dimensional PCA. IEEE Transactions on Systems, Man, and Cybernetics, Part B 2008, 38:1176-1180.
11. Chen S., Zhao H., Kong M., Luo B. 2d-lpp: A two-dimensional extension of locality preserving projections. Neurocomputing 2007, 70:912-921.
12. Yan S., Xu D., Zhang B., Zhang H., Yang Q., Lin S. Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Transactions on Pattern Analysis and Machine Intelligence 2007, 29(1):40-51.
13. Xu L., Yuille A. Robust principal component analysis by self-organizing rules based on statistical physics approach. IEEE Transactions on Neural Networks 1995, 6:131-143.
14. Locantore N., Marron J., Simpson D., Tripoli N., Zhang J., Cohen K. Robust principal component analysis for functional data. Test 1999, 8(1):1-73.
15. Maronna R.A. Principal components and orthogonal regression based on robust scales. Technometrics 2005, 47:264-273.
16. Pang Y., Li X., Yuan Y., Tao D., Pan J. Fast haar transform based feature extraction for face representation and recognition. IEEE Transactions on Information Forensics and Security 2009, 4:441-450.
17. Baccini A., Besse P., Falguerolles A. A l1-norm PCA and a heuristic approach. Ordinal and Symbolic Data Analysis 1996, 1:359-368.
18. Q. Ke, T. Kanade, Robust l1 norm factorization in the presence of outliers and missing data by alternative convex programming, in: Computer Vision and Pattern Recognition, 2005.
19. C. Ding, D. Zhou, X. He, H. Zha, R1-pca: rotational invariant l1-norm principal component analysis for robust subspace factorization, in: International Conference on Machine Learning, 2006.
20. Kwak N. Principal component analysis based on l1-norm maximization. IEEE Transactions on Pattern Analysis and Machine Intelligence 2008, 30(9):1672-1677.
21. Y.W. Pang, X.L. Li, Y. Yuan, L1-norm based tensor analysis, IEEE Transactions on Circuits and Systems for Video Technology 20 (2) (2010) 172-178.
22. Torre F., Black M. A framework for robust subspace learning. International Journal of Computer Vision 2003, 54(1-3):117-142.
23. Hubert M., Rousseeuwy P., Branden K. Robpca a new approach to robust principal component analysis. Technometrics 2005, 47:64-79.
24. Yang J., Zhang D., Yang J. Locally principal component learning for face representation and recognition. Neurocomputing 2006, 69:1697-1701.
25. Moller R., Hoffmannb H. An extension of neural gas to local pca. Neurocomputing 2004, 62:305-326.
26. Huang D., Yia Z., Pu X. A new local pca-som algorithm. Neurocomputing 2004, 71:16-18.
27. J. Park, Z. Zhang, H. Zha, R. Kasturi, Local smoothing for manifold learning, in: Computer Vision and Pattern Recognition, 2004.
28. D. Zhao, Z. Lin, X. Tang, Laplacian pca and its applications, in: International Conference on Computer Vision, 2007.
29. R. Vidal, Y. Ma, S. Sastry, Generalized principal component analysis, in: Computer Vision and Pattern Recognition, vol. 1, 2003, pp. 621-628.
30. J. Principe, D. Xu, J.W.F. Iii, Information-theoretic learning. http://www.cnel.ufl.edu/bib/./pdf_papers/chapter7.pdf.
31. D. Xu, Energy, entropy and information potential for neural computation, Ph.D. Thesis, University of Florida, 1999.
32. P. Viola, N. Schraudolph, T. Sejnowski, Empirical entropy manipulation for real-world problems, in: Neural Information Processing Systems (NIPS), 1995, pp. 851-857.
33. X. Yuan, B. Hu, Robust feature extraction via information theoretic learning, in: International Conference on Machine Learning (ICML 2009), Montreal, Canada, 2009.
34. Girolami M. Orthogonal series density estimation and the kernel eigenvalue problem. Neural Computation 2002, 14(3):669-688.
35. Jenssen R., Erdogmus D., Principe J., Eltoft T. Some equivalences between kernel methods and information theoretic methods. Journal of VLSI Signal Processing 2006, 45:49-65.
36. R. Jenssen, D. Erdogmus, J. Principe, T. Eltoft, Information theoretic angle-based spectral clustering: a theoretical analysis and an algorithm, in: International joint conference on neural networks, 2006, pp. 4904-4911.
37. A.R. Paiva, J. Wu, J. Principe, kernel principal component are maximum entropy projection, in: ICA, 2006, pp. 846-853.
38. R. Jenssen, T. Eltoft, M. Girolami, D. Erdogmus, Kernel maximum entropy data transformation and an enhanced spectral clustering algorithm, in: Neural Information Processing Systems (NIPS), 2006.
39. Pearson K. On lines and planes of closest fit to systems of points in space. The London, Edinburgh and Dublin Philosophical Magazine and Journal of Science, Sixth Series 1901, 2:559-572.
40. Luenberger D. Optimization by Vector Space Methods 1969, Wiley.
41. S. Roweis, Em algorithms for PCA and SPCA, in: Neural Information Processing Systems (NIPS), 1997, pp. 626-632.
42. Ahna J., Oha J., Choib S. Learning principal directions: integrated-squared-error minimization. Neurocomputing 2007, 70:1372-1381.
43. Golub G., Loan C.V. Matrix Computations 1996, Johns Hopkins, Baltimore. third ed.
44. Zhang T.H., Tao D.C., Li X.L., Yang J. Patch alignment for dimensionality reduction. IEEE Transactions on Knowledge and Data Engineering 2009, 21(9):1299-1313.
45. Q. Ke, T. Kanade, Robust subspace computation using l1 norm, Techical Report. http://citeseer.ist.psu.edu/ke03robust.html.
46. Danijel S., Ales L., Horst B. Weighted and robust learning of subspace representations. Pattern Recognition 2007, 40:1556-1569.
47. Jolliffe I.T. Principal Component Analysis 1986, Springer-Verlag, New York.
48. Scholkopf B., Smola A.J., Muller K.R. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 1998, 10:1299-1319.
49. Cover T., Thomas J.A. Elements of Information Theory 2005, John Wiley, New Jersey. second ed.
50. Torkkola K. Feature extraction by nonparametric mutual information maximization. Journal of Machine Learning Research 2003, 3:1415-1438.
51. Huber P. Robust Statistics 1981, Wiley.
52. Liu W., Pokharel P.P., Principe J.C. Correntropy: properties and applications in non-Gaussian signal processing. IEEE Transactions on Signal Processing 2007, 55(11):5286-5298.
53. Hyvarinen A. Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks 1999, 10:626-634.
54. Sharma A., Paliwal K.K. Fast principal component analysis using fixed-point algorithm. Pattern Recognition Letters 2007, 28:1151-1155.
55. Pang Y., Yuan Y., Li X. Iterative subspace analysis based on feature line distance. IEEE Transactions on Image Processing 2009, 18:903-907.
56. Nocedal J., Wright S.J. Numberical Optimization 2000, Springer.
57. A. Ng, Feature selection, l1 vs. l2 regularization, and rotational invariance, in: International Conference on Machine Learning, 2004.
58. Cover T., Thomas J.A. Elements of Information Theory 2005, John Wiley, New York. second ed.
59. D. Newman, S. Hettich, C. Blake, C. Merz, Uci repository of machine learning databases. http://www.ics.uci.edu/mlearn/MLRepository.html.
60. II K.H., Erdogmus D., Torkkola K., Principe C. Feature extraction using information-theoretic learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 2006, 28(9):1385-1392.
61. Serneels S., Verdonck T. Principal component analysis for data containing outliers and missing elements. Computational Statistics and Data Analysis 2008, 52:1712-1727.
62. van der Linde A. Variational Bayesian functional PCA. Computational Statistics and Data Analysis 2008, 53:517-533.
63. Hubert M., Rousseeuw P., Verdonck T. Robust pca for skewed data and its outlier map. Computational Statistics and Data Analysis 2009, 53:2264-2274.
64. Turk M., Pentland A. Eigenfaces for recognition. Journal of Cognitive Neuroscience 1991, 3(1):71-86.
65. P. Philips, P. Flynn, T. Sruggs, K. Bowyer, Overview of the face recognition grand challenge, in: Computer Vision and Pattern Recognition, 2005.
66. Duda R.O., Hart P.E., Stork D.G. Pattern Classification 2000, Wiley-Interscience.
67. S.H. Yang, H.Y. Zha, S. Zhou, B.-G. Hu, Variational graph embedding for globally and locally consistent feature extraction, in: Europe Conference on Machine Learning (ECML), 2009, pp. 538-553.
68. R. He, B. Hu, X. Yuan, Robust discriminant analysis based on nonparametric maximum entropy, in: Asian Conference on Machine Learning (ACML), NanJing, China, 2009.
69. Mizera I., Muller C. Breakdown points of cauchy regression-scale estimators. Statistics and Probability Letters 2002, 57:79-89.
70. Silverman B.W. Density Estimation for Statistics and Data Analysis 1986, Chapman and Hall, London, UK.
71. Rousseeuw P.J. Robust Regression and Outlier Detection 1987, Wiley, New York.
72. M. Rogers, J. Graham, Robust active shape model search, in: European Conference on Computer Vision, Springer, 2002, pp. 517-530.