Supervised principal component analysis: Visualization, classification and regression on subspaces and submanifolds

Pattern Recognition

Volumn 44, Issue 7, 2011, Pages 1357-1371

  Barshan, Elnaz ^a   Ghodsi, Ali ^b   Azimifar, Zohreh ^a   Zolghadri Jahromi, Mansoor ^a  

^a SHIRAZ UNIVERSITY   (Iran)

^b UNIVERSITY OF WATERLOO   (Canada)

Author keywords

Classification; Dimensionality reduction; Kernel methods; Principal component analysis (PCA); Regression; Supervised learning; Visualization

Indexed keywords

CLASSIFICATION; DIMENSIONALITY REDUCTION; KERNEL METHODS; PRINCIPAL COMPONENT ANALYSIS (PCA); REGRESSION;

COMPUTATIONAL COMPLEXITY; COMPUTATIONAL EFFICIENCY; REGRESSION ANALYSIS; SUPERVISED LEARNING; VISUALIZATION;

PRINCIPAL COMPONENT ANALYSIS;

EID: 79952190499     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2010.12.015     Document Type: Article

References (56)

1. I.T. Jolliffe Principal Component Analysis 1986 Springer-Verlag New York
2. R. Bellman, Adaptive Control Process: A Guided Tour, University Press, Princeton.
3. R.A. Fisher The use of multiple measurements in taxonomic problems Annals of Eugenics 7 1936 179 188
4. E.P. Xing, A.Y. Ng, M.I. Jordan, S. Russell, Distance metric learning with application to clustering with side-information, in: Advances in Neural Information Processing Systems (NIPS), vol. 15, 2002, pp. 505512.
5. M. Bilenko, S. Basu, R.J. Mooney, Integrating constraints and metric learning in semi-supervised clustering, in: ICML, vol. 69, 2004, p. 11.
6. H. Chang, D.-Y. Yeung, Locally linear metric adaptation for semi-supervised clustering, in: ICML, vol. 69, 2004, pp. 153160. (Pubitemid 40290803)
7. H. Chang, and D.-Y. Yeung Locally linear metric adaptation with application to semi-supervised clustering and image retrieval Pattern Recognition 39 7 2006 1253 1264
8. D.-Y. Yeung, and H. Chang Extending the relevant component analysis algorithm for metric learning using both positive and negative equivalence constraints Pattern Recognition 39 5 2006 1007 1010 (Pubitemid 43276113)
9. S. Basu, M. Bilenko, R.J. Mooney, A probabilistic framework for semi-supervised clustering, in: KDD, 2004, pp. 5968. (Pubitemid 40114916)
10. K.Q. Weinberger, J. Blitzer, L.K. Saul, Distance metric learning for large margin nearest neighbor classification, in: Advances in Neural Information Processing Systems, vol. 18, 2006, pp. 14731480.
11. A. Globerson, S.T. Roweis, Metric learning by collapsing classes, in: Advances in Neural Information Processing Systems, vol. 18, 2006, pp. 451458.
12. B. Alipanahi, M. Biggs, A. Ghodsi, Distance metric learning versus fisher discriminant analysis, in: Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, 2008, pp. 598603.
13. K. Li Sliced inverse regression for dimension reduction (with discussion) Journal of the American Statistical Association 86 1991 316 342
14. R.D. Cook, and S. Weisberg Discussion of li (1991) Journal of the American Statistical Association 86 1991 328 332
15. K. Li On principal hessian directions for data visualization and dimension reduction: another application of Stein's lemma Journal of the American Statistical Association 87 1992 1025 1039
16. A.M. Samarov Exploring regression structure using nonparametric functional estimation Journal of the American Statistical Association 88 1993 836 847
17. R.D. Cook, and X. Yin Dimension reduction and visualization in discriminant analysis (with discussion) Australian & New-Zealand Journal of Statistics 43 2001 147 199 (Pubitemid 33613436)
18. M. Hristache, A. Juditsky, J. Polzehl, and V. Spokoiny Structure adaptive approach for dimension reduction The Annals of Statistics 29 2001 1537 1566 (Pubitemid 33405985)
19. K. Torkkola Feature extraction by non-parametric mutual information maximization Journal of Machine Learning Research 3 2003 1415 1438
20. K. Fukumizu, F.R. Bach, and M.I. Jordan Dimensionality reduction for supervised learning with reproducing kernel Hilbert spaces Journal of Machine Learning Research 5 2004 73 99
21. E. Bair, T. Hastie, D. Paul, and R. Tibshirani Prediction by supervised principal components Journal of the American Statistical Association 101 2006 119 137 (Pubitemid 43500030)
22. N. Aronszajn Theory of reproducing kernels Transactions of the American Mathematical Society 68 3 1950 337 404
23. J. Nilsson, F. Sha, M.I. Jordan, Regression on manifolds using kernel dimension reduction, in: ICML, vol. 227, 2007, pp. 697704.
24. A. Gretton, O. Bousquet, A.J. Smola, B. Schlkopf, Measuring statistical dependence with HilbertSchmidt norms, in: Proceedings Algorithmic Learning Theory (ALT), vol. 3734, 2005, pp. 6377.
25. L. Song, A.J. Smola, A. Gretton, K.M. Borgwardt, J. Bedo, Supervised feature selection via dependence estimation, in: ICML, vol. 227, 2007, pp. 823830.
26. L. Song, A.J. Smola, K.M. Borgwardt, A. Gretton, Colored maximum variance unfolding, in: Advances in neural information processing systems, vol. 20, 2008, pp. 13851392.
27. L. Song, A.J. Smola, A. Gretton, K.M. Borgwardt, A dependence maximization view of clustering, in: ICML, vol. 227, 2007, pp. 815822.
28. C.R. Baker Joint measures and cross-covariance operators Transactions of the American Mathematical Society 186 1973 273 289
29. H. Lutkepohl Handbook of Matrices 1997 John Wiley and Sons
30. J.L. Alperin Local Representation Theory: Modular Representations as an Introduction to the Local Representation Theory of Finite Groups 1986 Cambridge University Press
31. A. Asuncion, D. Newman, Uci machine learning repository, 2007.
32. U. Alon, N. Barkai, D.A. Notterman, K. Gishdagger, S. Ybarradagger, D. Mackdagger, and A.J. Levine Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays Proceedings of the National Academy of Sciences of the United States of America 96 12 1999 6745 6750 (Pubitemid 29274954)
33. A.A. Alizadeh, M.B. Eisen, R.E. Davis, C. Ma, I.S. Lossos, A. Rosenwald, J.C. Boldrick, H. Sabet, T. Tran, X. Yu, J.I. Powell, L. Yang, G.E. Marti, T. Moore, J. Hudson, L. Lu, D.B. Lewis, R. Tibshirani, G. Sherlock, W.C. Chan, T.C. Greiner, D.D. Weisenburger, J.O. Armitage, R. Warnke, R. Levy, W. Wilson, M.R. Grever, J.C. Byrd, D. Botstein, P.O. Brown, and L.M. Staudt Distinct types of diffuse large b-cell lymphoma identified by gene expression profiling Nature 403 6769 2000 503 511 (Pubitemid 30082188)
34. J. Khan, J.S. Wei, M. Ringnr, L.H. Saal, M. Ladanyi, F. Westermann, F. Berthold, M. Schwab, C.R. Antonescu, C. Peterson, and P.S. Meltzer Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks Nature Medicine 7 6 2001 673 679 (Pubitemid 32588022)
35. L. Song, J. Bedo, K.M. Borgwardt, A. Gretton, and A.J. Smola Gene selection via the BAHSIC family of algorithms Bioinformatics 23 2007 490 498
36. W.F. Massy Principal components regression in exploratory statistical research Journal of the American Statistical Association 60 1965 234 256
37. B. Li, H. Zha, F. Chiaromonte, Contour regression: a general approach to dimension reduction, in: ICML, vol. 33, 2005, pp. 15801616. (Pubitemid 41423981)
38. A. Rosenwald, G. Wright, W.C. Chan, J.M. Connors, E. Campo, R.I. Fisher, R.D. Gascoyne, H.K. Muller-Hermelink, E.B. Smeland, and L.M. Staudt The use of molecular profiling to predict survival after chemotherapy for diffuse large b-cell lymphoma Annals of Statistics 346 4 2002 1937 1947 (Pubitemid 34651353)
39. H. Wold Soft modeling by latent variables: The nonlinear iterative partial least squares approach in: Perspectives in Probability and Statistics, Papers in Honour of MS Bartlett, 1975 520 540
40. S. Wold, H. Ruhe, H. Wold, and W.J. Dunn III The collinearity problem in linear regression The partial least squares (PLS) approach to generalized inverse, in: SIAM Journal of Scientific and Statistical Computations vol. 5 1984 735 743
41. J.A. Wegelin, A survey of partial least squares (pls) methods, with emphasis on the two-block case, Technical Report, University of Washington, 2000.
42. R. Rosipal, and L.J. Trejo Kernel partial least squares regression in reproducing kernel Hilbert space Journal of Machine Learning Research 2 2001 97 123
43. M. Momma, K. Bennett, Sparse kernel partial least squares regression, in: Proceedings of Conference on Learning Theory (COLT), 2003.
44. R. Rosipal, L.J. Trejo, B. Matthews, Kernel pls-svc for linear and nonlinear classification, in: Proceedings of International Conference on Machine Learning (ICML), vol. 20, 2003, p. 640.
45. L. Hoegaerts, J.A.K. Suykens, J. Vanderwalle, B.D. Moor, Primal space sparse kernel partial least squares regression for large problems, in: Proceedings of International Joint Conference on Neural Networks (IJCNN), 2004.
46. J. Arenas-Garca, K.B. Petersen, L.K. Hansen, Sparse kernel orthonormalized pls for feature extraction in large data sets, in: Advances in Neural Information Processing Systems, 2007, p. 33.
47. R. Rosipal Kernel partial least squares for nonlinear regression and discrimination Neural Network World 13 3 2003 291 300
48. H. Hotelling Relations between two sets of variables Biometrika 28 1936 312 377
49. D. Hardoon, S. Szedmak, and J. Shawe-taylor Canonical correlation analysis: an overview with application to learning methods Neural Computation 16 12 2004 2639 2664 (Pubitemid 39604012)
50. M.S. Bartlett Further aspects of the theory of multiple regression Proceedings of the Cambridge Philosophical Society vol. 34 1938 33 40
51. R. Rosipal, and N. Krmer Overview and recent advances in partial least squares Subspace, Latent Structure and Feature Selection 2006 34 51 (Pubitemid 44029878)
52. C. Fyfe, and P. Lai Kernel and nonlinear canonical correlation analysis IEEE-INNS-ENNS International Joint Conference on Neural Networks vol. 4 2000 4614
53. S. Roweis, and L. Saul Nonlinear dimensionality reduction by locally linear embedding Science 290 2000 2323 2326 (Pubitemid 32041578)
54. M. Belkin, P. Niyogi, Laplacian eigenmaps and spectral techniques for embedding and clustering, in: Advances in Neural Information Processing Systems, vol. 1, 2002, pp. 585592.
55. J. Tenenbaum, Mapping a manifold of perceptual observations, in: Advances in Neural Information Processing Systems, vol. 10, 1998, pp. 682687.
56. J. Ham, D. Lee, S. Mika, and B. Schlkopf A kernel view of the dimensionality reduction of manifolds International Conference on Machine Learning 2004