Generalized linear discriminant analysis: A unified framework and efficient model selection

IEEE Transactions on Neural Networks

Volumn 19, Issue 10, 2008, Pages 1768-1782

  Ji, Shuiwang ^a   Ye, Jieping ^a  

^a AZ 85281   (United States)

Author keywords

Dimensionality reduction; Linear discriminant analysis (LDA); Model selection; Principal component analysis (PCA); Regularization; Visualization

Indexed keywords

ALGORITHMS; BIOACTIVITY; DISCRIMINANT ANALYSIS; FACE RECOGNITION; GENE EXPRESSION; IMAGE ENHANCEMENT; IMAGE RECOGNITION; IMAGE RETRIEVAL; MATRIX ALGEBRA; MODAL ANALYSIS;

DIMENSIONALITY REDUCTION; LINEAR DISCRIMINANT ANALYSIS (LDA); MODEL SELECTION; PRINCIPAL COMPONENT ANALYSIS (PCA); REGULARIZATION;

PRINCIPAL COMPONENT ANALYSIS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; DISCRIMINANT ANALYSIS; MATHEMATICAL COMPUTING; METHODOLOGY; STATISTICAL MODEL; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DISCRIMINANT ANALYSIS; LINEAR MODELS; MODELS, THEORETICAL; NUMERICAL ANALYSIS, COMPUTER-ASSISTED; PATTERN RECOGNITION, AUTOMATED;

EID: 54349111811     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2008.2002078     Document Type: Article

References (37)

1. P. N. Belhumeour, J. P. Hespanha, and D. J. Kriegman, "Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 711-720, Jul. 1997.
2. M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Comput., vol. 15, no. 6, pp. 1373-1396, 2003.
3. R. E. Bellman, Adaptive Control Processes: A Guided Tour. Princeton, NJ: Princeton Univ. Press, 1961.
4. C. J. C. Burges, "Geometric methods for feature extraction and dimensional reduction," in Mining and Knowledge Discovery Handbook: A Complete Guide for Practitioners and Researchers, O. Maimon and L. Rokach, Eds. New York: Springer-Verlag, 2005, pp. 59-92.
5. H. Cevikalp, M. Neamtu, M. Wilkes, and A. Barkana, "Discriminative common vectors for face recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 1, pp. 4-13, Jan. 2005.
6. L. F. Chen, H. Y. M. Liao, J. C. Lin, M. D. Kao, and G. J. Yu, "A new LDA-based face recognition system which can solve the small sample size problem," Pattern Recognit., vol. 33, no. 10, pp. 1713-1726, 2000.
7. D. Donoho, "High-dimensional data analysis: The curses and blessings of dimensionality," Aug. 2000 [Online]. Available: http://www-stat.stanford.edu/~donoho/Lectures/AMS2000/AMS2000.html, American Mathematical Society Lecture-Math Challenges of the 21st Century
8. L. Duchene and S. Leclerq, "An optimal transformation for discriminant and principal component analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 10, no. 6, pp. 978-983, Jun. 1988.
9. R. O. Duda, P. E. Hart, and D. Stork, Pattern Classification. New York: Wiley, 2000.
10. R. A. Fisher, "The use of multiple measurements in taxonomic problems," Ann. Eugenics, vol. 7, pp. 179-188, 1936.
11. K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed. San Diego, CA: Academic, 1990.
12. G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore, MD: The Johns Hopkins University Press, 1996.
13. Y. Guo, T. Hastie, and R. Tibshirani, "Regularized linear discriminant analysis and its application in microarrays," Biostatistics, vol. 8, no. 1, pp. 86-100, 2007.
14. R. Gurunathan, B. Van Emden, S. Panchanathan, and S. Kumar, "Identifying spatially similar gene expression patterns in early stage fruit fly embryo images: Binary feature versus invariant moment digital representations," BMC Bioinf., vol. 5, no. 202, p. 13, 2004.
15. P. Hall, J. S. Marron, and A. Neeman, "Geometric representation of high dimension, low sample size data," J. Roy. Statist. Soc. B, Statist. Method., vol. 67, no. 18, pp. 427-444, Jun. 2005.
16. T. Hastie, R. Tibshirani, and J. H. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer-Verlag, 2001.
17. P. Howland, M. Jeon, and H. Park, "Structure preserving dimension reduction for clustered text data based on the generalized singular value decomposition," SIAM J. Matrix Anal. Appl., vol. 25, no. 1, pp. 165-179, 2003.
18. I. T. Jolliffe, Principal Component Analysis. New York: Springer-Verlag, 1986.
19. A. M. Martinez and R. Benavente, "The AR-Face database," CVC Tech. Rep. 24, 1998.
20. A. M. Martinez and A. C. Kak, "PCA versus LDA," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 2, pp. 228-233, Feb. 2001.
21. A. M. Martinez and M. Zhu, "Where are linear feature extraction methods applicable?," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 12, pp. 1934-1944, Dec. 2005.
22. H. Park, M. Jeon, and J. B. Rosen, "Lower dimensional representation of text data based on centroids and least squares," BIT Numer. Math. vol. 43, no. 2, pp. 1-22, 2003.
23. S. L. Pomeroy, P. Tamayo, M. Gaasenbeek, L. M. Sturla, M. Angelo, M. E. McLaughlin, J. Y. H. Kim, L. C. Goumnerova, P. M. Black, C. Lau, J. C. Allen, D. Zagzag, J. M. Olson, T. Curran, C. Wetmore, J. A. Biegel, T. Poggio, S. Mukherjee, R. Rifkin, A. Califano, G. Stolovitzky, D. N. Louis, J. P. Mesirov, E. S. Lander, and T. R. Golub, "Prediction of central nervous system embryonal tumour outcome based on gene expression," Nature, vol. 415, no. 6870, pp. 436-442, 2002.
24. S. Ramaswamy, P. Tamayo, R. Rifkin, S. Mukherjee, C.-H. Yeang, M. Angelo, C. Ladd, M. Reich, E. Latulippe, J. P. Mesirov, T. Poggio, W. Gerald, M. Loda, E. S. Lander, and T. R. Golub, "Multiclass cancer diagnosis using tumor gene expression signatures," Proc. Nat. Acad. Sci., vol. 98, no. 26, pp. 15149-15154, 2001.
25. S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, no. 5500, pp. 2323-6, 2000.
26. L. K. Saul, K. Q. Weinberger, J. H. Ham, F. Sha, and D. D. Lee, O. Chapelle, B. Schölkopf, and A. Zien, Eds., "Spectral methods for dimensionality reduction," in Semisupervised Learning. Cambridge, MA: The MIT Press, 2006.
27. A. I. Su, J. B.Welsh, L. M. Sapinoso, S. G. Kern, P. Dimitrov, H. Lapp, P. G. Schultz, S. M. Powell, C. A. Moskaluk, Jr., H. F. Frierson, and G. M. Hampton, "Molecular classification of human carcinomas by use of gene expression signatures," Cancer Res., vol. 61, no. 20, pp. 7388-7393, 2001.
28. J. B. Tenenbaum, V. d. Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, no. 5500, pp. 2319-2323, 2000.
29. X. Wang and X. Tang, "A unified framework for subspace face recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 9, pp. 1222-1228, Sep. 2004.
30. J. Weston, C. Leslie, D. Zhou, A. Elisseeff, and W. S. Noble, "Semi-supervised protein classification using cluster kernels," in Advances in Neural Information Processing Systems 16. Cambridge, MA: The MIT Press, 2004.
31. S. Yan, D. Xu, B. Zhang, H.-J. Zhang, Q. Yang, and S. Lin, "Graph embedding and extensions: A general framework for dimensionality reduction," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 1, pp. 40-51, Jan. 2007.
32. J. Ye, "Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems," J. Mach. Learn. Res., vol. 6, pp. 483-502, 2005.
33. J. Ye, J. Chen, Q. Li, and S. Kumar, "Classification of Drosophila embryonic developmental stage range based on gene expression pattern images," in Proc. Comput. Syst. Bioinf. Conf., 2006, pp. 293-298.
34. J. Ye, R. Janardan, C. H. Park, and H. Park, "An optimization criterion for generalized discriminant analysis on undersampled problems," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 8, pp. 982-994, Aug. 2004.
35. J. Ye and T. Xiong, "Computational and theoretical analysis of null space and orthogonal linear discriminant analysis," J. Mach. Learn. Res., vol. 7, pp. 1183-1204, Jul. 2006.
36. W. Zhao, R. Chellappa, and P. Phillips, "Subspace linear discriminant analysis for face recognition," Cntr. Autom. Res., Univ. Maryland, College Park, MD, Tech. Rep. CAR-TR-914, 1999.
37. M. Zhu and A. M. Martinez, "Pruning noisy bases in discriminant analysis," IEEE Trans. Neural Netw., vol. 19, no. 1, pp. 148-157, Jan. 2008.