A new perspective to null linear discriminant analysis method and its fast implementation using random matrix multiplication with scatter matrices

Pattern Recognition

Volumn 45, Issue 6, 2012, Pages 2205-2213

  Sharma, Alok ^a,b,c   Paliwal, Kuldip K ^a  

^a GRIFFITH UNIVERSITY   (Australia)

^b UNIVERSITY OF TOKYO   (Japan)

^c UNIVERSITY OF THE SOUTH PACIFIC   (Fiji)

Author keywords

Dimensionality Reduction; Null LDA; Small Sample Size Problem

Indexed keywords

DISCRIMINANT ANALYSIS; EIGENVALUES AND EIGENFUNCTIONS;

COMPUTATIONALLY EFFICIENT; DIMENSIONALITY REDUCTION; DIMENSIONALITY REDUCTION METHOD; EIGENVALUE DECOMPOSITION; LINEAR DISCRIMINANT ANALYSES (LDA); LINEAR DISCRIMINANT ANALYSIS; NULL LDA; SMALL SAMPLE SIZE PROBLEMS;

MATRIX ALGEBRA;

EID: 84857037026     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2011.11.018     Document Type: Article

References (40)

1. S.A. Armstrong, J.E. Staunton, L.B. Silverman, R. Pieters, M.L. den Boer, M.D. Minden, S.E. Sallan, E.S. Lander, T.R. Golub, and S.J. Korsemeyer MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia Nature Genetics 30 2002 41 47 [Data Source1:], [Data Source2: ] 〈http://sdmc.lit.org.sg/GEDatasets/Datasets.html〉 〈http://www. broad.mit.edu/cgi-bin/cancer/publications/pub-paper.cgi?mode=view&paper-id= 63〉
2. P.N. Belhumeur, J.P. Hespanha, and D.J. Kriegman Eigenfaces vs. fisherfaces: recognition using class specific linear projection IEEE Transactions on Pattern Analysis and Machine Intelligence 19 7 1997 711 720 (Pubitemid 127762263)
3. C.L. Blake and C.J. Merz, UCI repository of machine learning databases, 〈 http://www.ics.uci.edu/∼mlearn 〉, Irvine, CA, University of Calif., Dept. of Information and Comp. Sci., 1998.
4. H. Cevikalp, M. Neamtu, M. Wilkes, and A. Barkana Discriminative common vectors for face recognition IEEE Transactions on Pattern Analysis and Machine Intelligence 27 1 2005 4 13 (Pubitemid 40195893)
5. L.-F. Chen, H.-Y.M. Liao, M.-T. Ko, J.-C. Lin, and G.-J. Yu A new LDA-based face recognition system which can solve the small sample size problem Pattern Recognition 33 2000 1713 1726
6. D. Chu, and G.S. Thye A new and fast implementation for null space based linear discriminant analysis Pattern Recognition 43 2010 1373 1379
7. D. Coppersmith, and S. Winograd Matrix multiplication via arithmetic progressions Journal of Symbolic Computation 9 3 1990 251 280
8. R.O. Duda, and P.E. Hart Pattern Classification and Scene Analysis 1973 Wiley New York
9. J.H. Friedman Regularized discriminant analysis Journal of the American Statistical Association 84 1989 165 175
10. K. Fukunaga Introduction to Statistical Pattern Recognition 1990 Academic Press Inc., Hartcourt Brace Jovanovich, Publishers, San Diego, CA 92101-4495, USA
11. G.H. Golub, and C.F.V. Loan Matrix Computations 1996 The John Hopkins University Press, Baltimore, Maryland 21218-4363, USA
12. Y. Guo, T. Hastie, and R. Tinshirani Regularized discriminant analysis and its application in microarrays Biostatistics 8 1 2007 86 100 (Pubitemid 44906104)
13. O.C. Hamsici, and A.M. Martinez Bayes optimality in linear discriminant analysis IEEE Transactions on Pattern Analysis and Machine Intelligence 30 2008 647 657 (Pubitemid 351559718)
14. P. Howland, and H. Park Generalizing discriminant analysis using the generalized singular value decomposition IEEE Transactions on Pattern Analysis and Machine Intelligence 26 2004 995 1006
15. R. Huang, Q. Liu, H. Lu, and S. Ma Solving the small sample size problem of LDA Proceedings of ICPR 3 2002 29 32 2002 (Pubitemid 44844422)
16. X. Jiang, B. Mandal, and A. Kot Eigenfeature regularization and extraction in face recognition IEEE Transactions on Pattern Analysis and Machine Intelligence 30 3 2008 383 394 (Pubitemid 351280047)
17. X. Jiang Asymmetric principal component analysis and discriminant analysis for pattern classification IEEE Transactions on Pattern Analysis and Machine Intelligence 31 5 2009 931 937
18. X. Jiang Linear subspace learning-based dimensionality reduction IEEE Signal Processing Magazine 28 2 2011 16 26
19. J. Khan, J.S. Wei, M. Ringner, L.H. Saal, M. Ladanyi, F. Westermann, F. Berthold, M. Schwab, C.R. Antonescu, C. Peterson, and P.S. Meltzer Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural network Nature Medicine 7 2001 673 679 [Data Source: 〈http://research.nhgri.nih.gov/microarray/Supplement/〉 ] (Pubitemid 32588022)
20. Z. Jin, J.Y. Yang, Z.M. Tang, and Z.S. Hu A theorem on the uncorrelated optimal discriminant vectors Pattern Recognition 24 10 2001 2041 2047 (Pubitemid 32760633)
21. W.J. Krzanowski, P. Jonathan, W.V. McCarthy, and M.R. Thomas Discriminant analysis with singular covariance matrices: methods and applications to spectroscopic data Applied Statistics 44 1995 101 115
22. J. Liu, S. Chen, X. Tan, and D. Zhang Efficient pseudoinverse linear discriminant analysis and its non linear form for face recognition International Journal of Pattern Recognition and Artificial Intelligence 21 2007 1265 1278
23. W. Liu, Y. Wang, S.Z. Li and T. Tan, Null Space Approach of Fisher Discriminant Analysis for Face Recognition, ECCV Biometric Authentication Workshop, Prague, Czech, 2004.
24. R. Lotlikar, and R. Kothari Fractional-step dimensionality reduction IEEE TransPattern Analysis and Machine Intelligence 22 6 2000 623 627
25. J. Lu, K.N. Plataniotis, and A.N. Venetsanopoulos Face recognition using LDA-based algorithms IEEE Transactions on Neural Networks. 14 1 2003 195 200
26. A.M. Martinez Recognizing imprecisely localized, partially occluded, and expression variant faces from a single sample per class IEEE Transactions on Pattern Analysis and Machine Intelligence 24 6 2002 748 763 (Pubitemid 34669589)
27. C.H. Park, and H. Park A comparison of generalized linear discriminant analysis algorithms Pattern Recognition 41 2008 1083 1097 (Pubitemid 47632680)
28. S. Ramaswamy, P. Tamayo, R. Rifkin, S. Mukherjee, C.-H. Yeang, M. Angelo, C. Ladd, M. Reich, E. Latulippe, J.P. Mesirov, T. Poggio, W. Gerald, M. Loda, E.S. Lander, and T.R. Golub Multiclass cancer diagnosis using tumor gene expression signatures Proceedings of the National Academy of Sciences of the USA 98 26 2001 15149 15154 (Pubitemid 34013984)
29. S. Raudys, and R.P.W. Duin On expected classification error of the Fisher linear classifier with pseudo-inverse covariance matrix Pattern Recognition Letters 19 56 1998 385 392 (Pubitemid 128369345)
30. F. Samaria and A. Harter, Parameterization of a stochastic model for human face identification, Proc. Second IEEE Workshop Applications of Comp. Vision, pp. 138142, 1994.
31. A. Sharma, and K.K. Paliwal Cancer classification by gradient LDA technique using microarray gene expression data Data & Knowledge Engineering 66 2008 338 347
32. A. Sharma, and K.K. Paliwal A gradient linear discriminant analysis for small sample sized problem Neural Processing Letters 27 2008 17 24
33. D. Singh, P.G. Febbo, K. Ross, D.G. Jackson, J. Manola, C. Ladd, P. Tamayo, A.A. Renshaw, A.V. DAmico, J.P. Richie, E.S. Lander, M. Loda, P.W. Kantoff, T.R. Golub, and W.R. Sellers Gene expression correlates of clinical prostate cancer behavior Cancer Cell 1 2002 203 209 [Data Source:] 〈http://sdmc.lit.org.sg/GEDatasets/Datasets.html#Prostate〉 (Pubitemid 41039166)
34. D.L. Swets, and J. Weng Using discriminative eigenfeatures for image retrieval IEEE Transactions on Pattern Analysis and Machine Intelligence 18 8 1996 831 836 (Pubitemid 126772824)
35. Q. Tian, M. Barbero, Z.H. Gu, and S.H. Lee Image classification by the Foley-Sammon transform Optical Engineering 25 7 1986 834 840
36. J. Yang, D. Zhang, and J.-Y. Yang A generased KL expansion method which can deal with small samples size and high-dimensional problems Pattern Analysis and Applications 6 2003 47 54 (Pubitemid 36719924)
37. J. Ye Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems Journal of Machine Learning Research 6 2005 483 502
38. J. Ye, and T. Xiong Computational and theoretical analysis of null space and orthogonal linear discriminant analysis Journal of Machine Learning Research 7 2006 1183 1204 (Pubitemid 44015300)
39. E.J. Yeoh, M.E. Ross, S.A. Shurtleff, W.K. Williams, D. Patel, R. Mahfouz, F.G. Behm, S.C. Raimondi, M.V. Relling, A. Patel, C. Cheng, D. Campana, D. Wilkins, X. Zhou, J. Li, H. Liu, C.H. Pui, W.E. Evans, C. Naeve, L. Wong, and J.R. Downing Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling Cancer 1 2 2002 133 143 Data Source: 〈http://www.stjuderesearch.org/data/ALL1/〉 ] (Pubitemid 41039160)
40. H. Yu, and J. Yang A direct LDA algorithm for high-dimensional data-with application to face recognition Pattern Recognition 34 2001 2067 2070