Improving implementation of linear discriminant analysis for the high dimension/small sample size problem

Computational Statistics and Data Analysis

Volumn 52, Issue 1, 2007, Pages 423-437

  Duintjer Tebbens, Jurjen ^a   Schlesinger, Pavel ^b  

^a INSTITUTE OF COMPUTER SCIENCE   (Czech Republic)

^b CHARLES UNIVERSITY   (Czech Republic)

Author keywords

Dimension reduction; Linear discriminant analysis; Numerical aspects of FLDA; Small sample size problem; Sparsity

Indexed keywords

COMPUTATIONAL METHODS; CONVERGENCE OF NUMERICAL METHODS; FUNCTION EVALUATION; PROBLEM SOLVING; SAMPLING;

DIMENSION REDUCTION; FISHER LINEAR DISCRIMINANT ANALYSIS (FLDA); LINEAR DISCRIMINANT ANALYSIS; SMALL SAMPLE SIZE PROBLEM; SPARSITY;

DISCRIMINANT ANALYSIS;

EID: 34548273742     PISSN: 01679473     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.csda.2007.02.001     Document Type: Article

References (30)

1. Anderson E., Bai Z., Bischof C., Blackford L.S., Demmel J., Dongarra J., Croz J., Du J., Greeenhaum A., Hammarling S., and McKenney A. Lapack Users' Guide (2000), SIAM, Philadelphia
2. Bai, Z., Demmel, J., Dongarra, J., Ruhe, A., van der Vorst, H. (Eds.), 2000. Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide, 2000, SIAM, Philadelphia.
3. Bensmail H., and Celeux G. Regularized Gaussian discriminant analysis through eigenvalue decomposition. J. Amer. Statist. Assoc. 91 (1996) 1743-1748
4. Chen L.-F., Liao H.-Y.M., Ko M.-T., Lin J.-C., and Yu G.-J. A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recognition 33 10 (2000) 1713-1726
5. Cheng Y.-Q., Zhuang Y.-M., and Yang J.-Y. Optimal Fisher discriminant analysis using the rank decomposition. Pattern Recognition 25 1 (1992) 101-111
6. Duda R.O., Hart P.E., and Stork D.G. Pattern Recognition. second ed (2000), Wiley, New York
7. Fisher R.A. The use of multiple measurements in taxonomic problems. Ann. Eugenics 7 (1936) 179-188
8. Friedman J.H. Regularized discriminant analysis. J. Amer. Statist. Assoc. 84 (1989) 165-175
9. Golub G.H., and van Loan C.F. Matrix Computations. third ed (1996), John Hopkins University Press, Baltimore
10. Guo Y.-F., Li S.-J., Yang J.-Y., Shu T.-T., and Wu L.-D. A generalized Foley-Sammon transform based on generalized Fisher discriminant criterion and its application to face recognition. Pattern Recognition Lett. 24 (2003) 147-158
11. Hastie, T., Tibshirani, R., 2003. Expression arrays and the p ≫ n problem. See 〈http://www-stat.stanford.edu/∼hastie/Papers/pgtn.pdf〉.
12. Hoffbeck J.P., and Landgrebe D.A. Covariance matrix estimation and classification with limited training data. IEEE Trans. Pattern Anal. Mach. Intell. 18 7 (1996) 763-767
13. Hong Z.-Q., and Yang J.-Y. Optimal discriminant plane for a small number of samples and design method of classifier on the plane. Pattern Recognition 24 4 (1991) 317-324
14. Howland P., and Park H. Generalizing discriminant analysis using the generalized singular value decomposition. IEEE Trans. Pattern Anal. Mach. Intell. 26 8 (2004) 995-1006
15. Howland P., Jeon M., and Park H. Structure preserving dimension reduction for clustered text data based on the generalized singular value decomposition. SIAM J. Matrix Anal. Appl. 25 1 (2003) 165-179
16. Howland P., Wang J., and Park H. Solving the small sample size problem in face recognition using generalized discriminant analysis. Pattern Recognition 39 (2006) 277-287
17. Johnson R.A., and Wichern D.W. Applied Multivariate Statistical Analysis (1998), Prentice-Hall, Engelwood Cliffs, NJ
18. Kim H., Howland P., and Park P. Dimension reduction in text classification with support vector machines. J. Mach. Learn. Res. 6 (2005) 37-53
19. Krzanowski W.J., Jonathan P., McCarthy W.V., and Thomas M.R. Discriminant analysis with singular covariance matrices: methods and applications to spectroscopic data. Appl. Statist. 44 1 (1995) 101-115
20. Li Y., Kittler J., and Matas J. Effective implementation of linear discriminant analysis for face recognition and verification. In: Leonardis A., and Solina F. (Eds). Lecture Notes in Computer Science 1689 (1999), Springer, Berlin 234-242
21. MathWorks, Inc., 1984-2005. MATLAB 7.0, 〈http://www.mathworks.com/products/matlab/〉.
22. Moler C.B., and Stewart G.W. An algorithm for generalized matrix eigenvalue problems. SIAM J. Numer. Anal. 10 2 (1973) 241-256
23. Paige C.C., and Saunders M.A. Towards a generalized singular value decomposition. SIAM J. Numer. Anal. 18 3 (1981) 398-405
24. Parlett B.N. The Symmetric Eigenvalue Problem (1998), SIAM, Philadelphia
25. R Development Core Team, 2005. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna.
26. Ripley B.D. Pattern Recognition and Neural Networks (1996), Cambridge University Press, Cambridge
27. Tibshirani R., Hastie T., Narasimhan B., and Chu G. Diagnosis of multiple cancer types by shrunken centroids of gene expression. Proc. Nat. Acad. Sci. USA 99 10 (2002) 6567-6572
28. Venables W.N., and Ripley B.D. Modern Applied Statistics with S. fourth ed (2002), Springer, New York
29. Yang J., and Yang J.-Y. Why can LDA be performed in PCA transformed space?. Pattern Recognition 36 2 (2003) 563-566
30. Yang, J., Yu, H., Kunz, W., 2000. An efficient LDA algorithm for face recognition. Sixth International Conference on Control, Automation, Robotics and Vision (ICARCV2000).