A relationship between linear discriminant analysis and the generalized minimum squared error solution

SIAM Journal on Matrix Analysis and Applications

Volumn 27, Issue 2, 2006, Pages 474-492

  Park, Cheong Hee ^a   Park, Haesun ^b,c  

^a Chungnam National University   (South Korea)

^b University of Minnesota   (United States)

^c NATIONAL SCIENCE FOUNDATION   (United States)

Author keywords

Dimension reduction method; Generalized singular value decomposition; Linear discriminant analysis; Minimum squared error solution; Pseudoinverse; Undersampled problems

Indexed keywords

DIMENSION REDUCTION METHOD; GENERALIZED SINGULAR VALUE DECOMPOSITION; LINEAR DISCRIMINANT ANALYSIS; MINIMUM SQUARED ERROR SOLUTION; PSEUDOINVERSE; UNDERSAMPLED PROBLEMS;

ALGORITHMS; MATRIX ALGEBRA; PROBLEM SOLVING;

ERROR ANALYSIS;

EID: 33646267415     PISSN: 08954798     EISSN: 10957162     Source Type: Journal    
DOI: 10.1137/040607599     Document Type: Article

References (22)

1. R. A. FISHER, The use of multiple measurements in taxonomic problems, Annu. Eugenics, 7 (Part II) (1936), pp. 179-188.
2. F. ROSENBLATT, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, Spartan Books, Washington, D.C., 1962.
3. B. E. BOSER, I. M. GUYON, AND V. VAPNIK, A training algorithm for optimal margin classifiers, in Proceedings of the Fifth Annual Workshop on Computational Learning Theory (Pittsburgh, PA), ACM, New York, 1992.
4. R. O. DUDA AND P. E. HART, Pattern Classification and Scene Analysis, Wiley-Interscience, New York, 1973.
5. Y-C. HO AND R. L. KASHYAP, An algorithm for linear inequalities and its applications, IEEE Trans. Elec. Comp., 14 (1965), pp. 683-688.
6. R. O. DUDA, P. E. HART, AND D. G. STORK, Pattern Classification, Wiley-Interscience, New York, 2001.
7. K. FUKUNAGA, Introduction to Statistical Pattern Recognition, 2nd ed., Academic Press, New York, 1990.
8. J. S. KOFORD AND G. F.GRONER, The use of an adaptive threshold element to design a linear optimal pattern classifier, IEEE Trans. Inf. Theory, 12 (1966), pp. 42-50.
9. T. HASTIE, R. TIBSHIRANI, AND A. BUJA, Flexible discriminant analysis by optimal scoring, J. Amer. Statist. Assoc., 89 (1994), pp. 1255-1270.
10. T. HASTIE AND R. TIBSHIRANI, Discriminant analysis by Gaussian mixture, J. Roy. Statist. Soc. Ser. B, 58 (1996), pp. 155-176.
11. N. CAMPBELL, Canonical variate analysis - A general formulation, Austral. J. Statist., 26 (1984), pp. 86-96.
12. J. H. FRIEDMAN, Regularized discriminant analysis, J. Amer. Statist. Assoc., 84 (1989), pp. 165-175.
13. T. HASTIE, A. BUJA, AND R. TIBSHIRANI, Penalized discriminant analysis, Ann. Statist., 23 (1995), pp. 73-102.
14. L. CHEN, H. M. LIAO, M. KO, J. LIN, AND G. YU, A new LDA-based face recognition system which can solve the small sample size problem, Pattern Recogn., 33 (2000), pp. 1713-1726.
15. H. YU AND J. YANG, A direct LDA algorithm for high-dimensional data with application to face recognition, Pattern Recogn., 34 (2001), pp. 2067-2070.
16. X. WANG AND X. TANG, Dual-space linear discriminant analysis for face recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Washington, D.C., 2004.
17. J. YANG, J.-Y. YANG, AND D. ZHANG, What's wrong with Fisher criterion?, Pattern Recogn., 35 (2002), pp. 2665-2668.
18. P. HOWLAND, M. JEON, AND H. PARK, Structure preserving dimension reduction for clustered text data based on the generalized singular value decomposition, SIAM J. Matrix Anal. Appl., 25 (2003), pp. 165-179.
19. P. HOWLAND AND H. PARK, Generalizing discriminant analysis using the generalized singular value decomposition, IEEE Trans. Pattern Anal. Machine Intel., 26 (2004), pp. 995-1006.
20. C. C. PAIGE AND M. A. SAUNDERS, Towards a generalized singular value decomposition, SIAM J. Numer. Anal., 18 (1981), pp. 398-405.
21. G. H. GOLUB AND C. F. VAN LOAN, Matrix Computations, 3rd ed., Johns Hopkins University Press, Baltimore, MD, 1996.
22. T. G. KOLDA AND D. P. O'LEARY, A semidiscrete matrix decomposition for latent semantic indexing in information retrieval, ACM Trans. Inf. Syst., 16 (1998), pp. 322-346.