Computational and theoretical analysis of null space and orthogonal linear discriminant analysis

Journal of Machine Learning Research

Volumn 7, Issue , 2006, Pages 1183-1204

  Ye, Jieping ^a   Xiong, Tao ^b  

^a Arizona State University   (United States)

^b University of Minnesota   (United States)

Author keywords

Dimensionality reduction; Linear discriminant analysis; Null space; Orthogonal matrix; Regularization

Indexed keywords

ALGORITHMS; DATA REDUCTION; LEARNING SYSTEMS; LINEAR SYSTEMS; PROBLEM SOLVING; STATISTICAL METHODS; VECTORS;

DIMENSIONALITY REDUCTION; LINEAR DISCRIMINANT ANALYSIS (LDA); NULL SPACE; ORTHOGONAL MATRIX; REGULARIZATION;

COMPUTATION THEORY;

EID: 33745743918     PISSN: 15337928     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (32)

1. P. N. Belhumeour, J. P. Hespanha, and D. J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Trans Pattern Analysis and Machine Intelligence, 19 (7):711-720, 1997.
2. R. Bellmanna. Adaptive Control Processes: A Guided Tour. Princeton University Press, 1961.
3. M. W. Berry, S. T. Dumais, and G. W. O'Brie. Using linear algebra for intelligent information retrieval. SIAM Review, 37:573-595, 1995.
4. L. F. Chen, H. Y. M. Liao, M. T. Ko, J. C. Lin, and G. J. Yu. A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recognition, 33:1713-1726, 2000.
5. S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman. Indexing by latent semantic analysis. Journal of the Society for Information Science, 41:391-407, 1990.
6. L. Duchene and S. Leclerq. An optimal transformation for discriminant and principal component analysis. IEEE Trans. Pattern Analysis and Machine Intelligence, 10(6):978-983, 1988.
7. R. O. Duda, P. E. Hart, and D. Stork. Pattern Classification. Wiley, 2000.
8. S. Dudoit, J. Fridlyand, and T. P. Speed. Comparison of discrimination methods for the classification of tumors using gene expression data. Journal of the American Statistical Association, 97(457): 77-87, 2002.
9. D. H. Foley and J. W. Sammon. An optimal set of discriminant vectors. IEEE Trans Computers, 24 (3):281-289, 1975.
10. J. H. Friedman. Regularized discriminant analysis. Journal of the American Statistical Association, 84(405): 165-175, 1989.
11. K. Fukunaga. Introduction to Statistical Pattern Classification. Academic Press, San Diego, California, USA, 1990.
12. G. H. Golub and C. F. Van Loan. Matrix Computations. The Johns Hopkins University Press, Baltimore, MD, USA, third edition, 1996.
13. P. Hall, J. S. Marron, and A. Neeman. Geometric representation of high dimension, low sample size data. Journal of the Royal Statistical Society series B, 67:427-444, 2005.
14. T. Hastie, R. Tibshirani, and J. H. Friedman. The Elements of Statistical Learning : Data Mining, Inference, and Prediction. Springer, 2001.
15. R. Huang, Q. Liu, H. Lu, and S. Ma. Solving the small sample size problem of LDA. In Proc. International Conference on Pattern Recognition, pages 29-32, 2002.
16. Z. Jin, J. Y. Yang, Z. S. Hu, and Z. Lou. Face recognition based on the uncorrelated discriminant transformation. Pattern Recognition, 34:1405-1416, 2001.
17. I. T. Jolliffe. Principal Component Analysis. Springer-Verlag, New York, 1986.
18. W. J. Krzanowski, P. Jonathan, W.V McCarthy, and M. R. Thomas. Discriminant analysis with singular covariance matrices: methods and applications to spectroscopic data. Applied Statistics, 44:101-115, 1995.
19. T. Li, C. Zhang, and M. Ogihara. A comparative study of feature selection and multiclass classification methods for tissue classification based on gene expression. Bioinformatics, 20(15): 2429-2437, 2004.
20. W. Liu, Y. Wang, S. Z. Li, and T. Tan. Null space approach of Fisher discriminant analysis for face recognition. In Proc. European Conference on Computer Vision, Biometric Authentication Workshop, 2004.
21. J. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos. Face recognition using kernel direct discriminant analysis algorithms. IEEE Trans. Neural Networks, 14(1): 117-126, 2003.
22. S. Raudys and R. P. W. Duin. On expected classification error of the Fisher linear classifier with pseudo-inverse covariance matrix. Pattern Recognition Letters, 19(5-6):385-392, 1998.
23. B. Schökopf and A. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond. MIT Press, 2002.
24. M. Skurichina and R. P. W. Duin. Stabilizing classifiers for very small sample size. In Proc. International Conference on Pattern Recognition, pages 891-896, 1996.
25. D. L. Swets and J. Y. Weng. Using discriminant eigenfeatures for image retrieval. IEEE Trans. Pattern Analysis and Machine Intelligence, 18(8):831-836, 1996.
26. M. A. Turk and A. P. Pentland. Face recognition using Eigenfaces. In Proc. Computer Vision and Pattern Recognition Conference, pages 586-591, 1991.
27. V. N. Vapnik. Statistical Learning Theory. Wiley, New York, 1998.
28. J. Yang, A. F. Frangi, J. Y Yang, D. Zhang, and Z. Jin. KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition. IEEE Trans. Pattern Analysis and Machine Intelligence, 27(2):230-244, 2005.
29. J. Ye. Characterization of a family of algorithms for generalized discriminant analysis on under-sampled problems. Journal of Machine Learning Research, 6:483-502, 2005.
30. J. Ye, R. Janardan, Q. Li, and H. Park. Feature extraction via generalized uncorrelated linear discriminant analysis. In Proc. International Conference on Machine Learning, pages 895-902, 2004a.
31. J. Ye, T. Li, T. Xiong, and R. Janardan. Using uncorrelated discriminant analysis for tissue classification with gene expression data. IEEE/ACM Trans. Computational Biology and Bioinformatics, 1(4): 181-190, 2004b.
32. H. Yu and J. Yang. A direct LDA algorithm for high-dimensional data with applications to face recognition. Pattern Recognition, 34:2067-2070, 2001.