Approximate information discriminant analysis: A computationally simple heteroscedastic feature extraction technique

Pattern Recognition

Volumn 41, Issue 5, 2008, Pages 1548-1557

  Das, Koel ^a   Nenadic, Zoran ^a  

^a Irvine   (United States)

Author keywords

Bayes error; Classification; Entropy; Feature extraction; Information theory; Linear discriminant analysis; Mutual information

Indexed keywords

DISCRIMINANT ANALYSIS; ERROR ANALYSIS; FEATURE EXTRACTION; IMAGE CLASSIFICATION; LINEAR CONTROL SYSTEMS;

BAYES ERROR; LINEAR DISCRIMINANT ANALYSIS; MUTUAL INFORMATION;

INFORMATION THEORY;

EID: 38349174498     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2007.10.001     Document Type: Article

References (33)

1. Rao C.R. The utilization of multiple measurements in problems of biological classification. J. R. Stat. Soc. B Met. 10 2 (1948) 159-203
2. Fisher R.A. The use of multiple measurements in taxonomic problems. Ann. Eugen. 7 (1936) 179-188
3. Kumar N., and Andreou A.G. Heteroscedastic discriminant analysis and reduced rank hmms for improved speech recognition. Speech Commun. 26 4 (1998) 283-297
4. K. Torkkola, Discriminative features for document classification, in: Proceedings of the 16th International Conference on Pattern Recognition, vol. 1, 2002, pp. 472-475.
5. Chen L.-F., Liao H.-Y.M., M-T. K., Lin J.-C., and Yu G.-J. A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recognition 33 10 (2000) 1713-1726
6. McLachlan G.J. Discriminant Analysis and Statistical Pattern Recognition (1992), Wiley, New York
7. Devijver P.A., and Kittler J. Pattern Recognition: A Statistical Approach (1982), Prentice-Hall, Englewood Cliffs, NJ
8. H.P. Decell, J.A. Quirein, An iterative approach to the feature selection problem, in: Proceedings of Purdue Conference on Machine Processing of Remotely Sensed Data, 1972, pp. 3B1-3B12.
9. G. Saon, M. Padmanabhan, Minimum Bayes error feature selection for continuous speech recognition, in: NIPS, 2000, pp. 800-806.
10. Nenadic Z. Information discriminant analysis: feature extraction with and information theoretic objective. IEEE Trans. Pattern Anal. 29 8 (2007) 1394-1407
11. Padmanabhan M., and Dharanipragada S. Maximizing information content in feature extraction. IEEE Trans. Speech and Audio Process. (2005) 512-519
12. Hild K.E., Erdogmus D., Torkkola K., and Principe J.C. Feature extraction using information-theoretic learning. IEEE Trans. Pattern Anal. 28 9 (2006) 1385-1392
13. Loog M., and Duin R.P.W. Linear dimensionality reduction via a heteroscedastic extension of LDA: the Chernoff criterion. IEEE Trans. Pattern Anal. 26 (2004) 732-739
14. Tubbs J.D., Coberly W.A., and Young D.M. Linear dimension reduction and Bayes classification with unknown population parameters. Pattern Recognition 15 3 (1982) 167-172
15. Brunzell H., and Eriksson J. Feature reduction for classification of multidimensional data. Pattern Recognition 33 10 (2000) 1741-1748
16. Duda R.O., Hart P.E., and Stork D.G. Pattern Classification (2001), Wiley, New York
17. Zhu M., and Martinez A.M. Where are linear feature extraction methods applicable?. IEEE Trans. Pattern Anal. 27 12 (2005) 1934-1944
18. Zhu M., and Martinez A.M. Subclass discriminant analysis. IEEE Trans. Pattern Anal. 28 8 (2006) 1274-1286
19. Paredes R. Learning weighted metrics to minimize nearest-neighbor classification error. IEEE Trans. Pattern Anal. 28 7 (2006) 1100-1110
20. Hellman M.E., and Raviv J. Probability of error, equivocation, and the Chernoff bound. IEEE Trans. Inf. Theory IT-16 4 (1970) 368-372
21. Battiti R. Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Networks 5 4 (1994) 537-550
22. Principe J.C., Fisher III J.W., and Xu D. Information theoretic learning. In: Haykin S. (Ed). Unsupervised Adaptive Filtering (2000), Wiley, New York
23. Torkkola K. Feature extraction by non-parametric mutual information maximization. J. Mach. Learn. Res. 3 (2003) 1415-1438
24. He X., Yan S., Hu Y., and Niyogi P. Face recognition using laplacianfaces. IEEE Trans. Pattern Anal. 27 3 (2005) 328-340
25. S. Hettich, C.L. Blake, C.J. Merz, UCI repository of machine learning databases, 1998, University of California, Irvine, Department of Information and Computer Sciences, 〈http://www.ics.uci.edu/∼mlearn/MLRepository.html〉.
26. Nenadic Z., Rizzuto D.S., Andersen R.A., and Burdick J.W. Advances in cognitive neural prosthesis: Recognition of neural data with an information-theoretic objective. In: Dornhege G., Millan J.R., Hinterberger T., McFarland D.J., and Muller K.-R. (Eds). Toward Brain Computer Interfacing (2007), MIT Press, Cambridge, MA 175-190
27. R. Kohavi, A study of cross-validation and bootstrap for accuracy estimation and model selection, in: International Joint Conference on Artificial Intelligence, 1995, pp. 1137-1145.
28. Fukunaga K. Introduction to Statistical Pattern Recognition. second ed. (1990), Academic Press, New York
29. Schäfer J., and Strimmer K. A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Stat. Appl. Genet. Mol. Biol. 4 1 (2005) Article 32
30. Yu H., and Yang H. A direct lda algorithm for high-dimensional data-with application to face recognition. Pattern Recogn. Lett. 34 10 (2001) 2067-2070
31. X. Wang, X. Tang, Dual-space linear discriminant analysis for face recognition, in: 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'04), vol. 02, 2004, pp. 564-569.
32. Anton H. Elementary Linear Algebra. sixth ed. (1991), Wiley, New York
33. Noble B., and Daniel J.W. Applied Linear Algebra. second ed. (1977), Prentice-Hall, Englewood Cliffs, NJ