Feature extraction based on direct calculation of mutual information

International Journal of Pattern Recognition and Artificial Intelligence

Volumn 21, Issue 7, 2007, Pages 1213-1231

  Kwak, Nojun ^a  

^a Ajou University   (South Korea)

Author keywords

Classification; Feature extraction; Gradient descent; Mutual information; Optimization; Parzen window; Subspace method

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; INFORMATION ANALYSIS; OPTIMIZATION; PROBABILITY DENSITY FUNCTION; PROBLEM SOLVING;

CLASSIFICATION; GRADIENT DESCENT; MUTUAL INFORMATION; PARZEN WINDOW; SUBSPACE METHODS;

FEATURE EXTRACTION;

EID: 35948966389     PISSN: 02180014     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0218001407005892     Document Type: Article

References (28)

1. D. Addison, S. Wermter and G. Arevian, A comparison of feature extraction and selection techniques, Proc. Int. Conf. Artificial Neural Networks 2003, Istanbul, Turkey (June 2003), pp. 212-215.
2. S. Aeberhard, D. Coomans and O. de Vel, The classification performance of RDA, Technical Report 92-01, Department of Computer Science and Department of Mathematics and Statistics, James Cook University of North Queensland (1992).
3. G. A. Babich and O. I. Camps, Weighted Parzen window for pattern classification, IEEE Trans. Patt. Anal. Mach. Intell. 18(5) (1996) 567-570.
4. P. N. Belhumeur, J. P. Hespanha and D. J. Kriegman, Eigenfaces vs. fisherfaces: recognition using class specific linear projection, IEEE Trans. Patt. Anal. Mach. Intell. 19(7) (1997) 711-720.
5. A. J. Bell and T. J. Sejnowski, An information-maximization approach to blind separation and blind deconvolution, Neural Comput. 7(6) (1995) 1129-1159.
6. K. J. Cios, W. Pedrycz and R. W. Swiniarski, Data Mining Methods for Knowledge-Discovery (Kluwer Academic Publishers, 1998), Chap. 9.
7. T. M. Cover and J. A. Thomas, Elements of Information Theory (John Wiley & Sons, 1991).
8. P. A. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach (Prentice Hall, London, 1982).
9. J. W. Fisher III and J. C. Principe, A methodology for information theoretic feature extraction, Proc. Int Joint Conf. Neural Networks 1998, Anchorage, Alasca (May 1998).
10. K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd edn. (Academic Press, 1990).
11. K. Fukunaga, and R. R. Hayes, The reduced Parzen classifier, IEEE Trans. Patt. Anal. Mach. Intell. 11(4) (1989) 423-425.
12. R. P. Gorman and T. J. Sejnowski, Analysis of hidden units in a layered network trained to classify sonar targets, Neural Networks 1 (1988) 75-89.
13. M. E. Hellman and J. Raviv, Probability of error, equivocation and Chernoff bound, IEEE Trans. Inform. Th. 16 (1970) 368-372.
14. I. T. Joliffe, Principal Component Analysis (Springer-Verlag, 1986).
15. N. Kwak, Feature Selection and Extraction Based on Mutual Information for Classification, Ph.D. thesis, Seoul National University, Seoul, Korea (2003).
16. N. Kwak and C.-H. Choi, Input feature selection by mutual information based on Parzen window, IEEE Trans. Patt. Anal. Mach. Intell. 24(12) (2002) 1667-1671.
17. N. Kwak and C.-H. Choi, A new method of feature extraction and its stability, Proc. Int. Conf. Artificial Neural Networks 2002, Madrid Spain (August 2002), pp. 480-485.
18. N. Kwak, C.-H. Choi and N. Ahuja, Face recognition using feature extraction based on independent component analysis, Proc. Int. Conf. Image Processing 2002, Rochester, NY (September 2002), pp. 337-340.
19. N. Kwak and C.-H. Choi, Feature extraction based on ICA for binary classification problems, IEEE Trans. Knowl. Data Engin. 15(6) (2003) 1374-1388.
20. E. Parzen, On estimation of a probability density function and mode, Ann. Math. Stat. 33 (1962) 1065-1076.
21. R. Quinlan, C4- 5: Programs for Machine Learning (Morgan Kaufmann Publishers, San Mateo, CA, 1993).
22. S. Ruping, MYSVM - a support vector machine, For more information contact http://www-ai.ca.uni-dortmund.de/SOFTWARE/MYSVM/.
23. F. Samaria and A. Harter, Parameterisation of a stochastic model for human face identification, Proc. 2nd IEEE Workshop on Applications of Computer Vision, Sarasota FL (December 1994).
24. C. E. Shannon and W. Weaver, The Mathematical Theory of Communication (University of Illinois Press, Urbana, IL, 1949).
25. M. Turk and A. Pentland, Face recognition using eigenfaces, Proc. IEEE Conf. Computer Vision and Pattern Recognition (1991), pp. 586-591.
26. S. Ullman, M. Vidal-Naquet and E. Sali, Visual features of intermediate complexity and their use in classification, Nature Neurosci. 7 (2002) 682-687.
27. P. Viola and W. M. Wells III, Alignment by maximization of mutual information, Int. J. Comput. Vis. 24(2) (1997) 137-154.
28. M. H. Yang, Kernel eigenface vs. kernel Fisherface: face recognition using kernel methods, Proc. 5th Int. Conf. Automatic Face Gesture Recognition, Washington DC (May 2002), pp. 215-220.