Estimating mutual information using gaussian mixture model for feature ranking and selection

IEEE International Conference on Neural Networks - Conference Proceedings

Volumn , Issue , 2006, Pages 5034-5039

  Lan, Tian ^a   Erdogmus, Deniz ^a   Ozertem, Umut ^a   Huang, Yonghong ^a  

^a OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL EFFICIENCY; FEATURE EXTRACTION; PROBABILITY DENSITY FUNCTION;

GAUSSIAN MIXTURE MODEL (GMM); MUTUAL INFORMATION (MI); SELECTION STRATEGIES;

GAUSSIAN DISTRIBUTION;

EID: 40649100303     PISSN: 10987576     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. E. Oja, Subspace Methods of Pattern Recognition, Wiley, New York, 1983.
2. P.A. Devijver, J. Kittler, Pattern Recognition: A Statistical Approach, Prentice Hall, London, 1982.
3. nd ed., Academic Press, New York, 1990.
4. R. Everson, S. Roberts, "Independent Component Analysis: A Flexible Nonlinearity and Decorrelating Manifold Approach", Neural Computation, vol. 11, no. 8, pp. 1957-1983, 2003.
5. A. Hyvärinen, E. Oja, P. Hoyer, J. Hurri, "Image Feature Extraction by Sparse coding and Independent Component Analysis",Proceedings of ICPR'98,pp. 1268-1273, 1998.
6. T. Lan, D. Erdogmus, A. Adami, M. Pavel, "Feature Selection by Independent Component Analysis and Mutual Information Maximization in EEG Signal Classification," Proceedings of IJCNN'05, Montreal, Canada, pp. 3011-3016, Aug. 2005.
7. W. Duch, T. Wieczorek, J. Biesiada, M. Blachnik, "Comparison of feature ranking methods based on information entropy", Proc. of International Joint Conference on Neural Networks (IJCNN), Budapest 2004, IEEE Press, pp. 1415-1420.
8. R. M. Fano, Transmission of Information: A Statistical Theory of Communications. Wiley, New York, 1961.
9. M.E. Hellman, J. Raviv, "Probability of Error, Equivocation and the Chernoff Bound," IEEE Transactions on Information Theory, vol. 16, pp. 368-372, 1970.
10. Battiti, R., "Using Mutual Information for Selecting Features in Supervised Neural Net Training," IEEE Trans Neural Networks, vol 5, no 4, pp 537-550. July 1994.
11. Kira, K. and Rendell,L., "The feature selection problem: Traditional methods and a new algorithm," In Proceedings of the Tenth National Conference on Artificial Intelligence (AAAI-92), pages 129-134, Menlo Park, CA, USA, 1992. AAAI Press.
12. John,G.H., Kohavi,R., & Pfleger,K., "Irrelevant features and the subset selection problem". In Proceedings of the 11th International Conference on Machine Learning, pp 121-129, San Mateo, CA, Morgan Kaufmann, 1994.
13. Guyon,I., and Elisseeff, A., "An Introduction to Variable and Feature Selection," Journal of Machine Learning Research (Special Issue on Variable and Feature Selection), 2003.
14. T. Lan, D. Erdogmus, A. Adami, M. Pavel, S. Mathan, "Salient EEG Channel Selection in Brain Computer Interfaces by Mutual Information Maximization," Proceedings of EMBC'05, Shanghai, China, Sept. 2005.
15. D. Erdogmus, Information Theoretic Learning: Renyi's Entropy and its applications to Adaptive System Training, PhD Dissertation, University of Florida, 2002.
16. D. Erdogmus, J.C. Principe, "Lower and Upper Bounds for Misclassification Probability Based on Renyi's Information," Journal of VLSI Signal Processing Systems, vol. 37, no. 2/3, pp. 305-317, 2004.
17. K. Torkkola, "Feature Extraction by Non-Parametric Mutual Information Maximization," Journal of Machine Learning Research, vol. 3, pp. 1415-1438, 2003.
18. R. Battiti, "Using Mutual Information for Selecting Features in Supervised Neural Networks learning," IEEE Trans. Neural Networks, vol. 5, no. 4, pp. 537-550, 1994.
19. A.P. Dempster, N.M. Laird, D.B. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," Journal of the Royal Statistical Society, vol. 39, pp. 1-38, 1977.
20. http://www.ics.uci.edu/mlearn/MLRepository.html
21. T. Lan, A. Adami, D. Erdogmus, M. Pavel, "Estimating Cognitive State Using EEG Signals," Proceedings of EUSIPCO'05, Sep 2005.