A method for combining mutual information and canonical correlation analysis: Predictive Mutual Information and its use in feature selection

Expert Systems with Applications

Volumn 39, Issue 3, 2012, Pages 3333-3344

  Okan Sakar, C ^a   Kursun, Olcay ^b  

^a BAHCESEHIR UNIVERSITY   (Turkey)

^b ISTANBUL UNIVERSITY   (Turkey)

Author keywords

Canonical correlation; Gebelein's Maximal Correlation; Imbalanced datasets; Mutual information; Statistical dependence; Suspicious coincidences

Indexed keywords

CANONICAL CORRELATIONS; IMBALANCED DATA-SETS; MAXIMAL CORRELATION; MUTUAL INFORMATIONS; STATISTICAL DEPENDENCE; SUSPICIOUS COINCIDENCES;

ARTIFICIAL INTELLIGENCE; CORRELATION METHODS;

FEATURE EXTRACTION;

EID: 80255131236     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2011.09.020     Document Type: Article

References (31)

1. Akaho, S. (2001). A kernel method for canonical correlation analysis. International Meeting of Psychometric Society (IMPS2001).
2. A. Asuncion, and D.J. Newman UCI machine learning repository 2007 University of California, Department of Information and Computer Science Irvine, CA
3. F.R. Bach, and M.I. Jordan Kernel independent component analysis The Journal of Machine Learning Research 3 2003 1 48
4. S. Becker Mutual information maximization: Models of cortical self-organization Network 7 1 1996 7 31
5. C.B. Bell Mutual information and maximal correlation as measures of dependence Annals of Mathematical Statistics 33 1962 587 595
6. C.M. Bishop Neural networks for pattern recognition 1995 Oxford University Press Oxford
7. L. Breiman, and J.H. Friedman Estimating optimal transformations for multiple regression and correlation Journal of the American Statistical Association 80 391 1985 580 598
8. W. Bryc, A. Dembo, and A. Kagan On the maximum correlation coefficient Theory of Probability and Its Applications 49 2005 132 138 (Pubitemid 40554420)
9. C.J.C. Burges A tutorial on support vector machines for pattern recognition Knowledge Discovery and Data Mining 2 1998 1 43
10. D. Endres, and P. Földiák Bayesian bin distribution inference and mutual information IEEE Transactions on Information Theory 51 11 2005 3766 3779 (Pubitemid 41622579)
11. O. Favorov, and D. Ryder SINBAD: A neocortical mechanism for discovering environmental variables and regularities hidden in sensory input Biological Cybernetics 90 2004 191 202 (Pubitemid 40877328)
12. H. Gebelein Das statistische problem der correlation also variations und Eigenwert problem Zeitschrift fur Angewandte Mathematik und Mechanik 21 1941 364 379
13. Gurrera, M. D. C. (2005). Construction of bivariate distributions and statistical dependence operations. Ph.D. dissertation, University of Barcelona, Spain.
14. Guvenir, H. A.; Acar, B.; Demiroz, G.; & Cekin, A. (1997). A supervised machine learning algorithm for arrhythmia analysis. In Proceedings of the computers in cardiology conference. Lund, Sweden.
15. D.R. Hardoon, S. Szedmak, and J. Shawe-Taylor Canonical correlation analysis: An overview with application to learning methods Neural Computation 16 2004 2639 2664 (Pubitemid 39604012)
16. H. Hotelling Relations between two sets of variates Biometrika 28 1936 321 377
17. C.W. Hsu, and C.J. Lin A comparison of methods for multi-class support vector machines IEEE Transactions on Neural Networks 13 2002 415 425
18. N. Japkowicz, and S. Stephen The class imbalance problem: A systematic study Intelligent Data Analysis 6 5 2002 429 450
19. O. Kursun, and O. Favorov SINBAD automation of scientific discovery: From factor analysis to theory synthesis Natural Computing 3 2 2004 207 233 (Pubitemid 39127654)
20. O. Kursun, and O. Favorov Feature selection and extraction using an unsupervised biologically-suggested approximation to Gebelein's maximal correlation International Journal of Pattern Recognition and Artificial Intelligence 24 3 2010 337 358
21. N. Kwak, and C.H. Choi Input feature selection by mutual information based on Parzen window IEEE Transactions on Pattern Analysis and Machine Intelligence 24 2002 1667 1671
22. E. Mjolsness, and D. DeCoste Machine learning for science: State of the art and future prospects Science 293 2001 2051 2055 (Pubitemid 32848047)
23. S.Y. Novak On Gebelein's correlation coefficient Statistics and Probability Letters 69 2004 299 303 (Pubitemid 39173801)
24. H. Peng, F. Long, and C. Ding Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy IEEE Transactions on Pattern Analysis and Machine Intelligence 27 2005 1226 1238 (Pubitemid 41245053)
25. A. Renyi On measures of dependence Acta Mathematica Academiae Scieniarum Hungaricae 10 1959 441 451
26. Sakar, C. O.; & Kursun, O. (2010). A hybrid method for feature selection based on mutual information and canonical correlation analysis. 20th international conference on pattern recognition. Istanbul, Turkey.
27. B. Scholkopf, and A. Smola Learning with kernels 2002 MIT Press Cambridge, MA
28. C.E. Shannon A mathematical theory of communication Bell System Technical Journal 27 1948 379 423
29. Tavallaee, M.; Bagheri, E.; Lu, W.; & Ghorbani, A. A. (2009). A detailed analysis of the KDD CUP 99 data set. In Proceedings of the second IEEE international conference on computational intelligence for security and defense applications (Ottawa, Ontario, Canada, July 08-10, 2009) (pp. 53-58). IEEE Press, Piscataway, NJ.
30. V.N. Vapnik The nature of statistical learning theory 1995 Springer-Verlag New York
31. Z. Zhou, and X. Liu Training cost-sensitive neural networks with methods addressing the class imbalance problem IEEE Transactions on Knowledge and Data Engineering 18 1 2006 63 77 (Pubitemid 43145089)