Maximum relevance minimum common redundancy feature selection for nonlinear data

Information Sciences

Volumn 409-410, Issue , 2017, Pages 68-86

  Che, Jinxing ^a,b   Yang, Youlong ^a   Li, Li ^b   Bai, Xuying ^a   Zhang, Shenghu ^c   Deng, Chengzhi ^b  

^a XIDIAN UNIVERSITY   (China)

^b NANCHANG INSTITUTE OF TECHNOLOGY   (China)

^c UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

Author keywords

Feature selection; Maximal relevance; Minimal common redundancy; Mutual information; Normalization

Indexed keywords

ECONOMIC AND SOCIAL EFFECTS; FORESTRY; LEARNING SYSTEMS; NEAREST NEIGHBOR SEARCH; NONLINEAR PROGRAMMING; OPTIMIZATION; REDUNDANCY; TREES (MATHEMATICS);

ALGORITHMIC FRAMEWORK; BAYESIAN ADDITIVE REGRESSION TREES; EXPERIMENTAL COMPARISON; MAXIMAL RELEVANCE; MUTUAL INFORMATION FEATURE SELECTIONS; MUTUAL INFORMATIONS; NON-LINEAR OPTIMIZATION; NORMALIZATION;

FEATURE EXTRACTION;

EID: 85019245357     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2017.05.013     Document Type: Article

References (54)

1. Banerjee, M., Pal, N.R., Feature selection with SVD entropy: some modification and extension. Inf. Sci. 264 (2014), 118–134.
2. Battiti, R., Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Netw. 5:4 (1994), 537–550.
3. Bennasar, M., Hicks, Y., Setchi, R., Feature selection using joint mutual information maximisation. Expert Syst. Appl. 42 (2015), 8520–8532.
4. Brown, G., Pocock, A., et al. Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. J. Mach. Learn. Res. 13 (2012), 27–66.
5. Brusco, M.J., A comparison of simulated annealing algorithms for variable selection in principal component analysis and discriminant analysis. Comput. Stat. Data Anal. 77 (2014), 38–53.
6. Chakraborty, D., Pal, N.R., Selecting useful groups of features in a connectionist framework. IEEE Trans. Neural Netw. 19:3 (2008), 381–396.
7. Chakraborty, R., Pal, N.R., Feature selection using a neural framework with controlled redundancy. IEEE Trans. Neural Netw. Learn.Syst. 26:1 (2015), 35–50.
8. Chang, J., Lee, H.K.H., Variable selection via a multi-stage strategy. J. Appl. Stat. 42:4 (2015), 762–774.
9. Che, J.X., Yang, Y.L., Stochastic correlation coefficient ensembles for variable selection. J. Appl. Stat., 2016, 10.1080/02664763.2016.1221913.
10. Chernbumroong, S., Cang, S., Yu, H., Maximum relevancy maximum complementary feature selection for multi-sensor activity recognition. Expert Syst. Appl. 42 (2015), 573–583.
11. Chipman, H.A., George, E.I., McCulloch, R.E., BART: Bayesian additive regression trees. Ann. Appl. Stat. 4 (2010), 266–298.
12. H. Chipman, R. McCulloch, Bayestree: Bayesian methods for tree based models, 2010, R Package Version 0.3-1.1. Available at http://www.cran.r-project.org/web/packages/BayesTree/index.html.
13. Cover, T.M., Thomas, J.A., Elements of Information Theory. 1991, John Wiley and Sons, Inc., New York, NY, USA.
14. Das, A., Kempe, D., Algorithms for subset selection in linear regression. Proceedings of the Fortieth Annual ACM Symposium on Theory of Computing, 2008, 45–54.
15. Ding, F., Hierarchical estimation algorithms for multivariable systems using measurement information. Inf. Sci. 277:2 (2014), 396–405.
16. Ding, F., Wang, F., Xu, L., Hayat, T., Alsaedi, A., Parameter estimation for pseudo-linear systems using the auxiliary model and the decomposition technique. IET Control Theory Appl. 11 (2017), 390–400.
17. Ding, F., Wang, F., Xu, L., Wu, M., Decomposition based least squares iterative identification algorithm for multivariate pseudo-linear ARMA systems using the data filtering. J. Franklin Inst. 354 (2017), 1321–1339.
18. Ding, F., Xu, L., Zhu, Q., Performance analysis of the generalised projection identification for time-varying systems. IET Control Theory Appl. 10 (2016), 2506–2514.
19. Emary, E., Zawbaa, H.M., Hassanien, A.E., Binary grey wolf optimization approaches for feature selection. Neurocomputing 172 (2016), 371–381.
20. Estévez, P.A., Normalized mutual information feature selection. IEEE Trans. Neural Netw. 20:2 (2009), 189–201.
21. Fleuret, F., Fast binary feature selection with conditional mutual information. J. Mach. Learn. Res. 5 (2004), 1531–1555.
22. Freeman, C., Kulić, D., Basir, O., An evaluation of classifier-specific filter measure performance for feature selection. Pattern Recognit. 48 (2015), 1812–1826.
23. García-Torres, M., Gómez-Vela, F., Melián-Batista, B., Moreno-Vega, J.M., High-dimensional feature selection via feature grouping: a variable neighborhood search approach. Inf. Sci. 326 (2016), 102–118.
24. Gramacy, R.B., tgp: An r package for bayesian nonstationary, semiparametric nonlinear regression and design by treed gaussian process models. J. Stat. Softw. 19 (2007), 1–46.
25. Hausser, J., Strimmer, K., Entropy inference and the james-stein estimator, with application to nonlinear gene association networks. J. Mach. Learn. Res. 10 (2009), 1469–1484 See website for package: http://www.strimmerlab.org/software/entropy/.
26. Huang, D., Chow, T.W.S., Effective feature selection scheme using mutual information. Neurocomputing 63 (2005), 325–343.
27. G.H. John, R. Kohavi, K. Pfleger, Irrelevant feature and the subset selection problem, Proceedings of the Eleventh International Conference on Machine Learning1994, 121–129.
28. Kang, R., Zhang, T., Tang, H., Zhao, W., Adaptive region boosting method with biased entropy for path planning in changing environment. CAAI Trans. Intell. Technol. 1 (2016), 179–188.
29. Kwak, N., Kernel discriminant analysis for regression problems. Pattern Recognit. 45 (2012), 2019–2031.
30. Kwak, N., Choi, C.H., Input feature selection for classification problems. IEEE Trans. Neural Netw. 13:1 (2002), 143–159.
31. Koller, D., Sahami, M., Toward optimal feature selection. Proceedings of the Thirteenth International Conference on Machine Learning, 1996, 284–292.
32. Li, W., Jia, Y., Du, J., Distributed filtering for discrete-time linear systems with fading measurements and time-correlated noise. Digital Signal Process. 60 (2017), 211–219.
33. MacKay, D.J.C., Information Theory, Inference, and Learning Algorithms. 2003, Cambridge University Press, Cambridge.
34. Maldonado, S., Carrizosa, E., Weber, R., Kernel penalized k-means: a feature selection method based on kernel k-means. Inf. Sci. 322 (2015), 150–160.
35. Meyer, P.E., Bontempi, G., On the use of variable complementarity for feature selection in cancer classification. Proceedings of European Workshop on Applications of Evolutionary Computing, 2006, Evo Workshops, 91–102.
36. Meyer, P.E., Schretter, C., Bontempi, G., Information-theoretic feature selection in microarray data using variable complementarity. IEEE J. Sel. Top. Signal Process. 2 (2008), 261–274.
37. Mkhadri, A., Ouhourane, M., A group VISA algorithm for variable selection. Stat. Methods Appl. 24 (2015), 41–60.
38. Peng, H., Long, F., Ding, C., Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach.Intell. 27:8 (2005), 1226–1238.
39. Ryuhei, M., Yuichi, T., Mixed integer second-order cone programming formulations for variable selection in linear regression. Eur. J. Oper. Res. 247:3 (2015), 721–731.
40. Setiono, R., Liu, H., Neural network feature selector. IEEE Trans. Neural Netw. 8 (1997), 654–662.
41. Unler, A., Murat, A., Chinnam, R.B., mr2PSO: a maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification. Inf. Sci. 181:20 (2011), 4625–4641.
42. Vidal-Naquet, M., Ullman, S., Object recognition with informative features and linear classification. Proceedings of the 10th IEEE International Conference on Computer Vision, 2003, 281–289.
43. Vinh, L.T., Thang, N.D., Lee, Y.K., An improved maximum relevance and minimum redundancy feature selection algorithm based on normalized mutual information. Tenth International Symposium on Applications and the Internet, 2010, 395–398.
44. Wang, H., Bell, D., Murtagh, F., Axiomatic approach to feature subset selection based on relevance. IEEE Trans. Pattern Anal. Mach.Intell. 21:3 (1999), 271–277.
45. Wang, D., Ding, F., Chu, Y., Data filtering based recursive least squares algorithm for hammerstein systems using the key-term separation principle. Inf. Sci. 222:3 (2013), 203–212.
46. Wang, Z., Li, M., Li, J., A multi-objective evolutionary algorithm for feature selection based on mutual information with a new redundancy measure. Inf. Sci. 307 (2015), 73–88.
47. Wang, D., Ding, F., Parameter estimation algorithms for multivariable hammerstein CARMA systems. Inf. Sci. 355-356 (2016), 237–248.
48. Xu, L., The damping iterative parameter identification method for dynamical systems based on the sine signal measurement. Signal Process. 120 (2016), 660–667.
49. Xu, L., Application of the newton iteration algorithm to the parameter estimation for dynamical systems. J. Comput. Appl. Math. 288 (2015), 33–43.
50. Xu, R.F., Lee, S.J., Dimensionality reduction by feature clustering for regression problems. Inf. Sci. 299 (2015), 42–57.
51. Yang, H., Moody, J., Feature selection based on joint mutual information. Proceedings of International ICSC Symposium on Advances in Intelligent Data Analysis, 1999, 22–25.
52. Yin, S., Zhu, X., Kaynak, O., Improved PLS focused on key performance indictor related fault diagnosis. IEEE Trans. Ind. Electron. 62:3 (2015), 1651–1658.
53. Yin, S., Wang, G., Gao, H., Data-driven process monitoring based on modified orthogonal projections to latent structures. IEEE Trans. Control Syst. Technol. 24:4 (2016), 1480–1487.
54. Yu, L., Liu, H., Efficient feature selection via analysis of relevance and redundancy. J. Mach. Learn. Res. 5 (2004), 1205–1224.