Stochastic correlation coefficient ensembles for variable selection

Journal of Applied Statistics

Volumn 44, Issue 10, 2017, Pages 1721-1742

  Che, JinXing ^a   Yang, YouLong ^a  

^a XIDIAN UNIVERSITY   (China)

Author keywords

LASSO; maximal relevance; minimal redundancy; ranking; stochastic correlation coefficient ensemble

Indexed keywords

EID: 84982162233     PISSN: 02664763     EISSN: 13600532     Source Type: Journal    
DOI: 10.1080/02664763.2016.1221913     Document Type: Article

References (48)

1. R., Adhikari, A mutual association based nonlinear ensemble mechanism for time series forecasting, Appl. Intell. 43 (2015), pp. 233–250. doi:10.1007/s10489-014-0641-y
2. R., Battiti, Using mutual information for selecting features in supervised neural net learning, IEEE Transact Neural Netw. 5 (1994), pp. 537–550. doi:10.1109/72.298224
3. M., Branicki and A.J., Majda, An information-theoretic framework for improving imperfect dynamical predictions via multi-model ensemble forecasts, J. Nonlinear Sci. 25 (2015), pp. 489–538. doi:10.1007/s00332-015-9233-1
4. L., Breiman, Random forests, Mach. Learn. 45 (2001), pp. 5–32. doi:10.1023/A:1010933404324
5. M.J., Brusco, A comparison of simulated annealing algorithms for variable selection in principal component analysis and discriminant analysis, Comput. Stat. Data Anal. 77 (2014), pp. 38–53. doi:10.1016/j.csda.2014.03.001
6. R., Chakraborty and N.R., Pal, Feature selection using a neural framework with controlled redundancy, IEEE Trans. Neural Netw. Learn. Syst. 26 (2015), pp. 35–50. doi:10.1109/TNNLS.2014.2308902
7. P., Cortez, A., Cerdeira, F., Almeida, T., Matos and J., Reis, Modeling wine preferences by data mining from physicochemical properties, Decis. Support Syst. 47 (2009), pp. 547–553. doi:10.1016/j.dss.2009.05.016
8. T.M., Cover and J.A., Thomas, Elements of Information Theory, 1st ed., Wiley, New York, 1991.
9. C., Ding and H.C., Peng, Minimum redundancy feature selection from microarray gene expression data, Proc. Second IEEE Comput. Syst. Bioinf. Conf. (2003), pp. 523–528.
10. B., Efron, T., Hastie, I., Hohnstone and R., Tibshirani, Least angle regression, Ann. Stat. 32 (2004), pp. 407–499. doi:10.1214/009053604000000067
11. P.A., Estevez, M., Tesmer, C.A., Perez and J.M., Zurada, Normalized mutual information feature selection, IEEE Trans. Neural Netw. 20 (2009), pp. 189–201. doi:10.1109/TNN.2008.2005601
12. J., Fan and R., Li, Variable selection via nonconcave penalized likelihood and its oracle properties, J. Amer. Statist. Assoc. 96 (2001), pp. 1348–1360. doi:10.1198/016214501753382273
13. J., Fan, L., Xue and H., Zou, Strong oracle optimality of folded concave penalized estimation, Ann. Stat. 42 (2014), pp. 819–849. doi:10.1214/13-AOS1198
14. G., Grimmett and D., Stirzaker, Probability and Random Processes, Oxford University Press, New York, 2001.
15. I., Guyon and A., Elisseeff, An introduction to variable and feature selection, J. Mach. Learn. Res. 3 (2003), pp. 1157–1182.
16. M., Hamza and D., Larocque, An empirical comparison of ensemble methods based on classification trees, J. Statist. Comput. Simul. 75 (2005), pp. 629–643. doi:10.1080/00949650410001729472
17. D., Harrison and D.L., Rubinfeld, Hedonic prices and the demand for clean air, J. Environ. Econ. Manage. 5 (1978), pp. 81–102. doi:10.1016/0095-0696(78)90006-2
18. T., Hastie, R., Tibshirani and J., Friedman, The Elements of Statistical Learning:Data Mining, Inference, and Prediction, 2nd ed., Springer, Berlin, 2009.
19. A.K., Jain, R.P.W., Duin and J., Mao, Statistical pattern recognition:a review, IEEE Trans. Pattern Anal. Mach. Intell. 22 (2000), pp. 4–37. doi:10.1109/34.824819
20. I., Kojadinovic, Agglomerative hierarchical clustering of continuous variables based on mutual information, Comput. Stat. Data Anal. 46 (2004), pp. 269–294. doi:10.1016/S0167-9473(03)00153-1
21. A., Krogh and J., Vedelsby, Neural network ensembles, cross validation, and active learning, in Advances in Neural Information Processing Systems, Vol. 7, G. Tesauro, D.S. Touretzky and T.K. Leen, eds., MIT Press, Cambridge, 1995, pp. 231–238.
22. N., Kwak and C.H., Choi, Input feature selection for classification problems, IEEE Trans. Neural Netw. 13 (2002), pp. 143–159. doi:10.1109/72.977291
23. N., Meinshausen, Relaxed lasso, Comput. Stat. Data Anal. 52 (2007), pp. 374–393. doi:10.1016/j.csda.2006.12.019
24. N., Meinshausen and P., Bühlmann, Stability selection (with discussion), J. Royal Statist. Soc:Ser. B 72 (2010), pp. 417–473. doi:10.1111/j.1467-9868.2010.00740.x
25. J., Mendes-Moreira, C., Soares, A.M., Jorge and J.F., deSousa, Ensemble approaches for regression:A survey, ACM Comput. Surv. 45 (2012), pp. 10. Article 10. doi:10.1145/2379776.2379786
26. A., Miller, Subset Selection in Regression, 2nd ed., Chapman and Hall/CRC Press, New York, 2002.
27. A., Mkhadri and M., Ouhourane, An extended variable inclusion and shrinkage algorithm for correlated variables, Comput. Stat. Data Anal. 57 (2013), pp. 631–644. doi:10.1016/j.csda.2012.07.023
28. A., Mkhadri and M., Ouhourane, A group VISA algorithm for variable selection, Statist. Methods Appl. 24 (2015), pp. 41–60. doi:10.1007/s10260-014-0281-8
29. K., Pearson, Notes on regression and inheritance in the case of two parents, Proc. R. Soc. Lond. 58 (1895), pp. 240–242. doi:10.1098/rspl.1895.0041
30. H., Peng, F., Long and C., Ding, Feature selection based on mutual information:Criteria of max-dependency, max-relevance and min-redundancey, IEEE Trans. Pattern Anal. Mach. Intell. 27 (2005), pp. 1226–1238. doi:10.1109/TPAMI.2005.159
31. P., Radchenko and G.M., James, variable inclusion and shrinkage algorithms, J. Amer. Statist. Assoc. 103 (2008), pp. 1304–1315. doi:10.1198/016214508000000481
32. R.D., Shah and R.J., Samworth, Variable selection with error control:another look at stability selection, J. R. Statist. Soc.:Ser. B 75 (2013), pp. 55–80. doi:10.1111/j.1467-9868.2011.01034.x
33. C.E., Shannon, A mathematical theory of communication, Bell Syst. Tech. J. 27 (1948), pp. 379–423.623–656. doi:10.1002/j.1538-7305.1948.tb01338.x
34. R., Tibshirani, Regression shrinkage and selection via the lasso, J. R. Statist. Soc.:Ser. B 58 (1996), pp. 267–288.
35. R., Tibshirani, Regression shrinkage and selection via the lasso:a retrospective, J. R. Statist. Soc.:Ser. B (Statist. Methodol.) 73 (2011), pp. 273–282. doi:10.1111/j.1467-9868.2011.00771.x
36. A., Tsanas, M.A., Little, P.E., McSharry and L.O., Ramig, Accurate telemonitoring of Parkinson's disease progression by non-invasive speech tests, IEEE Trans. Biomed. Eng. 57 (2010), pp. 884–893. doi:10.1109/TBME.2009.2036000
37. A., Unler, A., Murat and R.B., Chinnam, mr2PSO:A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification, Inf. Sci. 181 (2011), pp. 4625–4641. doi:10.1016/j.ins.2010.05.037
38. L.T., Vinh, N.D., Thang and Y.K., Lee, An Improved maximum relevance and minimum redundancy feature selection algorithm based on normalized mutual information, in The International Symposium on Applications and the Internet, Y. Okabe, G. Agha and C. Seon Hong, eds., IEEE Computer Society, Seoul, 2010, pp. 395–398.
39. B., Wang, Variable ranking by solution-path algorithms, M.Math., University of Waterloo (Canada), 2011.
40. S., Wang, B., Nan, S., Rosset and J., Zhu, Random lasso, Ann. Appl. Stat. 5 (2011), pp. 468–485. doi:10.1214/10-AOAS377
41. J., Weston, A., Elisseff, B., Schoelkopf and M., Tipping, Use of the zero norm with linear models and kernel methods, J. Mach. Learn. Res. 3 (2003), pp. 1439–1461.
42. L., Xin and M., Zhu, Stochastic stepwise ensembles for variable selection, J. Comput. Graph. Stat. 21 (2012), pp. 275–294. doi:10.1080/10618600.2012.679223
43. E.P., Xing, M.I., Jordan and R.M., Karp, Feature selection for high-dimensional genomic microarray data, Proc. Eighteenth Int. Conf. Mach. Learn. (2001), pp. 601–608.
44. C.-X., Zhang, G.-W., Wang and J.-M., Liu, RandGA:Injecting randomness into parallel genetic algorithm for variable selection, J. Appl. Stat. 42 (2015), pp. 630–647. doi:10.1080/02664763.2014.980788
45. M., Zhu and H.A., Chipman, Darwinian evolution in parallel universes:A parallel genetic algorithm for variable selection, Technometrics 48 (2006), pp. 491–502. doi:10.1198/004017006000000093
46. H., Zou, The adaptive lasso and its oracle properties, J. Amer. Statist. Assoc. 101 (2006), pp. 1418–1429. doi:10.1198/016214506000000735
47. H., Zou and T., Hastie, Regulzarization variable selection via the elastic net, J. R. Statist. Soc.:Ser. B 67 (2005), pp. 301–320. doi:10.1111/j.1467-9868.2005.00503.x
48. H., Zou and R.Z., Li, One-step sparse estimates in nonconcave penalized likelihood models, Ann. Stat. 36 (2008), pp. 1509–1533. doi:10.1214/009053607000000802