RandGA: injecting randomness into parallel genetic algorithm for variable selection

Journal of Applied Statistics

Volumn 42, Issue 3, 2015, Pages 630-647

  Zhang, Chun Xia ^a   Wang, Guan Wei ^b   Liu, Jun Min ^a  

^a XI'AN JIAOTONG UNIVERSITY   (China)

^b XI'AN TECHNOLOGICAL UNIVERSITY   (China)

Author keywords

ensemble learning; genetic algorithm; randomness; strength diversity tradeoff; variable selection

Indexed keywords

EID: 84918810308     PISSN: 02664763     EISSN: 13600532     Source Type: Journal    
DOI: 10.1080/02664763.2014.980788     Document Type: Article

References (45)

1. L. Breiman, Better subset regression using the nonnegative garrote, Technometrics 37(4) (1995), pp. 373–384. doi: 10.1080/00401706.1995.10484371
2. L. Breiman, Heuristics of instability and stabilization in model selection, Ann. Stat. 24(6) (1996), pp. 2350–2383. doi: 10.1214/aos/1032181158
3. L. Breiman, Bagging predictors, Mach. Learn. 24(2) (1996), pp. 123–140.
4. L. Breiman, Random forests, Mach. Learn. 45(1) (2001), pp. 5–32. doi: 10.1023/A:1010933404324
5. G. Brown, J. Wyatt, R. Harris, and X. Yao, Diversity creation methods: A survey and categorization, Inf. Fusion 6(1) (2005), pp. 5–20. doi: 10.1016/j.inffus.2004.04.004
6. P. Bühlmann and T. Hothorn, Boosting algorithms: Regularization, prediction and model fitting, Statist. Sci. 22(4) (2007), pp. 477–505. doi: 10.1214/07-STS242
7. S. Chatterjee, M. Lauadto, and L.A. Lynch, Genetic algorithms and their statistical applications: An introduction, Comput. Stat. Data Anal. 22 (1996), pp. 633–651. doi: 10.1016/0167-9473(96)00011-4
8. B. Efron, T. Hastie, I. Hohnstone, and R. Tibshirani, Least angle regression, Ann. Stat. 32(2) (2004), pp. 407–499. doi: 10.1214/009053604000000067
9. J.Q. Fan and R.Z. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, J. Am. Statist. Assoc. 96(456) (2001), pp. 1348–1360. doi: 10.1198/016214501753382273
10. Y. Freund and R. Schapire, Experiments with a new boosting algorithm, Proceedings of the 13th International Conference on Machine Learning, Bari, Italy, 1996, pp. 148–156.
11. J.H. Friedman, Greedy function approximation: A gradient boosting machine, Ann. Stat. 29(5) (2001), pp. 1189–1232. doi: 10.1214/aos/1013203451
12. W.J. Fu, Penalized regression: The bridge versus the lasso, J. Comput. Graph. Stat. 7(3) (1998), pp. 397–346.
13. L. Guo and S. Boukir, Margin-based ordered aggregation for ensemble learning, Pattern Recognit. Lett. 34(6) (2013), pp. 603–609. doi: 10.1016/j.patrec.2013.01.003
14. M. Hamza and D. Larocque, An empirical comparison of ensemble methods based on classification trees, J. Statist. Comput. Simul. 75(8) (2005), pp. 629–643. doi: 10.1080/00949650410001729472
15. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., Springer, 2009.
16. A. Krogh and J. Vedelsby, Neural network ensembles, cross validation, and active learning, in Advances in Neural Information Processing Systems, Vol. 7, G. Tesauro, D.S. Touretzky, T.K. Leen, eds., MIT Press, Cambridge, pp. 231–238.
17. C.Y. Lin, Some recent developments in parametric and nonparametric regression models, Ph.D. diss., Raleigh, North Carolina State University, 2012.
18. S.S. Mao, L.C. Jiao, L. Xiong, and S.P. Gou, Greedy optimization classifiers ensemble based on diversity, Pattern Recognit. 44(6) (2011), pp. 1245–1261. doi: 10.1016/j.patcog.2010.11.007
19. D. Margineantu and T. Dietterich, Pruning adaptive boosting, in Proceedings of the 14th International Conference on Machine Learning, Morgan, Kafmann, San Francisco, CA, 1997, pp. 211–218.
20. G. Martínez-Muñoz, D. Hernández-Lobato, and A. Suárez, An analysis of ensemble pruning techniques based on ordered aggregation, IEEE Trans. Pattern Anal. Mach. Intell. 31(2) (2009), pp. 245–259. doi: 10.1109/TPAMI.2008.78
21. N. Meinshausen, Relaxed lasso, Comput. Stat. Data Anal. 52(1) (2007), pp. 374–393. doi: 10.1016/j.csda.2006.12.019
22. N. Meinshausen and P. Bühlmann, Stability selection (with discussion), J. R. Statist. Soc. (Ser. B) 72(4) (2010), pp. 417–473. doi: 10.1111/j.1467-9868.2010.00740.x
23. D. Mejri, R. Khanchel, and M. Limam, An ensemble method for concept drift in nonstationary environment, J. Stat. Comput. Simul. 83(6) (2013), pp. 1115–1128. doi: 10.1080/00949655.2011.651797
24. J. Mendes-Moreira, C. Soares, A.M. Jorge, and J.F. de Sousa, Ensemble approaches for regression: A survey, ACM Comput. Surv. 45(1) (2012), p. 10. Article 10. doi: 10.1145/2379776.2379786
25. A. Miller, Subset Selection in Regression, 2nd ed., Chapman & Hall/CRC Press, New York, 2002.
26. A. Mkhadri and M. Ouhourane, An extended variable inclusion and shrinkage algorithm for correlated variables, Comput. Stat. Data Anal. 57(1) (2013), pp. 631–644. doi: 10.1016/j.csda.2012.07.023
27. C.O. Plumpton, L.I. Kuncheva, N.N. Oosterhof, and S.J. Jognston, Naive random subspace ensemble with linear classifiers for real-time classification of fMRI data, Pattern Recognit. 45(5) (2012), pp. 2101–2108. doi: 10.1016/j.patcog.2011.04.023
28. P. Radchenko and G.M. James, variable inclusion and shrinkage algorithms, J. Amer. Statist. Assoc. 103(483) (2008), pp. 1304–1315. doi: 10.1198/016214508000000481
29. J.J. Rodríguez, L.I. Kuncheva, and C.J. Alonso, Rotation forest: A new classifier ensemble method, IEEE Trans. Pattern Anal. Mach. Intell. 28(10) (2006), pp. 1619–1631. doi: 10.1109/TPAMI.2006.211
30. L. Rokach, Taxonomy for characterizing ensemble methods in classification tasks: A review and annotated bibliography, Comput. Stat. Data Anal. 53(12) (2009), pp. 4046–4072. doi: 10.1016/j.csda.2009.07.017
31. R.D. Shah and R.J. Samworth, Variable selection with error control: another look at stability selection, J. R. Statist. Soc. (Ser. B) 75(1) (2013), pp. 55–80. doi: 10.1111/j.1467-9868.2011.01034.x
32. G. Shmueli, To explain or to predict? Statist. Sci. 25(3) (2010), pp. 289–310. doi: 10.1214/10-STS330
33. R. Tibshirani, Regression shrinkage and selection via the lasso, J. R. Statist. Soc. (Ser. B) 58(1) (1996), pp. 267–288.
34. S.J. Wang, B. Nan, S. Rosset, and J. Zhu, Random lasso, Ann. Stat. 5(1) (2011), pp. 468–485. doi: 10.1214/10-AOAS377
35. L. Xin and M. Zhu, Stochastic stepwise ensembles for variable selection, J. Comput. Graph. Stat. 21(2) (2012), pp. 275–294. doi: 10.1080/10618600.2012.679223
36. C.X. Zhang, G.W. Wang, and J.S. Zhang, An empirical bias-variance analysis of DECORATE ensemble method at different training sample sizes, J. Appl. Stat. 39(4) (2012), pp. 829–850. doi: 10.1080/02664763.2011.620949
37. C.X. Zhang and J.S. Zhang, A local boosting algorithm for solving classification problems, Comput. Stat. Data Anal. 52(4) (2008), pp. 1928–1941. doi: 10.1016/j.csda.2007.06.015
38. C.X. Zhang and J.S. Zhang, A novel method for constructing ensemble classifiers, Stat. Comput. 19(3) (2009), pp. 317–327. doi: 10.1007/s11222-008-9094-7
39. M.L. Zhang and Z.H. Zhou, Exploiting unlabeled data to enhance diversity, Data Min. Knowl. Discov. 26(1) (2013), pp. 98–126. doi: 10.1007/s10618-011-0243-9
40. Z.H. Zhou, J. Wu, and W. Tang, Ensembling neural networks: Many could be better than all, Artif. Intell. 137(1-2) (2002), pp. 239–263. doi: 10.1016/S0004-3702(02)00190-X
41. M. Zhu and H.A. Chipman, Darwinian evolution in parallel universes: A parallel genetic algorithm for variable selection, Technometrics 48(4) (2006), pp. 491–502. doi: 10.1198/004017006000000093
42. M. Zhu and G.Z. Fan, Variable selection by ensembles for the Cox model, J. Statist. Comput. Simul. 81(12) (2011), pp. 1983–1992. doi: 10.1080/00949655.2010.511622
43. H. Zou, The adaptive lasso and its oracle properties, J. Amer. Statist. Assoc. 101(476) (2006), pp. 1418–429. doi: 10.1198/016214506000000735
44. H. Zou and T. Hastie, Regulzarization variable selection via the elastic net, J. R. Statist. Soc. (Ser. B) 67(2) (2005), pp. 301–320. doi: 10.1111/j.1467-9868.2005.00503.x
45. H. Zou and R.Z. Li, One-step sparse estimates in nonconcave penalized likelihood models, Ann. Stat. 36(4) (2008), pp. 1509–1533. doi: 10.1214/009053607000000802