Separating the wheat from the chaff: On feature selection and feature importance in regression random forests and symbolic regression

Genetic and Evolutionary Computation Conference, GECCO'11 - Companion Publication

Volumn , Issue , 2011, Pages 623-630

  Stijven, Sean ^a   Minnebo, Wouter ^a   Vladislavleva, Katya ^a  

^a UNIVERSITY OF ANTWERP   (Belgium)

Author keywords

feature selection; genetic programming; random forests; symbolic regression; variable importance; variable selection

Indexed keywords

HIGH DIMENSIONAL DATA; OPEN PROBLEMS; RANDOM FORESTS; SYMBOLIC REGRESSION; TEST PROBLEM; VARIABLE IMPORTANCE; VARIABLE SELECTION;

DECISION TREES; GENETIC PROGRAMMING; REGRESSION ANALYSIS; SEMANTICS;

FEATURE EXTRACTION;

EID: 80051920124     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2001858.2002059     Document Type: Conference Paper

References (25)

1. H. D. Bondell and B. J. Reich. Simultaneous regression shrinkage, variable selection, and supervised clustering of predictors with oscar. Biometrics, 64:115-123, Mar. 2008.
2. L. Breiman. Bagging predictors. Machine Learning, 24:123-140, August 1996. (Pubitemid 126724382)
3. L. Breiman. Random forests. Machine Learning, 45:5-32, 2001. 10.1023/A:1010933404324.
4. Evolved Analytics LLC. DataModeler Release 1.0. Evolved Analytics LLC, 2010.
5. M. Gashler, C. Giraud-Carrier, and T. Martinez. Decision Tree Ensemble: Small Heterogeneous Is Better Than Large Homogeneous. In 2008 Seventh International Conference on Machine Learning and Applications, pages 900-905. IEEE, Dec. 2008.
6. R. Genuer, J.-M. Poggi, and C. Tuleau-Malot. Variable selection using random forests. Pattern Recogn. Lett., 31:2225-2236, October 2010.
7. U. Grömping. Variable importance assessment in regression: Linear regression versus random forest. The American Statistician, 64:308-319, Nov. 2009.
8. I. Guyon and A. Elisseeff. An introduction to variable and feature selection. J. Mach. Learn. Res., 3:1157-1182, March 2003.
9. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik. Gene selection for cancer classification using support vector machines. Machine Learning, 46:389-422, 2002. (Pubitemid 34129977)
10. Human Development Reports. http://www.hdr.undp.org/.
11. Human Development Research Papers. http://hdr.undp.org/en/reports/global/ hdr2010/papers/.
12. H. Ishwaran. Variable importance in binary regression trees and forests. Electronic Journal of Statistics, 1:519-537, 2007.
13. Y. Lin and Y. Jeon. Random forests and adaptive nearest neighbors. Journal of the American Statistical Association, 101:578-590, June 2006.
14. T. McConaghy. Latent Variable Symbolic Regression for High-Dimensional Inputs, pages 103-118. Springer, 2010.
15. R. K. McRee. Symbolic regression using nearest neighbor indexing. In Proceedings of the 12th annual conference companion on Genetic and evolutionary computation, GECCO '10, pages 1983-1990, New York, NY, USA, 2010. ACM.
16. H. Peng, F. Long, and C. Ding. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8):1226-1238, Aug. 2005. (Pubitemid 41245053)
17. A. Rakotomamonjy. Variable selection using svm-based criteria. Journal of Machine Learning Research, 3:1357-1370, 2003.
18. D. S. Siroky. Navigating random forests and related advances in algorithmic modeling. Statistics Surveys, 3:147-163, 2009.
19. G. Smits and M. Kotanchek. Pareto-front exploitation in symbolic regression. In U.-M. O'Reilly, T. Yu, R. L. Riolo, and B. Worzel, editors, Genetic Programming Theory and Practice II, chapter 17, pages 283-299. Springer, Ann Arbor, 13-15 May 2004.
20. C. Strobl, A. L. Boulesteix, T. Kneib, T. Augustin, and A. Zeileis. Conditional variable importance for random forests. BMC Bioinformatics, 9(1):307+, July 2008.
21. C. Strobl, A. L. Boulesteix, A. Zeileis, and T. Hothorn. Bias in random forest variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics, 8(1):25+, Jan. 2007.
22. K. Vladislavleva. Model-based Problem Solving through Symbolic Regression via Pareto Genetic Programming. PhD thesis, Tilburg University, 2008.
23. K. Vladislavleva, K. Veeramachaneni, M. Burland, J. Parcon, and U.-M. O'Reilly. Knowledge mining with genetic programming methods for variable selection in flavor design. In GECCO, pages 941-948, 2010.
24. Z. Zhao, F. Morstatter, S. Sharma, S. Alelyani, A. Anand, and H. Liu. Advancing feature selection research - asu feature selection repository. Technical report, Arizona State University, June 2010.
25. H. Zou and T. Hastie. Regularization and variable selection via the Elastic Net. Journal of the Royal Statistical Society B, 67:301-320, 2005. (Pubitemid 40465877)