Mining data with random forests: Current options for real-world applications

Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery

Volumn 4, Issue 1, 2014, Pages 55-63

  Ziegler, Andreas ^a,b   König, Inke R ^a  

^a UNIVERSITY HOSPITAL SCHLESWIG HOLSTEIN   (Germany)

^b UNIVERSITY OF LÜBECK   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

BACKWARD ELIMINATION; CLASSIFICATION AND REGRESSION TREE; CORRELATION BETWEEN FEATURES; MINING HIGH-DIMENSIONAL DATA; PROBABILITY ESTIMATION; RANDOM SELECTION; ROBUST APPROACHES; VARIABLE IMPORTANCES;

FORESTRY;

DECISION TREES;

DECISION MAKING; FORESTRY; RANDOM PROCESSES; TREES;

EID: 84890868650     PISSN: 19424787     EISSN: 19424795     Source Type: Journal    
DOI: 10.1002/widm.1114     Document Type: Article

References (48)

1. Breiman L, Friedman J, Olshen RA, Stone CJ. Classification and Regression Trees. Chapman & Hall/CRC: Boca Raton, FL; 1984.
2. Loh W-Y. Classification and regression trees. WIREs Data Mining Knowl Discov 2011, 1:14-23.
3. Gordon L, Olshen RA. Tree-structured survival analysis. Cancer Treat Rep 1985, 69:1065-1069.
4. Breiman L. Random forests. Mach Learn 2001, 45:5-32.
5. Breiman L. Bagging predictors. Mach Learn 1996, 24:123-140.
6. Hothorn T, Bühlmann P, Dudoit S, Molinaro A, van der Laan MJ. Survival ensembles. Biostatistics 2006, 7:355-373.
7. Ishwaran H, Kogalur UB, Blackstone EH, Lauer MS. Random survival forests. Ann App Statist 2008, 2:841-860.
8. Kruppa J, Liu Y, Biau G, Kohler M, König IR, Malley JD, Ziegler A. Probability estimation with machine learning methods for dichotomous and multi-category outcome: theory. Biom J, In press.
9. Nicodemus KK, Malley JD, Strobl C, Ziegler A. The behaviour of random forest permutation-based variable importance measures under predictor correlation. BMC Bioinformatics 2010, 11:110.
10. Boulesteix A-L, Janitza S, Kruppa J, König IR. Overview of random forest methodology and practical guidance with emphasis on computational biology and bioinformatics. WIREs Data Mining Knowl Discov 2012, 2:493-507.
11. Chen CC, Schwender H, Keith J, Nunkesser R, Mengersen K, Macrossan P. Methods for identifying SNP interactions: a review on variations of Logic Regression, Random Forest and Bayesian logistic regression. IEEE/ACM Trans Comput Biol Bioinform 2011, 8:1580-1591.
12. Chen X, Ishwaran H. Random forests for genomic data analysis. Genomics 2012, 99:323-329.
13. Cutler A, Stevens JR. Random forests for microarrays. Meth Enzymol 2006, 411:422-432.
14. Sun YV. Multigenic modeling of complex disease by random forests. Adv Genet 2010, 72:73-99.
15. Strobl C, Malley J, Tutz G. An introduction to recursive partitioning: rationale, application, and characteristics of classification and regression trees, bagging, and random forests. Psychol Methods 2009, 14:323-348.
16. Touw WG, Bayjanov JR, Overmars L, Backus L, Boekhorst J, Wels M, van Hijum SA. Data mining in the life sciences with random forest: a walk in the park or lost in the jungle? Brief Bioinform 2013, 14:315-326.
17. Verikas A, Gelzinis A, Bacauskiene M. Mining data with random forests: a survey and results of new tests. Pattern Recogn 2011, 44:330-349.
18. Mitchell MW. Bias of the random forest out-of-bag (OOB) error for certain input parameters. Open J Statist 2011, 1:205-211.
19. Schwarz DF, König IR, Ziegler A. On safari to Random Jungle: a fast implementation of Random Forests for high-dimensional data. Bioinformatics 2010, 26:1752-1758.
20. Genuer R, Poggi JM, Tuleau C. Random Forests: some methodological insights. arXiv: 0811.3619. 2008. Available at: http://hal.inria.fr/inria-00340725/en/;
21. Al Ghoson AM. Decision tree induction & clustering techniques in SAS Enterprise Miner, SPSS Clementine, and IBM Intelligent Miner - a comparative analysis. Int J Manag Inf Syst 2010, 14:57-70.
22. Rokach L, Maimon O. Decision tress. In: Maimon O, Rokach L, eds. The Data Mining and Knowledge Discovery Handbook. New York: Springer; 2005, 165-192.
23. Schapire RE. A brief introduction to boosting. In: Dean TL, ed. IJCAI-99: Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, vol. 2. San Francisco, CA: Morgan Kaufmann; 1999, 1401-1406.
24. Friedman J, Hastie T, Tibshirani R. Additive logistic regression: a statistical view of boosting. Ann Stat 2000, 28:337-407.
25. Caruana R, Niculescu-Mizil A. An empirical comparison of supervised learning algorithms. In: Cohen W, Moore A, eds. Proceedings of the 23rd International Conference on Machine Learning. New York: Association for Computing Machinery; 2006, 161-168.
26. Hothorn T, Hornik K, Zeileis A. Unbiased recursive partitioning: a conditional inference framework. J Comput Graph Stat 2006, 15:651-674.
27. Biau G, Devroye L, Lugosi G. Consistency of random forests and other averaging classifiers. J Mach Learn Res 2008, 9:2039-2057.
28. Biau G, Devroye L. On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification. J Multivariate Anal 2010, 101:2499-2518.
29. Biau G. Analysis of a random forests model. J Mach Learn Res 2012, 13:1063-1095.
30. Genuer R. Variance reduction in purely random forests. J Nonparametric Stat 2012, 24:543-562.
31. Breiman L. Some infinity theory for predictor ensembles. 2000. Available at: http://digitalassets.lib.berkeley.edu/sdtr/ucb/text/579.pdf.
32. Ishwaran H, Kogalur UB. Random survival forests for R. R-News 2007, 7:25-31.
33. Strobl C, Boulesteix AL, Zeileis A, Hothorn T. Bias in random forest variable importance measures: illustrations, sources and a solution. BMC Bioinformatics 2007, 8:25.
34. Nicodemus KK, Malley JD. Predictor correlation impacts machine learning algorithms: implications for genomic studies. Bioinformatics 2009, 25:1884-1890.
35. Meng YA, Yu Y, Cupples LA, Farrer LA, Lunetta KL. Performance of random forest when SNPs are in linkage disequilibrium. BMC Bioinformatics 2009, 10:78.
36. Boulesteix AL, Bender A, Lorenzo Bermejo J, Strobl C. Random forest Gini importance favours SNPs with large minor allele frequency: impact, sources and recommendations. Brief Bioinform 2012, 13:292-304.
37. Díaz-Uriarte R, Alvarez de Andrés S. Gene selection and classification of microarray data using random forest. BMC Bioinformatics 2006, 7:3.
38. Genuer R, Poggi JM, Tuleau-Malot C. Variable selection using random forests. Pattern Recogn Lett 2010, 31:2225-2236.
39. Kursa MB. Robustness of the random forest-based gene selection methods. 2013. Available at: http://arxiv.org/abs/1305.4525.
40. Rodin AS, Litvinenko A, Klos K, Morrison AC, Woodage T, Coresh J, Boerwinkle E. Use of wrapper algorithms coupled with a random forests classifier for variable selection in large-scale genomic association studies. J Comput Biol 2009, 16:1705-1718.
41. Sandri M, Zuccolotto P. Variable selection using random forests. In: Zani S, Cerioli A, Riani M, Vichi M, eds. Proceedings of the Meeting of the Classification and Data Analysis Group (CLADAG) of the Italian Statistical Society, University of Parma, June 6-8, 2005. Heidelberg, Germany: Springer; 2006, 263-270.
42. Zhang H, Wang M. Search for the smallest random forest. Stat. its interface 2009, 2:381.
43. Banerjee M, Ding Y, Noone AM. Identifying representative trees from ensembles. Stat Med 2012, 31:1601-1616.
44. Kruppa J, Ziegler A, König IR. Risk estimation and risk prediction using machine learning methods. Hum Genet 2012, 131:1639-1654.
45. Lin HF, Juo SH, Cheng R. Comparison of the power between microsatellite and single-nucleotide polymorphism markers for linkage and linkage disequilibrium mapping of an electrophysiological phenotype. BMC Genet 2005, 6:S7.
46. Brown I, Mues C. An experimental comparison of classification algorithms for imbalanced credit scoring data sets. Exp Syst Appl 2012, 39:3446-3453.
47. Deloncle A, Berk R, D'Andrea F, Ghil M. Weather regime prediction using statistical learning. J Atmos Sci 2007, 64:1619-1635.
48. Pal M. Random forest classifier for remote sensing classification. Int J Remote Sens 2005, 26:217-222.