Random forests: From early developments to recent advancements

Systems Science and Control Engineering

Volumn 2, Issue 1, 2014, Pages 602-609

  Fawagreh, Khaled ^a   Gaber, Mohamed Medhat ^a   Elyan, Eyad ^a  

^a ROBERT GORDON UNIVERSITY   (United Kingdom)

Author keywords

Bagging; Ensemble learning; Random forests; Supervised learning

Indexed keywords

DECISION TREES; SUPERVISED LEARNING;

BAGGING; CLASSIFICATION TECHNIQUE; ENSEMBLE CLASSIFICATION; ENSEMBLE LEARNING; HIGH-ACCURACY; HISTORICAL VIEW; RANDOM FORESTS; RESEARCH COMMUNITIES;

DATA MINING;

EID: 84962466332     PISSN: None     EISSN: 21642583     Source Type: Journal    
DOI: 10.1080/21642583.2014.956265     Document Type: Article

References (43)

1. Adeva, J. J. G., Beresi, U., & Calvo, R. (2005). Accuracy and diversity in ensembles of text categorisers. CLEI Electronic Journal, 9(1), 1–12.
2. Amaratunga, D., Cabrera, J., & Lee, Y.-S. (2008). Enriched random forests. Bioinformatics, 24(18), 2010–2014.
3. Amit Y., & Geman, D. (1997). Shape quantization and recognition with randomized trees. Neural Computation, 9(7), 1545–1588.
4. Bader-El-Den, M., & Gaber, M. (2012, November 12–15). Garf: Towards self-optimised random forests. In T. Huang, Z. Zeng, C. Li, & C.-S. Leung (Eds.), Neural Information Processing – 19th International Conference, ICONIP 2012, Doha, Qatar, Proceedings, Part II, Lecture Notes in Computer Science (pp. 506–515). Berlin: Springer.
5. Bernard, S., Heutte, L., & Adam, S. (2010, August 18–21). A study of strength and correlation in random forests. In D.-S. Huang, T. Martin McGinnity, L. Heutte, & X.-P. Zhang (Eds.), Advanced Intelligent Computing Theories and Applications – 6th International Conference on Intelligent Computing, ICIC 2010, Changsha, China, Proceedings, Communications in Computer and Information Science (pp. 186–191). Berlin: Springer.
6. Boinee, P., De Angelis, A., & Foresti, G. L. (2005). Meta random forests. International Journal of Computational Intelligence, 2(3), 138–147.
7. Boulesteix, A.-L., Janitza, S., Kruppa, J., & König, I. R. (2012). Overview of random forest methodology and practical guidance with emphasis on computational biology and bioinformatics. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2(6), 493–507.
8. Breiman, L. (1996a). Bagging predictors. Machine Learning, 24(2), 123–140.
9. Breiman, L. (1996b). Stacked regressions. Machine Learning, 24(1), 49–64.
10. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
11. Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees (1st ed.). New York/Boca Raton, FL: Chapman and Hall/CRC.
12. Brown, G., Wyatt, J., Harris, R., & Yao, X. (2005). Diversity creation methods: A survey and categorisation. Information Fusion, 6(1), 5–20.
13. Cutler, D. R., Edwards Jr., T. C., Beard, K. H., Cutler, A., Hess, K. T., Gibson, J., & Lawler, J. J. (2007). Random forests for classification in ecology. Ecology, 88(11), 2783–2792.
14. Dietterich, T. G. (2000). An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning, 40(2), 139–157.
15. Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996). From data mining to knowledge discovery in databases. AI Magazine, 17(3), 37.
16. Flaxman, A. D., Vahdatpour, A., Green, S., James, S. L., & Murray, C. J. L. (2011). Random forests for verbal autopsy analysis: Multisite validation study using clinical diagnostic gold standards. Population Health Metrics, 9(29), 1–11.
17. Freund Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119–139.
18. Gao, D., Zhang, Y.-X., & Zhao, Y.-H. (2009). Random forest algorithm for classification of multiwavelength data. Research in Astronomy and Astrophysics, 9(2), 14–39.
19. Goldberg, D. E. (1989). Genetic algorithms in search, optimization, and machine learning (1st ed.). Boston, MA: Addison-Wesley Longman Publishing.
20. Hawkins, D. M. (2004). The problem of overfitting. Journal of Chemical Information and Computer Sciences, 44(1), 1–12.
21. Ho, T. K. (1995). Random decision forests. In Document analysis and recognition, 1995, Proceedings of the third international conference, Montreal, Quebec, Canada (Vol. 1, pp. 278–282). New York City, NY: IEEE.
22. Ho, T. K. (1998). The random subspace method for constructing decision forests. Intelligence, IEEE Transactions on Pattern Analysis and Machine, 20(8), 832–844.
23. Hu, W. (2009). Identifying predictive markers of chemosensitivity of breast cancer with random forests. Cancer, 13, 59–64.
24. Klassen, M., Cummings, M., & Saldana, G. (2008, April 9– 11). Investigation of random forest performance with cancer microarray data. In T. Philip (Ed.), Proceedings of the ISCA 23rd International Conference on Computers and Their Applications, CATA 2008, Cancun, Mexico (pp. 64–69). Cary, NC: International Society for Computers and Their Applications.
25. Kuncheva, L. I., & Whitaker, C. J. (2003). Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning, 51(2), 181–207.
26. Lam, L., & Suen, C. Y. (1997). Application of majority voting to pattern recognition: An analysis of its behavior and performance. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 27(5), 553–568.
27. Latinne, P., Debeir, O., & Decaestecker, C. (2001, July 2–4). Limiting the number of trees in random forests. In J. Kittler & F. Roli (Eds.), Multiple Classifier Systems, Second International Workshop, MCS 2001 Cambridge, UK, Lecture Notes in Computer Science (pp. 178–187). Berlin: Springer.
28. Liaw, A., & Wiener, M. (2002). Classification and regression by randomforest. RNews, 2(3), 18–22.
29. Löw, F., Schorcht, G., Michel, U., Dech, S., & Conrad, C. (2012, September 24). Per-fieldcropclassificationinirrigatedagri-cultural regions in Middle Asia using random forest and support vector machine ensemble. In SPIE remote sensing, Edinburgh, United Kingdom (pp. 85380R–85380R). Bellingham, WA: International Society for Optics and Photonics.
30. Polikar, R. (2006). Ensemble based systems in decision making. IEEE Circuits and Systems Magazine, 6(3), 21–45.
31. Quinlan, J. R. (1993). C4. 5: Programs for machine learning (Vol. 1). Boston, MA: Morgan Kaufmann.
32. Robnik-Šikonja, M. (2004, September 20–24). Improving random forests. In J.-F. Boulicaut, F. Esposito, F. Giannotti, & D. Pedreschi (Eds.), Machine Learning: ECML 2004, 15th European Conference on Machine Learning, Pisa, Italy, 2004 Proceedings, Lecture Notes in Computer Science (pp. 359–370). Berlin: Springer.
33. Rokach, L. (2010). Ensemble-based classifiers. Artificial Intelligence Review, 33(1–2), 1–39.
34. Saffari, A., Leistner, C., Santner, J., Godec, M., & Bischof, H. (2009). On-line random forests. In 2009 IEEE 12th international conference on computer vision workshops (ICCV workshops), Kyoto, Japan (pp. 1393–1400). New York City, NY: IEEE.
35. Shahzad, R. K., & Lavesson, N. (2012). Veto-based malware detection. In 2012 seventh international conference on availability, reliability and security (ARES), Prague, Czech Republic (pp. 47–54). New York City, NY: IEEE.
36. Shahzad, R. K., & Lavesson, N. (2013). Comparative analysis of voting schemes for ensemble-based malware detection. Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications, 4(1), 98–117.
37. Smyth, P., & Wolpert, D. (1999). Linearly combining density estimators via stacking. Machine Learning, 36(1–2), 59–83.
38. Sun, Y.-A., & Dance, C. (2012). When majority voting fails: Comparing quality assurance methods for noisy human computation environment. CoRR. Retrieved from arXiv:1204.3516.
39. Tsymbal, A., Pechenizkiy, M., & Cunningham, P. (2006, September 18–22). Dynamic integration with random forests. In J. Fürnkranz, T. Scheffer, & M. Spiliopoulou (Eds.), Machine Learning: ECML 2006, 17th European Conference on Machine Learning, Berlin, Germany, 2006 Proceedings, Lecture Notes in Computer Science (pp. 801–808). Berlin: Springer.
40. Wolpert, D. H. (1992). Stacked generalization. Neural Networks, 5(2), 241–259.
41. Xu, B., Huang, J. Z., Williams, G., & Ye, Y. (2012). Hybrid weighted random forests for classifying very high-dimensional data. International Journal of Data Warehousing and Mining, 8(2), 44–63.
42. Yan, W., & Goebel, K. F. (2004). Designing classifier ensembles with constrained performance requirements. In B. V. Dasarathy (Ed.), Proceedings of the SPIE 5434, Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2004, Orlando, FL (pp. 59–68). Bellingham, WA: SPIE.
43. Zaklouta, F., Stanciulescu, B., & Hamdoun, O. (2011). Traffic sign classification using kd trees and random forests. In The 2011 international joint conference on neural networks (IJCNN), San Jose, CA, USA (pp. 2151–2155). New York City, NY: IEEE.