Random Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling

Journal of Chemical Information and Computer Sciences

Volumn 43, Issue 6, 2003, Pages 1947-1958

  Svetnik, Vladimir ^a   Liaw, Andy ^a   Tong, Christopher ^a   Christopher Culberson, J ^a   Sheridan, Robert P ^a   Feuston, Bradley P ^a  

^a MERCK RESEARCH LABORATORIES   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

MULTIPLE LINEAR REGRESSION (MLR); PARTIAL LEAST SQUARES (PLS);

ALGORITHMS; LEARNING SYSTEMS; MOLECULAR STRUCTURE; OPTIMIZATION; REGRESSION ANALYSIS; TREES (MATHEMATICS);

CHEMICAL ENGINEERING;

EID: 0345548657     PISSN: 00952338     EISSN: None     Source Type: Journal    
DOI: 10.1021/ci034160g     Document Type: Article

References (42)

1. Rusinko, A., III; Farmen, M. W.; Lambert, C. G.; Brown, P. L.; Young, S. S. Analysis of a large structure/biological activity data set using recursive partitioning. J. Chem. Inf. Comput. Sci. 1999, 39, 1017-1026.
2. Doniger, S.; Hofmann, T.; Yeh, J. Predicting CNS permeability of drug molecules: comparison of neural network and support vector machine algorithms. J. Comput. Biol. 2002, 9, 849-864.
3. Kauffman, G. W.; Jurs, P. C. QSAR and k-nearest neighbor classification analysis of selective cyclooxygenase-2 inhibitors using topologically based numerical descriptors. J. Chem. Inf. Comput. Sci. 2001, 41, 1553-1560.
4. Sheridan, R. P.; Nachbar, R. B.; Bush, B. L. Extending the trend vector: the trend matrix and sample-based partial least squares. J. Comput.-Aided Mol. Des. 1994, 8, 323-340.
5. Bakken, G. A.; Jurs, P. C. Classification of multidrug-resistance reversal agents using structure-based descriptors and linear discriminant analysis. J. Med. Chem. 2000, 43, 4534-4541.
6. Tong, W.; Hong, H.; Fang, H.; Xie, Q.; Perkins, R. Decision forest: combining the predictions of multiple independent decision tree models. J. Chem. Inf. Comput. Sci. 2003, 43, 525-531.
7. Dietterich, T. G. Ensemble learning. In The Handbook of Brain Theory and Neural Networks, 2nd ed.; Arbib, M. A., Ed.; The MIT Press: Cambridge, 2002.
8. Breiman, L. Random forests. Machine Learning 2001, 45, 5-32.
9. The Mathematics of Generalization; Wolpert, D. H., Ed.; Addison-Wesley: Reading, 1995.
10. Breiman, L. Bagging predictors. Machine Learning 1996, 24, 123-140.
11. Breiman, L.; Friedman, J. H.; Olshen, R. A.; Stone, C. J. Classification and Regression Trees; Chapman & Hall/CRC: Boca Raton, 1984.
12. Freund, Y.; Schapire, R. E. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. System Sci. 1997, 55, 119-139.
13. Friedman, J. H. Greedy function approximation: a gradient boosting machine. Ann. Stat. 2001, 29, 1189-1202.
14. Meyer, D.; Leisch, F.; Hornik, K. The support vector machine under test. Neurocomputing 2003, submitted.
15. Hawkins, D. M.; Musser, B. J. One tree or a forest? Alternative dendrographic models. Comput. Sci. Stat. 1999, 30, 534-542.
16. Hawkins, D. M.; Basak, S. C.; Mills, D. Assessing model fit by cross-validation. J. Chem. Inf. Comput. Sci. 2003, 43, 579-586.
17. Breiman, L. Out-of-bag estimation; Technical Report; Department of Statistics: UC Berkeley, 1996.
18. Hastie, T.; Tibshirani, R.; Friedman, J. H. The Elements of Statistical Learning; Springer-Verlag: New York, 2001.
19. Breiman, L. Using models to infer mechanisms, IMS Wald Lecture 2, 2002, http://oz.berkeley.edu/users/breiman/wald2002-2.pdf.
20. Schapire, R. E.; Freund, Y.; Bartlett, P.; Lee, W. S. Boosting the margin: A new explanation for the effectiveness of voting methods. Ann. Stat. 1998, 26, 1651-1686.
21. Brown, R. D.; Martin, Y. C. Use of structure-activity data to compare structure-based clustering methods and descriptors for use in compound selection. J. Chem. Inf. Comput. Sci. 1996, 36, 572-584.
22. Penzotti, J. E.; Lamb, M. L.; Evensen, E.; Grootenhuis, P. D. J. A computational ensemble pharmacophore model for identifying sub-strates of P-glycoprotein. J. Med. Chem. 2002, 45, 1737-1740.
23. Carhart, R. E.; Smith, D. H.; Venkataraghavan, R. Atom pairs as molecular features in structure-activity studies: definition and applications. J. Chem. Inf. Comput. Sci. 1985, 25, 64-73.
24. Klopman, G.; Shi, L. M.; Ramu, A. Quantitative structure-activity relationship of multidrug resistance reversal agents. Mol. Pharmacol. 1997, 52, 323-334.
25. DRAGON; Milano Chemometrics, http://www.disat.unimib.it/chm/Dragon.htm.
26. Gilligan, P. J.; Cain, G. A.; Christos, T. E.; Cook, L.; Drummond, S.; Johnson, A. L.; Kergaye, A. A.; McElroy, J. F.; Rohrbach, K. W.; Schmidt, W. K. ; Tam, S. W. Novel piperidine σ receptor ligands as potential antipsychotic drugs. J. Med. Chem. 1992, 35, 4344-4361.
27. Liaw, A.; Wiener, M. Classification and regression by randomForest. R News 2002, 2/3, 18-22.
28. Therneau, T. M.; Atkinson, E. J. An introduction to recursive partitioning using the RPART routines; Technical Report; Department of Health Sciences Research: Mayo Clinic, 1997.
29. pls.pcr; http://cran.r-project.org/src/contrib/PACKAGES.html#pls.pcr.
30. Nguyen, D. V.; Rocke, D. M. Tumor classification by partial least squares using microarray gene expression data. Bioinformatics 2002, 78, 39-50.
31. Barker, M.; Rayens, W. Partial least squares for discrimination. J. Chemometrics 2003, 17, 166-173.
32. Guyon, I.; Eliseeff, A. An introduction to variable and feature selection. J. Machine Learning Res. 2003, 3, 1157-1182.
33. Svetnik, V.; Liaw, A.; Tong, C. Variable selection in random forest with application to quantitative structure-activity relationship. In Proceedings of the 7th Course on Ensemble Methods for Learning Machines; Intrator, N., Masulli, F., Eds.; Springer-Verlag: Berlin, 2004; submitted.
34. Willett, P. Chemical similarity searching. J. Chem. Inf. Comput. Sci. 1998, 38, 983-996.
35. Liaw, A.; Svetnik, V. Application of random forest to QSAR modeling of HTS data, Presented at the 26th Annual Midwest Biopharmaceutical Statistics Workshop; Muncie, IN, 2003.
36. Breiman, L.; Cutler, A. In Manual-Setting up, using, and understanding Random Forests; v4.0, 2003, http://www.stat.berkeley.edu/users/breiman/rf.html.
37. The Comprehensive R Archive Network, http://cran.r-project.org.
38. Ripley, B. D. Pattern Recognition and Neural Networks; Cambridge University Press: Cambridge, 1996.
39. Breiman, L. Arcing classifiers. Ann. Stat. 1998, 26, 801-849.
40. Friedman, J. H. On bias, variance, 0/1-loss, and the curse-of-dimensionality. Data Mining Knowledge Discovery 1997, 1, 55-77.
41. Domingos, P. A unified bias-variance decomposition for zero-one and squared loss. In Proceedings of the Seventeenth National Conference on Artificial Intelligence; AAAI Press: Austin, 2000.
42. Valentini, G.; Dietterich, T. G. Bias-variance analysis and ensembles of SVM. In Third International Workshop on Multiple Classifier Systems; Kittler, J., Roli, F., Eds.; Springer: New York, 2002.