What are our models really telling us? A practical tutorial on avoiding common mistakes when building predictive models

Chemoinformatics for Drug Discovery

Volumn , Issue , 2013, Pages 1-31

  Walters, W P ^a  

^a VERTEX PHARMACEUTICALS   (United States)

Author keywords

Applicability domain; Experimental error; Kendall tau; Model comparison; Pearson r; Predictive models

Indexed keywords

APPLICABILITY DOMAIN; EXPERIMENTAL ERRORS; KENDALL TAUS; MODEL COMPARISON; PEARSON R; PREDICTIVE MODELS;

EID: 85016064263     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118742785.ch1     Document Type: Chapter

References (52)

1. Gleeson MP, Hersey A, Montanari D, et al. Probing the links between in vitro potency, ADMET and physicochemical parameters. Nat Rev Drug Discov 2011;10:197-208.
2. Delaney JS. Predicting aqueous solubility from structure. Drug Discov Today 2005;10:289-295.
3. Wang J, Hou T. Recent advances on aqueous solubility prediction. Comb Chem High Throughput Screen 2011;14:328-338.
4. Hughes LD, Palmer DS, Nigsch F, et al. Why are some properties more difficult to predict than others? A study of QSPR models of solubility, melting point, and log P. J Chem Info Model 2008;48:220-232.
5. Hudelson MG, Ketkar NS, Holder LB, et al. High confidence predictions of drug-drug interactions: Predicting affinities for cytochrome P450 2C9 with multiple computational methods. J Med Chem 2008;51:648-654.
6. Aronov AM. Predictive in silico modeling for hERG channel blockers. Drug Discov Today 2005;10:149-155.
7. Song M, Clark M. Development and evaluation of an in silico model for hERG binding. J Chem Info Model 2006;46:392-400.
8. Smith RD, Dunbar JB, Jr, Ung PM, et al. CSAR benchmark exercise of 2010: Combined evaluation across all submitted scoring functions. J Chem Info Model 2011;51: 2115-2131.
9. Ceder V. The Quick Python Book. 2nd ed. Greenwich: Manning Publications; 2010. p 400.
10. Beazley DM. Python Essential Reference. 4th ed. Addison-Wesley Professional; 2009. p 717.
11. Adler J. R in a Nutshell: A Desktop Quick Reference. O'Reilly Media; 2010. p 636.
12. Chambers J. Software for Data Analysis: Programming with R (Statistics and Computing). New York: Springer; 2010. p 514.
13. Lipinski C. Drug-like properties and the causes of poor solubility and poor permeability. J Pharmacol Toxicol Methods 2000;44:235-249.
14. Hou TJ, Xia K, Zhang W, et al. ADME evaluation in drug discovery. 4. Prediction of aqueous solubility based on atom contribution approach. J Chem Info Model 2004;44:266-275.
15. Huuskonen J. Estimation of aqueous solubility for a diverse set of organic compounds based on molecular topology. J Chem Info Model 2000;40:773-777.
16. Huuskonen J, Salo M, Taskinen J. Aqueous solubility prediction of drugs based on molecular topology and neural network modeling. J Chem Info Model 1998;38:450-456.
17. Tetko IV, Tanchuk VY, Kasheva TN, et al. Estimation of aqueous solubility of chemical compounds using E-state indices. J Chem Info Model 2001;41:1488-1493.
18. Myrdal P, Ward GH, Dannenfelser RM, et al. AQUAFAC 1: Aqueous functional group activity coefficients; application to hydrocarbons. Chemosphere 1992;24:1047-1061.
19. http://www.pharmacy.arizona.edu/outreach/aquasol/index.html. Accessed 2013 May 14.
20. http://www.srcinc.com/what-we-do/product.aspx?id=133. Accessed 2013 May 14.
21. Llinàs A, Glen RC, Goodman JM. Solubility challenge: Can you predict solubilities of 32 molecules using a database of 100 reliable measurements? J Chem Info Model 2008; 48:1289-1303.
22. Hopfinger AJ, Esposito EX, Llinàs A, et al. Findings of the challenge to predict aqueous solubility. J Chem Info Model 2009;49:1-5.
23. Guha R, Dexheimer TS, Kestranek AN, et al. Exploratory analysis of kinetic solubility measurements of a small molecule library. Bioorg Med Chem 2011;19:4127-4134.
24. http://flowingdata.com/2008/02/15/how-to-read-and-use-a-box-and-whisker-plot/. Accessed 2013 May 14.
25. http://flowingdata.com/2012/05/15/how-to-visualize-and-compare-distributions/. Accessed 2013 May 14.
26. http://en.wikipedia.org/wiki/Box_plot. Accessed 2013 May 14.
27. Todeschini R, Consonni V. Molecular Descriptors for Chemoinformatics (Methods and Principles in Medicinal Chemistry). Weinheim: Wiley-VCH; 2009. p 1257.
28. Karelson M. Molecular Descriptors in QSAR/QSPR. New York: Wiley-Interscience;2000. p 448.
29. Goldman BB, Walters WP. Machine learning in computational chemistry. Annu Rep Comput Chem 2006;2:127-140.
30. http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient. Accessed 2013 May 14.
31. Glantz, S. Primer of Biostatistics. New York: McGraw-Hill Medical; 2011. p 320.
32. http://en.wikipedia.org/wiki/Kendall_tau_rank_correlation_coefficient. Accessed 2013 May 14.
33. http://en.wikipedia.org/wiki/Root-mean-square_deviation. Accessed 2013 May 14.
34. Landrum G, Lewis R, Palmer A, et al. Making sure there's a "give" associated with the "take": Producing and using open-source software in big pharma. J Cheminform 2011;3:1-1.
35. http://www.rdkit.org/. Accessed 2013 May 14.
36. Breiman L. Random forests. Mach Learn 2001;45:5-32.
37. Liaw A, Wiener M. Classification and regression by random forest. R News 2002;2:18-22.
38. Svetnik V, Liaw A, Tong C, et al. Random forest: A classification and regression tool for compound classification and QSAR modeling. J Chem Info Comput Sci 2003;43:1947-1958.
39. Yan A, Wang Z, Cai Z. Prediction of human intestinal absorption by GA feature selection and support vector machine regression. Int J Mol Sci 2008;9:1961-1976.
40. Yan A, Gasteiger J. Prediction of aqueous solubility of organic compounds based on a 3D structure representation. J Chem Info Model 2003;43:429-434.
41. Brown SP, Muchmore SW, Hajduk PJ. Healthy skepticism: Assessing realistic model performance. Drug Discov Today 2009;14:420-427.
42. Sheridan RP, Feuston BP, Maiorov VN, et al. Similarity to molecules in the training set is a good discriminator for prediction accuracy in QSAR. J Chem Info Comput Sci 2004;44:1912-1928.
43. Grant J, Haigh J, Pickup B, et al. Lingos, finite state machines, and fast similarity searching. J Chem Inf Model 2006; 46:1912-1918.
44. Hert J, Willett P, Wilton DJ, et al. New methods for ligand-based virtual screening: Use of data fusion and machine learning to enhance the effectiveness of similarity searching. J Chem Info Model 2006;46:462-470.
45. Moffat K, Gillet VJ, Whittle M, et al. A comparison of field-based similarity searching methods: CatShape, FBSS, and ROCS. J Chem Info Model 2008;48:719-729.
46. Muchmore SW, Debe DA, Metz JT, et al. Application of belief theory to similarity data fusion for use in analog searching and lead hopping. J Chem Info Model 2008;48:941-948.
47. Bender A, Jenkins JL, Scheiber J, et al. How similar are similarity searching methods? A principal component analysis of molecular descriptor space. J Chem Info Model 2009;49:108-119.
48. Nicholls A, McGaughey GB, Sheridan RP, et al. Molecular shape and medicinal chemistry: A perspective. J Med Chem 2010;53:3862.
49. Pearson R. Exploring Data in Engineering, the Sciences, and Medicine. New York: Oxford University Press; 2011. p 792.
50. Nicholls A. What do we know and when do we know it? J Comput Aided Mol Des 2008;22:239-255.
51. Jain AN, Nicholls A. Recommendations for evaluation of computational methods. J Comput Aided Mol Des 2008;22:133-139.
52. Hawkins PCD, Warren GL, Skillman AG, et al. How to do an evaluation: Pitfalls and traps. J Comput Aided Mol Des 2008;22:179-190.