Beware of machine learning-based scoring functions-on the danger of developing black boxes

Journal of Chemical Information and Modeling

Volumn 54, Issue 10, 2014, Pages 2807-2815

  Gabel, Joffrey ^a   Desaphy, Jérémy ^a   Rognan, Didier ^a  

^a UNIVERSITÉ DE STRASBOURG   (France)

Author keywords

[No Author keywords available]

Indexed keywords

DECISION TREES; DOCKS; HYDRAULIC STRUCTURES; LEARNING SYSTEMS; LIGANDS; PROTEINS; SUPPORT VECTOR MACHINES; X RAYS;

ATOMIC COORDINATE; BINDING CONSTANT; RAY COORDINATES; RECEPTOR LIGANDS; ROOT MEAN SQUARE DEVIATIONS; SCORING FUNCTIONS; TRAINING MACHINES; VIRTUAL SCREENING;

LEARNING ALGORITHMS;

ENZYME INHIBITOR; LIGAND; MOLECULAR LIBRARY; PROTEIN; PROTEIN BINDING;

ANTAGONISTS AND INHIBITORS; ARTIFACT; BINDING SITE; CHEMICAL DATABASE; CHEMISTRY; COMPUTER INTERFACE; HIGH THROUGHPUT SCREENING; HUMAN; METHODOLOGY; MOLECULAR DOCKING; MOLECULAR DYNAMICS; MOLECULAR LIBRARY; SUPPORT VECTOR MACHINE; THERMODYNAMICS; X RAY CRYSTALLOGRAPHY;

ARTIFACTS; BINDING SITES; CRYSTALLOGRAPHY, X-RAY; DATABASES, CHEMICAL; ENZYME INHIBITORS; HIGH-THROUGHPUT SCREENING ASSAYS; HUMANS; LIGANDS; MOLECULAR DOCKING SIMULATION; MOLECULAR DYNAMICS SIMULATION; PROTEIN BINDING; PROTEINS; RESEARCH DESIGN; SMALL MOLECULE LIBRARIES; SUPPORT VECTOR MACHINES; THERMODYNAMICS; USER-COMPUTER INTERFACE;

EID: 84908242076     PISSN: 15499596     EISSN: 1549960X     Source Type: Journal    
DOI: 10.1021/ci500406k     Document Type: Article

References (34)

1. Bohm, H. J. The development of a simple empirical scoring function to estimate the binding constant for a protein-ligand complex of known three-dimensional structure J. Comput. Aided Mol. Des. 1994, 8, 243-256
2. Damm-Ganamet, K. L.; Smith, R. D.; Dunbar, J. B., Jr.; Stuckey, J. A.; Carlson, H. A. CSAR benchmark exercise 2011-2012: evaluation of results from docking and relative ranking of blinded congeneric series J. Chem. Inf. Model. 2013, 53, 1853-1870
3. Smith, R. D.; Dunbar, J. B., Jr.; Ung, P. M.; Esposito, E. X.; Yang, C. Y.; Wang, S.; Carlson, H. A. CSAR benchmark exercise of 2010: Combined evaluation across all submitted scoring functions J. Chem. Inf. Model. 2011, 51, 2115-2131
4. Plewczynski, D.; Lazniewski, M.; Augustyniak, R.; Ginalski, K. Can we trust docking results? Evaluation of seven commonly used programs on PDBbind database J. Comput. Chem. 2011, 32, 742-755
5. Ripphausen, P.; Nisius, B.; Peltason, L.; Bajorath, J. Quo vadis, virtual screening? A comprehensive survey of prospective applications J. Med. Chem. 2010, 53, 8461-8467
6. Dunbar, J. B., Jr.; Smith, R. D.; Damm-Ganamet, K. L.; Ahmed, A.; Esposito, E. X.; Delproposto, J.; Chinnaswamy, K.; Kang, Y. N.; Kubish, G.; Gestwicki, J. E.; Stuckey, J. A.; Carlson, H. A. CSAR data set release 2012: Ligands, affinities, complexes, and docking decoys J. Chem. Inf. Model. 2013, 53, 1842-1852
7. Dunbar, J. B., Jr.; Smith, R. D.; Yang, C. Y.; Ung, P. M.; Lexa, K. W.; Khazanov, N. A.; Stuckey, J. A.; Wang, S.; Carlson, H. A. CSAR benchmark exercise of 2010: Selection of the protein-ligand complexes J. Chem. Inf. Model. 2011, 51, 2036-2046
8. Sotriffer, C. A.; Sanschagrin, P.; Matter, H.; Klebe, G. SFCscore: Scoring functions for affinity prediction of protein-ligand complexes Proteins 2008, 73, 395-419
9. Hartshorn, M. J.; Verdonk, M. L.; Chessari, G.; Brewerton, S. C.; Mooij, W. T.; Mortenson, P. N.; Murray, C. W. Diverse, high-quality test set for the validation of protein-ligand docking performance J. Med. Chem. 2007, 50, 726-741
10. Wang, R.; Fang, X.; Lu, Y.; Yang, C. Y.; Wang, S. The PDBbind database: Methodologies and updates J. Med. Chem. 2005, 48, 4111-4119
11. Li, L.; Wang, B.; Meroueh, S. O. Support vector regression scoring of receptor-ligand complexes for rank-ordering and virtual screening of chemical libraries J. Chem. Inf. Model. 2011, 51, 2132-2138
12. Muegge, I.; Oloff, S. Advances in virtual screening Drug Discovery Today: Technol. 2006, 3, 405-411
13. Zilian, D.; Sotriffer, C. A. SFCscore(RF): A random forest-based scoring function for improved affinity prediction of protein-ligand complexes J. Chem. Inf. Model. 2013, 53, 1923-1933
14. Ballester, P. J.; Schreyer, A.; Blundell, T. L. Does a more precise chemical description of protein-ligand complexes lead to more accurate prediction of binding affinity? J. Chem. Inf. Model. 2014, 54, 944-955
15. Ballester, P. J.; Mitchell, J. B. A machine learning approach to predicting protein-ligand binding affinity with applications to molecular docking Bioinformatics 2010, 26, 1169-1175
16. Cheng, T.; Li, X.; Li, Y.; Liu, Z.; Wang, R. Comparative assessment of scoring functions on a diverse test set J. Chem. Inf. Model. 2009, 49, 1079-1093
17. Berman, H. M.; Westbrook, J.; Feng, Z.; Gilliland, G.; Bhat, T. N.; Weissig, H.; Shindyalov, I. N.; Bourne, P. E. The Protein Data Bank Nucleic Acids Res. 2000, 28, 235-242
18. http://www.pdbbind-cnr.org/download/CASF-2007.tar.gz (accessed June 2014).
19. SYBYL, version X2.1; Certara: St. Louis, MO, 2012.
20. Mysinger, M. M.; Carchia, M.; Irwin, J. J.; Shoichet, B. K. Directory of useful decoys, enhanced (DUD-E): better ligands and decoys for better benchmarking J. Med. Chem. 2012, 55, 6582-6594
21. Triballeau, N.; Acher, F.; Brabet, I.; Pin, J. P.; Bertrand, H. O. Virtual screening workflow development guided by the "receiver operating characteristic"curve approach. Application to high-throughput docking on metabotropic glutamate receptor subtype 4 J. Med. Chem. 2005, 48, 2534-2547
22. Desaphy, J.; Raimbaud, E.; Ducrot, P.; Rognan, D. Encoding protein-ligand interaction patterns in fingerprints and graphs J. Chem. Inf. Model. 2013, 53, 623-637
23. Liaw, A.; Wiener, M. Classification and regression by randomForest R News 2002, 2, 18-22
24. R Development Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2011, (ISBN 3-900051-07-0).
25. Jain, A. N. Surflex: fully automatic flexible molecular docking using a molecular similarity-based search engine J. Med. Chem. 2003, 46, 499-511
26. Truchon, J. F.; Bayly, C. I. Evaluating virtual screening methods: good and bad metrics for the -early recognition- problem J. Chem. Inf. Model. 2007, 47, 488-508
27. Swamidass, S. J.; Azencott, C. A.; Daily, K.; Baldi, P. A CROC stronger than ROC: measuring, visualizing and optimizing early retrieval Bioinformatics 2010, 26, 1348-1356
28. Cross, J. B.; Thompson, D. C.; Rai, B. K.; Baber, J. C.; Fan, K. Y.; Hu, Y.; Humblet, C. Comparison of several molecular docking programs: pose prediction and virtual screening accuracy J. Chem. Inf. Model. 2009, 49, 1455-1474
29. Ben Nasr, N.; Guillemain, H.; Lagarde, N.; Zagury, J. F.; Montes, M. Multiple structures for virtual ligand screening: Defining binding site properties-based criteria to optimize the selection of the query J. Chem. Inf. Model. 2013, 53, 293-311
30. DesJarlais, R. L.; Seibel, G. L.; Kuntz, I. D.; Furth, P. S.; Alvarez, J. C.; Ortiz de Montellano, P. R.; DeCamp, D. L.; Babe, L. M.; Craik, C. S. Structure-based design of nonpeptide inhibitors specific for the human immunodeficiency virus 1 protease Proc. Natl. Acad. Sci. U.S.A 1990, 87, 6644-6648
31. Rutenber, E.; Fauman, E. B.; Keenan, R. J.; Fong, S.; Furth, P. S.; Ortiz de Montellano, P. R.; Meng, E.; Kuntz, I. D.; DeCamp, D. L.; Salto, R. Structure of a non-peptide inhibitor complexed with HIV-1 protease. Developing a cycle of structure-based drug design J. Biol. Chem. 1993, 268, 15343-15346
32. Muegge, I.; Podlogar, B. L. 3D-Quantitative Structure Activity Relationships of Biphenyl Carboxylic Acid MMP-3 Inhibitors: Exploring Automated Docking as Alignment Method Quant. Struct.-Act. Relat. 2001, 20, 215-222
33. 2! J. Mol. Graph. Model. 2002, 20, 269-276
34. Li, Y.; Liu, Z.; Li, J.; Han, L.; Liu, J.; Zhao, Z.; Wang, R. Comparative assessment of scoring functions on an updated benchmark: 1. Compilation of the test set J. Chem. Inf. Model. 2014, 54, 1700-1716