Evaluating virtual screening methods: Good and bad metrics for the "early recognition" problem

Journal of Chemical Information and Modeling

Volumn 47, Issue 2, 2007, Pages 488-508

  Truchon, Jean François ^a   Bayly, Christopher I ^a  

^a MERCK FROSST CENTRE FOR THERAPEUTIC RESEARCH   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

INDEXING (OF INFORMATION); MEASUREMENT ERRORS; MOLECULAR RECOGNITION; MOLECULAR STRUCTURE; SIGNAL RECEIVERS; STATISTICAL METHODS;

RANKING METHODS; RECEIVER OPERATING CHARACTERISTIC CURVE (ROC); STATISTICAL ERRORS; VIRTUAL SCREENING (VS);

DATABASE SYSTEMS;

ARTICLE; BIOLOGY; DRUG SCREENING; METHODOLOGY; TIME;

COMPUTATIONAL BIOLOGY; DRUG EVALUATION, PRECLINICAL; TIME FACTORS;

EID: 34247272948     PISSN: 15499596     EISSN: 1549960X     Source Type: Journal    
DOI: 10.1021/ci600426e     Document Type: Article

References (27)

1. Chen, H. M.; Lyne, P. D.; Giordanetto, F.; Lovell, T.; Li, J. Evaluating molecular-docking methods for pose prediction and enrichment factors. J. Chem. Inf. Model. 2006, 46, 401-415.
2. Kairys, V.; Fernandes, M. X.; Gilson, M. K. Screening drug-like compounds by docking to homology models: A systematic study. J. Chem. Inf. Model. 2006, 46, 365-379.
3. Klon, A. E.; Glick, M.; Thoma, M.; Acklin, P.; Davies, J. W. Finding more needles in the haystack: A simple and efficient method for improving high-throughput docking results. J. Med. Chem. 2004, 47, 2743-2749.
4. Perola, E.; Walters, W. P.; Charifson, P. S. A detailed comparison of current docking and scoring methods on systems of pharmaceutical relevance. Proteins 2004, 56, 235-249.
5. Schulz-Gasch, T.; Stahl, M. Binding site characteristics in structure-based virtual screening: Evaluation of current docking tools. J. Mol. Model. 2003, 9, 47-57.
6. Muegge, I.; Enyedy, I. J. Virtual screening for kinase targets. Curr. Med. Chem. 2004, 11, 693-707.
7. McGaughey, G. B.; Sheridan, R. P.; Bayly, C. I.; Culberson, J. C.; Kreatsoulas, C.; Lindsley, S.; Maiorov, V.; Truchon, J. F.; Cornell, W. D. Comparison of topological, shape, and docking methods in virtual screening. J. Med. Chem. Submitted for publication.
8. Cornell, W.D. Recent evaluations of high throughput docking methods for pharmaceutical lead finding - Consensus and caveats. In Annual Reports in Computational Chemistry; Spellmeyer, D. C., Ed.; Elsevier: Amsterdam, The Netherlands, 2006; Vol. 2, pp 302-328.
9. Triballeau, N.; Acher, F.; Brabet, I.; Pin, J. P.; Bertrand, H. O. Virtual screening workflow development guided by the "receiver operating characteristic" curve approach. Application to high-throughput docking on metabotropic glutamate receptor subtype 4, J. Med. Chem. 2005, 48, 2534-2547.
10. Edgar, S. J.; Holliday, J. D.; Willett, P. Effectiveness of retrieval in similarity searches of chemical databases: A review of performance measures. J. Mol. Graphics Modell. 2000, 18, 343-357.
11. Cleves, A. E.; Jain, A. N. Robust ligand-based modeling of the biological targets of known drugs. J. Med. Chem. 2006, 49, 2921-2938.
12. Pham, T. A.; Jain, A. N. Parameter Estimation for Scoring Protein-Ligand Interactions Using Negative Training Data. J. Med. Chem. 2005, 47, 6128-6136.
13. Klon, A. E.; Glick, M.; Davies, J. W. Application of machine learning to improve the results of high-throughput docking against the HIV-1 protease. J. Chem. Inf. Comput. Sci. 2004, 44, 2216-2224.
14. Webb, G. I.; Ting, K. M. On the application of ROC analysis to predict classification performance under varying class distributions. Mach. Learn. 2005, 58, 25-32.
15. Drummond, C.; Holte, R. C. Cost curves: An improved method for visualizing classifier performance. Mach. Learn. 2006 [Online].
16. Seifert, M. H. J. Assessing the discriminatory power of scoring functions for virtual screening. J. Chem. Inf. Model. 2006, 46, 1456-1465.
17. Ferrara, P.; Gohlke, H.; Price, D. J.; Klebe, G.; Brooks, C. L. Assessing scoring functions for protein-ligand interactions, J. Med. Chem. 2004, 47, 3032-3047.
18. Halgren, T. A.; Murphy, R. B.; Friesner, R. A.; Beard, H. S.; Frye, L. L.; Pollard, W. T.; Banks, J. L. Glide: A new approach for rapid, accurate docking and scoring. 2. Enrichment factors in database screening. J. Med. Chem. 2004, 47, 1750-1759.
19. Sheridan, R. P.; Singh, S. B.; Fluder, E. M.; Kearsley, S. K. Protocols for bridging the peptide to nonpeptide gap in topological similarity searches. J. Chem. Inf. Comput. Sci. 2001, 41, 1395-1406.
20. Killeen, P. R.; Taylor, T. J. Symmetric receiver operating characteristics. J. Math. Psychol. 2004, 48, 432-434.
21. Swets, J. A. Indexes of Discrimination Or Diagnostic-Accuracy - Their Rocs and Implied Models. Psychol. Bull. 1986, 99, 100-117.
22. Ross, S. M. Introduction to Probability Models; Academic Press: San Diego, CA, 2002.
23. Fawcett, T. An introduction to ROC analysis. Pattern Recognit. Lett. 2006, 27, 861-874.
24. Hanley, J. A.; Mcneil, B. J. The meaning and use of the area under a receiver operating characteristic (Roc) Curve. Radiology 1982, 143, 29-36.
25. Swets, J. A.; Dawes, R. M.; Monahan, J. Better decisions through science. Sci. Am. 2000, 283, 82-87.
26. Swets, J. A. Measuring the Accuracy of Diagnostic Systems. Science 1988, 240, 1285-1293.
27. Bamber, D. The area above the ordinal dominance graph and the area below the receiver operating characteristic graph. J. Math. Psychol. 1975, 12, 387-415.