Comprehensive comparison of ligand-based virtual screening tools against the DUD data set reveals limitations of current 3D methods

Journal of Chemical Information and Modeling

Volumn 50, Issue 12, 2010, Pages 2079-2093

  Venkatraman, Vishwesh ^a   Pérez Nueno, Violeta I ^a   Mavridis, Lazaros ^a   Ritchie, David W ^a  

^a LORIA   (France)

Author keywords

[No Author keywords available]

Indexed keywords

BIOACTIVITY; VIRTUAL REALITY;

COMPREHENSIVE COMPARISONS; CORRELATION ANALYSIS; DOCKING ALGORITHMS; MOLECULAR REPRESENTATIONS; MOLECULAR SHAPES; SCORING FUNCTIONS; SIGNIFICANCE TEST; VIRTUAL SCREENING;

LIGANDS;

LIGAND;

ARTICLE; CHEMICAL STRUCTURE; COMPARATIVE STUDY; COMPUTER INTERFACE; CONFORMATION; DRUG SCREENING; FACTUAL DATABASE; HUMAN; METHODOLOGY; RECEIVER OPERATING CHARACTERISTIC;

DATABASES, FACTUAL; DRUG EVALUATION, PRECLINICAL; HUMANS; LIGANDS; MODELS, MOLECULAR; MOLECULAR CONFORMATION; ROC CURVE; USER-COMPUTER INTERFACE;

EID: 78650689245     PISSN: 15499596     EISSN: 1549960X     Source Type: Journal    
DOI: 10.1021/ci100263p     Document Type: Article

References (55)

1. Geppert, H.; Vogt, M.; Bajorath, J. Current Trends in Ligand-Based Virtual Screening: Molecular Representations, Data Mining Methods, New Application Areas, and Performance Evaluation J. Chem. Inf. Model. 2010, 50, 205-216
2. Eckert, H.; Bajorath, J. Molecular Similarity Analysis in Virtual Screening: Foundations, Limitations and Novel Approaches Drug Discovery Today 2007, 12, 225-233
3. Putta, S.; Beroza, P. Shapes of Things: Computer Modeling of Molecular Shape in Drug Discovery Curr. Top. Med. Chem. 2007, 7, 1514-1524
4. Brown, R. D.; Martin, Y. C. Use of Structure-Activity Data To Compare Structure-Based Clustering Methods and Descriptors for Use in Compound Selection J. Chem. Inf. Model. 1996, 36, 572-584
5. Schuffenhauer, A.; Gillet, V. J.; Willett, P. Similarity Searching in Files of Three-Dimensional Chemical Structures: Analysis of the BIOSTER Database Using Two-Dimensional Finger-prints and Molecular Field Descriptors J. Chem. Inf. Model. 2000, 40, 295-307
6. Nettles, J. H.; Jenkins, J. L.; Bender, A.; Deng, Z.; Davies, J. W.; Glick, M. Bridging Chemical and Biological Space: Target Fishing Using 2D and 3D Molecular Descriptors J. Med. Chem. 2006, 49, 6802-6810
7. Verdonk, M. L.; Berdini, V.; Hartshorn, M. J.; Mooij, W. T. M.; Murray, C. W.; Taylor, R. D.; Watson, P. Virtual Screening Using Protein-Ligand Docking: Avoiding Artificial Enrichment J. Chem. Inf. Model. 2004, 44, 793-806
8. Kellenberger, E.; Rodrigo, J.; Muller, P.; Rognan, D. Comparative Evaluation of Eight Docking Tools for Docking and Virtual Screening Accuracy Proteins: Struct., Func., Bioinf. 2004, 57, 225-242
9. Cross, J. B.; Thompson, D. C.; Rai, B. K.; Baber, J. C.; Fan, K. Y.; Hu, Y.; Humblet, C. Comparison of Several Molecular Docking Programs: Pose Prediction and Virtual Screening Accuracy J. Chem. Inf. Model. 2009, 49, 1455-1474
10. Deng, W.; Verlinde, C. L.; M., J. Evaluation of Different Virtual Screening Programs for Docking in a Charged Binding Pocket J. Chem. Inf. Model. 2008, 48, 2010-2020
11. Tiikkainen, P.; Markt, P.; Wolber, G.; Kirchmair, J.; Distinto, S.; Poso, A.; Kallioniemi, O. Critical Comparison of Virtual Screening Methods against the MUV Data Set J. Chem. Inf. Model. 2009, 49, 2168-2178
12. Cheeseright, T. J.; Mackey, M. D.; Melville, J. L.; Vinter, J. G. FieldScreen: Virtual Screening Using Molecular Fields. Application to the DUD Data Set J. Chem. Inf. Model. 2008, 48, 2108-2117
13. McGaughey, G. B.; Sheridan, R. P.; Bayly, C. I.; Culberson, J. C.; Kreatsoulas, C.; Lindsley, S.; Maiorov, V.; Truchon, J.; Cornell, W. D. Comparison of Topological, Shape, and Docking Methods in Virtual Screening J. Chem. Inf. Model. 2007, 47, 1504-1519
14. Hawkins, P. C. D.; Skillman, A. G.; Nicholls, A. Comparison of Shape-Matching and Docking as Virtual Screening Tools J. Med. Chem. 2007, 50, 74-82
15. Huang, N.; Shoichet, B. K.; Irwin, J. J. Benchmarking Sets for Molecular Docking J. Med. Chem. 2006, 49, 6789-6801
16. Rohrer, S. G.; Baumann, K. Maximum Unbiased Validation (MUV) Data Sets for Virtual Screening Based on PubChem Bioactivity Data J. Chem. Inf. Model. 2009, 49, 169-184
17. Lee, H. S.; Lee, C. S.; Kim, J. S.; Kim, D. H.; Choe, H. Improving Virtual Screening Per-formance against Conformational Variations of Receptors by Shape Matching with Ligand Binding Pocket J. Chem. Inf. Model. 2009, 49, 2419-2428
18. von Korff, M.; Freyss, J.; Sander, T. Comparison of Ligand- and Structure-Based Virtual Screening on the DUD Data Set J. Chem. Inf. Model. 2009, 49, 209-231
19. Irwin, J. J. Community Benchmarks for Virtual Screening J. Comput-Aided Mol. Des. 2008, 22, 193-199
20. Good, A. C.; Oprea, T. I. Optimization of CAMD Techniques 3. Virtual Screening Enrichment Studies: A Help or Hindrance in Tool Selection? J. Comput.-Aided Mol. Des. 2008, 22, 169-178
21. Vainio, M. J.; Puranen, J. S.; Johnson, M. S. ShaEP: Molecular Overlay Based on Shape and Electrostatic Potential J. Chem. Inf. Model. 2009, 49, 492-502
22. Jahn, A.; Hinselmann, G.; Fechner, N.; Zell, A. Optimal Assignment Methods for Ligand-Based Virtual Screening J. Cheminf. 2009, 1, 14
23. Kinnings, S. L.; Jackson, R. M. LigMatch: A Multiple Structure-Based Ligand Matching Method for 3D Virtual Screening J. Chem. Inf. Model. 2009, 49, 2056-2066
24. Giganti, D.; Guillemain, H.; Spadoni, J.; Nilges, M.; Zagury, J.; Montes, M. Comparative Evaluation of 3D Virtual Ligand Screening Methods: Impact of the Molecular Alignment on Enrichment J. Chem. Inf. Model. 2010, 50, 992-1004
25. Nicholls, A. What Do We Know and When Do We Know It? J. Comput.-Aided Mol. Des. 2008, 22, 239-255
26. Kirchmair, J.; Distinto, S.; Markt, P.; Schuster, D.; Spitzer, G. M.; Liedl, K. R.; Wolber, G. How To Optimize Shape-Based Virtual Screening: Choosing the Right Query and Including Chemical Information J. Chem. Inf. Model. 2009, 49, 678-692
27. The Open Babel Package, version 2.2.3, http://openbabel.org/wiki/Main- Page/ (accessed December 6, 2009).
28. Barnard, J. M.; Downs, G. M. Chemical Fragment Generation and Clustering Softwares J. Chem. Inf. Model. 1997, 37, 141-142
29. Molecular Operating Environment (MOE), version 2008, release 10; Chemical Computing Group: Montreal, Canada, 2009.
30. Daylight Version 4.62; Daylight Chemical Information Systems, Inc.: CA, U.S.A., 1999.
31. Bender, A.; Mussa, H. Y.; Glen, R. C.; Reiling, S. Similarity Searching of Chemical Databases Using Atom Environment Descriptors (MOLPRINT 2D): Evaluation of Performance J. Chem. Inf. Model. 2004, 44, 1708-1718
32. Grant, J. A.; Gallardo, M. A.; Pickup, B. T. a Fast Method of Molecular Shape Comparison: A Simple Application of a Gaussian Description of Molecular Shape J. Comput. Chem. 1996, 17, 1653-1666
33. Lin, J.; Clark, T. An Analytical, Variable Resolution, Complete Description of Static Molecules and Their Intermolecular Binding Properties J. Chem. Inf. Model. 2005, 45, 1010-1016
34. Ballester, P. J.; Richards, W. G. Ultrafast Shape Recognition to Search Compound Databases for Similar Molecular Shapes J. Comput. Chem. 2007, 28, 1711-1723
35. Mavridis, L.; Hudson, B. D.; Ritchie, D. W. Toward High Throughput 3D Virtual Screening Using Spherical Harmonic Surface Representations J. Chem. Inf. Model. 2007, 47, 1787-1796
36. Kirchmair, J.; Markt, P.; Distinto, S.; Wolber, G.; Langer, T. Evaluation of the Performance of 3D Virtual Screening Protocols: RMSD Comparisons, Enrichment Assessments, And Decoy Selection? What Can We Learn from Earlier Mistakes? J. Comput.-Aided Mol. Des. 2008, 22, 213-228
37. Barker, E. J.; Gardiner, E. J.; Gillet, V. J.; Kitts, P.; Morris, J. Further Development of Reduced Graphs for Identifying Bioactive Compounds J. Chem. Inf. Model. 2003, 43, 346-356
38. Mackey, M. D.; Melville, J. L. Better than Random? The Chemotype Enrichment Problem J. Chem. Inf. Model. 2009, 49, 1154-1162
39. Truchon, J.; Bayly, C. I. Evaluating Virtual Screening Methods: Good and Bad Metrics for the "Early Recognition" Problem J. Chem. Inf. Model. 2007, 47, 488-508
40. Zhao, W.; Hevener, K.; White, S.; Lee, R.; Boyett, J. A Statistical Framework to Evaluate Virtual Screening BMC Bioinf. 2009, 10, 225
41. Sheridan, R. P. Alternative Global Goodness Metrics and Sensitivity Analysis: Heuristics to Check the Robustness of Conclusions from Studies Comparing Virtual Screening Methods J. Chem. Inf. Model. 2008, 48, 426-433
42. Swamidass, S. J.; Azencott, C.-A.; Daily, K.; Baldi, P. A CROC Stronger than ROC: Measuring, Visualizing and Optimizing Early Retrieval Bioinformatics 2010, 26, 1348-1356
43. Hanley, J. A.; McNeil, B. J. The Meaning and Use of the Area under a Receiver Operating Characteristic (ROC) Curve Radiology 1982, 143, 29-36
44. Fawcett, T. An Introduction to ROC Analysis Pattern Recogn. Lett. 2006, 27, 861-874
45. Hawkins, P. C. D.; Warren, G. L.; Skillman, A. G.; Nicholls, A. How to Do an Evaluation: Pitfalls and Traps J. Comput.-Aided Mol. Des. 2008, 22, 179-190
46. Macskassy, S. A.; Provost, F.; Rosset, S. ROC Confidence Bands: an Empirical Evaluation. ICML ′05 In Proceedings of the 22nd International Conference on Machine Learning, Bonn, Germany, 2005; De Raedt, L.; Wrobel, S., Eds.; ACM: New York, 2005; pp 537 - 544.
47. Provost, F. J.; Fawcett, T.; Kohavi, R. The Case against Accuracy Estimation for Comparing Induction Algorithms. In Proceedings of the Fifteenth International Conference on Machine Learning, Madison, Wisconsin, USA, 1998; Shavlik, J. W., Ed.; Morgan Kaufmann Publishers Inc.: San Francisco, CA, 1998; pp 445 - 453.
48. Böstrom, J.; Greenwood, J. R.; Gottfries, J. Assessing the Performance of OMEGA with Respect to Retrieving Bioactive Conformations J. Mol. Graphics. Modell. 2003, 21, 449-462
49. Ritchie, D. W.; Kemp, G. J. Protein Docking Using Spherical Polar Fourier Correlations Proteins: Struct., Funct., Genet. 2000, 39, 178-194
50. Nicholls, A.; McGaughey, G. B.; Sheridan, R. P.; Good, A. C.; Warren, G.; Mathieu, M.; Muchmore, S. W.; Brown, S. P.; Grant, J. A.; Haigh, J. A.; Nevins, N.; Jain, A. N.; Kelley, B. Molecular Shape and Medicinal Chemistry: A Perspective J. Med. Chem. 2010, 53, 3862-3886
51. Sato, H.; Shewchuk, L. M.; Tang, J. Prediction of Multiple Binding Modes of the CDK2 Inhibitors, Anilinopyrazoles, Using the Automated Docking Programs GOLD, FlexX, and Ligand Fit: An Evaluation of Performance J. Chem. Inf. Model. 2006, 46, 2552-2562
52. Pargellis, C.; Tong, L.; Churchill, L.; Cirillo, P. F.; Gilmore, T.; Graham, A. G.; Grob, P. M.; Hickey, E. R.; Moss, N.; Pav, S.; Regan, J. Inhibition of p38 MAP Kinase by Utilizing a Novel Allosteric Binding Site Nat. Struct. Biol. 2002, 9, 268-272
53. Steuber, H.; Zentgraf, M.; Motta, C. L.; Sartini, S.; Heine, A.; Klebe, G. Evidence for a Novel Binding Site Conformer of Aldose Reductase in Ligand-Bound State J. Mol. Biol. 2007, 369, 186-197
54. Pérez-Nueno, V. I.; Ritchie, D. W.; Borrell, J. I.; Teixidó, J. Clustering and Classifying Di-verse HIV Entry Inhibitors Using a Novel Consensus Shape-Based Virtual Screening Approach: Further Evidence for Multiple Binding Sites within the CCR5 Extracellular Pocket J. Chem. Inf. Model. 2008, 48, 2146-2165
55. Baber, J. C.; Shirley, W. A.; Gao, Y.; Feher, M. The Use of Consensus Scoring in Ligand-Based Virtual Screening J. Chem. Inf. Model. 2006, 46, 277-288 (Pubitemid 43285403)