Impact of benchmark data set topology on the validation of virtual screening methods: Exploration and quantification by spatial statistics

Journal of Chemical Information and Modeling

Volumn 48, Issue 4, 2008, Pages 704-718

  Rohrer, Sebastian G ^a   Baumann, Knut ^a  

^a TECHNICAL UNIVERSITY OF BRAUNSCHWEIG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

BENCHMARKING; LIGANDS; STATISTICAL METHODS; SYSTEMS ANALYSIS;

SPATIAL STATISTICS; VIRTUAL SCREENING;

DATA ACQUISITION;

LIGAND;

ARTICLE; CHEMISTRY; VALIDATION STUDY;

CHEMISTRY, PHARMACEUTICAL; LIGANDS;

EID: 44449122375     PISSN: 15499596     EISSN: 1549960X     Source Type: Journal    
DOI: 10.1021/ci700099u     Document Type: Article

References (63)

1. Böhm, H. J.; Schneider, G. Virtual Screening for Bioactive Molecules; Wiley-VCH: Weinheim, Germany, 2000.
2. Stahura, F. L.; Bajorath, J. New methodologies for ligand-based virtual screening. Curr. Pharm. Des. 2005, 11, 1189-1202.
3. Kitchen, D. B.; Decornez, H.; Furr, J. R.; Bajorath, J. Docking and scoring in virtual screening for drug discovery: methods and applications. Nat. Rev. Drug. Discov. 2004, 3, 935-949.
4. Patterson, D. E.; Cramer, R. D.; Ferguson, A. M.; Clark, R. D.; Weinberger, L. E. Neighborhood behavior: a useful concept for validation of "molecular diversity" descriptors. J. Med. Chem. 1996, 39, 3049-3059.
5. Martin, Y. C.; Kofron, J. L.; Traphagen, L. M. Do structurally similar molecules have similar biological activity. J. Med. Chem. 2002, 45, 4350-4358.
6. Willett, P.; Barnard, J. M.; Downs, G. M. Chemical similarity searching. J. Chem. Inf. Comput. Sci. 1998, 38, 983-996.
7. Brown, R. D.; Martin, Y. C. Use of Structure-Activity Data To Compare Structure-Based Clustering Methods and Descriptors for Use in Compound Selection. J. Chem. Inf. Comput. Sci. 1996, 36, 572-584.
8. Brown, R. D.; Martin, Y. C. The information content of 2D and 3D structural descriptors relevant to ligand-receptor binding. J. Chem. Inf. Comput. Sci. 1997, 37, 1-9.
9. Hert, J.; Willett, P.; Wilton, D. J.; Acklin, P.; Azzaoui, K.; Jacoby, E.; Schuffenhauer, A. Comparison of fingerprint-based methods for virtual screening using multiple bioactive reference structures. J. Chem. Inf. Comput. Sci. 2004, 44, 1177-1185.
10. Hert, J.; Willett, P.; Wilton, D. J.; Acklin, P.; Azzaoui, K.; Jacoby, E.; Schuffenhauer, A. Comparison of topological descriptors for similarity-based virtual screening using multiple bioactive reference structures. Org. Biomol. Chem. 2004, 2, 3256-3266.
11. Pearlman, D. A.; Charifson, P. S. Improved Scoring of Ligand-Protein Interactions Using OWFEG Free Energy Grids. J. Med. Chem. 2001, 44, 502-511.
12. Bender, A.; Glen, R. C. A discussion of measures of enrichment in virtual screening: comparing the information content of descriptors with increasing levels of sophistication. J. Chem. Inf. Model. 2005, 45, 1369-1375.
13. Truchon, J.-F.; Bayly, C. I. Evaluating virtual screening methods: good and bad metrics for the "early recognition" problem. J. Chem. Inf. Model. 2007, 47, 488-508.
14. Sheridan, R. P.; Singh, S. B.; Fluder, E. M.; Kearsley, S. K. Protocols for Bridging the Peptide to Nonpeptide Gap in Topological Similarity Searches. J. Chem. Inf. Model. 2001, 41, 1395-1406.
15. Cleves, A. E.; Jain, A. N. Robust Ligand-Based Modeling of the Biological Targets of Known Drugs. J. Med. Chem. 2006, 49, 2921-2938.
16. Pham, T. A.; Jain, A. N. Parameter Estimation for Scoring Protein-Ligand Interactions Using Negative Training Data. J. Med. Chem. 2006, 49, 5856-5868.
17. Klon, A. E.; Glick, M.; Davies, J. W. Application of Machine Learning To Improve the Results of High-Throughput Docking Against the HIV-1 Protease. J. Chem. Inf. Model. 2004, 44, 2216-2224.
18. Triballeau, N.; Acher, F.; Brabet, I.; Pin, J. P.; Bertrand, H. O. Virtual Screening Workflow Development Guided by the "Receiver Operating Characteristic" Curve Approach. Application to High-Throughput Docking on Metabotropic Glutamate Receptor Subtype 4. J. Med. Chem. 2005, 48, 2534-2547.
19. MDL Drug Data Report (MDDR); Elsevier MDL: San Ramon, CA, 2005.
20. Birchall, K.; Gillet, V. J.; Harper, G.; Pickett, S. D. Training similarity measures for specific activities: application to reduced graphs. J. Chem. Inf. Model. 2006, 46, 577-586.
21. Barker, E. J.; Buttar, D.; Cosgrove, D. A.; Gardiner, E. J.; Kitts, P.; Willett, P.; Gillet, V. J. Scaffold hopping using clique detection applied to reduced graphs. J. Chem. Inf. Model. 2006, 46, 503-511.
22. Stiefl, N.; Watson, I. A.; Baumann, K.; Zaliani, A. ErG: 2D Pharmacophore Descriptors for Scaffold Hopping. J. Chem. Inf. Model. 2006, 46, 208-220.
23. Bissantz, C.; Folkers, G.; Rognan, D. Protein-based virtual screening of chemical databases. 1. Evaluation of different docking/scoring combinations. J. Med. Chem. 2000, 43, 4759-4767.
24. Yang, J.-M.; Chen, Y.-F.; Shen, T.-W.; Kristal, B.; Hsu, D. Consensus Scoring Criteria for Improving Enrichment in Virtual Screening. J. Chem. Inf. Model. 2005, 45, 1134-1146.
25. Bender, A.; Mussa, H. Y.; Glen, R. C.; Reiling, S. Similarity searching of chemical databases using atom environment descriptors (MOL-PRINT 2D): evaluation of performance. J. Chem. Inf. Comput. Sci. 2004, 44, 1708-1718.
26. Harper, G.; Bravi, G. S.; Pickett, S. D.; Hussain, J.; Green, D. V. S. The reduced graph descriptor in virtual screening and data-driven clustering of high-throughput screening data. J. Chem. Inf. Comput. Sci. 2004, 44, 2145-2156.
27. Jones, G.; Willett, P.; Glen, R. C. Molecular recognition of receptor sites using a genetic algorithm with a description of desolvation. J. Mol. Biol. 1995, 245, 43-53.
28. Jones, G.; Willett, P.; Glen, R. C.; Leach, A. R.; Taylor, R. Development and validation of a genetic algorithm for flexible docking. J. Mol. Biol. 1997, 267, 727-748.
29. Verdonk, M. L.; Berdini, V.; Hartshorn, M. J.; Mooij, W. T. M.; Murray, C. W.; Taylor, R. D.; Watson, P. Virtual screening using protein-ligand docking: avoiding artificial enrichment. J. Chem. Inf. Comput. Sci. 2004, 44, 793-806.
30. Huang, N.; Shoichet, B. K.; Irwin, J. J. Benchmarking sets for molecular docking. J. Med. Chem. 2006, 49, 6789-6801.
31. Good, A. C.; Hermsmeier, M. A.; Hindle, S. A. Measuring CAMD technique performance: a virtual screening case study in the design of validation experiments. J. Comput.-Aided. Mol. Des. 2004, 18, 529-536.
32. Good, A. C.; Hermsmeier, M. A. Measuring CAMD technique performance. 2. How "druglike" are drugs? Implications of Random test set selection exemplified using druglikeness classification models. J. Chem. Inf. Model. 2007, 47, 110-114.
33. Vogt, M.; Bajorath, J. Introduction of an information-theoretic method to predict recovery rates of active compounds for Bayesian in silico screening: theory and screening trials. J. Chem. Inf. Model. 2007, 47, 337-341.
34. Vogt, M.; Bajorath, J. Introduction of a Generally Applicable Method to Estimate Retrieval of Active Molecules for Similarity Searching using Fingerprints. ChemMedChem 2007, 2, 1311-1320.
35. Fortin, M.-J.; Dale, M. R. T. Spatial analysis: a guide for ecologists; Cambridge University Press: Cambridge, UK, 2005.
36. Diggle, P. J. Statistical methods for spatial point patterns in ecology. In Spatial and temporal analysis in ecology; Cormack, R. M., Ord, J. K., Eds.; International Cooperative Publishing House: Fairland, MD, 1979; pp 95-150.
37. MOE Molecular Operating Environment, 2002.03; Chemical Computing Group, Inc.: Montreal, Canada, 2002.
38. CTFile Formats; Elsevier MDL: San Ramon, CA, 2005.
39. 3D Structure Generator CORINA: Generation of High-Quality Three-Dimensional Molecular Models; Molecular Networks GmbH Computerchemie: Erlangen, Germany, 2006.
40. Jolliffe, I. T. Principal Component Analysis, 2nd ed.; Springer-Verlag New York, Inc.: New York, 2002.
41. BABELS, 2.2; OpenEye Scientific Software, Inc.: Santa Fe, NM, 2006.
42. FILTER, 2.0.1; OpenEye Scientific Software, Inc.: Santa Fe, NM, 2007.
43. De Aguiar, P. F.; Bourguignon, B.; Khots, M. S.; Massart, D. L.; Phan-Than-Luu, R. D-Optimal designs. Chemom. Intell. Lab. Syst. 1995, 30, 199-210.
44. Johnson, M. E.; Nachtsheim, C. J. Some guidelines for constructing exact D-optimal designs on convex design spaces. Technometrics 1983, 25, 271-277.
45. Olsson, I.-M.; Gottfries, J.; Wold, S. D-optimal onion designs in statistical molecular design. Chemom. Intell. Lab. Syst. 2004, 73, 3746.
46. Olsson, I. M.; Gottfries, J.; Wold, S. Controlling coverage of D-optimal onion designs and selections. J. Chemom. 2004, 18, 548-557.
47. Scott, D. W.; Thompson, J. R. Probability density estimation in higher dimensions, In Interface: Computer Science and Statistics; Proceedings of the Fifteenth Symposium, Houston, TX, 1983; Gentle, J., Ed.; North-Holland: Amsterdam, The Netherlands, 1983; pp 173-179.
48. Matlab 7; The Mathworks: Natick, MA, 2006.
49. Atkinson, A. C.; Donev, A. N. Optimum Experimental Designs; Oxford University Press: Oxford, UK, 1992.
50. Box, G. E. P.; Hunter, J. S.; Hunter, W. G. Statistics for Experimenters: Design, Discovery and Innovation, 2nd ed.; Wiley & Sons: Hoboken, NJ, 2005.
51. Godden, J. W.; Bajorath, J. A distance function for retrieval of active molecules from complex chemical space representations. J. Chem. Inf. Model. 2006, 46, 1094-1097.
52. Kellenberger, E.; Muller, P.; Schalon, C.; Bret, G.; Foata, N.; Rognan, D. sc-PDB: an annotated database of draggable binding sites from the Protein Data Bank. J. Chem. Inf. Model. 2006, 46, 717-727.
53. Klabunde, T.; Hessler, G. Drug design strategies for targeting G-protein-coupled receptors. ChemBioChem. 2002, 3, 928-944.
54. Ripley, B. D. 2nd-Order analysis of stationary point processes. J. Appl. Probab. 1976, 13, 255-266.
55. Guha, R.; Dutta, D.; Jurs, P. C.; Chen, T. R-NN curves: an intuitive approach to outlier detection using a distance based method. J. Chem. Inf. Model. 2006, 46, 1713-22.
56. Guha, R.; Dutta, D.; Wild, D. J.; Chen, T. Counting clusters using R-NN curves. J. Chem. Inf. Model. 2007, 47, 1308-18.
57. Upton, G. J. G.; Fingleton, B. Spatial Data Analysis by Example; Wiley & Sons Ltd: New York, 1985.
58. Breimann, L. Using convex pseudo-data to improve prediction accuracy; Statistics Department, University of California Berkeley: Berkeley, CA, 1998.
59. Kohonen, T. Self-organizing maps; Springer-Verlag New York, Inc.: Secaucus, NJ, 1997.
60. Vesanto, J. Data Mining Techniques Based on the Self-Organizing Map; Helsinki University of Technology: Helsinki, Finland, 1997.
61. Vesanto, J. SOM-Based Data Visualization Methods. Intell. Data Anal. 1999, 2, 111-126.
62. Vesanto, J.; Alhoniemi, E. Clustering of the Self-Organizing Map. IEEE Trans. Neural. Netw. 2000, 11, 586-600.
63. Alhoniemi, E.; Himberg, J.; Parhankangas, J.; Vesanto, J. SOM Toolbox, 2.0; SOM Toolbox Team, Laboratory of Computer and Information Science: Helsinki, Finland, 2005.