The importance of being earnest: Validation is the absolute essential for successful application and interpretation of QSPR models

QSAR and Combinatorial Science

Volumn 22, Issue 1, 2003, Pages 69-77

  Tropsha, Alexander ^a   Gramatica, Paola ^b   Gombar, Vijay K ^c  

^a UNIVERSITY OF NORTH CAROLINA   (United States)

^b UNIVERSITY OF INSUBRIA   (Italy)

^c GLAXOSMITHKLINE   (United States)

Author keywords

Applicability domain; Model validation; QSAR; Structure property relationship modeling

Indexed keywords

COMPUTATIONAL CHEMISTRY;

APPLICABILITY DOMAIN; INTEGRAL COMPONENTS; MODEL VALIDATION; QSAR; QUANTITATIVE STRUCTURE-PROPERTY RELATIONSHIP MODELS; RELATIONSHIP MODEL; RIGOROUS VALIDATION; STRUCTURE-PROPERTIES RELATIONSHIPS; STRUCTURE-PROPERTY RELATIONSHIP MODELING; TRAINING SETS;

STATISTICAL TESTS;

ACCURACY; CONFERENCE PAPER; MODEL; PRIORITY JOURNAL; QUANTITATIVE STRUCTURE PROPERTY RELATION; SCREENING; STATISTICAL ANALYSIS; VALIDATION PROCESS;

EID: 0038724207     PISSN: 1611020X     EISSN: None     Source Type: Journal    
DOI: 10.1002/qsar.200390007     Document Type: Conference Paper

References (71)

1. Walker J. D. (Ed.), Handbook on Quantitative Structure Activity Relationships (QSARs) for Pollution Prevention, Toxicity Screening, Risk Assessment and World Wide Web Applications, SETAC Press, Pensacola, FL 2002.
2. Walker J. D. (Ed.), Handbook on Quantitative Structure Activity Relationships (QSARs) for Predicting Physical Properties, Bioconcentration Potential and Environmental Fate of Chemicals, SETAC Press, Pensacola, FL 2002.
3. Walker J. D. (Ed.), Handbook on Quantitative Structure Activity Relationships (QSARs) for Predicting Endocrine Disruption Potential of Chemicals, SETAC Press, Pensacola, FL 2002.
4. Walker J. D. (Ed.), Handbook on Quantitative Structure Activity Relationships (QSARs) for Predicting Effects of Chemicals on Environmental-Human Health Interactions, SETAC Press, Pensacola, FL 2002.
5. Walker J. D. (Ed.), Handbook on Quantitative Structure Activity Relationships (QSARs) for Predicting Ecological Effects of Chemicals, SETAC Press, Pensacola, FL 2002.
6. Cronin, M. T. D., Dearden, J. C., Duffy, J. C., Edwards, R., Manga, N., Worth, A. P., and Worgan, A. D. P., The importance of hydrophobicity and Electrophilicity Descriptors in mechanistically-based QSARs for Toxicological Endpoints, SAR QSAR Environ. Res. 13, 167-176 (2002).
7. Moss, G. P., Dearden, J. C., Patel, H., and Cronin, M. T. D., Quantitative Structure-Permeability Relationships (QSPRs) for percutaneous absorption, Toxicol. in Vitro 16, 299-317 (2002).
8. Trohalaki, S., Gifford, E., and Pachter, R., Improved QSARs for predictive toxicology of halogenated hydrocarbons, Comput. Chem. 24, 421-427 (2000).
9. Schultz, T. W., Bowerrs, G. S., and Cronin, M. T. D., Structure-toxicity relationships for four classes of aliphatic electrophiles to Vibrio fischeri, Marine Environ. Res. 50, 61-81 (2000).
10. Schultz, T. W., Aptula, A. O., Netzeva, T. I., and Cronin, M. T. D., Quantitative Structure- Activity Relationships for the Toxicity of AliphaticCompounds to Tetrahymena pyriformis. A Mechanism of Action Approach, Presented at the QSAR 2002 meeting, May 25-29, Ottawa, Canada (2002).
11. Akers, K. S., Sinks, G. D., and Schultz, T. W., Structure-toxicity relationships for selected halogenated aliphatic chemicals, Environ. Toxicol. Pharm. 7, 33-39 (1999).
12. Benigni R., Giuliani, A., Franke, R.: and Gruska, A., Quantiative-Structure-Activity Relationships of mutagenic and Carcinogenic Aromatic Amines, Chem. Rev. 100, 3697-3714 (2000).
13. Devillers, J., QSAR Modeling of Large Heterogeneous Sets of Molecules, SAR QSAR Environ. Res. 12, 515-528 (2001).
14. Kaiser, K. L. E., Dearden, J. C., Klein, W., and Schultz, T. W., A Note of Caution to Users of ECOSAR, Water Qual. Res. J. Canada 34, 179-182 (1999).
15. ECOSAR, Version 0.99f, Jan 2000, (http://www.epa.gov/oppt/ newchems/21ecosar.htm).
16. Rousseeuw, P. J., and Leroy, A. M., Robust Regression and Outlier Detection, John Wiley & Sons, New York 1987.
17. Belsley, D. A., Kuh, E., and Welsch, R. E., Regression Diagnostics - Identifying Influential Data and Sources of collinearity, John Wiley & Sons, New York 1980.
18. Gironés, X., Gallegos, A., and Ramon, C.-D., Modeling Antimalarial Activity: Application of Kinetic Energy Density Quantum similarity Measures as Descriptors in QSAR, J. Chem Inf. Comput. Sci. 46, 1400-1407 (2000).
19. Bordás, B., Kömíves, T., Szántó, Z., and Lopata, A., Comparative Three-Dimensional Quantitative Structure-Activity Relationship Study of Safeners and Herbicides, J. Agricult. Food Chem. 48, 926-931 (2000).
20. Fan, Y., Shi, L. M., Kohn, K. W., Pommier, Y., and Weinstein, J. N., Quantitative Structure-Antitumor Activity Relationships of Camptothecin Analogues: Cluster Analysis and Genetic Algorithm-Based Studies, J. Med. Chem. 44, 3254-3263 (2001).
21. Randić, M., and Basak, S. C., Construction of high-quality structure-property-activity regressions: the boiling points of sulfides, J. Chem. Inf. Comput. Sci. 40, 899-905 (2000).
22. Suzuki, T., Ide, K., Ishida, M., and Shapiro, S., Classification of Environmental Estrogens by Physicochemical Properties Using Principal Component Analysis and Hierarchical Cluster Analysis, J. Chem. Inf Comput. Sci. 41, 718-726 (2001).
23. Basak, S. C., and Mills, D., Prediction of mutagenicity utilizing a hierarchical QSAR approach, SAR QSAR Environ. Res. 12, 481-496 (2001).
24. Wang, X., Yin, C., and Wang L., Structure-activity relationships and response-surface analysis of nitroaromatics toxicity to the yeast (Saccharomyces cerevisiae), Chemosphere 46, 1045-1051 (2002).
25. Kubinyi, H., Hamprecht, F. A., and Mietzner, T., Three-Dimensional Quantitative Similarity-Activity Relationships (3D QSiAR) from SEAL Similarity Matrices, J. Med. Chem. 41, 2553-2564 (1998).
26. Golbraikh, A., and Tropsha, A., Beware of q2!, J. Mol. Graph. Model. 20, 269-276 (2002).
27. Cramer III, R. D., Patterson, D. E., and Bunce, J. D., Comparative molecular field analysis (CoMFA). 1. Effect of shape on binding of steroids to carrier proteins, J. Amer. Chem. Soc. 110, 5959-5967 (1988).
28. Coats, E. A., The CoMFA steroids as a benchmark data set for development of 3D QSAR methods, in: Kubinyi, H., Folkers, G., and Martin, Y. C. (Eds.), 3D QSAR in Drug Design. V.3., Kluwer/ESCOM, Dordrecht (The Netherlands) 1998, pp. 199-213.
29. Novellino, E., Fattorusso, C., and Greco, G., Use of Comparative Molecular Field Analysis and Cluster Analysis in Series Design, Pharm. Acta Helv. 70, 149-154 (1995).
30. Zheng, W., and Tropsha, A., Novel Variable Selection Quantitative Structure-Property Relationship Approach Based on the k-Nearest-Neighbor Principle, J. Chem. Inform. Comput. Sci. 40, 185-194 (2000).
31. Wehrens, R., Putter, H., and Buydens, L. M. C., The bootstrap: a tutorial, Chemom. Intell. Lab. Systems 54, 35-52 (2002).
32. Efron, B., and Tibshirani, R. J., An Introduction to the Bootstrap, Chapman & Hall, New York 1993.
33. Wold, S., and Eriksson, L., Statistical Validation of QSAR Results, in: van de Waterbeemd H., (Ed.), Chemometrics Methods in Molecular Design, VCH, Weinheim (Germany) 1995, pp. 309-318.
34. Clark, R. D., Sprous, D. G., and Leonard, J. M., Validating Models Based on Large Dataset, in: Höltje, H.-D., and Sippl, W. (Eds.), Rational Approaches to Drug Design, Proceedings of the 13th European Symposium on Quantitative Structure-Activity Relationship, Aug 27-Sept 1, 2000, Prous Science Düsseldorf (Germany) 2001, pp. 475-485.
35. Wilcox, R. E., Huang, W.-H., Brusniak, M.-Y. K., Wilcox, D. M., Pearlman, R. S., Teeter, M. M., DuRand, C. J., Wiens, B. L., and Neve, K. A., CoMFA-Based Prediction of Agonist Affinities at Recombinant Wild Type versus Serine to Alanine Point Mutated D2 Dopamine Receptors, J. Med. Chem. 43, 3005-3019 (2000).
36. Wu, W., Walczak, B., Massart, D. L., Heuerding, S., Erni, F., Last, I. R., and Prebble, K. A., Artificial Neural Networks in Classification of NIR Spectral Data: Design of the Training Set, Chemometr. Intell. Lab. Syst. 33, 35-46 (1996).
37. Yasri, A., and Hartsough, D., Toward an Optimal Procedure for Variable Selection and QSAR Model Building, J. Chem. Inf. Comput. Sci. 41, 1218-1227 (2001).
38. Kauffman, G. V., and Jurs, P. C., QSAR and k-Nearest Neighbor Classification Analysis of Selective Cyclooxygenase-2 Inhibitors Using Topologically-Based Numerical Descriptors, J. Chem. Inf. Comput. Sci. 41, 1553-1560 (2001).
39. Mattioni, B. E., and Jurs, P. C., Development of Quantitative Structure-Activity Relationship and Classification Models for a Set of Carbonic Anhydrase Inhibitors, J. Chem. Inf. Comput. Sci. 42, 94-102 (2002).
40. Burden, F. R., and Winkler, D. A., Robust QSAR Models Using Bayesian Regularized Artificial Neural Networks, J. Med. Chem. 42, 3183-3187 (1999).
41. Burden, F. R., Ford, M. G., Whitley, D. C, and Winkler, D. A., Use of automatic relevance determination in QSAR studies using Bayesian neural networks, J. Chem. Inf. Comput. Sci. 40, 1423-1430 (2000).
42. Adams, M. J., Chemometrics in Analytical Spectroscopy, The Royal Society of Chemistry, Cambridge (UK) 1995.
43. Potter, T., and Matter, H., Random or Rational Design? Evaluation of Diverse Compound Subsets from Chemical Structure Databases, J. Med. Chem. 41, 478-488 (1998).
44. Lajiness, M., Johnson, M. A., and Maggiora, G. M., Implementing Drug Screening Programs Using Molecular Similarity Methods, in: F. J. Lauchere (Ed.), QSAR: Quantitative Structure-Activity Relationships in Drug Design., Alan R. Liss Inc., New York 1989, pp. 173-176.
45. Taylor, R., Simulation Analysis of Experimental Design Strategies for Screening Random Compounds as Potential New Drugs and Agrochemicals, J. Chem. Inf. Comput. Sci. 35, 59-67 (1995).
46. Snarey, M., Terrett, N. K., Willett, P., and Wilton, D. J., Comparison of Algorithms for Dissimilarity-Based Compound Selection, J. Mol. Graph. Model. 15, 372-385 (1997).
47. Gasteiger, J., and Zupan, J., Neural Networks in Chemistry, Angew. Chem. Int. Ed. Engl. 32, 503-527 (1993).
48. Gramatica, P., Consonni, V., and Todeschini, R., QSAR Study on the Tropospheric Degradation of Organic Compounds, Chemosphere 38, 1371-1378 (1999).
49. Kennard, R. W., and Stone, L. A., Computer Aided Design of Experiments, Technometrics 11, 137-148 (1969).
50. Bourguignon, B., Deaguiar, P. F., Thorre, K., and Massart, D. L., Application Of Nonlinear-Regression Functions For The Modeling Of Retention In Reversed-Phase Lc, J. Chromatogr. Sci. 32, 144-152 (1994).
51. Bourguignon, B., Deaguiar, P. F., Khots, M. S., and Massart, D. L., Optimization in Irregularly Shaped Regions - PH and Solvent Strength in Reversed-Phase High-Performance Liquid-Chromatography Separations, Anal. Chem. 66, 893-904 (1994).
52. Sjöström, M., and Eriksson, L., Applications of Statistical Experimental Design, in: van de Waterbeemd H., (Ed.), Chemometrics Methods in Molecular Design, VCH, Weinheim (Germany) 1995, pp. 63-90.
53. Eriksson, L., and Johansson, E., Multivariate Design and Modeling in QSAR. Tutorial. Chemometr. Intell. Lab. Syst. 34, 1-19 (1996).
54. Carlson, R., Design and Optimization in Organic Synthesis, Elsevier, Dordrecht (The Netherlands) 1992.
55. Martin, E.J., and Critchlow, R. E., Beyond Mere Diversity: Tailoring Combinatorial Libraries for Drug Discovery, J. Comb. Chem. 1, 32-45 (1999).
56. Miller, A., and Nguyen, N.-K., A Fedorov Exchange Algorithm of D-Optimal Design, Appl. Stat. 43, 669-678 (1994).
57. Mitchell, T. J., An Algorithm for the Construction of "D-optimal" Experimental Designs, Technometrics 16, 203-210 (1974).
58. Mitchell, T. J., An Algorithm for the Construction of "D-optimal" Experimental Designs, Technometrics 42, 48-54 (2000).
59. Golbraikh, A., and Tropsha, A., Predictive QSAR Modeling Based on Diversity Sampling of Experimental Datasets for the Test and Training Set Selection, J. Comput.-Aided Mol. Des., in press.
60. Golbraikh, A., J. Chem. Inf. Comput. Sci., 40, 414-425 (2000).
61. Gramatica, P., and Papa, E., QSAR Modeling of Bioconcentration Factor by Theoretical Molecular Descriptors, Quant. Struct-Act. Relat., in press.
62. Gramatica, P., Papa, E., and Pilutti P., QSAR Predictions of Ozone Tropospheric Degradation, Quant. Struct.-Act. Relat., in press.
63. Marengo, E., and Todeschini, R., A New Algorithm for Optimal, Distance-based Experimental Design, Chemom. Intell. Lab. Syst., 37-44 (1992).
64. Norinder, U., Single and Domain Made Variable Selection in 3D QSAR applications, J. Chemomet. 10, 95-105 (1996).
65. Zefirov, N. S., and Palyulin, V. A., QSAR for Boiling Points of "Small" Sulfides. Are the "High-Quality Structure-Property-Activity Regressions" the Real High Quality QSAR Models?, J. Chem. Inf. Comput. Sci. 41, 1022-1027 (2001).
66. Sachs, L., Applied Statistics. A Handbook of Techniques, Springer-Verlag, Heidelberg (Germany) 1982.
67. Atkinson, A. C., Plots, transformations and regression, Clarendon Press, Oxford (UK) 1985, p. 282.
68. Gramatica, P., Corradi, M., and Consonni, V., Modelling and prediction of soil sorption coefficients of non-ionic organic pesticides by molecular descriptors, Chemosphere 41, 763-777 (2000).
69. Mandel, J., The Regression Analysis of Collinear Data, J. Res. Nat. Bur. Stand. 90, 465-476 (1985).
70. Lindberg, W., Persson, J.-A., and Wold, S., Partial Least- Squares Method for Spectrofluorimetric Analysis of Mixtures of Humic Acid and Ligninsulfonate, Anal. Chem. 55, 643-648 (1983).
71. Cho, S. J., Zheng, W., and Tropsha, A., Rational Combinatorial Library Design. 2. Rational Design of Targeted Combinatorial Peptide Libraries Using Chemical Similarity Probe and the Inverse QSAR Approaches, J. Chem. Inf. Comput. Sci. 38, 259-268 (1998).