Does rational selection of training and test sets improve the outcome of QSAR modeling?

Journal of Chemical Information and Modeling

Volumn 52, Issue 10, 2012, Pages 2570-2578

  Martin, Todd M ^a   Harten, Paul ^a   Young, Douglas M ^a   Muratov, Eugene N ^b,c   Golbraikh, Alexander ^b   Zhu, Hao ^d,e   Tropsha, Alexander ^b  

^a U S ENVIRONMENTAL PROTECTION AGENCY   (United States)

^b University of North Carolina   (United States)

^c A V BOGATSKY PHYSICO CHEMICAL INSTITUTE   (Ukraine)

^d RUTGERS UNIVERSITY   (United States)

^e USA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL CHEMISTRY; DECISION TREES; HIERARCHICAL CLUSTERING; MOLECULAR GRAPHICS; NEAREST NEIGHBOR SEARCH; STATISTICAL TESTS;

EXCLUSION ALGORITHMS; INTELLIGENT FASHION; K-NEAREST NEIGHBOR METHOD; PREDICTIVE ABILITIES; PREDICTIVE MODELS; PREDICTIVE POWER; QUANTITATIVE STRUCTURE-ACTIVITY RELATIONSHIP MODELING; SPLITTING METHOD;

PREDICTIVE ANALYTICS;

BIOLOGICAL PRODUCT;

ALGORITHM; ANIMAL; ARTICLE; CHEMICAL STRUCTURE; CHEMISTRY; CYPRINIFORMES; DRUG DEVELOPMENT; DRUG EFFECT; FACTUAL DATABASE; GROWTH, DEVELOPMENT AND AGING; IC 50; LD 50; QUANTITATIVE STRUCTURE ACTIVITY RELATION; RAT; REPRODUCIBILITY; TETRAHYMENA PYRIFORMIS; VALIDATION STUDY;

ALGORITHMS; ANIMALS; BIOLOGICAL AGENTS; CYPRINIDAE; DATABASES, FACTUAL; DRUG DISCOVERY; INHIBITORY CONCENTRATION 50; LETHAL DOSE 50; MODELS, MOLECULAR; QUANTITATIVE STRUCTURE-ACTIVITY RELATIONSHIP; RATS; REPRODUCIBILITY OF RESULTS; TETRAHYMENA PYRIFORMIS; VALIDATION STUDIES AS TOPIC;

EID: 84867827686     PISSN: 15499596     EISSN: 1549960X     Source Type: Journal    
DOI: 10.1021/ci300338w     Document Type: Article

References (53)

1. Hawkins, D. M. The Problem of Overfitting J. Chem. Inf. Comput. Sci. 2004, 44, 1-12
2. Eriksson, L.; Jaworska, J. S.; Worth, A. P.; Cronin, M. T. D.; McDowell, R. M.; Gramatica, P. Methods for Reliability and Uncertainty Assessment and for Applicability Evaluations of Classification- and Regression-Based QSARs Environ. Health Perspect. 2003, 111, 1361-1375
3. Zupan, J.; Novic, M.; Ruisanchez, I. Kohonen and Counterpropagation Artificial Neural Networks in Analytical Chemistry Chemom. Intell. Lab. Syst. 1997, 38, 1-23
4. Gasteiger, J.; Zupan, J. Neuronale Netze in der Chemie Angew. Chem. 1993, 32, 503
5. Bourguignon, B.; de Aguiar, P. F.; Khots, M. S.; Massart, D. L. Optimization in Irregularly Shaped Regions: pH and Solvent Strength in Reversed-Phase High-Performance Liquid Chromatography Separations Anal. Chem. 1994, 66, 893-904
6. Bourguignon, B.; de Aguiar, P. F.; Thorre, K.; Massart, D. L. Application of Nonlinear-Regression Functions for the Modeling of Retention in Reversed-Phase LC J. Chromatogr. Sci. 1994, 32, 144-152
7. Kennard, R. W.; Stone, L. A. Computer Aided Design of Experiments Technometrics 1969, 11, 137-148
8. Snarey, M.; Terrett, N. K.; Willet, P.; Wilton, D. J. Comparison of Algorithms for Dissimilarity-Based Compound Selection J. Mol. Graphics Modell. 1997, 15, 372-385
9. Federov, V. V. Theory of Optimal Experiments, Moscow University; Academic Press: New York, 1972.
10. de Aguiar, P. F.; Bourguignon, B.; Khots, M. S.; Massart, D. L.; Phan-Than-Luu, R. D-Optimal Designs Chemom. Intell. Lab. Syst. 1995, 30, 199-210
11. Carlson, R. Design and Optimization in Organic Synthesis; Elsevier: Amsterdam, 1992.
12. Olsson, I.-M.; Gottfries, J.; Wold, S. D-Optimal Onion Designs in Statistical Molecular Design Chemom. Intell. Lab. Syst. 2004, 73, 37-46
13. Golbraikh, A.; Tropsha, A. Predictive QSAR Modeling Based on Diversity Sampling of Experimental Datasets for the Training and Test Set Selection J. Comput.-Aided. Mol. Des. 2002, 16, 357-369
14. Golbraikh, A.; Shen, M.; Xiao, Z.; Xiao, Y.-D.; Lee, K.-H.; Tropsha, A. Rational Selection of Training and Test sets for the Development of Validated QSAR Models J. Comput.-Aided. Mol. Des. 2003, 17, 241-253
15. Wu, W.; Walczak, B.; Massart, D. L.; Heuerding, S.; Erni, F.; Last, I. R.; Prebble, K. A. Artificial Neural Networks in Classification of NIR Spectral Data: Design of the Training Set Chemom. Intell. Lab. Syst. 1996, 33, 35-46
16. Gramatica, P.; Pilutti, P. Evaluation of different statistical approaches for the validation of quantitative structure-activity relationships; The European Commission-Joint Research Centre, Institute for Health & Consumer Protection-ECVAM: Ispra, Italy, 2004.
17. Marengo, E.; Todeschini, R. A New Algorithm for Optimal, Distance-Based Experimental Design Chemom. Intell. Lab. Syst. 1992, 16, 37-44
18. 3 Radical Tropospheric Degradability of Organic Pollutants by Theoretical Molecular Descriptors Atmos. Environ. 2003, 37, 3115-3124
19. Sjostrom, M.; Eriksson, L. Applications of Statistical Experimental Design. In Chemometric Methods in Molecular Design; Van de Warerbeemd, H., Ed.; VCH: New York, 1995.
20. Gramatica, P.; Pilutti, P.; Papa, E. Validated QSAR Prediction of OH Tropospheric Degradability: Splitting into Training-Test Set and Consensus Modeling J. Chem. Inf. Comput. Sci. 2004, 44, 1794-1802
21. Martin, T. M.; Harten, P.; Venkatapathy, R.; Das, S.; Young, D. M. A Hierarchical Clustering Methodology for the Estimation of Toxicity Toxicol. Mech. Methods 2008, 18, 251-266
22. Kuz'min, V. E.; Artemenko, A. G.; Muratov, E. N.; Volineckaya, I. L.; Makarov, V. A.; Riabova, O. B.; Wutzler, P.; Schmidtke, M. Quantitative Structure-Activity Relationship Studies of [(Biphenyloxy)Propyl]Isoxazole Derivatives-Human Rhinovirus 2 Replication Inhibitors J. Med. Chem. 2007, 50, 4205-4213
23. Polishchuk, P. G.; Muratov, E. N.; Artemenko, A. G.; O.G., K.; Muratov, N. N.; Kuz'min, V. E. Application of Random Forest Approach to QSAR Prediction of Aquatic Toxicity J. Chem. Inf. Model. 2009, 49, 2481-2488
24. USEPA. ECOTOX Database. http://cfpub.epa.gov/ecotox/ (accessed 10/23/08).
25. USEPA. User'sGuide for T.E.S.T. (Toxicity EstimationSoftware Tool) A Program to Estimate Toxicity from Molecular Structure,Version 3.0. http://www.epa.gov/ORD/NRMRL/std/cppb/qsar/testuserguide.pdf (accessed 10/27/09).
26. Eldred, D. V.; Weikel, C. L.; Jurs, P. C.; Kaiser, K. L. E. Prediction of Fathead Minnow Acute Toxicity of Organic Compounds from Molecular Structure Chem. Res. Toxicol. 1999, 12, 670-678
27. He, L.; Jurs, P. C. Assessing the Reliability of QSAR Model'S Predictions J. Mol. Graphics Modell. 2005, 23, 503-523
28. Kaiser, K. L. E.; Niculescu, S. P. Using Probabilistic Neural Networks To Model the Toxicity of Chemicals to the Fathead Minnow (Pimephales promelas): A Study Based on 865 Chemicals Chemosphere 1999, 38, 3237-3245
29. 50) of Organic Compounds to the Fathead Minnow (Pimephales promelas) Using a Group Contribution Method Chem. Res. Toxicol. 2001, 14, 1378-1385
30. Zhu, H.; Tropsha, A.; Fourches, D.; Varnek, A.; Papa, E.; Gramatica, P.; Öberg, T.; Dao, P.; Cherkasov, A.; Tetko, I. V. Combinational QSAR Model of Chemical Toxicants Tested against Tetrahymena pyriformis J. Chem. Inf. Model. 2008, 48, 766-784
31. Schultz, T. W. TETRATOX: Tetrahymena pyriformis Population Growth Impairment Endpoint-A Surrogate for Fish Lethality Toxicol. Methods 1997, 7, 289-309
32. Schultz, T. W. TETRATOX. http://www.vet.utk.edu/TETRATOX/index.php (accessed 2/24/11).
33. Schultz, T. W.; Netzeva, T. I. Development and Evaluation of QSARs for Ecotoxic Endpoints: The Benzene Response-Surface Model for Tetrahymena Toxicity. In Modeling Environmental Fate and Toxicity; Cronin, M. T. D.; Livingstone, D. J., Eds.; CRC Press: Boca Raton, FL, 2004.
34. Cronin, M. T. D.; Aptula, A. O.; Duffy, J. C.; Netzeva, T. I.; Rowe, P. H.; Valkova, I. V.; Schultz, T. W. Comparative Assessment of Methods To Develop QSARs for the Prediction of the Toxicity of Phenols to Tetrahymena pyriformis Chemosphere 2002, 49, 1201-1221
35. NLM. ChemIDplus Advanced Database. http://chem.sis.nlm.nih.gov/ chemidplus/ (accessed 2/24/11).
36. Zhu, H.; Martin, T. M.; Ye, L.; Sedykh, A.; Young, D. M.; Tropsha, A. Quantitative Structure-Activity Relationship Modeling of Rat Acute Toxicity by Oral Exposure Chem. Res. Toxicol. 2009, 22, 1913-1921
37. Dimitrov, S.; Dimitrova, N.; Parkerton, T.; Combers, M.; Bonnell, M.; Mekenyan, O. Base-Line Model for Identifying the Bioaccumulation Potential of Chemicals SAR QSAR Environ. Res. 2005, 16, 531-554
38. Arnot, J. A.; Gobas, F. A. P. C. A Review of Bioconcentration Factor (BCF) and Bioaccumulation Factor (BAF) Assessments for Organic Chemicals in Aquatic Organisms Environ. Rev. 2006, 14, 257-297
39. EURAS. Establishing a bioconcentration factor (BCF) GoldStandard Database. http://www.euras.be/eng/project.asp?ProjectId=92 (accessed 5/20/09).
40. Zhao, C. B., E.; Chana, A.; Roncaglioni, A.; Benfenati, E. A New Hybrid System of QSAR Models for Predicting Bioconcentration Factors (BCF) Chemosphere 2008, 73, 1701-1707
41. Hamelink, J. L. Current Bioconcentration Test Methods and Theory. In Aquatic Toxicology and Hazard Evaluation; Mayer, F. L.; Hamelink, J. L., Eds.; ASTM STP: West Conshohocken, PA, 1977; Vol. 634, pp 149-161.
42. Young, D. M.; Martin, T. M. Are the Chemical Structures in Your QSAR Correct? QSAR Comb. Sci. 2008, 27, 1337-1345
43. Talete. Dragon Version 5.4. http://www.talete.mi.it/ (accessed 5/26/09).
44. Artemenko, A. G.; Muratov, E. N.; Kuz'min, V. E.; Kovdienko, N. A.; Hromov, A. I.; Makarov, V. A.; Riabova, O. B.; Wutzler, P.; Schmidtke, M. Identification of Individual Structural Fragments of N,N′-(Bis-5- nitropyrimidyl)dispirotripiperazine Derivatives for Cytotoxicity and Antiherpetic Activity Allows the Prediction of New Highly Active Compounds J. Antimicrob. Chemother. 2007, 60, 68-77
45. Muratov, E. N.; Artemenko, A. G.; Kuz'min, V. E.; Lozitsky, V. P.; Fedchuk, A. S.; Lozitska, R. N.; Boschenko, Y. A.; Gridina., T. L. Investigation of Anti-Influenza Activity Using Hierarchic QSAR Technology on the Base of Simplex Representation of Molecular Structure Antiviral Res. 2005, 65, A62-A63
46. Kuz'min, V. E.; Artemenko, A. G.; Muratov, E. N. Hierarchical QSAR Technology on the Base of Simplex Representation of Molecular Structure J. Comput.-Aided Mol. Des. 2008, 22, 403-421
47. Kuz'min, V. E.; Artemenko, A. G.; Lozitsky, V. P.; Muratov, E. N.; Fedtchouk, A. S.; Dyachenko, N. S.; Nosach, L. N.; Gridina, T. L.; Shitikova, L. I.; Mudrik, L. M.; Mescheriakov, A. K.; Chelombitko, V. A.; Zheltvay, A. I.; Eynde, J.-J. V. The Analysis of Structure- Anticancer and Antiviral Activity Relationships for Macrocyclic Pyridinophanes and Their Analogues on the Basis of 4D QSAR Models (Simplex Representation of Molecular Structure) Acta Biochim. Pol. 2002, 49, 157-168
48. Romesburg, H. C. Cluster Analysis for Researchers; Lifetime Learning Publications: Belmont, CA, 1984.
49. Montgomery, D. C. Introduction to linear regression analysis; John Wiley and Sons: New York, 1982.
50. Breiman, L. Random forests Mach. Learn. 2001, 45, 5-32
51. Dalgaard, P. Introductory Statistics with R; Springer: New York, 2008.
52. Sachs, L. Applied Statistics. A Handbook of Techniques; Springer-Verlag: New York, 1984.
53. Svetnik, V.; Liaw, A.; Tong, C.; Culberson, J. C.; Sheridan, R. P.; Feuston, B. P. Random Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling J. Chem. Inf. Model. 2003, 43, 1947-1958