Criterion for evaluating the predictive ability of nonlinear regression models without cross-validation

Journal of Chemical Information and Modeling

Volumn 53, Issue 9, 2013, Pages 2341-2348

  Kaneko, Hiromasa ^a   Funatsu, Kimito ^a  

^a UNIVERSITY OF TOKYO   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

DATA HANDLING; MEAN SQUARE ERROR; NEAREST NEIGHBOR SEARCH;

CROSS VALIDATION; DETERMINATION COEFFICIENTS; K-NEAREST NEIGHBORS; NONLINEAR REGRESSION MODELS; PREDICTIVE ABILITIES; PREDICTIVE PERFORMANCE; ROOT MEAN SQUARE ERRORS; STRUCTURAL RELATIONSHIP;

REGRESSION ANALYSIS;

WATER;

ARTICLE; CHEMISTRY; DRUG EFFECT; INFORMATION SCIENCE; LD 50; METHODOLOGY; NONLINEAR SYSTEM; QUANTITATIVE STRUCTURE ACTIVITY RELATION; REGRESSION ANALYSIS; REPRODUCIBILITY; SOLUBILITY; TETRAHYMENA PYRIFORMIS; TIME; TOXICITY TESTING; DRUG EFFECTS; PROCEDURES;

INFORMATICS; LETHAL DOSE 50; NONLINEAR DYNAMICS; QUANTITATIVE STRUCTURE-ACTIVITY RELATIONSHIP; REGRESSION ANALYSIS; REPRODUCIBILITY OF RESULTS; SOLUBILITY; TETRAHYMENA PYRIFORMIS; TIME FACTORS; TOXICITY TESTS; WATER;

EID: 84884574602     PISSN: 15499596     EISSN: 1549960X     Source Type: Journal    
DOI: 10.1021/ci4003766     Document Type: Article

References (46)

1. Gasteiger, J. and Engel, T. Chemoinformatics-A Textbook; Wiley-VCH: Weinheim, Germany, 2003.
2. Wold, S. Principal component analysis Chemom. Intell. Lab. Syst. 1987, 2, 37-52
3. Wold, S.; Sjöström, M.; Eriksson, L. PLS-regression: A basic tool of chemometrics Chemom. Intell. Lab. Syst. 2001, 58, 109-130
4. Marini, F.; Roncaglioni, A.; Novič, M. Variable selection and interpretation in structure-affinity correlation modeling of estrogen receptor binders J. Chem. Inf. Model. 2005, 45, 1507-1519
5. Mazzatorta, M.; Smiesko, M.; Piparo, E. L.; Benfenati, E. QSAR model for predicting pesticide aquatic toxicity J. Chem. Inf. Model. 2005, 45, 1767-1774
6. Kim, K.; Lee, J. M.; Lee, I. B. A novel multivariate regression approach based on kernel partial least squares with orthogonal signal correction Chemom. Intell. Lab. Syst. 2005, 79, 22-30
7. Obrezanova, O.; Csanyi, G.; Gola, J. M. R.; Segall, M. D. Gaussian processes: A method for automatic QSAR modeling of ADME properties J. Chem. Inf. Model. 2007, 47, 1847-1857
8. Song, M. H.; Clark, M. Development and evaluation of an in silico model for hERG binding J. Chem. Inf. Model. 2006, 46, 392-400
9. Lowe, R.; Mussa, H. Y.; Mitchell, J. B. O.; Glen, R. C. Classifying molecules using a sparse probabilistic kernel binary classifier J. Chem. Inf. Model. 2011, 51, 1539-1544
10. Schuurmann, G.; Ebert, R. U.; Chen, J. W.; Wang, B.; Kuhne, R. External validation and prediction employing the predictive squared correlation coefficient-Test set activity mean vs training set activity mean J. Chem. Inf. Model. 2008, 48, 2140-2145
11. Chirico, N.; Gramatica, P. Real external predictivity of QSAR models: How to evaluate it? Comparison of different validation criteria and proposal of using the concordance correlation coefficient J. Chem. Inf. Model. 2011, 51, 2320-2335
12. Chirico, N.; Gramatica, P. Real external predictivity of QSAR models. Part 2. New intercomparable thresholds for different validation criteria and the need for scatter plot inspection J. Chem. Inf. Model. 2012, 51, 2044-2058
13. Golbraikh, A.; Tropsha, A. Beware of q2! J. Mol. Graph Model. 2002, 20, 269-276
14. Baumann, K. Cross-validation as the objective function for variable-selection techniques TrAC, Trends Anal. Chem. 2003, 22, 395-406
15. Todeschini, R.; Consonni, V.; Mauri, A.; Pavan, M. Detecting "bad" regression models: multicriteria fitness functions in regression analysis Anal. Chim. Acta 2004, 515, 199-208
16. Konovalov, D. A.; Sim, N.; Deconinck, E.; Heyden, Y. V.; Coomans, D. Statistical confidence for variable selection in QSAR models via Monte Carlo cross-validation J. Chem. Inf. Model. 2008, 48, 370-383
17. Sheridan, R. P. Time-split cross-validation as a method for estimating the goodness of prospective prediction J. Chem. Inf. Model. 2013, 53, 783-790
18. Rucker, C.; Rucker, G.; Meringer, M. y -Randomization and its variants in QSPR/QSAR J. Chem. Inf. Model. 2007, 47, 2345-2357
19. Papa, E.; Villa, F.; Gramatica, P. Statistically validated QSARs, based on theoretical descriptors, for modeling aquatic toxicity of organic chemicals in Pimephales promelas (fathead minnow) J. Chem. Inf. Model. 2005, 45, 1256-1266
20. Anderssen, E.; Dyrstad, K.; Westad, F.; Martens, H. Reducing over-optimism in variable selection by cross-model validation Chemom. Intell. Lab. Syst. 2006, 84, 69-74
21. Howe, D.; Costanzo, M.; Fey, P.; Gojobori, T.; Hannick, L.; Hide, W.; Hill, D. P.; Kania, R.; Schaeffer, M.; Pierre, S., St; Twigger, S.; White, O.; Rhee, S. Y. Big data: The future of biocuration Nature. 2008, 455, 47-50
22. Li, J. Z.; Gramatica, P. Classification and virtual screening of androgen receptor antagonists J. Chem. Inf. Model. 2010, 50, 861-874
23. Rodgers, S. L.; Davis, A. M.; Tomkinson, N. P.; Waterbeemd, H. V. D. Predictivity of simulated ADME autoQSAR models over time Mol. Inf. 2011, 30, 256-266
24. D'Souza, A.; Schaal, S. Incremental online learning in high dimensions Neural Comput. 2005, 17, 2602-2634
25. Kadlec, P.; Gabrys, B.; Strandt, S. Data-driven soft sensors in the process industry Comput. Chem. Eng. 2009, 33, 795-814
26. Kaneko, H.; Funatsu, K. Classification of the degradation of soft sensor models and discussion on adaptive models AIChE J. 2013, 59, 2339-2347
27. Kaneko, H.; Funatsu, K. Adaptive soft sensor model using online support vector regression with time variable and discussion of appropriate hyperparameter settings and window size Comput. Chem. Eng. 2013, 58, 288-297
28. Mallow, C. L. Some comments on Cp Technometrics 1973, 15, 661-675
29. Akaike, H. Factor analysis and AIC Psychometrika 1987, 52, 317-332
30. Schwarz, G. Estimating the dimension of a model Ann. Stat. 1978, 6, 461-464
31. Maesschalck, R. D.; Jouan-Rimbaud, D.; Massart, D. L. The Mahalanobis distance Chemom. Intell. Lab. Syst. 2000, 50, 1-18
32. Vapnik, V. N. The Nature of Statistical Learning Theory; Springer: New York, 1999.
33. Li, G.; Aute, V.; Azarm, S. An accumulative error based adaptive design of experiments for offline metamodeling Struct. Multidiscip. Optim. 2010, 40, 137-155
34. http://www.cadaster.eu/node/65 (accessed June 12, 2013).
35. Owen, J. R.; Nabney, I. T.; Medina-Franco, J. L.; Lopez-Vallejo, F. Visualization of molecular fingerprints J. Chem. Inf. Model. 2011, 21, 1552-1563
36. http://www.talete.mi.it/products/dragon-description.htm (accessed June 12, 2013).
37. Hou, T. J.; Xia, K.; Zhang, W.; Xu, X. J. ADME evaluation in drug discovery. 4. Prediction of aqueous solubility based on atom contribution approach J. Chem. Inf. Comput. Sci. 2004, 44, 266-275
38. Baurin, N. Drug-like annotation and duplicate analysis of a 23-supplier chemical database totalling 2.7 million compounds J. Chem. Inf. Comput. Sci. 2004, 44, 643-651
39. Sun, H. A Universal molecular descriptor system for prediction of logP, logS, logBB, and absorption J. Chem. Inf. Comput. Sci. 2004, 44, 748-757
40. Wegner, J. K.; Fröhlich, H.; Zell, A. Feature selection for descriptor based classification models. 1. Theory and GA-SEC algorithm J. Chem. Inf. Comput. Sci. 2004, 44, 921-930
41. Bergström, C. A. S.; Wassvik, C. M.; Norinder, U.; Luthman, K.; Artursson, P. Global and local computational models for aqueous solubility prediction of drug-like molecules J. Chem. Inf. Comput. Sci. 2004, 44, 1477-1488
42. Clark, M. Generalized fragment-substructure based property prediction method J. Chem. Inf. Model. 2005, 45, 30-38
43. Vidal, D.; Thormann, M.; Pons, M. LINGO, An efficient holographic text based method to calculate biophysical properties and intermolecular similarities J. Chem. Inf. Model. 2005, 45, 386-393
44. Kaneko, H.; Funatsu, K. Development of a new regression analysis method using independent component analysis J. Chem. Inf. Model. 2008, 48, 534-541
45. Bishop, C. M. Pattern Recognition and Machine Learning; Springer: New York, 2006.
46. Chang, C. C.; Lin, C. J. LIBSVM: A library for support vector machines. http://www.csie.ntu.edu.tw/∼cjlin/libsvm.