Be aware of error measures. Further studies on validation of predictive QSAR models

Chemometrics and Intelligent Laboratory Systems

Volumn 152, Issue , 2016, Pages 18-33

  Roy, Kunal ^a,b   Das, Rudra Narayan ^a   Ambure, Pravin ^a   Aher, Rahul B ^a  

^a JADAVPUR UNIVERSITY   (India)

^b NONE   (United States)

Author keywords

Dispersion; Error measures; MAE; QSAR; Validation

Indexed keywords

DIPEPTIDYL CARBOXYPEPTIDASE;

ARTICLE; COMPUTER PREDICTION; CYTOTOXICITY TEST; MATHEMATICAL COMPUTING; MEASUREMENT ERROR; NONHUMAN; PRIORITY JOURNAL; QUANTITATIVE STRUCTURE ACTIVITY RELATION; SOLUBILITY; STATISTICAL MODEL; VALIDATION PROCESS;

EID: 84958826961     PISSN: 01697439     EISSN: 18733239     Source Type: Journal    
DOI: 10.1016/j.chemolab.2016.01.008     Document Type: Article

References (26)

1. Benigni R., Giuliani A. Putting the predictive toxicology challenge into perspective: reflections on the results. Bioinformatics 2003, 19:1194-1200.
2. Kar S., Roy K. Predictive toxicology using QSAR: a perspective. J. Indian Chem. Soc. 2010, 87:1455-1515.
3. OECD environment health and safety publications series on testing and assessment No.69 Guidance Document on the Validation of (Quantitative) Structure Activity Relationship [(Q)SAR] Models 2007, Available via, (last accessed on November 18, 2015). 10.1787/9789264085442-en.
4. Roy K., Mitra I. On various metrics used for validation of predictive QSAR models with applications in virtual screening and focused library design. Comb. Chem. High Throughput Screen. 2011, 14:450-474.
5. Golbraikh A., Tropsha A. Beware of q 2!. J. Mol. Graph. Model. 2002, 20:269-276.
6. Tropsha A. Best practices for QSAR model development, validation, and exploitation. Mol. Inf. 2010, 29:476-488.
7. 2 metrics for judging quality of quantitative structure-activity relationship predictions: emphasis on scaling of response data. J. Comput. Chem. 2013, 34:1071-1082.
8. Chirico N., Gramatica P. Real external predictivity of QSAR models: how to evaluate it? Comparison of different validation criteria and proposal of using the concordance correlation coefficient. J. Chem. Inf. Model. 2011, 51:2320-2335.
9. Alexander D.L.J., Tropsha A., Winkler D.A. Beware of R 2: Simple, Unambiguous Assessment of the Prediction Accuracy of QSAR and QSPR Models. J. Chem. Inf. Model. 2015, 55:1316-1322.
10. Davis W.H., Pryor W.A. Measures of goodness of fit in linear free energy relationships. J. Chem. Educ. 1976, 53:285.
11. Lothar S., Reynarowych Z. Applied Statistics: A Handbook of Techniques 1982, Springer-Verlag, New York.
12. Roy K., Mitra I., Kar S., Ojha P.K., Das R.N., Kabir H. Comparative studies on some metrics for external validation of QSPR models. J. Chem. Inf. Model. 2012, 52:396-408.
13. 2 as a metric for validation of QSAR models. J. Chemom. 2010, 24:22-33.
14. Mendham J., Denney R.C., Barnes J.D., Thomas M.J.K. Vogels Textbook of Quantitative Chemical Analysis 2007, Pearson Education India. sixth ed.
15. Dearden J.C., Cronin M.T.D., Kaiser K.L.E. How not to develop a quantitative structure-activity or structure-property relationship (QSAR/QSPR). SAR QSAR Environ. Res. 2009, 20:241-266.
16. Fang K.-T., Yin H., Liang Y.-Z. New approach by Kriging models to problems in QSAR. J. Chem. Inf. Comput. Sci. 2004, 44:2106-2113.
17. Abraham M.H., Sanchez-Moreno R., Cometto-Muniz J.E., Cain W.S. A quantitative structure-activity analysis on the relative sensitivity of the olfactory and the nasal trigeminal chemosensory systems. Chem. Senses 2007, 32:711-719.
18. 2 parameter for QSAR validation. J. Chem. Inf. Model. 2009, 49:1669-1678.
19. Consonni V., Ballabio D., Todeschini R. Evaluation of model predictive ability by external validation techniques. J. Chemom. 2010, 24:194-201.
20. Chai T., Draxler R.R. Root mean square error (RMSE) or mean absolute error (MAE)? - arguments against avoiding RMSE in the literature. Geosci. Model Dev. 2014, 7:1247-1250.
21. Willmott C.J., Matsuura K. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim. Res. 2005, 30:79.
22. Glick J.H. Expression of random analytical error as a percentage of the range of clinical interest. Clin. Chem. 1976, 22:475-483.
23. Snedecor G.W., Cochran W.G. Statistical Methods 1967, The IOWA StateUniversitypress, AMES, IOWA, VSA.
24. Das R.N., Roy K. QSPR with extended topochemical atom (ETA) indices. 4. Modeling aqueous solubility of drug like molecules and agrochemicals following OECD guidelines. Struct. Chem. 2013, 24:303-331.
25. Brahmachari G., Choo C., Ambure P., Roy K. In vitro evaluation and in silico screening of synthetic acetylcholinesterase inhibitors bearing functionalized piperidine pharmacophores. Bioorg. Med. Chem. 2015, 23:4567-4575.
26. Das R.N., Roy K., Popelier P.L.A. Exploring simple, transparent, interpretable and predictive QSAR models for classification and quantitative prediction of rat toxicity of ionic liquids using OECD recommended guidelines. Chemosphere 2015, 139:163-173.