On some aspects of validation of predictive quantitative structure-activity relationship models

Expert Opinion on Drug Discovery

Volumn 2, Issue 12, 2007, Pages 1567-1577

  Roy, Kunal ^a  

^a JADAVPUR UNIVERSITY   (India)

Author keywords

Applicability domain; Cross validation; External validation; Internal validation; QSAR; validation

Indexed keywords

ACCURACY; BIOLOGICAL ACTIVITY; CORRELATION COEFFICIENT; COST EFFECTIVENESS ANALYSIS; DATA ANALYSIS; MATHEMATICAL MODEL; METHODOLOGY; PREDICTION; PRIORITY JOURNAL; QUANTITATIVE STRUCTURE ACTIVITY RELATION; REGRESSION ANALYSIS; REVIEW; RISK ASSESSMENT; STRUCTURE ACTIVITY RELATION; TECHNIQUE; VALIDATION PROCESS;

EID: 37349048522     PISSN: 17460441     EISSN: None     Source Type: Journal    
DOI: 10.1517/17460441.2.12.1567     Document Type: Review

References (71)

1. SELASSIE CD: History of quantitative structure-activity relationships. In: Burger's Medicinal Chemistry and Drug Discovery. Abraham DJ (Ed.), John Wiley, New York, USA (2003):1-48.
2. PERKINS R, FANG H, TONG W, WELSH WJ: Quantitative structure-activity relationship methods: perspectives on drug discovery and toxicology. Environ. Toxicol. Chem. (2003) 22(8):1666-1679.
3. WALKER JD, JAWORSKA J, COMBER JHI, SCHULTZ TW, DEARDEN JC: Guidelines for developing and using quantitive structure-activity relationships. Environ. Toxicol. Chem. (2003) 22(8):1653-1665.
4. YANG G-F, HUANG F: Development of quantitative structure-activity relationships and its application in rational drug design. Curr. Pharm. Des. (2006) 12:4601-4611.
5. MAZZATORTA P, BENFENATI E, LORENZINI P, VIGHI M: QSAR in ecotoxicity: an overview of modern classification techniques. J. Chem. Inf. Comput. Sci. (2004) 44(1):105-112.
6. TONG W, HONG H, XIE Q, SHI L, FANG H, PERKINS R: Assessing QSAR limitations - a regulatory perspective. Curr. Comput. Aided Drug Des. (2005) 1(2):195-205.
7. HE L, JURS PC: Assessing the reliability of a QSAR model's predictions. J. Mol. Graph. Mod. (2005) 23(6):503-523.
8. GHAFOURIAN T, CRONIN MTD: The impact of variable selection on the modelling of oestrogenicity. SAR QSAR Environ. Res. (2005) 16(1-2):171-190.
9. TROPSHA A, GRAMATICA P, GOMBAR VK: The importance of being earnest: validation is the absolute essential for successful application and interpretation of QSPR models. QSAR Comb. Sci. (2003) 22(1):69-77.
10. GOLBRAIKH A, TROPSHA A: Beware of q2! J. Mol. Graph. Mod. (2002) 20(4):269-276.
11. TONG W, XIE Q, HONG H, SHI L, FANG H, PERKINS R: Assessment of prediction confidence and domain extrapolation of two structure - activity relationship models for predicting estrogen receptor binding activity. Environ. Health Perspect. (2004) 112(12):1249-1254.
12. BALLS M, BLAAUBOER BJ, FENTEM JH et al.: Practical aspects of the validation of toxicity test procedures -the report and recommendations of ECVAM workshop 5. ATLA (1995) 23:129-147.
13. APTULA AO, JELIAZKOVA NG, SCHULTZ TW, CRONIN MTD: The better predictive model: high q2 for the timing set or low root mean square error of prediction for the test set? QSAR Comb. Sci. (2005) 24(3):385-396.
14. McKINNEY JD, RICHARD A, WALLER C, NEWMAN MC, GERBERICK F: The practice of structure activity relationships (SAR) in toxicology. Toxicol. Sci. (2000) 56:8-17.
15. TONG W, WELSH WJ, SHI L, FANG H, PERKINS R: Structure-activity relationship approaches and applications. Environ. Toxicol. Chem. (2003) 22:1680-1695.
16. WORTH AP, CRONIN MTD: Report of the workshop on the validation of QSARs and other computational prediction models. ATLA (2004) 32(Suppl. 1):703-706.
17. CRONIN MTD, JAWORSKA JS, WALKER JD, COMBER MHI, WATTS CD, WORTH AP: Use of QSARs in international decision-making frameworks to predict health effects of chemical substances. Environ. Health Perspect. (2003) 111:1391-1401.
18. CRONIN MTD, WALKER JD, JAWORSKA JS, COMBER MHI, WATTS CD, WORTH AP: Use of QSARs in international decision-making frameworks to predict ecologic effects and environmental late of chemical substances. Environ. Health Persp. (2003) 111:1376-1390.
19. WORTH AP, LEEUWEN CJ VAN, HARTUNG T: The role of the European centre for the validation of alternative methods (ECVAM) in the validation of (Q)SARs. SAR QSAR Environ. Res. (2004) 15(5-6):331-343.
20. WORTH AP, HARTUNG T, LEEUWEN CJ VAN: The prospects for using (Q)SARs in a changing political environment - high expectations and a key role for the European Commission's joint research center. SAR QSAR Environ. Res. (2004) 15(5-6):345-358.
21. VON DER OHE PC, KUHNE R, EBERT RU, ALTENBURGER R, LIESS M, SCHUURMANN G: Structural alerts - a new classification model to discriminate excess toxicity from narcotic effect levels of organic compounds in the acute daphnid assay. Chem. Res. Toxicol. (2005) 18:536-555.
22. COMBES R, BALLS M, BANSIL L et al.: An assessment of progress in the use of alternatives in toxicity testing since the publication of the report of the second FRAME toxicity committee. ATLA (1991) 30:365-406.
23. WORTH AP, CRONIN MTD, LEEUWEN CJ VAN: A frame-work for promoting the acceptance and regulatory use of (Quantitative) structure-activity relationships. In: Predicting Chemical Toxicity and Fate, Cronin MTD, Livingstone DJ (Eds), CRC Press LLC, Boca Raton, Florida, USA (2004):429-440.
24. CRONIN MTD, LIVINGSTONE DJ: Predicting Chemical Toxicity and Fate. CRC Press, Boca Raton, FL USA (2004):472.
25. JAWORSKA JS, COMBER M, AUER C, LEEUWEN CJ VAN: Summary of a workshop on regulatory acceptance of (Q)SARs for human health and environmental endpoints. Environ. Health Perspect. (2003) 111:1358-1360.
26. CLARK RD, SPROUS DG, LEONARD JM: In: Rational Approaches to Drug Design, Höltje HD, Sippl W (Eds), Prous Science, Barcelona, Spain (2001):475-485.
27. GOLBRAIKH A, TROPSHA A: Predictive QSAR modeling based on diversity sampling of experimental datasets for the training and test set selection. J. Comput. Aided Mol. Des. (2002) 16:357-369.
28. WOLD S, ERIKSSON L: Statistical validation of QSAR results. In: Chemometrics Methods in Molecular Design. van deWaterbeemd H (Ed.), VCH (1995):309-318.
29. STONE M: Cross-validatory choice and assessment of statistical predictions. J. R. Stat. Soc. (1974) B36:111-147.
30. WOLD S: Cross-validation estimation of the number of components in factor and principal components models. Technometrics (1978) 20:397-405.
31. WOLD S: Validation of QSAR's. Quant. Struct. Act. Relat. (1991) 10:191-193.
32. KUBINYI H, HAMPRECHT FA, MIETZNER T: Three-dimensional quantitative similarity - activity relationships (3D QSiAR) from SEAL similarity matrices. J. Med. Chem. (1998) 41:2553-2564.
33. LEONARD JT, ROY K: On selection of training and test sets for the development of predictive QSAR models. QSAR Comb. Sci. (2006) 25:235-251.
34. WEHRENS R, PUTTER H, BUYDENS LMC: The bootstrap: a tutorial. Chemom. Intell. Lab. Sys. (2000) 54:35-52.
35. WU W, WALCZAK B, MASSART DL et al.: Artificial neural networks in classification of NIR spectral data: design of the training set. Chemom. Intell. Lab. Syst. (1996) 33:35-46.
36. HAWKINS DM, BASAK SC, MILLS D: Assessing model fit by crossvalidation. J. Chem. Inf. Comput. Sci. (2003) 43(2):579-586.
37. HAWKINS DM: The problem of overfitting. J. Chem. Inf. Comput. Sci. (2003) 44(1):1-12.
38. NOVELLINO E, FATTORUSSO C, GRECO G: Use of comparative molecular field analysis and cluster analysis in series design. Pharm. Acta Helv. (1995) 70:149-154.
39. NORINDER U: Single and domain variable selection in 3D QSAR applications. J. Chemom. (1996) 10:95-105.
40. KUBINYI H: A general view on similarity and QSAR studies. In: Computer-Assisted Lead Finding and Optimization, van de Waterbeemd H, Testa B, Folkers G (Eds), VHChA and VCH, Basel, Weinheim (1997):9-28.
41. KUBINYI H, HAMPRECHT FA, MIETZNER T: Three-dimensional quantitative similarity-activity relationships (3D QSiAR) from SEAL similarity matrices. J. Med. Chem. (1998) 41:2553-2564.
42. GUHA R, JURS PC: Determining the validity of a QSAR model - a classification approach. J. Chem. Inf. Model. (2005) 45(1):65-73.
43. YASRI A, HARTSOUGH D: Toward an optimal procedure for variable selection and QSAR model building. J. Chem. Inf. Comput. Sci. (2001) 41:1218-1227.
44. KAUFFMAN GV, JURS PC: QSAR and k-nearest neighbor classification analysis of selective cyclooxygenase-2 inhibitors using topologically-based numerical descriptors. J. Chem. Inf. Comput. Sci. (2001) 41:1553-1560.
45. MATTIONI BE, JURS PC: Prediction of glass transition temperatures from monomer and repeat unit structure using computational neural networks. J. Chem. Inf. Comput. Sci. (2002) 42:232-240.
46. CARLSON R: Design and Optimization in Organic Synthesis. Elsevier (1992).
47. HUUSKONEN J: QSAR modeling with the electrotopological state: TIBO derivatives. J. Chem. Inf. Comput. Sci. (2001) 41:425-429.
48. TETKO IV, KOVALISHYN VV, LIVINGSTONE DJ: Volume learning algorithm artificial neural networks for 3D QSAR studies. J. Med. Chem. (2001) 44:2411-2420.
49. GASTEIGER J, ZUPAN J: Neural networks in chemistry. Angew. Chem. (1993) 32(4):503-527.
50. EVERITT BS, LANDAU S, LEESE M: Cluster Analysis. Edward Arnold, London, UK (2001).
51. KOWALSKI RB, WOLD S: Handbook of Statistics. North Holland Publishing Company, Amsterdam, The Netherlands (1982).
52. LAJINESS M, JOHNSON MA, MAGGIORA GM: In: QSAR: Quantitative Structure - Activity Relationships in Drug Design. Fauchere JL (Ed.), Alan R Liss Inc., New York, USA (1989):173-176.
53. TAYLOR R: Simulation analysis of experimental design strategies for screening random compounds as potential new drugs and agrochemicals. J. Chem. Inf. Comput. Sci. (1995) 35:59-67.
54. SNAREY M, TERRETT NK, WILLETT P, WILTON DJ: Comparison of algorithms for dissimilarity-based compound selection. J. Mol. Graph. Mod. (1997) 15:372-385.
55. KENNARD RW, STONE LA: Computer-aided design of experiments. Technometrics (1969) 11:137-148.
56. BOURGUIGNON B, DEAGUIAR PF, THORREAND K, MASSART DL: The application of nonlinear regression functions for the modelling of retention in reversed-phase liquid chromatography. J. Chromatogr. Sci. (1994) 32:144-152.
57. BOURGUIGNON B, DEAGUIAR PF, KHOTS MS, MASSART DL: Optimization in irregularly shaped regions: pH and solvent strength in reversed-phase HPLC separations. Anal. Chem. (1994) 66:893-904.
58. GOLBRAIKH A: Molecular dataset diversity indices and their applications to comparison of chemical databases and QSAR analysis. J. Chem. Inf. Comput. Sci. (2000) 40:414-425.
59. SZÀNTAI-KIS C, KÖVESDI I, KÈRI G, ORFI L: Validation subset selections for extrapolation oriented QSPAR models. Mol. Divers. (2003) 7:37-43.
60. GRAMATICA P: Principles of QSAR models validation: internal and external. QSAR Comb. Sci. (2007) 26(6):694-701.
61. ATKINSON AC: Plots, Transformation and Regression. Clarendon Press, Oxford, UK (1985).
62. JAWORSKA J, NIKOLOVA-JELIAZKOVA N, ALDENBERG T: QSAR applicability domain estimation by projection of the training set descriptor space: a review. Altern. Lab. Anim. (2005) 33(5):445-459.
63. STANFORTH RW, KOLOSSOV E, MIRKIN B: A measure of domain of applicability for QSAR modelling based on intelligent K-means clustering. QSAR Comb. Sci. (2007) 26(7):837-844.
64. BAUMANN K, STIEFL N: Validation tools for variable subset regression. J. Comput. Aided Mol. Des. (2004) 18:549-562.
65. ZHENG W, TROPSHA A: Novel variable selection quantitative structure - property relationship approach based on the k-nearest-neighbor principle. J. Chem. Inf. Comput. Sci. (2000) 40(1):185-194.
66. GHAFOURIAN T, CRONIN M: The effect of variable selection on the non-linear modeling of oestrogen receptor binding. QSAR Comb. Sci. (2006) 25(10):824-835.
67. ROY P, ROY K: On some aspects of variable selection for partial least squares regression models. QSAR Comb. Sci. (2007):[Epub ahead of print].
68. ROY PP, LEONARD JT, ROY K: Exploring the impact of the size of training sets for the development of predictive QSAR models. Chemom. Intell. Lab. Sys. (2007) [Epub ahead of print].
69. KAHN I, FARA D, KARELSON M, MARAN U: QSPR treatment of the soil sorption coefficients of organic pollutants. J. Chem. Inf. Model. (2005) 45:94-105.
70. oc prediction. J. Mol. Graph. Model. (2007) 25:755-766.
71. DEBNATH AK: Generation of predictive pharmacophore models for CCR5 antagonists: study with piperidine- and piperazine-based compounds as a new class of HIV-1 entry inhibitors. J. Med. Chem. (2003) 46:4501-4515.