Deep-learning: Investigating deep neural networks hyper-parameters and comparison of performance to shallow methods for modeling bioactivity data

Journal of Cheminformatics

Volumn 9, Issue 1, 2017, Pages

  Koutsoukas, Alexios ^a   Monaghan, Keith J ^a   Li, Xiaoli ^a   Huan, Jun ^a  

^a UNIVERSITY OF KANSAS   (United States)

Author keywords

Cheminformatics; Data mining; Deep learning; kNN; Machine learning; Na ve Bayes; Random forest; SARs; Support vector machines

Indexed keywords

EID: 85021424291     PISSN: None     EISSN: 17582946     Source Type: Journal    
DOI: 10.1186/s13321-017-0226-y     Document Type: Article

References (74)

1. Bains W, Gilbert R, Sviridenko L, Gascon JM, Scoffin R, Birchall K, Harvey I, Caldwell J (2002) Evolutionary computational methods to predict oral bioavailability QSPRs. Curr Opin Drug Discov Dev 5:44-51
2. Basak SC, Mills D (2001) Quantitative structure-property relationships (QSPRs) for the estimation of vapor pressure: a hierarchical approach using mathematical structural descriptors. J Chem Inf Model 41:692-701
3. Le T, Epa VC, Burden FR, Winkler DA (2012) Quantitative structure-property relationship modeling of diverse materials properties. Chem Rev 112:2889-2919
4. Cronin MTD (2010) Quantitative structure-activity relationships (QSARs) - applications and methodology. In: Puzyn T, Leszczynski J, Cronin MT (eds) Recent advances in QSAR studies methods and applications. Springer, pp 3-11
5. Kruhlak NL, Benz RD, Zhou H, Colatsky TJ (2012) (Q)SAR modeling and safety assessment in regulatory review. Clin Pharmacol Ther 91:529-534
6. Merlot C (2010) Computational toxicology - a tool for early safety evaluation. Drug Discov Today 15:16-22
7. Koutsoukas A, Simms B, Kirchmair J, Bond PJ, Whitmore AV, Zimmer S, Young MP, Jenkins JL, Glick M, Glen RC, Bender A (2011) From in silico target prediction to multi-target drug design: current databases, methods and applications. J Proteomics 74:2554-2574
8. Winkler DA (2004) Neural networks as robust tools in drug lead discovery and development. Mol Biotechnol 27:139-168
9. Mosier PD, Jurs PC (2002) QSAR/QSPR studies using probabilistic neural networks and generalized regression neural networks. J Chem Inf Model 42:1460-1470
10. Burden FR (1999) Robust QSAR models using bayesian regularized neural networks. J Med Chem 42:3183-3187
11. Huuskonen J, Salo M, Taskinen J (1998) Aqueous solubility prediction of drugs based on molecular topology and neural network modeling. J Chem Inf Comput Sci 38:450-456
12. Lucic B, Trinajstic N (1999) Multivariate regression outperforms several robust architectures of neural networks in QSAR modeling. J Chem Inf Model 39:121-132
13. Niculescu SP (2003) Artificial neural networks and genetic algorithms in QSAR. J Mol Struct (Thoechem) 622:71-83
14. Agrafiotis DK, Cedeno W, Lobanov VS (2002) On the use of neural network ensembles in QSAR and QSPR. J Chem Inf Model 42:903-911
15. Guha R, Jurs PC (2005) Interpreting computational neural network QSAR models: a measure of descriptor importance. J Chem Inf Model 45:800-806
16. Manallack DT, Livingstone DJ (1999) Neural networks in drug discovery: have they lived up to their promise? Eur J Med Chem 34:195-208
17. Schneider G (2000) Neural networks are useful tools for drug design. Neural Netw 13:15-16
18. Gawehn E, Hiss JA, Schneider G (2016) Deep learning in drug discovery. Mol Inform 35:3-14
19. Ma J, Sheridan RP, Liaw A, Dahl GE, Svetnik V (2015) Deep neural nets as a method for quantitative structure-activity relationships. J Chem Inf Model 55:263-274
20. Andreas Mayr GK, Unterthiner T, Hochreiter S (2016) DeepTox: toxicity prediction using deep learning. Front Environ Sci 3:80
21. Arel I, Rose DC, Karnowski TP (2010) Deep machine learning - a new frontier in artificial intelligence research [research frontier]. IEEE Comput Intell Mag 5:13-18
22. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436-444
23. Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35:1798-1828
24. Dahl G, Mohamed A-R, Hinton GE (2010) Phone recognition with the mean-covariance restricted Boltzmann machine. Adv Neural Inf Process Syst 2010:469-477
25. Ciresan D, Meier U, Schmidhuber J (2012) Multi-column deep neural networks for image classification. In: Proceedings of the IEEE international conference on computer vision and pattern recognition (CVPR), pp 3642-3649
26. Cireşan DC, Meier U, Gambardella LM, Schmidhuber J (2010) Deep, big, simple neural nets for handwritten digit recognition. Neural Comput 22:3207-3220
27. Basheer IA, Hajmeer M (2000) Artificial neural networks: fundamentals, computing, design, and application. J Microbiol Methods 43:3-31
28. McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5:115-133
29. Larochelle H, Bengio Y, Louradour M, Lamblin P (2009) Exploring strategies for training deep neural networks. J Mach Learn Res 10:1-40
30. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323:533-536
31. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929-1958
32. Owens JD, Houston M, Luebke D, Green S, Stone JE, Phillips JC (2008) GPU computing. Proc IEEE 96:879-899
33. Lusci A, Pollastri G, Baldi P (2013) Deep architectures and deep learning in chemoinformatics: the prediction of aqueous solubility for drug-like molecules. J Chem Inf Model 53:1563-1575
34. Xu Y, Dai Z, Chen F, Gao S, Pei J, Lai L (2015) Deep learning for drug-induced liver injury. J Chem Inf Model 55:2085-2093
35. Aliper A, Plis S, Artemov A, Ulloa A, Mamoshina P, Zhavoronkov A (2016) Deep learning applications for predicting pharmacological properties of drugs and drug repurposing using transcriptomic data. Mol Pharm 13:2524-2530
36. Aspuru-Guzik A, Duvenaud D, Maclaurin D, Aguilera-Iparraguire J, Gomez-Bombarelli R, Hirzel TD, Adams RP (2015) Convolutional networks on graphs for learning molecular fingerprints. In: Proceedings of neural information processing systems, Quebec, pp 2224-2232
37. Team TTD, Al-Rfou R, Alain G, Almahairi A, Angermueller C, Bahdanau D, Ballas N, Bastien F, Bayer J, Belikov A (2016) Theano: a Python framework for fast computation of mathematical expressions. arXiv preprint arXiv:1605.02688
38. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the ACM international conference on multimedia, 2014. ACM, pp 675-678
39. Zeiler MD, Ranzato M, Monga R, Mao M, Yang K, Le QV, Nguyen P, Senior A, Vanhoucke V, Dean J, Hinton GE (2013) On rectified linear units for speech processing. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, Vancouver, pp 3517-3521
40. Tiikkainen P, Franke L (2012) Analysis of commercial and public bioactivity databases. J Chem Inf Model 52:319-326
41. Williams AJ, Ekins S, Tkachenko V (2012) Towards a gold standard: regarding quality in public domain chemistry databases and approaches to improving the situation. Drug Discov Today 17:685-701
42. Kramer C, Kalliokoski T, Gedeck P, Vulpetti A (2012) The experimental uncertainty of heterogeneous PublicKiData. J Med Chem 55:5165-5173
43. Kramer C, Dahl G, Tyrchan C, Ulander J (2016) A comprehensive company database analysis of biological assay variability. Drug Discov Today 21:1213-1221
44. Glick M (2004) Enrichment of extremely noisy high-throughput screening data using a Naive Bayes classifier. J Biomol Screen 9:32-36
45. Guha R (2008) On the interpretation and interpretability of quantitative structure-activity relationship models. J Comput Aided Mol Des 22:857-871
46. Carlsson L, Helgee EA, Boyer S (2009) Interpretation of nonlinear QSAR models applied to AMES mutagenicity data. J Chem Inf Model 49:2551-2558
47. Rosenbaum L, Hinselmann G, Jahn A, Zell A (2011) Interpreting linear support vector machine models with heat map molecule coloring. J Cheminform 3:11
48. Bento AP, Gaulton A, Hersey A, Bellis LJ, Chambers J, Davies M, Krüger FA, Light Y, Mak L, McGlinchey S, Nowotka M, Papadatos G, Santos R, Overington JP (2014) The ChEMBL bioactivity database: an update. Nucleic Acids Res 42:D1083-D1090
49. Gaulton A, Bellis LJ, Bento AP, Chambers J, Davies M, Hersey A, Light Y, McGlinchey S, Michalovich D, Al-Lazikani B, Overington JP (2011) ChEMBL: a large-scale bioactivity database for drug discovery. Nucleic Acids Res 40:D1100-D1107
50. Molecular Operating Environment (MOE) 2013.08. Chemical Computing Group Inc.: Sherbooke St. West, Suite #910, Montreal, QC, Canada, H3A 2R7, 2016
51. Rogers D, Hahn M (2010) Extended-connectivity fingerprints. J Chem Inf Model 50:742-754
52. Landrum G. RDKit: Open-source cheminformatics. http://www.rdkit.org
53. Bender A, Jenkins JL, Scheiber J, Sukuru SCK, Glick M, Davies JW (2009) How similar are similarity searching methods? A principal component analysis of molecular descriptor space. J Chem Inf Model 49:108-119
54. McCallum A, Nigam K (1998) A comparison of event models for Naive Bayes text classification. In: Proceedings of the AAAI-98 workshop on learning for text categorization, vol 752, pp 41-48
55. Lewis DD (1998) Naive (Bayes) at forty: the independence assumption in information retrieval. In: Nédellec C, Rouveirol C (eds) Machine learning: ECML-98. ECML 1998. Lecture notes in computer science, vol 1398. Springer, Berlin, Heidelberg, pp 4-15
56. Jordan A (2002) On discriminative vs. generative classifiers: a comparison of logistic regression and naive bayes. Adv Neural Inf Process Syst 14:841
57. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825-2830
58. Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. Am Stat 46:175-185
59. Svetnik V, Liaw A, Tong C, Culberson JC, Sheridan RP, Feuston BP (2003) Random forest: a classification and regression tool for compound classification and QSAR modeling. J Chem Inf Model 43:1947-1958
60. Breiman L (2001) Random forests. Mach Learn 45:5-32
61. Saeys Y, Inza I, Larranaga P (2007) A review of feature selection techniques in bioinformatics. Bioinformatics 23:2507-2517
62. Díaz-Uriarte R, Alvarez de Andrés S (2006) Gene selection and classification of microarray data using random forest. BMC Bioinform 7:3
63. Mitchell JBO (2014) Machine learning methods in chemoinformatics. Wiley Interdiscipl Rev Comput Mol Sci 4:468-481
64. Barrett SJ, Langdon WB (2006) Advances in the application of machine learning techniques in drug discovery. Des Dev 36:99-110
65. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273-297
66. Chang C-C, Lin C-J (2011) Libsvm. ACM Trans Intell Syst Technol 2:1-27
67. Fan R-E, Chang K-W, Hsieh C-J, Wang X-R, Lin C-J (2008) LIBLINEAR: a library for large linear classification. J Mach Learn Res 9:1871-1874
68. Alvarsson J, Eklund M, Andersson C, Carlsson L, Spjuth O, Wikberg JE (2014) Benchmarking study of parameter variation when using signature fingerprints together with support vector machines. J Chem Inf Model 54:3211-3217
69. Chu H. OpenLDAP lighting memory-mapped database. http://symas.com/mdb/
70. Group TH. Hierarchical data format (HDF). www.hdfgroup.org
71. Matthews BW (1975) Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochim Biophys Acta (BBA) Protein Struct 405:442-451
72. Baldi P, Brunak S, Chauvin Y, Andersen CAF, Nielsen H (2000) Assessing the accuracy of prediction algorithms for classification: an overview. Bioinformatics 16:412-424
73. Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1:80
74. Team RC (2014) In: ISBN 3-900051-07-0