Is Multitask Deep Learning Practical for Pharma?

Journal of Chemical Information and Modeling

Volumn 57, Issue 8, 2017, Pages 2068-2076

  Ramsundar, Bharath ^a   Liu, Bowen ^b   Wu, Zhenqin ^b   Verras, Andreas ^c   Tudor, Matthew ^c   Sheridan, Robert P ^c   Pande, Vijay ^b  

^a Stanford University   (United States)

^b STANFORD UNIVERSITY   (United States)

^c MERCK RESEARCH LABORATORIES   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DECISION TREES; OPEN SOURCE SOFTWARE;

BARRIERS TO ADOPTION; BIOTECH INDUSTRY; DRUG DISCOVERY; HIGH QUALITY; LEARNING PERFORMANCE; OPEN SOURCE IMPLEMENTATION; OPEN SOURCE PLATFORMS;

DEEP LEARNING;

PROTEIN KINASE INHIBITOR; SERINE PROTEINASE INHIBITOR;

CHEMISTRY; DRUG DEVELOPMENT; IC50; MACHINE LEARNING; PROCEDURES; RADIATION ABSORPTION; SOFTWARE; ULTRAVIOLET RADIATION;

ABSORPTION, RADIATION; DRUG DISCOVERY; INHIBITORY CONCENTRATION 50; MACHINE LEARNING; PROTEIN KINASE INHIBITORS; SERINE PROTEINASE INHIBITORS; SOFTWARE; ULTRAVIOLET RAYS;

EID: 85028532336     PISSN: 15499596     EISSN: 1549960X     Source Type: Journal    
DOI: 10.1021/acs.jcim.7b00146     Document Type: Article

References (30)

1. LeCun, Y.; Bengio, Y.; Hinton, G. Deep Learning Nature 2015, 521, 436-444 10.1038/nature14539
2. Deep Learning How I Did It: Merck 1st Place Interview. http://blog.kaggle.com/2012/11/01/deep-learning-how-i-did-it-merck-1st-place-interview/ (accessed Dec 10, 2016).
3. Ma, J.; Sheridan, R. P.; Liaw, A.; Dahl, G. E.; Svetnik, V. Deep Neural Nets as a Method for Quantitative Structure-Activity Relationships J. Chem. Inf. Model. 2015, 55, 263-274 10.1021/ci500747n
4. Ramsundar, B.; Kearnes, S.; Riley, P.; Webster, D.; Konerding, D.; Pande, V. Massively Multitask Networks for Drug Discovery. arXiv:1502.02072 2015.
5. Unterthiner, T.; Mayr, A.; Ünter Klambauer, G.; Steijaert, M.; Wenger, J.; Ceulemans, H.; Hochreiter, S. Deep Learning as an Opportunity in Virtual Screening. Deep Learning and Representation Learning Workshop (NIPS 2014); 2014.
6. Kearnes, S.; Goldman, B.; Pande, V. Modeling Industrial ADMET Data with Multitask Networks. arXiv:1606.08793 2016.
7. Netflix Never Used Its $1 Million Algorithm Due to Engineering Costs. http://arstechnica.com/gadgets/2012/04/netflix-never-used-its-1-million-algorithm-due-to-engineering-costs/ (accessed Dec 10, 2016).
8. Netflix Recommendations Beyond the 5 stars (Part 1). http://techblog.netflix.com/2012/04/netflix-recommendations-beyond-5-stars.html (accessed Dec 10, 2016).
9. Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; Kudlur, M.; Levenberg, J.; Monga, R.; Moore, S.; Murray, D.; Steiner, B.; Tucker, P.; Vasudevan, V.; Warden, P.; Wicke, M.; Yu, Y.; Zheng, X. TensorFlow: A System for Large-Scale Machine Learning. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI), Savannah, GA, 2016; pp 265-283.
10. Bastien, F.; Lamblin, P.; Pascanu, R.; Bergstra, J.; Goodfellow, I.; Bergeron, A.; Bouchard, N.; Warde-Farley, D.; Bengio, Y. Theano: New Features and Speed Improvements. arXiv:1211.5590 2012.
11. Chollet, F. Keras: Deep Learning Library for Theano and Tensorflow; 2015.
12. Weininger, D. SMILES, a Chemical Language and Information System. 1. Introduction to Methodology and Encoding Rules J. Chem. Inf. Model. 1988, 28, 31-36 10.1021/ci00057a005
13. Bemis, G. W.; Murcko, M. A. The Properties of Known Drugs. 1. Molecular Frameworks J. Med. Chem. 1996, 39, 2887-2893 10.1021/jm9602928
14. DeepChem: Deep-learning models for Drug Discovery and Quantum Chemistry. https://github.com/deepchem/deepchem (accessed Dec 10, 2016).
15. Subramanian, G.; Ramsundar, B.; Pande, V.; Denny, R. A. Computational Modeling of β-secretase 1 (BACE-1) Inhibitors using Ligand Based Approaches J. Chem. Inf. Model. 2016, 56, 1936-1949 10.1021/acs.jcim.6b00290
16. Altae-Tran, H.; Ramsundar, B.; Pappu, A., S.; Pande, V. Low Data Drug Discovery with One-Shot Learning ACS Cent. Sci. 2017, 3, 283-293 10.1021/acscentsci.6b00367
17. Rusu, A. A.; Rabinowitz, N. C.; Desjardins, G.; Soyer, H.; Kirkpatrick, J.; Kavukcuoglu, K.; Pascanu, R.; Hadsell, R. Progressive Neural Networks. arXiv:1606.04671 2016.
18. Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; Vanderplas, J.; Passos, A.; Cournapeau, D.; Brucher, M.; Perrot, M.; Duchesnay, é. Scikit-learn: Machine learning in Python Journal of Machine Learning Research (JMLR) 2011, 12, 2825-2830
19. Srivastava, N.; Hinton, G. E.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: a Simple Way to Prevent Neural Networks from Overfitting Journal of Machine Learning Research (JMLR) 2014, 15, 1929-1958
20. Kingma, D.; Ba, J. A Method for Stochastic Optimization. arXiv:1412.6980 2014.
21. Carhart, R. E.; Smith, D. H.; Venkataraghavan, R. Atom Pairs as Molecular Features in Structure-Activity Studies: Definition and Applications J. Chem. Inf. Model. 1985, 25, 64-73 10.1021/ci00046a002
22. Kearsley, S. K.; Sallamack, S.; Fluder, E. M.; Andose, J. D.; Mosley, R. T.; Sheridan, R. P. Chemical Similarity using Physiochemical Property Descriptors J. Chem. Inf. Model. 1996, 36, 118-127 10.1021/ci950274j
23. Sheridan, R. P. Time-Split Cross-Validation as a Method for Estimating the Goodness of Prospective Prediction J. Chem. Inf. Model. 2013, 53, 783-790 10.1021/ci400084k
24. Breiman, L. Random Forests Machine learning 2001, 45, 5-32 10.1023/A:1010933404324
25. Svetnik, V.; Liaw, A.; Tong, C.; Culberson, J. C.; Sheridan, R. P.; Feuston, B. P. Random Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling J. Chem. Inf. Model. 2003, 43, 1947-1958 10.1021/ci034160g
26. Bergstra, J.; Bengio, Y. Random Search for Hyper-Parameter Optimization Journal of Machine Learning Research (JMLR) 2012, 13, 281-305
27. Anastassiadis, T.; Deacon, S. W.; Devarajan, K.; Ma, H.; Peterson, J. R. Comprehensive Assay of Kinase Catalytic Activity Reveals Features of Kinase Inhibitor Selectivity Nat. Biotechnol. 2011, 29, 1039-1045 10.1038/nbt.2017
28. Duvenaud, D. K.; Maclaurin, D.; Iparraguirre, J.; Bombarell, R.; Hirzel, T.; Aspuru-Guzik, A.; Adams, R. P. Convolutional Networks on Graphs for Learning molecular fingerprints Neural Inf. Proc. Sys. (NIPS) 2015, 2224-2232
29. Wu, Z.; Ramsundar, B.; Feinberg, E. N.; Gomes, J.; Geniesse, C.; Pappu, A. S.; Leswing, K.; Pande, V. MoleculeNet: A Benchmark for Molecular Machine Learning. arXiv:1703.00564 2017.
30. Kendall, A.; Gal, Y. What Uncertainties do We Need in Bayesian Deep Learning for Computer Vision? arXiv:1703.04977 2017.