Prediction Errors of Molecular Machine Learning Models Lower than Hybrid DFT Error

Journal of Chemical Theory and Computation

Volumn 13, Issue 11, 2017, Pages 5255-5264

  Faber, Felix A ^a   Hutchison, Luke ^b   Huang, Bing ^a   Gilmer, Justin ^b   Schoenholz, Samuel S ^b   Dahl, George E ^b   Vinyals, Oriol ^c   Kearnes, Steven ^b   Riley, Patrick F ^b   Von Lilienfeld, O Anatole ^a  

^a UNIVERSITY OF BASEL   (Switzerland)

^b GOOGLE INC   (United States)

^c GOOGLE   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CONFORMATION; DENSITY FUNCTIONAL THEORY; DIPOLE; ELASTIC TISSUE; ENTHALPY; HEAT; HISTOGRAM; LEARNING CURVE; LINEAR REGRESSION ANALYSIS; NEBULIZATION; PREDICTION; RANDOM FOREST; ROOM TEMPERATURE; SAMPLING ERROR;

EID: 85034091857     PISSN: 15499618     EISSN: 15499626     Source Type: Journal    
DOI: 10.1021/acs.jctc.7b00577     Document Type: Article

References (65)

1. Ramakrishnan et al. Sci. Data 2014, 1, 140022
2. Hohenberg, P.; Kohn, W. Inhomogeneous Electron Gas Phys. Rev. 1964, 136, B864 10.1103/PhysRev.136.B864
3. Kohn, W.; Sham, L. J. Self-Consistent Equations Including Exchange and Correlation Effects Phys. Rev. 1965, 140, A1133 10.1103/PhysRev.140.A1133
4. Burke, K. Perspective on density functional theory J. Chem. Phys. 2012, 136, 150901 10.1063/1.4704546
5. Koch, W.; Holthausen, M. C. A Chemist's Guide to Density Functional Theory; Wiley-VCH, 2002; DOI: 10.1002/3527600043.
6. Cohen, A. J.; Mori-Sánchez, P.; Yang, W. Challenges for Density Functional Theory Chem. Rev. 2012, 112, 289-320 10.1021/cr200107z
7. Plata, R. E.; Singleton, D. A. A Case Study of the Mechanism of Alcohol-Mediated Morita Baylis-Hillman Reactions. The Importance of Experimental Observations J. Am. Chem. Soc. 2015, 137, 3811-3826 10.1021/ja5111392
8. Medvedev, M. G.; Bushmarinov, I. S.; Sun, J.; Perdew, J. P.; Lyssenko, K. A. Density functional theory is straying from the path toward the exact functional Science 2017, 355, 49-52 10.1126/science.aah5975
9. Ramakrishnan, R.; Dral, P. O.; Rupp, M.; von Lilienfeld, O. A. Big Data Meets Quantum Chemistry Approximations: The I"-Machine Learning Approach J. Chem. Theory Comput. 2015, 11, 2087-2096 10.1021/acs.jctc.5b00099
10. Ramakrishnan, R.; Dral, P. O.; Rupp, M.; von Lilienfeld, O. A. Quantum chemistry structures and properties of 134 kilo molecules Sci. Data 2014, 1, 140022 10.1038/sdata.2014.22
11. Ruddigkeit, L.; van Deursen, R.; Blum, L. C.; Reymond, J.-L. Enumeration of 166 Billion Organic Small Molecules in the Chemical Universe Database GDB-17 J. Chem. Inf. Model. 2012, 52, 2864-2875 10.1021/ci300415d
12. Huang, B.; von Lilienfeld, O. A. Communication: Understanding molecular representations in machine learning: The role of uniqueness and target similarity J. Chem. Phys. 2016, 145, 161102 10.1063/1.4964627
13. Hansen, K.; Biegler, F.; Ramakrishnan, R.; Pronobis, W.; von Lilienfeld, O. A.; Müller, K.-R.; Tkatchenko, A. Machine Learning Predictions of Molecular Properties: Accurate Many-Body Potentials and Nonlocality in Chemical Space J. Phys. Chem. Lett. 2015, 6, 2326-2331 10.1021/acs.jpclett.5b00831
14. Ramakrishnan, R.; von Lilienfeld, O. A. Many Molecular Properties from One Kernel in Chemical Space Chimia 2015, 69, 182-186 10.2533/chimia.2015.182
15. Rupp, M.; Tkatchenko, A.; Müller, K.-R.; von Lilienfeld, O. A. Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning Phys. Rev. Lett. 2012, 108, 058301 10.1103/PhysRevLett.108.058301
16. Barker, J.; Bulin, J.; Hamaekers, J.; Mathias, S. Localized Coulomb Descriptors for the Gaussian Approximation Potential. arXiv preprint arXiv:1611.05126, 2016.
17. von Lilienfeld, O. A.; Ramakrishnan, R.; Rupp, M.; Knoll, A. Fourier series of atomic radial distribution functions: A molecular fingerprint for machine learning models of quantum chemical properties Int. J. Quantum Chem. 2015, 115, 1084-1093 10.1002/qua.24912
18. Huan, T. D.; Mannodi-Kanakkithodi, A.; Ramprasad, R. Accelerated materials property predictions and design using motif-based fingerprints Phys. Rev. B: Condens. Matter Mater. Phys. 2015, 92, 014106 10.1103/PhysRevB.92.014106
19. Bartók, A. P.; Kondor, R.; Csányi, G. On representing chemical environments Phys. Rev. B: Condens. Matter Mater. Phys. 2013, 87, 184115 10.1103/PhysRevB.87.184115
20. De, S.; Bartók, A. P.; Csányi, G.; Ceriotti, M. Comparing molecules and solids across structural and alchemical space Phys. Chem. Chem. Phys. 2016, 18, 13754-13769 10.1039/C6CP00415F
21. Collins, C. R.; Gordon, G. J.; von Lilienfeld, O. A.; Yaron, D. J. Constant Size Molecular Descriptors For Use With Machine Learning. arXiv preprint arXiv:1701.06649, 2016.
22. Smith, J. S.; Isayev, O.; Roitberg, A. E. ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost Chem. Sci. 2017, 8, 3192-3203 10.1039/C6SC05720A
23. Schütt, K. T.; Arbabzadah, F.; Chmiela, S.; Müller, K. R.; Tkatchenko, A. Quantum-chemical insights from deep tensor neural networks Nat. Commun. 2017, 8, 13890 10.1038/ncomms13890
24. Dral, P. O.; von Lilienfeld, O. A.; Thiel, W. Machine Learning of Parameters for Accurate Semiempirical Quantum Chemical Calculations J. Chem. Theory Comput. 2015, 11, 2120-2125 10.1021/acs.jctc.5b00141
25. Weber, W.; Thiel, W. Orthogonalization corrections for semiempirical methods Theor. Chem. Acc. 2000, 103, 495-506 10.1007/s002149900083
26. Dral, P. O.; Wu, X.; Spörkel, L.; Koslowski, A.; Thiel, W. Semiempirical Quantum-Chemical Orthogonalization-Corrected Methods: Benchmarks for Ground-State Properties J. Chem. Theory Comput. 2016, 12, 1097-1120 10.1021/acs.jctc.5b01047
27. Hansen, K.; Montavon, G.; Biegler, F.; Fazli, S.; Rupp, M.; Scheffler, M.; von Lilienfeld, O. A.; Tkatchenko, A.; Müller, K.-R. Assessment and Validation of Machine Learning Methods for Predicting Molecular Atomization Energies J. Chem. Theory Comput. 2013, 9, 3404-3419 10.1021/ct400195d
28. Kearnes, S.; McCloskey, K.; Berndl, M.; Pande, V.; Riley, P. Molecular Graph Convolutions: Moving Beyond Fingerprints J. Comput.-Aided Mol. Des. 2016, 30, 595-608 10.1007/s10822-016-9938-8
29. Li, Y.; Tarlow, D.; Brockschmidt, M.; Zemel, R. Gated Graph Sequence Neural Networks. Proceeding of ICLR'16; 2016.
30. Curtiss, L. A.; Raghavachari, K.; Redfern, P. C.; Pople, J. A. Assessment of Gaussian-2 and density functional theories for the computation of enthalpies of formation J. Chem. Phys. 1997, 106, 1063-1079 10.1063/1.473182
31. Hickey, A. L.; Rowley, C. N. Benchmarking quantum chemical methods for the calculation of molecular dipole moments and polarizabilities J. Phys. Chem. A 2014, 118, 3678-3687 10.1021/jp502475e
32. Stowasser, R.; Hoffmann, R. What do the Kohn-Sham orbitals and eigenvalues mean? J. Am. Chem. Soc. 1999, 121, 3414 10.1021/ja9826892
33. Sinha, P.; Boesch, S. E.; Gu, C.; Wheeler, R. A.; Wilson, A. K. Harmonic Vibrational Frequencies: Scaling Factors for HF, B3LYP, and MP2Methods in Combination with Correlation Consistent Basis Sets J. Phys. Chem. A 2004, 108, 9213-9217 10.1021/jp048233q
34. Geary, R. C. The Ratio of the Mean Deviation to the Standard Deviation as a Test of Normality Biometrika 1935, 27, 310-332 10.1093/biomet/27.3-4.310
35. DeTar, D. F. Calculation of Entropy and Heat Capacity of Organic Compounds in the Gas Phase. Evaluation of a Consistent Method without Adjustable Parameters. Applications to Hydrocarbons J. Phys. Chem. A 2007, 111, 4464-4477 10.1021/jp066312r
36. Huo, H.; Rupp, M. Unified Representation for Machine Learning of Molecules and Crystals. arXiv preprint arXiv:1704.06439, 2017.
37. Bartok, A. P.; De, S.; Poelking, C.; Bernstein, N.; Kermode, J.; Csanyi, G.; Ceriotti, M. Machine Learning Unifies the Modelling of Materials and Molecules. arXiv preprint arXiv:1706.00179, 2017.
38. Rappe, A. K.; Casewit, C. J.; Colwell, K. S., III; Goddard, W. A.; Skiff, W. M. UFF, a full periodic table force field for molecular mechanics and molecular dynamics simulations J. Am. Chem. Soc. 1992, 114, 10024-10035 10.1021/ja00051a040
39. Rogers, D.; Hahn, M. Extended-connectivity fingerprints J. Chem. Inf. Model. 2010, 50, 742-754 10.1021/ci100050t
40. Besnard, J.; Ruda, G. F.; Setola, V.; Abecassis, K.; Rodriguiz, R. M.; Huang, X.-P.; Norval, S.; Sassano, M. F.; Shin, A. I.; Webster, L. A.; Simeons, F. R. C.; Stojanovski, L.; Prat, A.; Seidah, N. G.; Constam, D. B.; Bickerton, G. R.; Read, K. D.; Wetsel, W. C.; Gilbert, I. H.; Roth, B. L.; Hopkins, A. L. Automated design of ligands to polypharmacological profiles Nature 2012, 492, 215-220 10.1038/nature11691
41. Lounkine, E.; Keiser, M. J.; Whitebread, S.; Mikhailov, D.; Hamon, J.; Jenkins, J. L.; Lavan, P.; Weber, E.; Doak, A. K.; Côté, S.; Shoichet, B. K.; Urban, L. Large-scale prediction and testing of drug activity on side-effect targets Nature 2012, 486, 361-367 10.1038/nature11159
42. Huigens, R. W., III; Morrison, K. C.; Hicklin, R. W.; Flood, T. A., Jr; Richter, M. F.; Hergenrother, P. J. A Ring Distortion Strategy to Construct Stereochemically Complex and Structurally Diverse Compounds from Natural Products Nat. Chem. 2013, 5, 195 10.1038/nchem.1549
43. Todeschini, R.; Consonni, V. Handbook of Molecular Descriptors; Wiley-VCH: Weinheim, 2009; DOI: 10.1002/9783527613106.
44. Faulon, J.-L.; Visco, D. P., Jr; Pophale, R. S. The Signature Molecular Descriptor. 1. Using Extended Valence Sequences in QSAR and QSPR Studies J. Chem. Inf. Comp. Sci. 2003, 43, 707 10.1021/ci020345w
45. Visco, J.; Pophale, R. S.; Rintoul, M. D.; Faulon, J. L. Developing a methodology for an inverse quantitative structure activity relationship using the signature molecular descriptor J. Mol. Graphics Modell. 2002, 20, 429-438 10.1016/S1093-3263(01)00144-9
46. O'Boyle, N. M.; Banck, M.; James, C. A.; Morley, C.; Vandermeersch, T.; Hutchison, G. R. Open Babel: An open chemical toolbox J. Cheminform. 2011, 3, 33 10.1186/1758-2946-3-33
47. Landrum, G. RDKit: Open-source cheminformatics software; 2014; Vol. 3, p 2012. http://www.rdkit.org (accessed Sept 26, 2017).
48. Faber, F. A.; Hutchison, L.; Huang, B.; Gilmer, J.; Schoenholz, S. S.; Dahl, G. E.; Vinyals, O.; Kearnes, S.; Riley, P. F.; von Lilienfeld, O. A. Fast machine learning models of electronic and energetic properties consistently reach approximation errors better than DFT accuracy. arXiv preprint arXiv:1702.05532, 2017.
49. Müller, K.-R.; Mika, S.; Rätsch, G.; Tsuda, K.; Schölkopf, B. An introduction to kernel-based learning algorithms IEEE transactions on neural networks 2001, 12, 181-201 10.1109/72.914517
50. Schölkopf, B.; Smola, A. J. Learning with kernels: support vector machines, regularization, optimization, and beyond; MIT Press: 2002.
51. Vovk, V. Kernel Ridge Regression. In Empirical Inference: Festschrift in Honor of Vladimir N. Vapnik; Schölkopf, B.; Luo, Z.; Vovk, V., Eds.; Springer Berlin Heidelberg: Berlin, Heidelberg, 2013; pp 105-116; DOI: 10.1007/978-3-642-41136-6-11.
52. Hastie, T.; Tibshirani, R.; Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed.; Springer: New York, 2011; DOI: 10.1007/978-0-387-84858-7.
53. Hoerl, A. E.; Kennard, R. W. Ridge Regression Biased Estimation for Nonorthogonal Problems Technometrics 2000, 42, 80-86 10.1080/00401706.2000.10485983
54. Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; Vanderplas, J.; Passos, A.; Cournapeau, D.; Brucher, M.; Perrot, M.; Duchesnay, E. scikit-learn: Machine Learning in Python J. Mach. Learn. Res. 2011, 12, 2825-2830
55. Neal, R. M. Bayesian Learning for Neural Networks; Springer-Verlag New York, Inc.: Secaucus, NJ, USA, 1996; DOI: 10.1007/978-1-4612-0745-0.
56. Zou, H.; Hastie, T. Regularization and variable selection via the elastic net J. R. Stat. Soc. Series. B Stat. Methodol. 2005, 67, 301-320 10.1111/j.1467-9868.2005.00503.x
57. Breiman, L. Random forests Machine learning 2001, 45, 5-32 10.1023/A:1010933404324
58. Duvenaud, D. K.; Maclaurin, D.; Iparraguirre, J.; Bombarell, R.; Hirzel, T.; Aspuru-Guzik, A.; Adams, R. P. Convolutional Networks on Graphs for Learning Molecular Fingerprints. Advances in Neural Information Processing Systems; 2015; pp 2215-2223.
59. Desautels, T.; Krause, A.; Burdick, J. W. Parallelizing Exploration-Exploitation Tradeoffs in Gaussian Process Bandit Optimization J. Mach. Learn. Res. 2014, 15, 4053-4103
60. Google HyperTune. https://cloud.google.com/ml/ (accessed 2016).
61. Tew, D. P.; Klopper, W.; Heckert, M.; Gauss, J. Basis Set Limit CCSD(T) Harmonic Vibrational Frequencies J. Phys. Chem. A 2007, 111, 11242-11248 10.1021/jp070851u
62. Müller, K.-R.; Finke, M.; Murata, N.; Schulten, K.; Amari, S. A numerical study on learning curves in stochastic multilayer feedforward networks Neural Comput. 1996, 8, 1085-1106 10.1162/neco.1996.8.5.1085
63. Huang, B.; von Lilienfeld, O. A. The "DNA" of chemistry: Scalable quantum machine learning with "amons. arXiv preprint arXiv:1707.04146, 2017.
64. Gilmer, J.; Schoenholz, S. S.; Riley, P. F.; Vinyals, O.; Dahl, G. E. Neural Message Passing for Quantum Chemistry. Proceedings of the 34nd International Conference on Machine Learning, ICML 2017, 2017.
65. Kirkpatrick, P.; Ellis, C. Chemical space Nature 2004, 432, 823 10.1038/432823a