High-dimensional neural network potentials for organic reactions and an improved training algorithm

Journal of Chemical Theory and Computation

Volumn 11, Issue 5, 2015, Pages 2187-2198

  Gastegger, Michael ^a   Marquetand, Philipp ^a  

^a UNIVERSITY OF VIENNA   (Austria)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTIFICIAL NEURAL NETWORK; THEORETICAL MODEL; THERMODYNAMICS;

ALGORITHMS; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); THERMODYNAMICS;

EID: 84929346813     PISSN: 15499618     EISSN: 15499626     Source Type: Journal    
DOI: 10.1021/acs.jctc.5b00211     Document Type: Article

References (97)

1. Levine, I. N. Quantum Chemistry, 7 th ed.; Prentice Hall: Boston, 2013.
2. Friesner, R. A. Ab initio quantum chemistry: Methodology and applications Proc. Natl. Acad. Sci. U. S. A. 2005, 102, 6648-6653
3. Brooks, B. R.; Bruccoleri, R. E.; Olafson, B. D.; States, D. J.; Swaminathan, S.; Karplus, M. CHARMM: A program for macromolecular energy, minimization, and dynamics calculations J. Comput. Chem. 1983, 4, 187-217
4. Mackerell, A. D. Empirical force fields for biological macromolecules: Overview and issues J. Comput. Chem. 2004, 25, 1584-1604
5. Liang, T.; Shin, Y. K.; Cheng, Y.-T.; Yilmaz, D. E.; Vishnu, K. G.; Verners, O.; Zou, C.; Phillpot, S. R.; Sinnott, S. B.; van Duin, A. C. T. Reactive Potentials for Advanced Atomistic Simulations Annu. Rev. Mater. Res. 2013, 43, 109-129
6. Handley, C. M.; Behler, J. Next generation interatomic potentials for condensed systems Eur. Phys. J. B 2014, 87, 1-16
7. Brown, A.; Braams, B. J.; Christoffel, K.; Jin, Z.; Bowman, J. M. Classical and quasiclassical spectral analysis of CH5+ using an ab initio potential energy surface J. Chem. Phys. 2003, 119, 8790-8793
8. Bartók, A. P.; Kondor, R.; Csányi, G. On representing chemical environments Phys. Rev. B 2013, 87, 184115
9. Rupp, M.; Tkatchenko, A.; Müller, K.-R.; von Lilienfeld, O. A. Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning Phys. Rev. Lett. 2012, 108, 058301
10. Ischtwan, J.; Collins, M. A. Molecular potential energy surfaces by interpolation J. Chem. Phys. 1994, 100, 8080-8088
11. Wu, T.; Werner, H.-J.; Manthe, U. First-Principles Theory for the H + CH4 → H2 + CH3 Reaction Science 2004, 306, 2227-2229
12. Takata, T.; Taketsugu, T.; Hirao, K.; Gordon, M. S. Ab initio potential energy surface by modified Shepard interpolation: Application to the CH3 + H2 → CH4 + H reaction J. Chem. Phys. 1998, 109, 4281-4289
13. Crespos, C.; Collins, M. A.; Pijper, E.; Kroes, G. J. Multi-dimensional potential energy surface determination by modified Shepard interpolation for a molecule-surface reaction: H2 + Pt(111) Chem. Phys. Lett. 2003, 376, 566-575
14. Dawes, R.; Thompson, D. L.; Guo, Y.; Wagner, A. F.; Minkoff, M. Interpolating moving least-squares methods for fitting potential energy surfaces: Computing high-density potential energy surface data from low-density ab initio data points J. Chem. Phys. 2007, 126, 184108
15. Dawes, R.; Wang, X.-G.; Carrington, T., Jr. CO Dimer: New Potential Energy Surface and Rovibrational Calculations J. Phys. Chem. A 2013, 117, 7612-7630
16. Vitek, A.; Stachon, M.; Kromer, P.; Snael, V. Towards the Modeling of Atomic and Molecular Clusters Energy by Support Vector Regression. International Conference on Intelligent Networking and Collaborative Systems (INCoS). 2013; pp 121-126.
17. Blank, T. B.; Brown, S. D.; Calhoun, A. W.; Doren, D. J. Neural network models of potential energy surfaces J. Chem. Phys. 1995, 103, 4129-4137
18. Lorenz, S.; Scheffler, M.; Gross, A. Descriptions of surface chemical reactions using a neural network representation of the potential-energy surface Phys. Rev. B 2006, 73, 115431
19. Ludwig, J.; Vlachos, D. G. Ab initio molecular dynamics of hydrogen dissociation on metal surfaces using neural networks and novelty sampling J. Chem. Phys. 2007, 127, 154716
20. Manzhos, S.; Yamashita, K.; Carrington, T., Jr. Fitting sparse multidimensional data with low-dimensional terms Comput. Phys. Commun. 2009, 180, 2002-2012
21. Carbogno, C.; Behler, J.; Groß, A.; Reuter, K. Fingerprints for Spin-Selection Rules in the Interaction Dynamics of O2 at Al(111) Phys. Rev. Lett. 2008, 101, 096104
22. Behler, J.; Reuter, K.; Scheffler, M. Nonadiabatic effects in the dissociation of oxygen molecules at the Al(111) surface Phys. Rev. B 2008, 77, 115421
23. Latino, D. A. R. S.; Fartaria, R. P. S.; Freitas, F. F. M.; Aires-de Sousa, J.; Silva Fernandes, F. M. S. Mapping Potential Energy Surfaces by Neural Networks: The ethanol/Au(111) interface J. Electroanal. Chem. 2008, 624, 109-120
24. Latino, D. A. R. S.; Fartaria, R. P. S.; Freitas, F. F. M.; Aires-De-Sousa, J.; Silva Fer-nandes, F. M. S. Approach to potential energy surfaces by neural networks. A review of recent work Int. J. Quantum Chem. 2010, 110, 432-445
25. Liu, T.; Fu, B.; Zhang, D. H. Six-dimensional potential energy surface of the dissociative chemisorption of HCl on Au(111) using neural networks Sci. China Chem. 2013, 57, 147-155
26. Tafeit, E.; Estelberger, W.; Horejsi, R.; Moeller, R.; Oettl, K.; Vrecko, K.; Reibnegger, G. Neural networks as a tool for compact representation of ab initio molecular potential energy surfaces J. Mol. Graphics Modell. 1996, 14, 12-18
27. Brown, D. F. R.; Gibbs, M. N.; Clary, D. C. Combining ab initio computations, neural networks, and diffusion Monte Carlo: An efficient method to treat weakly bound molecules J. Chem. Phys. 1996, 105, 7597-7604
28. Houlding, S.; Liem, S. Y.; Popelier, P. L. A. A polarizable high-rank quantum topological electrostatic potential developed using neural networks: Molecular dynamics simulations on the hydrogen fluoride dimer Int. J. Quantum Chem. 2007, 107, 2817-2827
29. No, K. T.; Chang, B. H.; Kim, S. Y.; Jhon, M. S.; Scheraga, H. A. Description of the potential energy surface of the water dimer with an artificial neural network Chem. Phys. Lett. 1997, 271, 152-156
30. Cho, K.-H.; No, K. T.; Scheraga, H. A. A polarizable force field for water using an artificial neural network J. Mol. Struct. 2002, 641, 77-91
31. Gassner, H.; Probst, M.; Lauenstein, A.; Hermansson, K. Representation of Intermolecular Potential Functions by Neural Networks J. Phys. Chem. A 1998, 102, 4596-4605
32. Prudente, F. V.; Acioli, P. H.; Soares Neto, J. J. The fitting of potential energy surfaces using neural networks: Application to the study of vibrational levels of H 3+ J. Chem. Phys. 1998, 109, 8801-8808
33. Rocha Filho, T. M.; Oliveira, Z. T.; Malbouisson, L. A. C.; Gargano, R.; Soares Neto, J. J. The use of neural networks for fitting potential energy surfaces: A comparative case study for the H3+ molecule Int. J. Quantum Chem. 2003, 95, 281-288
34. Malshe, M.; Raff, L. M.; Rockley, M. G.; Hagan, M.; Agrawal, P. M.; Komanduri, R. Theoretical investigation of the dissociation dynamics of vibrationally excited vinyl bromide on an ab initio potential-energy surface obtained using modified novelty sampling and feedforward neural networks. II. Numerical application of the method J. Chem. Phys. 2007, 127, 134105
35. Raff, L. M.; Malshe, M.; Hagan, M.; Doughan, D. I.; Rockley, M. G.; Komanduri, R. Ab initio potential-energy surfaces for complex, multichannel systems using modified novelty sampling and feedforward neural networks J. Chem. Phys. 2005, 122, 084104
36. Agrawal, P. M.; Raff, L. M.; Hagan, M. T.; Komanduri, R. Molecular dynamics investigations of the dissociation of SiO2 on an ab initio potential energy surface obtained using neural network methods J. Chem. Phys. 2006, 124, 134306
37. Le, H. M.; Huynh, S.; Raff, L. M. Molecular dissociation of hydrogen peroxide (HOOH) on a neural network ab initio potential surface with a new configuration sampling method involving gradient fitting J. Chem. Phys. 2009, 131, 014107
38. Manzhos, S.; Carrington, T., Jr. Using neural networks to represent potential surfaces as sums of products J. Chem. Phys. 2006, 125, 194105
39. Le, H. M.; Raff, L. M. Cis → trans, trans → cis isomerizations and N-O bond dissociation of nitrous acid (HONO) on an ab initio potential surface obtained by novelty sampling and feed-forward neural network fitting J. Chem. Phys. 2008, 128, 194310
40. Darley, M. G.; Handley, C. M.; Popelier, P. L. A. Beyond Point Charges: Dynamic Polarization from Neural Net Predicted Multipole Moments J. Chem. Theory Comput. 2008, 4, 1435-1448
41. Le, H. M.; Dinh, T. S.; Le, H. V. Molecular Dynamics Investigations of Ozone on an Ab Initio Potential Energy Surface with the Utilization of Pattern-Recognition Neural Network for Accurate Determination of Product Formation J. Phys. Chem. A 2011, 115, 10862-10870
42. Pukrittayakamee, A.; Malshe, M.; Hagan, M.; Raff, L. M.; Narulkar, R.; Bukkapatnum, S.; Komanduri, R. Simultaneous fitting of a potential-energy surface and its corresponding force fields using feedforward neural networks J. Chem. Phys. 2009, 130, 134101
43. Nguyen, H. T. T.; Le, H. M. Modified Feed-Forward Neural Network Structures and Combined-Function-Derivative Approximations Incorporating Exchange Symmetry for Potential Energy Surface Fitting J. Phys. Chem. A 2012, 116, 4629-4638
44. Chen, J.; Xu, X.; Xu, X.; Zhang, D. H. Communication: An accurate global potential energy surface for the OH + CO → H + CO2 reaction using neural networks J. Chem. Phys. 2013, 138, 221104
45. Li, J.; Jiang, B.; Guo, H. Permutation invariant polynomial neural network approach to fitting potential energy surfaces. II. Four-atom systems J. Chem. Phys. 2013, 139, 204103
46. Behler, J. Neural network potential-energy surfaces in chemistry: a tool for large-scale simulations Phys. Chem. Chem. Phys. 2011, 13, 17930-17955
47. Behler, J. Representing potential energy surfaces by high-dimensional neural network potentials J. Phys.: Condens. Matter 2014, 26, 183001
48. Handley, C. M.; Popelier, P. L. A. Potential Energy Surfaces Fitted by Artificial Neural Networks J. Phys. Chem. A 2010, 114, 3371-3383
49. Witkoskie, J. B.; Doren, D. J. Neural Network Models of Potential Energy Surfaces:Prototypical Examples J. Chem. Theory Comput. 2005, 1, 14-23
50. Behler, J.; Lorenz, S.; Reuter, K. Representing molecule-surface interactions with symmetry-adapted neural networks J. Chem. Phys. 2007, 127, 014705
51. Behler, J. Atom-centered symmetry functions for constructing high-dimensional neural network potentials J. Chem. Phys. 2011, 134, 074106
52. Behler, J.; Parrinello, M. Generalized Neural-Network Representation of High-Dimensional Potential-Energy Surfaces Phys. Rev. Lett. 2007, 98, 146401
53. Manzhos, S.; Carrington, T., Jr. A random-sampling high dimensional model representation neural network for building potential energy surfaces J. Chem. Phys. 2006, 125, 084109
54. Artrith, N.; Hiller, B.; Behler, J. Neural network potentials for metals and oxides-First applications to copper clusters at zinc oxide Phys. Status Solidi B 2013, 250, 1191-1203
55. Artrith, N.; Morawietz, T.; Behler, J. High-dimensional neural-network potentials for multicomponent systems: Applications to zinc oxide Phys. Rev. B 2011, 83, 153101
56. Behler, J.; Martoňák, R.; Donadio, D.; Parrinello, M. Metadynamics Simulations of the High-Pressure Phases of Silicon Employing a High-Dimensional Neural Network Potential Phys. Rev. Lett. 2008, 100, 185501
57. Behler, J.; Martoňák, R.; Donadio, D.; Parrinello, M. Pressure-induced phase transitions in silicon studied by neural network-based metadynamics simulations Phys. Status Solidi B 2008, 245, 2618-2629
58. Eshet, H.; Khaliullin, R. Z.; Kühne, T. D.; Behler, J.; Parrinello, M. Ab initio quality neural-network potential for sodium Phys. Rev. B 2010, 81, 184107
59. Khaliullin, R. Z.; Eshet, H.; Kühne, T. D.; Behler, J.; Parrinello, M. Graphite-diamond phase coexistence study employing a neural-network mapping of the ab initio potential energy surface Phys. Rev. B 2010, 81, 100103
60. Seema, P.; Behler, J.; Marx, D. Adsorption of Methanethiolate and Atomic Sulfur at the Cu(111) Surface: A Computational Study J. Phys. Chem. C 2013, 117, 337-348
61. Morawietz, T.; Behler, J. A Density-Functional Theory-Based Neural Network Potential for Water Clusters Including van der Waals Corrections J. Phys. Chem. A 2013, 117, 7356-7366
62. Morawietz, T.; Behler, J. A Full-Dimensional Neural Network Potential-Energy Surface for Water Clusters up to the Hexamer Z. Phys. Chem. 2013, 227, 1559-1581
63. Morawietz, T.; Sharma, V.; Behler, J. A neural network potential-energy surface for the water dimer based on environment-dependent atomic energies and charges J. Chem. Phys. 2012, 136, 064103
64. Puskorius, G. V.; Feldkamp, L. A. Decoupled extended Kalman filter training of feedforward layered networks. IJCNN-91-Seattle International Joint Conference on Neural Networks. 1991; pp 771-777.
65. Cybenko, G. Approximation by superpositions of a sigmoidal function Math. Control Signal Syst. 1989, 2, 303-314
66. Hornik, K.; Stinchcombe, M.; White, H. Multilayer feedforward networks are universal approximators Neural Networks 1989, 2, 359-366
67. Hornik, K. Approximation capabilities of multilayer feedforward networks Neural Networks 1991, 4, 251-257
68. Bishop, C. M. Pattern Recognition and Machine Learning, 1 st ed.; Springer: New York, 2006.
69. Rumelhart, D. E.; Hinton, G. E.; Williams, R. J. Learning representations by back-propagating errors Nature 1986, 323, 533-536
70. Hagan, M.; Menhaj, M. Training feedforward networks with the Marquardt algorithm IEEE Trans. Neural Networks 1994, 5, 989-993
71. Blank, T. B.; Brown, S. D. Adaptive, global, extended Kalman filters for training feedforward neural networks J. Chemom. 2005, 8, 391-407
72. Shah, S.; Palmieri, F.; Datum, M. Optimal filtering algorithms for fast learning in feedforward neural networks Neural Networks 1992, 5, 779-787
73. Murtuza, S.; Chorian, S. F. Node decoupled extended Kalman filter based learning algorithm for neural networks. Proceedings of the 1994 IEEE International Symposium on Intelligent Control. 1994; pp 364-369.
74. Neese, F. The ORCA program system Wiley Interdiscip. Rev.: Comput. Mol. Sci. 2012, 2, 73-78
75. Becke, A. D. Density-functional exchange-energy approximation with correct asymptotic behavior Phys. Rev. A 1988, 38, 3098-3100
76. Dirac, P. A. M. Quantum Mechanics of Many-Electron Systems Proc. R. Soc., Ser. A 1929, 123, 714-733
77. Perdew, J. P. Density-functional approximation for the correlation energy of the inhomogeneous electron gas Phys. Rev. B 1986, 33, 8822-8824
78. Slater, J. C. A Simplification of the Hartree-Fock Method Phys. Rev. 1951, 81, 385-390
79. Vosko, S. H.; Wilk, L.; Nusair, M. Accurate spin-dependent electron liquid correlation energies for local spin density calculations: a critical analysis Can. J. Phys. 1980, 58, 1200-1211
80. Weigend, F.; Ahlrichs, R. Balanced basis sets of split valence, triple zeta valence and quadruple zeta valence quality for H to Rn: Design and assessment of accuracy Phys. Chem. Chem. Phys. 2005, 7, 3297-3305
81. Eichkorn, K.; Treutler, O.; Öhm, H.; Häser, M.; Ahlrichs, R. Auxiliary basis sets to approximate Coulomb potentials Chem. Phys. Lett. 1995, 240, 283-290
82. Vahtras, O.; Almlöf, J.; Feyereisen, M. W. Integral approximations for LCAO-SCF calculations Chem. Phys. Lett. 1993, 213, 514-518
83. Grimme, S.; Antony, J.; Ehrlich, S.; Krieg, H. A consistent and accurate ab initio parametrization of density functional dispersion correction (DFT-D) for the 94 elements H-Pu J. Chem. Phys. 2010, 132, 154104
84. Johnson, E. R.; Becke, A. D. A post-Hartree-Fock model of intermolecular interactions J. Chem. Phys. 2005, 123, 024101-024101-7
85. Plessow, P. Reaction Path Optimization without NEB Springs or Interpolation Algorithms J. Chem. Theory Comput. 2013, 9, 1305-1310
86. Furche, F.; Ahlrichs, R.; Hättig, C.; Klopper, W.; Sierka, M.; Weigend, F. Turbomole Wiley Interdiscip. Rev.: Comput. Mol. Sci. 2014, 4, 91-100
87. TURBOMOLE V6.6 2014, a development of the University of Karlsruhe and Forschungszentrum Karlsruhe GmbH, 1989-2007, TURBOMOLE GmbH: since 2007. Available from http://www.turbomole.com (accessed 03.04.2015).
88. Marx, D.; Hutter, J. Ab Initio Molecular Dynamics: Basic Theory and Advanced Methods, Reprint ed.; Cambridge University Press: Cambridge, 2012.
89. Laio, A.; Parrinello, M. Escaping free-energy minima Proc. Natl. Acad. Sci. U. S. A. 2002, 99, 12562-12566
90. Swope, W. C.; Andersen, H. C.; Berens, P. H.; Wilson, K. R. A computer simulation method for the calculation of equilibrium constants for the formation of physical clusters of molecules: Application to small water clusters J. Chem. Phys. 1982, 76, 637-649
91. Berendsen, H. J. C.; Postma, J. P. M.; van Gunsteren, W. F.; DiNola, A.; Haak, J. R. Molecular dynamics with coupling to an external bath J. Chem. Phys. 1984, 81, 3684-3690
92. Tolman, R. C. The Principles of Statistical Mechanics, New ed.; Dover Publications: New York, 2010.
93. Nguyen, D. H.; Widrow, B. Neural networks for self-learning control systems IEEE Control Systems Magazine 1990, 10, 18-23
94. Plaut, D. C.; Nowlan, S. J.; Hinton, G. E. Experiments on Learning by Back Propagation; Technical Report; 1986.
95. van Rossum, G.; Drale, F. L. Python Reference Manual; PythonLabs: Virginia, USA, 2001., http://www.python.org (accessed 03.04.2015).
96. van der Walt, S.; Colbert, S. C.; Varoquaux, G. The NumPy Array: A Structure for Efficient Numerical Computation Comput. Sci. Eng. 2011, 13, 22-30
97. Ziegler, F. E. The thermal, aliphatic Claisen rearrangement Chem. Rev. 1988, 88, 1423-1452