Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics

Journal of Chemical Physics

Volumn 148, Issue 24, 2018, Pages

  Wehmeyer, Christoph ^a   Noé, Frank ^a  

^a FREIE UNIVERSITÄT BERLIN   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

CHEMICAL MODIFICATION; DEEP NEURAL NETWORKS; LEARNING SYSTEMS; MOLECULAR DYNAMICS; RANDOM PROCESSES; STOCHASTIC SYSTEMS;

CHEMICAL SCIENCE; COLLECTIVE VARIABLES; DIMENSION REDUCTION; HIGH-DIMENSIONAL FEATURE SPACE; LEARNING TECHNIQUES; LINEAR DIMENSION REDUCTION; LOW DIMENSIONAL; MOLECULAR KINETICS;

DEEP LEARNING;

ARTICLE; AUTOENCODER; DEEP LEARNING; EMBEDDING; KINETICS; MARKOV CHAIN;

EID: 85044071429     PISSN: 00219606     EISSN: None     Source Type: Journal    
DOI: 10.1063/1.5011399     Document Type: Article

References (97)

1. M. Shirts and V. S. Pande, Science 290, 1903 (2000). 10.1126/science.290.5498.1903
2. I. Buch, M. J. Harvey, T. Giorgino, D. P. Anderson, and G. De Fabritiis, J. Chem. Inf. Model. 50, 397 (2010). 10.1021/ci900455r
3. D. E. Shaw, P. Maragakis, K. Lindorff-Larsen, S. Piana, R. Dror, M. Eastwood, J. Bank, J. Jumper, J. Salmon, Y. Shan, and W. Wriggers, Science 330, 341 (2010). 10.1126/science.1187409
4. S. Pronk, S. Páll, R. Schulz, P. Larsson, P. Bjelkmar, R. Apostolov, M. R. Shirts, J. C. Smith, P. M. Kasson, D. van der Spoel, B. Hess, and E. Lindahl, Bioinformatics 29, 845 (2013). 10.1093/bioinformatics/btt055
5. S. Doerr, M. J. Harvey, F. Noé, and G. D. Fabritiis, J. Chem. Theory Comput. 12, 1845 (2016). 10.1021/acs.jctc.6b00049
6. J.-H. Prinz, H. Wu, M. Sarich, B. Keller, M. Senne, M. Held, J. D. Chodera, C. Schütte, and F. Noé, J. Chem. Phys. 134, 174105 (2011). 10.1063/1.3565032
7. An Introduction to Markov State Models and Their Application to Long Timescale Molecular Simulation, edited by G. R. Bowman, V. S. Pande, and F. Noé (Springer, The Netherlands, 2014).
8. M. Sarich and C. Schütte, Metastability and Markov State Models in Molecular Dynamics, Courant Lecture Notes (American Mathematical Society, 2013).
9. F. Noé, C. Schütte, E. Vanden-Eijnden, L. Reich, and T. R. Weikl, Proc. Natl. Acad. Sci. U. S. A. 106, 19011 (2009). 10.1073/pnas.0905466106
10. G. R. Bowman, K. A. Beauchamp, G. Boxer, and V. S. Pande, J. Chem. Phys. 131, 124101 (2009). 10.1063/1.3216567
11. K. Lindorff-Larsen, S. Piana, R. O. Dror, and D. E. Shaw, Science 334, 517 (2011). 10.1126/science.1208351
12. S. K. Sadiq, F. Noé, and G. De Fabritiis, Proc. Natl. Acad. Sci. U. S. A. 109, 20449 (2012). 10.1073/pnas.1210983109
13. K. J. Kohlhoff, D. Shukla, M. Lawrenz, G. R. Bowman, D. E. Konerding, D. Belov, R. B. Altman, and V. S. Pande, Nat. Chem. 6, 15 (2014). 10.1038/nchem.1821
14. I. Buch, T. Giorgino, and G. De Fabritiis, Proc. Natl. Acad. Sci. U. S. A. 108, 10184 (2011). 10.1073/pnas.1103547108
15. D.-A. Silva, G. R. Bowman, A. Sosa-Peinado, and X. Huang, PLoS Comput. Biol. 7, e1002054 (2011). 10.1371/journal.pcbi.1002054
16. N. Plattner and F. Noé, Nat. Commun. 6, 7653 (2015). 10.1038/ncomms8653
17. N. Plattner, S. Doerr, G. D. Fabritiis, and F. Noé, Nat. Chem. 9, 1005 (2017). 10.1038/nchem.2785
18. H. Wu, A. S. J. S. Mey, E. Rosta, and F. Noé, J. Chem. Phys. 141, 214106 (2014). 10.1063/1.4902240
19. E. Rosta and G. Hummer, J. Chem. Theory Comput. 11, 276 (2015). 10.1021/ct500719p
20. H. Wu, F. Paul, C. Wehmeyer, and F. Noé, Proc. Natl. Acad. Sci. U. S. A. 113, E3221 (2016). 10.1073/pnas.1525092113
21. A. S. J. S. Mey, H. Wu, and F. Noé, Phys. Rev. X 4, 041018 (2014). 10.1103/physrevx.4.041018
22. F. Paul, C. Wehmeyer, E. T. Abualrous, H. Wu, M. D. Crabtree, J. Schöneberg, J. Clarke, C. Freund, T. R. Weikl, and F. Noé, Nat. Commun. 8, 1095 (2017). 10.1038/s41467-017-01163-6
23. R. Casasnovas, V. Limongelli, P. Tiwary, P. Carloni, and M. Parrinello, J. Am. Chem. Soc. 139, 4780 (2017). 10.1021/jacs.6b12950
24. P. Tiwary, J. Mondal, and B. J. Berne, Sci. Adv. 3, e1700014 (2017). 10.1126/sciadv.1700014
25. F. Noé and C. Clementi, Curr. Opin. Struct. Biol. 43, 141 (2017). 10.1016/j.sbi.2017.02.006
26. J. Preto and C. Clementi, Phys. Chem. Chem. Phys. 16, 19181 (2014). 10.1039/c3cp54520b
27. J. McCarty and M. Parrinello, J. Chem. Phys. 147, 204109 (2017). 10.1063/1.4998598
28. A. L. Ferguson, A. Z. Panagiotopoulos, P. G. Debenedetti, and I. G. Kevrekidis, Proc. Natl. Acad. Sci. U. S. A. 107, 13597 (2010). 10.1073/pnas.1003293107
29. M. Ceriotti, G. A. Tribello, and M. Parrinello, Proc. Natl. Acad. Sci. U. S. A. 108, 13023 (2011). 10.1073/pnas.1108486108
30. B. Hashemian, D. Millán, and M. Arroyo, J. Chem. Phys. 139, 214101 (2013). 10.1063/1.4830403
31. Y. LeCun, Y. Bengio, and G. Hinton, Nature 521, 436 (2015). 10.1038/nature14539
32. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot et al., Nature 529, 484 (2016). 10.1038/nature16961
33. M. Rupp, A. Tkatchenko, K.-R. Müller, and O. A. von Lilienfeld, Phys. Rev. Lett. 108, 058301 (2012). 10.1103/physrevlett.108.058301
34. R. Gómez-Bombarelli, D. Duvenaud, J. M. Hernández-Lobato, J. Aguilera-Iparraguirre, T. D. Hirzel, R. P. Adams, and A. Aspuru-Guzik, "Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules," ACS Cent. Sci. (published online). 10.1021/acscentsci.7b00572
35. G. B. Goh, N. O. Hodas, and A. Vishnu, J. Comput. Chem. 38, 1291 (2017). 10.1002/jcc.24764
36. K. T. Schütt, F. Arbabzadah, S. Chmiela, K. R. Müller, and A. Tkatchenko, Nat. Commun. 8, 13890 (2017). 10.1038/ncomms13890
37. E. Schneider, L. Dai, R. Q. Topper, C. Drechsel-Grau, and M. E. Tuckerman, Phys. Rev. Lett. 119, 150601 (2017). 10.1103/physrevlett.119.150601
38. J. Jiménez, S. Doerr, G. Martínez-Rosell, A. S. Rose, and G. De Fabritiis, Bioinformatics 33, 3036 (2017). 10.1093/bioinformatics/btx350
39. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning Internal Representations by Error Propagation (MIT Press, Cambridge, MA, USA, 1986), pp. 318-362.
40. P. Baldi, in Proceedings of ICML Workshop on Unsupervised and Transfer Learning, edited by I. Guyon, G. Dror, V. Lemaire, G. Taylor, and D. Silver (Proceedings of Machine Learning Research, PMLR, Bellevue, Washington, USA, 2012), Vol. 27, pp. 37-49.
41. B. Peters and B. L. Trout, J. Chem. Phys. 125, 054108 (2006). 10.1063/1.2234477
42. M. A. Rohrdanz, W. Zheng, M. Maggioni, and C. Clementi, J. Chem. Phys. 134, 124116 (2011). 10.1063/1.3569857
43. M. A. Rohrdanz, W. Zheng, and C. Clementi, Annu. Rev. Phys. Chem. 64, 295 (2013). 10.1146/annurev-physchem-040412-110006
44. H. Stamati, C. Clementi, and L. E. Kavraki, Proteins 78, 223 (2010). 10.1002/prot.22526
45. P. Das, M. Moll, H. Stamati, L. E. Kavraki, and C. Clementi, Proc. Natl. Acad. Sci. U. S. A. 103, 9885 (2006). 10.1073/pnas.0603553103
46. B. Peters, J. Chem. Phys. 125, 241101 (2006). 10.1063/1.2409924
47. J.-H. Prinz, J. D. Chodera, and F. Noé, Phys. Rev. X 4, 011020 (2014). 10.1103/physrevx.4.011020
48. A. Altis, P. H. Nguyen, R. Hegger, and G. Stock, J. Chem. Phys. 126, 244111 (2007). 10.1063/1.2746330
49. S. V. Krivov and M. Karplus, Proc. Natl. Acad. Sci. U. S. A. 101, 14766 (2004). 10.1073/pnas.0406234101
50. F. Noé and F. Nüske, Multiscale Model. Simul. 11, 635 (2013). 10.1137/110858616
51. F. Nüske, B. G. Keller, G. Pérez-Hernández, A. S. J. S. Mey, and F. Noé, J. Chem. Theory Comput. 10, 1739 (2014). 10.1021/ct4009156
52. H. Wu and F. Noé, e-print arXiv:1707.04659 (2017).
53. G. Pérez-Hernández, F. Paul, T. Giorgino, G. D. Fabritiis, and F. Noé, J. Chem. Phys. 139, 015102 (2013). 10.1063/1.4811489
54. C. R. Schwantes and V. S. Pande, J. Chem. Theory Comput. 9, 2000 (2013). 10.1021/ct300878a
55. L. Molgedey and H. G. Schuster, Phys. Rev. Lett. 72, 3634 (1994). 10.1103/physrevlett.72.3634
56. A. Ziehe and K.-R. Müller, ICANN 98 (Springer Science and Business Media, 1998), pp. 675-680.
57. I. Mezić, Nonlinear Dyn. 41, 309 (2005). 10.1007/s11071-005-2824-x
58. P. J. Schmid and J. Sesterhenn, in 61st Annual Meeting of the APS Division of Fluid Dynamics (American Physical Society, 2008).
59. J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton, and J. N. Kutz, J. Comput. Dyn. 1, 391 (2014). 10.3934/jcd.2014.1.391
60. S. Klus, F. Nüske, P. Koltai, H. Wu, I. Kevrekidis, C. Schütte, and F. Noé, "Data-driven model reduction and transfer operator approximation," e-print arXiv:1703.10112 (2017).
61. S. Harmeling, A. Ziehe, M. Kawanabe, and K.-R. Müller, Neural Comput. 15, 1089 (2003). 10.1162/089976603765202677
62. C. R. Schwantes and V. S. Pande, J. Chem. Theory Comput. 11, 600 (2015). 10.1021/ct5007357
63. M. O. Williams, C. W. Rowley, and I. G. Kevrekidis, J. Computational Dynamics 2, 247-265 (2015). 10.3934/jcd.2015005
64. F. Nüske, R. Schneider, F. Vitalini, and F. Noé, J. Chem. Phys. 144, 054105 (2016). 10.1063/1.4940774
65. S. L. Brunton, J. L. Proctor, and J. N. Kutz, Proc. Natl. Acad. Sci. U. S. A. 113, 3932 (2016). 10.1073/pnas.1517384113
66. M. P. Harrigan and V. S. Pande, bioRxiv:123752 (2017).
67. M. O. Williams, I. G. Kevrekidis, and C. W. Rowley, J. Nonlinear Sci. 25, 1307 (2015). 10.1007/s00332-015-9258-5
68. A. Mardt, L. Pasquali, H. Wu, and F. Noé, Nat. Commun. 9, 5 (2018). 10.1038/s41467-017-02388-1
69. P. Baldi and K. Hornik, J. Neural Networks 2, 53 (1989). 10.1016/0893-6080(89)90014-2
70. G. E. Hinton, Science 313, 504 (2006). 10.1126/science.1127647
71. Y. Wang, H. Yao, and S. Zhao, Neurocomputing 184, 232 (2016). 10.1016/j.neucom.2015.08.104
72. W. M. Brown, S. Martin, S. N. Pollock, E. A. Coutsias, and J.-P. Watson, J. Chem. Phys. 129, 064118 (2008). 10.1063/1.2968610
73. S. Doerr, I. Ariz-Extreme, M. J. Harvey, and G. De Fabritiis, e-print arXiv:1710.10629 [stat.ML] (2017).
74. B. Lusch, J. N. Kutz, and S. L. Brunton, e-print arXiv:1712.09707 [math.DS] (2017).
75. C. X. Hernández, H. K. Wayment-Steele, M. M. Sultan, B. E. Husic, and V. S. Pande, e-print arXiv:1711.08576 [stat.ML] (2017).
76. M. M. Sultan, H. K. Wayment-Steele, and V. S. Pande, e-print arXiv:1801.00636 [stat.ML] (2018).
77. S. E. Otto and C. W. Rowley, e-print arXiv:1712.01378 [math.DS] (2017).
78. M. W. Diederik and P. Kingma, in ICLR, 2014.
79. A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, and B. Frey, e-print arXiv:1511.05644 (2016).
80. H. Wu, F. Nüske, F. Paul, S. Klus, P. Koltai, and F. Noé, J. Chem. Phys. 146, 154104 (2017). 10.1063/1.4979344
81. I. Horenko, C. Hartmann, C. Schütte, and F. Noé, Phys. Rev. E 76, 016706 (2007). 10.1103/physreve.76.016706
82. H. Hotelling, Biometrika 28, 321 (1936). 10.1093/biomet/28.3-4.321
83. F. Noé and C. Clementi, J. Chem. Theory Comput. 11, 5002 (2015). 10.1021/acs.jctc.5b00553
84. A. Paszke, S. Gross, S. Chintala, and G. Chanan, "Tensors and dynamic neural networks in python with strong gpu acceleration," https://github.com/pytorch/pytorch (2017).
85. A. L. Maas, A. Y. Hannun, and A. Y. Ng, in ICML Workshop on Deep Learning for Audio, Speech and Language Processing, 2013.
86. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, J. Mach. Learn. Res. 15, 1929-1958 (2014).
87. D. P. Kingma and J. Ba, e-print arXiv:1412.6980v9 [cs.LG] (2014).
88. K. He, X. Zhang, S. Ren, and J. Sun, in 2015 IEEE International Conference on Computer Vision (ICCV) (IEEE, 2015).
89. M. K. Scherer, B. Trendelkamp-Schroer, F. Paul, G. Pérez-Hernández, M. Hoffmann, N. Plattner, C. Wehmeyer, J.-H. Prinz, and F. Noé, J. Chem. Theory Comput. 11, 5525 (2015). 10.1021/acs.jctc.5b00743
90. W. C. Swope, J. W. Pitera, and F. Suits, J. Phys. Chem. B 108, 6571 (2004). 10.1021/jp037421y
91. F. Nüske, H. Wu, J.-H. Prinz, C. Wehmeyer, C. Clementi, and F. Noé, J. Chem. Phys. 146, 094104 (2017). 10.1063/1.4976518
92. M. J. Harvey, G. Giupponi, and G. D. Fabritiis, J. Chem. Theory Comput. 5, 1632 (2009). 10.1021/ct9000685
93. Y. Zheng, B. Lindner, J.-H. Prinz, F. Noé, and J. C. Smith, J. Chem. Phys. 139, 175102 (2013). 10.1063/1.4824071
94. B. Hashemian and M. Arroyo, J. Chem. Phys. 142, 044102 (2015). 10.1063/1.4906425
95. R. T. McGibbon and V. S. Pande, J. Chem. Phys. 142, 124105 (2015). 10.1063/1.4916292
96. J. D. Hunter, Comput. Sci. Eng. 9, 90 (2007). 10.1109/mcse.2007.55
97. W. Humphrey, A. Dalke, and K. Schulten, J. Molec. Graphics 14, 33-38 (1966). 10.1016/0263-7855(96)00018-5