Variational encoding of complex dynamics

Physical Review E

Volumn 97, Issue 6, 2018, Pages

  Hernández, Carlos X ^a   Wayment Steele, Hannah K ^a   Sultan, Mohammad M ^a   Husic, Brooke E ^a   Pande, Vijay S ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CHEMICAL ANALYSIS; DEEP LEARNING; SIGNAL ENCODING; TIME SERIES ANALYSIS;

BIOPHYSICAL SYSTEMS; BROWNIAN DYNAMICS; COMPLEX DYNAMICS; HIGH-DIMENSIONAL; INDIVIDUAL FEATURES; NONLINEAR PROCESS; UNDERLYING DYNAMICS; VARIATIONAL DYNAMICS;

DYNAMICS;

ARTICLE; AUTOENCODER; BIOPHYSICS; DEEP LEARNING; EMBEDDING; PROTEIN FOLDING;

EID: 85048887085     PISSN: 24700045     EISSN: 24700053     Source Type: Journal    
DOI: 10.1103/PhysRevE.97.062412     Document Type: Article

References (51)

1. M. Shirts and V. S. Pande, Screen savers of the world unite! Science 290, 1903 (2000). SCIEAS 0036-8075 10.1126/science.290.5498.1903
2. D. E. Shaw, Anton, a special-purpose machine for molecular dynamics simulation, Commun. ACM 51, 91 (2008). CACMA2 0001-0782 10.1145/1364782.1364802
3. D. Shukla, C. X. Hernández, J. K. Weber, and V. S. Pande, Markov state models provide insights into dynamic modulation of protein function, Acc. Chem. Res. 48, 414 (2015). ACHRE4 0001-4842 10.1021/ar5002999
4. J.-H. Prinz, Markov models of molecular kinetics: Generation and validation, J. Chem. Phys. 134, 174105 (2011). JCPSA6 0021-9606 10.1063/1.3565032
5. B. E. Husic and V. S. Pande, Markov state models: From an art to a science, J. Am. Chem. Soc. (2018).
6. F. Noé and F. Nüske, A variational approach to modeling slow processes in stochastic dynamical systems, Multiscale Model. Simul. 11, 635 (2013). 1540-3459 10.1137/110858616
7. Y. Naritomi and S. Fuchigami, Slow dynamics in protein fluctuations revealed by time-structure based independent component analysis: the case of domain motions, J. Chem. Phys. 134, 065101 (2011). JCPSA6 0021-9606 10.1063/1.3554380
8. F. Pérez-Hernández, F. Paul, T. Giorgino, G. De Fabritiis, and F. Noé, Identification of slow molecular order parameters for Markov model construction, J. Chem. Phys. 139, 015102 (2013). JCPSA6 0021-9606 10.1063/1.4811489
9. C. R. Schwantes and V. S. Pande, Improvements in Markov state model construction reveal many nonnative interactions in the folding of NTL9, J. Chem. Theory Comput. 9, 2000 (2013). 1549-9618 10.1021/ct300878a
10. C. R. Schwantes and V. S. Pande, Modeling molecular kinetics with tICA and the kernel trick, J. Chem. Theory Comput. 11, 600 (2015). 1549-9618 10.1021/ct5007357
11. R. T. McGibbon and V. S. Pande, Variational cross-validation of slow dynamical modes in molecular kinetics, J. Chem. Phys. 142, 124105 (2015). JCPSA6 0021-9606 10.1063/1.4916292
12. H. Wu, Variational koopman models: Slow collective variables and molecular kinetics from short off-equilibrium simulations, J. Chem. Phys. 146, 154104 (2017). JCPSA6 0021-9606 10.1063/1.4979344
13. M. P. Harrigan and V. S. Pande, Towards robust dynamical models of biomolecules. Ph.D. Thesis, Stanford University (2017).
14. R. R. Coifman and S. Lafon, Diffusion maps, Appl. Comput. Harmon. Anal. 21, 5 (2006). 1063-5203 10.1016/j.acha.2006.04.006
15. M. P. Harrigan and V. S. Pande, Landmark kernel tICA for conformational dynamics, bioRxiv 123752 (2017).
16. D. P. Kingma and M. Welling, Auto-encoding variational Bayes, arXiv:1312.6114 (2013).
17. D. J. Rezende, S. Mohamed, and D. Wierstra, in Proceedings of the 31st International Conference on Machine Learning, edited by E. P. Xing and T. Jebara, Vol. 32 (PMLR, Bejing, China, 2014), pp. 1278-1286.
18. R. Hecht-Nielsen, Replicator neural networks for universal optimal source coding, Science 269, 1860 (1995). 10.1126/science.269.5232.1860
19. G. E. Hinton and R. R. Salakhutdinov, Reducing the dimensionality of data with neural networks, Science 313, 504 (2006). 10.1126/science.1127647
20. Y. Bengio, L. Yao, G. Alain, and P. Vincent, Generalized denoising auto-encoders as generative models. In Proceedings of the 26th International Conference on Neural Information Processing Systems-Volume 1 (Curran Associates Inc., Lake Tahoe, 2013), p. 899.
21. K. Pearson, LIII. On lines and planes of closest fit to systems of points in space, London, Edinburgh, Dublin Philos. Mag. J. Sci. 2, 559 (2010). 1941-5982 10.1080/14786440109462720
22. R. T. McGibbon, B. E. Husic, and V. S. Pande, Identification of simple reaction coordinates from complex dynamics, J. Chem. Phys. 146, 044109 (2017). JCPSA6 0021-9606 10.1063/1.4974306
23. A. Mardt, L. Pasquali, H. Wu, and F. Noé, Vampnets for deep learning of molecular kinetics, Nat. Commun. 9, 5 (2018). 2041-1723 10.1038/s41467-017-02388-1
24. C. Wehmeyer and F. Noé, Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics, J. Chem. Phys. 148, 241703 (2018). JCPSA6 0021-9606 10.1063/1.5011399
25. J. Walker, C. Doersch, A. Gupta, and M. Hebert, Computer Vision-ECCV 2016, edited by B. Leibe, J. Matas, N. Sebe, and M. Welling (Springer, Cham, 2016), pp. 835-851.
26. W. Grathwohl and A. Wilson, Disentangling space and time in video with hierarchical variational auto-encoders, arXiv:1612.04440 (2016).
27. M. Fraccaro, S. Kamronn, U. Paquet, and O. Winther, Advances in Neural Information Processing Systems 30, edited by I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Curran Associates, Inc., 2017), pp. 3601-3610.
28. W. Bao, J. Yue, and Y. Rao, A deep learning framework for financial time series using stacked autoencoders and long-short term memory, PloS One 12, e0180944 (2017). 1932-6203 10.1371/journal.pone.0180944
29. J. Cui, X. Liu, Y. Wang, and H. Liu, Deep reconstruction model for dynamic pet images, PloS One 12, e0184667 (2017). 1932-6203 10.1371/journal.pone.0184667
30. S. Doerr, I. Ariz, M. J. Harvey, and G. De Fabritiis, Dimensionality reduction methods for molecular simulations, arXiv:1710.10629 (2017).
31. K. Simonyan, A. Vedaldi, and A. Zisserman, Deep inside convolutional networks: Visualising image classification models and saliency maps, arXiv:1312.6034 (2013).
32. J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller, Striving for simplicity: The all convolutional net, arXiv:1412.6806 (2014).
33. K. Simonyan, A. Vedaldi, and A. Zisserman, Deep inside convolutional networks: Visualising image classification models and saliency maps, arXiv:1312.6034 (2013).
34. B. E. Husic and V. S. Pande, Note: MSM lag time cannot be used for variational model selection, J. Chem. Phys. 147, 176101 (2017). JCPSA6 0021-9606 10.1063/1.5002086
35. H. K. Wayment-Steele and V. S. Pande, Note: Variational encoding of protein dynamics benefits from maximizing latent autocorrelation, arXiv:1803.06449 (2018).
36. F. Noé, Dynamical fingerprints for probing individual relaxation processes in biomolecular dynamics with simulations and kinetic experiments, Proc. Natl. Acad. Sci. USA 108, 4822 (2011). PNASA6 0027-8424 10.1073/pnas.1004646108
37. M. P. Harrigan, MSMBuilder: Statistical models for biomolecular dynamics, Biophys. J. 112, 10 (2017). BIOJAU 0006-3495 10.1016/j.bpj.2016.10.042
38. K. Müller and L. D. Brown, Location of saddle points and minimum energy paths by a constrained simplex optimization procedure, Theor. Chim. Acta 53, 75 (1979). TCHAAM 0040-5744 10.1007/BF00547608
39. P. Ramachandran, B. Zoph, and Q. V. Le, Searching for activation functions, CoRR abs/1710.05941 (2017).
40. D. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv:1412.6980 (2014).
41. D. Sculley, Web-scale k-means clustering. In Proceedings of the 19th International Conference on World Wide Web (WWW'10), 1177 (ACM Press, New York, 2010).
42. F. Pedregosa, Scikit-learn: Machine learning in Python, J. Mach. Learn. Res. 12, 2825 (2011).
43. P. Metzner, F. Noé, and C. Schütte, Estimating the sampling error: Distribution of transition matrices and functions of transition matrices for given trajectory data., Phys. Rev. E 80, 021106 (2009). PLEEE8 1539-3755 10.1103/PhysRevE.80.021106
44. C. X. Hernández and V. S. Pande, MDEntropy: Information-theoretic analyses for molecular dynamics, J. Open Source Softw. 2, 427 (2017). 2475-9066 10.21105/joss.00427
45. C. X. Hernández, M. P. Harrigan, M. M. Sultan, and V. S. Pande, MSMExplorer: Data visualizations for biomolecular dynamics, J. Open Source Softw. 2 (2017).
46. K. Lindorff-Larsen, S. Piana, R. O. Dror, and D. E. Shaw, How fast-folding proteins fold. 334, 517 (2011).
47. R. T. McGibbon, MDTraj: A modern open library for the analysis of molecular dynamics trajectories, Biophys. J. 109, 1528 (2015). BIOJAU 0006-3495 10.1016/j.bpj.2015.08.015
48. B. E. Husic, R. T. McGibbon, M. M. Sultan, and V. S. Pande, Optimized parameter selection reveals trends in Markov state models for protein folding, J. Chem. Phys. 145, 194103 (2016). JCPSA6 0021-9606 10.1063/1.4967809
49. F. Noé and C. Clementi, Kinetic distance and kinetic maps from molecular dynamics simulation, J. Chem. Theory Comput. 11, 5002 (2015). 1549-9618 10.1021/acs.jctc.5b00553
50. R. T. McGibbon, Osprey: Hyperparameter optimization for machine learning, J. Open Source Softw. 1, 34 (2016). 2475-9066 10.21105/joss.00034
51. See Supplemental Material at http://link.aps.org/supplemental/10.1103/PhysRevE.97.062412 for the following information: Fig. S1 contains a bootstrapped mutual information analysis showing that the VDE captures more information from the original features than previous linear methods; Fig. S2 shows that the VDE is able to produce similar models using internal protein coordinates, unlike previous linear methods.