Model reduction algorithms for optimal control and importance sampling of diffusions

Nonlinearity

Volumn 29, Issue 8, 2016, Pages 2298-2326

  Hartmann, Carsten ^a,b   Schütte, Christof ^a,c   Zhang, Wei ^a  

^a FREIE UNIVERSITÄT BERLIN   (Germany)

^b BRANDENBURG UNIVERSITY OF TECHNOLOGY   (Germany)

^c ZUSE INSTITUTE BERLIN   (Germany)

Author keywords

importance sampling; model reduction; Stochastic optimal control; transition path

Indexed keywords

EID: 84980008589     PISSN: 09517715     EISSN: 13616544     Source Type: Journal    
DOI: 10.1088/0951-7715/29/8/2298     Document Type: Article

References (54)

1. Leimkuhler B and Matthews Ch 2015 Molecular Dynamics: with Deterministic and Stochastic Numerical Methods (Berlin: Springer)
2. Di Nunno J and Øksendal B 2011 Advanced Mathematical Methods for Finance (Berlin: Springer)
3. Majda A J, Timofeyev I and Vanden-Eijnden E 2001 A mathematical framework for stochastic climate models Commun. Pure Appl. Math. 54 891-974
4. Chorin A J, Hald O H and Kupferman R 2000 Optimal prediction and the Mori-Zwanzig representation of irreversible processes Proc. Natl Acad. Sci. USA 97 2968-73
5. Chorin A J, Hald O H and Kupferman R 2002 Optimal prediction with memory Physica D 166 239-57
6. Legoll F and Lelièvre T 2010 Effective dynamics using conditional expectations Nonlinearity 23 2131
7. Hartmann C 2011 Balanced model reduction of partially observed Langevin equations: an averaging principle Math. Comput. Modelling Dyn. Syst. 17 463-90
8. Turkington B 2013 An optimization principle for deriving nonequilibrium statistical models of Hamiltonian dynamics J. Stat. Phys. 152 569-97
9. Majda A J 2012 Challenges in climate science and contemporary applied mathematics Comm. Pure Appl. Math. 65 920-48
10. Givon D, Kupferman R and Stuart A M 2004 Extracting macroscopic dynamics: model problems and algorithms Nonlinearity 17 R55-127
11. E W, Ren W and Vanden-Eijnden E 2002 String method for the study of rare events Phys. Rev. B 66 052301
12. E W, Ren W and Vanden-Eijnden E 2005 Finite temperature string method for the study of rare events J. Phys. Chem. B 109 6688-93
13. Metzner P, Schütte C and Vanden-Eijnden E 2009 Transition path theory for Markov jump processes Multiscale Model Simul. 7 1192-219
14. Zhao R, Shen J and Skeel R D 2010 Maximum flux transition paths of conformational change J. Chem. Theory Comput. 6 2411-23
15. Vanden-Eijnden E 2006 Transition path theory Computer Simulations in Condensed Matter Systems: from Materials to Chemical Biology volume (Lecture Notes in Physics vol 703) ed M Ferrario et al (Berlin: Springer) pp 453-93
16. E W and Vanden-Eijnden E 2010 Transition-path theory and path-finding algorithms for the study of rare events Annu. Rev. Phys. Chem. 61 391-420
17. Zhang W, Wang H, Hartmann C, Weber M and Schütte Ch 2014 Applications of the cross-entropy method to importance sampling and optimal control of diffusions SIAM J. Sci. Comput. 36 A2654-72
18. de Boer P, Kroese D P, Mannor S and Rubinstein R Y 2005 A tutorial on the cross-entropy method Ann. Oper. Res. 134 19-67
19. Fleming W H and Soner H M 2006 Controlled Markov Processes and Viscosity Solutions (Berlin: Springer)
20. Hartmann C, Latorre J C, Zhang W and Pavliotis G A 2014 Optimal control of multiscale systems using reduced-order models J. Comput. Dyn. 1 279-306
21. Schütte Ch, Winkelmann S and Hartmann C 2012 Optimal control of molecular dynamics using Markov state models Math. Program. Ser. B 134 259-82
22. Dupuis P and Wang H 2004 Importance sampling, large deviations, and differential games Stoch. Stoch. Rep. 76 481-508
23. Hartmann C and Schütte Ch 2012 Efficient rare event simulation by optimal nonequilibrium forcing J. Stat. Mech. P11004
24. Spiliopoulos K 2013 Large deviations and importance sampling for systems of slow-fast motion Appl. Math. Optim. 67 123-61
25. Spiliopoulos K, Dupuis P and Zhou X 2015 Escaping from an attractor: Importance sampling and rest points, part I Ann. Appl. Probab. 25 2909-58
26. Vanden-Eijnden E and Weare J 2012 Rare event simulation of small noise diffusions Commun. Pure Appl. Math. 65 1770-803
27. Øksendal B 2000 Stochastic Differential Equations: an Introduction with Applications 5th edn (Berlin: Springer)
28. Freidlin M 1985 Functional Integration and Partial Differential Equations (Princeton, NJ: Princeton University Press)
29. Dupuis P, Spiliopoulos K and Wang H 2012 Importance sampling for multiscale diffusions Multiscale Modelling Simul. 10 1-27
30. Boué M and Dupuis P 1998 A variational representation for certain functionals of Brownian motion Ann. Probab. 26 1641-59
31. Dai Pra P, Meneghini L and Runggaldier W J 1996 Connections between stochastic control and dynamic games Math. Control Signals Syst. 9 303-26
32. Lie H C, Schütte Ch and Hartmann C 2015 Martingale-based gradient descent algorithm for estimating free energy values of diffusions http://publications.mi.fu-berlin.de/1476/
33. Banisch R and Hartmann C 2016 A sparse Markov chain approximation of LQ-type stochastic control problems Math. Control Relat. Fields (at press)
34. Rubinstein R Y and Kroese D P 2004 The Cross-Entropy Method: a Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning (Information Science and Statistics) 1st edn (Berlin: Springer)
35. Ciccotti G, Lelièvre T and Vanden-Eijnden E 2008 Projection of diffusions on submanifolds: Application to mean force computation Commun. Pure Appl. Math. 61 371-408
36. Hartmann C 2007 Model reduction in classical molecular dynamics PhD Dissertation Fachbereich Mathematik und Informatik, Freie Universität Berlin
37. Freidlin M and Wentzell A D 1998 Random Perturbations of Dynamical Systems (Berlin: Springer)
38. E W, Ren W and Vanden-Eijnden E 2007 Simplified and improved string method for computing the minimum energy paths in barrier-crossing events J. Chem. Phys. 126 164103
39. Vanden-Eijnden E and Venturoli M 2009 Revisiting the finite temperature string method for the calculation of reaction tubes and free energies J. Chem. Phys. 130 194103
40. Gil A, Segura J and Temme N M 2007 Numerical Methods for Special Functions 1st edn (Philadelphia: SIAM)
41. Huo S and Straub J E 1997 The MaxFlux algorithm for calculating variationally optimized reaction paths for conformational transitions in many body systems at finite temperature J. Chem. Phys. 107 5000-6
42. E W, Ren W and Vanden-Eijnden E 2004 Minimal action method for the study of rare events Commun. Pure Appl. Math. 57 0637-56
43. Heymann M and Vanden-Eijnden E 2008 The geometric minimum action method: a least action principle on the space of curves Commun. Pure Appl. Math. 61 1052-117
44. Müller K 1980 Reaction paths on multidimensional energy hypersurfaces Angew. Chem. Int. Ed. Engl. 19 1-13
45. Müller K and Brown L 1979 Location of saddle points and minimum energy paths by a constrained simplex optimization procedure Theor. Chem. Acc. 53 75-93 Theor. Chim. Acta
46. Latorre J C, Metzner Ph, Hartmann C and Schütte C 2011 A structure-preserving numerical discretization of reversible diffusions Commun. Math. Sci. 9 1051-72
47. Balay S et al 2015 PETSc www.mcs.anl.gov/petsc
48. van der Vorst H A 1992 Bi-CGSTAB: a fast and smoothly converging variant of bi-cg for the solution of nonsymmetric linear systems SIAM J. Sci. Stat. Comput. 13 631-44
49. Hartmann C, Schütte Ch, Weber M and Zhang W 2015 Importance sampling in path space for diffusion processes with slow-fast variables Probab. Theory. Rel. Fields submitted (arXiv: 1502.07899)
50. Bolhuis P G, Chandler D, Dellago C and Geissler P L 2002 Transition path sampling: throwing ropes over rough mountain passes, in the dark Annu. Rev. Phys. Chem. 53 291-318
51. Pinskii F, Stuart A M and Theil F 2012 Gamma limits for transition paths of maximal probability J. Stat. Phys. 146 955-74
52. Pinski F J and Stuart A M 2010 Transition paths in molecules at finite temperature J. Chem. Phys. 132 184104
53. Badowski T 2016 Adaptive importance sampling via minimization of estimators of cross-entropy, mean square, and inefficiency constant (arXiv:1505.06448)
54. Spiliopoulos K 2015 Non-asymptotic performance analysis of importance sampling schemes for small noise diffusions J. Appl. Probab. 52 797-810