Markov chain approximations for deterministic control problems with affine dynamics and quadratic cost in the control

SIAM Journal on Numerical Analysis

Volumn 36, Issue 3, 1999, Pages 667-695

  Boué, Michelle ^a   Dupuis, Paul ^b  

^a INSTITUTO TECNOLÓGICO AUTÓNOMO DE MÉXICO   (Mexico)

^b BROWN UNIVERSITY   (United States)

Author keywords

Calculus of variations; Deterministic optimal control; Finite difference approximation; Markov chain approximation; Weak convergence

Indexed keywords

EID: 0001678817     PISSN: 00361429     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0036142997323521     Document Type: Article

References (18)

1. P. BILLINGSLEY, Convergence of Probability Measures, John Wiley, New York, 1968.
2. P. DUPUIS, A numerical method for a calculus of variations problem with discontinuous integrand, in Proceedings of the Rutgers Conference on Applied Stochastic Analysis, I. Karatzas and D. Ocone, eds., Springer-Verlag, Berlin, 1992, pp. 90-107.
3. P. DUPUIS AND R. S. ELLIS, A Weak Convergence Approach to the Theory of Large Deviations, John Wiley, New York, 1997.
4. P. DUPUIS AND M. R. JAMES, Rates of convergence for approximation schemes in optimal control, SIAM J. Control Optim., 36 (1998), pp. 719-741.
5. P. DUPUIS AND D. JARVIS, Computational Methods for Certain Large Deviation Functionals of Queueing Networks, Technical Report, Lefschetz Center for Dynamical Systems, Brown University, Providence, RI, 1997.
6. P. DUPUIS AND H. J. KUSHNER, Stochastic systems with small noise, analysis and simulation: A phase locked loop example, SIAM J. Appl. Math., 47 (1987), pp. 643-661.
7. P. DUPUIS AND J. OLIENSIS, An optimal control formulation and related numerical methods for a problem in shape reconstruction, Ann. Appl. Probab., 4 (1994), pp. 287-346.
8. S. N. ETHIER AND T. G. KURTZ, Markov Processes: Characterization and Convergence, John Wiley, New York, 1986.
9. W. H. FLEMING AND H. M. SONER, Controlled Markov Processes and Viscosity Solutions, Springer-Verlag, New York, 1992.
10. M. I. FREIDLIN AND A. D. WENTZELL, Random Perturbations of Dynamical Systems, Springer-Verlag, New York, 1984.
11. M. R. JAMES AND I. R. PETERSEN, Nonlinear State Estimation for Uncertain Systems with an Integral Constraint, Technical Report, The Australian National University, Canberra, Australia, 1996.
12. I. KARATZAS AND S. E. SHREVE, Brownian Motion and Stochastic Calculus, 2nd ed., Springer-Verlag, New York, 1991.
13. H. J. KUSHNER, Probability Methods for Approximations in Stochastic Control and for Elliptic Equations, Academic Press, New York, 1977.
14. H. J. KUSHNER, Control of trunk line systems in heavy traffic, SIAM J. Control Optim., 33 (1995), pp. 765-803.
15. H. J. KUSHNER AND P. DUPUIS, Numerical Methods for Stochastic Control Problems in Continuous Time, Springer-Verlag, New York, 1992.
16. ∞ filtering for nonlinear systems, Systems Control Lett., 33 (1998), pp. 315-325.
17. M. L. PUTERMAN, Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley, New York, 1994.
18. P. STOLL, C.-W. SHU, AND B. B. KIMIA, Shock Capturing Numerical Methods for Viscosity Solutions of Certain PDEs in Computer Vision: The Godunov, Osher-Sethian and ENO Schemes, Technical Report LEMS-132, Brown University, Providence, RI, 1996.