A relative value iteration algorithm for nondegenerate controlled diffusions

SIAM Journal on Control and Optimization

Volumn 50, Issue 4, 2012, Pages 1886-1902

  Arapostathis, Ari ^a   Borkar, Vivek S ^b  

^a The University of Texas at Austin   (United States)

^b INDIAN INSTITUTE OF TECHNOLOGY BOMBAY   (India)

Author keywords

Controlled diffusions; Ergodic control; Hamilton Jacobi Bellman equation; Monotone dynamical systems; Relative value iteration; Reverse martingales

Indexed keywords

CONTROLLED DIFFUSION; ERGODIC CONTROL; HAMILTON JACOBI BELLMAN EQUATION; MONOTONE DYNAMICAL SYSTEMS; RELATIVE VALUE; REVERSE MARTINGALES;

ALGORITHMS; DYNAMIC PROGRAMMING; DYNAMICAL SYSTEMS;

NONLINEAR EQUATIONS;

EID: 84866419009     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/110850529     Document Type: Article

References (20)

1. J. Abounadi, D. Bertsekas, and V. S. Borkar, Learning algorithms for Markov decision processes with average cost, SIAM J. Control Optim., 40 (2001), pp. 681-698.
2. A. Arapostathis, V. S. Borkar, and M. K. Ghosh, Ergodic Control of Diffusion Processes, Encyclopedia Math. Appl. 143, Cambridge University Press, Cambridge, UK, 2011.
3. Y. Aviv and A. Federgruen, The value iteration method for countable state Markov decision processes, Oper. Res. Lett., 24 (1999), pp. 223-234.
4. D. P. Bertsekas, A new value iteration method for the average cost dynamic programming problem, SIAM J. Control Optim., 36 (1998), pp. 742-759.
5. V. I. Bogachev, N. V. Krylov, and M. Röckner, On regularity of transition probabilities and invariant measures of singular diffusions under minimal conditions, Comm. Partial Differential Equations, 26 (2001), pp. 2037-2080.
6. D. Down, S. P. Meyn, and R. L. Tweedie, Exponential and uniform ergodicity of Markov processes, Ann. Probab., 23 (1995), pp. 1671-1691.
7. E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Grundlehren Math. Wiss. 235, Springer-Verlag, New York, 1979.
8. G. Fort and G. O. Roberts, Subgeometric ergodicity of strong Markov processes, Ann. Appl. Probab., 15 (2005), pp. 1565-1589.
9. I. Gyöngy and N. Krylov, Existence of strong solutions for Itô's stochastic equations via approximations, Probab. Theory Related Fields, 105 (1996), pp. 143-158.
10. R. Z. Hasminskiǐ, Ergodic properties of recurrent diffusion processes and stabilization of the solution of the Cauchy problem for parabolic equations, Theory Probab. Appl., 5 (1960), pp. 179-196.
11. O. Hernández-Lerma and J. B. Lasserre, Markov Chains and Invariant Probabilities, Progr. Math. 211, Birkhäuser Verlag, Basel, 2003.
12. N. V. Krylov, Controlled Diffusion Processes, Appl. Math. (N.Y.) 14, Springer-Verlag, New York, 1980.
13. N. V. Krylov, Lectures on Elliptic and Parabolic Equations in Sobolev Spaces, Grad. Stud. Math. 96, American Mathematical Society, Providence, RI, 2008.
14. O. A. Ladyženskaja, V. A. Solonnikov, and N. N. Ural'ceva, Linear and Quasilinear Equations of Parabolic Type, Transl. Math. Monogr. 23, American Mathematical Society, Providence, RI, 1967.
15. S. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability, 2nd ed., Cambridge University Press, Cambridge, UK, 2009.
16. M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley & Sons, New York, 1994.
17. O. Shlakhter, C.-G. Lee, D. Khmelev, and N. Jaber, Acceleration operators in the value iteration algorithms for Markov decision processes, Oper. Res., 58 (2010), pp. 193-202.
18. H. L. Smith, Monotone Dynamical Systems, Math. Surveys Monogr. 41, American Mathematical Society, Providence, RI, 1995.
19. W. Stannat, (Nonsymmetric) Dirichlet operators on L1: Existence, uniqueness and associated Markov processes, Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4), 28 (1999), pp. 99-140.
20. D. J. White, Dynamic programming, Markov chains, and the method of successive approximations, J. Math. Anal. Appl., 6 (1963), pp. 373-376.