Convergence of the relative value iteration for the ergodic control problem of nondegenerate diffusions under near-monotone costs

SIAM Journal on Control and Optimization

Volumn 52, Issue 1, 2014, Pages 1-31

  Arapostathis, Ari ^a   Borkar, Vivek S ^b   Kumar, K Suresh ^b  

^a The University of Texas at Austin   (United States)

^b INDIAN INSTITUTE OF TECHNOLOGY BOMBAY   (India)

Author keywords

Controlled diffusions; Ergodic control; Hamilton jacobi bellman equation; Parabolic cauchy problem; Relative value iteration

Indexed keywords

CAUCHY PROBLEMS; CONTROLLED DIFFUSION; ERGODIC CONTROL; HAMILTON JACOBI BELLMAN EQUATION; RELATIVE VALUE ITERATIONS;

CONTROL THEORY; DIFFUSION; DYNAMIC PROGRAMMING; INITIAL VALUE PROBLEMS; PARTIAL DIFFERENTIAL EQUATIONS;

ITERATIVE METHODS;

EID: 84897899776     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/130912918     Document Type: Article

References (23)

1. J. ABOUNADI, D. P. BERTSEKAS, AND V. S. BORKAR, Learning algorithms for Markov decision processes with average cost, SIAM J. Control Optim., 40(2001), pp. 681-698.
2. A. ARAPOSTATHIS, On the policy iteration algorithm for nondegenerate controlled diffusions under the ergodic criterion, in Optimization, Control, and Applications of Stochastic Systems, D. Hernández-Hernández and J. A. Minjárez-Sosa, eds., Systems Control Found. Appl., Birkhäuser Boston, Boston, MA, 2012, pp. 1-12.
3. A. ARAPOSTATHIS AND V. S. BORKAR, A relative value iteration algorithm for nondegenerate controlled diffusions, SIAM J. Control Optim., 50(2012), pp. 1886-1902.
4. A. ARAPOSTATHIS, V. S. BORKAR, AND M. K. GHOSH, Ergodic Control of Diffusion Processes, Encyclopedia Math. Appl. 143, Cambridge University Press, Cambridge, UK, 2011.
5. A. ARAPOSTATHIS, V. S. BORKAR, AND K. S. KUMAR, Relative value iteration for stochastic differential games, in Advances in Dynamical Games: Theory, Applications and Numerical Methods, V. Krivan and G. Zaccour, eds., Ann. Internat. Soc. Dynam. Games 13, Birkhäuser Boston, Boston, MA, 2013.
6. V. I. BOGACHEV, N. V. KRYLOV, AND M. RÖCKNER, On regularity of transition probabilities and invariant measures of singular diffusions under minimal conditions, Comm. Partial Differential Equations, 26(2001), pp. 2037-2080.
7. R.-R. CHEN AND S. MEYN, Value iteration and optimization of multiclass queueing networks, Queueing Systems Theory Appl., 32(1999), pp. 65-97.
8. S. EIDELMAN, S. KAMIN, AND F. PORPER, Uniqueness of solutions of the Cauchy problem for parabolic equations degenerating at infinity, Asymptot. Anal., 22(2000), pp. 349-358.
9. M. GRUBER, Harnack inequalities for solutions of general second order parabolic equations and estimates of their Hölder constants, Math. Z., 185(1984), pp. 23-43.
10. I. GYÖNGY AND N. KRYLOV, Existence of strong solutions for Ito's stochastic equations via approximations, Probab. Theory Related Fields, 105(1996), pp. 143-158.
11. R. Z. HASMINSKIǏ, Ergodic properties of recurrent diffusion processes and stabilization of the solution of the Cauchy problem for parabolic equations, Theory Probab. Appl., 5(1960), pp. 179-196.
12. N. ICHIHARA, Large time asymptotic problems for optimal stochastic control with superlinear cost, Stochastic Process. Appl., 122(2012), pp. 1248-1275.
13. N. ICHIHARA AND S.-J. SHEU, Large time behavior of solutions of Hamilton-Jacobi-Bellman equations with quadratic nonlinearity in gradients, SIAM J. Math. Anal., 45(2013), pp. 279-306.
14. N. V. KRYLOV, Controlled Diffusion Processes, Appl. Math. 14, Springer-Verlag, New York, 1980.
15. N. V. KRYLOV, Lectures on Elliptic and Parabolic Equations in Sobolev Spaces, Grad. Stud. Math. 96, American Mathematical Society, Providence, RI, 2008.
16. O. A. LADYŽENSKAJA, V. A. SOLONNIKOV, AND N. N. URAL'CEVA, Linear and quasi-linear equations of parabolic type, Transl. Math. Monogr. 23, American Mathematical Society, Providence, RI, 1967.
17. E. LÖCHERBACH, D. LOUKIANOVA, AND O. LOUKIANOV, Polynomial bounds in the ergodic theorem for one-dimensional diffusions and integrability of hitting times, Ann. Inst. H. Poincare Probab. Statist., 47(2011), pp. 425-449.
18. S. MEYN AND R. L. TWEEDIE, Markov Chains and Stochastic Stability, 2nd ed., Cambridge University Press, Cambridge, UK, 2009.
19. S. P. MEYN, The policy iteration algorithm for average reward Markov decision processes with general state space, IEEE Trans. Automat. Control, 42(1997), pp. 1663-1680.
20. S. P. MEYN AND R. L. TWEEDIE, Stability of Markovian processes. III. Foster-Lyapunov criteria for continuous-time processes, Adv. in Appl. Probab., 25(1993), pp. 518-548.
21. H. NAGAI, Downside risk minimization via a large deviations approach, Ann. Appl. Probab., 22(2012), pp. 608-669.
22. 1: Existence, uniqueness and associated Markov processes, Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4), 28(1999), pp. 99-140.
23. D. J. WHITE, Dynamic programming, Markov chains, and the method of successive approximations, J. Math. Anal. Appl., 6(1963), pp. 373-376.