Ergodic control for constrained diffusions: Characterization using HJB equations

SIAM Journal on Control and Optimization

Volumn 43, Issue 4, 2005, Pages 1467-1492

  Borkar, Vivek ^a   Budhiraja, Amarjit ^b  

^a TATA INSTITUTE OF FUNDAMENTAL RESEARCH   (India)

^b The University of North Carolina at Chapel Hill   (United States)

Author keywords

Constrained processes; Controlled reflected diffusions; Coupling; Domains with corners; Ergodic control; HJB equation; Oblique Neumann problem; Optimal Markov control; Pseudoatom; Viscosity solutions

Indexed keywords

CONSTRAINED PROCESSES; CONTROLLED REFLECTED DIFFUSIONS; COUPLING; DOMAINS WITH CORNERS; ERGODIC CONTROL; HJB EQUATIONS; OBLIQUE NEUMANN PROBLEM; OPTIMAL MARKOV CONTROL; PSEUDOATOM; VISCOSITY SOLUTIONS;

CONSTRAINT THEORY; DIFFUSION; FUNCTIONS; HAMILTONIANS; MARKOV PROCESSES; OPTIMAL CONTROL SYSTEMS; PROBLEM SOLVING; VISCOSITY;

CONTROL THEORY;

EID: 23244439620     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012902417619     Document Type: Article

References (31)

1. B. ATA, J. M. HARRISON, AND L. A. SHEPP, Drift Rate Control of a Brownian Processing System, preprint.
2. R. ATAR AND A. BUDHIRAJA, Stability properties of constrained jump-diffusion processes, Electron. J. Probab., 7 (2002), 31 (electronic).
3. R. ATAR, A. BUDHIRAJA, AND P. DUPUIS, On positive recurrence of constrained diffusion processes, Ann. Probab., 29 (2001), pp. 979-1000.
4. G. K. BASAK, V. S. BORKAR, AND M. K. GHOSH, Ergodic control of degenerate diffusions, Stochastic Anal. Appl., 1 (1997), pp. 1-17.
5. V. S. BORKAR, Dynamic programming for ergodic control with partial observations, Stochastic Process. Appl., 103 (2003), pp. 293-310.
6. V. S. BORKAR, A topology for Markov controls, Appl. Math. Optim., 20 (1989), pp. 55-62.
7. V. S. BORKAR, Optimal Control of Diffusion Processes, Longman Scientific and Technical, Harlow, UK, 1989.
8. V. S. BORKAR AND A. BUDHIRAJA, A further remark on dynamic programming for partially observed Markov processes, Stochastic Process. Appl., 112 (2004), pp. 79-93.
9. V. S. BORKAR AND M. K. GHOSH, Ergodic control of multidimensional diffusions. II. Adaptive control, Appl. Math. Optim., 21 (1990), pp. 191-220.
10. A. BUDHIRAJA, An ergodic control problem for constrained diffusion processes: Existence of optimal Markov control, SIAM J. Control Optim., 42 (2003), pp. 532-558.
11. A. BUDHIRAJA AND P. DUPUIS, Simple necessary and sufficient conditions for the stability of constrained processes, SIAM J. Appl. Math., 59 (1999), pp. 1686-1700.
12. A. BUDHIRAJA AND A. P. GHOSH, A large deviations approach to asymptotically optimal control of crisscross network in heavy traffic, Ann. Appl. Probab., to appear.
13. M. CHEN, C. PANDIT, AND S. P. MEYN, In search of sensitivity in network optimization, Queueing Systems Theory Appl., 44 (2003), pp. 313-363.
14. R. M. Cox, Stationary and discounted control of diffusion processes, Ph.D. thesis, Columbia University, New York, NY, 1984.
15. M. G. CRANDALL AND P. L. LIONS, Viscosity solutions of Hamilton-Jacobi equations, Trans. Amer. Math. Soc., 277 (1983), pp. 1-42.
16. P. DUPUIS AND H. ISHII, On Lipschitz continuity of the solution mapping to the Skorokhod problem, with applications, Stochastics Stochastics Rep., 35 (1991), pp. 31-62.
17. P. DUPUIS AND H. ISHII, On oblique derivative problems for fully onlinear second-order elliptic PDE's on domains with corners, Hokkaido Math. J., 20 (1991), pp. 135-164.
18. P. DUPUIS AND K. RAMANAN, Convex duality and the Skorokhod Problem. I, II, Probab. Theory Related Fields, 115 (1999), pp. 153-195, 197-236.
19. P. W. GLYNN AND S. P. MEYN, A Liapounov bound for solutions of the Poisson equation, Ann. Probab., 24 (1996), pp. 916-931.
20. J. M. HARRISON AND J. A. VAN MIEGHAM, Dynamic control of Brownian networks: State space collapse and equivalent workload formulation, Ann. Appl. Probab., 7 (1997), pp. 747-771.
21. J. M. HARRISON AND L. WEIN, Scheduling networks of queues: Heavy trafic analysis of a simple open network, Queueing Systems Theory Appl., 5 (1989), pp. 265-280.
22. F. P. KELLY AND C. N. LAWS, Dynamic routing in open queuing networks: Brownian models, cut constraints and resource pooling, Queueing Systems Theory Appl., 13 (1993), pp. 47-86.
23. H. J. KUSHNER, Heavy Traffic Analysis of Controlled Queueing and Communication Networks, Springer-Verlag, New York, May 2001.
24. H. J. KUSHNER AND L. F. MARTINS, Limit theorems for pathwise average cost per unit time problems for controlled queues in heavy traffic, Stochastics Stochastics Rep., 42 (1993), pp. 25-51.
25. P.-L. LIONS, Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations. II. Viscosity solutions and uniqueness, Comm. Partial Differential Equations, 8 (1983), pp. 1229-1276.
26. L. F. MARTINS AND H. J. KUSHNER, Routing and singular control for queuing networks in heavy traffic, SIAM J. Control Optim., 28 (1990), pp. 1209-1233.
27. S. P. MEYN, The policy iteration algorithm for average reward Markov decision processes with general state space, IEEE Trans. Automat. Control, 42 (1997), pp. 1663-1680.
28. S. P. MEYN, Stability, performance evaluation, and optimization, Handbook of Markov Decision Processes, Internat. Ser. Oper. Res. Management Sci. 40, Kluwer Academic Publishers, Boston, 2002, pp. 305-346.
29. S. MEYN AND R. TWEEDIE, Markov Chains and Stochastic Stability, Springer-Verlag, London, 1993.
30. S. MEYN AND R. TWEEDIE, Stability of Markovian processes, III. Foster-Lyapunov criteria for continuous-time processes, Adv. in Appl. Probab., 25 (1993), pp. 518-548.
31. S. MEYN AND R. TWEEDIE, Generalized resolvents and Harris recurrence of Markov processes, in Doeblin and Modern Probability (Blaubeuren, 1991), Contemp. Math. 149, AMS, Providence, RI, 1991, pp. 227-250.