An ergodic control problem for constrained diffusion processes: Existence of optimal Markov control

SIAM Journal on Control and Optimization

Volumn 42, Issue 2, 2003, Pages 532-558

  Budhiraja, Amarjit ^a  

^a UNIVERSITY OF NORTH CAROLINA   (United States)

Author keywords

Constrained processes; Control of queuing networks; Controlled martingale problem; Controlled reflected diffusions; Echeverria's criterion; Ergodic control; Optimal Markov control; Patchwork martingale problem

Indexed keywords

CONSTRAINED PROCESSES; CONTROL OF QUEUING NETWORKS; CONTROLLED MARTINGALE PROBLEM; CONTROLLED REFLECTED DIFFUSIONS; ECHEVERRIA'S CRITERION; ERGODIC CONTROL; OPTIMAL MARKOV CONTROL; PATCHWORK MARTINGALE PROBLEM;

BOUNDARY VALUE PROBLEMS; CONFORMAL MAPPING; CONSTRAINT THEORY; MARKOV PROCESSES; MATHEMATICAL MODELS; PROBLEM SOLVING; QUEUEING NETWORKS; THEOREM PROVING; TRAFFIC SURVEYS; VECTORS;

OPTIMAL CONTROL SYSTEMS;

EID: 1942541501     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012901379073     Document Type: Article

References (42)

1. M. K. GHOSH, A. ARAPOSTATHIS, AND S. I. MARCUS, Ergodic control of switching diffusions, SIAM J. Control Optim., 35 (1997), pp. 1952-1988.
2. R. ATAR, A. BUDHIRAJA, AND P. DUPUIS, On positive recurrence of constrained diffusion processes, Ann. Probab., 29 (2001), pp. 979-1000.
3. G. K. BASAK, V. S. BORKAR, AND M. K. GHOSH, Ergodic control of degenerate diffusions, Stochastic Anal. Appl., 1 (1997), pp. 1-17.
4. A. BENSOUSSAN, Perturbation Methods in Optimal Control, John Wiley, New York, 1988.
5. A. G. BHATT AND V. S. BORKAR, Occupation measures for controlled Markov processes: Characterization and optimality, Ann. Probab., 24 (1996), pp. 1531-1562.
6. V. BORKAR, Optimal Control of Diffusion Processes, Longman Scientific and Technical, Harlow, UK, 1989.
7. V. S. BORKAR AND M. K. GHOSH, Ergodic control of multidimensional diffusions I: The existence results, SIAM J. Control Optim., 26 (1988), pp. 112-126.
8. V. S. BORKAR AND M. K. GHOSH, Ergodic control of multidimensional diffusions. II. Adaptive control, Appl. Math. Optim., 21 (1990), pp. 191-220.
9. A. BUDHIRAJA AND P. DUPUIS, Simple necessary and sufficient conditions for the stability of constrained processes, SIAM J. Appl. Math., 59 (1999), pp. 1686-1700.
10. R. M. Cox, Stationary and Discounted Control of Diffusion Processes, Ph.D. thesis, Columbia University, New York, 1984.
11. M. G. CRANDALL AND P. L. LIONS, Viscosity solutions of Hamilton-Jacobi equations, Trans. Amer. Math. Soc., 277 (1983), pp. 1-42.
12. J. G. DAI AND R. J. WILLIAMS, Existence and uniqueness of semimartingale reflecting Brownian motions in convex polyhedrons, Theory Probab. Appl., 40 (1995), pp. 1-40.
13. P. DUPUIS AND H. ISHII, On Lipschitz continuity of the solution mapping to the Skorokhod problem, with applications, Stochastics, 35 (1991), pp. 31-62.
14. P. DUPUIS AND H. ISHII, On oblique derivative problems for fully nonlinear second-order elliptic PDEs on domains with corners, Hokkaido Math. J., 20 (1991), pp. 135-164.
15. P. DUPUIS AND K. RAMANAN, Convex duality and the Skorokhod Problem. I, II, Probab. Theory Related Fields, 2 (1999), pp. 153-195, 197-236.
16. P. DUPUIS AND K. RAMANAN, A multiclass feedback queueing network with a regular Skorokhod problem, Queuing Syst. Theory Appl., 36 (2000), pp. 327-349.
17. P. E. ECHEVERRIA, A criterion for invariant measures of Markov processes, Z. Warsch. Verw. Gebiete., 61 (1982), pp. 1-16.
18. J. M. HARRISON AND M. I. REIMAN, Reflected Brownian motion on an orthant, Ann. Probab., 9 (1981), pp. 302-308.
19. J. M. HARRISON AND J. A. VAN MIEGHAM, Dynamic control of Brownian networks: State space collapse and equivalent workload formulation, Ann. Appl. Probab., 7 (1997), pp. 747-771.
20. J. M. HARRISON AND L. WEIN, Scheduling networks of queues: Heavy traffic analysis of a simple open network, Queuing Systems Theory Appl., 5 (1989), pp. 265-280.
21. J. M. HARRISON AND R. J. WILLIAMS, Brownian models of open queueing networks with homogeneous customer populations, Stoohastics, 22 (1987), pp. 77-115.
22. U. G. HAUSSMANN AND J. P. LEPELTIER, On the existence of optimal controls, SIAM J. Control Optim., 28 (1990), pp. 851-902.
23. F. P. KELLY AND C. N. LAWS, Dynamic routing in open queuing networks: Brownian models, cut constraints and resource pooling, Queuing Systems Theory Appl., 13 (1993), pp. 47-86.
24. R. Z. KHASMINSKII, Ergodic properties of recurrent diffusion processes and stabilization of the solution to the Cauchy problem for parabolic equations, Theory Probab. Appl., 5 (1960), pp. 179-196.
25. R. Z. KHASMINSKII, Stochastic Stability of Differential Equations, Sijthoff & Noordhoff, Groningen, The Netherlands, 1980.
26. T. G. KURTZ, Martingale problems for constrained Markov processes, in Advances in Stochastic Calculus, J. S. Baras and V. Mirelli, eds., Springer-Verlag, New York, 1990, pp. 151-168.
27. T. G. KURTZ, A control formulation for constrained Markov processes, in Mathematics of Random Media, Lectures in Appl. Math., 27, AMS, Providence, RI, 1991, pp. 139-150.
28. T. G. KURTZ AND R. H. STOCKBRIDGE, Existence of Markov controls and characterization of optimal Markov controls, SIAM J. Control Optim., 36 (1998), pp. 609-653.
29. T. G. KURTZ AND R. H. STOCKBRIDGE, Stationary solutions and forward equations for controlled and singular martingale problems, Electron. J. Probab., 6 (2001), 52 pp.
30. H. J. KUSHNER, Optimality conditions for the average cost per unit time problem with a diffusion model, SIAM J. Control Optim., 16 (1978), pp. 330-346.
31. H. J. KUSHNER, Heavy Traffic Analysis of Controlled Queueing and Communication Networks, Springer-Verlag, New York, 2001.
32. F. MARTINS AND H. J. KUSHNER, Routing and singular control for queuing networks in heavy traffic, SIAM J. Control Optim, 28 (1990), pp. 1209-1233.
33. P.-L. LIONS, Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations. II, Viscosity solutions and uniqueness, Comm. Partial Differential Equations, 8 (1983), pp. 1229-1276.
34. J. L. MENALDI AND M. ROBIN, Ergodic control of reflected diffusions with jumps, Appl. Math. Optim., 35 (1997), pp. 117-138.
35. V. NGUYEN, Processing networks with parallel and sequential tasks: Heavy traffic analysis and Brownian limits, Ann. Appl. Probab., 3 (1993), pp. 28-55.
36. W. PETERSON, A heavy traffic limit theorem for networks of queues with multiple customer types, Math. Oper. Res., 16 (1991), pp. 90-118.
37. M. REIMAN AND R. J. WILLIAMS, A boundary property for semimartingale reflecting Brownian motions, Probab. Theory Related Fields, 77 (1988), pp. 87-97.
38. D. REVUZ AND M. YOR, Continuous Martingales and Brownian Motion, Springer-Verlag, Berlin, 1991.
39. M. ROBIN, Long-term average cost control problems for continuous time markov processes - A survey, Acta Appl. Math., 1 (1983), pp. 281-300.
40. R. H. STOCKBRIDGE, Time Average Control of Martingale Problems, Ph.D. thesis, University of Wisconsin, Madison, WI, 1987.
41. D. STROOCK AND S. VARADHAN, Multidimensional Diffusion Processes, Springer-Verlag, Berlin-New York, 1979.
42. A. WEISS, Invariant Measures of Diffusions in Bounded Domains, Ph.D. thesis, New York University, New York, 1981.