Continuity of optimal values and solutions for control of Markov chains with constraints

SIAM Journal on Control and Optimization

Volumn 38, Issue 4, 2000, Pages 1204-1222

  Tidball, Mabel M ^a   Lombardi, Ariel ^b   Pourtallier, Odile ^c   Altman, Eitan ^c  

^a CEMAGREF   (France)

^b UNIVERSIDAD DE BUENOS AIRES   (Argentina)

^c INRIA   (France)

Author keywords

[No Author keywords available]

Indexed keywords

CONTROL SYSTEM ANALYSIS; CONVERGENCE OF NUMERICAL METHODS; DECISION THEORY; MARKOV PROCESSES; OPTIMIZATION; ROBUSTNESS (CONTROL SYSTEMS); SENSITIVITY ANALYSIS;

CONSTRAINED MARKOV DECISION PROCESSES (CMDPS); ERGODIC STRUCTURES;

OPTIMAL CONTROL SYSTEMS;

EID: 0033683162     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012997280294     Document Type: Article

References (46)

1. E. ALTMAN, Asymptotic properties of constrained Markov decision processes, Z. Oper. Res., 37 (1993), pp. 151-170.
2. E. ALTMAN, Denumerable constrained Markov decision problems and finite approximations, Math. Oper. Res., 19 (1994), pp. 169-191.
3. E. ALTMAN, Constrained Markov Decision Processes, in Stochastic Modeling, Chapman and Hall/CRC, Boca Raton, FL, 1999.
4. E. ALTMAN AND V. A. GAITSGORY, Stability and singular perturbations in constrained Markov decision problems, IEEE Trans. Automat. Control, 38 (1993), pp. 971-975.
5. E. ALTMAN AND A. SHWARTZ, Optimal priority assignment: A time sharing approach, IEEE Trans. Automat. Control, AC-34 (1989), pp. 1089-1102.
6. E. ALTMAN AND A. SHWARTZ, Adaptive control of constrained Markov chains, IEEE Trans. Automat. Control, 36 (1991), pp. 454-462.
7. E. ALTMAN AND A. SHWARTZ, Markov decision problems and state-action frequencies, SIAM J. Control Optim., 29 (1991), pp. 786-809.
8. E. ALTMAN AND A. SHWARTZ, Sensitivity of constrained Markov decision problems, Ann. Oper. Res., 32 (1991), pp. 1-22.
9. E. ALTMAN AND A. SHWARTZ, Time-sharing policies for controlled Markov chains, Oper. Res., 41 (1993), pp. 1116-1124.
10. E. ALTMAN AND F. SPIEKSMA, The linear program approach in Markov decision problems revisited, Z. Oper. Res., 42 (1995), pp. 169-188.
11. F. J. BEUTLER AND K. W. ROSS, Optimal policies for controlled Markov chains with a constraint, J. Math. Anal. Appl., 112 (1985), pp. 236-252.
12. F. J. BEUTLER AND K. W. ROSS, Time-average optimal constrained semi-Markov decision processes, Adv. Appl. Probab., 18 (1986), pp. 341-359.
13. J. R. BIRGE AND R. J. WETS, Designing approximatin schemes for stochastic optimization problems, Math. Programming Stud., 27 (1986), pp. 54-102.
14. V. S. BORKAR, A convex analytic approach to Markov decision processes, Probab. Theory Related Fields, 78 (1988), pp. 583-602.
15. V. S. BORKAR, Ergodic control of Markov chains with constraints - the general case, SIAM J. Control Optim., 32 (1994), pp. 176-186.
16. G. DAL MASO, An Introduction to Γ-Convergence, Progr. Nonlinear Differential Equations Appl. 8, Birkhäuser, Basel, Boston, 1993.
17. G. B. DANTZIG, J. FOLKMAN, AND N. SHAPIRO, On the continuity of the minimum set of a continuous function, J. Math. Anal. Appl., 17 (1967), pp. 519-548.
18. C. DERMAN AND R. E. STRAUCH, A note on memoryless rules for controlling sequential control processes, Ann. Math. Statist., 37 (1966), pp. 276-278.
19. E. DYNKIN AND A. YUSHKEVICH, Controlled Markov Processes, Springer-Verlag, Berlin, 1979.
20. E. A. FEINBERG, Sufficient classes of strategies in discrete dynamic programming I: Decomposition of randomized strategies and embedded models, SIAM Theory Probab. Appl., 31 (1986), pp. 658-668.
21. E. A. FEINBERG, Constrained semi-Markov decision processes with average rewards, Z. Oper. Res., 39 (1995), pp. 257-288.
22. E. A. FEINBERG AND M. I. REIMAN, Optimality of randomized trunk reservation, Probab. Engrg. Inform. Sci., 8 (1994), pp. 463-489.
23. A. V. FIACCO, Convergence properties of local solutions of convex optimization problems, J. Optim. Theory Appl., 13 (1974), pp. 1-12.
24. V. A. GAITSGORY AND A. A. PERVOZVANSKII, Perturbation theory for mathematical programming problems, J. Optim. Theory Appl., 49 (1986), pp. 389-410.
25. A. HORDIJK AND L. C. M. KALLENBERG, Constrained undiscounted stochastic dynamic programming, Math. Oper. Res., 9 (1984), pp. 276-289.
26. A. HORDIJK AND F. SPIEKSMA, Constrained admission control to a queuing system, Adv. in Appl. Probab., 21 (1989), pp. 409-431.
27. L. C. M. KALLENBERG, Linear Programming and Finite Markovian Control Problems, Mathematical Centre Tracts 148, Amsterdam, 1983.
28. P. KANNIAPPAN AND S. M. A. SASTRY, Uniform convergence of convex optimization problems, J. Math. Anal. Appl., 96 (1974), pp. 1-12.
29. N. V. KRYLOV, The construction of an optimal strategy for a finite controlled chain, Theory Probab. Appl., 10 (1965), pp. 45-54.
30. A. LAZAR, Optimal flow control of a class of queuing networks in equilibrium, IEEE Trans. Automat. Control, 28 (1983), pp. 1001-1007.
31. M. B. LIGNOTA AND J. MORGAN, Topological existence and stability of min sup problems, J. Math. Anal. Appl., 151 (1990), pp. 164-180.
32. M. B. LIGNOTA AND J. MORGAN, Convergences of marginal functions with dependent constraints, Optimization, 23 (1992), pp. 189-213.
33. R. LUCCHETTI AND R. J. B. WETS, Convergence of minima of integral functionals, with applications to optimal control and stochastic optimization, Statist. Decisions, 11 (1993), pp. 69-84.
34. J. MORGAN AND R. RAUCCI, Continuity properties of ∈-solutions for generalized parametric saddle point problems and application to hierarchical games, J. Math. Anal. Appl., 211 (1997), pp. 30-48.
35. P. NAIN AND K. W. ROSS, Optimal priority assignment with hard constraint, IEEE Trans. Automat. Control, 31 (1986), pp. 883-888.
36. A. A. PERVOZVANSKII AND V. A. GAITSGORY, Theory of Suboptimal Decision: Decomposition and Aggregation, Kluwer Academic Publishers, Dordrecht, The Netherlands, 1988.
37. R. T. ROCKAFELLAR AND R. J. B. WETS, Variational Analysis, Grundlehren Math. Wiss. 317, Springer-Verlag, Berlin, 1998.
38. K. W. ROSS, Randomized and past-dependent policies for Markov decision processes with multiple constraints, Oper. Res., 37 (1989), pp. 474-477.
39. K. W. ROSS AND B. CHEN, Optimal scheduling of interactive and non interactive traffic in telecommunication systems, IEEE Trans. Automat. Control, 33 (1988), pp. 261-267.
40. K. ROSS AND R. VARADARAJAN, Markov decision processes with sample path constraints: The communicating case, Oper. Res., 37 (1989), pp. 780-790.
41. K. ROSS AND R. VARADARAJAN, Multichain Markov decision processes with a sample path constraint: A decomposition approach, Math. Oper. Res., 16 (1991), pp. 195-207.
42. M. SCHAL, A selection theorem for optimization problems, Arch. Math., 25 (1974), pp. 219-224.
43. I. E. SCHOCHETMAN, Pointwise versions of the maximum theorem with applications to optimization, Appl. Math. Lett., 3 (1990), pp. 89-92.
44. I. E. SCHOCHETMAN AND R. L. SMITH, Convergence of selections with applications in optimization, J. Math. Anal. Appl., 155 (1991), pp. 278-242.
45. L. I. SENNOTT, Constrained discounted Markov decision chains, Probab. Engrg. Inform. Sci., 5 (1991), pp. 463-475.
46. L. I. SENNOTT, Constrained average cost Markov decision chains, Probab. Engrg. Inform. Sci., 7 (1993), pp. 69-83.