Constrained average cost Markov control processes in Borel spaces

SIAM Journal on Control and Optimization

Volumn 42, Issue 2, 2003, Pages 442-468

  Hernández Lerma, Onésimo ^a   González Hernández, Juan ^b   López Martínez, Raquiel R ^c  

^a DEPARTAMENTO DE FÍSICA   (Mexico)

^b Cd Universitaria   (Mexico)

^c Facultad de Matematicas   (Mexico)

Author keywords

Average cost; Constrained Markov control processes; Discounted cost

Indexed keywords

AVERAGE COSTS; CONSTRAINED MARKOV CONTROL PROCESSES; DISCOUNTED COSTS;

CONSTRAINT THEORY; GRAPH THEORY; MATHEMATICAL MODELS; PROBABILITY; STATE SPACE METHODS; VECTORS;

MARKOV PROCESSES;

EID: 11144355176     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012999361627     Document Type: Article

References (43)

1. E. ALTMAN, Constrained Markov Decision Processes, Chapman and Hall/CRC, Boca Raton, FL, 1999.
2. E. J. ANDERSON AND P. NASH, Linear Programming in Infinite-Dimensional Spaces, Wiley, Chichester, U.K., 1987.
3. A. ARAPOSTATHIS, V. S. BORKAR, E. FERNÁNDEZ-GAUCHERAND, M. K. GHOSH, AND S. I. MARCUS, Discrete-time controlled Markov processes with average cost criterion: A survey, SIAM J. Control Optim., 31 (1993), pp. 282-344.
4. R. B. ASH, Real Analysis and Probability, Academic Press, New Yor, 1972.
5. D. P. BERTSEKAS AND S. E. SHREVE, Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York, 1978.
6. P. BILLINGSLEY, Convergence of Probability Measures, Wiley, New York, 1968.
7. D. BLACKWELL, Memoryless strategies in finite-stage dynamic programming, Ann. Math. Statist., 33 (1964), pp. 863-865.
8. V. S. BORKAR, Ergodic control of Markov chains with constraints-the general case, SIAM J. Control Optim., 32 (1994), pp. 176-186.
9. B. D. CRAVEN AND J. J. KOLIHA, Generalizations of Farkas' theorem, SIAM J. Math. Anal. Appl., 8 (1977), pp. 983-997.
10. R. DEKKER AND A. HORDIJK, Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards, Math. Oper. Res., 13 (1988), pp. 395-420.
11. E. B. DYNKIN AND A. A. YUSHKEVICH, Controlled Markov Processes, Springer-Verlag, Berlin, 1979.
12. E. FEINBERG AND A. SHWARTZ, Constrained discounted dynamic programming, Math. Oper. Res., 21 (1996), pp. 922-945.
13. E. FEINBERG AND A. SHWARTZ, Constrained dynamic programming with two discount factors: Applications and an algorithm, IEEE Trans. Automat. Control, 44 (1999), pp. 628-631.
14. K. GOLABI, R. B. KULKARNI, AND G. B. WAY, A statewide pavement management system, Interfaces, 12 (1982), pp. 5-21.
15. E. GORDIENKO AND O. HERNÁNDEZ-LERMA, Average cost Markov control processes with weighted norms: Existence of canonical policies, Appl. Math. (Warsaw), 23 (1995), pp. 199-218.
16. E. GORDIENKO AND O. HERNÁNDEZ-LERMA, Average cost Markov control processes with weighted norms: Value iteration, Appl. Math. (Warsaw), 23 (1995), pp. 219-237.
17. O. HERNÁNDEZ-LERMA AND J. GONZÁLEZ-HERNÁNDEZ, Infinite linear programming and multi-chain Markov control processes in uncountable spaces, SIAM J. Control Optim., 36 (1998), pp. 313-335.
18. O. HERNÁNDEZ-LERMA AND J. B. LASSERRE, Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer-Verlag, New York, 1996.
19. O. HERNÁNDEZ-LERMA AND J. B. LASSERRE, Cone-constrained linear equations in Banach spaces, J. Convex Anal., 4 (1997), pp. 149-164.
20. O. HERNÁNDEZ-LERMA AND J. B. LASSERRE, Approximation schemes for infinite linear programs, SIAM J. Optim., 8 (1998), pp. 973-988.
21. O. HERNÁNDEZ-LERMA AND J. B. LASSERRE, Linear programming approximations for Markov control processes in metric spaces, Acta Appl. Math., 51 (1998), pp. 123-139.
22. O. HERNÁNDEZ-LERMA AND J. B. LASSERRE, Further Topics on Discrete-Time Markov Control Processes, Springer-Verlag, New York, 1999.
23. K. HINDERER, Foundations of Non-Stationary Dynamic Programming with Discrete Time Parameter, Lecture Notes in Operations Research and Mathematical Systems 33, Springer-Verlag, Berlin, 1970.
24. A. HORDIJK AND F. SPIEKSMA, Constrained admission control to a queueing system, Adv. In Appl. Probab., 21 (1989), pp. 409-431.
25. A. HORDIJK AND A. A. YUSHKEVICH, Blackwell optimality in the class of stationary policies in Markov decision chains with a Borel state space and unbounded rewards, Math. Methods Oper. Res., 49 (1999), pp. 1-39.
26. A. HORDIJK AND A. A. YUSHKEVICH, Blackwell optimality in the class of all policies in Markov decision chains with a Borel state space and unbounded rewards, Math. Methods Oper. Res., 50 (1999), pp. 421-448.
27. Y. HUANG AND M. KURANO, The LP approach in average rewards MDPs with multiple cost constraints: The countable state case, J. Inform. Optim. Sci., 18 (1997), pp. 33-47.
28. M. KURANO, The existence of a minimum pair of state and policy for Markov decision processes under the hypothesis of Doeblin, SIAM J. Control Optim., 27 (1989), pp. 296-307.
29. M. KURANO, J.-I. NAKAGAMI, AND Y. HUANG, Constrained Markov decision processes with compact state and action spaces: The average case, Optimization, 48 (2000), pp. 255-269.
30. A. LAZAR, Optimal flow control of a class of queueing networks in equilibrium, IEEE Trans, Automat. Control, 28 (1983), pp. 1001-1007.
31. A. B. PIUNOVSKIY, Optimal Control of Random Sequences in Problems with Constraints, Kluwer, Boston, 1997.
32. K. Ross AND R. VARADARAJAN, Multichain Markov decision processes with a sample path constraint: A decomposition approach, Math. Oper. Res., 16 (1991), pp. 195-207.
33. L. I. SENNOTT, Constrained average cost Markov decision chains, Probab. Engrg. Inform. Sci., 7 (1993), pp. 69-83.
34. F. VAKIL AND A. A. LAZAR, Flow control protocols for integrated networks with partially observed voice traffic, IEEE Trans. Automat. Control, 32 (1987), pp. 2-14.
35. K. YOSIDA, Functional Analysis, 5th ed., Springer-Verlag, Berlin, 1978.
36. R. R. LÓPEZ-MARTÍNEZ AND O. HERNÁNDEZ-LERMA, The Lagrange and Pareto Approaches to Constrained Markov Control Processes, Internal Report 283, Mathematics Department, CINVESTAV-IPN.
37. O. HERNÁNDEZ-LERMA AND J. GONZÁLEZ-HERNÁNDEZ, Constrained Markov control processes in Borel spaces: The discounted case, Math. Methods. Oper. Res., 52 (2000), pp. 271-285.
38. O. HERNÁNDEZ-LERMA AND R. ROMERA, Pareto Optimality in Multiobjective Markov Control Processes, Internal Report 278, Mathematics Department, CINVESTAV-IPN.
39. O. HERNÁNDEZ-LERMA AND L. F. HOYOS-REYES, A multiobjective control approach to priority queues, Math. Methods Oper. Res., 53 (2001), pp. 265-277.
40. O. HERNÁNDEZ-LERMA AND L. F. HOYOS-REYES, A Multiobjective Formulation of Optimal Control Problems with Additive Costs, Internal Report 286, Mathematics Department, CINVESTAV-IPN.
41. J. I. GONZÁLEZ-TREJO, O. HERNÁNDEZ-LERMA, AND L. F. HOYOS-REYES, Minimax control of discrete-time stochastic systems, SIAM J. Control Optim., 41 (2003), pp. 1626-1659.
42. O. HERNÁNDEZ-LERMA AND R. ROMERA, Limiting discounted-cost control of partially observable stochastic systems, SIAM J. Control Optim., 40 (2001), pp. 348-369.
43. A. HORDIJK AND L. C. M. KALLENBERG, Constrained undiscounted stochastic dynamic programming, Math. Oper. Res., 9 (1984), pp. 276-289.