Average optimality in Markov control processes via discounted-cost problems and linear programming

SIAM Journal on Control and Optimization

Volumn 34, Issue 1, 1996, Pages 295-310

  Hernández Lerma, Onésimo ^a   Lasserre, Jean B ^b  

^a DEPARTAMENTO DE FÍSICA   (Mexico)

^b LAAS CNRS   (France)

Author keywords

(Discrete time) Markov control processes; Average cost criterion; Discounted cost criterion; Linear programming (in general vector spaces)

Indexed keywords

CONSTRAINT THEORY; COSTS; LINEAR PROGRAMMING; MARKOV PROCESSES; SET THEORY;

AVERAGE COST CRITERION; BOREL SPACES; DISCOUNTED COST CRITERIA; MARKOV CONTROL PROCESSES;

DISCRETE TIME CONTROL SYSTEMS;

EID: 0029733401     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012993245306     Document Type: Article

References (17)

1. E. J. ANDERSON AND P. NASH, Linear Programming in Infinite-Dimensional Spaces, John Wiley, Chichester, 1987.
2. A. ARAPOSTATHIS, V. S. BORKAR, E. FERNÁNDEZ-GAUCHERAND, M. K. GHOSH, AND S. I. MARCUS, Discrete-time controlled Markov processes with average cost criterion: A survey, SIAM J. Control Optim., 31 (1993), pp. 282-344.
3. E. V. DENARDO, On linear programming in a Markov decision problem, Manag. Sci., 16 (1970), pp. 281-288.
4. E. B. DYNKIN AND A. A. YUSHKEVICH, Controlled Markov Processes, Springer-Verlag, New York, 1979.
5. O. HERNÁNDEZ-LERMA, Adaptive Markov Control Processes, Springer-Verlag, New York, 1989.
6. _, Existence of average optimal policies in Markov control processes with strictly unbounded costs, Kybernetika (Prague), 29 (1993), pp. 1-17.
7. O. HERNÁNDEZ-LERMA AND D. HERNÁNDEZ-HERNÁNDEZ, Discounted cost Markov decision processes on Borel spaces: The linear programming formulation, J. Math. Anal. Appl., 183 (1994), pp. 335-351.
8. O. HERNÁNDEZ-LERMA AND J. B. LASSERRE, Linear programming and average optimality of Markov control processes on Borel spaces - unbounded costs, SIAM J. Control Optim., 32 (1994), pp. 480-500.
9. O. HERNÁNDEZ-LERMA AND M. MUNOZ DE OZAK, Discrete-time Markov control processes with discounted unbounded costs: Optimality criteria, Kybernetika (Prague), 28 (1992), pp. 191-213.
10. K. HINDERER, Foundations of Non-Stationary Dynamic Programming with Discrete-Time Parameter, Lecture Notes Oper. Res. 33, Springer-Verlag, New York, 1970.
11. M. KURANO, The existence of a minimum pair of state and policy for Markov decision processes under the hypothesis of Doeblin, SIAM J. Control Optim., 27 (1989), pp. 296-307.
12. L. C. M. KALLENBERG, Linear Programming and Finite Markovian Control Problems, Math. Centre Tracts 148, Mathematische Centrum, Amsterdam, 1983.
13. J. B. LASSERRE, Average optimal stationary policies and linear programming in countable state Markov decision processes, J. Math. Anal. Appl., 183 (1994), pp. 233-249.
14. S. P. MEYN AND R. L. TWEEDIE, Stability of Markovian processes I: Criteria for discrete-time chains, Adv. in Appl. Probab., 24 (1992), pp. 542-574.
15. R. MONTES-DE-OCA AND O. HERNÁNDEZ-LERMA, Conditions for average optimality in Markov control processes with unbounded costs and controls, summary in J. Math. Systems Estim. Control, 4 (1994), pp. 145-148. The full paper is available via anonymous ftp in trick.ntp.springer.de, file name /jmsec/11617.ps.
16. L. I. SENNOTT, Average cost optimal stationary policies in infinite state Markov decision processes with unbounded costs, Oper. Res., 37 (1989), pp. 626-633.
17. K. YOSIDA, Functional Analysis, 6th ed., Springer-Verlag, New York, 1980.