On the optimality equation for average cost Markov control processes with Feller transition probabilities

Journal of Mathematical Analysis and Applications

Volumn 316, Issue 2, 2006, Pages 495-509

  Jaśkiewicz, Anna ^a   Nowak, Andrzej S ^b  

^a WROCLAW UNIVERSITY OF TECHNOLOGY   (Poland)

^b UNIVERSITY OF ZIELONA GÓRA   (Poland)

Author keywords

Average cost; Average cost optimal policies; Borel state space; Discrete time Markov control processes; Dynamic programming; Optimality equation

Indexed keywords

EID: 29844439998     PISSN: 0022247X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jmaa.2005.04.065     Document Type: Article

References (34)

1. A. Arapostathis V.S. Borkar E. Fernnández-Gaucherand M.K. Ghosh S.I. Marcus Discrete-time controlled Markov processes with average cost criterion: A survey SIAM J. Control Optim. 31 1993 282-344
2. E. Berge Topological Spaces 1963 Macmillan New York
3. D.P. Bertsekas S.E. Shreve Stochastic Optimal Control: The Discrete Time Case 1978 Academic Press New York
4. R.N. Bhattacharya M. Majumdar Controlled semi-Markov model under long-run average rewards J. Statist. Plann. Inference 22 1989 223-242
5. L.D. Brown R. Purves Measurable selections of extrema Ann. Statist. 1 1973 902-912
6. P.K. Dutta What do discounted optima converge to? A theory of discount rate asymptotics in economic models J. Econ. Theory 55 1991 64-94
7. E.B. Dynkin A.A. Yushkevich Controlled Markov Processes 1979 Springer-Verlag New York
8. E.A. Feinberg M.E. Lewis Optimality of four-threshold policies in inventory systems with customer returns and borrowing/storage options Probab. Engrg. Inform. Sci. 19 2005 45-71
9. J.I. González-Trejo O. Hernández-Lerma L.F. Hoyos-Reyes Minimax control of discrete-time stochastic systems SIAM J. Control Optim. 41 2003 1626-1659
10. E. Gordienko O. Hernández-Lerma Average cost Markov control processes with weighted norms: Existence of canonical policies Appl. Math. (Warsaw) 23 1995 199-218
11. E. Gordienko O. Hernández-Lerma Average cost Markov control processes with weighted norms: Value iteration Appl. Math. (Warsaw) 23 1995 219-237
12. O. Hernández-Lerma Adaptive Markov Control Processes 1989 Springer-Verlag New York
13. O. Hernández-Lerma J.B. Lasserre Average cost optimal policies for Markov control processes with Borel state space and unbounded costs Systems Control Lett. 15 1990 349-356
14. O. Hernández-Lerma J.B. Lasserre Discrete-Time Markov Control Processes: Basic Optimality Criteria 1996 Springer-Verlag New York
15. O. Hernández-Lerma J.B. Lasserre Further Topics on Discrete-Time Markov Control Processes 1999 Springer-Verlag New York
16. A. Jaśkiewicz A.S. Nowak On the optimality equation for zero-sum ergodic stochastic games Math. Methods Oper. Res. 54 2001 291-301
17. A. Jaśkiewicz, A.S. Nowak, Zero-sum ergodic stochastic games with Feller transition probabilities, SIAM J. Control. Optim. (2005), in press
18. N.V. Kartashov Strong Stable Markov Chains 1996 VSP Utrecht, The Netherlands
19. E. Klein A.C. Thompson Theory of Correspondences 1984 Wiley New York
20. H.-U. Küenle, On Markov games with average reward criterion and weakly continuous transition probabilities, submitted for publication
21. A. Leizarowitz Overtaking and almost-sure optimality for infinite horizon Markov decision processes Math. Oper. Res. 21 1996 158-181
22. S.P. Meyn Stability, performance evaluation, and optimization in: F.A. Feinberg A. Shwartz (Eds.), Handbook of Markov Decision Processes, Methods and Applications 2002 Kluwer Academic Dordrecht 305-346
23. S.P. Meyn R.L. Tweedie Markov Chains and Stochastic Stability 1993 Springer-Verlag New York
24. S.P. Meyn R.L. Tweedie Computable bounds for geometric convergence rates of Markov chains Ann. Appl. Probab. 4 1994 981-1011
25. E. Nummelin General Irreducible Markov Chains and Non-Negative Operators 1983 Cambridge Univ. Press Cambridge
26. K.R. Parthasarathy Probability Measures on Metric Spaces 1967 Academic Press New York
27. M.L. Puterman Markov Decision Processes. Discrete Stochastic Dynamic Programming 1994 Wiley New York
28. R.K. Ritt L.I. Sennott Optimal stationary policies in general state space Markov decision chains with finite action sets Math. Oper. Res. 17 1992 901-909
29. W.J. Runggaldier Ł. Stettner Approximations of Discrete Time Partially Observed Control Problems 1994 Giardini Editori e Stampatori Pisa
30. M. Schäl Average optimality in dynamic programming with general state space Math. Oper. Res. 18 1993 163-172
31. L.I. Sennott A new condition for the existence of optimal stationary policies in average cost Markov decision processes Oper. Res. Lett. 5 1986 17-23
32. R. Serfozo Convergence of Lebesgue integrals with varying measures Sankhyã Ser. A 44 1982 380-402
33. H.C. Tijms Average reward optimality equation in Markov decision processes in: F.P. Kelly (Ed.), Probability, Statistics and Optimization: A Tribute to Peter Whittle 1994 Wiley New York 485-495
34. O. Vega-Amaya The average cost optimality equation: A fixed point approach Bol. Soc. Mat. Mexicana 9 2003 185-195