The convergence of value iteration in average cost Markov decision chains

Operations Research Letters

Volumn 19, Issue 1, 1996, Pages 11-16

  Sennott, Linn I ^a  

^a ILLINOIS STATE UNIVERSITY   (United States)

Author keywords

Average cost Markov decision process; Stochastic dynamic programming; Value iteration

Indexed keywords

EID: 0009245727     PISSN: 01676377     EISSN: None     Source Type: Journal    
DOI: 10.1016/0167-6377(96)00018-1     Document Type: Article

References (14)

1. A. Arapostathis, V.S. Borkar, E. Fernandez-Gaucherand, M.K. Ghosh and S.I. Marcus, "Discrete-time controlled Markov processes with average cost criterion: a survey", SIAM J. Control Optim. 31, 282-344 (1993).
2. D. Bertsekas, Dynamic Programming, Prentice-Hall, Englewood Cliffs, NJ, 1987.
3. V.S. Borkar, "On minimum cost per unit time control of Markov chains", SIAM J. Control Optim. 22, 965-978 (1984).
4. V.S. Borkar, "Control of Markov chains with long-run average cost criterion: the dynamic programming equations", SIAM J. Control Optim. 27, 642-657 (1989).
5. R. Cavazos-Cadena and L.I. Sennett, "Comparing recent assumptions for the existence of average optimal stationary policies", Oper. Res. Lett. 11, 33-37 (1992).
6. R. Cavazos-Cadena and E. Fernandez-Gaucherand, "Denumerable controlled Markov chains with strong average optimality criterion: bounded and unbounded costs", to appear in: ZOR: Math. Models Oper. Res.
7. M.K. Ghosh and S.I. Marcus, "On strong average optimality of Markov decision processes with unbounded costs", Oper. Res. Lett. 11, 99-104 (1992).
8. O. Hernandez-Lerma and J.B. Lasserre, "Linear Programming and average optimality of Markov control processes on Borel spaces - unbounded costs", SIAM J. Control Optim. 32, 480-500 (1994).
9. M.L. Puterman, Markov Decision Processes, Wiley, New York, 1994.
10. S.M. Ross, Introduction to Stochastic Dynamic Programming, Academic Press, New York, 1983.
11. L.I. Sennott, "Average cost optimal stationary policies in infinite state Markov decision processes with unbounded costs", Oper. Res. 37, 626-633 (1989).
12. L.I. Sennott, "The average cost optimality equation and critical number policies", Probab. Engrg. Inform. Sci. 7, 47-67 (1993).
13. L.I. Sennott, "Another set of conditions for average optimality in Markov control processes", Systems Control Lett. 24, 147-151 (1995).
14. H.C. Tijms, Stochastic Models, An Algorithmic Approach, Wiley, New York, 1994.