The computation of average optimal policies in denumerable state Markov decision chains

Advances in Applied Probability

Volumn 29, Issue 1, 1997, Pages 114-137

  Sennott, Linn I ^a  

^a ILLINOIS STATE UNIVERSITY   (United States)

Author keywords

Computation of average cost optimal stationary policies; Control of discrete time queues; Controlled Markov processes

Indexed keywords

EID: 0004270527     PISSN: 00018678     EISSN: None     Source Type: Journal    
DOI: 10.1017/S0001867800027816     Document Type: Article

References (25)

1. APOSTOL, T. (1974) Mathematical Analysis. Addison-Wesley, Reading, MA.
2. ARAPOSTATHIS, A., BORKAR, V. S., FERNÁNDEZ-GAUCHERAND, E., GHOSH, M. K. AND MARCUS, S. I. (1993) Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J. Control Optim. 31, 282-344.
3. BERTSEKAS, D. (1987) Dynamic Programming. Prentice-Hall, Englewood Cliffs, NJ.
4. BORKAR, V. S. (1984) On minimum cost per unit time control of Markov chains. SIAM J. Control. Optim. 22, 965-978.
5. BORKAR, V. S. (1989) Control of Markov chains with long-run average cost criterion: the dynamic programming equations. SIAM J. Control Optim. 27, 642-657.
6. CAVAZOS-CADENA, R. (1986) Finite-state approximations for denumerable state discounted Markov decision processes. Appl. Math. Optim. 14, 1-26.
7. CAVAZOS-CADENA, R. AND SENNOTT, L. I. (1992) Comparing recent assumptions for the existence of average optimal stationary policies. Operat. Res. Lett. 11, 33-37.
8. CHUNG, K. L. (1967) Markov Chains with Stationary Transition Probabilities. 2nd edn. Springer, New York.
9. FOX, B. L. (1971) Finite-state approximations to denumerable state dynamic programs. J. Math. Anal. Appl. 34, 665-670.
10. GIBSON, D. AND SENETA, E. (1986) Augmented truncations of infinite stochastic matrices. Report. Department of Mathematical Statistics, University of Sydney.
11. GIBSON, D. AND SENETA, E. (1987) Augmented truncations of infinite stochastic matrices. J. Appl. Prob. 24, 600-608.
12. GOLUB, G. H. AND SENETA, E. (1974) Computation of the stationary distribution of an infinite stochastic matrix of special form. Bull. Austral. Math. Soc. 10, 255-261.
13. GRASSMAN, W. K., TAKSAR, M. I. AND HEYMAN, D. P. (1985) Regenerative analysis and steady state distributions for Markov chains. Operat. Res. 33, 1107-1116.
14. HERNANDEZ-LERMA, O. (1989) Adaptive Markov Control Processes (Applied Mathematical Sciences 79). Springer, New York.
15. HEYMAN, D. P. (1991) Approximating the stationary distribution of an infinite stochastic matrix. J. Appl. Prob. 28, 96-103.
16. PUTERMAN, M. L. (1994) Markov Decision Processes. Wiley, New York.
17. ROSS, S. M. (1983) Introduction to Stochastic Dynamic Programming. Academic Press, New York.
18. ROYDEN, H. L. (1968) Real Analysis. 2nd edn. Macmillan, New York.
19. SENNOTT, L. I. (1989) Average cost optimal stationary policies in infinite state Markov decision processes with unbounded costs. Operat. Res. 37, 626-633.
20. SENNOTT, L. I. (1993) The average cost optimality equation and critical number policies. Prob. Eng. Inf. Sci. 7, 47-67.
21. SENNOTT, L. I. (1995) Another set of conditions for average optimality in Markov control processes. Sys. Control. Lett. 24, 147-151.
22. THOMAS, L. C. AND STENGOS, D. (1985) Finite state approximation algorithms for average cost denumerable state Markov decision processes. OR Spek. 7, 27-37.
23. TIDBALL, M. M. AND ALTMAN, E. (1996) Approximations in dynamic zero-sum games. I. SIAM J. Control Optim. 34, 311-328.
24. WHITE, D. J. (1980) Finite state approximations for denumerable state infinite horizon discounted Markov decision processes: the method of successive approximations. In Recent Developments in Markov Decision Processes. ed. R. Hartley, L. C. Thomas and D. J. White. Academic Press, New York. pp. 57-72.
25. WOLF, D. (1980) Approximation of the invariant probability measure of an infinite stochastic matrix. Adv. Appl. Prob. 12, 710-726.