On average reward semi-Markov decision processes with a general multichain structure

Mathematics of Operations Research

Volumn 29, Issue 2, 2004, Pages 339-352

  Jianyong, L ^a   Xiaobo, Z ^b  

^a INSTITUTE OF APPLIED MATHEMATICS   (China)

^b TSINGHUA UNIVERSITY   (China)

Author keywords

Average reward criterion; Data transformation method; Multichain structure; Optimal policy; Semi Markov decision processes

Indexed keywords

DECISION THEORY; MATHEMATICAL MODELS; OPTIMIZATION; PROBABILITY; STATE SPACE METHODS;

AVERAGE REWARD CRITERION; MULTICHAIN STRUCTURE; OPTIMAL POLICY; SEMI-MARKOV DECISION PROCESSES;

MARKOV PROCESSES;

EID: 4043081188     PISSN: 0364765X     EISSN: None     Source Type: Journal    
DOI: 10.1287/moor.1030.0077     Document Type: Article

References (32)

1. Arapostathis, A., V. S. Borkar, E. Fernandez-Gaucherand, M. K. Ghosh, S. I. Marcus. 1993. Discrete-time controlled Markov processes with average cost criterion: A survey. SIAM J. Contin. Optim. 31(2) 282-344.
2. Beutler, F. J., K. W. Ross. 1986. Time-average optimal constrained semi-Markov decision processes. Adv. Appl. Probab. 18 341-359.
3. Beutler, F. J., K. W. Ross. 1987. Uniformization for semi-Markov decision processes under stationary policies. J. Appl. Probab. 24 644-656.
4. Das, T. K., A. Gosavi, S. Mahadevan, N. Marchalleck. 1999. Solving semi-Markov decision problems using average reward reinforcement learning. Management Sci. 45(4) 560-574.
5. DeCani, J. S. 1964. A dynamic programming algorithm for embedded Markov chains when the planning horizon is at infinity. Management Sci. 10 716-733.
6. Denardo, E. V., B. L. Fox. 1968. Multichain Markov renewal programs. SIAM J. Appl. Math. 16(3) 468-487.
7. Deppe, H. 1984. On the existence of average optimal policies in semi-regenerative decision models. Math. Oper. Res. 9(4) 558-575.
8. Derman, C. 1970. Finite state Markovian Decision Processes. Academic, New York.
9. Dynkin, E. B., A. A. Yushkevich. 1979. Controlled Markov Processes. Springer-Verlag, New York.
10. Federgruen, A., H. C. Tijms. 1978. The optimality equation in average cost denumerable state semi-Markov decision problems, recurrency conditions and algorithms. J. Appl. Probab. 15 356-373.
11. Federgruen, A., P. J. Schweitzer. 1984. A fixed point approach to undiscounted Markov renewal programs. SIAM J. Algebra Discrete Math. 5(4 ) 539-550.
12. Federgruen, A., A. Hordijk, H. C. Tijms. 1979. Denumerable state semi-Markov decision processes with unbounded costs, average cost criterion. Stochastic Proc. Appl. 9 223-235.
13. Federgruen, A., P. J. Schweitzer, H. C. Tijms. 1983. Denumerable undiscounted semi-Markov decision processes with unbounded rewards. Math. Oper. Res. 8(2) 298-313.
14. Fox, B. 1966. Markov renewal programming by linear fractional programming. SIAM J. Appl. Math. 14(6) 1418-1432.
15. Fox, B. 1967. Existence of stationary optimal policies for some Markov renewal programs. SIAM Rev. 9(3) 573-576.
16. Hinderer, K., K-H. Waldmann. 1998. Approximate solution of Markov renewal programs with finite time horizon. SIAM J. Control Optim. 37(2) 502-520.
17. Howard, R. A. 1963. Semi-Markovian decision processes. Proc. Internat. Statist. Inst. Ottawa, Canada.
18. Hu, Q., J. Liu. 2000. An introduction to Markov decision processes (Chinese). Xidian University, Xian, China.
19. Jewell, W. S. 1963a. Markov-renewal programming. I: Formulation, finite return models. Oper Res. 11 938-948.
20. Jewell, W. S. 1963b. Markov-renewal programming. II: Infinite return models, example. Oper. Res. 11 949-971.
21. Kallenberg, L. C. M. 1983. Linear programming and finite Markov control problems. Math. Centre, Amsterdam, The Netherlands.
22. Lippman, S. A. 1971. Maximal average-reward policies for semi-Markov decision processes with arbitrary state and action space. Ann. Math. Statist. 42(5) 1717-1726.
23. Mann, E. 1986. The functional equations of undiscounted denumerable state Markov renewal programming. J. Janssen, ed. Semi-Markov Model. Plenum, New York, 79-96.
24. Platzman, L. 1977. Improved conditions for convergence in undiscounted Markov renewal programming. Oper. Res. 25 529-533.
25. Puterman, M. L. 1994. Markov Decision Processes: Discrete Stochastic Dynam. Programming. John Wiley and Sons, New York.
26. Ross, S. M. 1970. Applied Probability Models with Optimization Applications. Holden Day, San Francisco.
27. Schal, M. 1992. On the second optimality equation for semi-Markov decision models. Math. Oper. Res. 17(2) 470-486.
28. Schweitzer, P. J. 1971. Iterative solution of the functional equations of undiscounted Markov renewal programming. J. Math. Anal. Appl. 34 495-501.
29. Schweitzer, P. J. 1984. A value-iteration scheme for undiscounted multichain Markov renewal programs. Zeitschrift Oper. Res. 28 143-152.
30. Schweitzer, P. J. 1985. Iterative bounds on the relative value vector in undiscounted Markov renewal programming. Zeitschrift Oper. Res. 29 269-284.
31. Schweitzer, P. J., A. Federgruen. 1978. The functional equations of undiscounted Markov renewal programming. Math. Oper. Res. 3(4) 308-321.
32. Sennott, L. I. 1989. Average cost semi-Markov decision processes and the control of queueing systems. Probab. Engrg. Inform. Sci. 3 247-272.