First passage models for denumerable semi-Markov decision processes with nonnegative discounted costs

Acta Mathematicae Applicatae Sinica

Volumn 27, Issue 2, 2011, Pages 177-190

  Huang, Yong hui ^a   Guo, Xian ping ^a  

^a SUN YAT SEN UNIVERSITY   (China)

Author keywords

discounted cost; first passage time; optimal policy; Semi Markov decision processes; target set

Indexed keywords

EID: 79952542715     PISSN: 01689673     EISSN: 16183932     Source Type: Journal    
DOI: 10.1007/s10255-011-0061-2     Document Type: Article

References (18)

1. Derman, C. Finite state Markov decision processes. Academic Press, New York, 1970.
2. Guo, X. P., Hernández-Lerma, O. Continuous-time controlled Markov chains. Ann. Appl. Probab., 13: 363-388 (2003).
3. Guo, X. P., Hernández-Lerma, O. Continuous-time controlled Markov chains with discounted rewards. Acta. Appl. Math., 79: 195-216 (2003).
4. Guo X. P., Hernández-Lerma, O. Continuous-Time Markov Decision Processes: Theory and Applications (Chapter 3). Springer-Verlag, Berlin Heidelberg, 2009.
5. Hernández-Lerma, O., Lasserre, J. B. Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer-Verlag, New York, 1996.
6. Hernández-Lerma, O., Lasserre, J. B. Futher Topics on Discrete-Time Markov Control Processes. Springer-Verlag, New York, 1999.
7. Limnios, N., Oprisan, J. Semi-Markov Processes and Reliability. Birkhäuser, Boston, 2001.
8. Lin, Y. L. Continuous time first arrival target models (1)-discounted moment optimal models. Acta. Math. Appl. Sinica, 14: 115-124 (1991) (in Chinese).
9. Lin, Y. L., Tomkins, R. J., Wang, C. L. Optimal models for the first arrival time distribution function in continuous time-with a special case. Acta. Math. Appl. Sinica, 10: 194-212 (1994).
10. Lippman, S. A. Semi-Markov decision processes with unbounded rewards. Management Science, 19: 717-731 (1973).
11. Liu, J. Y., Huang, S. M. Markov decision processes with distribution function criterion of first-passage time. Appl. Math. Optim., 43: 187-201 (2001).
12. Liu, J. Y., Liu, K. Markov decision programming-the first passage model with denumerable state space. Systems Sci. Math. Sci., 5: 340-351 (1992).
13. Liu, J. Y., Liu, K. Markov decision programming-the moment optimal problem for the first-passage model. J. Austral. Math. Soc. Ser. B, 38: 542-562 (1997).
14. Ohtsubo, Y. Optimal threshold probability in undiscounted Markov decision processes with a target set. Appl. Math. Anal. Comp., 149: 519-532 (2004).
15. Puterman, M. L. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons. Inc., New York, 1994.
16. Ross, S. M. Average cost semi-Markov decision processes. J. Appl. Probab., 7: 649-656 (1970).
17. Sennott, L. I. Stochastic Dynamic Programming and the Control of Queuing Systems. John Wiley & Sons. Inc., New York, 1999.
18. Yu, S. X., Lin, Y. L., Yan, P. F. Optimization models for the first arrival target distribution function in discrete time. J. Math. Anal. Appl., 225: 193-223 (1998).