Finite horizon semi-Markov decision processes with application to maintenance systems

European Journal of Operational Research

Volumn 212, Issue 1, 2011, Pages 131-140

  Huang, Yonghui ^a,b   Guo, Xianping ^a,b  

^a SUN YAT SEN UNIVERSITY   (China)

^b Guangdong Province Key Laboratory of Information Security   (China)

Author keywords

Dynamic programming; Finite horizon semi Markov decision processes; Optimal policy; Optimality equation; Value function

Indexed keywords

COST RATES; EFFECTIVE ALGORITHMS; FINITE HORIZONS; MAINTENANCE SYSTEMS; NON-NEGATIVE SOLUTIONS; OPTIMAL POLICIES; OPTIMAL POLICY; OPTIMALITY; OPTIMALITY EQUATION; PLANNING HORIZONS; SEMI-MARKOV DECISION PROCESS; VALUE FUNCTION; VALUE FUNCTIONS;

MAINTENANCE; MARKOV PROCESSES; OPTIMIZATION;

DYNAMIC PROGRAMMING;

EID: 79952700174     PISSN: 03772217     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ejor.2011.01.027     Document Type: Article

References (30)

1. W.J. Anderson Continuous-time Markov chains An Applications-Oriented Approach 1991 Springer-Verlag New York
2. R.B. Ash Probability and Measure Theory second ed. 2000 Harcourt/Academic Press Burlington, MA
3. D.P. Bertsekas, and S.E. Shreve Stochastic Optimal Control: The Discrete-Time Case 1996 Athena Scientific Belmont, Massachusetts
4. X.R. Cao Semi-Markov decision problems and performance sensitivity analysis IEEE Transactions on Automatic Control 48 2003 758 769
5. B. Cekyay, and S. Ozekici Mean time to failure and availability of semi-Markov missions with maximal repair European Journal Operational Research 207 2010 1442 1454
6. E.A. Feinberg Continuous time discounted jump Markov decision processes: A discrete-event approach Mathematics of Operations Research 29 2004 492 524
7. S. Gunter, and L. Swanson Semi-Markov dynamic programming approach to competitive bidding with state space reduction considerations European Journal of Operational Research 32 1987 435 447 (Pubitemid 18530953)
8. X. Guo, and O. Hernández-Lerma Continuous-Time Markov Decision Processes: Theory and Applications 2009 Springer-Verlag Berlin
9. O. Hernández-Lerma, and J.B. Lasserre Discrete-Time Markov Control Processes: Basic Optimality Criteria 1996 Springer-Verlag New York
10. Y. Huang, and X. Guo Optimal risk probability for first passage models in semi-Markov decision processes Journal of Mathematical Analysis and Applications 359 2009 404 420
11. Y. Huang, and X. Guo Discounted semi-Markov decision processes with nonnegative costs Acta Mathematica Sinica (Chinese Series) 53 2010 503 514
12. Y. Huang, X. Guo, and First passage models for denumerable semi-Markov decision processes with nonnegative discounted costs Acta Mathematicae Applicatae Sinica 27 2011 263 276
13. A. Jaskiewicz An approximation approach to ergodic semi-Markov control processes Mathematical Methods in the Applied Sciences 54 2001 1 19 (Pubitemid 32991764)
14. A. Jaskiewicz On the equivalence of two expected average cost criteria for semi-Markov control processes Mathematics of Operations Research 29 2004 326 338
15. A. Jaskiewicz, and A.S. Nowak Optimality in Feller semi-Markov control processes Operations Research Letters 34 2006 713 718
16. M.Y. Kitaev, and V.V. Rykov Controlled Queueing Systems 1995 CRC Press Boca Raton, Florida
17. D. Klabjan, and D. Adelman Existence of optimal policies for semi-Markov decision processes using duality for infinite linear programming SIAM Journal on Control and Optimization 44 2006 2104 2122 (Pubitemid 44775253)
18. D. Klabjan, and D. Adelman An infinite-dimensional linear programming algorithm for deterministic semi-Markov decision processes on Borel spaces Mathematics of Operations Research 32 2007 528 550 (Pubitemid 351159467)
19. N. Limnios, and J. Oprisan Semi-Markov Processes and Reliability 2001 Birkhäuser Boston
20. C.E. Love, Z.G. Zhang, M.A. Zitron, and R. Guo A discrete semi-Markov decision model to determine the optimal repair/replacement policy under general repairs European Journal of Operational Research 125 2000 398 409
21. F. Luque-Vasquez, and J.A. Minjarez-Sosa Semi-Markov control processes with unknown holding times distribution under a discounted criterion Mathematical Methods of Operations Research 61 2005 455 468 (Pubitemid 41118345)
22. J.W. Mamer Successive approximations for finite horizon semi-Markov decision processes with application to asset liquidation Operational Research 34 1986 638 644
23. L.R. Nielsen, and A.R. Kristensen Finding the K best policies in a finite-horizon Markov decision process European Journal Operational Research 175 2006 1164 1179 (Pubitemid 44345078)
24. V. Nollau Solution of a discounted semi-Markovian decision problem by successive overrelaxation Optimization 39 1997 85 97
25. M.L. Puterman Markov Decision Processes: Discrete Stochastic Dynamic Programming 1994 John Wiley & Sons Inc. New York
26. S.M. Ross Average cost semi-Markov decision processes Journal of Applied Probability 7 1970 649 656
27. L.I. Sennott Stochastic Dynamic Programming and the Control of Queuing Systems 1999 John Wiley & Sons Inc. New York
28. S.S. Singh, V.B. Tadic, and A. Doucet A policy gradient method for semi-Markov decision processes with application to call admission control European Journal of Operational Research 178 2007 808 818 (Pubitemid 44765147)
29. D.J. White Finite horizon Markov decision processes with uncertain terminal payoffs Operational Research 43 1995 862 869
30. C.B. Wu, and Y.L. Lin Minimizing risk models in Markov decision processes with policies depending on target values Journal of Mathematical Analysis and Applications 231 1999 47 67