Numerical analysis of continuous time Markov decision processes over finite horizons

Computers and Operations Research

Volumn 38, Issue 3, 2011, Pages 651-659

  Buchholz, Peter ^a   Schulz, Ingo ^a  

^a TU DORTMUND UNIVERSITY   (Germany)

Author keywords

Continuous time Markov decision processes; Error bounds; Finite horizon; Uniformization

Indexed keywords

ACTION SPACES; ADAPTIVE TIME STEP; ANALYSIS APPROACH; CONTINUOUS TIME MARKOV DECISION PROCESS; DISCRETIZATION METHOD; DISCRETIZATIONS; ERROR BOUNDS; FINITE HORIZONS; FINITE INTERVALS; FINITE STATE; OPTIMAL POLICIES; PIECEWISE CONSTANT; PREDEFINED PRECISION; UNIFORMIZATION;

CONTINUOUS TIME SYSTEMS; DISCRETE EVENT SIMULATION; ERROR ANALYSIS; NUMERICAL METHODS; OPTIMIZATION;

MARKOV PROCESSES;

EID: 77957305140     PISSN: 03050548     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cor.2010.08.011     Document Type: Article

References (19)

1. D.P. Bertsekas Dynamic programming and optimal control vol. I 2005 Athena Scientific
2. D.P. Bertsekas Dynamic programming and optimal control vol. II 2007 Athena Scientific
3. Q. Hu, and W. Yue Markov decision processes with their applications. Advances in mechanics and mathematics 2008 Springer
4. M.L. Puterman Markov decision processes 2005 Wiley
5. D. Gross, and D. Miller The randomization technique as a modeling tool and solution procedure for transient Markov processes Operations Research 32 2 1984 926 944
6. A. Jensen Markoff chains as an aid in the study of Markoff processes Skandinavisk Aktuarietidskritft 36 1953 87 91
7. F.J. Beutler, and K.W. Ross Uniformization for semi-Markov decision processes under stationary policies Journal of Applied Probability 24 1987 644 656
8. R.F. Serfozo An equivalence between continuous and discrete time Markov decision processes Operations Research 27 3 1979 616 620
9. B.L. Miller Finite state continuous time Markov decision processes with a finite planning horizon SIAM Journal on Control 6 2 1968 266 280
10. M. Yasuda On the existence of optimal control in continuous time Markov decision processes Bulletin of Mathematical Statistics 15 1972 7 17
11. M.R. Lembersky On maximal rewards and ε optimal policies in continuous time Markov decision chains The Annals of Statistics 2 4 1974 159 169
12. A. Martin-Lfs Optimal control of a continuous-time Markov chain with periodic transition probabilities Operations Research 15 1967 872 881
13. S.A. Lippman Applying a new device in the optimization of exponential queuing systems Operations Research 23 4 1975 687 710
14. S.A. Lippman Countable-state, continuous-time dynamic programming with structure Operations Research 24 3 1976 477 490
15. S. Bhatnagar, and M.S. Abdulla Simulation-based optimization algorithms for finite-horizon Markov decision processes Simulation 84 12 2008 577 600
16. W.J. Stewart Introduction to the numerical solution of Markov chains 1994 Princeton University Press
17. A. Rindos, S. Woolet, I. Viniotis, and K. Trivedi Exact methods for the transient analysis of nonhomogeneous continuous time Markov chains W.J. Stewart, Computations with Markov chains 1995 Kluwer Academic Publishers 121 133
18. M. Arns, P. Buchholz, and A. Panchenko On the numerical analysis of inhomogeneous continuous time Markov chains INFORMS Journal on Computing 22 3 2010 416 432
19. C. Baier, H. Hermanns, J.P. Katoen, and B.R. Haverkort Efficient computation of time-bounded reachability probabilities in uniform continuous-time Markov decision processes Theoretical Computer Science 345 1 2005 2 26