Efficient computation of time-bounded reachability probabilities in uniform continuous-time Markov decision processes

Theoretical Computer Science

Volumn 345, Issue 1, 2005, Pages 2-26

  Baier, Christel ^a   Hermanns, Holger ^b,c   Katoen, Joost Pieter ^c   Haverkort, Boudewijn R ^c  

^a UNIVERSITY OF BONN   (Germany)

^b SAARLAND UNIVERSITY   (Germany)

^c UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

Continuous time; Markov decision process; Model checking; Temporal logic; Time bounded reachability

Indexed keywords

ALGORITHMS; COMPUTATIONAL METHODS; DECISION THEORY; LOGIC PROGRAMMING; MATHEMATICAL MODELS; PROBABILITY;

CONTINUOUS-TIME; MARKOV DECISION PROCESSES; MODEL CHECKING; TEMPORAL LOGIC; TIME-BOUNDED REACHABILITY;

MARKOV PROCESSES;

EID: 27644578953     PISSN: 03043975     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tcs.2005.07.022     Document Type: Conference Paper

References (31)

1. Y. Abdeddaïm, E. Asarin, O. Maler, On optimal scheduling under uncertainty, in: Tools and Algorithms for the Construction and Analysis of Systems, Lecture Notes in Computer Science, Vol. 2619, Springer, Berlin, 2003, pp. 240-253.
2. M. Ajmone Marsan, G. Balbo, G. Conte, S. Donatelli, and G. Franceschinis Modelling with Generalized Stochastic Petri Nets 1995 Wiley New York
3. L. de Alfaro, Formal verification of probabilistic systems, Ph.D. Dissertation, Stanford University, 1997.
4. L. de Alfaro How to specify and verify the long-run average behavior of probabilistic systems IEEE 13th Symp. on Logic in Computer Science 1998 IEEE CS Press Silver Spring, MD 174 183
5. A. Aziz, K. Sanwal, V. Singhal, and R. Brayton Model checking continuous time Markov chains ACM Trans. Comput. Logic 1 1 2000 162 170
6. C. Baier, B. Haverkort, H. Hermanns, and J.-P. Katoen Model-checking algorithms for continuous-time Markov chains IEEE Trans. Software Engrg. 29 6 2003 524 541
7. C. Baier, B. Haverkort, H. Hermanns, J.-P. Katoen, Efficient computation of time-bounded reachability probabilities in uniform continuous-time Markov decision processes, in: Tools and Algorithms for the Construction and Analysis of Systems, Lecture Notes in Computer Science, Vol. 2988, 2004, pp. 61-76.
8. C. Baier, H. Hermanns, J.-P. Katoen, V. Wolf, Comparative branching-time semantics for Markov chains (extended abstract), in: Concurrency Theory, Lecture Notes in Computer Science, Vol. 2761, Springer, Berlin, 2003, pp. 492-507.
9. C. Baier, and M.Z. Kwiatkowska Model checking for a probabilistic branching time logic with fairness Distrib. Comput. 11 1998 125 155
10. D.P. Bertsekas, Dynamic Programming and Optimal Control. Vol. II, Athena Scientific, 1995.
11. F. Beutler, and K.W. Ross Uniformization for semi-Markov decision processes under stationary policies J. Appl. Probab. 24 1987 644 656
12. A. Bianco, L. de Alfaro, Model checking of probabilistic and nondeterministic systems, in: Foundations of Software Technology and Theoretical Computer Science, Lecture Notes in Computer Science, Vol. 1026, Springer, Berlin, 1995, 499-513.
13. J.L. Bruno, P.J. Downey, and G.N. Frederickson Sequencing tasks with exponential service times to minimize the expected flow time or makespan J. Assoc. Comput. Mach. 28 1 1981 100 113
14. P. Buchholz Exact and ordinary lumpability in finite Markov chains J. Appl. Probab. 31 1994 59 75
15. P. Buchholz, J.-P. Katoen, P. Kemper, and C. Tepper Model-checking large structured Markov chains J. Logic Algebraic Programming 56 2003 69 97
16. G. Chiola, S. Donatelli, and G. Franceschinis GSPNs versus SPNs: what is the actual role of immediate transitions? Petri Nets and Performance Models 1991 1991 IEEE CS Press Silver Spring, MD
17. G. Ciardo, R. Zijal, Well-defined stochastic Petri nets, in: Proc. Fourth Internat. Workshop on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS'96), 1996, pp. 278-284.
18. P.R. D'Argenio, B. Jeannet, H.E. Jensen, K.G. Larsen., Reduction and refinement strategies for probabilistic analysis, in: Process Algebra and Probabilistic Methods, Lecture Notes in Computer Science, Vol. 2399, Springer, Berlin, 2001, pp. 57-76.
19. D.D. Deavours, and W.H. Sanders An efficient well-specified check Petri Nets and Performance Models, PNPM'99 1999 IEEE CS Press Silver spring, MD 124 133
20. E.A. Feinberg, Continuous time discounted jump Markov decision processes: a discrete-event approach, 1998. http://www.ams.sunysb.edu/ ∼ feinberg/public/.
21. B.L. Fox, and P.W. Glynn Computing Poisson probabilities Comm. ACM 31 4 1988 440 445
22. H. Hermanns, Interactive Markov Chains and the Quest for Quantified Quality, Lecture Notes in Computer Science, Vol. 2428, Springer, Berlin, 2002.
23. H. Hermanns, J.-P. Katoen, J. Meyer-Kayser, and M. Siegle A tool for model-checking Markov chains Internat. J. Software Tools for Technology Transfer 4 2 2003 153 172
24. A. Jensen Markov chains as an aid in the study of Markov processes Skand. Aktuarietidskrift 3 1953 87 91
25. J.-P. Katoen, M.Z. Kwiatkowska, G. Norman, D. Parker, Faster and symbolic CTMC model checking, in: Process Algebra and Probabilistic Methods, Lecture Notes in Computer Science, Vol. 2165, 2001, Springer, Berlin, pp. 23-38.
26. V.G. Kulkarni Modeling and Analysis of Stochastic Systems 1995 Chapman & Hall London
27. M.Z. Kwiatkowska, G. Norman, D. Parker, Probabilistic symbolic model checking with PRISM: a hybrid approach, in: Tools and Algorithms for the Construction and Analysis of Systems, Lecture Notes in Computer Science, Vol. 2280, 2002, pp. 52-66.
28. M.L. Puterman Markov Decision Processes: Discrete Stochastic Dynamic Programming 1994 Wiley New York
29. Q. Qiu, and M. Pedram Dynamic power management based on continuous-time Markov decision processes Design Automation Conference 99 1999 ACM Press New York 555 561
30. W.H. Sanders, and J.F. Meyer Reduced base model construction methods for stochastic activity networks IEEE J. Selected Areas in Commun. 9 1 1991 25 36
31. L. Sennot Stochastic Dynamic Programming and the Control of Queueing Systems 1999 Wiley New York