Efficient Computation of Time-Bounded Reachability Probabilities in Uniform Continuous-Time Markov Decision Processes

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 2988, Issue , 2004, Pages 61-76

  Baier, Christel ^a   Haverkort, Boudewijn ^b   Hermanns, Holger ^c   Katoen, Joost Pieter ^c  

^a UNIVERSITY OF BONN   (Germany)

^b UNIVERSITY OF TWENTE   (Netherlands)

^c SAARLAND UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; BEHAVIORAL RESEARCH; COMPUTATION THEORY; ENERGY MANAGEMENT; LEARNING ALGORITHMS; MARKOV PROCESSES; PROBABILITY; PROBABILITY DISTRIBUTIONS;

CONTINUOUS TIME MARKOV CHAIN; CONTINUOUS TIME MARKOV DECISION PROCESS; DELAY TIME DISTRIBUTIONS; EFFICIENT COMPUTATION; NON-DETERMINISM; NONDETERMINISTIC CHOICE; TIME BOUND; TIME-BOUNDED REACHABILITY;

CONTINUOUS TIME SYSTEMS;

EID: 27644517480     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-24730-2_5     Document Type: Article

References (24)

1. M. Ajmone Marsan, G. Balbo, G. Conte, S. Donatelli, and G. Franceschinis. Modelling with Generalized Stochastic Petri Nets. John Wiley & Sons, 1995.
2. A. Aziz, K. Sanwal, V. Singhal and R. Brayton. Model checking continuous time Markov chains. ACM TOOL, 1(1): 162-170, 2000.
3. C. Baier, B. Haverkort, H. Hermanns, and J.-P. Katoen. Efficient computation of time-bounded reachability probabilities in uniform continuous-time Markov decision processes. CTIT Technical Report 03-50, University of Twente, 2003.
4. C. Baier, B. Haverkort, H. Hermanns, and J.-P. Katoen. Model-checking algorithms for continuous-time Markov chains. IEEE Transactions on Software Engineering, 29(6): 524-541, 2003.
5. C. Baier and M.Z. Kwiatkowska. Model checking for a probabilistic branching time logic with fairness. Distributed Computing, 11: 125-155, 1998.
6. D.P. Bertsekas. Dynamic Programming and Optimal Control. Volume II. Athena Scientific, 1995.
7. F. Beutler and K.W. Ross. Uniformization for semi-Markov decision processes under stationary policies. Journal of Applied Probability, 24: 644-656, 1987.
8. A. Bianco and L. de Alfaro. Model checking of probabilistic and nondeterministic systems. In Foundations of Software Technology and Theoretical Computer Science, LNCS 1026: 499-513, 1995.
9. P. Buchholz. Exact and ordinary lumpability in finite Markov chains. Journal of Applied Probability, 31:.59-75, 1994.
10. P. Buchholz, J.-P. Katoen, P. Kemper, and C. Tepper. Model-checking large structured Markov chains. Journal of Logic and Algebraic Programming, 56:69-97, 2003.
11. G. Chiola, S. Donatelli, and G. Franceschinis. GSPNs versus SPNs: What is the actual role of immediate transitions? In Petri Nets and Performance Models 1991, IEEE CS Press, 1991.
12. G. Ciardo and R. Zijal. Well-defined stochastic Petri nets. In Proc. 4th International Workshop on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS'96), pages 278-284, 1996.
13. D.D. Deavours, and W.H. Sanders. An efficient well-specified check. In Petri Nets and Performance Models, IEEE CS Press, 1999.
14. E.A. Feinberg. Continuous time discounted jump Markov decision processes: a discrete-event approach. 1998. http://www.ams.sunysb.edu/~feinberg/public/.
15. B.L. Fox and P.W. Glynn. Computing Poisson probabilities. Communications of the ACM, 31(4): 440-445, 1988.
16. H. Hermanns, U. Herzog, and V. Mertsiotakis. Stochastic process algebras - Between LOTOS and Markov chains. Computer Networks and ISDN Systems, 30(9-10):901-924, 1998.
17. H. Hermanns. Interactive Markov Chains and the Quest for Quantified Quality. LNCS 2428, Springer, 2002.
18. H. Hermanns, J.-P. Katoen, J. Meyer-Kayser, and M. Siegle. A tool for model-checking Markov chains. Int. Journal on Software Tools for Technology Transfer, 4(2):153-172, 2003.
19. A. Jensen. Markov chains as an aid in the study of Markov processes. Skand. Aktuarietidskrift, 3: 87-91, 1953.
20. J.-P. Katoen, M.Z. Kwiatkowska, G. Norman and D. Parker. Faster and symbolic CTMC model checking. In Process Algebra and Probabilistic Methods, LNCS 2165: 23-38, Springer, 2001.
21. M.L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, 1994.
22. Q. Qiu and M. Pedram. Dynamic power managment based on continuous-time Markov decision processes. In Design Automation Conference 99, pp. 555-561, 1999.
23. W.H. Sanders and J.F. Meyer. Reduced base model construction methods for stochastic activity networks. IEEE Journal on Selected Areas in Communications, 9(1):25-36, January 1991.
24. L. Sennot. Stochastic Dynamic Programming and the Control of Queueing Systems. John Wiley & Sons, 1999.