Reduction techniques for model checking markov decision processes

Proceedings - 5th International Conference on the Quantitative Evaluation of Systems, QEST 2008

Volumn , Issue , 2008, Pages 45-54

  Ciesinski, Frank ^a   Baier, Christel ^a   Größer, Marcus ^a   Klein, Joachim ^a  

^a DRESDEN UNIVERSITY OF TECHNOLOGY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

HEURISTIC ALGORITHMS; LEARNING ALGORITHMS; MODEL CHECKING; NUMERICAL METHODS; PROBABILITY; TIME SERIES ANALYSIS;

LTL FORMULAS; MARKOV DECISION PROCESSES; OF GRAPHS; PROBABILISTIC MODELS; QUANTITATIVE ANALYSES; RANDOMIZED SYSTEMS; REACH ABILITIES; REDUCTION TECHNIQUES; SPEED-UP; STANDARD ALGORITHMS; TIME COMPLEXITIES;

HEURISTIC METHODS;

EID: 56649116055     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/QEST.2008.45     Document Type: Conference Paper

References (35)

1. [BCG04] C. Baier, F. Ciesinski, and M. Größer. Probmela: a modeling language for communicating probabilistic systems. In Proceeding MEMOCODE, 2004.
2. [BCGK08] C. Baier, F. Ciesinski, M. Groesser, and J. Klein. Reduction techniques for model checking Markov decision processes. In Technical report http://www.inf.tudresden.de/content/institutes/thi/algi/publikationen/texte/ Qest08-with.appendix.pdf, 2008.
3. [BdA95] A. Bianco and L. de Alfaro. Model checking of probabilistic and nondeterministic systems. In Proc. Foundations of Software Technology and Theoretical Computer Science (FST & TCS), volume 1026 of Lecture Notes in Computer Science, pages 499-513, 1995.
4. [Ber08] Michel Berkelaar. The lpsolve package. http://cran.r-project. org/web/packages/lpSolve, 2008.
5. [BGC04] C. Baier, M. Größer, and F. Ciesinski. Partial order reduction for probabilistic systems. In QEST 2004 [QES04], pages 230-239.
6. [BK98] C. Baier and M. Kwiatkowska. Model checking for a probabilistic branching time logic with fairness. Distributed Computing, 11:125-155, 1998.
7. [CB07] F. Ciesinski and C. Baier. LiQuor: a tool for qualitative and quantitative linear time analysis of reactive systems. In Proc. QEST, pages 131-132. IEEE CS Press, 2007.
8. [CGP99] E. Clarke, Orna Grumberg, and Doron Peled. Model Checking. MIT Press, 1999.
9. [CY90] C. Courcoubetis and M. Yannakakis. Markov decision processes and regular events (extended abstract). In Proc. 17th on International Colloquium Automata, Languages and Programming (ICALP), volume 443 of Lecture Notes in Computer Science, pages 336-349, 1990.
10. [CY95] C. Courcoubetis and M. Yannakakis. The complexity of probabilistic verification. Journal of the ACM, 42(4):857-907, 1995.
11. [dA97a] L. de Alfaro. Formal Verification of Probabilistic Systems. PhD thesis, Stanford University, Department of Computer Science, 1997.
12. [dA97b] L. de Alfaro. Temporal logics for the specification of performance and reliability. In Proc. STACS, volume 1200 of Lecture Notes in Computer Science, pages 165-179, 1997.
13. [DG06] P. Dai and J. Goldsmith. Faster Dynamic Programming for Markov Decision Processes. In Proc. ICAPS, 2006.
14. [DG07] P. Dai and J. Goldsmith. Topological value iteration algorithm for markov decision processes. In Proc. IJCAI, pages 1860-1865, 2007.
15. [DH07] P. Dai and E. Hansen. Prioritizing Bellman backups without a priority queue. In Proc. ICAPS. AAAI Press, 2007.
16. [DJJL01] P.R. D'Argenio, B. Jeannet, H.E. Jensen, and K.G. Larsen. Reachability analysis of probabilistic systems by successive refinements. In L. de Alfaro and S. Gilmore, editors, Proceedings of Process Algebra and Probabilistic Methods. Performance Modeling and Verification. Joint International Workshop, PAPM-PROBMIV 2001, Aachen, Germany, volume 2165 of Lecture Notes in Computer Science, pages 29-56. Springer, 2001.
17. [DJJL02] P. D'Argenio, B. Jeannet, H. Jensen, and K. Larsen. Reduction and refinement strategies for probabilistic analysis. In Proc. PAPM-PROBMIV, volume 2399 of LNCS, pages 335-372, 2002.
18. [DN04] P.R. D'Argenio and P. Niebert. Partial order reduction on concurrent probabilistic programs. In QEST 2004 [QES04], pages 240-249.
19. [Hol03a] G. Holzmann. The SPIN Model Checker, Primer and Reference Manual. Addison Wesley, 2003.
20. [Hol03b] G.J. Holzmann. The Spin Model Checker, Primer and Reference Manual. Addison-Wesley, Reading, Massachusetts, 2003.
21. [HP94] G. Holzmann and D. Peled. An improvement in formal verification. In Proc. Formal Description Techniques, FORTE94, pages 197-211, Berne, Switzerland, October 1994. Chapman & Hall.
22. [KB06] J. Klein and C. Baier. Experiments with deterministic omega-automata for formulas of linear temporal logic. Theoretical Computer Science, 363(2):182-195, 2006.
23. [KB07] J. Klein and C. Baier. On-the-fly stuttering in the construction of deterministic omega-automata. In Proceedings CIAA'07, volume 4783 of Lecture Notes in Computer Science, pages 51-61, 2007.
24. +98] R. Kurshan, V. Levin, M. Minea, D. Peled, and H. Yenign. Static partial order reduction. In Proc. Tools and Algorithms for Construction and Analysis of Systems (TACAS), volume 1384 of Lecture Notes in Computer Science, pages 345-357, 1998.
25. [KNP04] M. Kwiatkowska, G. Norman, and D. Parker. Probabilistic symbolic model checking with prism: A hybrid approach. International Journal on Software Tools for Technology Transfer (STTT), 6(2):128-142, 2004.
26. [KPN08] Marta Kwiatkowska, Dave Parker, and Gethin Norman. Symmetry reduction in prism (prototype implementation). http://www.prismmodelchecker. org/symm/, 2008.
27. [KTZJ07] J.P. Katoen, Kemna T, I. Zapreev, and D. Jansen. Bisimulation minimisation mostly speeds up probabilistic model checking. In Proc. TACAS, volume 4424 of LNCS, pages 87-101, 2007.
28. [Lyn96] N. Lynch. Distributed Algorithms. MIT Press, 1996.
29. [MA93] A. Moore and C. Atkeson. Prioritized sweeping: Reinforcement learning with less data and less real time. Machine Learning, 13:103-130, 1993.
30. [Pel93] D. Peled. All from one, one for all: On model checking using representatives. In Proc. 5th International Computer Aided Verification Conference (CAV), volume 697 of Lecture Notes in Computer Science, pages 409-423, 1993.
31. [Pel96] D. Peled. Partial order reduction: Linear and branching time logics and process algebras. In [PPH97], pages 79-88, 1996.
32. [PPH97] D. Peled, V. Pratt, and G. Holzmann, editors. Partial Order Methods in Verification, volume 29(10) of DIMACS Series in Discrete Mathematics and Theoretical Computer Science. American Mathematical Society, American Mathematical Society, 1997.
33. [Put94] Martin L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc., New York, NY, 1994.
34. [QES04] Proceedings of the 1st International Conference on Quantitative Evaluation of SysTems (QEST 2004). Enschede, the Netherlands. IEEE Computer Society Press, 2004.
35. [Saf88] Shmuel Safra. On the complexity of omega-automata. In Proceedings of the 29th Annual Symposium on Foundations of Computer Science, FoCS'88, pages 319-327, Los Alamitos, California, October 1988. IEEE Computer Society Press.