A game-based abstraction-refinement framework for Markov decision processes

Formal Methods in System Design

Volumn 36, Issue 3, 2010, Pages 246-280

  Kattenbelt, Mark ^a   Kwiatkowska, Marta ^a   Norman, Gethin ^b   Parker, David ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

^b UNIVERSITY OF GLASGOW   (United Kingdom)

Author keywords

Abstraction; Abstraction refinement; Markov decision processes; Probabilistic verification

Indexed keywords

ABSTRACTION; ABSTRACTION PROCESS; ABSTRACTION REFINEMENT; AUTOMATIC GENERATION; EFFICIENT ALGORITHM; LOWER AND UPPER BOUNDS; MARKOV DECISION PROCESSES; NON-DETERMINISM; PROBABILISTIC VERIFICATION; PROTOTYPE IMPLEMENTATIONS; QUANTITATIVE MEASURES; REACHABILITY; REAL-WORLD; REFINEMENT METHODS; STATE-SPACE EXPLOSION; TWO-PLAYER GAMES;

ALGORITHMS; MARKOV PROCESSES; MATHEMATICAL MODELS; MODEL CHECKING; SOFTWARE PROTOTYPING;

ABSTRACTING;

EID: 77956718255     PISSN: 09259856     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10703-010-0097-6     Document Type: Article

References (44)

1. C Baier M Kwiatkowska 1998 Model checking for a probabilistic branching time logic with fairness Distrib Comput 11 3 125 155 10.1007/s004460050046
2. D Bertsekas J Tsitsiklis 1991 An analysis of stochastic shortest path problems Math Oper Res 16 3 580 595 0751.90077 10.1287/moor.16.3.580 1120471
3. Billingsley P (1979) Probability and measure. Wiley, New York
4. Chadha R, Viswanathan M (2010) A counterexample guided abstraction-refinement framework for Markov decision processes. ACM Trans Comput Logic (to appear)
5. Chatterjee K, de Alfaro L, Henzinger T (2004) Trading memory for randomness. In: Proc. 1st int. conf. quantitative evaluation of systems (QEST'04). IEEE Comput. Soc., Los Alamitos, pp 206-217
6. Chatterjee K, Henzinger T, Jhala R, Majumdar R (2005) Counterexample-guided planning. In: Proc. 21st conference in uncertainty in artificial intelligence (UAI'05), pp 104-111
7. Cheshire S, Adoba B, Guttman E (2002) Dynamic configuration of IPv4 link-local addresses (draft August 2002). Zeroconf Working Group of the Internet Engineering Task Force ( www.zeroconf.org )
8. Clarke E, Grumberg O, Jha S, Lu Y, Veith H (2000) Counterexample-guided abstraction refinement. In: Emerson A, Sistla A (eds) Proc. 12th int. conf. computer aided verification (CAV'00). Lecture notes in computer science, vol 1855. Springer, Berlin, pp 154-169
9. A Condon 1992 The complexity of stochastic games Inf Comput 96 2 203 224 0756.90103 10.1016/0890-5401(92)90048-K 1147987
10. A Condon 1993 On algorithms for simple stochastic games. Advances in computational complexity theory DIMACS Ser Discrete Math Theor Comput Sci 13 51 73 1255328
11. D'Argenio P, Jeannet B, Jensen H, Larsen K (2001) Reachability analysis of probabilistic systems by successive refinements. In: de Alfaro L, Gilmore S (eds) Proc. 1st joint int workshop process algebra and probabilistic methods, performance modelling and verification (PAPM/PROBMIV'01). Lecture notes in computer science, vol 2165. Springer, Berlin, pp 39-56
12. de Alfaro L (1999) Computing minimum and maximum reachability times in probabilistic systems. In: Baeten J, Mauw S (eds) Proc. 10th int. conf. concurrency theory (CONCUR'99). Lecture notes in computer science, vol 1664. Springer, Berlin, pp 66-81
13. de Alfaro L (1997) Formal verification of probabilistic systems. Ph.D. thesis, Stanford University
14. de Alfaro L, Roy P (2007) Magnifying-lens abstraction for Markov decision processes. In: Damm W, Hermanns H (eds) Proc. 19th int. conf. computer aided verification (CAV'07). Lecture notes in computer science, vol 4590. Springer, Berlin, pp 325-338
15. de Alfaro L, Henzinger T, Kupferman O (1998) Concurrent reachability games. In: Proc. 39th symp. foundations of computer science (FOCS'98). IEEE Comput. Soc., Los Alamitos, pp 564-575
16. L de Alfaro T Henzinger O Kupferman 2007 Concurrent reachability games Theor Comput Sci 386 3 188 217 1154.91306
17. J Desharnais V Gupta R Jagadeesan P Panangaden 2003 Approximating labelled Markov processes Inf Comput 184 1 160 200 1028.68091 10.1016/S0890-5401(03)00051-8 1983184
18. Fecher H, Leucker M, Wolf V (2006) Don't know in probabilistic systems. In: Valmari A (ed) Proc. 13th int. spin workshop on model checking of software (SPIN'06). Lecture notes in computer science, vol 3925. Springer, Berlin, pp 71-88
19. Graf S, Saidi H (1997) Construction of abstract state graphs with PVS. In: Grumberg O (ed) Proc. 9th int. conf. computer aided verification (CAV'97). Lecture notes in computer science, vol 1254. Springer, Berlin, pp 72-83
20. T Han JP Katoen B Damman 2009 Counterexample generation in probabilistic model checking IEEE Trans Softw Eng 35 2 241 257 10.1109/TSE.2009.5
21. Hermanns H, Wachter B, Zhang L (2008) Probabilistic CEGAR. In: Gupta A, Malik S (eds) Proc. 20th int. conf. computer aided verification (CAV'08). Lecture notes in computer science, vol 5123. Springer, Berlin, pp 162-175
22. Hinton A, Kwiatkowska M, Norman G, Parker D (2006) PRISM: A tool for automatic verification of probabilistic systems. In: Hermanns H, Palsberg J (eds) Proc. 12th int. conf. tools and algorithms for the construction and analysis of systems (TACAS'06). Lecture notes in computer science, vol 3920. Springer, Berlin, pp 441-444 (Pubitemid 44019387)
23. J Hurd A McIver C Morgan 2005 Probabilistic guarded commands mechanized in HOL Theor Comput Sci 346 1 96 112 1080.68063 10.1016/j.tcs.2005.08.005 2182231 (Pubitemid 41606836)
24. Huth M (2004) An abstraction framework for mixed nondeterministic and probabilistic systems. In: Baier C, Haverkort B, Hermanns H, Katoen JP, Siegle M (eds) Validation of stochastic systems. Lecture notes in computer science, vol 2925. Springer, Berlin, pp 419-444
25. Kattenbelt M, Kwiatkowska M, Norman G, Parker D (2008) Game-based probabilistic predicate abstraction in PRISM. In: Proc. 6th workshop quantitative aspects of programming languages (QAPL'08)
26. Kattenbelt M, Kwiatkowska M, Norman G, Parker D (2009) Abstraction refinement for probabilistic software. In: Jones N, Muller-Olm M (eds) Proc. 10th int. conf. verification, model checking and abstract interpretation (VMCAI'09). Lecture notes in computer science, vol 5403. Springer, Berlin, pp 182-197
27. Kemeny J, Snell J, Knapp A (1976) Denumerable Markov chains, 2nd edn. Springer, Berlin
28. Kwiatkowska M, Norman G, Parker D (2006) Game-based abstraction for Markov decision processes. In: Proc. 3th int. conf. quantitative evaluation of systems (QEST'06). IEEE Comput. Soc., Los Alamitos, pp 157-166
29. M Kwiatkowska G Norman D Parker J Sproston 2006 Performance analysis of probabilistic timed automata using digital clocks Form Methods Syst Des 29 33 78 1105.68063 10.1007/s10703-006-0005-2 (Pubitemid 44075773)
30. Kwiatkowska M, Norman G, Parker D (2009) Stochastic games for verification of probabilistic timed automata. In: Ouaknine J, Vaandrager F (eds) Proc. 7th international conference on formal modelling and analysis of timed systems (FORMATS'09). Lecture notes in computer science, vol 5813. Springer, Berlin, pp 212-227
31. K Larsen A Skou 1991 Bisimulation through probabilistic testing Inf Comput 94 1 28 0756.68035 10.1016/0890-5401(91)90030-6 1123153
32. McIver A, Morgan C (2004) Abstraction, refinement and proof for probabilistic systems. Monographs in computer science. Springer, Berlin
33. D Monniaux 2005 Abstract interpretation of programs as Markov decision processes Sci Comput Program 58 1-2 179 205 1088.68039 10.1016/j.scico.2005.02. 008 2167568 (Pubitemid 41261381)
34. Norman G (2004) Analysing randomized distributed algorithms. In: Baier C, Haverkort B, Hermanns H, Katoen JP, Siegle M (eds) Validation of stochastic systems. Lecture notes in computer science, vol 2925. Springer, Berlin, pp 384-418
35. PASS tool homepage. http://depend.cs.uni-sb.de/PASS/
36. Pierro AD, Hankin C, Wiklicky H (2006) Abstract interpretation for worst and average case analysis. In: Reps T, Sagiv M, Bauer J (eds) Program analysis and compilation, theory and practice, essays dedicated to Reinhard Wilhelm on the occasion of his 60th birthday. Lecture notes in computer science, vol 4444. Springer, Berlin, pp 160-174
37. PRISM web site. http://www.prismmodelchecker.org/
38. Segala R (1995) Modelling and verification of randomized distributed real time systems. Ph.D. thesis, Massachusetts Institute of Technology
39. Sen K, Viswanathan M, Agha G (2006) Model-checking Markov chains in the presence of uncertainties. In: Hermanns H, Palsberg J (eds) Proc. 12th int. conf. tools and algorithms for the construction and analysis of systems (TACAS'06). Lecture notes in computer science, vol 3920. Springer, Berlin, pp 394-410 (Pubitemid 44019384)
40. Shapley L (1953) Stochastic games. In: Proc. national academy of science, vol 39, pp 1095-1100
41. Smith M (2008) Probabilistic abstract interpretation of imperative programs using truncated normal distributions. In: Aldini A, Baier C (eds) Proc. 6th workshop on quantitative aspects of programming languages (QAPL'08). Electronic notes in theoretical computer science, vol 220(3). Elsevier, Dordrecht, pp 43-59
42. Stoelinga M, Vaandrager F (1999) Root contention in IEEE 1394. In: Katoen JP (ed) Proc. 5th int. AMAST workshop real-time and probabilistic systems (ARTS'99). Lecture notes in computer science, vol 1601. Springer, Berlin, pp 53-74
43. Wachter B, Zhang L (2010) Best probabilistic transformers. In: Barthe G, Hermenegildo M (eds) Proc. 11th int. conf. verification, model checking and abstract interpretation (VMCAI'10). Lecture notes in computer science, vol 5944. Springer, Berlin, pp 362-379
44. Wachter B, Zhang L, Hermanns H (2006) Probabilistic model checking modulo theories. In: Proc. 4th int. conf. quantitative evaluation of systems (QEST'07). IEEE Comput. Soc., Los Alamitos, pp 129-138