Recursive Markov Decision Processes and Recursive Stochastic Games

Lecture Notes in Computer Science

Volumn 3580, Issue , 2005, Pages 891-903

  Etessami, Kousha ^a   Yannakakis, Mihalis ^b  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b Columbia University ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; DECISION THEORY; MARKOV PROCESSES; PROBABILITY;

CONTEXT-FREE GRAMMERS; MULTI-TYPE BRANCHING; RECURSIVE MARKOV CHAINS (RMC);

GAME THEORY;

EID: 26444468097     PISSN: 03029743     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1007/11523468_72     Document Type: Conference Paper

References (28)

1. [AEY01] R. Alur, K. Etessami, and M. Yannakakis. Analysis of recursive state machines. In Proc. of 13th CAV, pages 304-313, 2001.
2. [ATM03] R. Alur, S. La Torre, and P. Madhusudan. Modular strategies for recursive game graphs. In TACAS, volume 2619 of LNCS, pages 363-378, 2003.
3. [BC03] V. Blondel and V. Canterini. Undecidable problems for probabilistic automata of fixed dimension. Theory of Comput. Sys., 36:231-245, 2003.
4. [BKS05] T. Brázdil, A. Kučera, and O. Straž ovský. Decidability of temporal properties of probabilistic pushdown automata. In Proc. of STACS'05, 2005.
5. [BPR96] S. Basu, R. Pollack, and M. F. Roy. On the combinatorial and algebraic complexity of quantifier elimination. J. ACM, 43(6): 1002-1045, 1996.
6. [Can88] J. Canny. Some algebraic and geometric computations in PSPACE. In Prof. of 20th ACM STOC, pages 460-467, 1988.
7. [CL89] A. Condon and R. Lipton. The complexity of space bounded interactive proofs. In Proc. of 30th IEEE FOCS, 1989.
8. [Con92] A. Condon. The complexity of stochastic games. Inf. & Comp., 96(2):203-224, 1992.
9. [CY95] C. Courcoubetis and M. Yannakakis. The complexity of probabilistic verification. Journal of the ACM, 42(4):857-907, 1995.
10. [CY98] C. Courcoubetis and M. Yannakakis. Markov decision processes and regular events. IEEE Trans. on Automatic Control, 43(10):1399-1418, 1998.
11. +00] L. de Alfaro, M. Kwiatkowska, G. Norman, D. Parker, and R. Segala. Symbolic model checking of probabilistic processes using MTBDDs and the kronecker representation. In Proc. of 6th TACAS, pages 395-410, 2000.
12. [EKM04] J. Esparza, A. Kučera, and R. Mayr. Model checking probabilistic push-down automata. In Proc. of 19th IEEE LICS'04, 2004.
13. [Ete04] K. Etessami. Analysis of recursive game graphs using data flow equations. In Proc. 5th VMCAI, vol. 2937 LNCS, pages 282-296. Springer, 2004.
14. [EY05a] K. Etessami and M. Yannakakis. Recursive markov chains, stochastic grammars, and monotone systems of non-linear equations. In Proc. of 22nd STACS'05. Springer, 2005. (Tech. Report, U. Edinburgh, June 2004).
15. [EY05b] K. Etessami and M. Yannakakis. Algorithmic verification of recursive probabilistic state machines. In Proc. 11th TACAS, vol. 3440 of LNCS, 2005.
16. [FS02] E. Feinberg and A. Shwartz, editors. Handbook of Markov Decision Processes. Kluwer, 2002.
17. [FV97] J. Filar and K. Vrieze. Competitive Markov Decision Processes. Springer, 1997.
18. [Har63] T. E. Harris. The Theory of Branching Processes. Springer-Verlag, 1963.
19. [HSP83] S. Hart, M. Sharir, and A. Pnueli. Termination of probabilistic concurrent programs. ACM Trans. Prog. Lang. & Sys., 5(3):356-380, 1983.
20. [Jag75] P. Jagers. Branching Processes with Biological Applications. Wiley, 1975.
21. [Kwi03] M. Kwiatkowska. Model checking for probability and time: from theory to practice. In 18th IEEE LICS, pages 351-360, 2003.
22. [Mar98] D. A. Martin. Determinacy of Blackwell games. J. Symb. Logic, 63(4):1565-1581, 1998.
23. [MS99] C. Manning and H. Schütze. Foundations of Statistical Natural Language Processing. MIT Press, 1999.
24. [Paz71] A. Paz. Introduction to Probabilistic Automata. Academic Press, 1971.
25. [Put94] M. L. Puterman. Markov Decision Processes. Wiley, 1994.
26. [Ren92] J. Renegar. On the computational complexity and geometry of the first-order theory of the reals, parts I-III. J. Symb. Comp., 13(3):255-352, 1992.
27. +94] Y. Sakakibara, M. Brown, R Hughey, I.S. Mian, K. Sjolander, R. Underwood, and D. Haussler. Stochastic context-free grammars for tRNA modeling. Nucleic Acids Research, 22(23):5112-5120, 1994.
28. [Var85] M. Vardi. Automatic verification of probabilistic concurrent finite-state programs. In Proc. of 26th IEEE FOCS, pages 327-338, 1985.