Markov decision processes and regular events

IEEE Transactions on Automatic Control

Volumn 43, Issue 10, 1998, Pages 1399-1418

  Courcoubetis, Costas ^a   Yannakakis, Mihalis ^b  

^a UNIVERSITY OF CRETE   (Greece)

^b LUCENT TECHNOLOGIES   (United States)

Author keywords

automata; Complexity; Discrete event systems; Markov decision processes; Probabilistic games; Regular languages; Verification

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; DECISION THEORY; FORMAL LANGUAGES; GAME THEORY; MARKOV PROCESSES; SET THEORY;

FINITE-STATE MARKOV DECISION PROCESSES; PROBABILISTIC GAMES;

DISCRETE TIME CONTROL SYSTEMS;

EID: 0032188357     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/9.720497     Document Type: Article

References (35)

1. S. Aggarwal, C. Courcoubetis, and P. Wolper, "Adding liveness properties to coupled finite-state machines," ACM TOPLAS, 1990.
2. A. Aho, J. Hopcroft, and J. Ullman, The Design and Analysis of Computer Algorithms. Reading, MA: Addison-Wesley, 1974.
3. M. Abadi, L. Lamport, and P. Wolper, "Realizable and unrealizable specifications of reactive systems," in Proc. 16th ICALP, 1989, vol. LNCS 372, pp. 1-17.
4. F. Beutler and K. Ross, "Optimal policies for controlled Markov chains with a constraint," J. Math. Analysis and Appl., vol. 112, pp. 236-252, 1985.
5. _, "Time-average optimal constrained semi-Markov decision processes," Adv. Appl. Prob., vol. 18, no. 2, pp. 341-359, 1986.
6. R. Cieslak, C. Desclaux, A. Fawaz, and P. Varaiya, "Supervisory control of discrete-event processes with partial information," IEEE Trans. Automat. Contr., vol. 33, pp. 249-260, Mar. 1988.
7. C. Courcoubetis and M. Yannakakis, "Verifying temporal properties of finite-state probabilistic programs," in Proc. 29th FOCS, 1988, pp. 338-345.
8. _, "The complexity of probabilistic verification," J. ACM, vol. 42, pp. 857-907, Apr. 1995.
9. C. Derman, Finite-State Markovian Decision Processes. New York, Academic, 1970.
10. S. Hart and M. Sharir, "Probabilistic temporal logic for finite and bounded models," in Proc. 16th ACM Symp. Theory of Computing, J. Thistle, Ed. 1984, pp. 1-13.
11. L. C. M. Kallenberg, Linear Programming and Finite Markovian Control Problems. Amsterdam, The Netherlands: Mathematical Center Tracts, 1983.
12. D. Lehman and S. Shelah, "Reasoning with time and chance," Inform. and Contr., vol. 53, pp. 165-198, 1982.
13. J. Lin and W. M. Wonham, "On observability of discrete-event systems," Inform. Sci., vol. 44, no. 2, pp. 173-198, 1988.
14. R. McNaughton, "Testing and generating infinite sequences by a finite automaton," Inform. and Contr., vol. 9, pp. 521-530, 1966.
15. A. Pnueli, "The temporal logic of concurrent programs," Theoretical Computer Sci., vol. 13, pp. 45-60, 1981.
16. A. Pnueli and R. Rosner, "On the synthesis of a reactive module," in Proc. 16th ACM Symp. Principles of Programming Languages, 1989, pp. 179-190.
17. _, "Distributed reactive systems are hard to synthesize," in Proc. 31th IEEE Symp. Foundations of Computer Science, 1990, pp. 746-757.
18. A. Pnueli and L. D. Zuck, "Probabilistic verification," Inform. and Computation, vol. 103, pp. 1-29, 1993.
19. M. L. Puterman, Markov Decision Processes. New York: Wiley, 1994.
20. J. P. Queille and J. Sifakis, "Fairness and related properties in transition systems," IMAG, Grenoble, France, Res. Rep. 292, 1982.
21. M. O. Rabin, "Automata on infinite objects and Church's problem," in Proc. Regional AMS Conf. Series in Math., 1972, vol. 13, pp. 1-22.
22. P. Ramadge, "Some tractable supervisory control problems for discrete event systems modeled by Buchi automata," IEEE Trans. Automat. Contr., vol. 34, pp. 10-19, Jan. 1989.
23. S. Ross, Introduction to Stochastic Dynamic Programming. New York: Academic, 1983.
24. K. Ross and R. Varadarajan, "Markov decision processes with sample path constraints; the communicating case," Operations Res., 1990.
25. P. Ramadge and W. M. Wonham, "Supervisory control of a class of discrete-event processes," SIAM J. Contr. and Optimiz., vol. 25, no. 1, pp. 206-230, Jan. 1987.
26. S. Safra, "On the complexity of ω-automata," in Proc. 29th FOCS, Oct. 1988, pp. 319-327.
27. W. Thomas, "Automata on infinite objects," in Handbook of Theoretical Computer Science, vol. 2, J. van Leeuwen, Ed. Cambridge, MA: MIT Press, 1990, pp. 133-192.
28. J. Thistle, "Control of infinite behavior of discrete event systems," Ph.D. dissertation, Univ. of Toronto, 1991.
29. J. Thistle and W. Wonham, "On the synthesis of supervisors subject to ω-language specifications," 22nd Annual Conf. Information Sciences and Systems, Princeton NJ, Mar. 1988, pp. 440-444.
30. M. Vardi, "Automatic verification of probabilistic concurrent finite-state programs," in Proc. 26th STOC, 1985.
31. M. Vardi and P. Wolper, "An automata-theoretic approach to automatic program verification," in Proc. 1st Symp. Logic in Computer Science, 1986.
32. H. Wong-Toi and D. Dill, "Synthesizing processes and schedulers from temporal specifications," Computer-Aided Verfication (Proc. CAV 90 Workshop), DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol. 3. American Mathematical Society, 1991.
33. P. Wolper, "Temporal logic can be more expressive," Inform. and Contr., vol. 56, pp. 72-99, 1983.
34. W. M. Wonham and P. Ramadge, "On the supremal controllable sublanguage of a given language," SIAM J. Contr. and Optim., vol. 25, no. 3, pp. 637-659, May 1987.
35. P. Wolper, M. Y. Vardi, and A. P. Sistla, "Reasoning about infinite computation paths," in Proc. 24th IEEE Symp. Foundations of Computer Science, 1983, pp. 185-194.