Robust control of uncertain Markov Decision Processes with temporal logic specifications

Proceedings of the IEEE Conference on Decision and Control

Volumn , Issue , 2012, Pages 3372-3379

  Wolff, Eric M ^a   Topcu, Ufuk ^a   Murray, Richard M ^a  

^a CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BEHAVIORAL RESEARCH; COMPUTER CIRCUITS; MARKOV PROCESSES; MODEL CHECKING; ROBUST CONTROL; SPECIFICATIONS; STOCHASTIC SYSTEMS; TEMPORAL LOGIC;

FINITE MARKOV DECISION PROCESS; MARKOV DECISION PROCESSES; PROBABILISTIC MODEL CHECKING; STOCHASTIC SHORTEST PATHS; SYSTEM MODELING; TEMPORAL LOGIC SPECIFICATIONS; TIME COMPLEXITY; TRANSITION PROBABILITIES;

DYNAMIC PROGRAMMING;

EID: 84874255767     PISSN: 07431546     EISSN: 25762370     Source Type: Conference Proceeding    
DOI: 10.1109/CDC.2012.6426174     Document Type: Conference Paper

References (21)

1. A. Abate, A. D'Innocenzo, and M. Di Benedetto. Approximate abstractions of stochastic hybrid systems. IEEE Transactions on Automatic Control, 56:2688-2694, 2011.
2. R. Alur and S. La Torre. Deterministic generators and games for LTL fragments. ACM Trans. Comput. Logic, 5(1):1-25, 2004.
3. S. Azuma and G. Pappas. Discrete abstraction of stochastic nonlinear systems: A bisimulation function approach. In American Control Conference, 2010.
4. J. A. Bagnell, A. Y. Ng, and J. G. Schneider. Solving uncertain Markov decision processes. Technical report, Carnegie Mellon University, 2001.
5. C. Baier and J.-P. Katoen. Principles of Model Checking. MIT Press, 2008.
6. D. P. Bertsekas. Dynamic Programming and Optimal Control (Vol. I and II). Athena Scientific, 2001.
7. O. Buffet. Reachability analysis for uncertain ssps. In Proceedings of the IEEE International Conf. on Tools with Artificial Intelligence, 2005.
8. K. Chatterjee, K. Sen, and T. Henzinger. Model-checking omegaregular properties of interval Markov chains. In Foundations of Software Science and Computation Structure (FoSSaCS), pages 302-317, 2008.
9. C. Courcoubetis and M. Yannakakis. The complexity of probabilistic verification. Journal of the Association for Computing Machinery, 42:857-907, 1995.
10. L. de Alfaro. Formal Verification of Probabilistic Systems. PhD thesis, Stanford University, 1997.
11. X. C. Ding, S. L. Smith, C. Belta, and D. Rus. LTL control in uncertain environments with probabilistic satisfaction guarantees. In Proceedings of 18th IFAC World Congress, 2011.
12. R. Givan, S. Leach, and T. Dean. Bounded-parameter Markov decision processes. Artificial Intelligence, 122:71-109, 2000.
13. J. Klein and C. Baier. Experiments with deterministic omega-automata for formulas of linear temporal logic. Theoretical Computer Science, 363:182-195, 2006.
14. M. Kwiatkowska, G. Norman, and D. Parker. PRISM 4.0: verification of probabilistic real-time systems. In Proceedings of 23rd International Conference on Computer Aided Verification, 2011.
15. M. Lahijanian, S. B. Andersson, and C. Belta. Control of Markov decision processes from PCTL specifications. In Proceedings of the American Control Conference, 2011.
16. E. L. Lehmann and J. P. Romano. Testing Statistical Hypotheses. Springer, 2005.
17. R. Majumdar, E. Render, and P. Tabuada. Robust discrete synthesis against unspecified disturbances. In Proc. Hybrid Systems: Computation and Control, pages 211-220, 2011.
18. A. W. Naylor and G. R. Sell. Linear Operator Theory in Engineering and Science. Springer-Verlag, 1982.
19. A. Nilim and L. El Ghaoui. Robust control of Markov decision processes with uncertain transition matrices. Operations Research, 53:780-798, 2005.
20. J. K. Satia and R. E. L. Jr. Markovian decision processes with uncertain transition probabilities. Operations Research, 21(3):pp. 728-740, 1973.
21. D. Wu and X. Koutsoukos. Reachability analysis of uncertain systems using bounded-parameter Markov decision processes. Artif. Intell., 172:945-954, 2008.