Reachability analysis of uncertain systems using bounded-parameter Markov decision processes

Artificial Intelligence

Volumn 172, Issue 8-9, 2008, Pages 945-954

  Wu, Di ^a   Koutsoukos, Xenofon ^a  

^a VANDERBILT UNIVERSITY   (United States)

Author keywords

Markov decision processes; Reachability analysis; Uncertain systems

Indexed keywords

DECISION MAKING; MARKOV PROCESSES; PARAMETER ESTIMATION; PROBLEM SOLVING; VERIFICATION;

MARKOV DECISION PROCESSES; REACHABILITY ANALYSIS;

UNCERTAIN SYSTEMS;

EID: 41049112648     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.artint.2007.12.002     Document Type: Article

References (24)

1. H.L.S. Younes, D.J. Musliner, Probabilistic plan verification through acceptance sampling, in: Proceedings of the AIPS-02 Workshop on Planning via Model Checking, 2002, pp. 81-88
2. J. Heath, M. Kwiatkowska, G. Norman, D. Parker, O. Tymchyshun, Probabilistic model checking of complex biological pathways, in: Proc. International Conference on Computational Methods in Systems Biology, 2006
3. Pham H. Minimizing shortfall risk and applications to finance and insurance problems. The Annals of Applied Probability 312 1 (2002) 143-172
4. Courcoubetis C., and Yannakakis M. Markov decision processes and regular events. IEEE Transaction on Automatic Control 43 10 (1998) 1399-1418
5. L. de Alfaro, Formal verification of probabilistic systems, PhD thesis, Tech. Rep. STAN-CS-TR-98-1601, Stanford University, Stanford, CA, 1997
6. Rutten J., Kwiatkowska M., Norman G., and Parker D. Mathematical Techniques for Analyzing Concurrent and Probabilistic Systems. CRM Monograph Series vol. 23 (2004), American Mathematical Society
7. Jaeger M. Probabilistic decision graphs-combining verification and AI techniques for probabilistic inference. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 12 (2004) 19-42
8. Kushner H.J., and Dupuis P. Numerical Methods for Stochastic Control Problems in Continuous Time. second ed. (2001), Springer-Verlag, New York
9. Feinberg E., and Shwartz A. Handbook of Markov Decision Processes: Methods and Applications (2002), Kluwer Academic Publishers, Boston, MA
10. A. Nilim, L.E. Ghaoui, Robust control of Markov decision processes with uncertain transition matrices, Operations Research 53 (5)
11. Wu D., and Koutsoukos X.D. Probabilistic verification of bounded-parameter Markov decision processes. In: Torra V., Narukawa Y., and Miyamoto S. (Eds). Modeling Decisions for Artificial Intelligence, MDAI 2006. Tarragona, Catalonia, Spain, April 3-5, 2006. Lecture Notes in Artificial Intelligence vol. 3885 (2006), Springer 283-294
12. Givan R., Leach S., and Dean T. Bounded-parameter Markov decision process. Artificial Intelligence 122 1-2 (2000) 71-109
13. Puterman M.L. Markov Decision Processes: Discrete Stochastic Dynamic Programming (1994), John Wiley & Sons, Inc., New York
14. Satia J.K., and Lave R.E. Markovian decision processes with uncertain transition probabilities. Operations Research 39 (1953) 1095-1100
15. White C.C., and Eldeib H.K. Markov decision processes with imprecise transition probabilities. Operations Research 43 (1994) 739-749
16. J.A. Bagnell, A.Y. Ng, J.G. Schneider, Solving uncertain Markov decision problems, Tech. Rep. CMU-RI-TR-01-25, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, August 2001
17. Kurano M., Yasuda M., and Nakagami J. Interval methods for uncertain Markov decision processes. Markov Processes and Controlled Markov Chains (2002) 223-232
18. Tewari A., and Bartlett P.L. Bounded parameter Markov decision processes with average reward criterion. Proceedings of the 20th Annual Conference on Learning Theory. Lecture Notes in Computer Science vol. 4539 (2007), Springer 263-277
19. S. Kalyanasundaram, E.K.P. Chong, N.B. Shroff, Markovian decision processes with uncertain transition rates: Sensitivity and robust control, in: Proc. 41th IEEE Conference on Decision and Control, 2002
20. O. Buffet, Reachability analysis for uncertain ssps, in: ICTAI '05: Proceedings of the 17th IEEE International Conference on Tools with Artificial Intelligence, 2005, pp. 515-522
21. I.O. Kozine, L.V. Utkin, Interval-valued finite Markov chains, Reliable Computing 8 (2)
22. Sen K., Viswanathan M., and Agha G. Model-checking Markov chains in the presence of uncertainties. TACAS. Lecture Notes in Computer Science vol. 3920 (2006), Springer 394-410
23. Bertsekas D.P., and Tsitsiklis J.N. Parallel and Distributed Computation: Numerical Methods (1989), Prentice-Hall, Inc., Upper Saddle River, NJ
24. *. Operations Research Letters 9 5 (1990) 287-297