Maximizing the probability of attaining a target prior to extinction

Nonlinear Analysis: Hybrid Systems

Volumn 5, Issue 2, 2011, Pages 367-381

  Chatterjee, Debasish ^a   Cinquemani, Eugenio ^b   Lygeros, John ^a  

^a ETH ZURICH   (Switzerland)

^b INRIA RHÔNE ALPES   (France)

Author keywords

Dynamic programming; Markov control processes; Probability maximization

Indexed keywords

DISCRETE-TIME MARKOV CONTROL PROCESS; FIRST HITTING TIME; MARKOV CONTROL PROCESSES; MARTINGALE CHARACTERIZATION; MAXIMUM VALUES; OPTIMAL POLICIES; STATIONARY POLICY;

OPTIMIZATION; PROBABILITY; PROCESS CONTROL; PUBLIC WORKS;

DYNAMIC PROGRAMMING;

EID: 79954419208     PISSN: 1751570X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.nahs.2010.12.003     Document Type: Article

References (37)

1. Dynkin E.B. Optimum choice of the stopping moment of a Markov process. Doklady Academii Nauk SSR 1963, 150:238-240.
2. Peskir G., Shiryaev A.N. Optimal Stopping and Free-Boundary Problems. Lectures in Mathematics ETH Zürich 2006, Birkhäuser Verlag, Basel.
3. Dubins L.E., Savage L.J. Inequalities for Stochastic Processes (How to Gamble if You Must) 1976, Dover Publications Inc., New York. corrected republication of the 1965 edition.
4. Bouakiz M., Kebir Y. Target-level criterion in markov decision processes. Journal of Optimization Theory and Applications 1995, 86(1):1-15.
5. Boda K., Filar J.A., Lin Y., Spanjers L. Stochastic target hitting time and the problem of early retirement. IEEE Transactions on Automatic Control 2004, 49(3):409-419.
6. Schmidli H. Stochastic Control in Insurance. Probability and its Applications 2008, Springer-Verlag London Ltd., London.
7. Simon H.A. Models of man, social and rational. Mathematical Essays on Rational Human Behavior in a Social Setting 1957, John Wiley & Sons Inc., New York.
8. Ohtsubo Y. Value iteration methods in risk minimizing stopping problems. Journal of Computational and Applied Mathematics 2003, 152(1-2):427-439.
9. Digaǐlova I.A., Kurzhanskiǐ A.B. The attainability problem under stochastic perturbations. Differentsial'nye Uravneniya 2004, 40(11):1494-1499. 1582.
10. Borkar V.S. Topics in Controlled Markov Chains. Pitman Research Notes in Mathematics Series 1991, vol. 240. Longman Scientific & Technical, Harlow.
11. Gao Y., Lygeros J., Quincampoix M. On the reachability problem for uncertain hybrid systems. IEEE Transactions on Automatic Control 2007, 52(9):1572-1586.
12. Tomlin C.J., Lygeros J., Sastry S. A game theoretic approach to controller design for hybrid systems. Proceedings of IEEE 2000, 88:949-969.
13. O. Watkins, J. Lygeros, Stochastic reachability for discrete-time systems: an application to aircraft collision avoidance. In: 42nd IEEE Conference on Decision and Control. vol. 5. 2003, pp. 5314-5319.
14. Prandini M., Hu J. A stochastic approximation method for reachability computations. Lecture Notes in Control and Information Sciences. 2006, vol. 337:107-139. Springer, Berlin.
15. Abate A., Prandini M., Lygeros J., Sastry S. Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems. Automatica 2008, 44(11):2724-2734.
16. Prajna S., Jadbabaie A., Pappas G.J. A framework for worst-case and stochastic safety verification using barrier certificates. IEEE Transactions on Automatic Control 2007, 52(8):1415-1428.
17. Summers S., Lygeros J. Probabilistic reach-avoid problem. IFAC Conference on Analysis and Design of Hybrid Systems 2009, Zaragoza, Spain.
18. Hernández-Lerma O., Lasserre J.B. Further Topics on Discrete-Time Markov Control Processes. Applications of Mathematics 1999, vol. 42. Springer-Verlag, New York.
19. Eaton J.H., Zadeh L.A. Optimal pursuit strategies in discrete-state probabilistic systems. Transactions of the ASME Series D., Journal of Basic Engineering 1962, 84:23-29.
20. Whittle P. Optimization Over Time. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics 1983, vol. II. John Wiley & Sons Ltd., Chichester.
21. Derman C. Finite State Markovian Decision Processes. Mathematics in Science and Engineering 1970, vol. 67. Academic Press, New York.
22. Kushner H. Introduction to Stochastic Control 1971, Holt, Rinehart and Winston, Inc., New York.
23. Bertsekas D.P. Dynamic Programming and Optimal Control 2007, vol. 2. Athena Scientific. 3rd ed.
24. Borkar V.S. A convex analytic approach to Markov decision processes. Probabability Theory and Related Fields 1988, 78(4):583-602.
25. Kesten H., Spitzer F. Controlled Markov chains. Annals of Probability 1975, 3:32-40.
26. I. Karatzas, W. Sudderth, Two characterizations of optimality in dynamic programming. Applied Mathematics & Optimization. 2009. http://www.springerlink.com/content/340m82862p817446/?p%3Df54a99eb8ef3432cb45724c3d5ee8baa%26pi%3D0.
27. Zhu Q., Guo X. A semimartingale characterization of average optimal stationary policies for Markov decision processes. Journal of Applied Mathematics and Stochastic Analysis, Art. ID 81593 2006, http://www.hindawi.com/GetArticle.aspx?doi=10.1155/JAMSA/2006/81593.
28. Chatterjee D., Cinquemani E., Chaloulos G., Lygeros J. Stochastic control up to a hitting time: optimality and rolling-horizon implementation 2008, arxiv:0806.3008.
29. Hernández-Lerma O., Lasserre J.B. Discrete-Time Markov Control Processes: Basic Optimality Criteria. Applications of Mathematics 1996, vol. 30. Springer-Verlag, New York.
30. Meyn S.P. Control Techniques for Complex Networks 2008, Cambridge University Press, Cambridge.
31. Aliprantis C., Border K.C. Infinite Dimensional Analysis: A Hitchhiker's Guide 2006, Springer-Verlag, Berlin. 3rd ed.
32. Rao M.M., Swift R.J. Probability Theory with Applications. Mathematics and its Applications 2006, vol. 582. Springer-Verlag. 2nd ed.
33. Levin D.A., Peres Y., Wilmer E.L. Markov Chains and Mixing Times 2009, American Mathematical Society, USA, with an Appendix written by James G. Propp and David B. Wilson.
34. Bertsekas D., Tsitsiklis J. Neuro-Dynamic Programming 1996, Athena Scientific.
35. Powell W.B. Approximate Dynamic Programming. Wiley Series in Probability and Statistics 2007, Wiley-Interscience [John Wiley & Sons], Hoboken, NJ.
36. Kwiatkowska M., Norman G., Parker D. Stochastic Model Checking. Lecture Notes in Comuter Science 2007, vol. 4486. Springer-Verlag.
37. Ramponi F., Chatterjee D., Summers S., Lygeros J. On the connections between PCTL and dynamic programming. HSCC (Hybrid Systems: Computation and Control) 2010, Stockholm, Sweden.