Overtaking and almost-sure optimality for infinite horizon Markov decision processes

Mathematics of Operations Research

Volumn 21, Issue 1, 1996, Pages 158-181

  Leizarowitz, Arie ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Almost sure optimality; Controlled Markov chains; Infinite horizon optimal control; Infinite horizon optimality equation; Overtaking optimality

Indexed keywords

DECISION THEORY; DISCRETE TIME CONTROL SYSTEMS; EIGENVALUES AND EIGENFUNCTIONS; MARKOV PROCESSES; OPTIMAL CONTROL SYSTEMS; STATE SPACE METHODS;

ALMOST SURE OPTIMALITY; BELLMAN EQUATION; FINITE STATE SPACES; INFINITE HORIZON OPTIMAL CONTROL; MARKOV DECISION PROCESSES; OVERTAKING OPTIMALITY;

OPTIMIZATION;

EID: 0030087007     PISSN: 0364765X     EISSN: None     Source Type: Journal    
DOI: 10.1287/moor.21.1.158     Document Type: Article

References (39)

1. Asriev, A. V., V. I. Rotar' (1990). On asymptotic optimality in probability and almost surely in dynamic control Stochastics Stochastics Rep. 33 1-16.
2. Bather, J. (1973). Optimal decision procedures for finite Markov chains, Parts I-III, Adv. Appl. Probab. 5 328-339, 521-553.
3. Bertsekas, D. P., S. F. Shreve (1978). Stochastic Optimal Control, Academic Press, New York.
4. Bhattacharya, R. N., M. Majumdar (1989). Controlled semi-Markov models under long-run average rewards, J. Statist. Plann. Inference 22 223-242.
5. Blackwell, D. (1962). Discrete dynamic programming, Ann. Math. Statist. 33 719-726.
6. _ (1967). Positive dynamic programming. Proc. 5th Berkeley Symp. Mathematical Statistics and Probability 1 415-418.
7. Brock, W. A., A. Haurie (1976). On existence of overtaking optimal trajectories over an infinite time horizon, Math. Oper. Res. 1 337-346.
8. Carlson, D. A. (1986). On the existence of catching up optimal solutions for Lagrange problems defined on unbounded intervals, J. Optim. Theory Appl. 49 1-19.
9. Denardo, E. V. (1967). Contraction mappings in the theory underlying dynamic programming, SIAM Rev. 9 165-177.
10. _, B. L. Miller (1968). An optimality condition for discrete dynamical programming with no discounting, Ann. Math. Statist. 39 1220-1227.
11. _, U. G. Rothblum (1979). Overtaking optimality for Markov Decisions Chains, Math. Oper. Res. 4 144-152.
12. Doob, J. L. (1967). Stochastic Processes, John Wiley and Sons, Inc., New York, London.
13. Dynkin, E. B., A. A. Yushkevich (1979). Controlled Markov Processes, Springer Verlag, New York.
14. Fainberg, E. A. (1975). On controlled finite state Markov processes with compact control sets (English translation), Theory Probab. Appl. 20 856-862.
15. Flynn, J. (1980). On optimality criteria for dynamic programs with long finite horizons, J. Math. Anal. Appl. 76 202-208.
16. _ (1989). Optimal steady states, excessive functions and deterministic dynamic programming, J. Math. Anal. Appl. 144 586-594.
17. Gale, D. (1967). On optimal development in a multisector economy, Rev. Econom. Stud. 34 1-19.
18. Gubenko, L. G., E. S. Štatland (1975). On controlled discrete-time Markov Decision Processes (English translation), Theory Probab. Math. Statist. 7 47-61.
19. Hall, P., C. C. Heyde (1980). Martingale Limit Theory and Its Applications, Academic Press, New York, London.
20. Haurie, A. (1976). Optimal control on an infinite horizon: The turnpike approach, J. Math. Econom. 3 81-102.
21. Hernández-Lerma, O. (1991). Average optimality in dynamic programming on Borel spaces-unbounded costs and controls, Systems Control Lett. 17 232-242.
22. _, J. C. Hennet, J. B. Lasserre (1991). Average cost Markov Decision Processes: Optimality conditions, J. Math. Anal. Appl. 158 396-406.
23. Hinderer, K. (1970). Foundations of Non-Stationary Dynamic Programming with Discrete-Time Parameter, Lecture Notes in Operations Research 33, Springer-Verlag, New York.
24. Hordijk, A. (1974). Dynamic programming and Markov potential theory, Math. Centre Tracts 51, Amsterdam.
25. Ikeda, N., S. Watanabe (1989). Stochastic Differential Equations and Diffusion Processes, 2nd ed., North-Holland, Amsterdam.
26. Kuratowski, K. (1968). Topology. Vol. II, Academic Press, New York.
27. Kushner, H. (1971). Introduction to Stochastic Control, Holt, Rinehart and Winston Inc., New York.
28. Leizarowitz, A. (1987). Infinite horizon optimization for finite state Markov chain, SIAM J. Control Optim. 25 1601-1618.
29. _ (1992). On infinite products of stochastic matrices, Linear Algebra Appl. 168 189-219.
30. Lippman, S. A. (1971). Maximal average-reward policies for semi-Markov decision processes with arbitrary state and action space, Ann. Math. Statist. 42 1717-1726.
31. Puterman, M. L. (1990). Markov Decision Processes, Chapter 8 Handbooks in OR & MS vol. 2, Eds. D. P. Heyman & M. J. Sobel, Elsevier Science Publications B. V. (North-Holland).
32. Ross, S. M. (1968). Arbitrary state Markovian Decision Processes, Ann. Math. Statist. 29 2118-2122.
33. _ (1983). Introduction to Stochastic Dynamic Programming, Academic Press, New York.
34. Schäl, M. (1975). Conditions for optimality in dynamic programming and for the limit of n-stage optimal policy to be optimal, Z. Wahrch. verw. Gebiete 32 179-196.
35. _ (1990). Averaging optimality in dynamic programming with general state space (Preprint).
36. Schweitzer, P. J., A. Federgrüen (1977). The asymptotic behaviour of undiscounted value iteration in Markov Decision Programs, Math. Oper. Res. 2 360-381.
37. Strauch, R. E. (1966). Negative dynamic programming, Ann. Math. Statist. 37 871-890.
38. Veinott, Jr., A. F. (1966). On finding optimal policies in discrete dynamic programming with no discounting, Ann. Math. Statist. 37 1284-294.
39. von Weizsäcker, C. C. (1968). Existence of optimal programs of accumulation for an infinite time horizon, Rev. Econom. Stud. 32 85-164.