Sample path optimality for a Markov optimization problem

Stochastic Processes and their Applications

Volumn 115, Issue 5, 2005, Pages 769-779

  Hunt, F Y ^a  

^a NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY   (United States)

Author keywords

Azuma's inequality; Markov decision process; Stochastic control

Indexed keywords

ASYMPTOTIC STABILITY; MATHEMATICAL MODELS; OPTIMIZATION; PROBABILITY; STOCHASTIC CONTROL SYSTEMS;

AZUMA INEQUALITY; MARKOV DECISION PROCESS;

MARKOV PROCESSES;

EID: 16244407393     PISSN: 03044149     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.spa.2004.12.005     Document Type: Article

References (15)

1. E. Altman Constrained Markov Decision Processes 1999 Chapman & Hall London
2. E. Altman A. Shwartz Markov decision problems and state-action frequencies SIAM J. Control Optim. 29 4 1991 786-809
3. A. Asriev V. Rotar On asymptotic optimality in probability and almost surely in dynamic control Stochastics Stochastic Rep. 33 1990 1-16
4. P. Bremaud Markov Chains, Gibbs Fields, Monte Carlo Simulation and Queues, Springer Texts in Applied Mathematics 1999 Springer New York
5. T. Belkina, V. Rotar, On optimality in probability and almost surely for controlled stochastic processes with a communication property, preprint available at http://www-rohan.sdsu.edu/~rotar/index.html, 2002.
6. P. Dai Pra B. Giovanni B. Trivellato Pathwise optimality in stochastic control SIAM J. Control Optim. 39 5 2001 1540-1557
7. J.L. Doob Stochastic Processes 1953 Wiley New York, London
8. P. Glynn D. Ormoneit Hoeffding's inequality for uniformly ergodic Markov chains Statist. Probab. Lett. 56 2002 143-146
9. S. Meyn R. Tweedie Markov Chains and Stochastic Stability 1993 Springer London
10. V. Rotar Probability Theory 1997 World Scientific Singapore
11. J.M. Steele, Probability Theory and Combinatorial Optimization, Society for Industrial and Applied Mathematics, Philadelphia, 1997.
12. J.B. Lasserre, Average optimal stationary policies and linear programming in countable space markov decision processes, J. Math. Anal. Appl. 183 (1994) 233-249.
13. A. Leizarowitz, On almost sure optimization for stochastic control systems, Stochastics 23 (1988) 85-107.
14. S. Ross, Applied Probability Models with Optimization Applications, Holden-Day, San Francisco 1970.
15. S. Ross, Probability Models, seventh ed, Harcourt Academic Press, San Diego, 2000.