Drift and monotonicity conditions for continuous-time controlled Markov chains with an average criterion

IEEE Transactions on Automatic Control

Volumn 48, Issue 2, 2003, Pages 236-245

  Guo, Xianping ^a,b   Hernández Lerma, Onésimo ^a  

^a DEPARTAMENTO DE FÍSICA   (Mexico)

^b SUN YAT SEN UNIVERSITY   (China)

Author keywords

Average (or ergodic) reward cost criterion; Continuous time controlled Markov chains (or continuous time Markov decision processes); Drift and monotonicity conditions; Optimal stationary policy; Unbounded transition and reward cost rates

Indexed keywords

ITERATIVE METHODS; MATHEMATICAL MODELS; OPTIMAL CONTROL SYSTEMS; QUEUEING NETWORKS; STOCHASTIC CONTROL SYSTEMS;

CONTINUOUS-TIME CONTROLLED MARKOV CHAINS;

MARKOV PROCESSES;

EID: 0037291699     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2002.808469     Document Type: Article

References (23)

1. W. J. Anderson, Continuous-Time Markov Chains. New York: Springer-Verlag, 1991.
2. X. R. Cao, "The relations among potentials, perturbation analysis, and Markov decision processes," Discrete Event Dyna. Syst.: Theory Appl., vol. 8, pp. 71-87, 1998.
3. K. L. Chung, Markov Chains With Stationary Transition Probabilities. Berlin, Germany: Springer-Verlag, 1960.
4. M. H. A. Davis, Markov Models and Optimization. London, U.K.: Chapman and Hall, 1993.
5. E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes. New York: Springer-Verlag, 1979.
6. W. Feller, "On the integro-differential equations of purely discontinuous Markoff processes," Trans. Amer. Math. Soc., vol. 48, pp. 488-515, 1940.
7. L. Fisher, "On the recurrent denumerable decision process," Ann. Math. Statist., vol. 39, pp. 424-434, 1968.
8. X. P. Guo and O. Hernández-Lerma, Continuous-time controlled Markov chains with discounted rewards, submitted for publication.
9. X. P. Guo and W. P. Zhu, "Denumerable state continuous-time Markov decision processes with unbounded cost and transition rates under average criterion," ANZIAM J., pt. 541, p. 557, 2002.
10. _, "Optimality conditions for continuous-time Markov decision processes with average cost criterion," in Markov Processes and Controlled Markov Chains, Z. T. Hou, J. A. Filar, and A. Y. Chan, Eds. Dordrecht, The Netherlands: Kluwer, 2001.
11. X. P. Guo and K. Liu, "A note on optimality conditions for continuous-time Markov decision processes with average cost criterion," IEEE Trans. Automat. Contr., vol. 46, pp. 1984-1988, Dec. 2001.
12. O. Hernández-Lerma and J. B. Lasserre, Further Topics on Discrate-Time Markov Control Processes. New York: Springer-Verlag, 1999.
13. O. Hernández-Lerma, Lectures on Continuous-Time Markov Control Processes. Mexico City, Mexico: Sociedad Matematica Mexicana, 1994, vol. 3.
14. Z. T. Hou and X. P. Guo, Markov Decision Processes (in Chinese). Changsha, China: Science and Technology Press, 1998.
15. P. Kakumanu, "Nondiscounted continuous-time Markov decision processes with countable state space," SIAM J. Control, vol. 10, pp. 210-220, 1972.
16. _, "Continuous time Markov decision processes with average return criterion," J. Math. Anal. Appl., vol. 52, pp. 173-188, 1975.
17. R. B. Lund, S. P. Meyn, and R. L. Tweedie, "Computable exponential convergence rates for stochastically ordered Markov proceses," Ann. Appl. Prob., vol. 6, pp. 218-237, 1996.
18. S. P. Meyn and R. L. Tweedie, "Stability of Markovian processes III: Foster-Lyapunov criteria for continuous-time processes," Adv. Appl. Prob., vol. 25, pp. 518-548, 1993.
19. M. L. Puterman, Markov Decision Processes. New York: Wiley, 1994.
20. L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems. New York: Wiley, 1999.
21. J. S. Song, "Continuous-time Markov decision programming with nonuniformly bounded transition rates," Sci. Sin., vol. 12, pp. 1258-1267, 1987.
22. A. A. Yushkevich and E. A. Feinberg, "On homogeneous Markov model with continuous-time and finite or countable state space," Theory Prob. Appl., vol. 24, pp. 156-161, 1979.
23. S. H. Zheng, "Continuous-time Markov decision programming with average reward criterion and unbounded reward rate," Acta Math. Appl. Sinica, vol. 7, pp. 6-16, 1991.