Unified approach to Markov decision problems and performance sensitivity analysis

Automatica

Volumn 36, Issue 5, 2000, Pages 771-774

  Cao, Xi Ren ^a  

^a HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

Author keywords

[No Author keywords available]

Indexed keywords

LYAPUNOV METHODS; PERTURBATION TECHNIQUES; SENSITIVITY ANALYSIS; THEOREM PROVING;

MARKOV DECISION PROBLEMS (MDP);

MARKOV PROCESSES;

EID: 0033884215     PISSN: 00051098     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0005-1098(99)00207-1     Document Type: Article

References (13)

1. Bertsekas, D. P. (1995). Dynamic programming and optimal control, Vols. I, II. Belmont, MA: Athena Scientific.
2. Berman, A., & Plemmons, R. J. (1994). Nonnegative matrices in the mathematical sciences. Philadelphia: SIAM.
3. Cao, X. R. (1994). Realization Probabilities: The dynamics of queueing systems. New York: Springer.
4. Cao Xi-Ren. The relation among potentials, perturbation analysis, Markov decision processes, and other topics. Journal of Discrete Event Dynamic Systems. 8:1998;71-87.
5. Cao X.R., Chen H.F. Potentials, perturbation realization, and sensitivity analysis of Markov processes. IEEE Transactions on Automatic Control. 42:1997;1382-1393.
6. Dai L.Y., Ho Y.C. Structural infinitesimal perturbation analysis (SIPA) for derivative estimation of discrete event dynamic systems. IEEE Transactions on Automatic Control. 40:1995;1154-1166.
7. Glasserman, P. (1991). Gradient estimation via perturbation analysis. Boston: Kluwer.
8. Glynn P.W., Meyn S.P. A Lyapunov bound for solutions of Poisson's equation. Annals of Probability. 24:1996;916-931.
9. Ho, Y. C., & Cao, X. R. (1991). Perturbation analysis of discrete-event dynamic systems, Boston: Kluwer.
10. Kemeny, J. G., & Snell, J. L. (1990). Finite Markov chains. New York: Van Nostrand.
11. Makowski, A. M., & Shwartz, A. (1994). On the Poisson equation for countable Markov Chains: Existence of solutions and parameter dependence by probabilistic methods. Technical Report, T.R. 94-87, Institute for Systems Research, University of Maryland, College Park, MD 20742, USA.
12. Meyn, S. P., & Tweedie, R. L. (1993). Markov chains and stochastic stability. London: Springer.
13. Puterman, M. L. (1994). Markov decision processes: discrete stochastic dynamic programming. New York: Wiley.