Single sample path-based optimization of Markov chains

Journal of Optimization Theory and Applications

Volumn 100, Issue 3, 1999, Pages 527-548

  Cao, X R ^a  

^a HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

Author keywords

Markov decision processes; On line optimization; Performance potentials; Perturbation analysis

Indexed keywords

BEHAVIORAL RESEARCH; ITERATIVE METHODS;

ENGINEERING SYSTEMS; MARKOV DECISION PROCESSES; ONLINE OPTIMIZATION; OPTIMISATIONS; PATH-BASED; PERFORMANCE POTENTIALS; PERTURBATION ANALYSIS; REAL-WORLD; SAMPLE PATH; SINGLE SAMPLE;

MARKOV PROCESSES;

EID: 0033247533     PISSN: 00223239     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1022634422482     Document Type: Article

References (17)

1. 1. SURI, R., and LEUNG, Y. T., Single-Run Optimization of Discrete-Event Simulations: An Empirical Study Using the M/M/1 Queue, IIE Transactions, Vol. 21, pp. 35-49, 1989.
2. 2. CHONG, E. K. P., and RAMADGE, P. J., Stochastic Optimization of Regenerative Systems Using Infinitesimal Perturbation Analysis, IEEE Transactions on Automatic Control, Vol. 39, pp. 1400-1410, 1994.
3. 3. PLAMBECK, E. L., FU, B. R., ROBINSON, S. M., and SURI, R., Sample-Path Optimization of Convex Stochastic Performance Functions, Mathematical Programming, (to appear).
4. 4. CAO, X. R., and CHEN, H. F., Potentials, Perturbation Realization, and Sensitivity Analysis of Markov Processes, IEEE Transactions on Automatic Control, Vol. 42, pp. 1382-1393, 1997.
5. 5. ARAPOSTATHIS, A., BORKAR, V. S., FERNANDEZ-GAUCHERAND, E., GHOSH, M. H., and MARCUS, S. I., Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey, SIAM Journal on Control and Optimization, Vol. 31, pp. 282-344, 1993.
6. 6. BERTSEKAS, D. P., Dynamic Programming and Optimal Control, Vols. 1-2, Athena Scientific, Belmont, Massachusetts, 1995.
7. 7. BERTSEKAS, D. P., and TSITSIKLIS, T. N., Neurodynamic Programming, Athena Scientific, Belmont, Massachusetts, 1996.
8. 8. GLYNN, P. W., and L'ECUYER, P., Likelihood Ratio Gradient Estimation for Stochastic Recursions, Advances in Applied Probability, Vol. 27, pp. 1019-1053, 1995.
9. 9. MEYN, S. P., and TWEEDIE, R. L., Markov Chains and Stochastic Stability, Springer Verlag, London, England, 1993.
10. 10. MEYN, S. P., The Policy Improvement Algorithm for Markov Decision Processes with General State Space, IEEE Transactions on Automatic Control, (to appear).
11. 11. PUTERMAN, M. L., Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, New York, New York, 1994.
12. 12. ROSS, S. M., Introduction to Stochastic Dynamic Programming, Academic Press, Reading, Massachusetts, 1983.
13. 13. TSITSIKLIS, J. N., and VAN ROY, B., Feature-Based Methods for Large-Scale Dynamic Programming, Machine Learning, Vol. 22, pp. 59-94, 1996.
14. 14. ÇINLAR, E., Introduction to Stochastic Processes, Prentice Hall, Englewood Cliffs, New Jersey, 1975.
15. 15. CAO, X. R., and WAN, Y. W., Algorithms for Sensitivity Analysis of Markov Systems through Potentials and Perturbation Realization, IEEE Transactions on Control System Technology, Vol. 6, pp. 472-494, 1998.
16. 16. SCHWEITZER, P. J., Perturbation Theory and Finite Markov Chains, Journal of Applied Probability, Vol. 5, pp. 401-413, 1968.
17. 17. GOLUB, G. H., and MEYER, C. D., Using the QR Factorization and Group Inversion to Compute, Differentiate, and Estimate the Sensitivity of Stationary Probabilities for Markov Chains, SIAM Journal on Algorithms and Discrete Methods, Vol. 7, pp. 273-281, 1986.