Implementation of Gradient Estimation to a Constrained Markov Decision Problem

Proceedings of the IEEE Conference on Decision and Control

Volumn 5, Issue , 2003, Pages 4841-4846

  Krishnamurthy, Vikram ^a,b   Martin, Katerine ^c   Abad, Felisa Vázquez ^b,c  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^b UNIVERSITY OF MELBOURNE   (Australia)

^c UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

CONSTRAINT THEORY; DECISION THEORY; MARKOV PROCESSES; MATHEMATICAL MODELS; OPTIMIZATION; PERTURBATION TECHNIQUES; PROBABILITY; PROBLEM SOLVING;

GRADIENT ESTIMATION; MARKOV DECISION PROBLEM;

CONTROL SYSTEM ANALYSIS;

EID: 1542350203     PISSN: 07431546     EISSN: 25762370     Source Type: Conference Proceeding    
DOI: 10.1109/CDC.2003.1272362     Document Type: Conference Paper

References (10)

1. D. Bertsekas. Nonlinear Programming. Athena Scientific, Belmont, MA., 2000.
2. X. R. Cao. The relations amongst potentials, perturbation analysis and Markov decision processes. J DEDS, 8:71-87, 1998.
3. H.J. Kushner and G. Yin. Stochastic Approximation Algorithms and Applications. Springer-Verlag, New York, 1997.
4. M. Puterman. Markov Decision Processes. John Wiley, 1994.
5. F.J. Vázquez-Abad. and V. Krishnamurthy (2003) "Self Learning Control of Constrained Markov Decision Processes - A Gradient Approach", submitted to IEEE Trans on Autom Control.
6. E. Altman. Constrained Markov Decision Processes. Stochastic Modelling. Chapman and Hall, London, 1999.
7. L. Dai. Perturbation analysis via coupling. IEEE Trans. Auto. Control, 41(4):614-628, 2000.
8. B. Heidergott and F.J. Vázquez-Abad. Measure valued differentiation for stochastic processes: The finite horizon case, preprint.
9. V. Krishnamurthy and A. Logothetis. Iterative and recursive estimators for hidden Markov errors-in-variables models. IEEE Transactions on Signal Processing, 44(3):629-639, 1996.
10. K.W. Ross and R. Varadarajan. Markov decision processes with sample path constraints: The communication case. Operations Research, 37(5):780-790, Sept-Oct 1989.