A policy gradient method for semi-Markov decision processes with application to call admission control

European Journal of Operational Research

Volumn 178, Issue 3, 2007, Pages 808-818

  Singh, Sumeetpal S ^a   Tadić, Vladislav B ^b   Doucet, Arnaud ^c  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^b UNIVERSITY OF SHEFFIELD   (United Kingdom)

^c UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Call admission control; Policy gradient; Semi Markov decision process; Stochastic processes; Two time scale

Indexed keywords

APPROXIMATION THEORY; DECISION MAKING; MARKOV PROCESSES; OPTIMIZATION;

CALL ADMISSION CONTROL; SEMI-MARKOV DECISION PROCESS; TWO TIME-SCALE;

OPERATIONS RESEARCH;

EID: 33751077547     PISSN: 03772217     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ejor.2006.02.023     Document Type: Article

References (21)

1. Adelman D. Price-directed replenishment of subsets: Methodology and its application to inventory routing. Manufacturing and Service Operations Management 5 4 (2003) 348-371
2. Baxter J., and Bartlett P.L. Infinite-horizon policy-gradient estimation. Journal of Artificial Intelligence Research 15 (2001) 319-350
3. Bertsekas D.P. Dynamic Programming and Optimal Control vol. 2 (1995), Athena Scientific, Belmont
4. Bertsekas D.P., and Tsitsiklis J.N. Gradient convergence in gradient methods with errors. SIAM Journal on Optimization 10 3 (2000) 627-642
5. Bertsekas D.P., and Tsitsiklis J.N. Neuro-dynamic Programming (1996), Athena Scientific, Belmont
6. Bradtke S.J., and Duff M.O. Reinforcement learning methods for continuous-time Markov decision problems. Advances in Neural Information Processing Systems 7 (1995)
7. Cao X.-R. Semi-Markov decision problems and performance sensitivity analysis. IEEE Transactions on Automatic Control 48 5 (2003) 758-769
8. Cinlar E. Introduction to Stochastic Processes (1974), Prentice-Hall, Englewood Cliffs, NJ
9. Comaniciu C., Mandayam N.B., and Poor H.V. Wireless Networks: Multiuser Detection in Cross-Layer Design (2005), Springer, New York
10. Gosavi A. Reinforcement learning for long-run average cost. European Journal of Operational Research 155 (2004) 654-674
11. Marbach P., Mihatsch O., and Tsitsiklis J.N. Call admission control and routing in integrated services networks using neuro-dynamic programming. IEEE JSAC 18 2 (2000) 197-208
12. Meyn S.P., and Tweedie R.L. Markov Chains and Stochastic Stability (1993), Springer-Verlag, London
13. Ross K.W. Multiservice Loss Models for Broadband Telecommunication Networks (1995), Springer-Verlag, London
14. Singh S., Krishnamurthy V., and Poor H.V. Integrated voice/data call admission control for wireless DS-CDMA systems with fading. IEEE Transactions on Signal Processing 50 6 (2002) 1483-1495
15. Suton R.S., McAllester D., Singh S., and Mansour Y. Policy gradient methods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems 12 (2000) 1057-1063
16. Sutton R.S., Precup D., and Singh S.P. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112 (1999) 181-211
17. Tadić V. On the convergence of temporal-difference learning with linear function approximation. Machine Learning 42 3 (2001) 241-267
18. Tijms H.C. Stochastic Models: An Algorithmic Approach (1994), John Wiley & Sons, Chichester
19. V. Tadić, A. Doucet, Two time-scale stochastic approximation for constrained stochastic optimization and constrained Markov decision problems, in: Proceedings of ACC, 2003.
20. V. Tadić, S.P. Meyn, Asymptotic properties of two time-scale stochastic approximation algorithms with constant step sizes, in: Proceedings of ACC, 2003.
21. Intelligent Techniques in High Speed Networks. IEEE Journal on Selected Areas in Communications 18 2 (2000). (special issue). Available from: