Reinforcement learning for long-run average cost

European Journal of Operational Research

Volumn 155, Issue 3, 2004, Pages 654-674

  Gosavi, Abhijit ^a  

^a UNIVERSITY AT BUFFALO   (United States)

Author keywords

Reinforcement learning; Stochastic processes; Two time scales

Indexed keywords

COMPUTER SIMULATION; COSTS; DYNAMIC PROGRAMMING; LEARNING SYSTEMS; MARKOV PROCESSES; PROBABILITY; QUALITY CONTROL; RANDOM PROCESSES; REINFORCEMENT;

REINFORCEMENT LEARNING; TWO TIME SCALES;

DECISION MAKING;

EID: 0742319170     PISSN: 03772217     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0377-2217(02)00874-3     Document Type: Conference Paper

References (35)

1. J. Abounadi, D. Bertsekas, V. Borkar, Ode analysis for Q-learning algorithms, LIDS Report, MIT, Cambridge, MA, 1996.
2. Bellman R. The theory of dynamic programming. Bulletin of American Mathematical Society. 60:1954;503-516.
3. Bertsekas D.P. Dynamic Programming and Optimal Control. 1995;Athena Scientific, Belmont, MA.
4. Bertsekas D.P., Tsitsiklis J.N. Neuro-Dynamic Programming. 1996;Athena Scientific, Belmont, MA.
5. Borkar V.S. Stochastic approximation with two-time scales. System and Control Letters. 29:1997;291-294.
6. Borkar V.S. Asynchronous stochastic approximation. SIAM Journal on Control and Optimization. 36(3):1998;840-851.
7. V.S. Borkar, S.P. Meyn, The ode method for convergence of stochastic approximation and reinforcement learning, Working paper.
8. Borkar V.S., Soumyanath K. An analog scheme for fixed point computation, Part i: Theory. IEEE Transactions Circuits and Systems I. Fundamental Theory and Application. 44:1997;351-354.
9. Cassandras C.G., Han Y. Optimal inspection policies for a manufacturing station. European Journal of Operational Research. 63:1992;35-53.
10. Das T.K., Gosavi A., Mahadevan S., Marchalleck N. Solving semi-Markov decision problems using average reward reinforcement learning. Management Science. 45(4):1999;560-574.
11. Das T.K., Sarkar S. Optimal preventive maintenance in a production inventory system. IIE Transactions on Quality and Reliability. 31:1999;537-551.
12. M. Elhafsi, S. Bai, Optimal and near-optimal control of a two-part stochastic manufacturing system with dynamic setups, Research Report 95-10, at the Department of Industrial Engineering, University of Florida, Gainesville, 1997.
13. A. Gosavi, An algorithm for solving semi-Markov decision problems using reinforcement learning: Convergence analysis and numerical results, Unpublished Ph.D. Dissertation, Department of Industrial and Management Systems Engineering, University of South Florida, Tampa, FL, May 1999.
14. A. Gosavi, On the convergence of some reinforcement learning algorithms. Working paper, Department of Engineering, University of Southern Colorado, Pueblo, 2000.
15. Gosavi A., Bandla N., Das T.K. Airline seat allocation among multiple fare classes with overbooking. IIE Transactions. 34(9):2002;729-742.
16. A. Gosavi, T.K. Das, S. Sarkar, A simulation-based learning automata framework for solving semi-Markov decision problems under long-run average reward. IIE Transactions (in press).
17. V.R. Konda, V.S. Borkar, Actor-critic type learning algorithms for Markov decision processes, Working paper, Indian Institute of Sciences, Bangalore, India.
18. Kushner H.J., Clark D.S. Stochastic Approximation Methods for Constrained and Unconstrained Systems. 1978;Springer-Verlag, Berlin.
19. Lewis E.E. Introduction to Reliability Engineering. 1994;John Wiley and Sons, New York.
20. Ljung L. Analysis of recursive stochastic algorithms. IEEE Transactions on Automatic Control. 22:1977;551-575.
21. Mahadevan S. Average reward reinforcement learning: Foundations, algorithms, and empirical results. Machine Learning. 22(1):1996;159-195.
22. M.L. Littman, Algorithms for sequential decision-making, Unpublished Ph.D. Thesis, Brown University, Providence, RI, 1996.
23. Narendra K., Thatachar M.A.L. Learning Automata: An Introduction. 1989;Prentice Hall, Englewood Cliffs, NJ.
24. Puterman M.L. Markov Decision Processes. 1994;Wiley Interscience, New York.
25. Robbins H., Monro S. A stochastic approximation method. Annals Mathematical Statistics. 22:1951;400-407.
26. A. Schwartz. A reinforcement learning method for maximizing undiscounted rewards, in: Proceedings of the Tenth Annual Conference on Machine Learning, 1993, pp. 298-305.
27. Seidmann A., Schweitzer P.J. Part selection policy for a flexible manufacturing cell feeding several production lines. IIE Transactions. 16(4):1984;355-362.
28. Sennott L. Stochastic Dynamic Programming and the Control of Queueing Systems. 1999;John Wiley and Sons, New York.
29. Shioyama T. Optimal control of a queuing network system with two types of customers. European Journal of Operational Research. 52:1991;367-372.
30. Singh S. Reinforcement learning algorithms for average-payoff Markovian decision processes. Proceedings of the 12th AAAI. 1994;MIT Press, Cambridge, MA.
31. Sutton R. Reinforcement Learning. Machine Learning Journal. 8(3):1992;5. (special issue).
32. Button R., Barto A.G. Reinforcement Learning. 1998;The MIT Press, Cambridge, MA.
33. Tadepalli P., Ok D. Scaling up average reward reinforcement learning by approximating the domain models and the value function. Proceedings of the Thirteenth International Machine Learning Conference. 1996;471-479 Morgan Kaufmann, New York.
34. Tsitsiklis J. Asynchronous stochastic approximation and q-learning. Machine Learning. 16:1994;185-202.
35. C.J. Watkins, Learning from Delayed Rewards, Ph.D. Thesis, Kings College, Cambridge, England, May 1989.