On step sizes, stochastic shortest paths, and survival probabilities in reinforcement learning

Proceedings - Winter Simulation Conference

Volumn , Issue , 2008, Pages 525-531

  Gosavi, Abhijit ^a  

^a Architectural and Environmental Engineering   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AVERAGE REWARDS; DOWNSIDE RISKS; EMPIRICAL STUDIES; FUNCTION APPROXIMATIONS; LEARNING RULES; MARKOV DECISION PROCESS; NUMBER OF STATE; SIMULATION-BASED; STEP SIZES; STOCHASTIC SHORTEST PATHS; SURVIVAL PROBABILITIES; TRANSITION PROBABILITIES;

CONVERGENCE OF NUMERICAL METHODS; RANDOM PROCESSES; REINFORCEMENT; REINFORCEMENT LEARNING;

LEARNING ALGORITHMS;

EID: 60749124483     PISSN: 08917736     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/WSC.2008.4736109     Document Type: Conference Paper

References (20)

1. Bertsekas, D. 1995. Dynamic programming and optimal control. Belmont: Athena.
2. Borkar, V. 2002. Q-learning for risk-sensitive control. Mathematics of Operations Research 27(2):294-311.
3. Borkar, V. S. 1998. Asynchronous stochastic approximation. SIAM Journal of Control and Optimization 36 No 3:840-851.
4. Borkar, V. S., and S. Meyn. 2000. The ODE method for convergence of stochastic approximation and reinforcement learning. SIAM Journal of Control and Optimization 38 (2):447-469.
5. Even-Dar, E., and Y. Mansour. 2003. Learning rates for Q- learning. Journal of Machine Learning Research 5:1-25.
6. Filar, J., L. Kallenberg, and H. Lee. 1989. Variance-penalized Markov decision processes. Mathematics of Operations Research 14(1):147-161.
7. Geibel, P., and F. Wysotzki. 2005. Risk-sensitive reinforcement learning applied to control under constraints. Journal of Artificial Intelligence Research 24:81-108.
8. Gosavi, A. 2004a. A reinforcement learning algorithm based on policy iteration for average reward: Empirical results with yield management and convergence analysis. Machine Learning 55(l):5-29.
9. Gosavi, A. 2004b. Reinforcement learning for long-run average cost. European Journal of Operational Research 155:654-674.
10. Gosavi, A. 2006. A risk-sensitive approach to total productive maintenance. Automatica 42:1321-1330.
11. Gosavi, A. 2008. Markov decision processes subject to semivariance risk. Working paper at University at Buffalo, SUNY, ISE Department.
12. Gosavi, A., and S. Meyn. 2008. The variance-penalized Bellman equation. Working paper at SUNY Buffalo and University of Illinois at Urbana-Champaign.
13. Grossi, P., and H. Kunreuther. 2005. Catastrophe modeling: A new appraoch to managing risk. Springer.
14. Heger, M. 1994. Consideration of risk in reinforcement learning. Proceedings of the 11th International Conference on Machine Learning:105-111.
15. Markowitz, H. 1952. Portfolio selection. Journal of Finance 7(1):77-91.
16. Puterman, M. L. 1994. Markov decision processes. New York: Wiley Interscience.
17. Ross, S. M. 1997. Introduction to Probability Models. Academic Press, San Diego, CA, USA.
18. Sato, M., and S. Kobayashi. 2001. Average-reward reinforcement learning for variance-penalized Markov decision problems. In Proceedings of the 18th International Conference on Machine Learning, 473-480. Morgan Kauffman.
19. Sutton, R., and A. G. Barto. 1998. Reinforcement learning: An introduction. Cambridge, MA, USA: The MIT Press.
20. Watkins, C. 1989, May. Learning from delayed rewards. Ph. D. thesis, Kings College, Cambridge, England.