A new Q(λ) with interim forward view and Monte Carlo equivalence

31st International Conference on Machine Learning, ICML 2014

Volumn 3, Issue , 2014, Pages 1973-1988

  Sutton, Richard S ^a   Mahmood, A Rupam ^a   Precup, Doina ^b   Van Hasselt, Hado ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

^b MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; MONTE CARLO METHODS; PARALLEL PROCESSING SYSTEMS; PARAMETER ESTIMATION; REINFORCEMENT LEARNING;

ALGORITHMIC COMPLEXITY; ASYMPTOTIC PERFORMANCE; DERIVATION TECHNIQUES; ELIGIBILITY TRACES; EN-ROUTE; Q-LEARNING; RAPID LEARNING; TIME STEP;

LEARNING ALGORITHMS;

EID: 84919913727     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. Downey, C., Sanner, S. (2010). Temporal difference Bayesian model averaging: A Bayesian perspective on adapting lambda. In Proceedings of the 27th International Conference on Machine Learning, pp. 311-318.
2. Geist, M., Scherrer, B. (2014). Off-policy learning with eligibility traces: A survey. Journal of Machine Learning Research 15:289-333.
3. Kearns, M. J., Singh, S. P. (2000). Bias-variance error bounds for temporal difference updates. In Proceedings of the 13th Annual Conference on Computational Learning Theory, pp. 142-147.
4. Maei, H. R. (2011). Gradient Temporal-Difference Learning Algorithms. PhD thesis, University of Alberta.
5. Maei, H. R., Sutton, R. S. (2010). GQ(A): A general gradient algorithm for temporal-difference prediction learning with eligibility traces. In Proceedings of the Third Conference on Artificial General Intelligence, pp. 91-96. Atlantis Press.
6. Modayil, J., White, A., Sutton, R. S. (2014). Multi-timescale nexting in a reinforcement learning robot. Adaptive Behavior 22(2): 146-160.
7. Peng, J. (1993). Efficient Dynamic Programming-Based Learning for Control. PhD thesis, Northeastern University, Boston.
8. Precup, D., Sutton, R. S., Singh, S. (2000). Eligibility traces for off-policy policy evaluation. In Proceedings of the 17th International Conference on Machine Learning, pp. 759-766. Morgan Kaufmann.
9. Precup, D., Sutton, R. S., Dasgupta, S. (2001). Off-policy temporal-difference learning with function approximation. In Proceedings of the 18th International Conference on Machine Learning, pp. 417-424.
10. Rummery, G. A. (1995). Problem Solving with Reinforcement Learning. PhD thesis, Cambridge University.
11. Singh, S. P., Dayan, P. (1998). Analytical mean squared error curves for temporal difference learning. Machine Learning 52:5-40.
12. Singh, S. P., Sutton, R. S. (1996). Reinforcement learning with replacing eligibility traces. Machine Learning 22:123-158.
13. Sutton, R. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning 3:9-44.
14. Sutton, R. S. (1995). TD models: Modeling the world at a mixture of time scales. In Proceedings of the Twelfth International Conference on Machine Learning, pp. 531- 539. Morgan Kaufmann.
15. Sutton, R. S., Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press.
16. Sutton, R. S., Modayil, J., Delp, M., Degris, T., Pilarski, P. M., White, A., Precup, D. (2011). Horde: A scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction. In Proceedings of the Tenth International Conference on Autonomous Agents and Multiagent Systems, Taipei, Taiwan.
17. Sutton, R. S., Singh, S. (1994). On bias and step size in temporal-difference learning. In Proceedings of the Eighth Yale Workshop on Adaptive and Learning Systems, pp. 91-96, New Haven, CT. Yale University.
18. van Hasselt, H., Mahmood, A. R., Sutton, R. S. (2014). Off-policy TD(A) with a true online equivalence. In Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, Quebec City, Canada.
19. van Seijen, H., Sutton, R. S. (2014). True online TD(A). In Proceedings of the 31st International Conference on Machine Learning. Beijing, China. JMLR: W&CP volume 32.
20. van Seijen, H., van Hasselt, H., Whiteson, S., Wiering, M. (2009). A theoretical and empirical analysis of Expected Sarsa. In Proceedings of the IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, pp. 177-184.
21. Watkins, C. J. C. H. (1989). Learning from Delayed Rewards. PhD thesis, Cambridge University.
22. Yu, H. (2010). Convergence of least-squares temporal difference methods under general conditions. In Proceedings of the 27th International Conference on Machine Learning, pp. 1207-1214.