Dyna-style planning with linear function approximation and prioritized sweeping

Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, UAI 2008

Volumn , Issue , 2008, Pages 528-536

  Sutton, Richard S ^a   Szepesvári, Csaba ^a   Geramifard, Alborz ^a   Bowling, Michael ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

BACKING-UP; LEAST SQUARE; LIMIT POINTS; LINEAR APPROXIMATIONS; LINEAR FUNCTIONS; MODEL BASED APPROACH; MODEL FREE; NATURAL CONDITIONS; ON-LINE SETTING; OPTIMAL CONTROL POLICY; POLICY EVALUATION; PRIORITIZED SWEEPING; STATE SPACE; STATE TRANSITIONS; VALUE FUNCTIONS; WORLD MODEL;

ARTIFICIAL INTELLIGENCE; LEARNING ALGORITHMS; OPTIMAL CONTROL SYSTEMS; REINFORCEMENT LEARNING;

UNCERTAINTY ANALYSIS;

EID: 80053284668     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (26)

1. Atkeson, C. (1993). Using local trajectory optimizers to speed up global optimization in dynamic programming. Advances in Neural Information Processing Systems, 5, 663-670.
2. Baird, L. C. (1995). Residual algorithms: Reinforcement learning with function approximation. In Proceedings of the Twelfth International Conference on Machine Learning, pp. 30-37.
3. Bertsekas, Dimitri P., Tsitsiklis. J. (1996). Neuro-Dynamic Programming. Athena Scientific, 1996.
4. Boutilier, C., Dearden, R., Goldszmidt, M. (2000). Stochastic dynamic programming with factored representations. Artificial Intelligence 121: 49-107.
5. Bowling, M., Geramifard, A., Wingate, D. (2008). Sigma point policy iteration. In Proceedings of the Seventh International Conference on Autonomous Agents and Multiagent Systems.
6. Boyan, J. A. (1999). Least-squares temporal difference learning. In Proceedings of the Sixteenth International Conference on Machine Learning, 49-56.
7. Boyan, J. A. (2002). Technical update: Least-squares temporal difference learning. Machine Learning, 49:233-246. (Pubitemid 34325688)
8. Bradtke, S., Barto, A. G. (1996). Linear least-squares al gorithms for temporal difference learning. Machine Learning, 22:33-57. (Pubitemid 126724362)
9. Degris, T., Sigaud, O., Wuillemin, P. (2006). Learning the structure of factored markov decision processes in reinforcement learning problems. Proceedings of the 23rd International Conference on Machine Learning.
10. Delyon, B. (1996). General results on the convergence of stochastic algorithms. IEEE Transactions on Automatic Control, 41:1245-1255. (Pubitemid 126768500)
11. Geramifard, A., Bowling, M., Sutton, R. S. (2006). Incremental least-square temporal difference learning. Proceedings of the National Conference on Artificial Intelligence, pp. 356-361. (Pubitemid 44705310)
12. Kuvayev, L., Sutton, R. S. (1996). Model-based reinforcement learning with an approximate, learned model. Proceedings of the Ninth Yale Workshop on Adaptive and Learning Systems, pp. 101-105, Yale University, New Haven, CT.
13. Lagoudakis, M., Parr, R. (2003). Least squares policy iteration. Journal of Machine Learning Research, 4:1107-1149.
14. McMahan H. B., Gordon G. J. (2005). Fast exact planning in Markov decision processes. Proceedings of the 15th International Conference on Automated Planning and Scheduling.
15. Moore, A. W., Atkeson, C. G. (1993). Prioritized sweeping: Reinforcement learning with less data and less real time. Machine Learning, 13:103-130.
16. Paduraru, C. (2007). Planning with Approximate and Learned Models of Markov Decision Processes. MSc thesis, Department of Computing Science, University of Alberta.
17. Peng, J.,Williams, R.J. (1993). Efficient learning and planning within the Dyna framework, Adaptive Behavior 1, 437-454.
18. Peters, J., Vijayakumar, S. and Schaal, S. (2005). Natural actor-critic. Proceedings of the 16th European Conference on Machine Learning, pp. 280-291.
19. Schaeffer, J., Hlynka, M., Jussila, V. (2001). Temporal difference learning applied to a high-performance game-playing program. Proceedings of the International Joint Conference on Artificial Intelligence, pp. 529-534.
20. Silver, D., Sutton, R. S., Müller, M. (2007). Reinforcement learning of local shape in the game of Go. Proceedings of the 20th International Joint Conference on Artificial Intelligence, pp. 1053-1058.
21. Singh, S. P. (1992). Reinforcement learning with a hierarchy of abstract models. Proceedings of the Tenth National Conference on Artificial Intelligence, pp. 202-207.
22. Sutton, R. S. (1988). Learning to predict by the method of temporal differences. Machine Learning, 3:9-44.
23. Sutton, R. S. (1990). Integrated architectures for learning, planning, and reacting based on approximating dynamic programming, Proceedings of the Seventh International Conference on Machine Learning, pp. 216-224.
24. Sutton, R. S. (1996). Generalization in reinforcement learning: Successful examples using sparse coarse coding. Advances in Neural Information Processing Systems: Proceedings of the 1995 Conference, pp. 1038-1044. MIT Press.
25. Sutton, R. S., Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press.
26. Wingate, D., Seppi, K. D. (2005). Prioritization methods for accelerating MDP solvers. Journal of Machine Learning Research, 6: 851-881.