Using prior knowledge to accelerate online least-squares policy iteration

2010 IEEE International Conference on Automation, Quality and Testing, Robotics, AQTR 2010 - Proceedings

Volumn 1, Issue , 2010, Pages 44-49

  Buşoniu, Lucian ^a   De Schutter, Bart ^a   Babuška, Robert ^a   Ernst, Damien ^b  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^b UNIVERSITY OF LIÈGE   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

CONTROL POLICY; EMPIRICAL EVALUATIONS; IN-CONTROL; LEAST SQUARE; MONOTONICITY; ONLINE LEARNING; OPTIMAL CONTROLS; POLICY ITERATION; PRIOR KNOWLEDGE; SYSTEM STATE;

ITERATIVE METHODS;

ROBOTICS;

EID: 77958522395     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/AQTR.2010.5520917     Document Type: Conference Paper

References (12)

1. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Athena Scientific, 1996.
2. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. MIT Press, 1998.
3. D. P. Bertsekas, Dynamic Programming and Optimal Control, 3rd ed. Athena Scientific, 2007, vol. 2.
4. J. Boyan, "Technical update: Least-squares temporal difference learning, " Machine Learning, vol. 49, pp. 233-246, 2002.
5. A. Nedić and D. P. Bertsekas, "Least-squares policy evaluation algorithms with linear function approximation, " Discrete Event Dynamic Systems: Theory and Applications, vol. 13, no. 1-2, pp. 79-110, 2003.
6. M. G. Lagoudakis and R. Parr, "Least-squares policy iteration, " Journal of Machine Learning Research, vol. 4, pp. 1107-1149, 2003.
7. D. Ernst, P. Geurts, and L. Wehenkel, "Tree-based batch mode reinforcement learning, " Journal ofMachine Learning Research, vol. 6, pp. 503-556, 2005.
8. H. Yu and D. P. Bertsekas, "Convergence results for some temporal difference methods based on least squares, " IEEE Transactions on Automatic Control, vol. 54, no. 7, pp. 1515-1531, 2009.
9. L. Buşoniu, D. Ernst, B. De Schutter, and R. Babǔska, "Online least-squares policy iteration for reinforcement learning control, " in Proceedings 2010 American Control Conference (ACC-10), Baltimore, US, 30 June - 2 July 2010, accepted for publication.
10. R. S. Sutton, "Learning to predict by the method of temporal differences, " Machine Learning, vol. 3, pp. 9-44, 1988.
11. L. Li, M. L. Littman, and C. R. Mansley, "Online exploration in least-squares policy iteration, " in Proceedings 8th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS- 09), vol. 2, Budapest, Hungary, 10-15 May 2009, pp. 733-739.
12. T. Jung and D. Polani, "Kernelizing LSPE(λ?), " in Proceedings 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL-07), Honolulu, US, 1-5 April 2007, pp. 338-345.