Online least-squares policy iteration for reinforcement learning control

Proceedings of the 2010 American Control Conference, ACC 2010

Volumn , Issue , 2010, Pages 486-491

  Buşoniu, Lucian ^a   Ernst, Damien ^b   De Schutter, B ^a   Babuška, Robert ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^b UNIVERSITY OF LIÈGE   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

E-LEARNING; ITERATIVE METHODS; LEARNING ALGORITHMS; LEAST SQUARES APPROXIMATIONS; MACHINE LEARNING; PETROLEUM RESERVOIR EVALUATION;

EXPERIMENTAL EVALUATION; LEAST SQUARES METHODS; LEAST SQUARES POLICY ITERATIONS; LEAST SQUARES TECHNIQUES; LEAST-SQUARES TEMPORAL DIFFERENCES; ON-LINE ALGORITHMS; REINFORCEMENT LEARNING CONTROL; STATE TRANSITIONS;

REINFORCEMENT LEARNING;

EID: 77957782880     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/acc.2010.5530856     Document Type: Conference Paper

References (14)

1. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. MIT Press, 1998.
2. D. P. Bertsekas, Dynamic Programming and Optimal Control, 3rd ed. Athena Scientific, 2007, vol. 2.
3. J. Boyan, "Technical update: Least-squares temporal difference learning," Machine Learning, vol. 49, pp. 233-246, 2002.
4. A. Nedić and D. P. Bertsekas, "Least-squares policy evaluation algorithms with linear function approximation," Discrete Event Dynamic Systems: Theory and Applications, vol. 13, no. 1-2, pp. 79-110, 2003.
5. M. G. Lagoudakis and R. Parr, "Least-squares policy iteration," Journal of Machine Learning Research, vol. 4, pp. 1107-1149, 2003.
6. D. Ernst, P. Geurts, and L. Wehenkel, "Tree-based batch mode reinforcement learning," Journal ofMachine Learning Research, vol. 6, pp. 503-556, 2005.
7. V. Konda, "Actor-critic algorithms," Ph.D. dissertation, Massachusetts Institute of Technology, Cambridge, US, 2002.
8. H. Yu and D. P. Bertsekas, "Convergence results for some temporal difference methods based on least squares," IEEE Transactions on Automatic Control, vol. 54, no. 7, pp. 1515-1531, 2009.
9. V. R. Konda and J. N. Tsitsiklis, "On actor-critic algorithms," SIAM Journal on Control and Optimization, vol. 42, no. 4, pp. 1143-1166, 2003.
10. T. Jung and D. Polani, "Kernelizing LSPE(λ)," in Proceedings 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL-07), Honolulu, US, 1-5 April 2007, pp. 338-345.
11. L. Li, M. L. Littman, and C. R. Mansley, "Online exploration in least-squares policy iteration," in Proceedings 8th International Joint Conference on Autonomous Agents andMultiagent Systems (AAMAS- 09), vol. 2, Budapest, Hungary, 10-15 May 2009, pp. 733-739.
12. D. P. Bertsekas and S. Ioffe, "Temporal differences-based policy iteration and applications in neuro-dynamic programming," Massachusetts Institute of Technology, Cambridge, US, Tech. Rep. LIDS-P-2349, 1996, available at http://web.mit.edu/dimitrib/www/Tempdif.pdf.
13. R. S. Sutton, "Learning to predict by the method of temporal differences," Machine Learning, vol. 3, pp. 9-44, 1988.
14. S. Singh, T. Jaakkola, M. L. Littman, and Cs. Szepesvári, "Convergence results for single-step on-policy reinforcement-learning algorithms," Machine Learning, vol. 38, no. 3, pp. 287-308, 2000.