Adaptive bases for reinforcement learning

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 6321 LNAI, Issue PART 1, 2010, Pages 312-327

  Di Castro, Dotan ^a   Mannor, Shie ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ACTOR CRITIC; FUNCTION APPROXIMATION; SQUARE ERRORS; VALUE FUNCTIONS;

REINFORCEMENT LEARNING;

EID: 78049343060     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-15880-3_26     Document Type: Conference Paper

References (18)

1. Archibald, T., McKinnon, K., Thomas, L.: On the Generation of Markov Decision Processes. Journal of the Operational Research Society 46, 354-361 (1995)
2. Bradtke, S. J., Barto, A. G.: Linear least-squares algorithms for temporal difference learning. Machine Learning 22, 33-57 (1996)
3. Bertsekas, D.: Dynamic programming and optimal control, 3rd edn. Athena Scientific, Belmont (2007)
4. Bertsekas, D., Tsitsiklis, J.: Neuro-dynamic programming. Athena Scinetific, Belmont (1996)
5. Bhatnagar, S., Sutton, R., Ghavamzadeh, M., Lee, M.: Natural actor-critic algorithms. Technical report Univ. of Alberta (2007)
6. Borkar, V.: Stochastic approximation with two time scales. Systems & Control Letters 29, 291-294 (1997)
7. Borkar, V., Meyn, S.: The ODE method for convergence of stochastic approximation and reinforcement learning. SIAM Journal on Cont. and Optim. 38, 447-469 (2000)
8. Busoniu, L., Ernst, D., De Schutter, B., Babuska, R.: Cross-Entropy Optimization of Control Policies With Adaptive Basis Functions. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics (99), 1-14 (2010)
9. Girgin, S., Preux, P.: Basis expansion in natural actor critic methods. In: Girgin, S., Loth, M., Munos, R., Preux, P., Ryabko, D. (eds.) EWRL 2008. LNCS (LNAI), vol. 5323, pp. 110-123. Springer, Heidelberg (2008)
10. Kushner, H., Yin, G.: Stochastic approximation and recursive algorithms and applications. Springer, Heidelberg (2003)
11. Leslie, D., Collins, E.: Convergent multiple-timescales reinforcement learning algorithms in normal form games. The Annals of App. Prob. 13, 1231-1251 (2003)
12. Menache, I., Mannor, S., Shimkin, N.: Basis function adaptation in temporal difference reinforcement learning. Annals of Operations Research 134, 215-238 (2006)
13. Mokkadem, A., Pelletier, M.: Convergence rate and averaging of nonlinear twotime-scale stochastic approximation algorithms. Annals of Applied Prob. 16, 1671
14. Schoknecht, R.: Optimality of reinforcement learning algorithms with linear function approximation. In: Proceedings of Neural Information Processing and Systems, pp. 1555-1562 (2002)
15. Sutton, R. S., Barto, A. G.: Reinforcement Learning - an Introduction. MIT Press, Cambridge (1998)
16. Sutton, R. S., Maei, H. R., Precup, D., Bhatnagar, S., Silver, D., Szepesvári, C., Wiewiora, E.: Fast gradient-descent methods for temporal-difference learning with linear function approximation. In: Proceedings of the 26th Annual International Conference on Machine Learning (2009)
17. Sutton, R. S., Szepesvari, C., Maei, H. R.: A convergent o (n) temporal-difference algorithm for off-policy learning with linear function approximation. In: Advances in Neural Information Processing Systems, vol. 21, pp. 1609-1616 (2009b)
18. Yu, H., Bertsekas, D.: Basis function adaptation methods for cost approximation in MDP. In: Proc. of IEEE International Symposium on Adaptive Dynamic Programming and Reinforcement Learning, Nashville, TN (2009)