An empirical analysis of value function-based and policy search reinforcement learning

Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS

Volumn 1, Issue , 2009, Pages 554-561

  Kalyanakrishnan, Shivaram ^a   Stone, Peter ^a  

^a University of Texas at Austin   (United States)

Author keywords

Function approximation; Policy search; Reinforcement learning; Temporal difference learning

Indexed keywords

MULTI AGENT SYSTEMS; REINFORCEMENT LEARNING;

EMPIRICAL ANALYSIS; EMPIRICAL STUDIES; FUNCTION APPROXIMATION; PARTIAL OBSERVABILITY; POLICY SEARCH; SEQUENTIAL DECISION MAKING; TEMPORAL DIFFERENCE LEARNING; UNKNOWN ENVIRONMENTS;

AUTONOMOUS AGENTS;

EID: 84899831232     PISSN: 15488403     EISSN: 15582914     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. J. Baxter and P. L. Bartlett. Infinite-horizon policy-gradient estimation. Journal of Artificial Intelligence Research, 15:319-350, 2001.
2. R. E. Bellman. Dynamic Programming. Princeton University Press. Princeton, NJ. June 1957.
3. S. Bhatnagar, R. Sutton, M. Ghavamzadeh, and M. Lee. Incremental natural actor-critic algorithms. In J. Piatt, D. Koller, Y. Singer, and S. Roweis, editors. Advances in Neural Information Processing Systems 20, pages 105-112. MIT Press, Cambridge, MA, 2008.
4. A. R. Cassandra, L. P. Kaelbling, and M. L. Littman. Acting optimally in partially observable stochastic domains. In Proceedings of the Twelfth National Conference on Artificial Intelligence, volume 2. pages 1023-1028, Seattle, Washington, USA, 1994. AAAI Press/MIT Press.
5. R. H. Crites and A. G. Barto. Improving elevator performance using reinforcement learning. In D. S. Touretzky, M. Mozer, and M. E. Hassehno, editors, Advances in Neural Information Processing Systems 8, NIPS, Denver, CO, November 27-30, 1995, pages 1017-1023. MIT Press, 1996.
6. P. T. De Boer, D. P. Kroese, S. Mannor, and R. Rubinstein. A tutorial on the cross-entropy method. Annals of Operations Research, 134(1):19-67, 2005.
7. S. Kakade. A natural policy gradient. In T. G. Dietterich, S. Becker, and 2. Ghahramani, editors, Advances in Neural Information Processing Systems 14, pages 1531-1538. MIT Press. 2001.
8. N. Kohl and P. Stone. Machine learning for fast quadrupedal locomotion. In The Nineteenth National Conference on Artificial Intelligence, pages 611-616, July 2004.
9. M. G. Lagoudakis and R. Parr. Least-squares policy iteration. Journal of Machine Learning Research, 4:1107-1149, 2003.
10. J. Loch and S. Singh. Using eligibility traces to find the best memoryless policy in partially observable Markov decision processes. In Proceedings of the Fifteenth International Conference on Machine Learning, pages 323-331. Morgan Kaufmann, 1998.
11. A. Y. Ng, H. J. Kim, M. I. Jordan, and S. Sastry. Autonomous helicopter flight via reinforcement learning. In S. Thrun, L. Saul, and B. Scholkopf, editors, Advances in Neural Information Processing Systems 16. MIT Press, Cambridge, MA, 2004.
12. T. J. Perkins and D. Precup. A convergent form of approximate policy iteration. In S. T. S. Becker and K. Obermayer, editors, Advances in Neural Information Processing Systems 15, pages 1595-1602. MIT Press, Cambridge, MA, 2003.
13. J. Peters and S. Schaal. Natural actor-critic. Neurocomputing, 71 (7-9): 1180-1190, 2008.
14. G. A. Rummery and M. Niranjan. On-line Q-learning using connectionist systems. Technical Report CUED/F-INFENG/TR 166, Cambridge University Engineering Department, 1994.
15. K. O. Stanley. Efficient evolution of neural networks through complexification. Technical Report Al-TR-04-314, Department of Computer Sciences. University of Texas at Austin, August 2004.
16. P. Stone, R. S. Sutton, and G. Kuhlmann. Reinforcement learning for RoboCup-soccer keepaway. Adaptive Behavior, 13(3):165-188. 2005.
17. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 1998.
18. I. Szita and A. Lorincz. Learning Tetris using the noisy cross-entropy method. Neural Computation, 18:2936-2941, 2006.
19. M. Taylor, S. Whiteson. and P. Stone. Comparing evolutionary and temporal difference methods for reinforcement learning. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 1321-28, July 2006.
20. M. E. Taylor and P. Stone. Behavior transfer for value-function-based reinforcement learning. In F. Dignum, V. Dignum, S. Koenig. S. Kraus, M. P. Singh, and M. Wooldridge, editors, The Fourth International Joint Conference on Autonomous Agents and Multiagent Systems, pages 53-59, New York, NY, July 2005. ACM Press.
21. G. Tesauro, N. K. Jong, R. Das, and M. N. Bennani. On the use of hybrid reinforcement learning for autonomic resource allocation. Cluster Computing, 10(3):287-299, 2007.
22. C. J. C. H. Watkins and P. Dayan. Q-learning. Machine Learning, 8(3-4):279-292; 1992.
23. S. Whiteson and P. Stone. Evolutionary function approximation for reinforcement learning. Journal of Machine Learning Research, 7:877-917, May 2006.