Automatic shaping and decomposition of reward functions

ACM International Conference Proceeding Series

Volumn 227, Issue , 2007, Pages 601-608

  Marthi, Bhaskara ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ABSTRACTING; APPROXIMATION ALGORITHMS; DECISION MAKING; LEARNING ALGORITHMS; PROBLEM SOLVING; REINFORCEMENT LEARNING;

AUTOMATIC SHAPING; REWARD FUNCTIONS;

MARKOV PROCESSES;

EID: 34547964974     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1273496.1273572     Document Type: Conference Paper

References (28)

1. Agogino, A., & Turner, K. (2006). QUICR-learning for multi-agent coordination. AAAI 2006.
2. Alstrom, J. R. P. (1998). Learning to drive a bicycle using reinforcement learning and shaping. ICML 1998.
3. Andre, D., & Russell, S. J. (2002). State abstraction for programmable reinforcement learning agents. Proceedings of the 17th National Conference on Artificial Intelligence (pp. 119-125).
4. Bagnell, J., & Ng, A. (2006). On local rewards and scaling distributed reinforcement learning. Neural Information Processing Systems. MIT Press.
5. Bertsekas, D. P., & Tsitsiklis, J. N. (1996). Neuro-dynamic programming. Athena Scientific.
6. Chang, Y.-H., Ho, T., & Kaelbling, L. P. (2004). All learning is local: Multi-agent learning in global reward games. In S. Thrun, L. Saul and B. Scholkopf (Eds.), Advances in neural information processing systems 16. Cambridge, MA: MIT Press.
7. Chow, C., & Tsitsiklis, J. (1991). An optimal one-way multigrid algorithm for discrete-time stochastic control. IEEE transactions on automatic control, 36, 898-914.
8. Dietterieh, T. G. (2000). Hierarchical reinforcement learning with the MAXQ value function decomposition. JAIR, 13.
9. Guestrin, C., Koller, D., Parr, R., & Venkataraman, S. (2003). Efficient solution algorithms for factored MDPs. JAIR, 19.
10. Hauskrecht, M., Meuleau, N., Kaelbling, L. P., Dean, T., & Boutilier, C. (1998). Hierarchical solution of Markov decision processes using macro-actions. Proceedings of the fourteenth conference on Uncertainty in Artificial Intelligence (pp. 220-229).
11. Konidaris, C., & Barto, A. (2006). Autonomous shaping: knowledge transfer in reinforcement learning. Proceedings of the 23rd international conference on Machine learning.
12. Laud, A., & DeJong, G. (2002). Reinforcement learning and shaping: Encouraging intended behaviors. ICML (pp. 355-362).
13. Laud, A., & Dejong, G. (2003). The influence of reward on the speed of reinforcement learning: An analysis of shaping. ICML 2003.
14. Li, L., Walsh, T., & Littman, M. (2006). Towards a unified theory of state abstraction for MDPs. Proceedings of the ninth international symposium on AI and mathematics.
15. Littman, M., & Boyan, J. (1993). A distributed reinforcement learning scheme for network routing (Technical Report). Carnegie Mellon University, Pittsburgh, PA, USA.
16. Marthi, B. (2007). Automatic shaping and decomposition of reward functions: website. http://people.csail.mit.edu/bhaskara/autoshape/.
17. Marthi, B., Russell, S., Latham, D., & Guestrin, C. (2005). Concurrent hierarchical reinforcement learning. ICML 2005.
18. Mataric, M. J. (1994). Reward functions for accelerated learning. ICML 1994,
19. Ng, A., Harada, D., & Russell, S. (1999). Policy invariance under reward transformations: Theory and application to reward shaping. ICML 1999.
20. Russell, S., & Zimdars, A. (2003). Q-decomposition for reinforcement learning agents. ICML 2003.
21. Schneider, J., Wong, W., Moore, A., & Riedmiller, M. (1999). Distributed value functions. ICML 1999 (pp. 371-378).
22. Singh, S. P., & Yee, R. C. (1994). An upper bound on the loss from approximate optimal-value functions. Machine Learning, 16, 227-233.
23. Steinkraus, K., & Kaelbling, L. (2004). Combining dynamic abstractions in large MDPs (Technical Report). MIT.
24. Stone, P., & Sutton, R. S. (2001). Scaling reinforcement learning toward RoboCup soccer. ICML 2001.
25. Sutton, R. S., Precup, D., & Singh, S. P. (1999). Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112, 181-211.
26. VanEck, N., & VanWezel, M. (2005). Reinforcement learning and its application to Othello (Technical Report). Erasmus University.
27. Wiewiora, E. (2003). Potential-based shaping and Q-value initialization are equivalent. Journal of Artificial Intelligence Research, 19, 205-208.
28. Yoon, S. W., Fern, A., & Givan, R. (2002). Inductive policy selection for first-order mdps. UAI 2002.