Horde: A scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction

10th International Conference on Autonomous Agents and Multiagent Systems 2011, AAMAS 2011

Volumn 2, Issue , 2011, Pages 713-720

  Sutton, Richard S ^a   Modayil, Joseph ^a   Degris, Michael Delp Thomas ^a   Pilarski, Patrick M ^a   White, Adam ^a   Precup, Doina ^b  

^a UNIVERSITY OF ALBERTA   (Canada)

^b MCGILL UNIVERSITY   (Canada)

Author keywords

Artificial intelligence; Knowledge representation; Off policy learning; Real time; Reinforcement learning; Robotics; Temporal difference learning; Value function approximation

Indexed keywords

ARTIFICIAL INTELLIGENCE; AUTONOMOUS AGENTS; KNOWLEDGE REPRESENTATION; MULTI AGENT SYSTEMS; ROBOTICS;

ARTIFICIAL INTELLIGENCE SYSTEMS; FUNCTION APPROXIMATION; GOAL-ORIENTED BEHAVIOR; OFF-POLICY LEARNING; REAL-TIME; REAL-TIME ARCHITECTURE; TEMPORAL DIFFERENCE LEARNING; VALUE FUNCTION APPROXIMATION;

REINFORCEMENT LEARNING;

EID: 84899464022     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. Becker, J. D. (1973). A model for the encoding of experiential information. In Computer Models of Thought and Language, Schank, R. C, Colby, K. M., Eds. W. H. Freeman and Company.
2. Chaslot, G. M. J-B. (2010). Monte-Carlo tree search. PhD thesis, Dutch Research School for Information and Knowledge Systems.
3. Cohen, P. R., Atkin, M. S., Oates, T., Beal, C. R. (1997). Neo: Learning conceptual knowledge by sensorimotor interaction with an environment. In Agents '97, Marina del Rey, CA. ACM.
4. Cunningham, M. (1972). Intelligence: Its Organization and Development. Academic Press.
5. Drescher, G. L. (1991). Made-Up Minds: A Constructivist Approach to Artificial Intelligence. MIT Press, Cambridge, MA.
6. Kaelbling, L. P., Oates, T., Hernandez, N., Finney, S. (2001). Learning in worlds with objects. Working Notes of the AAAI Stanford Spring Symposium on Learning Grounded Representations.
7. Maei, H. R., Sutton, R. S. (2010). GQ(A): A general gradient algorithm for temporal-difference prediction learning with eligibility traces. In Proceedings of the Third Conference on Artificial General Intelligence, Lugano, Switzerland.
8. Maei, H. R., Szepesvári, Cs., Bhatnagar, S., Precup, D., Silver, D., Sutton, R. S. (2009). Convergent temporal-difference learning with arbitrary smooth function approximation. In Advances in Neural Information Processing Systems 22, Vancouver, BC. MIT Press.
9. Maei, H. R., Szepesvári, Cs., Bhatnagar, S., Sutton, R. S. (2010). Toward off-policy learning control with function approximation. In Proceedings of the 27th International Conference on Machine Learning, Haifa, Israel.
10. Natale, L. (2005). Linking action to perception in a humanoid robot: A developmental approach to grasping. MIT PhD thesis.
11. Oates, T., Schmill, M. D., Cohen, P. R. (2000). A method for clustering the experiences of a mobile robot that accords with human judgments. Proceedings AAAI, 846-851, AAAI/MIT Press.
12. Pasula, H., Zettlemoyer, L., Kaelbling L. (2007). Learning symbolic models of stochastic domains. Journal of Artificial Intelligence Research 59:309-352.
13. Pierce, D. M., Kuipers, B. J. (1997). Map learning with uninterpreted sensors and effectors. Artificial Intelligence 92:169-227.
14. Ring, M. B. (1997). CHILD: A first step toward continual learning. Machine Learning, 28:77-104.
15. Sutton, R. S. (1988). Learning to predict by the method of temporal differences. Machine Learning 3:9-44.
16. Sutton, R. S. (1990). Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Proceedings of the Seventh International Conference on Machine Learning, pp. 216-224. Morgan Kaufmann, San Mateo, CA.
17. Sutton, R. S., Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press.
18. Sutton, R. S., Maei, H. R., Precup, D., Bhatnagar, S., Silver, D., Szepesvari, Cs., Wiewiora, E. (2009). Fast gradient-descent methods for temporal-difference learning with linear function approximation. In Proceedings of the 26th International Conference on Machine Learning, Montreal, Canada.
19. Sutton, R. S., Precup D., Singh, S. (1999). Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112:181-211.
20. Sutton, R. S., Rafols, E. J., Koop, A. (2006). Temporal abstraction in temporal-difference networks. Advances in Neural Information Processing Systems 18.
21. Sutton, R. S., Szepesvári, Cs., Maei, H. R. (2008). A convergent O(n) algorithm for off-policy temporal-difference learning with linear function approximation. Advances in Neural Information Processing Systems 21.
22. Yu, C., Ballard, D. (2004). A multimodal learning interface for grounding spoken language in sensory perceptions. ACM Transactions on Applied Perception 1:57-80.