Accelerating reinforcement learning through implicit imitation

Journal of Artificial Intelligence Research

Volumn 19, Issue , 2003, Pages 569-629

  Price, Bob ^a   Boutilier, Craig ^b  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^b UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

FORMAL LOGIC; MATHEMATICAL MODELS; MULTI AGENT SYSTEMS;

LEARNING AGENTS; REINFORCEMENT LEARNING;

LEARNING SYSTEMS;

EID: 27344432348     PISSN: 10769757     EISSN: 10769757     Source Type: Journal    
DOI: 10.1613/jair.898     Document Type: Article

References (73)

1. Alissandrakis, A., Nehaniv, C. L., & Dautenhahn, K. (2000). Learning how to do things with imitation. In Bauer, M., & Rich, C. (Eds.), AAAI Fall Symposium on Learning How to Do Things, pp. 1-6 Cape Cod, MA.
2. Atkeson, C. G., & Schaal, S. (1997). Robot learning from demonstration. In Proceedings of the Fourteenth International Conference on Machine Learning, pp. 12-20 Nashville, TN.
3. Atkeson, C. G., Moore, A. W., & Schaal, S. (1997). Locally weighted learning for control. Artificial Intelligence Review, 11(1-5), 75-113.
4. Bakker, P., & Kuniyoshi, Y. (1996). Robot see, robot do: An overview of robot imitation. In AISB96 Workshop on Learning in Robots and Animals, pp. 3-11 Brighton,UK.
5. Bellman, R. E. (1957). Dynamic Programming. Princeton University Press, Princeton.
6. Bertsekas, D. P. (1987). Dynamic Programming: Deterministic and Stochastic Models. Prentice-Hall, Englewood Cliffs.
7. Bertsekas, D. P., & Tsitsiklis, J. N. (1996). Neuro-dynamic Programming. Athena, Belmont, MA.
8. Billard, A., & Hayes, G. (1997). Learning to communicate through imitation in autonomous robots. In Proceedings of The Seventh International Conference on Artificial Neural Networks, pp. 763-68 Lausanne, Switzerland.
9. Billard, A., & Hayes, G. (1999). Drama, a connectionist architecturefor control and learning in autonomous robots. Adaptive Behavior Journal, 7, 35-64.
10. Billard, A., Hayes, G., & Dautenhahn, K. (1999). Imitation skills as a means to enhance learning of a synthetic proto-language in an autonomous robot. In Proceedings of the AISB'99 Symposium on Imitation in Animals and Artifacts, pp. 88-95 Edinburgh.
11. Boutilier, C. (1999). Sequential optimality and coordination in multiagent systems. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, pp. 478-485 Stockholm.
12. Boutilier, C., Dean, T., & Hanks, S. (1999). Decision theoretic planning: Structural assumptions and computational leverage. Journal of Artificial Intelligence Research, 11, 1-94.
13. Bowling, M., & Veloso, M. (2001). Rational and convergent learning in stochastic games. In Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, pp. 1021-1026 Seattle.
14. Breazeal, C. (1999). Imitation as social exchange between humans and robot. In Proceedings of the AISB'99 Symposium on Imitation in Animals and Artifacts, pp. 96-104 Edinburgh.
15. Byrne, R. W., & Russon, A. E. (1998). Learning by imitation: a hierarchical approach. Behavioral and Brain Sciences, 21, 667-721.
16. Cañamero, D., Arcos, J. L., & de Mantaras, R. L. (1999). Imitating human performances to automatically generate expressive jazz ballads. In Proceedings of the AISB'99 Symposium on Imitation in Animals and Artifacts, pp. 115-20 Edinburgh.
17. Cassandra, A. R., Kaelbling, L. P., & Littman, M. L. (1994). Acting optimally in partially observable stochastic domains. In Proceedings of the Twelfth National Conference on Artificial Intelligence, pp. 1023-1028 Seattle.
18. Conte, R. (2000). Intelligent social learning. In Proceedings of the AISB'00 Symposium on Starting from Society: the Applications of Social Analogies to Computational Systems Birmingham.
19. Crites, R., & Barto, A. G. (1998). Elevator group control using multiple reinforcement learning agents. Machine-Learning, 55(2-3), 235-62.
20. Dean, T., & Givan, R. (1997). Model minimization in Markov decision processes. In Proceedings of the Fourteenth National Conference on Artificial Intelligence, pp. 106-111 Providence.
21. Dearden, R., & Boutilier, C. (1997). Abstraction and approximate decision theoretic planning. Artificial Intelligence, 89, 219-283.
22. Dearden, R., Friedman, N., & Andre, D. (1999). Model-based bayesian exploration. In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 150-159 Stockholm.
23. DeGroot, M. H. (1975). Probability and statistics. Addison-Wesley, Reading, MA.
24. Demiris, J., &: Hayes, G. (1997). Do robots ape?. In Proceedings of the AAAI Fall Symposium on Socially Intelligent Agents, pp. 28-31 Cambridge, MA.
25. Demiris, J., & Hayes, G. (1999). Active and passive routes to imitation. In Proceedings of the AISB'99 Symposium on Imitation in Animals and Artifacts, pp. 81-87 Edinburgh.
26. Fiorito, G., &: Scotto, P. (1992). Observational learning in octopus vulgaris. Science, 256, 545-47.
27. Forbes, J., & Andre, D. (2000). Practical reinforcement learning in continuous domains. Tech. rep. UCB/CSD-00-1109, Computer Science Division, University of California, Berkeley.
28. Friedrich, H., Munch, S., Dillmann, R., Bocionek, S., & Sassin, M. (1996). Robot programming by demonstration (RPD): Support the induction by human interaction. Machine Learning, 23, 163-189.
29. Hartmanis, J., & Stearns, R. E. (1966). Algebraic Structure Theory of Sequential Machines. Prentice-Hall, Englewood Cliffs.
30. Hu, J., & Wellman, M. P. (1998). Multiagent reinforcement learning: Theoretical framework and an algorithm. In Proceedings of the Fifthteenth International Conference on Machine Learning, pp. 242-250 Madison, WI.
31. Kaelbling, L. P. (1993). Learning in Embedded Systems. MIT Press, Cambridge,MA.
32. Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence. Research, 4, 237-285.
33. Kearns, M., & Singh, S. (1998). Near-optimal reinforcement learning in polynomial time. In Proceedings of the Fifthteenth International Conference on Machine Learning, pp. 260-268 Madison, WI.
34. Kuniyoshi, Y., Inaba, M., & Inoue, H. (1994). Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6), 799-822.
35. Lee, D., & Yannakakis, M. (1992). Online miminization of transition systems. In Proceedings of the 24th Annual ACM Symposium on the Theory of Computing (STOC-92), pp. 264-274 Victoria, BC.
36. Lieberman, H. (1993). Mondrian: A teachable graphical editor. In Cypher, A. (Ed.), Watch What I Do: Programming by Demonstration, pp. 340-358. MIT Press, Cambridge, MA.
37. Lin, L.-J. (1991). Self-improvement based on reinforcement learning, planning and teaching. Machine Learning: Proceedings of the Eighth International Workshop (ML91), 8, 323-27.
38. Lin, L.-J. (1992). Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning, 8, 293-321.
39. Littman, M. L. (1994). Markov games as a framework for multi-agent reinforcement learning. In Proceedings of the Eleventh International Conference on Machine Learning, pp. 157-163 New Brunswick, NJ.
40. Lovejoy, W. S. (1991). A survey of algorithmic methods for partially observed Markov decision processes. Annals of Operations Research, 28, 47-66.
41. Mataric, M. J. (1998). Using communication to reduce locality in distributed multi-agent learning. Journal Experimental and Theoretical Artificial Intelligence, 10(3), 357-369.
42. Matarić, M. J., Williamson, M., Demiris, J., & Mohan, A. (1998). Behaviour-based primitives for articulated control. In R. Pfiefer, B. Blumberg, J.-A. M. . S. W. W. (Ed.), Fifth International conference on simulation of adaptive behavior SAB'98, pp. 165-170 Zurich. MIT Press.
43. Meuleau, N., & Bourgine, P. (1999). Exploration of multi-state environments: Local mesures and back-propagation of uncertainty. Machine Learning, 32(2), 117-154.
44. Mi, J., & Sampson, A. R. (1993). A comparison of the Bonferroni and Scheffé bounds. Journal of Statistical Planning and Inference, 36, 101-105.
45. Michie, D. (1993). Knowledge, learning and machine intelligence. In Sterling, L. (Ed.), Intelligent Systems. Plenum Press, New York.
46. Mitchell, T. M., Mahadevan, S., & Steinberg, L. (1985). LEAP: A learning apprentice for VLSI design. In Proceedings of the Ninth International Joint Conference on Artificial Intelligence, pp. 573-580 Los Altos, California. Morgan Kaufmann Publishers, Inc.
47. Moore, A. W., & Atkeson, C. G. (1993). Prioritized sweeping: Reinforcement learning with less data and less real time. Machine Learning, 13(1), 103-30.
48. Myerson, R. B. (1991). Game Theory: Analysis of Conflict. Harvard University Press, Cambridge.
49. Nehaniv, C., & Dautenhahn, K. (1998). Mapping between dissimilar bodies: Affordances and the algebraic foundations of imitation. In Proceedings of the Seventh European Workshop on Learning Robots, pp. 64-72 Edinburgh.
50. Ng, A. Y., & Russell, S. (2000). Algorithms for inverse reinforcement learning. In Proceedings of the Seventeenth International Conference on Machine Learning, pp. 663-670 Stanford.
51. Noble, J., & Todd, P. M. (1999). Is it really imitation? a review of simple mechanisms in social information gathering. In Proceedings of the AISB'99 Symposium on Imitation in Animals and Artifacts, pp. 65-73 Edinburgh.
52. Oliphant, M. (1999). Cultural transmission of communications systems: Comparing observational and reinforcement learning models. In Proceedings of the AISB'99 Symposium on Imitation in Animals and Artifacts, pp. 47-54 Edinburgh.
53. Price, B., & Boutilier, C. (2003). A Bayesian approach to imitation in reinforcement learning. In Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence Acapulco. to appear.
54. Puterman, M. L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley and Sons, Inc., New York.
55. Russon, A., & Galdikas, B. (1993). Imitation in free-ranging rehabilitant orangutans (pongo-pygmaeus). Journal of Comparative Psychology, 107(2), 147-161.
56. Sammut, C., Hurst, S., Kedzier, D., & Michie, D. (1992). Learning to fly. In Proceedings of the Ninth International Conference on Machine Learning, pp. 385-393 Aberdeen, UK.
57. Scassellati, B. (1999). Knowing what to imitate and knowing when you succeed. In Proceedings of the AISB'99 Symposium on Imitation in Animals and Artifacts, pp. 105-113 Edinburgh.
58. Scheffer, T., Greiner, R., & Darken, C. (1997). Why experimentation can be better than perfect guidance. In Proceedings of the Fourteenth International Conference on Machine Learning, pp. 331-339 Nashville.
59. Seber, G. A. F. (1984). Multivariate Observations. Wiley, New York.
60. Shapley, L. S. (1953). Stochastic games. Proceedings of the National Academy of Sciences, 39, 327-332.
61. Singh, S. P., & Bertsekas, D. (1997). Reinforcement learning for dynamic channel allocation in cellular telephone systems. In Advances in Neural information processing systems, pp. 974-980 Cambridge, MA. MIT Press.
62. Smallwood, R. D., & Sondik, E. J. (1973). The optimal control of partially observable Markov processes over a finite horizon. Operations Research, 21, 1071-1088.
63. Sue, D., & Bratko, I. (1997). Skill reconstruction as induction of LQ controllers with subgoals. In Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, pp. 914-919 Nagoya.
64. Sutton, R. S. (1988). Learning to predict by the method of temporal differences. Machine Learning, 3, 9-44.
65. Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA.
66. Tan, M. (1993). Multi-agent reinforcement learning: Independent vs. cooperative agents. In ICML-93, pp. 330-37.
67. Urbancic, T., & Bratko, I. (1994). Reconstruction human skill with machine learning. In Eleventh European Conference on Artificial Intelligence, pp. 498-502 Amsterdam.
68. Utgoff, P. E., & Clouse, J. A. (1991). Two kinds of training information for evaluation function learning. In Proceedings of the Ninth National Conference on Artificial Intelligence, pp. 596-600 Anaheim, CA.
69. van Lent, M., & Laird, J. (1999). Learning hierarchical performance knowledge by observation. In Proceedings of the Sixteenth International Conference on Machine Learning, pp. 229-238 Bled, Slovenia.
70. Visalberghi, E., & Fragazy, D. (1990). Do monkeys ape?. In Parker, S., & Gibson, K. (Eds.), Language and Intelligence in Monkeys and Apes, pp. 247-273. Cambridge University Press, Cambridge.
71. Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine Learning, 8, 279-292.
72. Whitehead, S. D. (1991a). Complexity analysis of cooperative mechanisms in reinforcement learning. In Proceedings of the Ninth National Conference on Artificial Intelligence, pp. 607-613 Anaheim.
73. Whitehead, S. D. (1991b). Complexity and cooperation in q-learning. In Machine Learning. Proceedings of the Eighth International Workshop (ML91), pp. 363-367.