A Formalism for Learning from Demonstration*

Paladyn

Volumn 1, Issue 1, 2010, Pages 1-13

  Billing, Erik A ^a   Hellström, Thomas ^a  

^a UMEÅ UNIVERSITY   (Sweden)

Author keywords

ambiguities; behavior; bias; generalization; learning from demonstration; robot learning

Indexed keywords

BEHAVIORAL RESEARCH; ROBOT LEARNING; ROBOTS;

AMBIGUITY; BEHAVIOR; BEHAVIOUR RECOGNITION; BIAS; GENERALISATION; INFORMATION SPACES; LEARNING FROM DEMONSTRATION; LEARNING PROCESS; LEARNING TECHNIQUES; ROBOT BEHAVIOR;

DEMONSTRATIONS;

EID: 85141587915     PISSN: None     EISSN: 20814836     Source Type: Journal    
DOI: 10.2478/s13230-010-0001-5     Document Type: Article

References (87)

1. A. Alissandrakis, C. L. Nehaniv, and K. Dautenhahn. Imitation with ALICE: learning to imitate corresponding actions across dissimilar embodiments. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 32:482-496, 2002.
2. A. Alissandrakis, C. L. Nehaniv, and K. Dautenhahn. Action, state and effect metrics for robot imitation. In 15th IEEE International Symposium on Robot and Human Interactive Communication (ROMAN 2006), pages 232-237, Hatfield, September 2006.
3. R. Amit and M. Mataric. Parametric primitives for motor representation and control. In Int. Conf. on Robotics and Automation (ICRA), Washington DC, May 2002.
4. B. D. Argall, S. Chernova, M. Veloso, and B. Browning. A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5):469-483, May 2009.
5. R. C. Arkin. Behaviour-Based Robotics. MIT Press, 1998.
6. P. Bakker and Y. Kuniyoshi. Robot see, robot do: An overview of robot imitation. In Proceedings of the AISBWorkshop on Learning in Robots and Animals, pages 3-11, Brighton, 1996.
7. D. Baldwin, A. Andersson, J. Saffran, and M. Meyer. Segmenting dynamic human action via statistical structure. Cognition, 106(3): 1382-1407, March 2008.
8. D. C. Bentivegna. Learning from Observation using Primitives. PhD thesis, College of Computing, Georgia Institute of Technology, 2004.
9. D. C. Bentivegna, C. G. Atkeson, and G. Cheng. Learning similar tasks from observation and practice. In Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2677-2683, Beijing, China, October 2006.
10. A. Billard, Y. Epars, G. Cheng, and S. Schaal. Discovering imitation strategies through categorization of multi-dimensional data. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, volume 3, pages 2398-2403 vol. 3, 2003.
11. A. Billard, Y. Epars, S. Calinon, S. Schaal, and G. Cheng. Discovering optimal imitation strategies. Robotics and Autonomous Systems, 47(2-3):69-77, June 2004.
12. A. Billard, S. Calinon, R. Dillmann, and S. Schaal. Robot programming by demonstration. In B. Siciliano and O. Khatib, editors, Handbook of Robotics. Springer, 2008.
13. E. A. Billing. Cognition Reversed-Robot Learning from Demonstration. PhD thesis, Umea University, Department of Computing Science, Umea, Sweden, December 2009.
14. E. A. Billing. Cognitive perspectives on robot behavior. In J. Filipe, A. Fred, and B. Sharp, editors, Proceedings of 2nd International Conference on Agents and Artificial Intelligence (ICAART), Special Session LAMAS, pages 373-382, Valencia, Spain, January 2010.
15. E. A. Billing and T. Hellström. Behavior recognition for segmentation of demonstrated tasks. In IEEE SMC International Conference on Distributed Human-Machine Systems, pages 228-234, Athens, Greece, March 2008.
16. E. A. Billing, T. Hellström, and L. E. Janlert. Model-free learning from demonstration. In J. Filipe, A. Fred, and B. Sharp, editors, Proceedings of 2nd International Conference on Agents and Artificial Intelligence (ICAART), pages 62-71, Valencia, Spain, January 2010.
17. E. A. Billing, T. Hellström, and L. E. Janlert. Behavior recognition for learning from demonstration. In Proceedings of IEEE International Conference on Robotics and Automation, Anchorage, Alaska, May 2010.
18. C. Breazeal and B. Scassellati. Challanges in building robots that imitate people. In K. Dautenhahn and C. L. Nehahiv, editors, Imitation in Animals and Artifacts. MIT Press, 2002.
19. C. Breazeal and B. Scassellati. Infant-like social interactions between a robot and a human caretaker. Adaptive Behavior, 8(1): 49-74, 1998.
20. C. Breazeal and B. Scassellati. Robots that imitate humans. Trends in Cognitive Sciences, 6(11):481-487, November 2002.
21. R. A. Brooks. New approaches to robotics. Science, 253(13): 1227-1232, 1991.
22. R.W. Byrne and A. E. Russon. Learning by imitation: A hierarchical approach. The Journal of Behavioral and Brain Sciences, 16(3), 1998.
23. S. Calinon and A. Billard. Recognition and reproduction of gestures using a probabilistic framework combining PCA, ICA and HMM. In Proceedings of the 22nd international conference on Machine learning, pages 105-112, Bonn, Germany, 2005. ACM.
24. S. Calinon, F. Guenter, and A. Billard. On learning, representing and generalizing a task in a humanoid robot. IEEE Transactions on Systems, Man and Cybernetics, Part B. Special issue on robot learning by observation, demonstration and imitation, 37(2):286-298, 2007.
25. P. Cohen, N. Adams, and H. B. Voting experts: An unsupervised algorithm for segmenting. Intelligent Data Analysis, 11(6):607-625, 2007.
26. A. Cypher, editor. Watch What I Do: Programming by Demonstration. MIT Press, 1993.
27. T. S. Dahl. Behavior-Based Learning. PhD thesis, Faculty of Engineering, University of Bristol, UK, 2002.
28. N. Delson and H. West. Robot programming by human demonstration: The use of human inconsistency in improving 3D robot trajectories. In Proceedings of the IEEE/RSJ/GI International Conference on Intelligent Robots and Systems ?94. Advanced Robotic Systems and the Real World, IROS ?94., volume 2, pages 1248-1255, Munich, Germany, September 1994.
29. J. Demiris and G. Hayes. Do robots ape? In Proceedings of the AAAI Fall Symposium on Socially Intelligent Agents, pages 28-31, 1997.
30. Y. Demiris and A. Dearden. From motor babbling to hierarchical learning by imitation: A robot developmental pathway. In Proceedings of the 5th International Workshop on Epigenetic Robotics, pages 31-37, 2005.
31. Y. Demiris and M. Johnson. Distributed, predictive perception of actions: A biologically inspired robotics architecture for imitation and learning. Connection Science, 15(4):231-243, 2003.
32. Y. Demiris and B. Khadhouri. Hierarchical attentive multiple models for execution and recognition of actions. Robotics and Autonomous Systems, 54(5):361-369, May 2006.
33. R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification (2nd Edition). Wiley-Interscience, 2001.
34. A. Fod, M. Mataric, and O. C. Jenkins. Automated derivation of primitives for movement classification. Autonomous Robots, pages 39-54, 2002.
35. C. Giovannangeli and P. Gaussier. Human-Robot interactions as a cognitive catalyst for the learning of behavioral attractors. In 16th IEEE International Symposium on Robot and Human interactive Communication (RO-MAN 2007), pages 1028-1033, August 2007.
36. S. F. Giszter, F. A. Mussa-Ivaldi, and E. Bizzi. Convergent force field organized in the frog?s spinal cord. Journal of Neuroscience, 13(2):467-491, 1993.
37. J. G. Greeno. Special issue on situated action. In Cognitive Science, volume 17, pages 1-147. Ablex Publishing Corporation, Norwood, New Jersey, 1993.
38. F. Guenter, M. Hersch, S. Calinon, and A. Billard. Reinforcement learning for imitating constrained reaching movements. RSJ Advanced Robotics, Special Issue on Imitative Robots, 21(13): 1521-1544, 2007.
39. M. Haruno, D. M. Wolpert, and M. M. Kawato. MOSAIC model for sensorimotor learning and control. Neural Comput., 13(10): 2201-2220, 2001.
40. M. Haruno, D. M. Wolpert, and M. Kawato. Hierarchical MOSAIC for movement generation. In International Congress Series 1250, pages 575-590. Elsevier Science B.V., 2003.
41. T. Hastie, R. Tibshirani, and J. H Friedman. The Elements of Statistical Learning. Springer, August 2001.
42. T. Hellström. Teaching a robot to behave like a cockroach. In Proceedings of the Third International Symposium on Imitation in Animals and Artifacts in Hatfield UK, pages 54-61, 2005.
43. T. Hellström, T. Johansson, and O. Ringdahl. Development of an autonomous forest machine for path tracking. In P. Corke and S. Sukkariah, editors, Field and Service Robotics-Results of the 5th International Conference FSR, volume 25 of Springer Tracts in Advanced Robotics, pages 603-614. Springer, 2006.
44. M. Hersch, F. Guenter, S. Calinon, and A. Billard. Dynamical system modulation for robot learning via kinesthetic demonstrations. Proceedings of IEEE Transactions on Robotics, 24(6): 1463-1467, 2008.
45. E. Hutchins. Cognition in the Wild. MIT Press, Cambridge, Massachusetts, 1995.
46. R. A. Peters II and C. L. Campbell. Robonaut task learning through teleoperation. In Proceedings of the 2003 IEEE, International Conference on Robotics and Automation, pages 23-27, Taipei, Taiwan, September 2003.
47. L. E. Janlert. Modeling change-The frame problem. In Z. W. Pylyshyn, editor, The Robot?s Dilemma, pages 1-41. Ablex Publishing, Norwood, New Jersey, 1987.
48. K-Team. Khepera robot. http://www.k-Team.com, 2007.
49. H. Kadone and Y. Nakamura. Segmentation, memorization, recognition and abstraction of humanoid motions based on correlations and associative memory. In Proceedings of the 6th IEEERAS International Conference on Humanoid Robots, pages 1-6, University of Genova, Genova, Italy, 2006.
50. H. Kadone and Y. Nakamura. Symbolic memory for humanoid robots using hierarchical bifurcations of attractors in nonmonotonic neural networks. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2900-2905, Edmonton, AB, Canada, 2005.
51. N. Koenig and M. J. Mataric. Behavior-Based segmentation of demonstrated tasks. In International Conference on Development and Learning (ICDL), Bloomington, USA, May 2006.
52. D Kulic, W Takano, and Y Nakamura. Incremental learning, clus-Tering and hierarchy formation of whole body motion patterns using adaptive hidden markov chains. The International Journal of Robotics Research, 27(7):761-784, July 2008.
53. S. M. LaValle. Planning Algorithms. Cambridge University Press, Cambridge, U.K., 2006. Available at http://planning.cs.uiuc.edu/.
54. H. Lieberman, editor. Your Wish is My Command: Programming by Example. Morgan Kaufmann, San Francisco, 2001.
55. L. Ljung. System Identification. Prentice-Hall, Simon & Schuster, Englewood Cliffs, New Jersey, 1987.
56. P. Maes and R. A. Brooks. Learning to coordinate behaviors. In National Conference on Artificial Intelligence (AAAI), pages 796-802, 1990.
57. P. Martin and U. Nehmzow. Programming by teaching: Neural network control in themanchestermobile robot. In Proc. Intelligent Autonomous Vehicles, Helsinki. Springer Verlag, 1995.
58. M. J. Mataric. Behavior-Based control: Examples from navigation, learning, and group behavior. Journal of Experimental and Theoretical Artificial Intelligence, 9(2-3):323-336, 1997.
59. M. J. Mataric. Designing and understanding adaptive group behavior. Adaptive Behavior, 4(1):51-80, 1995.
60. M. J. Mataric. Integration of representation into Goal-Driven Behavior-Based robots. In IEEE Transactions on Robotics and Automation, volume 8, pages 304-312, 1992.
61. M. J. Mataric and M. J. Marjanovic. Synthesizing complex behaviors by composing simple primitives. In Proceedings of the European Conference on Artificial Life (ECAL-93), volume 2, pages 698-707, Brussels, Belgium, May 1993.
62. J. McCarthy and P. J. Hayes. Some philosophical problems from the standpoint of artificial intelligence. In B. Meltzer and D. Michie, editors, Machine Intelligence 4, pages 463-502. Edinburgh University Press, 1969.
63. T. M. Mitchell. The need for biases in learning generalizations. Technical Report CBM-TR-117, Rutgers Computer Science Department Technical Report, New Brunswick, New Jersey, 1980.
64. F. A. Mussa-Ivaldi and S. F. Giszter. Vector field approximation: A computational paradigm for motor control and learning. Biological cybernetics, 67:479-489, 1992.
65. S. Nakaoka, A. Nakazawa, K. Yokoi, and K. Ikeuchi. Recognition and generation of leg primitivemotions for dance imitation by a humanoid robot. In Proceedings of 2nd International Symposium on Adaptive Motion of Animals and Machines, Kyoto, Japan, 2003.
66. C. L. Nehaniv and K. Dautenhahn. The correspondence problem. In K. Dautenhahn and C. L. Nehahiv, editors, Imitation in Animals and Artifacts. MIT Press, 2002.
67. C. L. Nehaniv and K. Dautenhahn. Of hummingbirds and helicopters: An algebraic framework for interdisciplinary studies of imitation and its applications. In J. Demiris and A. Birk, editors, Learning Robots: An Interdisciplinary Approach, volume 24, pages 136-161. World Scientific Press, 2000.
68. M. Nicolescu. A Framework for Learning from Demonstration, Generalization and Practice in Human-Robot Domains. PhD thesis, University of Southern California, 2003.
69. A. Olenderski, M. Nicolescu, and S. Louis. Robot learning by demonstration using forward models of Schema-Based behaviors. In Proceedings of International Conference on Informatics in Control, Automation and Robotics, Barcelona, Spain, 2005.
70. N. Otero, J. Saunders, K. Dautenhahn, and C. L. Nehaniv. Teaching robot companions: The role of scaffolding and event structuring. Connection Science, 20:111-134, June 2008.
71. J. Peters and S. Schaal. Policy learning for motor skills. In Proceedings of 14th International Conference on Neural Information Processing (ICONIP 2007), pages 1-10, Berlin, Germany, November 2007. Springer.
72. R. Pfeifer and C. Scheier. Sensory-motor coordination: The metaphor and beyond. Robotics and Autonomous Systems, 20 (2):157-178, June 1997.
73. R. Pfeifer and C. Scheier. Understanding Intelligence. MIT Press. Cambrage, Massachusetts, 2001.
74. P. K. Pook and D. H. Ballard. Recognizing teleoperated manipulations. In Proceedings of the IEEE International Conference on Robotics and Automation, pages 578-585, 1993.
75. B. Rohrer and S. Hulet. BECCA-A brain emulating cognition and control architecture. Technical report, Cybernetic Systems Integration Department, Univeristy of Sandria National Laboratories, Alberquerque, NM, USA, 2006.
76. B. Rohrer and S. Hulet. A learning and control approach based on the human neuromotor system. In Proceedings of Biomedical Robotics and Biomechatronics, BioRob, 2006.
77. S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, NJ, 1995.
78. J. Saunders, C. L. Nehaniv, and K. Dautenhahn. Using Self-Imitation to direct learning. In 15th IEEE International Symposium on Robot and Human Interactive Communication, pages 244-250, 2006.
79. J. Saunders, C. L. Nehaniv, K. Dautenhahn, and A. Alissandrakis. Self-Imitation and environmental scaffolding for robot teaching. International Journal of Advanced Robotics Systems, 4(1):109-124, 2007.
80. B. Scassellati. Imitation and mechanisms of joint attention: A developmental structure for building social skills on a humanoid robot. Lecture Notes in Computer Science, 1562:176-195, 1999.
81. H. A. Simon. The Sciences of the Artificial. MIT Press, Cambridge, Massachusetts, 1969.
82. L. A. Suchman. Plans and Situated Actions. PhD thesis, Intelligent Systems Laboratory, Xerox Palo Alto Research Center, USA, 1987.
83. J. Tani. On the interactions between top-down anticipation and bottom-up regression. Frontiers in Neurorobotics, 1:2, 2007.
84. J. Tani and M. Ito. Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment. IEEE Trans. on Systems, Man, and Cybernetics Part A: Systems and Humans, 33(4):481-488, 2003.
85. D. H. Wolpert and W. G Macready. No free lunch theorems for optimization. In IEEE Transactions on Evolutionary Computation, volume 1, pages 67-82, April 1997.
86. D. M.Wolpert. A unifying computational framework for motor control and social interaction. Phil. Trans. R. Soc. Lond., B(358):593-602, March 2003.
87. D. Wood, J. Bruner, and G. Ross. The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17:89-100, 1976.