Robot learning from human teachers

Synthesis Lectures on Artificial Intelligence and Machine Learning

Volumn 28, Issue , 2014, Pages 1-121

  Chernova, Sonia ^a   L Thomaz, Andrea ^b  

^a WORCESTER POLYTECHNIC INSTITUTE   (United States)

^b GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Human robot Interaction; Imitation learning; Learning from Demonstration

Indexed keywords

HUMAN ROBOT INTERACTION; ROBOT LEARNING; ROBOTICS; SURVEYS; ARTIFICIAL INTELLIGENCE; DEMONSTRATIONS; EDUCATION; ROBOTS; TEACHING;

HUMAN DEMONSTRATIONS; IMITATION LEARNING; LEARNING FROM DEMONSTRATION; MODELING TECHNIQUE; PRIMITIVE ACTIONS; SOCIAL LEARNING; STATE-OF-THE-ART APPROACH; TECHNICAL CHALLENGES; SAMPLE LEARNING;

EDUCATION; HUMAN ROBOT INTERACTION;

EID: 84899837449     PISSN: 19394608     EISSN: 19394616     Source Type: Book Series    
DOI: 10.2200/S00568ED1V01Y201402AIM028     Document Type: Article

References (274)

1. P. Abbeel, A. Coates, M. Quigley, and A. Y. Ng. An application of reinforcement learning to aerobatic helicopter flight. In Neural Information Proccessing (NIPS07), 2007. 18, 19, 44
2. P. Abbeel, A. Coates, and A. Y. Ng. Autonomous Helicopter Aerobatics through Apprenticeship Learning. The International Journal of Robotics Research, 29(13):1608-1639, June 2010. DOI: 10.1177/0278364910371999. 26, 44
3. P. Abbeel and A. Ng. Apprenticeship learning via inverse reinforcement learning. In Proceedings of the 21st International Conference on Machine Learning, 2004. DOI: 10.1145/1015330.1015430. 19, 44
4. D. W. Aha, D. Kibler, and M. K. Albert. Instance-based learning algorithms. Machine learning, 6(1):37-66, 1991. DOI: 10.1023/A:1022689900470. 39
5. B. Akgun, M. Cakmak, J. Yoo, and A. L. Thomaz. Trajectories and Keyframes for Kinesthetic Teaching: A Human-Robot Interaction Perspective. In Proceedings of theInternational Conference on Human-Robot Interaction, 2012. DOI: 10.1145/2157689.2157815. 34
6. E. E. Aksoy, a. Abramov, J. Dorr, K. Ning, B. Dellen, and F. Worgotter. Learning the semantics of object-Action relations by observation. The International Journal of Robotics Research, 30(10):1229-1249, August 2011. DOI: 10.1177/0278364911410459. 48
7. R. Aler, O. Garcia, and J. M. Valls. Correcting and improving imitation models of humans for Robosoccer agents. Evolutionary Computation, 3(2-5):2402-2409, September 2005. DOI: 10.1109/CEC.2005.1554994. 19
8. Javier Almingol, Luis Montesano, and Manuel Lopes. Learning multiple behaviors from unlabeled demonstrations in a latent controller space. In International Conference on Machine Learning, 2013. 29
9. Ethem Alpaydin. Introduction to Machine Learning. MIT Press, 2004. 38
10. R Amit and Maja Mataric. Learning movement sequences from demonstration. In ICDL 02: Proceedings of the 2nd International Conference on Development and Learning, page 203, Washington, DC, USA, 2002. IEEE Computer Society. DOI: 10.1109/DEVLRN. 2002.1011867. 22
11. B. Argall, B. Browning, and M. Veloso. Learning from Demonstration with the Critique of a Human Teacher. In Second Annual Conference on Human-Robot Interactions, Arlington, Virginia, March 2007. DOI: 10.1145/1228716.1228725. 19
12. B. D. Argall, S. Chernova, M. Veloso, and B. Browning. A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5):469-483, 2009. DOI: 10.1016/j.robot.2008.10.024. 81
13. B. D. Argall, E. L. Sauser, and A. G. Billard. Tactile Guidance for Policy Adapatation. pages 79-133, 2010. 10, 19
14. Brenna Argall and Aude Billard. Learning from Demonstration and Correction via Multiple Modalities for a Humanoid Robot. In The International Conference SKILLS, volume 1, 2011. DOI: 10.1051/bioconf/20110100003. 10, 34, 54
15. Brenna D. Argall. Learning Mobile Robot Motion Control from Demonstration and Corrective Feedback. Ph.D. thesis, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, 2009. 59
16. A. Aron, E. N. Aron, and E. J. Coups. Statistics for Psychology, 5th Ed. Pearson Prentice Hall, New Jersey, 2009. 76, 78
17. H. Asada. Teaching and learning of compliance using neural nets: Representation and generation of nonlinear compliance. In Robotics and Automation, 1990. Proceedings., 1990 IEEE International Conference on, pages 1237-1244 vol.2, 1990. DOI: 10.1109/ROBOT.1990.126167. 25, 26
18. Tamim Asfour, Pedram Azad, Florian Gyarfas, and Rüdiger Dillmann. Imitation learning of dual-Arm manipulation tasks in humanoid robots. International Journal of Humanoid Robotics, 5(02):183-202, 2008. DOI: 10.1142/S0219843608001431. 32
19. C. G. Atkeson and S. Schaal. Learning Tasks From A Single Demonstration. In IEEE International Conference on Robotics and Automation, number April, 1997. DOI: 10.1109/ROBOT.1997.614389. 43
20. Christopher G Atkeson and Stefan Schaal. Robot learning from demonstration. In Douglas H. Fisher, Jr., editor, Proc. 14th International Conference on Machine Learning, pages 12-20. Morgan Kaufmann, 1997. 28
21. Monica Babes, Vukosi Marivate, Kaushik Subramanian, and Michael L Littman. Apprenticeship learning about multiple intentions. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 897-904, 2011. 44
22. C. Bartneck, T. Suzuki, T. Kanda, and T. Nomura. The influence of peoples culture and prior experiences with aibo on their attitude towards robots. AI and Society, 21(1):217-230, 2007. DOI: 10.1007/s00146-006-0052-7. 70
23. R. Bellamy. Designing Educational Technology: Computer-Mediated Change. In B Nardi, editor, Context and Consciousness: Activity Theory and Human-Computer Interaction. MIT Press, Cambridge, MA, 1996. 11
24. M. Bennewitz. Learning Motion Patterns of People for Compliant Robot Motion. The International Journal of Robotics Research, 24(1):31-48, January 2005. DOI: 10.1177/0278364904048962. 10
25. D. C. Bentivegna, A. Ude, C. G. Atkeson, and G. Cheng. Humanoid Robot Learning and Game Playing Using PC-Based Vision. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Swiss Federal Institute of Technology Lausanne, Switzerland, October 2002. DOI: 10.1109/IRDS.2002.1041635. 21, 22
26. M. Berlin, C. Breazeal, and C. Chao. Spatial scaffolding cues for interactive robot learning. In Proceedings of the23st Conference on Artificial Intelligence, 2008. DOI: 10.1109/IROS.2008.4651180. 10
27. C.L. Bethel, J.L. Burke, R.R. Murphy, and K. Salomon. Psychophysiological experimental design for use in human-robot interaction studies. In Proceedings of the International Symposium on Collaborative Technologies and Systems, 2007. DOI: 10.1109/CTS.2007.4621744. 78
28. T. W. Bickmore and R. W. Picard. Towards caring machines. In CHI04 Extended Abstracts on Human Factors in Computing Systems, pages 1489-1492. ACM, 2004. DOI: 10.1145/985921.986097. 70
29. A. Billard, Y. Epars, G. Cheng, and S. Schaal. Discovering Imitation Strategies through Categorization of Multi-Dimensional Data. In Proceedings of the 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, Las Vegas, Nevada, October 2003. DOI: 10.1109/IROS.2003.1249229. 22
30. A. Billard and M. Mataric. Learning Human Arm Movements by Imitation: Evaluation of Biologically Inspired Connectionist Architecture. Robotics and Autonomous Systems, 37(2-3):145-160, November 2001. DOI: 10.1016/S0921-8890(01) 00155-5. 22
31. Aude Billard, Sylvain Callinon, Rudiger Dillmann, and Stefan Schaal. Robot programming by demonstration. In B Siciliano and O Khatib, editors, Handbook of Robotics, chapter 59. Springer, New York, NY, USA, 2008. DOI: 10.1007/978-3-540-30301-5. 81
32. Erik A Billing and Thomas Hellström. A formalism for learning from demonstration. Paladyn. Journal of Behavioral Robotics, 1(1):1-13, 2010. DOI: 10.2478/s13230-010-0001-5. 81
33. B. Blumberg, M. Downie, Y. Ivanov, M. Berlin, M. P. Johnson, and B. Tomlinson. Integrated learning for interactive synthetic characters. In Proceedings of theACM SIGGRAPH, 2002. DOI: 10.1145/566654.566597. 23, 55
34. M. Bratman. Shared cooperative activity. The Philosophical Review, 101(2):327-341, 1992. DOI: 10.2307/2185537. 12
35. C Breazeal. Designing Sociable Robots. MIT Press, Cambridge, MA, May 2002. 9
36. C. Breazeal, A. Brooks, J. Gray, G. Hoffman, J. Lieberman, H. Lee, A. Thomaz, and D. Mulanda. Tutelage and Collaboration for Humanoid Robots. International Journal of Humanoid Robotics, 1(2), 2004. DOI: 10.1142/S0219843604000150. 20, 43, 59
37. C. Breazeal, D. Buchsbaum, J. Grey, and B. Blumberg. Learning from and about others: Toward using imitation to bootstrap the social competence of robots. Artificial Life, 11, 2005. DOI: 10.1162/1064546053278955. 22
38. C. Breazeal, N. DePalma, J. Orkin, S. Chernova, and M. Jung. Crowdsourcing humanrobot interaction: New methods and system evaluation in a public environment. Journal of Human-Robot Interaction, 2(1):82-111, 2013. DOI: 10.5898/JHRI.2.1.Breazeal. 39, 50
39. C. Breazeal, J. Gray, and M. Berlin. An embodied cognition approach to mindreading skills for socially intelligent robots. The International Journal of Robotics Research, 28(5):656-680, May 2009. DOI: 10.1177/0278364909102796. 43
40. B. Browning, L. Xu, and M. Veloso. Skill acquisition and use for a dynamically-balancing soccer robot. In Proceedings of Nineteenth National Conference on Artificial Intelligence, 2004. 19
41. B. G. Buchanan and T. M. Mitchell. Model-directed learning of production rules. In D. Watermah and F. Hayes-Roth, editors, Pattern-directed inference systems. Academic Press, New York, 1978. 43
42. R. R. Burton, J. S. Brown, and G. Fischer. Skiing as a model of instruction. In B Rogoff and J Lave, editors, Everyday cognition: Its development in social context. Harvard University Press, Cambridge, MA, 1984. 10
43. Jesse Butterfield, Sarah Osentoski, Graylin Jay, and Odest Chadwicke Jenkins. Learning from demonstration using a multi-valued function regressor for time-series data. In Humanoid Robots (Humanoids), 2010 10th IEEE-RAS International Conference on, pages 328-333. IEEE, 2010. DOI: 10.1109/ICHR.2010.5686284. 50
44. M. Cakmak. Guided teaching interactions with robots: Embodied queries and teaching heuristics. Ph.D. thesis, Georgia Institute of Technology, 2012. 61, 63
45. M. Cakmak, C. Chao, and A. L. Thomaz. Designing interactions for robot active learners. IEEE Transactions on Autonomous Mental Development, 2(2):108-118, 2010. DOI: 10.1109/TAMD.2010.2051030. 54, 60
46. M. Cakmak, N. DePalma, R. Arriaga, and A. L. Thomaz. Social learning mechanisms for robots. Autonomous Robots, 2010. DOI: 10.1007/s10514-010-9197-9. 8
47. M. Cakmak and A. L. Thomaz. Learning about objects with human teachers. In Proceedings of theInternational Conference on Human-Robot Interaction, 2009. DOI: 10.1145/1514095.1514101. 48
48. M. Cakmak and A. L. Thomaz. Designing robot learners that ask good questions. In Proceedings of theInternational Conference on Human-Robot Interaction, 2012. DOI: 10.1145/2157689.2157693. 61, 63
49. S. Calinon. Robot Programming by Demonstration: A Probabilistic Approach. EPFL/CRC Press, 2009. EPFL Press ISBN 978-2-940222-31-5, CRC Press ISBN 978-1-4398-0867-2. 18, 81
50. S. Calinon and A. Billard. Incremental learning of gestures by imitation in a humanoid robot. In In Proc. of the ACM/IEEE International Conference on Human-Robot Interaction, pages 255-262, 2007. DOI: 10.1145/1228716.1228751. 22
51. S Calinon and A Billard. What is the teachers role in robot programming by demonstration?-Toward benchmarks for improved learning. Interaction Studies. Special Issue on Psychological Benchmarks in Human-Robot Interaction, 8(3), 2007. 18, 20, 34
52. S. Calinon and A. Billard. A probabilistic programming by demonstration framework handling skill constraints in joint space and task space. In Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008. DOI: 10.1109/IROS.2008.4650593. 34
53. S. Calinon and A. Billard. Statistical learning by imitation of competing constraints in joint space and task space. Advanced Robotics, 23(15):2059-2076, 2009. DOI: 10.1163/016918609X12529294461843. 26, 62
54. S. Calinon, F. Dhalluin, E.L. Sauser, D.G. Caldwell, and A.G. Billard. Learning and reproduction of gestures by imitation. Robotics Automation Magazine, IEEE, 17(2):44-54, 2010. DOI: 10.1109/MRA.2010.936947. 34
55. S. Calinon, F. Guenter, and A. Billard. On learning, representing and generalizing a task in a humanoid robot. IEEE Transactions on Systems, Man and Cybernetics, Part B. Special issue on robot learning by observation, demonstration and imitation, 37(2):286-298, 2007. DOI: 10.1109/TSMCB.2006. 886952. 33, 34
56. J. Call and M. Carpenter. Three sources of information in social learning. In K. Dautenhahn and C.L. Nehaniv, editors, Imitation in animals and artifacts. MIT Press, 2002. 8
57. C. Cazden. Performance before competence: Assistance to child in the ZPD. In M Cole, Y Engeström, and O Vasquez, editors, Mind, Culture, and Activity: Seminal Papers from the Laboratory of Comparative Human Cognition. Cambridge University Press, Cambridge, 1997. 11
58. T. Cederborg, M. Li, A. Baranes, and P.-Y. Oudeyer. Incremental local online gaussian mixture regression for imitation learning of multiple tasks. In Intelligent Robots and Systems , 2010 IEEE/RSJ International Conference on, pages 267-274. IEEE, 2010. DOI: 10.1109/IROS.2010.5652040. 46
59. H. S. Chang. Reinforcement learning with supervision by combining multiple learnings and expert advices. In Proc. of the American Control Conference, 2006. DOI: 10.1109/ACC.2006.1657371. 55
60. C. Chao, M. Cakmak, and A. L. Thomaz. Transparent active learning for robots. In Autonomous Mental Development, special issue on Active Learning, 2010. DOI: 10.1145/1734454.1734562. 60
61. C. Chao, M. Cakmak, and A. L. Thomaz. Towards grounding concepts for transfer in goal learning from demonstration. In International Conference on Development and Learning, 2011. DOI: 10.1109/DEVLRN.2011.6037321. 43
62. C. Chao and A. L. Thomaz. Timing in multimodal turn-Taking interactions: Control and analysis using timed Petri nets. International Journal of Human-Robot Interaction, 1(1):4-25, 2012. DOI: 10.5898/JHRI.1.1.Chao. 7
63. C. Chao and A. L. Thomaz. Controlling social dynamics with a parameterized model of floor regulation. Journal of Human Robot Interaction, 2013. DOI: 10.5898/JHRI.2.1.Chao. 7
64. Jason Chen and Alex Zelinsky. Programing by demonstration: Coping with suboptimal teaching actions. The International Journal of Robotics Research, 22(5):299-319, May 2003. DOI: 10.1177/0278364903022005002. 19, 34
65. N. Chentanez, A. G. Barto, and S. P. Singh. Intrinsically motivated reinforcement learning. In Advances in Neural Information Processing Systems, pages 1281-1288, 2004. DOI: 10.1007/978-3-642-32375-1-2. 8
66. S. Chernova and M. Veloso. Interactive Policy Learning through Confidence-Based Autonomy. Journal of Artificial Intelligence Research, 34, 2009. DOI: 10.1613/jair.2584. 19, 39, 54, 59, 60, 61
67. Sonia Chernova and Manuela Veloso. Learning equivalent action choices from demonstration. In Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on, pages 1216-1221. IEEE, 2008. DOI: 10.1109/IROS.2008.4650995. 50
68. Silvia Chiappa and Jan Peters. Movement extraction by detecting dynamics switches and repetitions. Advances in Neural Information Processing Systems, 23:388-396, 2010. 29
69. J. Clouse and P. Utgoff. A teaching method for reinforcement learning. In Proc. of the Nineth International Conference on Machine Learning, pages 92-101, 1992. 23, 58
70. J. A. Clouse. On integrating apprentice learning and reinforcement learning. Ph.D. thesis, University of Massachisetts, Department of Computer Science, 1996. 19
71. Luis C Cobo, Peng Zang, Charles L Isbell Jr, Andrea L Thomaz, and Charles L Isbell Jr. Automatic state abstraction from demonstration. In Twenty-Second International Joint Conference on Artificial Intelligence, pages 1243-1248. AAAI Press, 2009. 45
72. P. R. Cohen, H. J. Levesque, J. H. T. Nunes, and S. L. Oviatt. Task-oriented dialogue as a consequence of joint activity. In Proceedings of thePacific Rim International Conference on Artificial Intelligence, Nagoya, Japan, November 1990. 11
73. D. A. Cohn, Z. Ghahramani, and M. I. Jordan. Active learning with statistical models. Journal of Artificial Intelligence Research, 4:129-145, 1996. 60
74. K. Conn and R. A. Peters. Reinforcement learning with a supervisor for a mobile robot in a real-world environment. In Computational Intelligence in Robotics and Automation, 2007. CIRA 2007. International Symposium on, pages 73-78. IEEE, 2007. DOI: 10.1109/CIRA.2007.382878. 58
75. C Crick, S Osentoski, G Jay, and O Jenkins. Human and robot perception in large-scale learning from demonstration. In ACM/IEEE International Conference on Human-Robot Interaction (HRI 2011), 2011. DOI: 10.1145/1957656.1957788. 38
76. G. Csibra. Teleological and referential understanding of action in infancy. Phil. Trans. The Royal Society of London, 358:447-458, 2003. DOI: 10.1098/rstb.2002.1235. 10
77. K. Dautenhahn and C. L. Nehaniv, editors. Imitation in animals and artifacts. MIT Press, Cambridge, MA, USA, 2002. 81
78. K. Dautenhahn and I. Werry. A quantitative technique for analysing robot-human interactions. In Intelligent Robots and Systems, 2002. IEEE/RSJ International Conference on, volume 2, pages 1132-1138 vol.2, 2002. DOI: 10.1109/IRDS.2002.1043883. 71
79. M. P. Deisenroth, C. E. Rasmussen, and D. Fox. Learning to control a low-cost manipulator using data-efficient reinforcement learning. In Robotics: Science & Systems, 2011. 22
80. J. Demiris and G. Hayes. Imitation as a dual-route process featuring predictive and learning components: A biologically plausible computational model. In K Dautenhahn and C L Nehaniv, editors, Imitation in Animals and Artifacts. MIT Press, Cambridge, 2002. 22
81. A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society. Series B (Methodological), pages 1-38, 1977. DOI: 10.2307/2984875. 31
82. R Dillmann. Teaching and learning of robot tasks via observation of human performance. Robotics and Autonomous Systems, 47(2-3):109-116, 2004. DOI: 10.1016/j.robot.2004.03.005. 41
83. K. R. Dixon. Predictive robot programming: Theoretical and experimental analysis. The International Journal of Robotics Research, 23(9):955-973, September 2004. DOI: 10.1177/0278364904044401. 34
84. Shuonan Dong and Brian Williams. Motion learning in variable environments using probabilistic flow tubes. In Robotics and Automation , 2011 IEEE International Conference on, pages 1976-1981. IEEE, 2011. DOI: 10.1109/ICRA.2011.5980530. 46
85. Shuonan Dong and Brian Williams. Learning and recognition of hybrid manipulation motions in variable environments using probabilistic flow tubes. International Journal of Social Robotics, 4(4):357-368, 2012. DOI: 10.1007/s12369-012-0155-x. 46, 47
86. John V Draper and Linda M Blair. Workload, flow, and telepresence during teleoperation. In Robotics and Automation, 1996. Proceedings. 1996 IEEE International Conference on, volume 2, pages 1030-1035. IEEE, 1996. DOI: 10.1109/ROBOT.1996.506844. 70
87. G. Druck, B. Settles, and A. McCallum. Active learning by labeling features. In In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 81-90, 2009. 62
88. B. Dufay and J.-C. Latombe. An approach to automatic robot programming based on inductive learning. The International Journal of Robotics Research, 3(4):3-20, 1984. DOI: 10.1177/027836498400300401. 26
89. Bruno Dufay and Jean-Claude Latombe. An approach to automatic robot programming based on inductive learning. The International Journal of Robotics Research, 3(4):3-20, 1984. DOI: 10.1177/027836498400300401. 25
90. M Ehrenmann, R Zollner, O Rogalla, and R Dillmann. Programming service tasks in household environments by human demonstration. In Robot and Human Interactive Communication, 2002. Proceedings 11th IEEE International Workshop on, pages 460-467. IEEE, 2002. DOI: 10.1109/ROMAN.2002.1045665. 41
91. R. Evans. Varieties of Learning. In S Rabin, editor, AI Game Programming Wisdom, pages 567-578. Charles River Media, Hingham, MA, 2002. 23
92. P. Fitzpatrick, G. Metta, L. Natale, S. Rao, and G. Sandini. Learning about objects through action-initial steps towards artificial cognition. In Robotics and Automation, 2003. Proceedings. ICRA03. IEEE International Conference on, volume 3, pages 3140-3145. IEEE, 2003. DOI: 10.1109/ROBOT.2003. 1242073. 47
93. Terrence Fong, Illah Nourbakhsh, Robert Ambrose, Reid Simmons, Alan Schultz, and Jean Scholtz. The peer-To-peer human-robot interaction project. In AIAA Space, volume 2005, 2005. 70
94. H. Friedrich, S. Münch, R. Dillmann, S. Bocionek, and M. Sassin. Robot programming by demonstration (rpd): Supporting the induction by human interaction. Machine Learning, 23(2-3):163-189, 1996. DOI: 10.1007/BF00117443. 43
95. A. Garland and N. Lesh. Learning hierarchical task models by demonstration. Technical Report TR2003-01, Mitsubishi Electric Research Laboratories, 2003. 40
96. J. J. Gibson. The Ecological Approach to Visual Perception. Routledge, 1986. 47
97. A. Gopnik, D. Sobel, L. Schulz, and C. Glymour. Causal learning mechanisms in very young children: Two, three, and four-year-olds infer causal relations from patterns of variation and covariation. Developmental Psychology, 37(5):620-629, 2001. DOI: 10.1037/0012-1649.37.5.620. 12
98. A. C. Graesser and N. K. Person. Question asking during tutoring. American Educational Research Journal, 31(1):104-137, 1994. DOI: 10.3102/00028312031001104. 12
99. P. M. Greenfield. Theory of the teacher in learning activities of everyday life. In B Rogoff and J Lave, editors, Everyday cognition: Its development in social context. Harvard University Press, Cambridge, MA, 1984. 7, 9
100. E Gribovskaya, Florent DHalluin, and Aude Billard. An active learning interface for bootstrapping robots generalization abilities in learning from demonstration. In RSS Workshop Towards Closing the Loop: Active Learning for Robotics, 2010. 62
101. D. H. Grollman and A. G. Billard. Learning from failure. In Proceedings of the 6th International Conference on Human-Robot Interaction, pages 145-146. ACM, 2011. 62
102. D. H. Grollman and O. C. Jenkins. Dogged learning for robots. In Proceedings of the IEEE International Conference on Robotics and Automation, Roma, Italy, April 2007. DOI: 10.1109/ROBOT.2007.363692. 19, 24, 39, 54, 60
103. Daniel H. Grollman and Aude G. Billard. Robot learning from failed demonstrations. International Journal of Social Robotics, 4(4):331-342, June 2012. DOI: 10.1007/s12369-012-0161-z. 35
104. G. Z. Grudic and P. D. Lawrence. Human-To-robot skill transfer using the SPORE approximation. In Proceedings of the IEEE International Conference on Robotics and Automation, Minneapolis, Minnnesota, April 1996. DOI: 10.1109/ROBOT.1996.509162. 24
105. F. Guenter and A. G. Billard. Using reinforcement learning to adapt an imitation task. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007. DOI: 10.1109/IROS.2007.4399449. 44
106. Isabelle Guyon and André Elisseeff. An introduction to variable and feature selection. The Journal of Machine Learning Research, 3:1157-1182, 2003. 45
107. P. Hakkarainen. Play and motivation. In Y. Engeström, R. Miettinen, and R-L. Punamäki, editors, Perspectives on Activity Theory. Cambridge University Press, Cambridge, MA, 1996. 7
108. Sandra G Hart and Lowell E Staveland. Development of nasa-Tlx (task load index): Results of empirical and theoretical research. Human Mental Workload, 1(3):139-183, 1988. DOI: 10.1016/S0166-4115(08)62386-9. 70
109. G. Hayes and J. Demiris. A Robot Controller Using Learning by Imitation. In Proceedings of the 2nd International Symposium on Intelligent Robotic Systems, Grenoble, France, July 1994. 20
110. A. Holroyd, C. Rich, C. Sidner, and B. Ponsler. Generating connection events for human-robot collaboration. In Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication, pages 241-246, 2011. DOI: 10.1109/ROMAN. 2011.6005245. 7
111. K. Hornbaek. Some whys and hows of experiments in human-computer interaction. Foundations and Trends in Human-Computer Interaction, 5(4):299-373, 2011. DOI: 10.1561/1100000043. 78
112. A. O. Horvath and L. S. Greenberg. Development and validation of the working alliance inventory. Journal of Counseling Psychology, 36(2):223, 1989. DOI: 10.1037/0022-0167.36.2.223. 70
113. G. Hovland, P. Sikka, and B. McCarragher. Skill acquisition from human demonstration using a hidden markov model. In IEEE International Conference on Robotics and Automation, 1996. DOI: 10.1109/ROBOT.1996.506571. 26
114. A. J. Ijspeert, J. Nakanishi, and S. Schaal. Learning rhythmic movements by demonstration using nonlinear oscillators. In Proceedings of the IEEE/RSJ Int. Conference on Intelligent Robots and Systems, 2002. DOI: 10.1109/IRDS.2002.1041514. 21, 27
115. A. J. Ijspeert, J. Nakanishi, and S. Schaal. Movement imitation with nonlinear dynamical systems in humanoid robots. In Proceedings of the IEEE International Conference on Robotics and Automation, 2002. DOI: 10.1109/ROBOT.2002.1014739. 21
116. Katsushi Ikeuchi. Assembly plan from observation. In Electronic Manufacturing Technology Symposium, 1995, Proceedings of 1995 Japan International, 18th IEEE/CPMT International, pages 9-12. IEEE, 1995. 40
117. M. Imai, T. Kanda, T. Ono, H. Ishiguro, and K. Mase. Robot mediated round table: Analysis of the effect of robots gaze. In Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication, pages 411-416, 2002. DOI: 10.1109/ROMAN. 2002.1045657. 10
118. T. Inamura, M. Inaba, and H. Inoue. Acquisition of probabilistic behavior decision model based on the interactive teaching method. In Proceedings of the Ninth International Conference on Advanced Robotics, pages 523-528, 1999. 19, 39, 59
119. C. Isbell, C. Shelton, M. Kearns, S. Singh, and P. Stone. Cobot: A social reinforcement learning agent. 5th International Conference on Autonomous Agents, 2001. 23
120. C. L. Isbell, C. Shelton, M. Kearns, S. Singh, and P. Stone. A social reinforcement learning agent. In Proc. of the 19th AAAI Conference on Artificial Intelligence, pages 377-384, 2001. 55
121. D. Blank J. B. Marshall and L. Meeden. An emergent framework for self-motivation in developmental robotics. In International Conference on Development and Learning, pages 104-111, 2004. 8
122. O. C. Jenkins and M. Matarić. Deriving action and behavior primitives from human motion data. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2551-2556, 2002. DOI: 10.1109/IRDS.2002.1041654. 22, 26
123. Odest Chadwicke Jenkins and Maja J Matarić. Performance-derived behavior vocabularies: Data-driven acquisition of skills from motion. International Journal of Humanoid Robotics, 2004. DOI: 10.1142/ S0219843604000186. 22
124. Nikolay Jetchev and Marc Toussaint. Task space retrieval using inverse feedback control. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 449-456, 2011. 63
125. Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. Reinforcement learning: A survey. arXiv preprint cs/9605103, 1996. 55
126. M. Kaiser, H. Friedrich, and R. Dillmann. Obtaining good performance from a bad teacher. In Programming by Demonstration vs. Learning from Examples Workshop at ML95, 1995. 34
127. T. Kanda, H. Ishiguro, M. Imai, and T. Ono. Body movement analysis of human-robot interaction. pages 177-182, August 2003. 10
128. T. Kanda, H. Ishiguro, M. Imai, and T. Ono. Development and evaluation of interactive humanoid robots. In Proceedings of the IEEE, volume 92, pages 1839-1850, 2004. DOI: 10.1109/JPROC.2004.835359. 7
129. F. Kaplan, P.-Y. Oudeyer, E. Kubinyi, and A. Miklosi. Robotic clicker training. Robotics and Autonomous Systems, 38(3-4):197-206, 2002. DOI: 10.1016/S0921-8890(02)00168-9. 10, 23, 55
130. K. Kaye. Infants effects upon their mothers teaching strategies. In J Glidewell, editor, The Social Context of Learning and Development. Gardner Press, New York, 1977. 7
131. S. M. Khansari-Zadeh and A. Billard. Learning stable non-linear dynamical systems with Gaussian mixture models. IEEE Transaction on Robotics, 2011. DOI: 10.1109/TRO.2011.2159412. 34
132. C. D. Kidd and C. Breazeal. Robots at home: Understanding long-Term human-robot interaction. In Intelligent Robots and Systems, pages 3230-3235. IEEE, 2008. DOI: 10.1109/IROS.2008.4651113. 70
133. Hedvig Kjellström, Javier Romero, and Danica Kragić. Visual object-Action recognition: Inferring object affordances from human demonstration. Computer Vision and Image Understanding, 115(1):81-90, 2011. DOI: 10.1016/j.cviu.2010.08.002. 48, 49
134. W. B. Knox and P. Stone. Tamer: Training an agent manually via evaluative reinforcement. In Proc. of the 7th IEEE International Conference on Development and Learning, pages 292-297, 2008. 55, 56
135. W. B. Knox and P. Stone. Interactively shaping agents via human reinforcement: The tamer framework. In The Fifth International Conference on Knowledge Capture, September 2009. DOI: 10.1145/1597735.1597738. 23, 56
136. W. B. Knox and P. Stone. Combining manual feedback with subsequent mdp reward signals for reinforcement learning. In Proc. of the 9th International Conference on Autonomous Agents and Multiagent Systems, pages 5-12, 2010. DOI: 10.1145/1838206.1838208. 58
137. W. B. Knox and P. Stone. Reinforcement learning from simultaneous human and mdp reward. In Proc. of the 11th International Conference on Autonomous Agents and Multiagent Systems, pages 475-482, 2012. 58
138. W. B. Knox, P. Stone, and C. Breazeal. Training a robot via human feedback: A case study. In International Conference on Social Robotics, pages 460-470. Springer, 2013. DOI: 10.1007/978-3-319-02675-6-46. 23, 56
139. J. Kober, J. A. Bagnell, and J. Peters. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, 32(11):1238-1274, 2013. DOI: 10.1177/0278364913495721. 59
140. J. Kober and J. Peters. Reinforcement learning in robotics: A survey. In Reinforcement Learning, pages 579-610. Springer, 2012. DOI: 10.1007/978-3-642- 27645-3-18. 53, 55, 59
141. J. Koenemann, F. Burget, and M. Bennewitz. Real-Time imitation of human whole-body motions by humanoids. In Proc. of the IEEE International Conference on Robotics & Automation (ICRA), 2014. 19, 20
142. Jens Kohlmorgen and Steven Lemm. An on-line method for segmentation and identification of non-stationary time series. In Neural Networks for Signal Processing XI, 2001. Proceedings of the 2001 IEEE Signal Processing Society Workshop, pages 113-122. IEEE, 2001. DOI: 10.1109/NNSP.2001.943116. 31
143. J Zico Kolter, Pieter Abbeel, and Andrew Y Ng. Hierarchical apprenticeship learning with application to quadruped locomotion. Advances in Neural Information Processing Systems, 20:769-776, 2008. 21, 44
144. George Konidaris, Scott Kuindersma, Roderic Grupen, and Andrew Barto. Robot learning from demonstration by constructing skill trees. The International Journal of Robotics Research, 31(3):360-375, December 2011. DOI: 10.1177/0278364911428653. 44
145. P Kormushev, S Calinon, and D G Caldwell. Robot Motor Skill Coordination with EMbased Reinforcement Learning. In Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010. DOI: 10.1109/IROS.2010.5649089. 26
146. Sotiris B Kotsiantis, ID Zaharakis, and PE Pintelas. Supervised machine learning: A review of classification techniques. 2007. 38
147. V. Kruger, D. Herzog, S. Baby, A. Ude, and D. Kragic. Learning actions from observations. Robotics & Automation Magazine, IEEE, 17(2):30-43, 2010. DOI: 10.1109/MRA.2010.936961. 32
148. G. Kuhlmann, P. Stone, R. J. Mooney, and J. W. Shavlik. Guiding a reinforcement learner with natural language advice: Initial results in roboCup soccer. In Proceedings of theAAAI-2004 Workshop on Supervisory Control of Learning and Adaptive Systems, San Jose, CA, July 2004. 23, 58
149. D. Kulic, C. Ott, D. Lee, J. Ishikawa, and Y. Nakamura. Incremental learning of full body motion primitives and their sequencing through human motion observation. The International Journal of Robotics Research, 31(3):330-345, November 2011. DOI: 10.1109/ICHR.2008.4756000. 31, 32
150. D. Kulic, W. Takano, and Y. Nakamura. Incremental learning, clustering and hierarchy formation of whole body motion patterns using adaptive hidden markov chains. International Journal of Robotics Research, 27(7):761-784, July 2008. DOI: 10.1177/0278364908091153. 31, 32
151. Dana Kulic, Wataru Takano, and Yoshihiko Nakamura. Online segmentation and clustering from continuous observation of whole body motions. IEEE Transactions on Robotics, 25(5):641-647, 2009. DOI: 10.1109/TRO.2009.2026508. 31, 32
152. Yasuo Kuniyoshi, Masayuki Inaba, and Hirochika Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. In IEEE Transactions on Robotics and Automation, volume 10, pages 799-822, 1994. DOI: 10.1109/70.338535. 22, 40
153. Yoshinori Kuno, Kazuhisa Sadazuka, Michie Kawashima, Keiichi Yamazaki, Akiko Yamazaki, and Hideaki Kuzuoka. Museum guide robot based on sociological interaction analysis. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems-CHI 07, page 1191, New York, New York, USA, April 2007. ACM Press. DOI: 10.1145/1240624.1240804. 10
154. John Lafferty, Andrew McCallum, and Fernando CN Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. 2001. 49
155. J. Lave and E. Wenger. Situated Learning: Legitimate Peripheral Participation. Cambridge University Press, Cambridge, 1991. DOI: 10.1017/CBO9780511815355. 8
156. D. Lee and Y. Nakamura. Mimesis model from partial observations for a humanoid robot. The International Journal of Robotics Research, 29(1):60-80, August 2009. DOI: 10.1177/0278364909342282. 32
157. D. Lee, C. Ott, and Y. Nakamura. Mimetic communication model with compliant physical contact in human-humanoid interaction. The International Journal of Robotics Research, 29(13):1684-1704, May 2010. DOI: 10.1177/0278364910364164. 32
158. S. Levine, Z. Popovic, and V. Koltun. Feature construction for inverse reinforcement learning. In Advances in Neural Information Processing Systems, pages 1342-1350, 2010. 45
159. J. Lieberman and C. Breazeal. Improvements on action parsing and action interpolation for learning through demonstration. In 4th IEEE/RAS International Conference on Humanoid Robots, volume 1, pages 342-365, 2004. DOI: 10.1109/ICHR.2004.1442131. 19
160. R. Likert. A technique for the measurement of attitudes. Archives of Psychology, 140:1-55, 1932. 69
161. M. Likhachev and R. C. Arkin. Spatio-Temporal case-based reasoning for behavioral selection. In Robotics and Automation, 2001. Proceedings 2001 ICRA. IEEE International Conference on, volume 2, pages 1627-1634. IEEE, 2001. DOI: 10.1109/ROBOT.2001.932844. 39
162. Y. Lin, S. Ren, M. Clevenger, and Y. Sun. Learning grasping force from demonstration. In Robotics and Automation , 2012 IEEE International Conference on, pages 1526-1531, 2012. DOI: 10.1109/ICRA.2012.6225222. 26
163. B. Litowitz. Just say no: Responsibility and resistance. In M Cole, Y Engeström, and O Vasquez, editors, Mind, Culture, and Activity: Seminal Papers from the Laboratory of Comparative Human Cognition. Cambridge University Press, Cambridge, 1997. 8
164. Huan Liu and Hiroshi Motoda. Feature Selection for Knowledge Discovery and Data Mining. Springer, 1998. DOI: 10.1007/978-1-4615-5689-3. 45
165. A. Lockerd and C. Breazeal. Tutelage and socially guided robot learning. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2004. DOI: 10.1109/IROS.2004.1389954. 39, 59
166. R. Lomasky, C. Brodley, M. Aernecke, D. Walt, and M. Friedl. Active class selection. In Machine Learning: ECML 2007, volume 4701 of Lecture Notes in Computer Science, pages 640-647. Springer-Verlag, 2007. DOI: 10.1007/978-3-540-74958-5-63. 62
167. M. Lopes, F. Melo, and L. Montesano. Active learning for reward estimation in inverse reinforcement learning. In Machine Learning and Knowledge Discovery in Databases, pages 31-46. Springer Berlin/Heidelberg, 2009. DOI: 10.1007/978-3-642-04174-7-3. 44, 61
168. Manuel Lopes, Francisco Melo, Luis Montesano, and Jose Santon-Victor. Abstraction levels for robotic imitation: Overview and computational approaches. In From Motor Learning to Interaction Learning in Robots, pages 313-355. 2010. DOI: 10.1007/978-3-642-05181-4-14. 48
169. Manuel Lopes, Francisco S. Melo, and Luis Montesano. Affordance-based imitation learning in robots. 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1015-1021, October 2007. DOI: 10.1109/IROS.2007.4399517. 48
170. R. Maclin, J. Shavlik, L. Torrey, T. Walker, and E. Wild. Giving advice about preferred actions to reinforcement learners via knowledge-based kernel regression. In Proceedings of theThe Twentieth National Conference on Artificial Intelligence, Pittsburgh, PA, July 2005. 23, 58
171. Y. Matsusaka, S. Fujie, and T. Kobayashi. Modeling of conversational strategy for the robot participating in the group conversation. In INTERSPEECH01, pages 2173-2176, 2001. 10
172. B. J. McCarragher and H. Asada. The discrete event modeling and trajectory planning of robotic assembly tasks. Journal of Dynamic Systems, Measurement, and Control, 117:394-394, 1995. DOI: 10.1115/1.2799130. 25
173. Wim Meeussen, Johan Rutgeerts, Klaas Gadeyne, Herman Bruyninckx, and Joris De Schutter. Contact-state segmentation using particle filters for programming by human demonstration in compliant-motion tasks. IEEE Transactions on Robotics, 23(2):218-231, April 2007. DOI: 10.1109/TRO.2007.892227. 26
174. A. N. Meltzoff. The Human infant as imitative generalist: A 20-year progress report on infant imitation with implications for comparative psychology. In B G Galef, C. M. Heyes, editor, Social Learning in Animals: The Roots of Culture. Academic Press, San Diego, CA, 1996. 7
175. Cetin Meriçli, Steven D Klee, Jack Paparian, and Manuela Veloso. An interactive approach for situated task teaching through verbal instructions. 2013. 20, 59
176. C. Meriçli, Manuela Veloso, and H Levent Akin. Multi-resolution corrective demonstration for efficient task execution and refinement. International Journal of Social Robotics, 4(4):423-435, 2012. DOI: 10.1007/s12369-012-0159-6. 45
177. L. Montesano, M. Lopes, A. Bernardino, and J. Santos-Victor. Learning object affordances: From sensory-motor coordination to imitation. IEEE Transactions on Robotics, 24:15-26, 2008. DOI: 10.1109/TRO.2007.914848. 48
178. J. R. Movellan. An infomax controller for real time detection of social contingency. In Proceedings of the 4th International Conference on Development and Learning, volume 1, pages 19-24, 2005. DOI: 10.1109/DEVLRN.2005.1490937. 12
179. M Muhlig, Michael Gienger, Jochen J Steil, and Christian Goerick. Automatic selection of task spaces for imitation learning. In Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Conference on, pages 4996-5002. IEEE, 2009. DOI: 10.1109/IROS.2009.5353894. 34, 63
180. K. Mulling, J. Kober, O. Kroemer, and J. Peters. Learning to select and generalize striking movements in robot table tennis. The International Journal of Robotics Research, 32(3):263-279, January 2013. DOI: 10.1177/ 0278364912472380. 26, 29
181. B. Mutlu, J. Forlizzi, and J. Hodgins. A storytelling robot: Modeling and evaluation of human-like gaze behavior. In 2006 6th IEEE-RAS International Conference on Humanoid Robots, pages 518-523. IEEE, December 2006. DOI: 10.1109/ICHR.2006.321322. 10
182. B. Mutlu, T. Shiwa, T. Kanda, H. Ishiguro, and N. Hagita. Footing in humanrobot conversations: How robots might shape participant roles using gaze cues. In Proceedings of the 2009 ACM Conference on Human-Robot Interaction, 2009. DOI: 10.1145/1514095.1514109. 10
183. Y. Nagai, C. Muhl, and K. J. Rohlfing. Toward designing a robot that learns actions from parental demonstrations. In Proceedings of the 2008 IEEE International Conference on Robotics and Automation, pages 3545-3550, 2008. DOI: 10.1109/ROBOT.2008.4543753. 10
184. Jun Nakanishi, Jun Morimoto, Gen Endo, Gordon Cheng, Stefan Schaal, and Mitsuo Kawato. Learning from demonstration and adaptation of biped locomotion. Robotics and Autonomous Systems, 47:79-91, 2004. DOI: 10.1016/S0921-8890(04) 00039-9. 21
185. M. Nakano, Y. Hasegawa, K. Funakoshi, J. Takeuchi, T. Torii, K. Nakadai, N. Kanda, K. Komatani, H. G. Okuno, and H. Tsujino. A multi-expert model for dialogue and behavior control of conversational robots and agents. Knowledge-Based Systems, 24(2):248-256, 2011. DOI: 10.1016/j.knosys.2010.08.004. 7
186. Chrystopher L Nehaniv and Kerstin Dautenhahn. The correspondence problem. Imitation in Animals and Artifacts, pages 41-61, 2002. 18
187. U. Nehmzow, O. Akanyeti, C. Weinrich, T. Kyriacou, and S. A. Billings. Robot programming by demonstration through system identification. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007. DOI: 10.1109/IROS.2007.4399087. 20
188. Gergely Neu and Csaba Szepesvári. Apprenticeship learning using inverse reinforcement learning and gradient methods. arXiv preprint arXiv:1206.5264, 2012. 44
189. A. Y. Ng and S. J. Russell. Algorithms for inverse reinforcement learning. In Icml, pages 663-670, 2000. 44
190. M. N. Nicolescu and M. J. Matarić. Experience-based representation construction: Learning from human and robot teachers. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Maui, Hawaii, 2001. DOI: 10.1109/IROS.2001.976257. 20
191. M. N. Nicolescu and M. J. Matarić. Methods for robot task learning: Demonstrations, generalization and practice. In Second International Joint Conference on Autonomous Agents and Multi-Agent Systems, Melbourne, Australia, July 2003. DOI: 10.1145/860575.860614. 40, 59
192. Scott Niekum, Sachin Chitta, Bhaskara Marthi, Sarah Osentoski, and Andrew G Barto. Incremental semantically grounded learning from demonstration. In Robotics Science and Systems, 2013. 41
193. Scott Niekum, Sarah Osentoski, George Konidaris, and Andrew G Barto. Learning and generalization of complex tasks from unstructured demonstrations. In Intelligent Robots and Systems , 2012 IEEE/RSJ International Conference on, pages 5239-5246. IEEE, 2012. DOI: 10.1109/IROS.2012.6386006. 29, 41, 46
194. Scott D Niekum. Semantically grounded learning from unstructured demonstrations. Ph.D. thesis, University of Massachusetts Amherst, 2013. 41, 42
195. T. Nomura, T. Suzuki, T. Kanda, and K. Kato. Measurement of negative attitudes toward robots. Interaction Studies, 7(3):437-454, 2006. DOI: 10.1075/is.7.3.14nom. 70
196. M. Ollis, W. H. Huang, and M. Happold. A Bayesian approach to imitation learning for robot navigation. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007. DOI: 10.1109/IROS.2007.4399220. 45
197. D. R. Olsen and M. A. Goodrich. Metrics for evaluating human-robot interactions. In Proceedings of the Performance Metrics for Intelligent Systems Workshop (PerMIS), 2003. 78
198. S. Ontañón, K. Mishra, N. Sugandh, and A. Ram. Case-based planning and execution for real-Time strategy games. In Case-Based Reasoning Research and Development, pages 164-178. Springer, 2007. DOI: 10.1007/978-3-540-74141-1-12. 39
199. C. C. Graylin, J. Dong, S. Grollman, D. Suay, H.B. Osentoski, S. Pitzer, B. Jenkins, O.C., Sarah Osentoski, Benjamin Pitzer, Christopher Crick, Graylin Jay, Shuonan Dong, Daniel Grollman, Halit Bener Suay, and Odest Chadwicke Jenkins. Remote robotic laboratories for learning from demonstration. International Journal of Social Robotics, pages 1-13, 2012. DOI: 10.1007/s12369-012-0157-8. 79
200. Pierre-yves Oudeyer, Adrien Baranes, and Frédéric Kaplan. Intrinsically motivated exploration for developmental and active sensorimotor learning. In From Motor Learning to Interaction Learning in Robots, pages 107-146. 2010. DOI: 10.1007/978-3-642-05181-4-6. 8
201. Peter Pastor, Heiko Hoffmann, Tamim Asfour, and Stefan Schaal. Learning and generalization of motor skills by learning from demonstration. In IEEE International Conference on Robotics and Automation, pages 763-768. IEEE, 2009. DOI: 10.1109/ROBOT.2009.5152385. 28
202. Peter Pastor, Mrinal Kalakrishnan, Sachin Chitta, Evangelos Theodorou, and Stefan Schaal. Skill learning and task outcome prediction for manipulation. IEEE International Conference on Robotics and Automation, pages 3828-3834, May 2011. DOI: 10.1109/ICRA.2011.5980200. 26, 28, 29, 53
203. R. Pea. Practices of distributed intelligence and designs for education. In G Salomon, editor, Distributed Cognitions: Psychological and Educational Considerations. Cambridge University Press, New York, 1993. 7
204. N. K. Person, A. C. Graesser, J. P. Magliano, and R. J. Kreuz. Inferring what the student knows in one-To-one tutoring: The role of student questions and answers. Learning and Individual Differences, 6(2):205-229, 1994. DOI: 10.1016/1041-6080(94)90010-8. 12
205. Jan Peters and Stefan Schaal. Reinforcement learning of motor skills with policy gradients. Neural networks: The Official Journal of the International Neural Network Society, 21(4):682-697, May 2008. DOI: 10.1016/j.neunet.2008.02. 003. 29
206. Thair Nu Phyu. Survey of classification techniques in data mining. In Proceedings of the International MultiConference of Engineers and Computer Scientists, volume 1, pages 18-20, 2009. 38
207. J. Piaget. The Origins of Intelligence in Children. International Universities Press, 1952. DOI: 10.1037/11494-000. 12
208. P. M. Pilarski, M. R. Dawson, T. Degris, F. Fahimi, J. P. Carey, and R. S. Sutton. Online human training of a myoelectric prosthesis controller via actor-critic reinforcement learning. In Proc. of the IEEE International Conference on Rehabilitation Robotics, pages 1-7, 2011. 55
209. N. Pollard and J. K. Hodgins. Generalizing demonstrated manipulation tasks. In Workshop on the Algorithmic Foundations of Robotics, December 2002. DOI: 10.1007/978-3-540-45058-0-31. 22
210. John Ross Quinlan. C4. 5: Programs for Machine Learning, volume 1. Morgan Kaufmann, 1993. 38
211. L. R. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257-286, 1989. DOI: 10.1109/5.18626. 30
212. H. Raghavan, O. Madani, and R. Jones. Active learning with feedback on features and instances. Journal of Machine Learning Research, 7:1655-1686, 2006. 62
213. D. Ramachandran and E. Amir. Bayesian inverse reinforcement learning. In International Joint Conference on Artificial Intelligence, pages 2586-2591, 2007. 44
214. Rajesh P N Rao, Aaron P Shon, and Andrew N Meltzoff. A Bayesian model of imitation in infants and robots. Imitation and Social Learning in Robots, Humans, and Animals: Behavioural, Social and Communicative Dimensions, 2004. 19, 39
215. N. Ratliff, J. A. Bagnell, and M. A. Zinkevich. Maximum margin planning. In Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, Pennsylvannia, 2006. DOI: 10.1145/1143844.1143936. 21
216. N. Ratliff, D. Bradley, J. A. Bagnell, and J. Chestnutt. Boosting structured prediction for imitation learning. Advances in Neural Information Processing Systems 19, 2007. 21
217. Nathan D Ratliff, David Silver, and J Andrew Bagnell. Learning to search: Functional gradient techniques for imitation learning. Autonomous Robots, 27(1):25-53, 2009. DOI: 10.1007/s10514-009-9121-3. 44, 79
218. C. Rich, B. Ponsler, A. Holroyd, and C. L. Sidner. Recognizing engagement in humanrobot interaction. In Proceedings of the 2010 ACM Conference on Human-Robot Interaction, 2010. DOI: 10.1109/HRI.2010.5453163. 7
219. B Rogoff and H Gardner. Adult guidance of cognitive development. In B Rogoff and J Lave, editors, Everyday Cognition: Its Development in Social Context. Harvard University Press, Cambridge, MA, 1984. 7, 10, 11
220. R. Ros, R. L. De Mántaras, J. L. Arcos, and M. Veloso. Team playing behavior in robot soccer: A case-based reasoning approach. In Case-Based Reasoning Research and Development, pages 46-60. Springer, 2007. DOI: 10.1007/978-3-540-74141-1-4. 39
221. M T Rosenstein and A G Barto. Supervised Actor-Critic Reinforcement Learning. John Wiley & Sons, Inc., New York, NY, USA, 2004. 19, 58
222. S Rosenthal, A K Dey, and M Veloso. How robots questions affect the accuracy of the human responses. In Proceedings of the IEEE Symposium on Robot and Human Interactive Communication, pages 1137-1142, 2009. DOI: 10.1109/ROMAN.2009.5326291. 61
223. Stephanie Rosenthal and Manuela Veloso. Modeling humans as observation providers using pomdps. In RO-MAN, 2011 IEEE, pages 53-58. IEEE, 2011. DOI: 10.1109/ROMAN. 2011.6005272. 61
224. R. M. Ryan and E. L. Deci. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1):68-78, 2000. DOI: 10.1037/0003-066X.55.1.68. 6
225. Paul E Rybski, Kevin Yoon, Jeremy Stolarz, and Manuela M Veloso. Interactive robot task training through dialog and demonstration. In HRI 07: Proceedings of the ACM/IEEE international conference on Human-robot interaction, pages 49-56, New York, NY, USA, 2007. ACM. DOI: 10.1145/1228716.1228724. 20, 40
226. L M Saksida, S M Raymond, and D S Touretzky. Shaping robot behavior using principles from instrumental conditioning. Robotics and Autonomous Systems, 22(3/4):231, 1998. DOI: 10.1016/S0921-8890(97)00041-9. 55
227. J Saunders, C Nehaniv, and K Dautenhahn. Teaching robots by moulding behavior and scaffolding the environment,. In Proceedings of theACM SIGCHI/SIGART Conference on Human-Robot Interaction, pages 118-125, 2006. DOI: 10.1145/1121241.1121263. 10, 39
228. Joe Saunders, Chrystopher L Nehaniv, Kerstin Dautenhahn, and Aris Alissandrakis. Selfimitation and environmental scaffolding for robot teaching. International Journal of Advanced Robotics Systems, 4(1):109-124, 2007. DOI: 10.5772/5703. 10
229. J. Schmidhuber. A possibility for implementing curiosity and boredom in model-building neural controllers. In International Conference on Simulation of Adaptive Behavior, pages 222-227, 1991. 8
230. Candace L Sidner, Cory D Kidd, Christopher Lee, and Neal Lesh. Where to look: A study of human-robot engagement. In Knowledge Creation Diffusion Utilization, IUI 04, pages 78-84. Mitsubishi Electric Research Labs and MIT Media Lab, ACM, 2004. DOI: 10.1145/964442.964458. 10
231. O. Sigaud and J. Peters. From Motor Learning to Interaction Learning in Robots. Springer, 2010. DOI: 10.1007/978-3-642-05181-4. 81
232. David Silver, J. Andrew Bagnell, and Anthony Stentz. Learning from demonstration for autonomous navigation in complex unstructured terrain. The International Journal of Robotics Research, 29(12):1565-1592, June 2010. DOI: 10.1177/0278364910369715. 34, 44, 79
233. S Singh, A G Barto, and N Chentanez. Intrinsically motivated reinforcement learning. In Proceedings ofAdvances in Neural Information Processing Systems 17, 2005. 8
234. B. F. Skinner. Science and Human Behavior. Colliler-Macmillian, 1953. 23
235. Marjorie Skubic and Richard A Volz. Identifying contact formations from sensory patterns and its applicability to robot programming by demonstration. In Intelligent Robots and Systems 96, IROS 96, Proceedings of the 1996 IEEE/RSJ International Conference on, volume 2, pages 458-464. IEEE, 1996. DOI: 10.1109/IROS.1996.570817. 26
236. W D Smart and L P Kaelbling. Effective reinforcement learning for mobile robots. In In Proceedings of the IEEE International Conference on Robotics and Automation, pages 3404-3410, 2002. DOI: 10.1109/ROBOT.2002.1014237. 58
237. William D Smart. Making reinforcement learning work on real robots. Ph.D. thesis, Department of Computer Science, Brown University, Providence, RI, 2002. 18, 19
238. Dimitrios Stefanidis, Fikre Wang, James R Korndorffer Jr, J Bruce Dunne, and Daniel J Scott. Robotic assistance improves intracorporeal suturing performance and safety in the operating room while decreasing operator workload. Surgical Endoscopy, 24(2):377-382, 2010. DOI: 10.1007/s00464-009-0578-0. 70
239. A. Steinfeld, T. Fong, D. Kaber, M. Lewis, J. Scholtz, A. Schultz, and M. Goodrich. Common metrics for human-robot interaction. In Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction, HRI 06, pages 33-40, New York, NY, USA, 2006. ACM. DOI: 10.1145/1121241.1121249. 78
240. Aaron Steinfeld, Terrence Fong, David Kaber, Michael Lewis, Jean Scholtz, Alan Schultz, and Michael Goodrich. Common metrics for human-robot interaction. In HRI 06: Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction, pages 33-40, New York, NY, USA, 2006. ACM. DOI: 10.1145/1121241.1121249. 69
241. A Stern, A Frank, and B Resner. Virtual petz (video session): A hybrid approach to creating autonomous, lifelike dogz and catz. In AGENTS 98: Proceedings of the Second International Conference on Autonomous Agents, pages 334-335, New York, NY, USA, 1998. ACM Press. DOI: 10.1145/280765.280852. 23
242. H. B. Suay, R. Toris, and S. Chernova. A practical comparison of three robot learning from demonstration algorithms. International Journal of Social Robotics, 2012. DOI: 10.1007/s12369-012-0158-7. 23, 58, 79
243. Keith Sullivan. Multiagent hierarchical learning from demonstration. In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence-Volume Volume Three, pages 2852-2853. AAAI Press, 2011. DOI: 10.5591/978-1-57735-516-8/IJCAI11-498. 38
244. Keith Sullivan, Sean Luke, and Vittoria Amos Ziparo. Hierarchical learning from demonstration on humanoid robots. In Proceedings of Humanoid Robots Learning from Human Interaction Workshop, 2010. 38
245. Richard S Sutton and Andrew G Barto. Reinforcement Learning: An Introduction. The MIT Press, Cambridge, MA, London, England, 1998. 2, 23, 55
246. Richard S Sutton, Doina Precup, and Satinder Singh. Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112(1):181-211, 1999. DOI: 10.1016/S0004-3702(99)00052- 1. 44
247. Umar Syed, Michael Bowling, and Robert E Schapire. Apprenticeship learning using linear programming. In Proceedings of the 25th International Conference on Machine Learning, pages 1032-1039. ACM, 2008. DOI: 10.1145/1390156.1390286. 44
248. Umar Syed and Robert E Schapire. A game-Theoretic approach to apprenticeship learning. In Advances in Neural Information Processing Systems, pages 1449-1456, 2007. 44
249. J. Takamatsu, K. Ogawara, H. Kimura, and K. Ikeuchi. Recognizing assembly tasks through human demonstration. The International Journal of Robotics Research, 26(7):641-659, July 2007. DOI: 10.1177/0278364907080736. 22
250. A. Tenorio-Gonzalez, E. Morales, and L. Villase nor Pineda. Dynamic reward shaping: Training a robot by voice. In Advances in Artificial Intelligence-IBERAMIA, pages 483-492, 2010. DOI: 10.1007/978-3-642-16952-6-49. 55
251. Evangelos Theodorou, Jonas Buchli, and Stefan Schaal. Reinforcement learning of motor skills in high dimensions: A path integral approach. In Robotics and Automation , 2010 IEEE International Conference on, pages 2397-2403. IEEE, 2010. DOI: 10.1109/ROBOT.2010.5509336. 29
252. A L Thomaz and C Breazeal. Reinforcement learning with human teachers: Evidence of feedback and guidance with implications for learning performance. In Proceedings of the 21st National Conference on Artificial Intelligence, 2006. 23, 58, 79
253. A L Thomaz and C Breazeal. Teachable robots: Understanding human teaching behavior to build more effective robot learners. Artificial Intelligence Journal, 172:716-737, 2008. DOI: 10.1016/j.artint.2007.09.009. 23, 58, 79
254. M. Tomasello. The Cultural Origins of Human Cognition. Harvard University Press, March 2001. 8
255. R. Toris, D. Kent, and S. Chernova. The robot management system: A framework for conducting human-robot interaction studies through crowdsourcing. International Journal of Human-Robot Interaction, 2013. 79
256. J.G. Trafton, M.D. Bugajska, B.R. Fransen, and R.M. Ratwani. Integrating vision and audition within a cognitive architecture to track conversations. In Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction, pages 201-208, 2008. DOI: 10.1145/1349822.1349849. 10
257. E Tronik, H Als, L Adamson, and C Trevarthen. Communication and cooperation in early infancy: A description of primary intersubjectivity. In M Bullowa, editor, Before Speech: The Beginning of Interpersonal Communication, pages 389-450. Cambridge University Press, Cambridge, 1979. 7, 10
258. K. M. Tsui, M. Desai, H. A. Yanco, H. Cramer and N. Kemp. Using the negative attitudes toward robots scale with telepresence robots. In Proceedings of the Performance Metrics for Intelligent Systems Workshop ,2010. DOI: 10.1145/2377576.2377621.70
259. Ales Ude, Christopher G Atkeson, and Marcia Riley. Programming full-body movements for humanoid robots by observation. Robotics and Autonomous Systems, 47:93-108, 2004. DOI: 10.1016/S0921-8890(04)00040-5. 22
260. Michael van Lent and John E Laird. Learning procedural knowledge through observation. In K-CAP 01: Proceedings of the 1st International Conference on Knowledge Capture, pages 179-186, New York, NY, USA, 2001. ACM Press. DOI: 10.1145/500737.500765. 40, 43
261. D. Vasquez, T. Fraichard, and C. Laugier. Growing Hidden Markov Models: An incremental tool for learning and predicting human and vehicle motion. The International Journal of Robotics Research, 28(11-12):1486-1506, August 2009. DOI: 10.1177/0278364909342118. 31
262. H Veeraraghavan and Manuela Veloso. Teaching sequential tasks with repetition through demonstration (short paper). In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems, May 2008. 40
263. Manuela Veloso, Felix Von Hundelshausen, and Paul E Rybski. Learning visual object definitions by observing human activities. In Humanoid Robots, 2005 5th IEEE-RAS International Conference on, pages 148-153. IEEE, 2005. DOI: 10.1109/ICHR.2005.1573560. 49
264. Sethu Vijayakumar and Stefan Schaal. Locally weighted projection regression: An O(n) algorithm for incremental real time learning in high dimensional space. In Proceedings of Seventeenth International Conference on Machine Learning, pages 1079-1086, 2000. 28
265. L. S. Vygotsky. Mind in society: The development of higher psychological processes. Harvard University Press, Cambridge, MA, 1978. 9
266. A. R. Wagner. The role of trust and relationships in human-robot social interaction, 2009. 70
267. Q. Wang, J. De Schutter, W. Witvrouw, and S. Graves. Derivation of compliant motion programs based on human demonstration. In Robotics and Automation, 1996. Proceedings., 1996 IEEE International Conference on, volume 3, pages 2616-2621 vol.3, 1996. DOI: 10.1109/ROBOT.1996.506557. 26
268. J V Wertsch, N Minick, and F J Arns. Creation of context in joint problem solving. In B Rogoff and J Lave, editors, Everyday Cognition: Its Development in Social Context. Harvard University Press, Cambridge, MA, 1984. 9
269. C. Wickens, J. Gordon, and Y. Liu. An Introduction to Human Factors Engineering. Pearson-Prentice Hall, New Jersey, 2004. 78
270. Akiko Yamazaki, Keiichi Yamazaki, Yoshinori Kuno, Matthew Burdelski, Michie Kawashima, and Hideaki Kuzuoka. Precision timing in human-robot interaction: Coordination of head movement and utterance. Ratio, 08(1):131-139, 2008. DOI: 10.1145/1357054.1357077. 10
271. P Zang, R Tian, A L Thomaz, and C Isbell. Batch versus interactive learning by demonstration. In Proceedings of the International Conference on Development and Learning, 2010. 54
272. Brian D Ziebart, Andrew Maas, James (Drew) Bagnell, and Anind K Dey. Maximum entropy inverse reinforcement learning. In Proceeding of AAAI 2008, July 2008. 44
273. Matthew Zucker and J Andrew Bagnell. Reinforcement planning: Rl for optimal planners. In Robotics and Automation , 2012 IEEE International Conference on, pages 1850-1855. IEEE, 2012. DOI: 10.1109/ICRA.2012.6225036. 45
274. P Zukow-Goldring, M A Arbib, and E Oztop. Language and the mirror system: A perception/ action based approach to cognitive development. Cognition, Brain, Behavior, pages 239-272, 2005. 7, 9.