Object schemas for grounding language in a responsive robot

Connection Science

Volumn 20, Issue 4, 2008, Pages 253-276

  Hsiao, Kai Yuh ^a   Tellex, Stefanie ^a   Vosoughi, Soroush ^a   Kubat, Rony ^a   Roy, Deb ^a  

^a MIT MEDIA LAB   (United States)

Author keywords

Human robot interaction; Language grounding; Object schema; Semantics

Indexed keywords

COUPLED PROCESS; GROUNDING LANGUAGE; INTERACTION PROCESS; LANGUAGE GROUNDING; NATURAL LANGUAGES; OBJECT SCHEMA; PHYSICAL CHANGES; PHYSICAL OBJECTS;

NATURAL LANGUAGE PROCESSING SYSTEMS; SEMANTICS;

ROBOTS;

EID: 56749176329     PISSN: 09540091     EISSN: 13600494     Source Type: Journal    
DOI: 10.1080/09540090802445113     Document Type: Article

References (41)

1. Arbib, M.A., Iberall, T., and Lyons, D. (1987), 'Schemas That Integrate Vision and Touch for Hand Control,' in Vision, Brain, and Cooperative Computation, eds. M.A. Arbib and A.R. Hanson, Cambridge, MA: MIT Press, pp. 489-510.
2. Bauckhage, C., Fritsch, J., Rohlfing, K., Wachsmuth, S., and Sagerer, G. (2002), 'Evaluating Integrated Speech and Image Understanding,' in Proceedings of the IEEE International Conference on Multimodal Interfaces, pp. 9-14.
3. Beeson, P., MacMahon, M., Modayil, J., Murarka, A., Kuipers, B., and Stankiewicz, B. (2007), 'Integrating Multiple Representations of Spatial Knowledge for Mapping, Navigation, and Communication,' in Proceedings of the AAAI 2007 Spring Symposium on Interaction. Challenges for Intelligent Assistants, pp. 1-9.
4. Belongie, S., Malik, J., and Puzicha, J. (2002), 'Shape Matching and Object Recognition. Using Shape Contexts,' IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 509-522.
5. st National Conference on Artificial Intelligence, pp. 1444-1450.
6. Breazeal, C., Berlin, M., Brooks, A., Gray, J., and Thomaz, A.L. (2006), 'Using Perspective Taking to Learn From Ambiguous Demonstrations,' Robotics and Autonomous Systems, Special Issue on. Robot Programming by Demonstration, 54, pp. 385-393.
7. Brooks, R. A. (1986), 'A Robust Layered Control System for a Mobile Robot,' IEEE Journal of Robotics and Automation, 2, pp. 14-23.
8. Brooks, R.A. (1991), 'Intelligence Without Representation,' Artificial Intelligence, 47, 1991, pp.139-160.
9. Bruce, J., Balch, T., and Veloso, M. (2000), 'Fast and Inexpensive Colour Image Segmentation for Interactive Robots,' in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2061-2066.
10. Bryson, J.J. (2001), Intelligence, by Design: Principles of Modularity and Coordination for Engineering Complex Adaptive Agents, PhD thesis, Massachusetts Institute of Technology.
11. Bryson, J.J., and Stein, L.A. (2001), 'Modularity and Design in Reactive Intelligence,' in Proceedings of the International Joint Conference on Artificial Intelligence, pp. 1115-1120.
12. Burke, R., Isla, D., Downie, M., Ivanov, Y., and Blumberg, B. (2001), 'Creature Smarts: The Art and Architecture of a Virtual Brain,' in Proceedings of the Game Developers Conference, pp. 147-166.
13. Cangelosi, A. (2006), "The Grounding and Sharing of Symbols,' Pragmatics and Cognition, 14, 2006, 275-285.
14. Clark, A., and Toribio, J. (1994), 'Doing with Representing,' Synthese, 101, 1994, pp. 401-431.
15. Comaniciu, D., and Meer, P. (2002), 'Mean Shift: A Robust Approach Toward Feature Space Analysis,' IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 2002, 603-619.
16. Drescher, G. (1991), Made-Up Minds: A Constructivist Approach to Artificial Intelligence, Cambridge, MA: MIT Press.
17. Fitzpatrick, P. (2003), From First Contact to Close Encounters: A Developmentally Deep Perceptual System for a Humanoid Robot, PhD thesis, Massachusetts Institute of Technology.
18. Fitzpatrick, P., and Metta, G. (2003), 'Grounding Vision Through Experimental Manipulation,' Philosophical Transactions of the Royal Society: Mathematical, Physical, and Engineering Sciences, 361, pp. 2165-2185.
19. th National Conference on Artificial Intelligence, 1992, pp. 809-815.
20. Gat, E. (1998), 'Three-layer Architectures,' in Artificial Intelligence and Mobile Robots, eds. D. Kortenkamp, R.P. Bannasso and R. Murphy, Cambridge, MA: AAAI Press, pp. 195-210.
21. Gibson, J.J. (1979), The Ecological Approach to Visual Perception, Thllsdale, NJ: Erlbaum.
22. Gorniak, P., and Roy, D. (2007), 'Situated Language Understanding as Filtering Perceived Affordances,' Cognitive. Science, 31, 197-231.
23. Hsiao, K., and Roy, D. (2005), 'A Habit System for an Interactive Robot,' in AAAI Fall Symposium: From Reactive to Anticipatory Cognitive Embodied Systems, pp. 99-106.
24. Isla, D., Burke, R., Downie, M., and Blumberg, B. (2001), 'A Layered Brain Architecture for Synthetic Creatures,' in Proceedings of the International Joint Conferences on Artificial Intelligence, pp. 1051-1058.
25. Modayil, J., and Kuipers, B. (2007), 'Where do Actions Come From? Autonomous Robot Learning of Objects and Actions,' in Proceedings of the AAAI 2007 Spring Symposium on Control Mechanisms for Spatial Knowledge Processing in Cognitive/Intelligent Systems, pp. 41-46.
26. Nilsson, N.J. (1984), 'Shakey the Robot,' Technical Report 323, AI Centre, SRI International.
27. Piaget, J. (1954), The Construction of Reality in the Child, New York, NY: Basic Books.
28. Regier, T., and Carlson, L. (2001), 'Grounding Spatial Language in Perception: An Empirical and Computational Investigation,' Journal of Experimental Psychology, 130, 273-298.
29. Roy, D. (2005a), 'Grounding Words in Perception and Action: Computational Insights,' Trends in Cognitive Science, 9, 389-396.
30. Roy, D. (2005b), 'Semiotic Schemas: A Framework for Grounding Language in Action and Perception,' Artificial Intelligence, 167, 170-205.
31. Roy, D. (in press), 'A Mechanistic Model of Three Facets of Meaning,' in Symbols and Embodiment, eds. M.de Vega, A. Glenberg and A. Graesser, Oxford: Oxford University Press.
32. Roy, D., Hsiao, K., and Mavridis, N. (2004), 'Mental Imagery for a Conversational Robot,' IEEE Transactions on Systems, Man, and Cybernetics, 34, 1374-1383.
33. Roy, D., and Reiter, E. (2005), 'Connecting Language to the World,' Artificial Intelligence, 167, 1-12.
34. Seabra Lopes, L., and Connell, J.H. (2001), 'Semisentient Robots: Routes to Integrated Intelligence,' IEEE Intelligent Systems, 16, 10-14.
35. Seabra Lopes, L., and Teixeira, A. (2000), 'Human-Robot Interaction Through Spoken Language Dialogue,' in Proceedings of the 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 528-534.
36. Siskind, J.M. (2001), 'Grounding the Lexical Semantics of Verbs in Visual Perception Using Force Dynamics and Event Logic,' Journal of Artificial Intelligence Research, 15, 31-90.
37. Skubic, M., Perzanowski, D., Blisard, S., Schultz, A., Adams, W., Bugajska, M., and Brock, D. (2004), 'Spatial Language for Human-Robot Dialogs,' IEEE Transactions on Systems, Man and Cybernetics, Part C, Special Issue on Human-Robot Interaction, 34, 154-167.
38. Steels, L., and Belpaeme, T. (2005), 'Coordinating Perceptually Grounded Categories Through Language: A Case Study for Colour,' Behavioural and Brain Sciences, 28, 469-529.
39. Stoytchev, A. (2005), 'Behavior-grounded Representation of Tool Affordances,' in Proceedings of IEEE International Conference on Robotics and Automation, pp. 3060-3065.
40. Vogt, P., and Divina, F. (2007), 'Social Symbol Grounding and Language Evolution,' Interaction Studies, 8, 31-52.
41. Winograd, T. (1972), Understanding Natural Language, New York: Academic Press.