Mental imagery for a conversational robot

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 34, Issue 3, 2004, Pages 1374-1383

  Roy, Deb ^a   Hsiao, Kai Yuh ^a   Mavridis, Nikolaos ^a  

^a MIT MEDIA LAB   (United States)

Author keywords

Active vision; Grounding; Language; Mental imagery; Mental models; Mental simulation; Robots

Indexed keywords

COGNITIVE SYSTEMS; COMPUTER SIMULATION; MAN MACHINE SYSTEMS; MANIPULATORS; NATURAL LANGUAGE PROCESSING SYSTEMS; ROBOTICS; SEMANTICS;

ACTIVE VISION; CONVERSATIONAL ROBOT; LANGUAGE; MENTAL IMAGERY; MENTAL MODELS;

COMPUTER VISION;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; COMPARATIVE STUDY; COMPUTER ASSISTED DIAGNOSIS; COMPUTER INTERFACE; EVALUATION; IMAGINATION; INTERPERSONAL COMMUNICATION; LINGUISTICS; MENTAL FUNCTION; METHODOLOGY; NATURAL LANGUAGE PROCESSING; PHYSIOLOGY; ROBOTICS; THREE DIMENSIONAL IMAGING; VALIDATION STUDY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMMUNICATION; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGINATION; IMAGING, THREE-DIMENSIONAL; MENTAL PROCESSES; NATURAL LANGUAGE PROCESSING; ROBOTICS; USER-COMPUTER INTERFACE; VOCABULARY, CONTROLLED;

EID: 2942522638     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2004.823327     Document Type: Article

References (47)

1. J. Barwise and J. Perry, Situations and Attitudes. Bradford,MA: MIT-Bradford, 1983.
2. H. Clark, Using Language. Cambridge, MA: Cambridge Univ. Press, 1996.
3. M. K. Brown, B. M. Buntschuh, and J. G. Wilpon, "SAM: A perceptive spoken language understanding robot," IEEE Trans. Syst., Man, Cybern., vol. 22, pp. 1390-1402, Nov./Dec. 1992.
4. C. Crangle and P. Suppes, Language and Learning for Robots. Stanford, CA: CSLI Publications, 1994.
5. P. McGuire, J. Fritsch, J. Steil, F. Roethling, G. Fink, S. Wachsmuth, G. Sagerer, and H. Ritter, "Multi-modal human-machine communication for instructing robot grasping tasks," in Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS), vol. 2, Aug. 2002, pp. 1082-1088.
6. D. Perzanowski, A. Schultz, W. Adams, K. Wauchope, E. Marsh, and M. Bugajska, "Interbot: A multi-modal interface to mobile robots," in Proc. Language Technologies 2001. Pittsburgh, PA, 2001.
7. W. Bluethmann, R. Ambrose, M. Diftler, S. Askew, E. Huber, M. Goza, F. Rehnmark, C. Lovchik, and D. Magruder, "Robonaut, A Robot designed to work with humans in space," Auton. Robots, vol. 14, pp. 179-197, 2003.
8. Z. Kazi, S. Chen, M. Beitler, D. Chester, and R. Foulds, "Grasping at straws: An intelligent multimodal assistive robot," in Proc. Int. Conf. Rehabilitation Robotics, Bath, U.K., 1997.
9. J. Feldman, G. Lakoff, D. Bailey, S. Narayanan, T. Regier, and A. Stolcke, "Lzero: The first five years," Artific. Intell. Rev., vol. 10, pp. 103-129, 1996.
10. J. M. Lammens, "A computational model of color perception and color naming," Ph.D. dissertation, State Univ. New York, New York, 1994.
11. T. Regier, The Human Semantic Potential. Cambridge, MA: MIT Press, 1996.
12. T. Regier and L. Carlson, "Grounding spatial language in perception: An empirical and computational investigation," J. Experim. Psych., vol. 130, no. 2, pp. 273-298, 2001.
13. D. Roy, "Learning words from sights and sounds: A computational model," Ph.D. dissertation, Massachusetts Inst. Technol., Cambridge, MA, 1999.
14. ____, "Learning visually-grounded words and syntax for a scene description task," Comput. Speech Lang., vol. 16, no. 3, 2002.
15. ____, "Grounded spoken language acquisition: Experiments in word learning," IEEE Trans. Multimedia, vol. 5, pp. 197-209, June 2003.
16. J. Siskind, "Naive physics, event perception, lexical semantics, and language acquisition," Ph.D. dissertation, Massachusetts Inst. Technol., Cambridge, MA, 1992.
17. D. Roy, "Grounding the lexical semantics of verbs in visual perception using force dynamics and event logic," J. Artific. Intell. Res., vol. 15, pp. 31-90, 2001.
18. C. Yu, D. Ballard, and R. Aslin, "The role of embodied intention in early lexical acquisition," in Proc. Cognitive Science Soc., Boston, MA, 2003.
19. D. Bailey, "When push comes to shove: A computational model of the role of motor control in the acquisition of action verbs," Ph.D. dissertation, EECS Dept., Univ. California at Berkeley, Berkeley, CA, 1997.
20. K. Hsiao, N. Mavridis, and D. Roy, "Coupling perception and simulation: Steps toward conversational robotics," in Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, Las Vegas, NV, 2003.
21. S. Narayanan, "Karma: Knowledge-based active representations for metaphor and aspect," Ph.D. dissertation, Univ. California at Berkeley, Berkeley, CA, 1997.
22. P. Johnson-Laird, Mental Models: Toward a Cognitive Science of Language, Inference, and Consciousness. Cambridge, MA: Cambridge Univ. Press, 1983.
23. G. Fauconnier, Mental Spaces. Cambridge, MA: MIT Press, 1985.
24. The Stanford Encyclopedia of Philosophy [Online]. Available: http://plato.stanford.edu
25. R. A. Finke, Principles of Mental Imagery. Cambridge, MA: MIT Press, 1989.
26. P. McKellar, Imagination and Thinking. London, U.L.:Cohen & West, 1957.
27. L. Barsalou, "Perceptual symbol systems," Behav. Brain Sci., vol. 22, pp. 577-609, 1999.
28. A. Glenberg, "What memory is for," Behav. Brain Sci., vol. 20, pp. 1-19, 1997.
29. J. Piaget and B. Inhelder, "The childs conception of space," in The Essential Piaget. New York: Basic Books, 1948, pp. 576-642.
30. B. Tversky, "Structures of mental spaces: How people think about space," Environ. Behav., vol. 35, pp. 66-80.
31. M. Skubic, G. Chronis, P. Matsakis, and J. Keller, "Spatial relations for tactical robot navigation," in Proc. SPIE, Unmanned Ground Vehicle Technology, Orlando, FL, 2001.
32. G. Miller and P. Johnson-Laird, Language and Perception. Cambridge, MA: Harvard Univ. Press, 1976.
33. D. Roy, P. Gorniak, N. Mukherjee, and J. Juster, "A trainable spoken language understanding system for visual object selection," in Proc. Int. Conf. Spoken Language Processing, Denver, CO, 2002.
34. F. Mussa-Ivaldi and E. Bizzi, "Motor learning through the combination of primitives," Philos. Trans. R. Soc. London, vol. 355, pp. 1755-1769, 2000.
35. Open Source Computer Vision Library [Online]. Available: http://www.intel.com/research/mrl/research/opencv
36. R. Smith. (2003) ODE: Open Dynamics Engine. [Online]. Available: http://q12.org/ode/
37. F. Cao and B. Shepherd, "Mimic: A robot planning environment integrating real and simulated worlds," in Proc. IEEE Int. Symp. Intelligent Control, 1989, p. 459 464.
38. W. J. Davis, "On-line simulation: Need and evolving research requirements," in Handbook of Simulation: Principles, Methodology, Advances, Applications and Practice, J. Banks, Ed. New York: Wiley, 1998.
39. J. R. Surdu, "Connecting simulation to the mission operational environment," Ph.D. dissertation, Texas A&M, College Station, 2000.
40. L. Talmy, Toward a Cognitive Semantics. Cambridge, MA: MIT Press, 2000.
41. OpenGL website [Online]. Available: www.opengl.org
42. J. J. Gibson, The Ecological Approach to Visual Perception. Mahwah, NJ: Erlbaum, 1979.
43. D. Roy, B. Schiele, and A. Pentland, "Learning audio-visual associations from sensory input," in Proc. Int. Conf. Computer Vision Workshop on the Integration of Speech and Image Understanding, Corfu, Greece, 1999.
44. B. Yoder, "Spontaneous Speech Recognition Using Hidden Markov Models," Master's thesis, Massachusetts Inst. Technol., Cambridge, MA, 2001.
45. J. Allen, Natural Language Understanding. New York: Benjamin Cummins, 1995.
46. P. Gorniak and D. Roy, "Grounded semantic composition for visual scenes," J. Artif. Intell. Res., to be published.
47. N. Mukherjee and D. Roy, "A visual context-aware multimodal system for spoken language processing," in Proc. Eurospeech, Geneva, Switzerland, 2003.