Using vision, acoustics, and natural language for disambiguation

HRI 2007 - Proceedings of the 2007 ACM/IEEE Conference on Human-Robot Interaction - Robot as Team Member

Volumn , Issue , 2007, Pages 73-80

  Fransen, Benjamin ^a   Morariu, Vlad ^a,b   Martinson, Eric ^a,c   Blisard, Samuel ^a,d   Marge, Matthew ^a,e   Thomas, Scott ^a,b   Schultz, Alan ^a   Perzanowski, Dennis ^a  

^a NAVAL RESEARCH LABORATORY   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

^c GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

^d UNIVERSITY OF MISSOURI   (United States)

^e UNIVERSITY OF EDINBURGH   (United Kingdom)

Author keywords

Acoustics; Artificial intelligence; Auditory perspective taking; Dialog; Human robot interaction; Natural language understanding; Spatial reasoning; Vision

Indexed keywords

KNOWLEDGE ENGINEERING; NATURAL LANGUAGE PROCESSING SYSTEMS; PROBLEM SOLVING; ROBUSTNESS (CONTROL SYSTEMS);

AUDITORY PERSPECTIVE TAKING; HUMAN ROBOT INTERACTION; NATURAL LANGUAGE UNDERSTANDING; SPATIAL REASONING;

HUMAN COMPUTER INTERACTION;

EID: 34548265670     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1228716.1228727     Document Type: Conference Paper

References (28)

1. Argyros, A. A., and Lourakis, M.I.A. Real-time tracking of multiple skin-colored objects with a possibly moving camera. In European Conf. on Computer Vision (ECCV 2004), vol. 3, 2004, 368-379.
2. Arrières, T., Marchand, J.-M., Gallinari, P., and Dorizzi, B. Multimodal segmental models for on-line handwriting recognition. In Intl. Conf. On Pattern Recognition, 2000, 2247-2250.
3. Bischoff, R. and Graefe, V. Design principles for dependable robotic assistants. In Intl. Journal on Humanoid Robotics, vol. 1, no. 1, (March 2004), 95-125.
4. Bolt, R. A. "Put-that-there": Voice and gesture at the graphics interface. In Proceedings of the 7th Annual Conf. on Computer Graphics and Interactive Techniques, (July 1980), 262-270.
5. Bradski, G., Kaehler, A., and Pisarevsky, V. Learning-based computer vision with Intel's Open Source computer vision library. In Intel Technology Journal, (May 2005).
6. Bradski, G.R. Computer Vision Face Tracking for Use in a Perceptual User Interface, Intel Technology Journal, 1998.
7. Brooks, A.G., and Breazeal, C. Working with robots and objects: Revisiting deictic reference for achieving spatial common ground. In Proceedings of the 2006 ACM Conference on Human-Robot Interaction, (March 2006), 297-304.
8. Collins, R.T., Liu, Y., and Leordeanu, M. Online selection of discriminative tracking features. In IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 27, no. 10, (October 2005), 1631-1643.
9. Erman, L.D., Hayes-Roth, F., Lesser, V.R., and Reddy, D. The Hearsay-II Speech-Understanding System: Integrating Knowledge to Resolve Uncertainty. In Computing Surveys, vol. 12, no. 2, (June 1980), 213-253.
10. Harris, T.K., Banerjee, S., Rudnicky, A., Sison, J., Bodine, K., and Black, A. A research platform for multi-agent dialogue dynamics. In Proceedings of the IEEE Intl. Workshop on Robotics and Human Interactive Communications, (September) 2004, 497-502.
11. Iba, S., Weghe, M.V., Paredis, C., and Khosla, P. An architecture for gesture based control of mobile robots. In Proceedings of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS'99), vol. 2, (October 1999), 851-857.
12. Isard, M., and Blake, A. Condensation-conditional density propagation for visual tracking. In Intl. Journal of Computer Vision, (August 1998), vol. 29, no. 1, 5-28.
13. Kim, S., Smyth, P., and Luther, S. Modeling waveform shapes with random effects segmental Hidden Markov models. In AUAI '04: Proceedings of the 20th Conf. on Uncertainty in Artificial Intelligence, 2004, 309-316.
14. Kruppa, H., Castrillon-Santana, M., and Schiele, B. Fast and robust face finding via local context. In Joint IEEE Intl. Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, (October 2003), 157-164.
15. Lienhart, L. and Maydt, J. An extended set of Haar-like features for rapid object detection. In Intl. Conf. on Image Processing, vol.1, 2002, 900-903.
16. Lowe, D.G. Object recognition from local scale invariant features. In Proceedings of the Seventh Intl. Conf. On Computer Vision (ICCV'99), (September 1999), 1150-1157.
17. Luke, R.H., Blisard, S.N., Keller, J.M., Skubic, M. Linguistic spatial relations of three dimensional scenes using SIFT keypoints. In IEEE Intl. Workshop on Robot and Human Interactive Communication: RO-MAN 2005, (August 2005), 704-709.
18. Mungamuru, B., and Aarabi, P. Enhanced sound localization. In IEEE Trans. on Systems, Man, and Cybernetics Part B, vol. 34, no. 3, (June 2004), 1526-1540.
19. Ostendorf, M., Digalakis, V., and Kimball, O. From HMMs to segment models: a unified view of stochastic modeling for speech recognition. In IEEE Trans. on Acoustics, vol. 4, 1996, 360-378.
20. Paquin, V., and Cohen, P. A vision-based gestural guidance interface for mobile robotic platforms. In Computer Vision in Human-Computer Interaction: ECCV Workshop in HCI Proceedings, 2004, 39-47.
21. Perzanowski, D., Schultz, A., Adams, W., Bugajska, M., Marsh, E., Trafton, G., Brock, D., Skubic, M., and Abramson, M. Communicating with teams of cooperative robots. In Multi-Robot Systems: From Swarms to Intelligent Automata. Kluwer: The Netherlands, 2002, 185-193.
22. Quatieri, T. Discrete Time Speech Signal Processing, Pearson Education, Inc.: Dehli, India, 2002.
23. Sidner, C.L., Kidd, C.D., Lee, C.H., and Lesh, N., Where to look: A study of human robot engagement. In ACM International Conference on Intelligent User Interfaces, (January 1994), 78-84.
24. Skubic, M., Perzanowski, D., Blisard, S., Schultz, A., Adams W., Bugajska, M. Spatial language for human-robot dialogs. In IEEE Trans. on Systems, Man, and Cybernetics, Special Issue on Human-Robot Interaction, (May 2004), vol. 34, no. 2, 154-167.
25. Sobottka, K., and Pitas, I. Face localization and facial feature extraction based on shape and color information. In Proc. of the Intl.Conf on Image Processing, (September 1996), 483-486.
26. Starner, T, and Pentland, A. Visual recognition of American Sign Language using Hidden Markov Models. In Intl. Workshop on Automatic Face and Gesture Recognition, 1995, 189-194.
27. Yang, C., Duraiswami, R., and Davis, L.S. Fast multiple object tracking via a hierarchical particle filter. In IEEE Intl. Conf. On Computer Vision, vol. 1, 2005, 212-219.
28. Zhao, L., and Davis, L.S. Iterative figure-ground discrimination. In Intl. Conf. on Pattern Recognition (ICPR), 2005, 67-70.