Semantic based information fusion in a multimodal interface

Proceedings of the 2005 International Conference on Human-Computer Interaction, HCI'05

Volumn , Issue , 2005, Pages 94-100

  Russ, Gerhard ^a   Sallans, Brian ^a   Hareter, Harald ^a  

^a ARC SEIBERSDORF RESEARCH GMBH   (Austria)

Author keywords

Dictionary; Frames; Fusion; Multimodal interface; Semantic network

Indexed keywords

COMPUTERS; INFORMATION THEORY; INTERACTIVE COMPUTER SYSTEMS; INTERFACES (COMPUTER); KNOWLEDGE MANAGEMENT; SEMANTICS;

DICTIONARY; FRAMES; FUSION; MULTIMODAL INTERFACE; SEMANTIC NETWORK;

HUMAN COMPUTER INTERACTION;

EID: 60749136069     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. Agirre, E., Arregi, X., Artola, X., Diaz de Darraza, A., Evrard, F., Sarasola, K.: Lexical Knowledge Representation in an Intelligent Dictionary Help System. Proceedings of COLING'94, Kyoto, Japan, pp 544-550, 1994.
2. Althoff, F., Al-Hames, M., McGlaun, G., Lang M. "Towards a new approach for integrating multimodal user input based on evolutionary computation". Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '02, vol. 2, pp 2033-2036, 2002.
3. Bolt, R. A.: Put that there 'Voice and gesture at the graphics interface". ACM Computer Graphics, vol. 14, no. 3, pp 262-270, 1980.
4. Campana, E., Baldridge, J., Dowding, J., Hockey, B. A., Remington, R. W., Stone, L. S. "Using eye movements to determine referents in a spoken dialogue system". Proceedings of the Workshop on Perceptive User Interface, Orland, Florida, 2001.
5. Chai, J., Shimei, P., Zhou, M. X., Houck, K. "Context-based multimodal input understanding in conversational systems". Proceedings of the Fourth IEEE International Conference on Multimodal Interfaces, pp 87-92, 14-16 Oct. 2002.
6. Flanagan, J. L., Marsic, I., Medl, A., Burdea, G., Wilder, J. "Multimodal Human/Machine Communication". 10th Tyrrhenian International Workshop on Digital Communications, Multimedia Communications, Ischia, Italy, pp 16-18, September 1998.
7. Kettebekov, S., Sharma, R. "Toward Natural Gesture/Speech Control of a Large Display". Lecture Notes in Computer Science, vol. 2254, Proceedings of the 8th MP International Conference on Engineering for Human-Computer Interaction, pp 221-234, 2001, ISBN 3-540-43044-X.
8. Kettebekov, S., Yeasin, M., Sharma, R. "Prosody based co-analysis for continuous recognition of coverbal gestures". Proceedings of the Fourth IEEE International Conference on Multimodal Interfaces, pp 161-166, 14-16 Oct 2002.
9. Koons, D. B., Sparrell, C. J., Thorisson, K. R. "Integrating simultaneous input from speech, gaze and hand gestures". In Intelligent Multimedia Interfaces, Editor M.Maybury, Menlo Park, CA, MIT, pp 257-276, 1993.
10. Krahnstoever, N., Schapira, E., Kettebekov, S., Sharma, R. "Multimodal Human-Computer Interaction for Crisis Management Systems". Sixth IEEE Workshop on Applications of Computer Vision, Orlando, Florida, December 2002.
11. Nigay, L., Coutaz, J. "A Design Space for Multimodal Systems: Concurrent Processing and Data Fusion". Proceedings of INTERCHI '93, ACM Press: New York, pp 172-178, 1993.
12. Oviatt, S. "User-Centered Modeling and Evaluation of Multimodal Interfaces". Proceedings of the IEEE, vol. 91, no. 9, Sept. 2003.
13. Pavlovic, V. I., Berry, G. A., Huang, T. S. "A Multimodal Human-Computer Interface for me Control of a Virtual Environment". American Association for Artificial Intelligence, Spring Symposium on Intelligent Environments, 1998.
14. Poddar, I., Sethi, Y., Ozyildiz, E., Sharma. R. "Toward Natural Gesture/Speech HCI: A Case Study of Weather Narration". Proceedings of the Workshop on Perceptual User Interfaces, PUI98, San Francisco, CA, pp 1-6, November 1998.
15. Sharma, R., Yeasin, M., Krahnstoever, N., Rauschen, I., Cai, G., Brewer, I., MacEachren, A., Sengupta, K. "Speech-Gesture Driven Multimodal Interfaces for Crisis Management". Proceedings of the IEEE, vol. 91, no. 9, pp 1327-1354, 2003.
16. Sharma, R., Pavlovic, V. I., Huang, T. S. "Toward Multimodal Human-Computer Interface". Proceedings of the IEEE, special issue on Multimedia Signal Processing, 86(5), pp 853-869, May 1998. [ 17] Tanaka, K. "A robust selection system using realtime multi-modal user-agent interactions". Proceedings of the International Conference on Intelligent User Interfaces, Redondo Beach CA USA PP 105-108, 1999.
17. Vo, M. T., Waibel, A. "Modeling and interpreting multimodal inputs: A semantic integration approach". Technical Report CMU-CS-97-192. Carnegie Mellon University, 1997.
18. Zhang, Q., Imamiya, A., Mao, X., Go, K. "A gaze and speech multimodal interface". Proceedings of the 24th International Conference on Distributed Computing Systems Workshops, pp 208-213, 23-24 March 2004.