An exploration of eye gaze in spoken language processing for multimodal conversational interfaces

NAACL HLT 2007 - Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference

Volumn , Issue , 2007, Pages 284-291

  Qu, Shaolin ^a   Chai, Joyce Y ^a  

^a Michigan State University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONVERSATIONAL INTERFACE; CONVERSATIONAL SYSTEMS; EXPLORATORY STUDIES; EYE-GAZE; GRAPHICAL DISPLAYS; HUMAN-MACHINE; IN-DEPTH INVESTIGATION; LANGUAGE PROCESSING; LANGUAGE UNDERSTANDING; MULTI-MODAL; SPOKEN LANGUAGE PROCESSING; SPOKEN LANGUAGE UNDERSTANDING;

COMPUTATIONAL LINGUISTICS; SPEECH COMMUNICATION;

SPEECH RECOGNITION;

EID: 84858415585     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. E. Campana, J. Baldridge, J. Dowding, B. Hockey, R. Remington, and L. Stone. 2001. Using eye movements to determine referents in a spoken dialogue system. In Proceedings of the Workshop on Perceptive User Interface.
2. J. Chai and S. Qu. 2005. A salience driven approach to robust input interpretation in multimodal conversational systems. In Proceedings of HLT/EMNLP'05.
3. A. T. Duchowski. 2002. A breath-first survey of eye tracking applications. Behavior Research methods, Instruments, and Computers, 33(4).
4. J. Eisenstein and C. M. Christoudias. 2004. A salience-based approach to gesture-speech alignment. In Proceedings of HLT/NAACL'04.
5. Z. M. Griffin and K. Bock. 2000. What the eyes say about speaking. Psychological Science, 11:274-279.
6. J. M. Henderson and F. Ferreira. 2004. The interface of language, vision, and action: Eye movements and the visual world. New York: Taylor & Francis.
7. C. Huls, E. Bos, and W. Classen. 1995. Automatic referent resolution of deictic and anaphoric expressions. Computational Linguistics, 21(1):59-79.
8. R. J. K. Jacob. 1990. What you look is what you get: Eye movement-based interaction techniques. In Proceedings of CHI'90.
9. R. J. K. Jacob. 1995. Eye tracking in advanced interface design. In W. Barfield and T. Furness, editors, Advanced Interface Design and Virtual Environments, pages 258-288. Oxford University Press.
10. M. Just and P. Carpenter. 1976. Eye fixations and cognitive processes. Cognitive Psychology, 8:441-480.
11. S. Katz. 1987. Estimation of probabilities from sparse data for the language model component of a speech recogniser. IEEE Trans. Acous., Speech and Sig. Processing, 35(3):400-401. (Pubitemid 17549162)
12. M. Kaur, M. Termaine, N. Huang, J. Wilder, Z. Gacovski, F. Flippo, and C. S. Mantravadi. 2003. Where is "it"? event synchronization in gaze-speech input systems. In Proceedings of ICMI'03.
13. A. Kehler. 2000. Cognitive status and form of reference in multimodal human-computer interaction. In Proceedings of AAAI'00.
14. S. Qu and J. Chai. 2006. Salience modeling based on nonverbal modalities for spoken language understanding. In Proceedings of ICMI'06.
15. P. Qvarfordt and S. Zhai. 2005. Conversing with the user based on eye-gaze patterns. In Proceedings of CHI'05.
16. D. Roy and N. Mukherjee. 2005. Towards situated speech understanding: Visual context priming of language models. Computer Speech and Language, 19(2):227-248.
17. I. Starker and R. A. Bolt. 1990. A gaze-responsive self-disclosing display. In Proceedings of CHI'90.
18. M. K. Tanenhaus, M. J. Spivey-Knowlton, K. M. Eberhard, and J. E. Sedivy. 1995. Integration of visual and linguistic information in spoken language comprehension. Science, 268:1632-1634.
19. B. M. Velichkovsky. 1995. Communicating attention-gaze position transfer in cooperative problem solving. Pragmatics and Cognition, 3:99-224.
20. R. Vertegaal. 1999. The gaze groupware system: Mediating joint attention in multiparty communication and collaboration. In Proceedings of CHI'99.
21. S. Zhai, C. Morimoto, and S. Ihde. 1999. Manual and gaze input cascaded (magic) pointing. In Proceedings of CHI'99.