Where is "it"? Event synchronization in gaze-speech input systems

ICMI'03: Fifth International Conference on Multimodal Interfaces

Volumn , Issue , 2003, Pages 151-158

  Kaur, Manpreet ^a   Tremaine, Marilyn ^b   Huang, Ning ^a   Wilder, Joseph ^a   Gacovski, Zoran ^a   Flippo, Frans ^a   Mantravadi, Chandra Sekhar ^a  

^a RUTGERS UNIVERSITY   (United States)

^b NEW JERSEY INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Eye Tracking; Gaze Speech Co occurrence; Multimodal Fusion; Multimodal Interfaces

Indexed keywords

DATA REDUCTION; MODAL ANALYSIS; OBJECT RECOGNITION; PROBABILITY; SYNCHRONIZATION; TRACKING (POSITION);

EYE-TRACKING; GAZE-SPEECH CO-OCCURANCES; MULTIMODAL INTERFACES;

SPEECH ANALYSIS;

EID: 9444226930     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/958432.958463     Document Type: Conference Paper

References (34)

1. Bernsen, N. O. and Dybkjær, L.: Is speech the right thing for your application? In Proceedings of the International Conference for Spoken Language Processing, ICSLP'98, Sydney. Australian Speech Science and Technology Association 1998, 3209-3212.
2. Bolt, R. A. The Human Interface. Lifetime Learning Publications, Belmont, CA, 1984.
3. Corno, F., Farineti, L. and Signorile, I. A cost-effective solution for eye-gaze assistive technology. In Proceedings of the ICME 2002 IEEE Conference on Multimedia and Expo, IEEE Press, Piscataway, NJ, 2002.
4. Curry, R., Hung, G. K., Wilder, J. and Julesz, B. Context effect of common objects on visual processing. Optometry and Vision Science Vol. 72, 1995, 452-460.
5. Dabbs, J. M., Jr., Evans, M. S., Hooper, C. H., & Purvis, J. A. Self monitors in conversation: Patterns of speech and gaze. Journal of Personality and Social Psychology Vol. 39, 1980, 278-284.
6. Farid, M. M. and Murtagh, F. Eye-movements and voice as interface modalities to computer systems. In Proceedings of OPTO Ireland, SPIE Press, Bellingham, WA, September 5-6, 2002, CD-ROM.
7. Glenstrup, A. J. and Engell-Nielsen, T. Eye controlled media: present and future state. Technical report, University of Copenhagen, Denmark, 1995.
8. rd Symposium on Human Interaction with Complex Systems, HICS/96, Dayton, OH, August 25-27, 1996, 10-19.
9. Hung, G. K., Wilder, J., Curry, R., and Julesz, B. Simultaneous better than sequential for brief presentations. Journal of the Optical Society of America Vol. 12, 1995, 441-449.
10. Hung, G. K., Wilder, J., Weiss, F. and Curry, R. K, Random and direct path eye movements during target search. Medical Science Research Vol. 21, 1993, 389-391.
11. Jacob, R J. K. The use of eye movements in human-computer interaction techniques: What you look at is what you get. ACM Transactions on Information Systems, Vol. 9 (3), 1991, pp. 152-169.
12. Kapoula, Z., and Robinson, D. A., "Saccadic undershoot is not inevitable: saccades can be accurate," Vision Research Vol. 26, 1986. 735-743.
13. Kaur, M. Integration of Gaze and Speech for Multimodal Human-Computer Interaction. Unpublished Ph.D. dissertation, Department of Biomedical Engineering, Rutgers, the State University, 2000, 142 pages.
14. Koons, D. B., Sparrell, C., J., and Thorisson, K.R. Integrating simultaneous input from speech, gaze, and hand gestures. In M. Maybury (Ed.) Intelligent Multimedia Interfaces, MIT Press, Menlo Park, CA, 1993, 252-276.
15. Kowler, E. and Blaser, E. The accuracy and precision of saccades to small and large targets. Vision Research Vol. 35 (12), 1995, 1741-1754.
16. Lin, W., Kaur, M., Tremaine, M., Hung, G. and Wilder, J., Performance analysis of an eye-tracker. In Proceedings of the SPIE Conference on Machine Vision Applications, Architectures and Systems Integration V, 1999, CD-ROM.
17. Maglio, P. P., Matlock, T., Campbell, C. S., Zhai, S. and Smith, B. A. Gaze and speech in attentive user interfaces. In Proceedings of the Third International Conference on Multimodal Interfaces, ICM 2000, Beijing, China, October 14-16, 2000, 1-7.
18. Mantravadi, C. S., Wilder, J., Grove, D. and Yuan, X. A Java-based multimodal human-computer interface architecture. In Proceedings of ICICS-2001, Singapore, IEEE Press, Piscataway, NJ, 2001, CD-ROM.
19. Nielsen, J. Noncommand user interfaces. Communications of the ACM, 36:83-99, 1993
20. Oviatt, S., Cohen, P., Wu, L., Vergo, J., Duncan, L., Subh, B., Bers, J., Holzman, T., Winograd, T., Landay, J., Larson, J. and Ferro, D. Designing the user interface for multimodal speech and pen-based gesture applications: State-of-the-art systems and future research directions. Human Computer Interaction, Vol. 15 (4), 2000, pp. 263-322.
21. Oviatt, S., DeAngeli, A. and Kuhn, K. Integration and synchronization of input modes during multimodal human-computer interaction. In Proceedings of the ACM CHI'97 Conference on Human Factors in Computing Systems, ACM Press, New York, NY, March 22-27, 1997, pp. 415-422.
22. Pelz, J. B., Canosa, R. and Babcock, J. Extended tasks elicit complex eye movement patterns. In Proceedings of the ACM ETRA 2000 Eye Tracking Research and Applications Symposium, ACM Press, New York, NY, 2000, 37-44.
23. Press, W.H., Vettering, W.T., Teukolsky, S. A., and Flannery, B. P., Numerical Recipes in C, Second Edition, Cambridge University Press, 1992, 691-692.
24. Salvucci, D. D. Interring intent in eye-based interfaces: Tracing eye movements with process models. In Proceedings of the ACM CHI'99 Conference on Human Factors in Computing Systems. ACM Press, New York, NY, May 15-20, 1999, 254-261.
25. Salvucci, D. D. and Anderson, J. R. Intelligent gaze-added interfaces. In Proceedings of the ACM CHI 2000 Conference on Human Factors in Computing Systems, ACM Press, New York, NY, 2000, 273-280.
26. Salvucci, D. D. and Goldberg, J. H. Identifying fixations and saccades in eye-tracking protocols. In Proceedings of the ACM ETRA 2000 Eye Tracking Research and Applications Symposium, ACM Press, New York, NY, 2000, 71-78.
27. Sharma, R., Pavlovic, V. I. and Huang, T. S. Toward multimodal human-computer interfaces. In Proceedings of the IEEE, Vol. 86, (5), May 1998, 853-869.
28. Siebert, L. E. and Jacob, R. J. K. Evaluation of eye gaze interaction. In Proceedings of the ACM CHI 2000 Conference on Human Factors in Computing Systems, ACM Press, New York, NY, April 1-6, 2000, 281-288.
29. Starker, I. and Bolt, R. A., A gaze-responsive self-disclosing display. In Proceedings of the CHI'90 Conference on Human Factors in Computing Systems, ACM Press, New York, NY, 1990, pp. 3-9.
30. Tanenhaus, M. K., Spivey-Knowlton, M., Eberhard, K. and Sedivy, J. Integration of visual and linguistic information during spoken language comprehension. Science, Vol. 268, 1995, pp. 1632-1634.
31. Velichkovsky, B. M. and Hansen, J. P. New technological windows into mind: There is more in eyes and brains for human-computer interaction. In Proceedings of the CHI'96 Conference on: Human Factors in Computing Systems, ACM Press, New York, NY, 1996, 496-503.
32. Vertegaal, R., Slagter, R., van der Veer, G. and Nijholt, A. Eye gaze patterns in conversations: There is more the conversational agents than meets the eyes; In Proceedings of the CHI 2001 Conference on Human factors in Computing Systems, New York, NY, ACM Press, 2001, 301-308.
33. Ware, C. and Mikaelian, H. H. An evaluation of an eye tracker as a device for computer input. In Proceedings of the ACM CHI+GI'87 Conference on Human Factors in Computing Systems, ACM Press, New York, NY, 1987, 183-188.
34. Zhai, S., Morimoto, C. and Ihde, S. Manual and gaze input cascaded (MAGIC) pointing. In Proceedings of the ACM CHI'99 Conference on Human Factors in Computing Systems, ACM Press, New York, NY, May 15-20, 1999, 246-253