A salience-based approach to gesture-speech alignment

HLT-NAACL 2004 - Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, Proceedings of the Main Conference

Volumn , Issue , 2004, Pages 25-32

  Eisenstein, Jacob ^a   Christoudias, C Mario ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

SPEECH;

BASIC STRUCTURE; DATA DRIVEN; GESTURE-SPEECH; KNOWLEDGE BASED; MULTI-MODAL; NOVEL TECHNIQUES; PRONOUN RESOLUTION; RECALL AND PRECISION; SET OF RULES;

KNOWLEDGE BASED SYSTEMS;

EID: 85117714184     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. Richard A. Bolt. 1980. Put-That-There: Voice and gesture at the graphics interface. Computer Graphics, 14(3):262–270.
2. Justine Cassell. 1998. A framework for gesture generation and interpretation. In Computer Vision in Human-Machine Interaction, pages 191–215. Cambridge University Press.
3. Joyce Y. Chai, Pengyu Hong,, and Michelle X. Zhou. 2004. A probabilistic approach to reference resolution in multimodal user interfaces. In Proceedings of 2004 International Conference on Intelligent User Intefaces (IUI’04), pages 70–77.
4. Nicole Chovil. 1992. Discourse-oriented facial displays in conversation. Research on Language and Social Interaction, 25:163–194.
5. Philip R. Cohen, M. Johnston, D. McGee, S. Oviatt, J. Pittman, I. Smith, L. Chen, and J. Clow. 1997. Quickset: Multimodal interaction for distributed applications. In ACM Multimedia’97, pages 31–40. ACM Press.
6. Philip R. Cohen, Rachel Coulston, and Kelly Krout. 2002. Multimodal interaction during multiparty dialogues: Initial results. In IEEE Conference on Multimodal Interfaces.
7. Jacob Eisenstein and Randall Davis. 2003. Natural gesture in descriptive monologues. In UIST’03 Supplemental Proceedings, pages 69–70.
8. Niyu Ge, John Hale, and Eugene Charniak. 1998. A statistical approach to anaphora resolution. In Proceedings of the Sixth Workshop on Very Large Corpora, pages 161–171.
9. David E. Goldberg. 1989. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley.
10. Michael Johnston and Srinivas Bangalore. 2000. Finite-state multimodal parsing and understanding. In Proceedings of COLING-2000. ICCL.
11. Sanshzar Kettebekov and Rajeev Sharma. 2001. Toward natural gesture/speech control of a large display. In Engineering for Human-Computer Interaction (EHCI’01). Lecture Notes in Computer Science. Springer Verlag.
12. Sanshzar Kettebekov, Mohammed Yeasin, and Rajeev Sharma. 2002. Prosody based co-analysis for continuous recognition of coverbal gestures. In International Conference on Multimodal Interfaces (ICMI’02), pages 161–166, Pittsburgh, USA.
13. David B. Koons, Carlton J. Sparrell, and Kristinn R. Thorisson. 1993. Integrating simultaneous input from speech, gaze, and hand gestures. In Intelligent Multimedia Interfaces, pages 257–276. AAAI Press.
14. Shalom Lappin and Herbert J. Leass. 1994. An algorithm for pronominal anaphora resolution. Computational Linguistics, 20(4):535–561.
15. David McNeill. 1992. Hand and Mind. The University of Chicago Press.
16. Ruslan Mitkov, Richard Evans, and Constantin Orasan. 2002. A new, fully automatic version of mitkov’s knowledge-poor pronoun resolution method. In Intelligent Text Processing and Computational Linguistics (CICLing’02), Mexico City, Mexico, February, 17 – 23.
17. Ruslan Mitkov. 1998. Robust pronoun resolution with limited knowledge. In COLING-ACL, pages 869–875.
18. Sharon L. Oviatt, Antonella DeAngeli, and Karen Kuhn. 1997. Integration and synchronization of input modes during multimodal human-computer interaction. In Human Factors in Computing Systems (CHI’97), pages 415–422. ACM Press.
19. Sharon L. Oviatt. 1999. Ten myths of multimodal interaction. Communications of the ACM, 42(11):74–81.
20. Francis Quek, David McNeill, Robert Bryll, Susan Duncan, Xin-Feng Ma, Cemil Kirbas, Karl E. McCullough, and Rashid Ansari. 2002. Multimodal human discourse: gesture and speech. Transactions on Computer-Human Interaction, 9(3):171–193.
21. Elaine Rich and Susann Luperfoy. 1988. An architecture for anaphora resolution. In Proceedings of the Second Conference on Applied Natural Language Processing (ANLP-2), pages 18–24, Texas, USA.
22. Lizhong Wu, Sharon L. Oviatt, and Philip R. Cohen. 1999. Multimodal integration - a statistical view. IEEE Transactions on Multimedia, 1(4):334–341.