Using maximum entropy (ME) model to incorporate gesture cues for SU detection

ICMI'06: 8th International Conference on Multimodal Interfaces, Conference Proceeding

Volumn , Issue , 2006, Pages 185-192

  Chen, Lei ^a   Harper, Mary ^a   Huang, Zhongqiang ^a  

^a PURDUE UNIVERSITY   (United States)

Author keywords

Gesture; Language models; Meetings; Multimodal fusion; Prosody; Sentence boundary detection

Indexed keywords

ARTIFICIAL INTELLIGENCE; HIDDEN MARKOV MODELS; IMAGE RECOGNITION; LEARNING SYSTEMS; SPEECH RECOGNITION;

MAXIMUM ENTROPY MODEL; ME MODELING APPROACH; MULTIMODAL INTEGRATION; SENTENCE UNITS;

USER INTERFACES;

EID: 34547201931     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1180995.1181035     Document Type: Conference Paper

References (24)

1. M. Argyle and M. Cook. Gaze and Mutual Gaze. Cambridge Univ. Press, 1976.
2. A. Berger, S. Pietra, and V. Pietra. A maximum entropy approach to natural language processing. Computational Linguistics, 22:39-72, 1996.
3. L. Chen, Y. Liu, M. Harper, and E. Shriberg. Multimodal model integration for sentence unit detection. In Proc. of Int. Conf. on Multimodal Interface (ICMI), University Park, PA, Oct 2004.
4. L. Chen, T. Rose, F. Parrill, X. Han, J. Tu, Z. Huang, I. Kimbara, H. Welji, M. Harper, F. Quek, D. McNeill, S. Duncan, R. Tuttle, and T. Huang. VACE multimodal meeting corpus. In Proc. of the Joint Workshop on Multimodal Interaction and Related Machine Learning Algorithms (MLMI), 2005.
5. S. Coquoz. Broadcast news segmentation using MDE and STT information to improve speech recognition. Technical report, International Computer Science Institute, 2004.
6. J. Eisenstein and R. Davis. Gestural cues for sentence segmentation. MIT AI Memo, 2005.
7. L. Ferrer. Prosodic features extraction. Technical report, SRI, 2002.
8. S. Goldin-Meadow. The role of gesture in communication and thinking. Trends in Cognitive Sciences, 3(11), Nov 1999.
9. M. Harper, B. Dorr, B. Roark, J. Hale, Z. Shafran, Y. Liu, M. Lease, M. Snover, L. Young, R. Stewart, and A. Krasnyanskaya. Final report: Parsing speech and structural event detection. Technical report, John Hopkins CSLP Summer Workshop, 2005.
10. Z. Huang, L. Chen, and M. Harper. An open source prosodic feature extraction tool. In Proc. of the Language Resources and Evaluation Conference (LREC), May 2006.
11. M. Kipp. Anvil: A generic annotation tool for multimodal dialogue. In Proc. of European Conf. on Speech Processing (EuroSpeech), 2001.
12. Y. Liu. Structural Event Detection for Rich Transcription of Speech. PhD thesis, Purdue University, 2004.
13. Y. Liu, N. V. Chawla, E. Shriberg, A. Stolcke, and M. Harper. Resampling techniques for Sentence Boundary Detection: A Case Study in Machine Learning from Imbalanced Data for Spoken Language Processing. Computer Speech and Language, to appear.
14. Y. Liu, E. Shriberg, A. Stockle, and M. Harper. Comparing HMM, Maximum Entropy, and Conditional Random Fields for disfluency detection. In Proc. of InterSpeech, Lisbon, Sept. 2005.
15. Y. Liu, E. Shriberg, A. Stolcke, B. Peskin, J. Ang, H. D., M. Ostendorf, M. Tomalin, P. Woodland, and M. Harper. Structural Metadata, Research in the EARS Program. In Proc. of ICASSP, 2005.
16. Y. Liu, A. Stolcke, E. Shriberg, and M. Harper. Comparing and combining generative and posterior probability models: Some advances in sentence boundary detection in speech. In Proc. of the Conf. on Empirical Methods in Natural Language Processing (EMNLP), 2004.
17. D. McNeill. Hand and Mind: What Gestures Reveal about Thought. Univ. Chicago Press, 1992.
18. D. McNeill. Growth points, catchments, and contexts. Cognitive Studies: Bulletin of the Japanese Cognitive Science Society, 7(1), 2000.
19. F. Quek, D. McNeill, R. Bryll, S. Duncan, X. Ma, C. Kirbas, K. E. McCullough, and R. Ansari. Multimodal human discourse: Gesture and speech. ACM Trans. Comput.-Hum. Interact., 9(3):171-193, 2002.
20. L. R. Rabiner and B. H. Juang. An introduction to hidden Markov models. IEEE ASSP Magazine, 3(1):4-16, 1986.
21. T. Rose, F. Quek, and Y. Shi. Macvissta: A system for multimodal analysis. In Proc. of Int. Conf. on Multimodal Interface (ICMI), 2004.
22. E. Shriberg, A. Stolcke, D. Hakkani-Tur, and G. Tur. Prosody-based automatic segmentation of speech into sentences and topics. Speech Communication, 32(1-2):127-154, 2000.
23. A. Stockle. SRILM - An extensible language modeling toolkit. In Proc. of Int. Conf. on Spoken Language Processing (ICSLP), 2002.
24. L. Zhang. Maximum Entropy Modeling Toolkit for Python and C++. http://homepages.inf.ed.ac.uk/s0450736/maxent.toolkit.html.