Salience modeling based on non-verbal modalities for spoken language understanding

ICMI'06: 8th International Conference on Multimodal Interfaces, Conference Proceeding

Volumn , Issue , 2006, Pages 193-200

  Qu, Shaolin ^a   Chai, Joyce Y ^a  

^a Michigan State University   (United States)

Author keywords

Language modeling; Multimodal interfaces; Salience modeling; Spoken language understanding

Indexed keywords

GESTURE RECOGNITION; MATHEMATICAL MODELS; USER INTERFACES;

CONVERSATIONAL SYSTEMS; DEICTIC GESTURE; LANGUAGE PROCESSING; MULTIMODAL INTERFACES; SALIENCE MODELING; SPOKEN LANGUAGE UNDERSTANDING;

SPEECH RECOGNITION;

EID: 34547181265     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1180995.1181036     Document Type: Conference Paper

References (18)

1. P. F. Brown, V. J. D. Pietra, P. V. deSouza, J. C. Lai, and R. L. Mercer. Class-based n-gram models of natural language. Computational Linguistics, 18(4):467-479, 1992.
2. J. Chai, P. Hong, M. Zhou, and Z. Prasov. Optimization in multimodal interpretation. In Proceedings of 42nd Annual Meeting of Association for Computational Linguistics (ACL), 2004.
3. J. Chai and S. Qu. A salience driven approach to robust input interpretation in multimodal conversational systems. In Conferences on Human Language Technology and Empirical Methods in Natural Language Processing (HLT/EMNLP), 2005.
4. J. Y. Chai, P. Hong, and M. X. Zhou. A probabilistic approach to reference resolution in multimodal user interfaces. In Proceedings of IUI'04, pages 70-77, 2004.
5. C. Huls, E. Bos, and W. Classen. Automatic referent resolution of deictic and anaphoric expressions. Computational Linguistics, 21(1):59-79, 1995.
6. E. J. and C. C. A salience-based approach to gesture-speech alignment. In Proceedings of HLT/NAACL'04, 2004.
7. M. Johnston. Unification-based multimodal parsing. In Proceedings of COLING-ACL'98, 1998.
8. M. Johnston and S. Bangalore. Finite-state multimodal parsing and understanding. In Proceedings of COLING'00, 2000.
9. S. Katz. Estimation of probabilities from sparse data for the language model component of a speech recogniser. IEEE Trans. Acoustics, Speech and Signal Processing, 35(3):400-401, 1987.
10. A. Kehler. Cognitive status and form of reference in multimodal human-computer interaction. In Proceedings of AAAI'00, pages 685-689, 2000.
11. D. Klein and C. D. Manning. Accurate unlexicalized parsing. In Proceedings of the 41st Meeting of the Association for Computational Linguistics, 2003.
12. S. Lappin and H. Leass. An algorithm for pronominal anaphora resolution. Computational Linguistics, (4):535-561, 1994.
13. S. Oviatt. Breaking the robustness barrier: Recent progress on the design of robust multimodal systems. Advances in Computers, 56:305-325, 2002.
14. S. L. Oviatt. Mutual disambiguation of recognition errors in a multimodal architecture. In Proceedings of Conference on Human Factors in Computing Systems: CHI'99, 1999.
15. D. Roy and N. Mukherjee. Towards situated speech understanding: Visual context priming of language models. Computer Speech and Language, 19(2):227-248, 2005.
16. R. Stevenson. The role of salience in the production of referring expressions: A psycholinguistic perspective. In K. van Deemter and R. Kibble, editors, Information Sharing. CSLI Publ., 2002.
17. P. Taylor, S. King, S. Isard, H. Wright, and J. Kowtko. Using intonation to constrain language models in speech recognition. In Proceedings of ICASSP, 1997.
18. W. Walker, P. Lamere, P. Kwok, B. Raj, R. Singh, E. Gouvea, P. Wolf, and J. Woelfel. Sphinx-4: A flexible open source framework for speech recognition. Technical Report TR-2004-139, Sun Microsystems Laboratories, 2004.