Performance evaluation and error analysis for multimodal reference resolution in a conversation system

HLT-NAACL 2004 - Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, Short Papers

Volumn , Issue , 2004, Pages 41-44

  Chai, Joyce Y ^a   Prasov, Zahar ^a   Hong, Pengyu ^b  

^a Michigan State University   (United States)

^b HARVARD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

'CURRENT; CONVERSATION SYSTEMS; FUTURE IMPROVEMENTS; HUMAN-MACHINE; MULTI-MODAL; PERFORMANCE ERROR; PERFORMANCES EVALUATION; REFERENCE RESOLUTION; STATE OF THE ART; USER STUDY;

EID: 33644586409     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (17)

1. Bolt, R.A. 1980. Put that there: Voice and Gesture at the Graphics Interface. Computer Graphics14(3): 262-270.
2. Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjalmsson, H. and Yan, H. 1999. Embodiment in Conversational Interfaces: Rea. In Proceedings of the CHI'99 Conference, pp. 520-527. Pittsburgh, PA.
3. Chai, J. Y., Hong, P., and Zhou, M. X. 2004. A probabilistic approach to reference resolution in multimodal user interfaces, Proceedings of 9th International Conference on Intelligent User Interfaces (IUI): 70-77. Madeira, Portugal, January.
4. Chai, J., Pan, S., Zhou, M., and Houck, K. 2002. Context-based Multimodal Interpretation in Conversational Systems.
5. Fourth International Conference on Multimodal Interfaces. Chen, L., Harper, M. and Quek, F. 2002. Gesture patterns during speech repairs. Proceedings of International Conference on Multimodal Interfaces (ICMI).
6. Cohen, P., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., and Clow, J. 1996. Quickset: Multimodal Interaction for Distributed Applications. Proceedings of ACM Multimedia, pp. 31– 40.
7. Gold, S. and Rangarajan, A. 1996. A graduated assignment algorithm for graph-matching. IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 4.
8. Gorniak, P. and Roy, D. 2003.Grounded Semantic Composition for Visual Scenes. Journal of Artificial Intelligence Research.
9. Huls, C., Bos, E., and Classen, W. 1995. Automatic Referent Resolution of Deictic and Anaphoric Expressions. Computational Linguistics, 21(1):59-79.
10. Johnston, M. 1998. Unification-based Multimodal parsing, Proceedings of COLING-ACL.
11. Johnston, M. and Bangalore, S. 2000. Finite-state multimodal parsing and understanding. Proceedings of COLING.
12. Johnston, M., Bangalore, S., Visireddy G., Stent, A., Ehlen, P., Walker, M., Whittaker, S., and Maloor, P. 2002. MATCH: An Architecture for Multimodal Dialog Systems, in Proceedings of ACL.
13. Kehler, A. 2000. Cognitive Status and Form of Reference in Multimodal Human-Computer Interaction, Proceedings of AAAI.
14. Neal, J. G., Thielman, C. Y., Dobes, Z. Haller, S. M., and Shapiro, S. C. 1998. Natural Language with Integrated Deictic and Graphic Gestures. Intelligent User Interfaces, M. Maybury and W. Wahlster (eds.), 38-51.
15. Oviatt, S., DeAngeli, A., and Kuhn, K. 1997. Integration and Synchronization of Input Modes during Multimodal Human-Computer Interaction, In Proceedings of Conference on Human Factors in Computing Systems: CHI '97.
16. Tsai, W.H. and Fu, K.S. 1979. Error-correcting isomorphism of attributed relational graphs for pattern analysis. IEEE Transactions on Systems, Man and Cybernetics, vol. 9, pp. 757−768.
17. Zancanaro, M., Stock, O., and Strapparava, C. 1997. Multimodal Interaction for Information Access: Exploiting Cohesion. Computational Intelligence 13(7):439-464.