Optimization in multimodal interpretation

Proceedings of the Annual Meeting of the Association for Computational Linguistics

Volumn , Issue , 2004, Pages 1-8

  Chai, Joyce Y ^a   Hong, Pengyu ^b   Zhou, Michelle X ^c   Prasov, Zahar ^a  

^a Michigan State University   (United States)

^b HARVARD UNIVERSITY   (United States)

^c IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

VISUAL COMMUNICATION;

DOMAIN SEMANTICS; INTERACTION CHANNEL; MULTI-MODAL; MULTIMODAL CONVERSATION; OPTIMISATIONS; OPTIMIZATION APPROACH; REFERRING EXPRESSIONS; TEMPORAL ALIGNMENT; USER INPUT; VISUAL FEEDBACK;

SEMANTICS;

EID: 85149108619     PISSN: 0736587X     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Bolt, R.A. 1980. Put that there: Voice and Gesture at the Graphics Interface. Computer Graphics, 14(3): 262-270.
2. Blutner, R., 1998. Some Aspects of Optimality In Natural Language Interpretation. Journal of Semantics, 17, 189-216.
3. Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjalmsson, H. and Yan, H. 1999. Embodiment in Conversational Interfaces: Rea. In Proceedings of the CHI'99 Conference, 520-527.
4. Chai, J., Prasov, Z, and Hong, P. 2004b. Performance Evaluation and Error Analysis for Multimodal Reference Resolution in a Conversational System. Proceedings of HLTNAACL 2004 (Companion Volumn).
5. Chai, J. Y., Hong, P., and Zhou, M. X. 2004a. A Probabilistic Approach to Reference Resolution in Multimodal User Interfaces, Proceedings of 9th International Conference on Intelligent User Interfaces (IUI): 70-77.
6. Chai, J., Pan, S., Zhou, M., and Houck, K. 2002. Context-based Multimodal Interpretation in Conversational Systems. Fourth International Conference on Multimodal Interfaces.
7. Cohen, P., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., and Clow, J. 1996. Quickset: Multimodal Interaction for Distributed Applications. Proceedings of ACM Multimedia.
8. Eisner, Jason. 1997. Efficient Generation in Primitive Optimality Theory. Proceedings of ACL’97.
9. Gold, S. and Rangarajan, A. 1996. A Graduated Assignment Algorithm for Graph-matching. IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 4.
10. Gustafson, J., Bell, L., Beskow, J., Boye J., Carlson, R., Edlund, J., Granstrom, B., House D., and Wiren, M. 2000. AdApt – a Multimodal Conversational Dialogue System in an Apartment Domain. Proceedings of 6th International Conference on Spoken Language Processing (ICSLP).
11. Johnston, M, Cohen, P., McGee, D., Oviatt, S., Pittman, J. and Smith, I. 1997. Unification-based Multimodal Integration, Proceedings of ACL’97.
12. Johnston, M. 1998. Unification-based Multimodal Parsing, Proceedings of COLING-ACL’98.
13. Johnston, M. and Bangalore, S. 2000. Finite-state Multimodal Parsing and Understanding. Proceedings of COLING’00.
14. Johnston, M., Bangalore, S., Visireddy G., Stent, A., Ehlen, P., Walker, M., Whittaker, S., and Maloor, P. 2002. MATCH: An Architecture for Multimodal Dialog Systems, Proceedings of ACL’02, Philadelphia, 376-383.
15. Kehler, A. 2000. Cognitive Status and Form of Reference in Multimodal Human-Computer Interaction, Proceedings of AAAI’01, 685-689.
16. Koons, D. B., Sparrell, C. J. and Thorisson, K. R. 1993. Integrating Simultaneous Input from Speech, Gaze, and Hand Gestures. In Intelligent Multimedia Interfaces, M. Maybury, Ed. MIT Press: Menlo Park, CA.
17. Neal, J. G., and Shapiro, S. C. 1991. Intelligent Multimedia Interface Technology. In Intelligent User Interfaces, J. Sullivan & S. Tyler, Eds. ACM: New York.
18. Oviatt, S. L. 1996. Multimodal Interfaces for Dynamic Interactive Maps. In Proceedings of Conference on Human Factors in Computing Systems: CHI '96, 95-102.
19. Oviatt, S., DeAngeli, A., and Kuhn, K., 1997. Integration and Synchronization of Input Modes during Multimodal Human-Computer Interaction, In Proceedings of Conference on Human Factors in Computing Systems: CHI '97.
20. Oviatt, S., Coulston, R., Tomko, S., Xiao, B., Bunsford, R. Wesson, M., and Carmichael, L. 2003. Toward a Theory of Organized Multimodal Integration Patterns during Human-Computer Interaction. In Proceedings of Fifth International Conference on Multimodal Interfaces, 44-51.
21. Prince, A. and Smolensky, P. 1993. Optimality Theory. Constraint Interaction in Generative Grammar. ROA 537. http://roa.rutgers.edu/view.php3?id=845.
22. Stent, A., J. Dowding, J. M. Gawron, E. O. Bratt, and R. Moore. 1999. The Commandtalk Spoken Dialog System. Proceedings of ACL’99, 183–190.
23. Tsai, W.H. and Fu, K.S. 1979. Error-correcting Isomorphism of Attributed Relational Graphs for Pattern Analysis. IEEE Transactions on Systems, Man and Cybernetics., vol. 9.
24. Wahlster, W., 1998. User and Discourse Models for Multimodal Communication. Intelligent User Interfaces, M. Maybury and W. Wahlster (eds.), 359-370.
25. Wu, L., Oviatt, S., and Cohen, P. 1999. Multimodal Integration – A Statistical View, IEEE Transactions on Multimedia, Vol. 1, No. 4, 334-341.