Multimodal new vocabulary recognition Through speech and handwriting in a whiteboard scheduling application

International Conference on Intelligent User Interfaces, Proceedings IUI

Volumn , Issue , 2005, Pages 51-58

  Kaiser, Edward C ^a  

^a Center for Human Computer Communication   (United States)

Author keywords

Multimodal interaction; Mutual disambiguation; Vocabulary learning

Indexed keywords

ERROR ANALYSIS; INTERFACES (COMPUTER); LEARNING SYSTEMS; SEMANTICS;

MULTIMODAL INTERACTION; MUTUAL DISAMBIGUATION; VOCABULARY LEARNING;

SPEECH RECOGNITION;

EID: 32344450610     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1040830.1040851     Document Type: Conference Paper

References (39)

1. Breazeal, C., A. Brooks, J. Gray, G. Hoffman, C. Kidd, H. Lee, J. Lieberman, A. Lockerd, and D. Mulanda, Humanoid Robots as Cooperative Partners for People. International Journal of Humanoid Robots (Forthcoming), 2004.1(2).
2. Atkeson, C.G., J.G. Hale, F. Pollick, M. Riley, S. Kotosaka, S. Schaal, T. Shibata, G. Tevatia, A. Ude, S. Vijayakumar, and M. Kawato, Using Humanoid Robots to Study Human Behavior. IEEE Intelligent Systems, 2000. 16(4): p. 46-56.
3. Bluethmann, W., R.O. Ambrose, M. Diftler, S. Askew, E. Huber, M. Goza, F. Rehnmark, C. Lovchik, and D. Magruder, Robonaut: A Robot Designed to Work with Humans in Space. Autonomous Robots, 2003.14(2-3): p. 179-197.
4. Franklin, D. and K. Hammond. The Intelligent Classroom: Providing Competent Assistance, in In Proceedings of International Comference on Autonomous Agents (Agents-2001). 2001.
5. Kaiser, E., A. Olwal, D. McGee, H. Benko, A. Corradini, X. Li, P. Cohen, and S. Feiner. Mutual Disambiguation of 3D Multimodal Interaction in Augmented and Virtual Reality, in International Conference on Mutimodal Interfaces (ICMI '03). 2003.
6. Chung, G., C. Wang, S. Seneff, E. FIlisko, and M. Tang. Combining Linguistic Knowledge and Acoustic Information in Automatic Pronunciation Lexicon Generation, in Interspeech '04. 2004. Jeju Island, Korea.
7. Chung, G., S. Seneff, and C. Wang. Automatic Acquistion of Names Using Speak and Spell Mode in Spoken Dialogue Systems, in Proceedings of HLT-NAACL 2003. 2003. Edmonton, Canada.
8. Chung, G., S. Seneff, C. Wang, and L. Hetherington. A Dynamic Vocabulary Spoken Dialogue Interface, in Interspeech '04. 2004. Jeju Island, Korea.
9. Roy, D. and N. Mukherjee, Visual Context Driven Semantic Priming of Speech Recognition and Understanding. Computer Speech and Language (In press).
10. Kara, L.B. and T.F. Stahovich. An Image-Based Trainable Symbol Recognizer for Sketch-Based Interfaces, in AAAI Fall Symposium Series 2004: Making Pen-Based Interaction Intelligent and Natural. 2004. Arlington, Virginia.
11. Porzel, R. and M. Strube, Towards Context-adaptive Natural Language Processing Systems, in Computational Linguistics for the New Millenium: Divergence or Synergy, M. Klenner and H. Visser, Editors. 2002: Lang, Frankfurt am Main.
12. Chronis, G. and M. Skubic. Sketched-Based Navigation for Mobile Robots, in In Proceedings of the 2003 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2003). 2003. St. Louis, MO.
13. Saund, E. and J. Mahoney. Perceptual Support of Diagram Creation and Editing, in Diagrams 2004 - International Conference on the Theory and Applications of Diagrams. 2004. Cambridge, England.
14. Landay, J.A. and B.A. Myers, Sketching Interfaces: Toward More Human Interface Design. IEEE Computer, 2001. 34(3): p. 56-64.
15. Pook, P.K. and D.H. Ballard. Deictic Teleassistance. in Proc. IEEE/RSJ/GI Int'l Conf. on Intelligent Robots and Systems. 1994. Muenchen, Germany.
16. Tenenbaum, J.B. and F. Xu. Word learning as Bayesian inference, in Proceedings of the 22nd Annual Conference of the Cognitive Science Society. 2000.
17. Furnas, G.W., T.K. Landauer, L.M. Gomez, and S.T. Dumais, The vocabulary problem in human-system communication. Communications of the Association for Computing Machinery, 1987. 30(11): p. 964-971.
18. Asadi, A.O., Automatic Detection and Modeling of New Words in a Large Vocabulary Continuous Speech Recognition System, in Department of Electrical and Computer Engineering. 1991, Northeastern University: Boston.
19. Bazzi, I. and J.R. Glass. Modeling Out-of-Vocabulary Words for Robust Speech Recognition, in Proceedings of the 6th International Conference on Spoken Language Processing. 2000. Beijing, China.
20. Galescu, L., Sub-lexical language models for unlimited vocabulary speech recognition. 2002, ATR: Kyoto, Japan.
21. Meliani, R.E. and D. O'Shaughnessy. New efficient fillers for unlimited word recognition and keyword spotting. in ICSLP '96. 1996. Philadelphia, Pennsylvania, USA.
22. Gorniak, P. and D.K. Roy. Augmenting User Interfaces with Adaptive Speech Commands, in In Proceedings of the International Conference for Multimodal Interfaces. 2003. Vancouver, B.C., Canada.
23. Roy, D. and A. Pentland, Learning Words from Sights and Sounds: A Computational Model. Cognitive Science, 2002. 26(1): p. 113-146.
24. Yu, C. and D.H. Ballard. A Multimodal Learning Interface for Grounding Spoken Language in Sensory Perceptions, in International Conference on Multimodal Interfaces (ICMI '03). 2003. Vancouver, B.C., Canada: ACM Press.
25. Kaiser, E., D. Demirdjian, A. Gruenstein, X. Li, J. Niekrasz, M. Wesson, and S. Kumar. Demo: A Multimodal Learning Interface for Sketch, Speak and Point Creation of a Schedule Chart, in International Conference on Multimodal Interfaces (ICM '04). 2004. State College, PA.
26. Oviatt, S. and E. Olsen. Integration Themes in Multimodal Human-Computer Interaction, in International ConferenceonSpoken Language Processing (ICSLP '94). 1994.
27. Oviatt, S.L., A. DeAngeli, and K. Kuhn. Integration and synchronization of input modes during multimodal humancomputer interaction, in Proceedings of Conference on Human Factors in Computing Systems: CHI '97. 1997. New York:: ACM Press.
28. Anderson, R.J., R. Anderson, C. Hoyer, and S.A. Wolfman. A Study of Digital Ink in Lecture Presentation, in CHI 2004: The 2004 Conference on Human Factors in Computing Systems. 2004. Vienna, Austria.
29. Anderson, R., C. Hoyer, C. Prince, J. Su, F. Videon, and S. Wolfman. Speech, Ink and Slides: The Interaction of Content Channels, in ACM Multimedia. 2004.
30. Neti, C., G. Potamianos, J. Luettin, I. Matthews, H. Glotin, and D. Vergyri. Large-vocabulary audio-visual speech recognition: A summary of the Johns Hopkins Summer 2000 Workshop, in Proc. IEEE Workshop on Multimedia Signal Processing. 2001. Cannes.
31. Kaiser, E.G. and P.R. Cohen. Implementation Testing of a Hybrid Symbolic/Statistical Multimodal Architecture, in Proceedings of the International Conference on Spoken Language Processing (ICSLP 2002). 2002. Denver.
32. Bazzi, I., Modelling Out-of-Vocabulary Words for Robust Speech Recognition, in Electrical Engineering and Computer Science. 2002, Massachusetts Institute of Technology, p. 153.
33. Roy, D., Grounded Spoken Language Acquisition: Experiments in Word Learning. IEEE Transactions on Multimedia, 2003. 5(2): p. 197-209.
34. Demirdjian, D., T. Ko, and T. Darrell. Constraining Human Body Tracking, in Proceedings of the International Conference on Computer Vision. 2003. Nice, France.
35. Oviatt, S.L. Mutual Disambiguation of Recognition Errors in a Multimodal Architecture, in Proceedings of the ACM Conference on Human Factors in Computing Systems. 1999.
36. Black, A.W. and K.A. Lenzo. Flite: a small fast run-time synthesis engine, in The 4th ISCA Worskop on Speech Synthesis. 2001. Perthshire, Scotland.
37. Yu, C. and D.H. Ballard, A Computational Model of Embodied Language Learning. 2003, Computer Science Deptartment, University of Rochester: Rochester, New York.
38. Gogate, L.J., A.S. Walker-Andrews, and L.E. Bahrick, The Intersensory Origins of Word Comprehension: an Ecological-Dynamic Systems View. Development Science, 2001. 4(1): p. 1-37.
39. Yu, C., D.H. Ballard, and R.N. Aslin. The Role of Embodied Intention in Early Lexical Acquisition, in 25th Annual Meeting of Cognitive Science Society (CogSci 2003). 2003. Boston, MA.