Multimodal Error Correction for Speech User Interfaces

ACM Transactions on Computer-Human Interaction

Volumn 8, Issue 1, 2001, Pages 60-98

  Suhm, Bernhard ^a   Myers, Brad ^b   Waibel, Alex ^c  

^a BBN TECHNOLOGIES   (United States)

^b Carnegie Mellon University   (United States)

^c CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Design; dictation systems; Experimentation; Human Factors; interactive error correction; Measurement; Multimodal interfaces; pen input; performance model; speech input; speech user interfaces

Indexed keywords

EID: 85009262210     PISSN: 10730516     EISSN: 15577325     Source Type: Journal    
DOI: 10.1145/371127.371166     Document Type: Article

References (40)

1. Ainsworth, W. A. and Pratt, S. R. 1992. Feedback strategies for error correction in speech recognition systems. Int. J. Man-Mach. Stud. 36, 6 (June), 833-842
2. Alto, P., Brandetti, M., Ferretti, M., Maltese, G., and Scarci, S. 1989. Experimenting natural-language dictation with a 20.000-word speech recognizer. VLSI and Computer Peripherals: VLSI Comp. Periph. 2, 78-81
3. Baber, C. and Hone, K. S. 1993. Modeling error recovery and repair in automatic speech recognition. Int. J. Man-Mach. Stud. 39, 3 (Sept.), 495-515
4. Brinton, B., Fujiki, M., and Sonnenberg, E. A. 1988. Responses to requests for clarification in linguistically normal and language-impaired children in conversation. J. Sp. Hrg. Dis. 53, 383-391
5. Chase, L. L. 1997. Error-response feedback mechanisms for speech recognizers. Ph.D. Dissertation. Computer Science Department, Carnegie Mellon University, Pittsburgh, PA
6. Cohen, P. R., Johnston, M., Mcgee, D., Oviatt, S. L., Clow, J., and Smith, I. 1998. The efficiency of multimodal interaction: A case study. In Proceedings of the International Conference on Spoken Language Systems (Sydney, Australia)
7. Gould, J. D. 1978. How experts dictate. J. Exp. Psychol. 4, 648-661
8. Gould, J. D., Conti, J., and Hovanyecz, T. 1983. Composing letters with a simulated listening typewriter. Commun. ACM 26, 4 (Apr.), 295-308
9. Hild, H. 1997. Buchstabiererkennung mit neuronalen Netzen in Auskunftssystemen (Spelling recognition for directory assistance systems with neural networks). Ph.D. Dissertation
10. Jelinek, F. 1990. Self-organized language modeling for speech recognition. In Readings in Speech Recognition, A. Waibel and K.-F. Lee, Eds. Morgan Kaufmann Publishers Inc., San Francisco, CA, 450-506
11. Karat, C.-M., Halverson, C., Horn, D., and Karat, J. 1999. Patterns of entry and correction in large vocabulary continuous speech recognition systems. In Proceedings of the International Conference on Computer-Human Interaction (Pittsburgh, PA). 568-575
12. Karat, J., Horn, D. B., Halverson, C. A., and Karat, C. -M. 2000. Overcoming unusability: Developing efficient strategies in speech recognition systems. In Proceedings of the International Conference on Computer-Human Interaction (Amsterdam, The Netherlands)
13. Kemp, T. and Schaaf, T. 1997. Estimating confidence using word lattices. In Proceedings of the European Conference on Speech Communication and Technology (Rhodos, Greece). 827-830
14. Kieras, D. E., Wood, S. D., and Meyer, D. E. 1997. Predictive engineering models based on the EPIC architecture for a multimodal high-performance human-computer interaction task. ACM Trans. Comput. Hum. Interact. 4, 3, 230-275
15. Lai, J. and Vergo, J. 1997. MedSpeak: Report creation with continuous speech recognition. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ‘97, Atlanta, GA, Mar. 22-27), S. Pemberton, Ed. ACM Press, New York, NY, 431-438
16. Levow, G.-A. 1998. Characterizing and recognizing spoken corrections in human-computer dialogue. In Proceedings of the Annual Meeting of the Association on Computer Linguistic (Montreal, Canada). 736-742
17. Manke, S. 1978. On-line erkennung kursiver handschrift bei groBen vokabularien (On-line recognition of cursive handwriting with large vocabularies). Ph.D. Dissertation
18. Manke, S., Finke, M., and Waibel, A. 1995. NPen+ +: A writer independent, large vocabulary on-line cursive handwriting recognition system. In Proceedings of the International Conference on Document Analysis and Recognition (Montreal, Quebec)
19. Mankoff, J., Hudson, S., and Abowd, G. 2000. Providing integrated toolkit-level support for ambiguity in recognition-based interfaces. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI’ 00, The Hauge, The Netherlands, Apr.). ACM Press, New York, NY, 368-375
20. Martin, T. B. and Welch, J. R. 1980. Practical speech recognizers and some performance effectiveness parameters. In Trends in Speech Recognition, W. A. Lea, Ed. Prentice Hall Press, Upper Saddle River, NJ
21. Mcnair, A. E. and Waibel, A. 1994. Improving recognizer acceptance through robust, natural speech repair. In Proceedings of the International Conference on Spoken Language Processing (Yokohama, Japan). 1299-1302
22. Mellor, B. and Baber, C. 1997. Modelling of speech-based user interfaces. In Proceedings of the European Conference on Speech Communication and Technology (Rhodos, Greece). 2263-2266
23. Murray, A. C., Frankish, C. R., and Jones, D. M. 1993. Data-entry by voice: Facilitating correction of misrecognitions. In Interactive Speech Technology: Human Factors Issues in the Application of Speech Input/Output to Computers, C. Baber and J. M. Noyes, Eds. Taylor and Francis, Inc., Bristol, PA, 137-144
24. Oviatt, S. 1999. Mutual disambiguation of recognition errors in a multimodal architecture. In Proceedings of the International Conference on Computer-Human Interaction (Pittsburgh, PA). 576-583
25. Oviatt, S. and Vangent, R. 1994. Error resolution during multimodal human-computer interaction. in Proceedings of the International Conference on Spoken Language Processing. Accoustical Society of Japan, Japan, 204-207
26. Oviatt, S., Deangeli, A., and Kuhn, K. 1997. Integration and synchronization of input modes during multimodal human-computer interaction. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ‘97, Atlanta, GA, Mar. 22-27), S. Pemberton, Ed. ACM Press, New York, NY, 415-422
27. Oviatt, S., Levow, G. A., Maceachern, M., and Kuhn, K. 1994. Modeling hyperarticulate speech during human-computer error resolution. in Proceedings of the International Conference on Spoken Language Processing. Accoustical Society of Japan, Japan, 797-800
28. Rhyne, J. R. and Wolf, C. G. 1993. Recognition-based user interfaces. In Advances in Human-Computer Interaction, H. R. Hartson and D. Hix, Eds. Ablex Publishing Corp., Norwood, NJ, 191-212
29. Robbe, S., Carbonell, N., and Valot, C. 1994. Towards usable multimodal command languages: Definition and ergonomic assessment of constraints on users’ spontaneous speech and gestures. in Proceedings of the International Conference on Spoken Language Processing (Yokohama, Japan). 1655-1658
30. Rogina, I. and Waibel, A. 1995. The JANUS speech recognizer. In Proceedings of the ARPA Workshop on Spoken Language Technology (Austin, TX). 166-169
31. Rubine, D. 1991. Specifying gestures by example. SIGGRAPH Comput. Graph. 25, 4 (July), 329-337
32. Shneiderman, B. 1997. Designing the User Interface: Strategies for Effective Human-Computer Interaction. Addison-Wesley, Reading, MA
33. Soltau, H. and Waibel, A. 1998. On the influence of hyperarticulated speech on recognition performance. in Proceedings of the International Conference on Spoken Language Systems (Sydney, Australia)
34. Suhm, B. 1997. Exploiting repair context in interactive error recovery. In Proceedings of the European Conference on Speech Communication and Technology (Rhodos, Greece). 1659-1661
35. Suhm, B. 1998. Multimodal interactive error recovery for non-conversational speech user interfaces. Ph.D. Dissertation
36. Suhm, B., Myers, B., and Waibel, A. 1996. Interactive recovery from speech recognition errors in speech user interfaces. In Proceedings of the International Conference on Spoken Language Processing (Philadelphia, PA). 861-864
37. Suhm, B., Myers, B., and Waibel, A. 1999. Model-based and Empirical evaluation of multimodal interactive error correction. in Proceedings of the International Conference on Computer-Human Interaction (Pittsburgh, PA). 584-591
38. Vergo, J. 1999. Multimodal text creation and manipulation. Presented at the CHI Workshop on Designing the User Interface for Pen and Speech Multimedia Applications. Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY
39. Vo, M. T. 1998. A framework and toolkit for the construction of multimodal learning interfaces. Ph.D. Dissertation. Computer Science Department, Carnegie Mellon University, Pittsburgh, PA
40. Wolf, C. G. and Morrel-Samuels, P. 1987. The use of hand-drawn gestures for text editing. Int. J. Man-Mach. Stud. 27, 1 (July 1), 91-102