Multimodal interfaces

Artificial Intelligence Review

Volumn 10, Issue 3-4, 1996, Pages 299-319

  Waibel, Alex ^a,b   Vo, Minh Tue ^a   Duchnowski, Paul ^b   Manke, Stefan ^b  

^a Carnegie Mellon University   (United States)

^b UNIVERSITY OF KARLSRUHE   (Germany)

Author keywords

Gesture recognition; Handwriting recognition; Lip reading; Multimodal interfaces; Neural networks; Speech recognition

Indexed keywords

ACOUSTIC WAVES; COMMUNICATION; EYE MOVEMENTS; HUMAN COMPUTER INTERACTION; NEURAL NETWORKS; PATTERN RECOGNITION; SPEECH RECOGNITION;

GESTURE RECOGNITION; HANDWRITING RECOGNITION; LIP READING; MULTIMODAL INTERFACES;

USER INTERFACES;

EID: 0030215826     PISSN: 02692821     EISSN: None     Source Type: Journal    
DOI: 10.1007/BF00127684     Document Type: Article

References (41)

1. Baluja, S. & Pomerleau, D. (1994). Non-Intrusive Gaze Tracking Using Artificial Neural Networks. To appear in Advances in Neural Information Processing Systems 6, Morgan Kaufmann Publishers.
2. Bodenhausen, U., Manke, S. & Waibel, A. (1993). Connectionist Architectural Learning for High Performance Character and Speech Recognition. In Proceedings of ICASSP'93, Vol. 1, 625-628. Minneapolis, MN, U.S.A.
3. Braida, L. D. (1991). Crossmodal Integration in the Identification of Consonant Segments. The Quarterly Journal of Experimental Psychology 43A(3): 647-677.
4. Bregler, C., Hild, H., Manke, S. & Waibel, A. (1993). Improving Connected Letter Recognition by Lipreading. In Proceedings of ICASSP'93, Vol. 1, 557-560, Minneapolis, MN, U.S.A.
5. Bregler, C. (1993). Lippenlesen als Unterstützung zur robusten automatischen Spracherkennung. M.S. Thesis. Fakultaet für Informatik, Universität Karlsruhe.
6. Goldschen, A. J. (1993). Continuous Automatic Speech Recognition by Lipreading. Ph.D. Dissertation, George Washington University.
7. Guyon, I., Albrecht, P., LeCun, Y., Denker, J. & Hubbard, W. (1991). Design of a Neural Network Character Recognizer for a Touch Terminal. Pattern Recognition 24(2): 105-119.
8. Haffner, P., Franzini, M. & Waibel, A. (1991). Integrating Time Alignment and Neural Networks for High Performance Continuous Speech Recognition. In Proceedings of ICASSP'91, Vol. 1, 105-108. Toronto, Canada.
9. Haffner, P. & Waibel, A. (1991). Multi-State Time Delay Neural Networks for Continuous Speech Recognition. Advances in Neural Network Information Processing Systems 4, 135-142. Morgan Kaufmann Publishers.
10. Hauptmann, A. (1989). Speech and Gestures for Graphic Image Manipulation. In Proceedings of CHI'89, 241-245. Austin, TX, U.S.A.
11. Hild, H. & Waibel, A. (1993). Connected Letter Recognition with a Multi-State Time Delay Neural Network. Advances in Neural Information Processing Systems 5, 712-719. Morgan Kaufmann Publishers.
12. Huang, X., Alleva, F., Hon. H., Hwang, M., Lee, K. & Rosenfeld, R. (1993). The SPHINX-II Speech Recognition System: An Overview. Computer Speech and Language 7(2): 137-148.
13. Jackson, P. L. (1988). The Theoretical Minimal Unit for Visual Speech Perception: Visemes and Coarticulation. The Volta Review 90(5): 99-115.
14. Manke, S. & Bodenhausen, U. (1994). A Connectionist Recognizer for On-Line Cursive Handwriting Recognition. In Proceedings of ICASSP'94, Vol. 2, 633-636. Adelaide, Australia.
15. Miller, G. A. & Nicely, P. E. (1955). An Analysis of Perceptual Confusions Among Some English Consonants. Journal of the Acoustical Society of America 27(2): 338-352.
16. Ney, H. (1984). The Use of a One-Stage Dynamic Programming Algorithm for Connected Word Recognition. In IEEE Transactions on Acoustics, Speech and Signal Processing 32(2): 263-271.
17. Nodine, C., Kundel, H., Toto, L. & Krupinski, E. (1992). Recording and Analyzing Eye-position Data Using a Microcomputer Workstation. Behavior Research Methods, Instruments & Computers 24(3): 475-584.
18. Mase, K. & Pentland, A. (1991). Automatic Lipreading by Optical-Flow Analysis. Systems and Computers in Japan 22(6): 67-76.
19. Petajan, E. D. (1984). Automatic Lipreading to Enhance Speech Recognition. Ph.D. Thesis, University of Illinois.
20. Petajan, E. D., Bischoff, B. & Bodoff, D. (1988). An Improved Automatic Lipreading System to Enhance Speech Recognition. In Proceedings of CHI'88, 19-25. Washington, DC, U.S.A.
21. Pomerleau, D., (1992). Neural Network Perception for Mobile Robot Guidance. Ph.D. Thesis, Carnegie Mellon University, CMU-CS-92-115.
22. Rose, R. & Paul, D. (1990). A Hidden Markov Model Based Keyword Recognition Systems. In Proceedings of ICASSP'90, Vol. 1, 129-132. Albuquerque, NM, U.S.A.
23. Rubine, D., (1991). The Automatic Recognition of Gestures. Ph.D. Thesis, Carnegie Mellon University.
24. Rubine, D., (1991). Specifying Gestures by Examples. Computer Graphics 25(4): 329-337.
25. Schwartz, R. & Austin, S. (1991). A Comparison of Several Approximate Algorithms for Finding N-best Hypotheses. In Proceedings of ICASSP'91, Vol. 1, 701-704. Toronto, Canada.
26. Schenkel, M., Guyon, I. & Henderson, D. (1994). On-Line Cursive Script Recognition Using Time Delay Neural Networks and Hidden Markov Models. In Proceedings of ICASSP'94, Vol. 2, 637-640. Adelaide, Australia.
27. Schmidbauer, O. & Tebelskis, J. (1992). An LVQ-based Reference Model for Speaker-Adaptive Speech Recognition. In Proceedings of ICASSP'92, Vol. 1, 441-444. San Francisco, CA, U.S.A.
28. Stork, D. G., Wolff, G. & Levine, E. (1992). Neural Network Lipreading System for Improved Speech Recognition. In Proceedings of IJCNN'92, Vol. 2, 289-295. Baltimore, MD, U.S.A.
29. Summerfield, Q. (1983). Audio-visual Speech Perception, Lipreading and Artificial Stimulation. In Lutman, M. E. & Haggard, M. P. (eds.) Hearing Science and Hearing Disorders, Academic Press: New York.
30. Tebelskis, J. & Waibel, A. (1993). Performance Through Consistency: MS-TDNNs for Large Vocabulary Continuous Speech Recognition. In Advances in Neural Information Processing Systems 5, 696-703. Morgan Kaufmann Publishers.
31. Turk, M. & Pentland, A. (1991). Eigenfaces for Recognition. Journal of Cognitive Neuro-Science 3(1): 71-86.
32. Vo, M. T. & Waibel, A. (1993). A Multimodal Human-Computer Interface: Combination of Speech and Gesture Recognition. In Adjunct Proc. InterCHI'93. Amtersdam, The Netherlands.
33. Vo, M. T. (1994). Incremental Learning using the Time Delay Neural Network. In Proceedings of ICASSP'94, Vol. 2, 629-632. Adelaide. Australia.
34. Waibel, A., Hanazawa, T., Hinton, G., Shikano, K. & Lang, K. (1989). Phoneme Recognition Using Time-Delay Neural Networks. IEEE Transactions on Acoustics, Speech, and Signal Processing 37(3): 328-339.
35. Waibel, A., Jain, A., McNair, A., Saito, H., Hauptmann, A. & Tebelskis, J. (1991). JANUS: A Speech-to-speech Translation System Using Connectionist and Symbolic Processing Strategies. In Proceedings of ICASSP'91, Vol. 2, 793-796. Toronto, Canada.
36. Ward, W. (1991). Understanding Spontaneous Speech: The Phoenix System. In Proceedings of ICASSP'91, Vol. 1, 365-367. Toronto, Canada.
37. Ware, C. & Mikaelian, H. (1987). An Evaluation of an Eye Tracker as a Device for Computer Input. In SIGCHI Bulletin, Spec. Issue, CHI+GI'87, 183-188. Toronto, Canada.
38. Woszczyna, M. et al. (1993). Recent Advances in Janus: A Speech Translation System. In Proceedings of EUROSPEECH'93, Vol. 2, 1295-1298. Berlin, Germany.
39. Yuhas, B. P., Goldstein, M. H., Sejnowski, Jr. & T. J. (1989). Integration of Acoustic and Visual Speech Signals Using Neural Networks. IEEE Communications Magazine 27(11): 65-71.
40. Zeppenfeld, T., & Waibel, A., (1992). A Hybrid Neural Network, Dynamic Programming Word Spotter. In Proceedings of ICASSP'92, Vol. 2, 77-80. San Francisco, CA, U.S.A.
41. Zeppenfeld, T., Houghton, R., & Waibel, A. (1993). Improving the MS-TSNN for Word Spotting. In Proceedings of ICASSP'93, Vol. 2, 475-478. Minneapolis, MN, U.S.A.