Development of speechreading supplements based on automatic speech recognition

IEEE Transactions on Biomedical Engineering

Volumn 47, Issue 4, 2000, Pages 487-496

  Duchnowski, Paul ^a   Lum, David S ^b   Krause, Jean C ^b   Sexton, Matthew G ^b   Bratakos, Maroula S ^b   Braida, Louis D ^b  

^a IEEE   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Aids for the deaf; Autocuer; Automatic speech recognition; Cued speech; Deafness; Speech perception; Speechreading; Transliteration

Indexed keywords

MARKOV PROCESSES; SPEECH RECOGNITION; SYNCHRONIZATION;

SPEECHREADING;

BIOMEDICAL ENGINEERING;

ADULT; ARTICLE; ASSOCIATION; CLINICAL ARTICLE; HAND MOVEMENT; HEARING IMPAIRMENT; HUMAN; INTERPERSONAL COMMUNICATION; SPEECH DISCRIMINATION; SPEECH PERCEPTION; SPEECH THERAPY;

ADULT; CHILD; COMMUNICATION AIDS FOR DISABLED; COMPUTER SIMULATION; CUES; DATA DISPLAY; DEAFNESS; GESTURES; HUMANS; LIPREADING; MARKOV CHAINS; MODELS, BIOLOGICAL; NATURAL LANGUAGE PROCESSING; SIGN LANGUAGE; SPEECH INTELLIGIBILITY; SPEECH PERCEPTION; SPEECH PRODUCTION MEASUREMENT; USER-COMPUTER INTERFACE;

EID: 0034071757     PISSN: 00189294     EISSN: None     Source Type: Journal    
DOI: 10.1109/10.828148     Document Type: Article

References (43)

1. K. Grant and L. Braida, "Evaluating the articulation index for audiovisual input," J. Acoust. Soc. Amer., vol. 89, pp. 2952-2960, 1991.
2. E. Douglas-Cowie, "Acquired deafness and communication," Coping with Acquired Hearing Loss, pp. 13-22, 1988.
3. E. Klima and U. Bellugi, The Signs of Language. Cambridge, MA: Harvard Univ. Press, 1979.
4. J. E. Wandel, "Use of internal speech in reading by hearing and hearing impaired students in oral, total communication, and cued speech programs," Ph.D. dissertation, Teacher's College, Columbia Univ., New York, 1989.
5. M. J. Osberger, L. Fisher, S. Zimmerman-Phillips, L. Geier, and M. J. Baker, "Speech recognition performance of older children with cochlear implants," Amer. J. Otol, vol. 19, pp. 152-157, March 1998.
6. G. M. Clark, "Cochlear implants in the second and third millennia," in Proc. 1CSLP-98, December 1998, pp. 1-6.
7. H. W. Upton, "Wearable eyeglass speechreading aid," Amer. Ann. Deaf, vol. 113, pp. 222-229, March 1968.
8. D. Ebrahimi and H. Kunov, "Peripheral vision lipreading aid," IEEE Trans. Biomed. Eng., vol. 38, pp. 944-952, Oct. 1991.
9. D. K. Oiler, Ed., Tactile Aids for the Hearing Impaired. New York: Int. Sensory Aids Soc., Thieme Medical, November 1995.
10. R.O.Cornett,"Cuedspeech,"Amer. Ann. Deaf, vol. 112,pp.3-13, 1967.
11. G. Fant, "Q-codes," in Proc. Int. Symp. Speech Communication Ability and Profound Deafness, G. Fant, Ed., Washington, DC, 1970, pp. 261-299.
12. S. Hiki and Y. Fukuda, "Proposal of a system of manual signs as an aid for Japanese lipreading," J. Acoust. Soc. Jpn., vol. 2, no. 2, pp. 127-129, 1981.
13. R. Uchanski, L. Delhorne, A. Dix, C. M. Reed, L. Braida, and N. Durlach, "Automatic speech recognition to aid the hearing impaired: Prospects for the automatic generation of cued speech," J. Rehab. Res. Dev., vol. 31, no. 1, pp. 201, 1994.
14. H. Kaplan, "The effects of cued speech on the speechreading ability of the deaf," Ph.D. dissertation, Univ. Maryland, College Park, 1974.
15. D. Ling and B. Clarke, "Cued speech: An evaluative study," Amer. Ann. Deaf, vol. 120, pp. 480-488, 1975.
16. B. Clarke and D. Ling, "The effects of using cued speech: A follow up study," Volta Rev., vol. 78, pp. 23-34, 1976.
17. G. Nicholls and D. Ling, "Cued speech and the reception of spoken language," J. Speech Hearing Res., vol. 25, pp. 262-269, 1982.
18. "IEEE recommended practices for speech quality measurements," IEEE, New York, Tec. Rep., 1969.
19. M.S. Bratakos, P. Duchnowski, and L. D. Braida, "Toward the automatic generation of cued speech," The Cued Speech J., vol. 6, pp. 1-37, June 1997.
20. O. Cornett, R. Beadles, and B. Wilson, "Automatic cued speech," Processing Aids for the Deaf, pp. 224-239, 1977.
21. R. Beadles and L. D. Braida, "Describing the performance of the RTI Autocuer and the results of tests with simulations of the Autocuer," unpublished, August 31, 1989.
22. D. O'Shaughnessy, Speech Communication, Human and Machine. Reading, MA: Addison-Wesley, 1990.
23. H. Hermansky and N. Morgan, "RASTA processing of speech," IEEE Trans. Acoust., Speech, Signal Processing, vol. 24, pp. 52-59, Oct. 1994.
24. L. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, pp. 257-286, Feb. 1989.
25. S. Y P. Woodland and W. Byrne, HTK: Hidden Marchkov Model Toolkit, Version 1.5 User's Manual. Washington, DC: Entropies Inc., 1993.
26. P. Woodland, C. Leggetter, J. Odell, V. Valtchev, and S. Young, "The 1994 HTK large vocabulary speech recognition system," in Proc. 1995 Int. Conf. Acoust. Speech and Signal Processing, 1995, pp. 73-76.
27. S. Furui, "Speaker-independent isolated word recognition using dynamic features of the speech spectrum," IEEE Trans. Acoust. Speech Signal Processing, vol. ASSP-34, pp. 52-59, Feb. 1986.
28. L. Baum, "An inequality and associated maximization technique in statistical estimation of probabilistic functions of Markov processes," Inequalities, vol. 3, pp. 1-8, 1972.
29. L. Bahl, F. Jelinek, and R. Mercer, "A maximum likelihood approach to continuous speech recognition," IEEE Trans. Pattern Anal. Machine Intell, vol. PAMI-5, pp. 179-190, Feb. 1983.
30. L. Lamel, R. Kassel, and S. Seneff, "Speech database development: Design and analysis of the acoustic-phonetic corpus," in Proc. DARPA Speech Recognition Workshop, L. Baumann, Ed., Feb. 1986, pp. 100-109.
31. W. Fisher, V. Zue, J. Bernstein, and D. Pallett, "An acoustic-phonetic data base," J. Acoust. Soc. Amer., vol. 81, no. SI, p. S92, 1987.
32. S. Austin, R. Schwartz, and P. Placeway, "The forward-backward search algorithm," in Proc. 1991 Int. Conf. Speech, Acoustics, and Signal Processing, vol. 1, 1991, pp. 697-700.
33. P. Duchnowski, L. D. Braida, M. S. Bratakos, D. S. Lum, M. G. Sexton, and J. C. Krause, "A speechreading aid based on phonetic ASR," in Proc. ICSLP-98, Dec. 1998, pp. 3289-3293.
34. M. G. Sexton, "A video display system for an automatic cue generator," Master's thesis, M.I.T., Cambridge, MA, Feb. 1997.
35. A. Boothroyd, T. Hnath-Chisolm, and L. Hanin, "A sentence test of speech perception: Reliability, set-equivalence, and short-term learning," City Univ. New York, New York, Tech. Rep. RCI10, 1985.
36. A. Huggins, R. Houde, and E. Colwell, "Vidvox human factors investigation," BBN Laboratories, Cambridge, MA, Tech. Rep. 6187, 1986.
37. S. Young, "A review of large-vocabulary continuous-speech recognition," in IEEE Signal Processing Mag., vol. 13, 1996, pp. 45-57.
38. D. Systems. (1997, December) Product information for naturally speaking speech recognition software. [Online] Available: HTTP: http://www.dragonsystems.com
39. S. Young, J. Odell, and P. Woodland, "Tree-based state tying for high accuracy acoustic modelling," in Proc. ARPA Human Lang. Tech. Workshop, March 1994, pp. 615-621.
40. R. W. Gengel, "Upton's wearable eyeglass speechreading aid: History and current status," in Hearing and Davis: Essays Honoring Hallowell Davis, S. Hirsh, D. Eldridge, I. J. Hirsh, and S. Silverman, Eds. St. Louis, MO: Washington Univ. Press, 1976, pp. 291-299.
41. C. Reed and L. Delhorne, "Current results of a field study of adult users of tactile aids," Seminars Hearing, pp. 305-315, 1995.
42. J. Yang and A. Waibel, "Tracking human faces in real time," Carnegie Mellon University, School of Computer Science, Pittsburgh, PA, Tech. Rep. CMU-CS-95-210, 1995.
43. N. Oliver, A. Pentland, F. Berard, and J. Coutaz, "LAFTER: Lips and face tracker," MIT Media Lab., Cambridge, MA, Tech. Rep. 396, 1996.