Spectral and temporal cues for speech recognition: Implications for auditory prostheses

Hearing Research

Volumn 242, Issue 1-2, 2008, Pages 132-140

  Xu, Li ^a,b   Pfingst, Bryan E ^b  

^a OHIO UNIVERSITY   (United States)

^b UNIVERSITY OF MICHIGAN HEALTH SYSTEM   (United States)

Author keywords

Cochlear implants; Spectral cues; Speech recognition; Temporal cues

Indexed keywords

ARTICLE; ASSOCIATION; COCHLEA PROSTHESIS; CONSONANT; HEARING AID; INFORMATION PROCESSING; NOISE; PHONEME; PRIORITY JOURNAL; SIMULATION; SPEECH DISCRIMINATION; TONOTOPY; VOWEL;

ACOUSTICS; AUDITORY PATHWAYS; AUDITORY PERCEPTION; COCHLEAR IMPLANTS; CUES; ELECTRIC STIMULATION; HUMANS; LANGUAGE; PATTERN RECOGNITION, PHYSIOLOGICAL; PITCH PERCEPTION; SPEECH PERCEPTION;

EID: 48449104918     PISSN: 03785955     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.heares.2007.12.010     Document Type: Article

References (81)

1. Baer T., and Moore B.C.J. Effects of spectral smearing on the intelligibility of sentences in noise. J. Acoust. Soc. Am. 94 (1993) 1229-1241
2. Baer T., and Moore B.C.J. Effects of spectral smearing on the intelligibility of sentences in the presence of interfering speech. J. Acoust. Soc. Am. 95 (1994) 2277-2280
3. Baskent D. Speech recognition in normal hearing and sensorineural hearing loss as a function of the number of spectral channels. J. Acoust. Soc. Am. 120 (2006) 2908-2925
4. Boersma, P., Weenink, D., 2007. Praat: Doing phonetics by computer (Version 4.6.09). Retrieved July 10, 2007, from .
5. Boothroyd A., Mulhearn B., Gong J., and Ostroff J. Effects of spectral smearing on phoneme and word recognition. J. Acoust. Soc. Am. 100 (1996) 1807-1818
6. Ciocca V., Francis A.L., Aisha R., and Wong L. The perception of Cantonese lexical tones by early-deafened cochlear implantees. J. Acoust. Soc. Am. 111 (2002) 2250-2256
7. Dorman M.F., Loizou P.C., and Rainey D. Speech intelligibility as a function of the number of channels of stimulation for signal processors using sine-wave and noise-band outputs. J. Acoust. Soc. Am. 102 (1997) 2403-2411
8. Drullman R., Festen J.M., and Plomp R. Effect of temporal envelope smearing on speech perception. J. Acoust. Soc. Am. 95 (1994) 1053-1064
9. Drullman R., Festen J.M., and Plomp R. Effect of reducing slow temporal modulations on speech reception. J. Acoust. Soc. Am. 95 (1994) 2670-2680
10. Duanmu S. The Phonology of Standard Chinese (2002), Oxford University Press, Oxford
11. Dudley H. The vocoder. Bell Labs Rec. 17 (1939) 122-126
12. Fishman K.E., Shannon R.V., and Slattery W.H. Speech recognition as a function of the number of electrodes used in the SPEAK cochlear implant speech processor. J. Speech Lang. Hear. Res. 40 (1997) 1201-1215
13. Friesen L.M., Shannon R.V., Baskent D., and Wang X. Speech recognition in noise as a function of the number of spectral channels: Comparison of acoustic hearing and cochlear implants. J. Acoust. Soc. Am. 110 (2001) 1150-1163
14. Friesen L.M., Shannon R.V., and Cruz R.J. Effects of stimulation rate on speech recognition with cochlear implants. Audiol. NeuroOtol. 10 (2005) 169-184
15. Fujita S., and Ito J. Ability of nucleus cochlear implantees to recognize music. Ann. Otol. Rhinol. Laryngol. 108 (1999) 634-640
16. Fu Q.-J., and Shannon R.V. Effect of stimulation rate on phoneme recognition by Nucleus-22 cochlear implant listeners. J. Acoust. Soc. Am. 107 (2000) 589-597
17. Fu Q.-J., and Zeng F.-G. Identification of temporal envelope cues in Chinese tone recognition. Asia Pac. J. Speech, Lang. Hear. 5 (2000) 45-57
18. Fu Q.-J., Shannon R.V., and Wang X. Effects of noise and spectral resolution on vowel and consonant recognition: Acoustic and electric hearing. J. Acoust. Soc. Am. 104 (1998) 3586-3596
19. Fu Q.-J., Hsu C.-J., and Horng M.-J. Effects of speech processing strategy on Chinese tone recognition by Nucleus-24 cochlear implant patients. Ear Hear. 25 (2004) 501-508
20. Geurts L., and Wouters J. Coding of the fundamental frequency in continuous interleaved sampling processors for cochlear implants. J. Acoust. Soc. Am. 109 (2001) 713-726
21. Geurts L., and Wouters J. Better place-coding of the fundamental frequency in cochlear implants. J. Acoust. Soc. Am. 115 (2004) 844-852
22. Gfeller K., Turner C., Woodworth G., Mehr M., Fearn R., Witt S., and Stordahl J. Recognition of familiar melodies by adult cochlear implant recipients and normal hearing adults. Cochlear Implant Int. 3 (2002) 29-53
23. Gfeller K., Turner C., Oleson J., Zhang X., Gantz B., Froman R., and Olszewski C. Accuracy of cochlear implant recipients on pitch perception, melody recognition, and speech reception in noise. Ear Hear. 28 3 (2007) 412-423
24. Green T., Faulkner A., and Rosen S. Enhancing temporal cues to voice pitch in continuous interleaved sampling cochlear implants. J. Acoust. Soc. Am. 116 (2004) 2298-2310
25. Green T., Faulkner A., Rosen S., and Macherey O. Enhancement of temporal periodicity cues in cochlear implants: Effects on prosodic perception and vowel identification. J. Acoust. Soc. Am. 118 (2005) 375-385
26. Hamilton N., Green T., and Faulkner A. Use of a single channel dedicated to conveying enhanced temporal periodicity cues in cochlear implants: Effects on prosodic perception and vowel identification. Int. J. Audiol. 46 (2007) 244-253
27. Hill F.J., McRae L.P., and McClellan R.P. Speech recognition as a function of channel capacity in a discrete set of channels. J. Acoust. Soc. Am. 44 (1968) 13-18
28. Hillenbrand J., Getty L.A., Clark M.J., and Wheeler K. Acoustic characteristics of American English vowels. J. Acoust. Soc. Am. 97 (1995) 3099-3111
29. Huang T.S., Wang N.M., and Liu S.Y. Tone perception of Mandarin-speaking postlingually deaf implantees using the nucleus 22-channel cochlear mini system. Ann. Otol. Rhinol. Laryngol. Suppl. 166 (1995) 294-298
30. Huang T.S., Wang N.M., and Liu S.Y. Nucleus 22-channel cochlear mini-system implantations in Mandarin-speaking patients. Am. J. Otol. 17 (1996) 46-52
31. Kong Y.-Y., and Zeng F.-G. Temporal and spectral cues in Mandarin tone recognition. J. Acoust. Soc. Am. 120 (2006) 2830-2840
32. Kong Y.-Y., Cruz R., Jones J.A., and Zeng F.-G. Music perception with temporal cues in acoustic and electric hearing. Ear Hear. 25 (2004) 173-185
33. Laneau J., Wouters J., and Moonen M. Improving music perception with explicit pitch coding in cochlear implants. Audiol. NeuroOtol. 11 (2006) 38-52
34. Lee K.Y.S., Van Hasselt C.A., Chiu S.N., and Cheung D.M.C. Cantonese tone perception ability of cochlear implant children in comparison with normal-hearing children. Int. J. Pediatr. Otorhinolaryngol. 63 (2002) 137-147
35. Litvak L., Delgutte B., and Eddington D. Improved neural representation of vowels in electric stimulation using desynchronizing pulse trains. J. Acoust. Soc. Am. 114 (2003) 2099-2111
36. Litvak L.M., Delgutte B., and Eddington D.K. Improved temporal coding of sinusoids in electric stimulation of the auditory nerve using desynchronizing pulse trains. J. Acoust. Soc. Am. 114 (2003) 2079-2098
37. Litvak L.M., Smith Z.M., Delgutte B., and Eddington D.K. Desynchronization of electrically evoked auditory-nerve activity by high-frequency pulse trains of long duration. J. Acoust. Soc. Am. 114 (2003) 2066-2078
38. Liu T.-C., Chen H.-P., and Lin H.-C. Effects of limiting the number of active electrodes on Mandarin tone perception in young children using cochlear implants. Acta Otolaryngol. 124 (2004) 1149-1154
39. Loizou P.C., Dorman M., and Tu Z. On the number of channels needed to understand speech. J. Acoust. Soc. Am. 106 (1999) 2097-2103
40. Luo X., and Fu Q.-J. Enhancing Chinese tone recognition by manipulating amplitude envelope: Implications for cochlear implants. J. Acoust. Soc. Am. 116 (2004) 3659-3667
41. McDermott H. Music perception with cochlear implants: A review. Trends Amplification 8 (2004) 49-81
42. Nie K., Barco A., and Zeng F.-G. Spectral and temporal cues in cochlear implant speech perception. Ear Hear. 27 (2006) 208-217
43. Nilsson M., Sali S.D., and Sullivan J.A. Development of the Hearing in Noise Test for the measurement of speech reception thresholds in quiet and in noise. J. Acoust. Soc. Am. 95 (1994) 1085-1099
44. Qin M.K., and Oxenham A.J. Effects of simulated cochlear implant processing on speech reception in fluctuating maskers. J. Acoust. Soc. Am. 114 (2003) 446-454
45. Remez R.E., Rubin P.E., Pisoni D.B., and Carrell T.D. Speech perception without traditional speech cues. Science 212 (1981) 947-950
46. Rosen S. Temporal information in speech: acoustic, auditory and linguistic aspects. Philos. Trans. R. Soc. London, Ser. B 336 (1992) 367-373
47. Rubinstein J.T., and Hong R.S. Signal coding in cochlear implants: exploiting effects of electrical stimulation. Ann. Otol. Rhinol. Laryngol. 112 Suppl. 191 (2003) 14-19
48. Rubinstein J.T., Wilson B.S., Finley C.C., and Abbas P.J. Pseudospontaneous activity: stochastic independence of auditory nerve fibers with electrical stimulation. Hear. Res. 127 (1999) 108-118
49. Schroeder M.R. Vocoders: analysis and synthesis of speech. Proc. IEEE 54 (1966) 352-366
50. Shannon R.V. Multichannel electrical stimulation of the auditory nerve in man. I. Basic psychophysics. Hear. Res. 11 (1983) 157-189
51. Shannon R.V. Temporal modulation transfer functions in patients with cochlear implants. J. Acoust. Soc. Am. 91 (1992) 2156-2164
52. Shannon R.V., Zeng F.-G., Kamath V., Wygonski J., and Ekelid M. Speech recognition with primarily temporal cues. Science 270 (1995) 303-304
53. Shannon R.V., Jensvold A., Padilla M., Robert M.E., and Wang X. Consonant recordings for speech testing. J. Acoust. Soc. Am. 106 (1999) L71-L74
54. Shannon R.V., Fu Q.-J., and Galvin J. The number of spectral channels required for speech recognition depends on the difficulty of the listening situation. Acta Otolaryngol. Suppl. 552 (2004) 50-54
55. Skinner M.W., Arndt P.L., and Staller S.J. Nucleus 24 advanced encoder conversion study: performance vs. preference. Ear Hear. 23 (2002) 2S-25S
56. Smith Z.M., Delgutte B., and Oxenham A.J. Chimaeric sounds reveal dichotomies in auditory perception. Nature 416 (2002) 87-90
57. Stickney G.S., Zeng F.-G., Litovsky R., and Assmann P. Cochlear implant speech recognition with speech maskers. Acoust. Soc. Am. 116 (2004) 1081-1091
58. Sun J.C., Skinner M.W., Liu S.Y., Wang F.N.M., Huang T.S., and Lin T. Optimization of speech processor fitting strategies for Chinese-speaking cochlear implantees. Laryngoscope 108 (1998) 560-568
59. ter Keurs M., Festen J.M., and Plomp R. Effect of spectral envelope smearing on speech reception I. J. Acoust. Soc. Am. 91 (1992) 2872-2880
60. ter Keurs M., Festen J.M., and Plomp R. Effect of spectral envelope smearing on speech reception. II. J. Acoust. Soc. Am. 93 (1993) 1547-1552
61. Turner C.W., Chi S.-L., and Flock S. Limiting spectral resolution in speech for listeners with sensorineural hearing loss. J. Speech Lang. Hear. Res. 42 (1999) 773-784
62. van der Horst R., Leeuw A.R., and Dreschler W.A. Importance of temporal-envelope cues in consonant recognition. J. Acoust. Soc. Am. 105 (1999) 1801-1809
63. Van Tassel D.J., Soli S.D., Kirby V.M., and Widin G.P. Speech waveform envelope cues for consonant recognition. J. Acoust. Soc. Am. 82 (1987) 1152-1161
64. Vandali A.E., Sucher C., Tsang D.J., McKay C.M., Chew J.W., and McDermott H.J. Pitch ranking ability of cochlear implant recipients: a comparison of sound-processing strategies. J. Acoust. Soc. Am. 117 (2005) 3126-3138
65. Vandali, A.E., Ciocca, V., Wong, L.L.N., Luk, B., Ip, V.W.K., Murray, B., Yu, H.C., Chung, I., Ng, E., Yuen, K., 2007. Pitch and tonal language perception in cochlear implant users. In: Conference on Implantable Auditory Prostheses Abst. p. 204.
66. Villchur E. Electronic models to simulate the effect of sensory distortions on speech perception by the deaf. J. Acoust. Soc. Am. 62 (1977) 665-674
67. Wei W.I., Wong R., Hui Y., Au D.K.K., Wong B.Y.K., Ho W.K., Tsang A., Kung P., and Chung E. Chinese tonal language rehabilitation following cochlear implantation in children. Acta Otolaryngol. 120 (2000) 218-221
68. Wei C.-G., Cao K., and Zeng F.-G. Mandarin tone recognition in cochlear-implant subjects. Hear. Res. 197 (2004) 87-95
69. Wei C., Cao K., Jin X., Chen X.X., and Zeng F.-G. Psychophysical performance and Mandarin tone recognition in noise by cochlear implant users. Ear Hear. 28 (2007) 62S-65S
70. Whalen D.H., and Xu Y. Information for Mandarin tones in the amplitude contour and in brief segments. Phonetica 49 (1992) 25-47
71. Wilson B.S., Finley C.C., Lawson D.T., and Zerbi M. Temporal representations with cochlear implants. Am. J. Otol. 18 Suppl. 6 (1997) S30-S34
72. Wong A.O., and Wong L.L. Tone perception of Cantonese-speaking prelingually hearing-impaired children with cochlear implants. Otolaryngol. Head Neck Surg. 130 (2004) 751-758
73. Xu L., and Pfingst B.E. Relative importance of temporal envelope and fine structure in lexical-tone perception. J. Acoust. Soc. Am. 114 (2003) 3024-3027
74. Xu L., and Zheng Y. Spectral and temporal cues for phoneme recognition in noise. J. Acoust. Soc. Am. 122 (2007) 1758-1764
75. Xu L., Tsai Y., and Pfingst B.E. Features of stimulation affecting tonal-speech perception: Implications for cochlear prostheses. J. Acoust. Soc. Am. 112 (2002) 247-258
76. Xu L., Thompson C.S., and Pfingst B.E. Relative contributions of spectral and temporal cues for phoneme recognition. J. Acoust. Soc. Am. 117 (2005) 3255-3267
77. ® 120 speech-processing strategy in Mandarin-speaking children. In: Asia-Pacific Symposium on Cochlear Implants, Sydney, Australia.
78. Zeng F.-G. Cochlear implants in China. Audiology 34 (1995) 61-75
79. Zeng F.-G. Trends in cochlear implants. Trends Amplification 8 (2004) 1-34
80. Zeng F.-G., Oba S., Garde S., Sininger Y., and Starr A. Temporal and speech processing deficits in auditory neuropathy. NeuroReport 10 16 (1999) 3429-3435
81. Zeng F.-G., Nie K., Stickney G.S., Kong Y.-Y., Vongphoe M., Bhargave A., Wei C., and Cao K. Speech recognition with amplitude and frequency modulations. Proc. Natl. Acad. Sci. U.S.A. 102 (2005) 2293-2298