Improved perception of music with a harmonic based algorithm for cochlear implants

IEEE Transactions on Neural Systems and Rehabilitation Engineering

Volumn 21, Issue 4, 2013, Pages 684-694

  Li, Xing ^a   Nie, Kaibao ^b   Imennov, Nikita S ^b   Rubinstein, Jay T ^b   Atlas, Les E ^a  

^a University of Washington   (United States)

^b University of Washington   (United States)

Author keywords

Cochlear implant; fine structure; harmonic coding; pitch perception; timbre perception

Indexed keywords

ENCODING STRATEGY; FINE STRUCTURES; HARMONIC CODING; MELODY RECOGNITION; MUSIC PERCEPTION; PITCH PERCEPTION; RECOGNITION PERFORMANCE; TEMPORAL FINE STRUCTURE;

ATOMIC PHYSICS; AUDITION; COCHLEAR IMPLANTS; COMPUTER SIMULATION; NEURAL NETWORKS; VOCODERS;

HARMONIC ANALYSIS;

ALGORITHM; ARTICLE; AUDITORY STIMULATION; BIOLOGICAL MODEL; BRAIN MAPPING; COCHLEA PROSTHESIS; COCHLEAR NERVE; COMPUTER SIMULATION; ELECTROSTIMULATION; EQUIPMENT DESIGN; FOURIER ANALYSIS; HEARING; HUMAN; MUSIC; NEUROPROSTHESIS; PHYSIOLOGY; PSYCHOLOGICAL ASPECT; RECOGNITION; PSYCHOLOGY;

ACOUSTIC STIMULATION; ALGORITHMS; AUDITORY PERCEPTION; BRAIN MAPPING; COCHLEAR IMPLANTS; COCHLEAR NERVE; COMPUTER SIMULATION; ELECTRIC STIMULATION; EQUIPMENT DESIGN; FOURIER ANALYSIS; HUMANS; MODELS, NEUROLOGICAL; MUSIC; NEURAL PROSTHESES; PITCH PERCEPTION; RECOGNITION (PSYCHOLOGY);

EID: 84880397498     PISSN: 15344320     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNSRE.2013.2257853     Document Type: Article

References (43)

1. K. Nie, L. Atlas, and J. Rubinstein, "Single sideband encoder for music coding in cochlear implants," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., 2008, pp. 4209-4212.
2. X. Li, K. Nie, L.Atlas, and J. Rubinstein, "Harmonic coherent demodulation for improving sound coding in cochlear implants," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., 2010, pp. 5462-5465.
3. X. Li, K. B. Nie, N. S. Imennov, J. H. Won, W. R. Drennan, J. T. Rubenstien, and L. E. Atals, "Improved perception of speech in noise and mandarin tones with acoustic simulations of harmonic coding for cochlear implants," J. Acoust. Soc. Am., vol. 132, no. 5, pp. 3387-3398, 2012.
4. F.-G. Zeng,K.Nie,G. S. Stickney,Y.-Y.Kong, M.Vongphoe, A. Bhargave, C.Wei, and K. Cao, "Speech recognition with amplitude and frequency modulations," Proc. Nat. Acad. Sci. USA, vol. 102, no. 7, pp. 2293-2298, 2005. (Pubitemid 40279369)
5. L. Xu, X. Chen, H. Lu, N. Zhou, S. Wang, Q. Liu, Y. Li, X. Zhao, and D. Han, "Tone perception and production in pediatric cochlear implants users," Acta Otolaryngol., vol. 131, pp. 395-398, 2011.
6. K. Gfeller,C.Turner,M.Mehr,G.Woodworth, R. Fearn, J. F. Knutson, S. Witt, and J. Stordahl, "Recognition of familiar melodies by adult cochlear implant recipients and normal-hearing adults," Cochlear Implants Int., vol. 3, no. 1, pp. 31-55, 2002.
7. Y.-Y. Kong, R. Cruz, J. A. Jones, and F.-G. Zeng, "Music perception with temporal cues in acoustic and electric hearing," Ear Hear., vol. 25, pp. 173-185, 2004. (Pubitemid 38451766)
8. J. J. Galvin, Q.-J. Fu, and G. Nogaki, "Melodic contour identification by cochlear implant listeners," Ear Hear., vol. 28, no. 3, pp. 302-319, 2007. (Pubitemid 46718011)
9. R. Kang, G. L. Nimmons, W. R. Drennan, J. Longnion, C. Ruffin, K. Nie, J. H. Won, T. Worman, B. Yueh, and J. T. Rubinstein, "Development and validation of the University ofWashington clinical assessment of music perception test," Ear. Hear., vol. 30, no. 4, pp. 411-418, 2009.
10. Y. Y. Kong, A. Mullangi, J. Marozeau, and M. Epstein, "Temporal and spectral cues for musical timbre perception in electric hearing," J. Speech Lang. Hear. Res., vol. 54, no. 3, pp. 981-994, 2011.
11. H. J. McDermott, "Music perception with cochlear implants: A review," Trends Amplif., vol. 8, no. 2, pp. 49-82, 2004. (Pubitemid 39435144)
12. W. R. Drennan and J. T. Rubinstein, "Music perception in cochlear implant users and its relationship with psychophysical capabilities," J. Rehabil. Res. Develop., vol. 45, no. 5, pp. 779-789, 2008.
13. K.H. Jung, J.H. Won, W. R.Drennan, E. Jameyson,G.Miyasaki, S. J. Norton, and J. T. Rubinstein, "Psychoacoustic performance and music and speech perception in prelingually deafened children with cochlear implants," Audiol. Neurootol., vol. 17, no. 3, pp. 189-197, 2012.
14. F. G. Zeng, S. Rebscher,W. Harrison, X. Sun, and H. Feng, "Cochlear implants: System design, integration, and evaluation," IEEE Rev. Biomed. Eng., vol. 1, pp. 115-142, Nov. 2008.
15. B. S.Wilson andM. F.Dorman, "Cochlear implants:A remarkable past and a brilliant future," Hear. Res., vol. 242, no. 1-2, pp. 3-21, 2008.
16. H. Dudley, "Remaking speech," J. Acoust. Soc. Am., vol. 11, pp. 169-177, 1939.
17. R. V. Shannon, F.-G. Zeng, V. Kamath, J. Wygonski, and M. Ekelid, "Speech recognition with primarily temporal cues," Science, vol. 270, pp. 303-304, 1995.
18. B. C. J. Moore, "Coding of sounds in the auditory system and its relevance to signal processing and coding in cochlear implants," Otol. Neurotol., vol. 24, no. 2, pp. 243-254, 2003. (Pubitemid 36337136)
19. R. Plomp, "Auditory psychophysics," Annu. Rev. Psychol., vol. 26, pp. 207-232, 1975.
20. P. A. Cariani and B. Delgutte, "Neural correlates of the pitch of complex tones.I. Pitch and pitch salience," J. Neurophysiol, vol. 76, no. 3, pp. 1698-716, 1996. (Pubitemid 26322613)
21. T. M. Shackleton and R. P. Carlyon, "The role of resolved and unresolved harmonics in pitch perception and frequency modulation discrimination," J. Acoust. Soc. Am., vol. 95, no. 6, pp. 3529-3540, 1994. (Pubitemid 24189504)
22. ANSI, "American national standard acoustical terminology," 1994.
23. J. F. Schouten, "The perception of timbre," in Proc. 6th Int. Conf. Acoust., 1968, vol. 1, p. GP-6-2.
24. J. M. Grey and J. W. Gordon, "Perceptual effects of spectral modifications on musical timbres," J. Acoust. Soc. Am., vol. 63, no. 5, pp. 1493-1500, 1978. (Pubitemid 8346203)
25. A. Caclin, S. McAdams, B. K. Smith, and S.Winsberg, "Acoustic correlates of timbre space dimensions: A confirmatory study using synthetic tones," J. Acoust. Soc. Am., vol. 118, no. 1, pp. 471-482, 2005. (Pubitemid 40981501)
26. R. Shannon, "Temporal modulation transfer functions in patients with cochlear implants," J. Acoust. Soc. Am., vol. 91, pp. 2156-2164, Apr. 1992.
27. F.-G. Zeng, "Temporal pitch in electric hearing," Hear. Res., vol. 174, pp. 101-106, 2002. (Pubitemid 35305037)
28. K. Nie, G. Stickney, and F.-G. Zeng, "Encoding frequency modulation to improve cochlear implant performance in noise," IEEE Trans. Biomed. Eng., vol. 52, no. 1, pp. 64-73, Jan. 2005. (Pubitemid 40065610)
29. R. Schatzer, A. Krenmayr, D. K. Au,M. Kals, and C. Zierhofer, "Temporal fine structure in cochlear implants: Preliminary speech perception results in Cantonese-speaking implant users," Acta. Otolaryngol., vol. 130, pp. 1031-1039, 2010.
30. J. Laneau, J. Wouters, and M. Moonen, "Improved music perception with explicitly pitch coding in cochlear implants," Audiol. Neurootol., vol. 11, pp. 38-52, 2006. (Pubitemid 43087558)
31. M. Milczynski, J. Wouters, and A. van Wieringen, "Improved fundamental frequency coding in cochlear implant signal processing," J. Acoust. Soc. Am., vol. 125, pp. 2260-2271, 2009.
32. A. E. Vandali and R. J. van Hoesel, "Development of a temporal fundamental frequency coding strategy for cochlear implants," J. Acoust. Soc. Am., vol. 129, pp. 4023-4036, 2011.
33. A. E. Vandali, L. A. Whitford, K. L. Plant, and G. M. Clark, "Speech perception as a fucntion of electrical stimulation rate:Using the nucleus 24 cochlear implant system," Ear Hear., vol. 21, no. 6, pp. 608-624, 2000.
34. B. H. Bonham and L. M. Litvak, "Current focusing and steering: Modeling, physiology, and psychophysics," Hear. Res., vol. 242, pp. 141-153, 2008.
35. H. Mino, J. T. Rubinstein, C. A. Miller, and P. J. Abbas, "Effects of electrode-to-fiber distance on temporal neural response with electrical stimulation," IEEE Trans. Biomed. Eng., vol. 51, no. 1, pp. 13-20, Jan. 2004. (Pubitemid 38009958)
36. N. S. Imennov and J. T. Rubinstein, "Stochastic population model for electrical stimulation of the auditory nerve," IEEE Trans. Biomed. Eng., vol. 56, no. 10, pp. 2493-2501, Oct. 2009.
37. Q. Li and L. E. Atlas, "Time-variant least squares harmonic modeling," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., 2003, p. II-41-4.
38. S. Schimmel and L. Atlas, "Coherent envelope detection for modulation filtering of speech," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., 2005, pp. 221-224.
39. P. Clark and L. E. Atlas, "Time-frequency coherent modulation filtering of nonstationary signals," IEEE Trans. Signal Process., vol. 57, no. 11, pp. 4323-4332, Nov. 2009.
40. B. Swanson and H. Mauch, NucleusMATLAB Toolbox 4.20 Software User Manual Cochlear Ltd. Lane Cove, Australia, 2006.
41. A. J. Oxenham, J. G. Bernstein, and H. Penagos, "Correct tonotopic representation is necessary for complex pitch perception," Proc. Nat. Acad. Sci. USA, vol. 101, no. 5, pp. 1421-1425, 2004. (Pubitemid 38182704)
42. P. de la Cuadra, A.Master, and C. Sapp, "Efficient pitch detection techniques for interactive music," in Proc. Int. Comput. Music Conf., 2001.
43. M. Goto, "A real-timemusic-scene-description system: Predominantf0 estimation for detecting melody and bass lines in real-world audio signals," Speech Commun., vol. 43, pp. 311-329, 2004.