Towards a neurocomputational model of speech production and perception

Speech Communication

Volumn 51, Issue 9, 2009, Pages 793-809

  Kröger, Bernd J ^a   Kannampuzha, Jim ^a   Neuschaefer Rube, Christiane ^a  

^a UNIVERSITY HOSPITAL RWTH AACHEN   (Germany)

Author keywords

Artificial neural networks; Neurocomputational model; Self organizing networks; Speech; Speech perception; Speech production

Indexed keywords

ARTIFICIAL NEURAL NETWORKS; NEUROCOMPUTATIONAL MODEL; SELF-ORGANIZING NETWORKS; SPEECH PERCEPTION; SPEECH PRODUCTION;

BACKPROPAGATION; CONFORMAL MAPPING; SELF ORGANIZING MAPS; SENSORY PERCEPTION; SPEECH ANALYSIS; SPEECH SYNTHESIS;

SPEECH RECOGNITION;

EID: 67649083911     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2008.08.002     Document Type: Article

References (114)

1. Ackermann H., and Riecker A. The contribution of the insula to motor aspects of speech production: a review and a hypothesis. Brain Lang. 89 (2003) 320-328
2. Arbib M.A. From monkey-like action recognition to human language: an evolutionary framework for neurolinguists. Behav. Brain Sci. 28 (2005) 105-167
3. Bailly G. Learning to speak: sensory-motor control of speech movements. Speech Comm. 22 (1997) 251-267
4. Batchelder E.O. Bootstrapping the lexicon: a computational model of infant speech segmentation. Cognition 83 (2002) 167-206
5. Benson R.R., Whalen D.H., Richardson M., Swainson B., Clark V.P., Lai S., and Liberman A.M. Parametrically dissociating speech and nonspeech perception in the brain using fMRI. Brain Lang. 78 (2001) 364-396
6. Benzeghiba M., De Mori R., Deroo O., Dupont S., Erbes T., Jouvet D., Fissore L., Laface P., Mertins A., Ris C., Rose R., Tyagi V., and Wellekens C. Automatic speech recognition and speech variability: a review. Speech Comm. 49 (2007) 763-786
7. Binder J.R., Frost J.A., Hammeke T.A., Bellgowan P.S.F., Springer J.A., Kaufman J.N., and Possing E.T. Human temporal lobe activation by speech and nonspeech sounds. Cereb. Cortex 10 (2000) 512-528
8. Birkholz, P., Jackèl, D., 2004. Influence of temporal discretization schemes on formant frequencies and bandwidths in time domain simulations of the vocal tract system. In: Proc. Internat. Conf. on Speech and Language Processing (Interspeech 2004, Jeju, Korea), pp. 1125-1128.
9. Birkholz, P., Kröger, B.J., 2006. Vocal tract model adaptation using magnetic resonance imaging. In: Proc. 7th Internat. Seminar on Speech Production (Belo Horizonte, Brazil), pp. 493-500.
10. Birkholz, P., Kröger, B.J., 2007. Simulation of vocal tract growth for articulatory speech synthesis. In: Proc. 16th Internat. Congress of Phonetic Sciences (Saarbrücken, Germany), pp. 377-380.
11. Birkholz, P., Jackèl, D., Kröger, B.J., 2006. Construction and control of a three-dimensional vocal tract model. In: Proc. Internat. Conf. on Acoustics, Speech, and Signal Processing (ICASSP 2006, Toulouse, France), pp. 873-876.
12. Birkholz P., Jackèl D., and Kröger B.J. Simulation of losses due to turbulence in the time-varying vocal system. IEEE Trans. Audio Speech Lang. Process. 15 (2007) 1218-1225
13. Blank S.C., Scott S.K., Murphy K., Warburton E., and Wise R.J.S. Speech production: Wernike, Broca and beyond. Brain 125 (2002) 1829-1838
14. Boatman D. Cortical bases of speech perception: evidence from functional lesion studies. Cognition 92 (2004) 47-65
15. Bookheimer S.Y., Zeffiro T.A., Blaxton T.A., Gaillard W., and Theodore W.H. Activation of langauge cortex with automatic speech tasks. Neurology 55 (2000) 1151-1157
16. Bradlow A.R. A comparative acoustic study of English and Spanish vowels. J. Acoust. Soc. Amer. 97 (1995) 1916-1924
17. Brent M.B. Speech segmentation and word discovery: a computational perspective. Trends Cognit. Sci. 3 (1999) 294-301
18. Browman C., and Goldstein L. Articulatory gestures as phonological units. Phonology 6 (1989) 201-251
19. Browman C., and Goldstein L. Articulatory phonology: an overview. Phonetica 49 (1992) 155-180
20. Bullock D., Grossberg S., and Guenther F. A self-organizing neural model of motor equivalent reaching and tool use by a multijoint arm. J. Cognit. Neurosci. 5 (1993) 408-435
21. Callan D.E., Tsytsarev V., Hanakawa T., Callan A.K., Katsuhara M., Fukuyama H., and Turner R. Song and speech: brain regions involved with perception and covert production. Neuroimage 31 (2006) 1327-1342
22. Cervera T., Miralles J.L., and Gonzales-Alvarez J. Acoustical analysis of Spanish vowels produced by laryngectomized subjects. J. Speech Lang. Hear. Res. 44 (2001) 988-996
23. Clark R.A.J., Richmond K., and King S. Multisyn: open-domain unit selection for the Festival speech synthesis system. Speech Comm. 49 (2007) 317-330
24. Damper R.I., and Harnad S.R. Neural network models of categorical perception. Percept. Psychophys. 62 (2000) 843-867
25. Dell G.S., Chang F., and Griffin Z.M. Connectionist models of language production: lexical access and grammatical encoding. Cognit. Sci. 23 (1999) 517-541
26. Eimas P.D. The relation between identification and discrimination along speech and non-speech continua. Lang. Speech 6 (1963) 206-217
27. Fadiga L., and Craighero L. Electrophysiology of action representation. J. Clin. Neurophysiol. 21 (2004) 157-168
28. Fadiga L., Craighero L., Buccino G., and Rizzolatti G. Speech listening specifically modulates the excitability of tongue muscles: a TMS study. Eur. J. Neurosci. 15 (2002) 399-402
29. Fowler C.A. An event approach to the study of speech perception from a direct-realist perspective. J. Phonetics 14 (1986) 3-28
30. Fry D.B., Abramson A.S., Eimas P.D., and Liberman A.M. The identification and discrimination of synthetic vowels. Lang. Speech 5 (1962) 171-189
31. Gaskell M.G., and Marslen-Wilson W.D. Integrating form and meaning: a distributed model of speech perception. Lang. Cognit. Process. 12 (1997) 613-656
32. Goldstein L., Byrd D., and Saltzman E. The role of vocal tract action units in understanding the evolution of phonology. In: Arbib M.A. (Ed). Action to Language via the Mirror Neuron System (2006), Cambridge University Press, Cambridge 215-249
33. Goldstein L., Marianne P., Larissa C., Elliot S., and Dani B. Dynamic action units slip in speech production errors. Cognition 103 (2007) 386-412
34. Grossberg S. Resonant neural dynamics of speech perception. J. Phonetics 31 (2003) 423-445
35. Guenther F.H. A neural network model of speech acquisition and motor equivalent speech production. Biological Cybernet. 72 (1994) 43-53
36. Guenther F.H. Speech sound acquisition, coarticulation, and rate effects in a neural model of speech production. Psychol. Rev. 102 (1995) 594-621
37. Guenther F.H. Cortical interaction underlying the production of speech sounds. J. Comm. Disorders 39 (2006) 350-365
38. Guenther F.H., Ghosh S.S., and Tourville J.A. Neural modeling and imaging of the cortical interactions underlying syllable production. Brain Lang. 96 (2006) 280-301
39. Hartsuiker R.J., and Kolk H.H.J. Error monitoring in speech production: a computational test of the perceptual loop theory. Cognit. Psychol. 42 (2001) 113-157
40. Heim S., Opitz B., Müller K., and Friederici A.D. Phonological processing during language production: fMRI evidence for a shared production-comprehension network. Cognit. Brain Res. 16 (2003) 285-296
41. Hickok G., and Poeppel D. Towards a functional neuroanatomy of speech perception. Trends Cognit. Sci. 4 (2000) 131-138
42. Hickok G., and Poeppel D. Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language. Cognition 92 (2004) 67-99
43. Hickok G., and Poeppel D. Towards a functional neuroanatomy of speech perception. Trends Cognit. Sci. 4 (2007) 131-138
44. Hillis A.E., Work M., Barker P.B., Jacobs M.A., Breese E.L., and Maurer K. Re-examining the brain regions crucial for orchestrating speech articulation. Brain 127 (2004) 1479-1487
45. Huang J., Carr T.H., and Cao Y. Comparing cortical activations for silent and overt speech using event-related fMRI. Hum. Brain Mapp. 15 (2001) 39-53
46. Iacoboni M. Neural mechanisms of imitation. Curr. Opin. Neurobiol. 15 (2005) 632-637
47. Indefrey P., and Levelt W.J.M. The spatial and temporal signatures of word production components. Cognition 92 (2004) 101-144
48. Ito T., Gomi H., and Honda M. Dynamical simulation of speech cooperative articulation by muscle linkages. Biological Cybernet. 91 (2004) 275-282
49. Jardri R., Pins D., Bubrovszky M., Despretz P., Pruvo J.P., Steinling M., and Thomas P. Self awareness and speech processing: an fMRI study. Neuroimage 35 (2007) 1645-1653
50. Jusczyk P.W. How infants begin to extract words from speech. Trends Cognit. Sci. 3 (1999) 323-328
51. Kandel E.R., Schwartz J.H., and Jessell T.M. Principles of Neural Science. fourth ed. (2000), McGraw-Hill, New York
52. Kemeny S., Ye F.Q., Birn R., and Braun A.R. Comparison of continuous overt speech fMRI using BOLD and arerial spin labeling. Hum. Brain Mapp. 24 (2005) 173-183
53. Kohler E., Keysers C., Umilta M.A., Fogassi L., Gallese V., and Rizzolatti G. Hearing sounds, understanding actions: action representation in mirror neurons. Science 297 (2002) 846-848
54. Kohonen T. Self-Organizing Maps (2001), Springer, Berlin NewYork
55. Kröger B.J. A gestural production model and its application to reduction in German. Phonetica 50 (1993) 213-233
56. Kröger B.J., and Birkholz P. A gesture-based concept for speech movement control in articulatory speech synthesis. In: Esposito A., Faundez-Zanuy M., Keller E., and Marinaro M. (Eds). Verbal and Nonverbal Communication Behaviours, LNAI 4775 (2007), Springer-Verlag, Berlin, Heidelberg 174-189
57. Kröger, B.J., Birkholz, P., Kannampuzha, J., Neuschaefer-Rube, C., 2006a. Modeling sensory-to-motor mappings using neural nets and a 3D articulatory speech synthesizer. In: Proc. 9th Internat. Conf. on Spoken Language Processing (Interspeech 2006 - ICSLP), pp. 565-568.
58. Kröger, B.J., Birkholz, P., Kannampuzha, J., Neuschaefer-Rube, C., 2006b. Learning to associate speech-like sensory and motor states during babbling. In: Proc. 7th Internat. Seminar on Speech Production (Belo Horizonte, Brazil), pp. 67-74.
59. Kröger, B.J., Birkholz, P., Kannampuzha, J., Neuschaefer-Rube, C., 2006c. Spatial-to-joint mapping in a neural model of speech production. In: DAGA-Proc. 32th Annu. Meet. German Acoustical Society (Braunschweig, Germany), pp. 561-562. .
60. Kröger B.J., Lowit A., and Schnitker R. The organization of a neurocomputational control model for articulatory speech synthesis. In: Esposito A., Bourbakis N., Avouris N., and Hatzilygeroudis I. (Eds). Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction. Selected papers from COST Action 2102 International Workshop (2008), Springer-Verlag 121-135
61. Kuriki S., Mori T., and Hirata Y. Motor planning center for speech articulation in the normal human brain. Neuroreport 10 (1999) 765-769
62. Latorre J., Iwano K., and Furui S. New approach to the polyglot speech generation by means of an HMM-based speaker adaptable synthesizer. Speech Comm. 48 (2006) 1227-1242
63. Levelt W.J.M., and Wheeldon L. Do speakers have access to a mental syllabary?. Cognition 50 (1994) 239-269
64. Levelt W.J.M., Roelofs A., and Meyer A. A theory of lexical access in speech production. Behav. Brain Sci. 22 (1999) 1-75
65. Liberman A.M., and Mattingly I.G. The motor theory of speech perception revised. Cognition 21 (1985) 1-36
66. Liberman A.M., Harris K.S., Hoffman H.S., and Griffith B.C. The discrimination of speech sounds within and across phoneme boundaries. J. Exp. Psychol. 54 (1957) 358-368
67. Liberman A.M., Cooper F.S., Shankweiler D.P., and Studdert-Kennedy M. Perception of the speech code. Psychol. Rev. 74 (1967) 431-461
68. Liebenthal E., Binder J.R., Spitzer S.M., Possing E.T., and Medler D.A. Neural substrates of phonemic perception. Cereb. Cortex 15 (2005) 1621-1631
69. Li P., Farkas I., and MacWhinney B. Early lexical development in a self-organizing neural network. Neural Networks 17 (2004) 1345-1362
70. Luce P.A., Goldinger S.D., Auer E.T., and Vitevitch M.S. Phonetic priming, neighborhood activation, and PARSYN. Percept. Psychophys. 62 (2000) 615-625
71. Maass W., and Schmitt M. On the complexity of learning for spiking neurons with temporal coding. Inform. Comput. 153 (1999) 26-46
72. McClelland J.L., and Elman J.L. The TRACE model of speech perception. Cognit. Psychol. 18 (1986) 1-86
73. Murphy K., Corfield D.R., Guz A., Fink G.R., Wise R.J.S., Harrison J., and Adams L. Cerebral areas associated with motor control of speech in humans. J. Appl. Physiol. 83 (1997) 1438-1447
74. Nasir S.M., and Ostry D.J. Somatosensory precision in speech production. Curr. Biology 16 (2006) 1918-1923
75. Norris D., Cutler A., McQueen J.M., and Butterfield S. Phonological and conceptual activation in speech comprehension. Cognit. Psychol. 53 (2006) 146-193
76. Obleser J., Boecker H., Drzezga A., Haslinger B., Hennenlotter A., Roettinger M., Eulitz C., and Rauschecker J.P. Vowel sound extraction in anterior superior temporal cortex. Hum. Brain Mapp. 27 (2006) 562-571
77. Obleser J., Wise R.J.S., Dresner M.A., and Scott S.K. Functional integration across brain regions improves speech perception under adverse listening conditions. J. Neurosci. 27 (2007) 2283-2289
78. Okada K., and Hickok G. Left posterior auditory-related cortices participate both in speech perception and speech production: neural overlap revealed by fMRI. Brain Lang. 98 (2006) 112-117
79. Oller D.K., Eilers R.E., Neal A.R., and Schwartz H.K. Precursors to speech in infancy: the prediction of speech and language disorders. J. Comm. Disorders 32 (1999) 223-245
80. Perkell J.S., Matthies M.L., Svirsky M.A., and Jordan M.I. Trading relations between tongue-body raising and lip rounding in production of the vowel /u/: a pilot "motor equivalence" study. J. Acoust. Soc. Amer. 93 (1993) 2948-2961
81. Poeppel D., Guillemin A., Thompson J., Fritz J., Bavelier D., and Braun A.R. Auditory lexical decision, categorical perception, and FM direction discrimination differentially engage left and right auditory cortex. Neurophysiologia 42 (2004) 183-200
82. Postma A. Detection of errors during speech production: a review of speech monitoring models. Cognition 77 (2000) 97-131
83. Raphael L.J., Borden G.J., and Harris K.S. Speech Science Primer: Physiology, Acoustics, and Perception of Speech. fifth ed. (2007), Lippincott Williams & Wilkins
84. Riecker A., Kassubek J., Gröschel K., Grodd W., and Ackermann H. The cerebral control of speech tempo: opposite relationship between speaking rate and BOLD signal change at stratal and cerebellar structures. Neuroimage 29 (2006) 46-53
85. Rimol L.M., Specht K., Weis S., Savoy R., and Hugdahl K. Processing of sub-syllabic speech units in the posterior temporal lobe: an fMRI study. Neuroimage 26 (2005) 1059-1067
86. Ritter H., and Kohonen T. Self-organizing semantic maps. Biological Cybernet. 61 (1989) 241-254
87. Rizzolatti G., and Arbib M.A. Language within our grasp. Trends Neurosci. 21 (1998) 188-194
88. Rizzolatti G., and Craighero L. The mirror neuron system. Annu. Rev. Neurosci. 27 (2004) 169-192
89. Rosen H.J., Ojemann J.G., Ollinger J.M., and Petersen S.E. Comparison of brain activation during word retrieval done silently and aloud using fMRI. Brain Cognit. 42 (2000) 201-217
90. Saltzman E. Levels of sensorimotor representation. J. Math. Psychol. 20 (1979) 91-163
91. Saltzman E., and Byrd D. Task-dynamics of gestural timing: phase windows and multifrequency rhythms. Hum. Movement Sci. 19 (2000) 499-526
92. Saltzman E., and Munhall K.G. A dynamic approach to gestural patterning in speech production. Ecol. Psychol. 1 (1989) 333-382
93. Sanguineti V., Laboissiere R., and Payan Y. A control model of human tongue movements in speech. Biological Cybernet. 77 (1997) 11-22
94. Scharenborg O. Reaching over the gap: a review of efforts to link human and automatic speech recognition research. Speech Comm. 49 (2007) 336-347
95. Scott S.K., Blank C.C., Rosen S., and Wise R.J.S. Identification of a pathway for intelligible speech in the left temporal lobe. Brain 123 (2000) 2400-2406
96. Shadmehr R., and Mussa-Ivaldi A. Adaptive representation of dynamics during learning of a motor task. J. Neurosci. 14 (1994) 3208-3224
97. Shuster L.I., and Lemieux S.K. An fMRI investigation of covertly and overtly produced mono- and multisyllabic words. Brain Lang. 93 (2005) 20-31
98. Sober S.J., and Sabes P.N. Multisensory integration during motor planning. J. Neurosci. 23 (2003) 6982-6992
99. Sörös R., Guttman Sakoloff L., Bose A., McIntosh A.R., Graham S.J., and Stuss D.T. Clustered functional MRI of overt speech production. Neuroimage 32 (2006) 376-387
100. Studdert-Kennedy M. Mirror neurons, vocal imitation, and the evolution of particulate speech. In: Stamenov M.I., and Gallese V. (Eds). Mirror Neurons and the Evolution of Brain and Language (2002), Benjamin, Philadelphia 207-227
101. Tani J., Masato I., and Sugita Y. Self-organization of distributed represented multiple behavior schemata in a mirror system: reviews of robot experiments using RNNPB. Neural Networks 17 (2004) 1273-1289
102. Todorov E. Optimality principles in sensorimotor control. Nature Neurosci. 7 (2004) 907-915
103. Tremblay S., Shiller D.M., and Ostry D.J. Somatosensory basis of speech production. Nature 423 (2003) 866-869
104. Ullman M.T. A neurocognitive perspective on language: the declarative/procedural model. Nature Rev. Neurosci. 2 (2001) 717-726
105. Uppenkamp S., Johnsrude I.S., Norris D., Marslen-Wilson W., and Patterson R.D. Locating the initial stages of speech-sound processing in human temporal cortex. Neuroimage 31 (2006) 1284-1296
106. Vanlancker-Sidtis D., McIntosh A.R., and Grafton S. PET activation studies comparing two speech tasks widely used in surgical mapping. Brain Lang. 85 (2003) 245-261
107. Varley R., and Whiteside S. What is the underlying impairment in acquired apraxia of speech. Aphasiology 15 (2001) 39-49
108. Werker J.F., and Yeung H.H. Infant speech perception bootstraps word learning. Trends Cognit. Sci. 9 (2005) 519-527
109. Westerman G., and Miranda E.R. A new model of sensorimotor coupling in the development of speech. Brain Lang. 89 (2004) 393-400
110. Wilson M., and Knoblich G. The case for motor involvement in perceiving conspecifics. Psychol. Bull. 131 (2005) 460-473
111. Wilson S.M., Saygin A.P., Sereno M.I., and Iacoboni M. Listening to speech activates motor areas involved in speech production. Nature Neurosci. 7 (2004) 701-702
112. Wise R.J.S., Greene J., Büchel C., and Scott S.K. Brain regions involved in articulation. The Lancet 353 (1999) 1057-1061
113. Zekveld A., Heslenfeld D.J., Festen J.M., and Schoonhoven R. Top-down and bottom-up processes in speech comprehension. Neuroimage 32 (2006) 1826-1836
114. Zell A. Simulation neuronaler Netze (2003), Oldenbourg Verlag, München, Wien