The multisensory roles for auditory cortex in primate vocal communication

Hearing Research

Volumn 258, Issue 1-2, 2009, Pages 113-120

  Ghazanfar, Asif A ^a  

^a PRINCETON UNIVERSITY   (United States)

Author keywords

Action specific representations; Audiovisual speech; Crossmodal; Embodied cognition; Multimodal; Multisensory

Indexed keywords

ANIMAL BEHAVIOR; ANIMAL COMMUNICATION; ARTICLE; AUDITORY CORTEX; BRAIN FUNCTION; BRAIN REGION; CONTROLLED STUDY; ETHOLOGY; FACIAL EXPRESSION; FEEDBACK SYSTEM; MONKEY; MOTOR PERFORMANCE; NONHUMAN; PRIMATE; PRIORITY JOURNAL; PROPRIOCEPTION; SENSORIMOTOR FUNCTION; SENSORIMOTOR INTEGRATION; SOMATOSENSORY STIMULATION; VERBAL BEHAVIOR; VISUAL DISCRIMINATION;

ACOUSTIC STIMULATION; ANIMAL COMMUNICATION; ANIMALS; AUDITORY CORTEX; AUDITORY PERCEPTION; BEHAVIOR, ANIMAL; BRAIN MAPPING; COGNITION; EVOKED POTENTIALS, AUDITORY; HAPLORHINI; HUMANS; PRIMATES; VISION, OCULAR; VISUAL PERCEPTION; VOICE;

EID: 68249120332     PISSN: 03785955     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.heares.2009.04.003     Document Type: Article

References (106)

1. Abry C., Lallouache M.-T., and Cathiard M.-A. How can coarticulation models account for speech sensitivity in audio-visual desynchronization?. In: Stork D., and Henneke M. (Eds). Speechreading by Humans and Machines: Models, Systems and Applications (1996), Springer-Verlag, Berlin 247-255
2. Barnes C.L., and Pandya D.N. Efferent cortical connections of multimodal cortex of the superior temporal sulcus in the rhesus-monkey. Journal of Comparative Neurology 318 (1992) 222-244
3. Barraclough N.E., Xiao D., Baker C.I., Oram M.W., and Perrett D.I. Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. Journal of Cognitive Neuroscience 17 (2005) 377-391
4. Beiser A. Processing of twitter-call fundamental frequencies in insula and auditory cortex of squirrel monkeys. Experimental Brain Research 122 (1998) 139-148
5. Bendor D., and Wang X.Q. The neuronal representation of pitch in primate auditory cortex. Nature 436 (2005) 1161-1165
6. Benevento L.A., Fallon J., Davis B.J., and Rezak M. Auditory-visual interactions in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey. Experimental Neurology 57 (1977) 849-872
7. Bernstein L.E., Auer E.T., and Takayanagi S. Auditory speech detection in noise enhanced by lipreading. Speech Communication 44 (2004) 5-18
8. Besle J., Fort A., Delpuech C., and Giard M.H. Bimodal speech: early suppressive visual effects in human auditory cortex. European Journal of Neuroscience 20 (2004) 2225-2234
9. Bizley J.K., Nodal F.R., Bajo V.M., Nelken I., and King A.J. Physiological and anatomical evidence for multisensory interactions in auditory cortex. Cerebral Cortex 17 (2007) 2172-2189
10. Bruce C., Desimone R., and Gross C.G. Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. Journal of Neurophysiology 46 (1981) 369-384
11. Brumm H., Voss K., Kollmer I., and Todt D. Acoustic communication in noise: regulation of call characteristics in a New World monkey. Journal of Experimental Biology 207 (2004) 443-448
12. Cappe C., and Barone P. Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. European Journal of Neuroscience 22 (2005) 2886-2902
13. Cappe C., Thut G., Romei V., and Murray M.M. Selective integration of auditory-visual looming cues by humans. Neuropsycholgia 47 (2009) 1045-1052
14. Chandrasekaran C., and Ghazanfar A.A. Different neural frequency bands integrate faces and voices differently in the superior temporal sulcus. Journal of Neurophysiology 101 (2009) 773-788
15. Cheney D.L., and Seyfarth R.M. How Vervet monkeys perceive their grunts - field playback experiments. Animal Behaviour 30 (1982) 739-751
16. Clark A. Being There: Putting Brain, Body, and World Together Again (1997), MIT Press, Cambridge, MA
17. de la Mothe L.A., Blumell S., Kajikawa Y., and Hackett T.A. Cortical connections of the auditory cortex in marmoset monkeys: Core and medial belt regions. Journal of Comparative Neurology 496 (2006) 27-71
18. Driver J., and Noesselt T. Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. Neuron 57 (2008) 11-23
19. Egnor S.E.R., and Hauser M.D. Noise-induced vocal modulation in cotton-top tamarins (Saguinus oedipus). American Journal of Primatology 68 (2006) 1183-1190
20. Egnor S.E.R., Iguina C.G., and Hauser M.D. Perturbation of auditory feedback causes systematic perturbation in vocal structure in adult cotton-top tamarins. Journal of Experimental Biology 209 (2006) 3652-3663
21. Egnor S.E.R., Wickelgren J.G., and Hauser M.D. Tracking silence: adjusting vocal production to avoid acoustic interference. Journal of Comparative Physiology A - Neuroethology Sensory Neural and Behavioral Physiology 193 (2007) 477-483
22. Eliades S.J., and Wang X.Q. Neural substrates of vocalization feedback monitoring in primate auditory cortex. Nature 453 (2008) 1102-1107
23. Ettlinger G., and Wilson W.A. Cross-modal performance: behavioural processes, phylogenetic considerations and neural mechanisms. Behavioural Brain Research 40 (1990) 169-192
24. Evans T.A., Howell S., and Westergaard G.C. Auditory-visual cross-modal perception of communicative stimuli in tufted capuchin monkeys (Cebus apella). Journal of Experimental Psychology - Animal Behavior Processes 31 (2005) 399-406
25. Fitch W.T. Vocal tract length and formant frequency dispersion correlate with body size in rhesus macaques. Journal of the Acoustical Society of America 102 (1997) 1213-1222
26. Fitch W.T., and Hauser M.D. Vocal production in nonhuman-primates - acoustics, physiology, and functional constraints on honest advertisement. American Journal of Primatology 37 (1995) 191-219
27. Fowler C.A. Speech as a supramodal or amodal phenomenon. In: Calvert G.A., Spence C., and Stein B.E. (Eds). The Handbook of Multisensory Processes (2004), MIT Press, Cambridge, MA. 189-201
28. Fu K.M.G., Johnston T.A., Shah A.S., Arnold L., Smiley J., Hackett T.A., Garraghty P.E., and Schroeder C.E. Auditory cortical neurons respond to somatosensory stimulation. Journal of Neuroscience 23 (2003) 7510-7515
29. Fu K.M.G., Shah A.S., O'Connell M.N., McGinnis T., Eckholdt H., Lakatos P., Smiley J., and Schroeder C.E. Timing and laminar profile of eye-position effects on auditory responses in primate auditory cortex. Journal of Neurophysiology 92 (2004) 3522-3531
30. Ghazanfar A.A., and Hauser M.D. The neuroethology of primate vocal communication: substrates for the evolution of speech. Trends in Cognitive Sciences 3 (1999) 377-384
31. Ghazanfar A.A., and Hauser M.D. The auditory behaviour of primates: a neuroethological perspective. Current Opinion in Neurobiology 11 (2001) 712-720
32. Ghazanfar A.A., and Logothetis N.K. Facial expressions linked to monkey calls. Nature 423 (2003) 937-938
33. Ghazanfar A.A., and Schroeder C.E. Is neocortex essentially multisensory?. Trends in Cognitive Sciences 10 (2006) 278-285
34. Ghazanfar A.A., and Chandrasekaran C.F. Paving the way forward: integrating the senses through phase-resetting of cortical oscillations. Neuron 53 (2007) 162-164
35. Ghazanfar A.A., and Rendall D. Evolution of human vocal production. Current Biology 18 (2008) R457-R460
36. Ghazanfar A.A., Nielsen K., and Logothetis N.K. Eye movements of monkeys viewing vocalizing conspecifics. Cognition 101 (2006) 515-529
37. Ghazanfar A.A., Chandrasekaran C., and Logothetis N.K. Interactions between the superior temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys. Journal of Neuroscience 28 (2008) 4457-4469
38. Ghazanfar A.A., Flombaum J.I., Miller C.T., and Hauser M.D. The units of perception in the antiphonal calling behavior of cotton-top tamarins (Saguinus oedipus): playback experiments with long calls. Journal of Comparative Physiology A - Neuroethology Sensory Neural and Behavioral Physiology 187 (2001) 27-35
39. Ghazanfar A.A., Smith-Rohrberg D., Pollen A.A., and Hauser M.D. Temporal cues in the antiphonal long-calling behaviour of cottontop tamarins. Animal Behaviour 64 (2002) 427-438
40. Ghazanfar A.A., Maier J.X., Hoffman K.L., and Logothetis N.K. Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. Journal of Neuroscience 25 (2005) 5004-5012
41. Ghazanfar A.A., Turesson H.K., Maier J.X., van Dinther R., Patterson R.D., and Logothetis N.K. Vocal tract resonances as indexical cues in rhesus monkeys. Current Biology 17 (2007) 425-430
42. Gogate L.J., Walker-Andrews A.S., and Bahrick L.E. The intersensory origins of word comprehension: an ecological-dynamic systems view. Developmental Science 4 (2001) 1-18
43. Gordon M.S., and Rosenblum L.D. Effects of intrastimulus modality change on audiovisual time-to-arrival judgments. Perception & Psychophysics 67 (2005) 580-594
44. Gothard K.M., Battaglia F.P., Erickson C.A., Spitler K.M., and Amaral D.G. Neural responses to facial expression and face identity in the monkey amygdala. Journal of Neurophysiology 97 (2007) 1671-1683
45. Graziano M.S., and Aflalo T.N. Rethinking cortical organization: moving away from discrete areas arranged in hierarchies. The Neuroscientist 13 (2007) 138-147
46. Hackett T.A., Stepniewska I., and Kaas J.H. Subdivisions of auditory cortex and ipsilateral cortical connections of the parabelt auditory cortex in macaque monkeys. Journal of Comparative Neurology 394 (1998) 475-495
47. Hackett T.A., Stepniewska I., and Kaas J.H. Prefrontal connections of the parabelt auditory cortex in macaque monkeys. Brain Research 817 (1999) 45-58
48. Hackett T.A., Preuss T.M., and Kaas J.H. Architectonic identification of the core region in auditory cortex of macaques, chimpanzees, and humans. Journal of Comparative Neurology 441 (2001) 197-222
49. Hackett T.A., de la Mothe L.A., Ulbert I., Karmos G., Smiley J., and Schroeder C.E. Multisensory convergence in auditory cortex, II. Thalamocortical connections of the caudal superior temporal plane. Journal of Comparative Neurology 502 (2007) 924-952
50. Hackett T.A., Smiley J.F., Ulbert I., Karmos G., Lakatos P., de la Mothe L.A., and Schroeder C.E. Sources of somatosensory input to the caudal belt areas of auditory cortex. Perception 36 (2007) 1419-1430
51. Harries M.H., and Perrett D.I. Visual processing of faces in temporal cortex - physiological evidence for a modular organization and possible anatomical correlates. Journal of Cognitive Neuroscience 3 (1991) 9-24
52. Hauser M.D., and Ybarra M.S. The role of lip configuration in monkey vocalizations - experiments using xylocaine as a nerve block. Brain and Language 46 (1994) 232-244
53. Hauser M.D., Evans C.S., and Marler P. The role of articulation in the production of rhesus-monkey, Macaca mulatta, vocalizations. Animal Behaviour 45 (1993) 423-433
54. Ito T., Tiede M., and Ostry D.J. Somatosensory function in speech perception. Proceedings of the National Academy of Sciences of the United States of America 106 (2009) 1245-1248
55. Iyengar S., Qi H., Jain N., and Kaas J.H. Cortical and thalamic connections of the representations of the teeth and tongue in somatosensory cortex of new world monkeys. Journal of Comparative Neurology 501 (2007) 95-120
56. Izumi A., and Kojima S. Matching vocalizations to vocalizing faces in a chimpanzee (Pan troglodytes). Animal Cognition 7 (2004) 179-184
57. Jiang J.T., Alwan A., Keating P.A., Auer E.T., and Bernstein L.E. On the relationship between face movements, tongue movements, and speech acoustics. Eurasip Journal on Applied Signal Processing 2002 (2002) 1174-1188
58. Jones J.A., and Munhall K.G. Learning to produce speech with an altered vocal tract: the role of auditory feedback. Journal of the Acoustical Society of America 113 (2003) 532-543
59. Jones J.A., and Munhall K.G. Remapping auditory-motor representations in voice production. Current Biology 15 (2005) 1768-1772
60. Jordan K.E., Brannon E.M., Logothetis N.K., and Ghazanfar A.A. Monkeys match the number of voices they hear with the number of faces they see. Current Biology 15 (2005) 1034-1038
61. Kayser, C., Logothetis, N.K. in press. Directed interactions between auditory and superior temporal cortices and their role in sensory integration. Frontiers in Integrative Neuroscience.
62. Kayser C., Petkov C.I., and Logothetis N.K. Visual modulation of neurons in auditory cortex. Cerebral Cortex 18 (2008) 1560-1574
63. Kayser C., Petkov C.I., Augath M., and Logothetis N.K. Integration of touch and sound in auditory cortex. Neuron 48 (2005) 373-384
64. Kayser C., Petkov C.I., Augath M., and Logothetis N.K. Functional imaging reveals visual modulation of specific fields in auditory cortex. Journal of Neuroscience 27 (2007) 1824-1835
65. Klin A., Jones W., Schultz R., Volkmar F., and Cohen D. Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry 59 (2002) 809-816
66. Kuhl P.K., Williams K.A., and Meltzoff A.N. Cross-modal speech perception in adults and infants using nonspeech auditory stimuli. Journal of Experimental Psychology: Human Perception and Performance. 17 (1991) 829-840
67. Kuraoka K., and Nakamura K. Responses of single neurons in monkey amygdala to facial and vocal emotions. Journal of Neurophysiology 97 (2007) 1379-1387
68. Lakatos P., Chen C.-M., O'Connell M.N., Mills A., and Schroeder C.E. Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53 (2007) 279-292
69. Lakatos P., Shah A.S., Knuth K.H., Ulbert I., Karmos G., and Schroeder C.E. An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. Journal of Neurophysiology 94 (2005) 1904-1911
70. Lansing I.R., and McConkie G.W. Word identification and eye fixation locations in visual and visual-plus-auditory presentations of spoken sentences. Perception & Psychophysics 65 (2003) 536-552
71. Liberman A.M., and Mattingly I. The motor theory revised. Cognition 21 (1985) 1-36
72. Maier J.X., Chandrasekaran C., and Ghazanfar A.A. Integration of bimodal looming signals through neuronal coherence in the temporal lobe. Current Biology 18 (2008) 963-968
73. Maier J.X., Neuhoff J.G., Logothetis N.K., and Ghazanfar A.A. Multisensory integration of looming signals by rhesus monkeys. Neuron 43 (2004) 177-181
74. McGurk H., and MacDonald J. Hearing lips and seeing voices. Nature 264 (1976) 229-239
75. Meltzoff A.N., and Moore M. Explaining facial imitation: a theoretical model. Early Development & Parenting 6 (1997) 179-192
76. Nasir S.M., and Ostry D.J. Speech motor learning in profoundly deaf adults. Nature Neuroscience 11 (2008) 1217-1222
77. Noesselt T., Rieger J.W., Schoenfeld M.A., Kanowski M., Hinrichs H., Heinze H.-J., and Driver J. Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. Journal of Neuroscience 27 (2007) 11431-11441
78. Oram M.W., and Perrett D.I. Responses of anterior superior temporal polysensory (STPa) neurons to biological motion stimuli. Journal of Cognitive Neuroscience 6 (1994) 99-116
79. Palombit R.A., Cheney D.L., and Seyfarth R.M. Male grunts as mediators of social interaction with females in wild chacma baboons (Papio cynocephalus ursinus). Behaviour 136 (1999) 221-242
80. Parr L.A. Perceptual biases for multimodal cues in chimpanzee (Pan troglodytes) affect recognition. Animal Cognition 7 (2004) 171-178
81. Patterson M.L., and Werker J.F. Two-month-old infants match phonetic information in lips and voice. Developmental Science 6 (2003) 191-196
82. Petkov C.I., Kayser C., Augath M., and Logothetis N.K. Functional imaging reveals numerous fields in the monkey auditory cortex. PLoS Biology 4 (2006) 1213-1226
83. Rauschecker J.P., Tian B., and Hauser M. Processing of complex sounds in the macaque nonprimary auditory-cortex. Science 268 (1995) 111-114
84. Recanzone G.H. Representation of con-specific vocalizations in the core and belt areas of the auditory cortex in the alert macaque monkey. Journal of Neuroscience 28 (2008) 13184-13193
85. Remedios R., Logothetis N.K., and Kayser C. An auditory region in the primate insular cortex responding preferentially to vocal communication sounds. Journal of Neuroscience 29 (2009) 1034-1045
86. Robinson D.A., and Fuchs A.F. Eye movements evoked by stimulation of frontal eye fields. Journal of Neurophysiology 32 (1969) 637-648
87. Romanski, L.M., Ghazanfar, A.A. in press. The primate frontal and temporal lobes and their role in multisensory vocal communication. In: Platt, M.L., Ghazanfar, A.A. (Eds.), Primate Neuroethology. Oxford University Press, Oxford.
88. Romanski L.M., Bates J.F., and Goldman-Rakic P.S. Auditory belt and parabelt projections to the prefrontal cortex in the rhesus monkey. Journal of Comparative Neurology 403 (1999) 141-157
89. Romanski L.M., Averbeck B.B., and Diltz M. Neural representation of vocalizations in the primate ventrolateral prefrontal cortex. Journal of Neurophysiology 93 (2005) 734-747
90. Rosenblum L.D. Primacy of multimodal speech perception. In: Pisoni D.B., and Remez R.E. (Eds). Handbook of Speech Perception (2005), Blackwell, Malden, MA 51-78
91. Sams M., Mottonen R., and Sihvonen T. Seeing and hearing others and oneself talk. Cognitive Brain Research 23 (2005) 429-435
92. Schall J.D., Morel A., King D.J., and Bullier J. Topography of visual cortex connections with frontal eye field in macaque: convergence and segregation of processing streams. Journal of Neuroscience 15 (1995) 4464-4487
93. Schroeder C.E., and Foxe J.J. The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Cognitive Brain Research 14 (2002) 187-198
94. Schroeder C.E., Lakatos P., Kajikawa Y., Partan S., and Puce A. Neuronal oscillations and visual amplification of speech. Trends in Cognitive Sciences 12 (2008) 106-113
95. Schroeder C.E., Lindsley R.W., Specht C., Marcovici A., Smiley J.F., and Javitt D.C. Somatosensory input to auditory association cortex in the macaque monkey. Journal of Neurophysiology 85 (2001) 1322-1327
96. Schwartz J.-L., Berthommier F., and Savariaux C. Seeing to hear better: evidence for early audio-visual interactions in speech identification. Cognition 93 (2004) B69-B78
97. Seltzer B., and Pandya D.N. Frontal-lobe connections of the superior temporal sulcus in the rhesus-monkey. Journal of Comparative Neurology 281 (1989) 97-113
98. Seltzer B., and Pandya D.N. Parietal, temporal, and occipital projections to cortex of the superior temporal sulcus in the rhesus monkey: a retrograde tracer study. Journal of Comparative Neurology 343 (1994) 445-463
99. Sinnott J.M., Stebbins W.C., and Moody D.B. Regulation of voice amplitude by monkey. Journal of the Acoustical Society of America 58 (1975) 412-414
100. Smiley J.F., Hackett T.A., Ulbert I., Karmas G., Lakatos P., Javitt D.C., and Schroeder C.E. Multisensory convergence in auditory cortex, I. Cortical connections of the caudal superior temporal plane in macaque monkeys. Journal of Comparative Neurology 502 (2007) 894-923
101. Sugihara T., Diltz M.D., Averbeck B.B., and Romanski L.M. Integration of auditory and visual communication information in the primate ventrolateral prefrontal cortex. Journal of Neuroscience 26 (2006) 11138-11147
102. Tremblay S., Shiller D.M., and Ostry D.J. Somatosensory basis of speech production. Nature 423 (2003) 866-869
103. van Wassenhove V., Grant K.W., and Poeppel D. Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences of the United States of America 102 (2005) 1181-1186
104. Werner-Reiss U., Kelly K.A., Trause A.S., Underhill A.M., and Groh J.M. Eye position affects activity in primary auditory cortex of primates. Current Biology 13 (2003) 554-562
105. Yehia H., Rubin P., and Vatikiotis-Bateson E. Quantitative association of vocal-tract and facial behavior. Speech Communication 26 (1998) 23-43
106. Yehia H.C., Kuratate T., and Vatikiotis-Bateson E. Linking facial animation, head motion and speech acoustics. Journal of Phonetics 30 (2002) 555-568