Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex

Journal of Neuroscience

Volumn 25, Issue 20, 2005, Pages 5004-5012

  Ghazanfar, Asif A ^a,b   Maier, Joost X ^a   Hoffman, Kari L ^a   Logothetis, Nikos K ^a  

^a MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

^b PRINCETON UNIVERSITY   (United States)

Author keywords

Bimodal; Crossmodal; Speech; Superior temporal sulcus; Temporal lobe; Vocalization

Indexed keywords

ARTICLE; AUDITORY CORTEX; ELECTRIC POTENTIAL; FACIAL EXPRESSION; HEARING; INTERPERSONAL COMMUNICATION; NONHUMAN; PERCEPTIVE DISCRIMINATION; PRIORITY JOURNAL; RHESUS MONKEY; SENSORY CORTEX; TEMPORAL LOBE; VISION; VISUAL CORTEX; VOCALIZATION; VOICE;

ACOUSTIC STIMULATION; ANALYSIS OF VARIANCE; ANIMALS; AUDITORY CORTEX; AUDITORY PERCEPTION; BRAIN MAPPING; ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS, AUDITORY; FACIAL EXPRESSION; MACACA MULATTA; MALE; MODELS, NEUROLOGICAL; PATTERN RECOGNITION, VISUAL; PHOTIC STIMULATION; REACTION TIME; TIME FACTORS; VOICE;

EID: 19044387911     PISSN: 02706474     EISSN: None     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.0799-05.2005     Document Type: Article

References (51)

1. Abry C, Lallouache M-T, Cathiard M-A (1996) How can coarticulation models account for speech sensitivity in audio-visual desynchronization? In: Speechreading by humans and machines: models, systems and applications (Stork D, Henneke M, eds), pp 247-255. Berlin: Springer.
2. Adolphs R (2003) The cognitive neuroscience of human social behaviour. Nat Rev Neurosci 4:165-178.
3. Barbour DL, Wang X (2003) Contrast tuning in auditory cortex. Science 299:1073-1075.
4. Barradough NE, Xiao D, Baker CI, Oram MW, Perrett DI (2005) Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J Cogn Neurosci 17:377-391.
5. Beauchamp MS, Lee KE, Argall BD, Martin A (2004) Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41:809-823.
6. Bell AH, Corneil BD, Munoz DP, Meredith MA (2003) Engagement of visual fixation suppresses sensory responsiveness and multisensory integration in the primate superior colliculus. Eur J Neurosci 18:2867-2873.
7. Bernstein LE, Auer ET, Moore JK, Ponton CW, Don M, Singh M (2002) Visual speech perception without primary auditory cortex activation. NeuroReport 13:311-315.
8. Bernstein LE, Auer ET, Takayanagi S (2004) Auditory speech detection in noise enhanced by lipreading. Speech Comm 44:5-18.
9. Besle J, Fort A, Delpuech C, Giard MH (2004) Bimodal speech: early suppressive visual effects in human auditory cortex. Eur J Neurosci 20:2225-2234.
10. Bullock TH (1997) Signals and signs in the nervous system: the dynamic anatomy of electrical activity is probably information-rich. Proc Natl Acad Sci USA 94:1-6.
11. Callan DE, Jones JA, Munhall K, Callan AM, Kroos C, Vatikiotis-Bateson E (2003) Neural processes underlying perceptual enhancement by visual speech gestures. NeuroReport 14:2213-2218.
12. Calvert GA (2001) Crossmodal processing in the human brain: Insights from functional neuroimaging studies. Cereb Cortex 11:1110-1123.
13. Calvert GA, Brammer MJ, Bullmore ET, Campbell R, Iversen SD, David AS (1999) Response amplification in sensory-specific cortices during cross-modal binding. NeuroReport 10:2619-2623.
14. Calvert GA, Campbell R, Brammer MJ (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10:649-657.
15. Cheney DL, Seyfarth RM (1982) How vervet monkeys perceive their grunts: field playback experiments. Anim Behav 30:739-751.
16. Frens MA, Van Opstal AJ (1998) Visual-auditory interactions modulate saccade-related activity in monkey superior colliculus. Brain Res Bull 46:211-224.
17. Fu K-MG, Shah AS, O'Connell MN, McGinnis T, Eckholdt H, Lakatos P, Smiley J, Schroeder CE (2004) Timing and laminar profile of eye-position effects on auditory responses in primate auditory cortex. J Neurophysiol 92:3522-3531.
18. Gail A, Brinksmeyer HJ, Eckhorn R (2004) Perception-related modulations of local field potential power and coherence in primary visual cortex of awake monkey during binocular rivalry. Cereb Cortex 14:300-313.
19. Ghazanfar AA, Logothetis NK (2003) Facial expressions linked to monkey calls. Nature 423:937-938.
20. Ghazanfar AA, Santos LR (2004) Primate brains in the wild: the neural bases for social interactions. Nat Rev Neurosci 5:603-616.
21. Hackett TA (2002) The comparative anatomy of the primate auditory cortex. In: Primate audition: ethology and neurobiology (Ghazanfar AA, ed), pp 199-226. Boca Raton, FL: CRC.
22. Harries MH, Perrett DI (1991) Visual processing of faces in temporal cortex - physiological evidence for a modular organization and possible an-atomical correlates. J Cogn Neurosci 3:9-24.
23. Hauser MD, Marler P (1993) Food-associated calls in rhesus macaques (Macaca mulatta). I. Socioecological factors. Behav Ecol 4:194-205.
24. Hauser MD, Evans CS, Marier P (1993) The role of articulation in the production of rhesus monkey, Macaca mulatta, vocalizations. Anim Behav 45:423-433.
25. Henrie JA, Shapley R (2005) LFP power spectra in V1 cortex: the graded effect of stimulus contrast. J Neurophysiol, in press.
26. Izumi A, Kojima S (2004) Matching vocalizations to vocalizing faces in a chimpanzee (Pan troglodytes). Anim Cogn 7:179-184.
27. Kayser C, Kim M, Ugurbil K, Kim D-S, Konig P (2004) A comparison of hemodynamic and neural responses in cat visual cortex using complex stimuli. Cereb Cortex 14:881-891.
28. Lauritzen M (2001) Relationship of spikes, synaptic activity, and local changes of cerebral blood flow. J Cereb Blood Flow Metab 21:1367-1383.
29. Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A (2001) Neurophysiological investigation of the basis of the fMRI signal. Nature 412:150-157.
30. Logothetis NK, Merkle H, Augath M, Trinath T, Ugurbil K (2002) Ultra high-resolution fMRI in monkeys with implanted RF coils. Neuron 35:227-242.
31. Massaro DW (1998) Perceiving talking faces: from speech perception to a behavioral principle. Cambridge, MA: MIT.
32. Mathiesen C, Cesar K, Algoren N, Lauritzen M (1998) Modification of activity-dependent increases of cerebral blood flow by excitatory synaptic activity and spikes in rat cerebellar cortex. J Physiol (Lond) 512:555-566.
33. Mehring C, Rickert J, Vaadia E, Cardoso de Oliveira S, Aertsen A, Rotter S (2003) Inference of hand movements from local field potentials in monkey motor cortex. Nat Neurosci 6:1253-1254.
34. Meredith MA, Stein BE (1986) Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol 56:640-662.
35. Meredith MA, Nemitz JW, Stein BE (1987) Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci 7:3215-3229.
36. Norena A, Eggermont JJ (2002) Comparison between local field potentials and unit cluster activity in primary auditory cortex and anterior auditory field in the cat. Hear Res 166:202-213.
37. Olson IR, Gatenby JC, Gore JC (2002) A comparison of bound and unbound audio-visual information processing in the human cerebral cortex. Brain Res Cogn Brain Res 14:129-138.
38. Palombit RA, Cheney DL, Seyfarth RM (1999) Male grunts as mediators of social interaction with females in wild chacma baboons (Papio cynocephalus ursinus). Behaviour 136:221-242.
39. Partan SR (2002) Single and multichannel signal composition: facial expressions and vocalizations of rhesus macaques (Macaca mulatta). Behaviour 139:993-1027.
40. Pesaran B, Pezaris JS, Sahani M, Mitra PP, Andersen RA (2002) Temporal structure in neuronal activity during working memory in macaque parietal cortex. Nat Neurosci 5:805-811.
41. Pfingst BE, O'Connor TA (1980) Vertical stereotaxic approach to auditory-cortex in the unanesthetized monkey. J Neurosci Methods 2:33-45.
42. Rauschecker JP, Tian B, Hauser M (1995) Processing of complex sounds in the macaque nonprimary auditory-cortex. Science 268:111-114.
43. Recanzone GH, Guard DC, Phan ML (2000) Frequency and intensity response properties of single neurons in the auditory cortex of the behaving macaque monkey. J Neurophysiol 83:2315-2331.
44. Sams M, Aulanko R, Hamalainen M, Hari R, Lounasmaa OV, Lu S-T, Simola J (1991) Seeing speech: visual information from lip movements modifies activity in the human auditory cortex. Neurosci Lett 127:141-145.
45. Schroeder CE, Foxe JJ (2002) The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res Cogn Brain Res 14:187-198.
46. Schroeder CE, Lindsey RW, Specht C, Marcovici A, Smiley JF, Javitt DC (2001) Somatosensory input to auditory association cortex in the macaque monkey. J Neurophysiol 85:1322-1327.
47. Seltzer B, Pandya DN (1994) Parietal, temporal, and occipital projections to cortex of the superior temporal sulcus in the rhesus monkey: a retrograde tracer study. J Comp Neurol 343:445-463.
48. Stein BE, Meredith MA (1993) The merging of the senses. Cambridge, MA: MIT.
49. van Wassenhove V, Grant KW, Poeppel D (2005) Visual speech speeds up the neural processing of auditory speech. Proc Natl Acad Sci USA 102:1181-1186.
50. Wallace MT, Wilkinson LK, Stein BE (1996) Representation and integration of multiple sensory inputs in primate superior colliculus. J Neurophysiol 76:1246-1266.
51. Wright TM, Pelphrey KA, Allison T, McKeown MJ, McCarthy G (2003) Polysensory interactions along lateral temporal regions evoked by audio-visual speech. Cereb Cortex 13:1034-1043.