Crossmodal interactions in audiovisual emotion processing

NeuroImage

Volumn 60, Issue 1, 2012, Pages 553-561

  Müller, Veronika I ^a,b,c   Cieslik, Edna C ^a,b,c   Turetsky, Bruce I ^d   Eickhoff, Simon B ^a,b,c,e  

^a RWTH AACHEN UNIVERSITY   (Germany)

^b FORSCHUNGSZENTRUM JÜLICH   (Germany)

^c JARA TRANSLATIONAL BRAIN MEDICINE   (Germany)

^d UNIVERSITY OF PENNSYLVANIA   (United States)

^e HEINRICH HEINE UNIVERSITY   (Germany)

Author keywords

Amygdala; Audiovisual; DCM; FMRI; Incongruence; PSTS

Indexed keywords

AMYGDALOID NUCLEUS; ARTICLE; AUDITORY DISCRIMINATION; AUDITORY STIMULATION; BAYES THEOREM; CAUSAL MODELING; CELL INTERACTION; CONTROLLED STUDY; EMOTIONALITY; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUSIFORM GYRUS; HEMISPHERIC DOMINANCE; HUMAN; HUMAN EXPERIMENT; INTERHEMISPHERIC TRANSFER; MALE; NERVE CELL NETWORK; NEUROMODULATION; NORMAL HUMAN; POSITIVE FEEDBACK; PRIORITY JOURNAL; SENSORIMOTOR INTEGRATION; SUPERIOR TEMPORAL SULCUS; TASK PERFORMANCE; VISUAL DISCRIMINATION; VISUAL STIMULATION;

AUDITORY PERCEPTION; BRAIN; EMOTIONS; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; VISUAL PERCEPTION;

EID: 84856084082     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2011.12.007     Document Type: Article

References (70)

1. Abbott L.F., Regehr W.G. Synaptic computation. Nature 2004, 431:796-803.
2. Adolphs R. What does the amygdala contribute to social cognition?. Ann. N. Y. Acad. Sci. 2010, 1191:42-61.
3. Ban T., Naito J., Kawamura K. Commissural afferents to the cortex surrounding the posterior part of the superior temporal sulcus in the monkey. Neurosci. Lett. 1984, 49:57-61.
4. Beauchamp M.S., Lee K.E., Argall B.D., Martin A. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 2004, 41:809-823.
5. Beauchamp M.S., Nath A.R., Pasalar S. fMRI-Guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect. J. Neurosci. 2010, 30:2414-2417.
6. Belin P., Zatorre R.J., Lafaille P., Ahad P., Pike B. Voice-selective areas in human auditory cortex. Nature 2000, 403:309-312.
7. Belin P., Fecteau S., Bedard C. Thinking the voice: neural correlates of voice perception. Trends Cogn. Sci. 2004, 8:129-135.
8. Benoit M.M., Raij T., Lin F.H., Jaaskelainen I.P., Stufflebeam S. Primary and multisensory cortical activity is correlated with audiovisual percepts. Hum. Brain Mapp. 2010, 31:526-538.
9. Calvert G.A., Campbell R., Brammer M.J. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol. 2000, 10:649-657.
10. Cieslik E.C., Zilles K., Grefkes C., Eickhoff S.B. Dynamic interactions in the fronto-parietal network during a manual stimulus-response compatibility task. NeuroImage 2011, 58(3):860-869.
11. Clarke S., Miklossy J. Occipital cortex in man: organization of callosal connections, related myelo- and cytoarchitecture, and putative boundaries of functional visual areas. J. Comp. Neurol. 1990, 298:188-214.
12. Dien J. A tale of two recognition systems: implications of the fusiform face area and the visual word form area for lateralized object recognition models. Neuropsychologia 2009, 47:1-16.
13. Dietrich S., Hertrich I., Alter K., Ischebeck A., Ackermann H. Semiotic aspects of human nonverbal vocalizations: a functional imaging study. Neuroreport 2007, 18:1891-1894.
14. Dolan R.J., Morris J.S., de Gelder B. Crossmodal binding of fear in voice and face. Proc. Natl. Acad. Sci. U. S. A. 2001, 98:10006-10010.
15. Eickhoff S.B., Stephan K.E., Mohlberg H., Grefkes C., Fink G.R., Amunts K., Zilles K. A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. NeuroImage 2005, 25:1325-1335.
16. Eickhoff S.B., Heim S., Zilles K., Amunts K. Testing anatomically specified hypotheses in functional imaging using cytoarchitectonic maps. NeuroImage 2006, 32:570-582.
17. Eickhoff S.B., Paus T., Caspers S., Grosbras M.H., Evans A.C., Zilles K., Amunts K. Assignment of functional activations to probabilistic cytoarchitectonic areas revisited. NeuroImage 2007, 36:511-521.
18. Eickhoff S.B., Heim S., Zilles K., Amunts K. A systems perspective on the effective connectivity of overt speech production. Philos. Transact. A Math. Phys. Eng. Sci 2009, 367:2399-2421.
19. Ethofer T., Anders S., Wiethoff S., Erb M., Herbert C., Saur R., Grodd W., Wildgruber D. Effects of prosodic emotional intensity on activation of associative auditory cortex. Neuroreport 2006, 17:249-253.
20. Ethofer T., Pourtois G., Wildgruber D. Investigating audiovisual integration of emotional signals in the human brain. Prog. Brain Res. 2006, 156:345-361.
21. Fairhall S.L., Ishai A. Effective connectivity within the distributed cortical network for face perception. Cereb. Cortex 2007, 17:2400-2406.
22. Fecteau S., Belin P., Joanette Y., Armony J.L. Amygdala responses to nonlinguistic emotional vocalizations. NeuroImage 2007, 36:480-487.
23. Fusar-Poli P., Placentino A., Carletti F., Allen P., Landi P., Abbamonte M., Barale F., Perez J., McGuire P., Politi P.L. Laterality effect on emotional faces processing: ALE meta-analysis of evidence. Neurosci. Lett. 2009, 452:262-267.
24. Gainotti G. What the study of voice recognition in normal subjects and brain-damaged patients tells us about models of familiar people recognition. Neuropsychologia 2011, 49:2273-2282.
25. Grefkes C., Eickhoff S.B., Nowak D.A., Dafotakis M., Fink G.R. Dynamic intra- and interhemispheric interactions during unilateral and bilateral hand movements assessed with fMRI and DCM. NeuroImage 2008, 41:1382-1394.
26. Grill-Spector K., Knouf N., Kanwisher N. The fusiform face area subserves face perception, not generic within-category identification. Nat. Neurosci. 2004, 7:555-562.
27. Gur R.C., Schroeder L., Turner T., McGrath C., Chan R.M., Turetsky B.I., Alsop D., Maldjian J., Gur R.E. Brain activation during facial emotion processing. NeuroImage 2002, 16:651-662.
28. Habel U., Windischberger C., Derntl B., Robinson S., Kryspin-Exner I., Gur R.C., Moser E. Amygdala activation and facial expressions: explicit emotion discrimination versus implicit emotion processing. Neuropsychologia 2007, 45:2369-2377.
29. Hackett T.A., Stepniewska I., Kaas J.H. Callosal connections of the parabelt auditory cortex in macaque monkeys. Eur. J. Neurosci. 1999, 11:856-866.
30. Hagan C.C., Woods W., Johnson S., Calder A.J., Green G.G., Young A.W. MEG demonstrates a supra-additive response to facial and vocal emotion in the right superior temporal sulcus. Proc. Natl. Acad. Sci. U. S. A. 2009, 106:20010-20015.
31. Hein G., Knight R.T. Superior temporal sulcus-it's my area: or is it?. J. Cogn. Neurosci. 2008, 20:2125-2136.
32. Herrington J.D., Taylor J.M., Grupe D.W., Curby K.M., Schultz R.T. Bidirectional communication between amygdala and fusiform gyrus during facial recognition. NeuroImage 2011, 56:2348-2355.
33. Hocking J., Price C.J. The role of the posterior superior temporal sulcus in audiovisual processing. Cereb. Cortex 2008, 18:2439-2449.
34. Hunter M.D., Eickhoff S.B., Pheasant R.J., Douglas M.J., Watts G.R., Farrow T.F., Hyland D., Kang J., Wilkinson I.D., Horoshenkov K.V., Woodruff P.W. The state of tranquility: subjective perception is shaped by contextual modulation of auditory connectivity. NeuroImage 2010, 53:611-618.
35. Ishai A., Schmidt C.F., Boesiger P. Face perception is mediated by a distributed cortical network. Brain Res. Bull. 2005, 67:87-93.
36. James T.W., Vanderklok R.M., Stevenson R.A., James K.H. Multisensory perception of action in posterior temporal and parietal cortices. Neuropsychologia 2011, 49:108-114.
37. Kanwisher N., Yovel G. The fusiform face area: a cortical region specialized for the perception of faces. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2006, 361:2109-2128.
38. Kanwisher N., McDermott J., Chun M.M. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neurosci. 1997, 17:4302-4311.
39. Klasen M., Kenworthy C.A., Mathiak K.A., Kircher T.T., Mathiak K. Supramodal representation of emotions. J. Neurosci. 2011, 31:13635-13643.
40. Kreifelts B., Ethofer T., Grodd W., Erb M., Wildgruber D. Audiovisual integration of emotional signals in voice and face: an event-related fMRI study. NeuroImage 2007, 37:1445-1456.
41. Kreifelts B., Ethofer T., Shiozawa T., Grodd W., Wildgruber D. Cerebral representation of non-verbal emotional perception: fMRI reveals audiovisual integration area between voice- and face-sensitive regions in the superior temporal sulcus. Neuropsychologia 2009, 47:3059-3066.
42. Kreifelts B., Ethofer T., Huberle E., Grodd W., Wildgruber D. Association of trait emotional intelligence and individual fMRI-activation patterns during the perception of social signals from voice and face. Hum. Brain Mapp. 2010, 31:979-991.
43. McDonald A.J. Cortical pathways to the mammalian amygdala. Prog. Neurobiol. 1998, 55:257-332.
44. McGurk H., MacDonald J. Hearing lips and seeing voices. Nature 1976, 264:746-748.
45. Mechelli A., Crinion J.T., Long S., Friston K.J., Lambon Ralph M.A., Patterson K., McClelland J.L., Price C.J. Dissociating reading processes on the basis of neuronal interactions. J. Cogn. Neurosci. 2005, 17:1753-1765.
46. Meyer M., Zysset S., von Cramon D.Y., Alter K. Distinct fMRI responses to laughter, speech, and sounds along the human peri-sylvian cortex. Brain Res. Cogn. Brain Res. 2005, 24:291-306.
47. Morris J.P., Pelphrey K.A., McCarthy G. Face processing without awareness in the right fusiform gyrus. Neuropsychologia 2007, 45:3087-3091.
48. Müller V.I., Habel U., Derntl B., Schneider F., Zilles K., Turetsky B.I., Eickhoff S.B. Incongruence effects in crossmodal emotional integration. NeuroImage 2011, 54:2257-2266.
49. Nath A.R., Beauchamp M.S. A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion. NeuroImage 2012, 59(1):781-787.
50. Oldfield R.C. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 1971, 9:97-113.
51. Olson I.R., Gatenby J.C., Gore J.C. A comparison of bound and unbound audio-visual information processing in the human cerebral cortex. Brain Res. Cogn. Brain Res. 2002, 14:129-138.
52. Pandya D.N., Rosene D.L. Laminar termination patterns of thalamic, callosal, and association afferents in the primary auditory area of the rhesus monkey. Exp. Neurol. 1993, 119:220-234.
53. Phillips M.L., Young A.W., Scott S.K., Calder A.J., Andrew C., Giampietro V., Williams S.C., Bullmore E.T., Brammer M., Gray J.A. Neural responses to facial and vocal expressions of fear and disgust. Proc. Biol. Sci. 1998, 265:1809-1817.
54. Pourtois G., de Gelder B., Bol A., Crommelinck M. Perception of facial expressions and voices and of their combination in the human brain. Cortex 2005, 41:49-59.
55. Rehme A.K., Eickhoff S.B., Wang L.E., Fink G.R., Grefkes C. Dynamic causal modeling of cortical activity from the acute to the chronic stage after stroke. NeuroImage 2011, 55:1147-1158.
56. Rhodes G., Byatt G., Michie P.T., Puce A. Is the fusiform face area specialized for faces, individuation, or expert individuation?. J. Cogn. Neurosci. 2004, 16:189-203.
57. Robins D.L., Hunyadi E., Schultz R.T. Superior temporal activation in response to dynamic audio-visual emotional cues. Brain Cogn. 2009, 69:269-278.
58. Sabatinelli D., Fortune E.E., Li Q., Siddiqui A., Krafft C., Oliver W.T., Beck S., Jeffries J. Emotional perception: meta-analyses of face and natural scene processing. NeuroImage 2011, 54:2524-2533.
59. Sander K., Scheich H. Auditory perception of laughing and crying activates human amygdala regardless of attentional state. Brain Res. Cogn. Brain Res. 2001, 12:181-198.
60. Sekiyama K., Kanno I., Miura S., Sugita Y. Auditory-visual speech perception examined by fMRI and PET. Neurosci. Res. 2003, 47:277-287.
61. Stephan K.E., Kasper L., Harrison L.M., Daunizeau J., den Ouden H.E., Breakspear M., Friston K.J. Nonlinear dynamic causal models for fMRI. NeuroImage 2008, 42:649-662.
62. Stephan K.E., Penny W.D., Daunizeau J., Moran R.J., Friston K.J. Bayesian model selection for group studies. NeuroImage 2009, 46:1004-1017.
63. Stephan K.E., Penny W.D., Moran R.J., den Ouden H.E., Daunizeau J., Friston K.J. Ten simple rules for dynamic causal modeling. NeuroImage 2010, 49:3099-3109.
64. Taylor K.I., Moss H.E., Stamatakis E.A., Tyler L.K. Binding crossmodal object features in perirhinal cortex. Proc. Natl. Acad. Sci. U. S. A. 2006, 103:8239-8244.
65. Van Essen D.C., Newsome W.T., Bixby J.L. The pattern of interhemispheric connections and its relationship to extrastriate visual areas in the macaque monkey. J. Neurosci. 1982, 2:265-283.
66. Wallace M.T., Ramachandran R., Stein B.E. A revised view of sensory cortical parcellation. Proc. Natl. Acad. Sci. U. S. A. 2004, 101:2167-2172.
67. Wang L.E., Fink G.R., Diekhoff S., Rehme A.K., Eickhoff S.B., Grefkes C. Noradrenergic enhancement improves motor network connectivity in stroke patients. Ann. Neurol. 2011, 69:375-388.
68. Wiethoff S., Wildgruber D., Kreifelts B., Becker H., Herbert C., Grodd W., Ethofer T. Cerebral processing of emotional prosody-influence of acoustic parameters and arousal. NeuroImage 2008, 39:885-893.
69. Worsley K.J., Marrett S., Neelin P., Vandal A.C., Friston K.J., Evans A.C. A unified statistical approach for determining significant signals in images of cerebral activation. Hum. Brain Mapp. 1996, 4:58-73.
70. Yukie M. Connections between the amygdala and auditory cortical areas in the macaque monkey. Neurosci. Res. 2002, 42:219-229.