Involvement of Right STS in Audio-Visual Integration for Affective Speech Demonstrated Using MEG

PLoS ONE

Volumn 8, Issue 8, 2013, Pages

  Hagan, Cindy C ^a,b   Woods, Will ^a,c   Johnson, Sam ^a   Green, Gary G R ^a   Young, Andrew W ^a  

^a UNIVERSITY OF YORK   (United Kingdom)

^b UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^c SWINBURNE UNIVERSITY OF TECHNOLOGY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ALPHA RHYTHM; ARTICLE; AUDIO VISUAL INTEGRATION; AUDITORY STIMULATION; BETA RHYTHM; BRAIN FUNCTION; BRAIN REGION; CONTROLLED STUDY; EMOTION; EVOKED AUDITORY RESPONSE; EVOKED VISUAL RESPONSE; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; GAMMA RHYTHM; HABITUATION; HUMAN; HUMAN EXPERIMENT; MAGNETOENCEPHALOGRAPHY; MALE; NORMAL HUMAN; NUCLEAR MAGNETIC RESONANCE IMAGING; SPEECH PERCEPTION; STIMULUS RESPONSE; SUPERIOR TEMPORAL SULCUS; TASK PERFORMANCE; VISUAL CORTEX; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; AUDITORY PERCEPTION; BRAIN MAPPING; BRAIN WAVES; EMOTIONS; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MAGNETOENCEPHALOGRAPHY; MALE; PHOTIC STIMULATION; SPEECH; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84881529048     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0070648     Document Type: Article

References (53)

1. De Gelder B, Böcker KB, Tuomainen J, Hensen M, Vroomen J, (1999) The combined perception of emotion from voice and face: Early interaction revealed by human electric brain responses. Neurosci Letters 260: 133-136.
2. Klasen M, Chen YH, Mathiak K, (2012) Multisensory émotions: perception, combination and underlying neural processes. Rev Neurosci 23: 381-392.
3. McGurk H, MacDonald J, (1976) Hearing lips and seeing voices. Nature 264: 746-748.
4. Haxby JV, Hoffman EA, Gobbini MI, (2000) The distributed human neural system for face perception. Trends Cog Sci 4: 223-233.
5. Haxby JV, Hoffman EA, Gobbini MI, (2002) Human neural systems for face recognition and social communication. Bio Psych 51: 59-67.
6. Calder AJ, Young AW, (2005) Understanding the recognition of facial identity and facial expression. Nat Neurosci Reviews 6: 641-651.
7. Calvert GA, (2001) Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb Cortex 11: 1110-1123.
8. Meredith MA, Stein BE, (1983) Interactions among converging sensory inputs in the superior colliculus. Science 221: 389-391.
9. Stein BE, Jiang W, Stanford TR (2004) Multisensory integration in single neurons of the midbrain. In Calvert G, Spence C, Stein BE, editors, The Handbook of Multisensory Processes: MIT Press. pp. 243-264.
10. Barraclough NE, Xiao D, Baker CI, Oram MW, Perrett DI, (2005) Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J Cog Neurosci 17: 377-391.
11. Beauchamp MS, (2005) See me, hear me, touch me: Multisensory integration in lateral occipitaltemporal cortex. Curr Op Neurobi 15: 145-153.
12. Hagan CC, Woods W, Johnson S, Calder AJ, Green GGR, et al. (2009) MEG demonstrates a supra-additive response to facial and vocal emotion in the right superior temporal sulcus. Proc Nat Acad Sci, USA 106: 20010-20015.
13. Calvert GA, Campbell R, Brammer MJ, (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Bio 10: 649-657.
14. Blonder LX, Bowers D, Heilman KM, (1991) The role of the right hemisphere in emotional communication. Brain 114: 1115-1127.
15. Ethofer T, Anders S, Erb M, Herbert C, Wiethoff S, et al. (2006) Cerebral pathways in processing of affective prosody: a dynamic causal modeling study. Neuroimage 30: 580-587.
16. Robins DL, Hunyadi E, Schultz RT, (2009) Superior temporal activation in response to dynamic audio-visual emotional cues. Brain Cogn 69: 269-278.
17. Kreifelts B, Ethofer T, Grodd W, Erb M, Wildgruber D, (2007) Audiovisual integration of emotional signals in voice and face: an event-related fMRI study. Neuroimage 37: 1445-1456.
18. Chen YH, Edgar JC, Holroyd T, Dammers J, Thönnessen H, et al. (2010) Neuromagnetic oscillations to emotional faces and prosody. Eur J Neurosci 31: 1818-1827.
19. Jones JA, Callan DE, (2003) Brain activity during audiovisual speech perception: An fMRI study of the McGurk effect. NeuroReport 14: 1129-1133.
20. Hocking J, Price CJ, (2008) The role of the posterior superior temporal sulcus in audiovisual processing. Cereb Cortex 18: 2439-2449.
21. Adolphs R, Tranel D, Damasio H, Damasio A, (1994) Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala. Nature 372: 669-672.
22. Scott SK, Young AW, Calder AJ, Hellawell DJ, Aggleton JP, et al. (1997) Impaired auditory recognition of fear and anger following bilateral amygdala lesions. Nature 385: 254-257.
23. Young AW, Aggleton JP, Hellawell DJ, Johnson M, Broks P, et al. (1995) Face processing impairments after amygdalotomy. Brain 118: 15-24.
24. Calder AJ, Keane J, Manes F, Antoun N, Young AW, (2000) Impaired recognition and experience of disgust following brain injury. Nat Neurosci 3: 1077-1078.
25. Ethofer T, Pourtois G, Wildgruber D (2006) Investigating audiovisual integration of emotional signals in the human brain. In Anders, Ende, Junghofer, Kissler, Wildgruber, editors, Progress in Brain Research (Vol. 156).pp. 345-361.
26. Giard MH, Peronnet F, (1999) Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. J Cog Neurosci 11: 473-490.
27. Pourtois G, de Gelder B, Vroomen J, Rossion B, Crommelinck M, (2000) The time-course of intermodal binding between seeing and hearing affective information. Neuroreport 11: 1329-1333.
28. Bradley MM, Lang PJ (1999) Affective norms for English words (ANEW): Stimuli, instruction manual and affective ratings. Technical report C-1, Gainesville, FL. The Center for Research in Psychophysiology, University of Florida.
29. Kozinska D, Carducci F, Nowinski K, (2001) Automatic alignment of EEG/MEG and MRI data sets. Clin Neurophys 112: 1553-1561.
30. van Veen BD, van Drongelen W, Yuchtman M, Suzuki A, (1997) Localization of brain electrical activity via linearly constrained minimum variance spatial filtering. IEEE Trans Biomed Eng 44: 867-880.
31. Huang MX, Shih JJ, Lee RR, Harrington DL, Thoma RJ, et al. (2004) Commonalities and differences among vectorized beamformers in electromagnetic source imaging. Brain Topo 16: 139-158.
32. Hillebrand A, Singh KD, Holliday IE, Furlong PL, Barnes GR, (2005) A new approach to neuroimaging with magnetoencephalography. Human Brain Map 25: 199-211.
33. Herdman AT, Fujioka T, Chau W, Ross B, Pantev C, et al. (2006) Cortical oscillations related to processing congruent and incongruent grapheme-phoneme pairs. Neurosci Letters 399: 61-66.
34. van Wassenhove V, Grant KW, Poeppel D, (2005) Visual speech speeds up the neural processing of auditory speech. Proc Natl Acad Sci U S A 102: 1181-1186.
35. Arnal LH, Morillon B, Kell CA, Giraud AL, (2009) Dual neural routing of visual facilitation in speech processing. J Neurosci 29: 13445-13453.
36. van Wassenhove V, Grant KW, Poeppel D, (2007) Temporal window of integration in bimodal speech. Neuropsychologia 45: 598-607.
37. De Gelder B, Bertelson P, Vroomen J (1996) Aspects of modality in audio-visual processes. In Stork DG and Hennecke ME, editors, Speechreading by humans and machines. NATO ASI Series F, (vol. 150,). Berlin: Springer-Verlag, Gmbh.pp.179-192.
38. Miller LM, D'Esposito M, (2005) Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J Neurosci 25: 5884-5893.
39. Stein BE, Meredith MA (1993) Merging of the senses. Cambridge, MA: MIT Press.
40. Stevenson RA, James TW, (2009) Audiovisual integration in human superior temporal sulcus: Inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 44: 1210-1223.
41. Frith CD, Frith U, (2006) How we predict what other people are going to do. Brain Research 1079: 36-46.
42. De Gelder B, Vroomen J, (2000) The perception of emotions by ear and by eye. Cog Emo 14: 289-311.
43. Talsma D, Doty TJ, Woldorff MG, (2007) Selective attention and audio-visual integration: is attending to both modalities a prerequisite for early integration? Cereb Cortex 17: 679-690.
44. Schroeder CE, Lakatos P, Kajikawa Y, Partan S, Puce A, (2008) Neuronal oscillations and visual amplification of speech. Trends Cog Sci 12: 106-113.
45. Lakatos P, Shah AS, Knuth KH, Ulbert I, Karmos G, et al. (2005) An oscillatory heirarchy controlling neuronal excitability and stimulus processing in the auditory cortex. J Neurophys 94: 1904-1911.
46. Fingelkurts AA, Fingelkurts AA, Krause CM, (2007) Composition of brain oscillations and their functions in the maintenance of auditory, visual and audio-visual speech percepts: An exploratory study. Cog Proc 8: 183-199.
47. Luo H, Poeppel D, (2007) Phase patterns of neuronal responses reliably discriminate speech in human auditory cortex. Neuron 54: 1001-1010.
48. Classen J, Gerloff C, Honda M, Hallett M, (1998) Integrative visuomotor behavior is associated with interregionally coherent oscillations in the human brain. J Neurophys 79: 1567-1573.
49. Jokisch D, Jensen O, (2007) Modulation of gamma and alpha activity during a working memory task engaging the dorsal or ventral stream. J Neurosci 27: 3244-3251.
50. Tallon-Baudry C, Bertrand O, (1999) Oscillatory gamma activity in humans and its role in object representation. Trends Cog Sci 3: 151-162.
51. Senkowski D, Schneider TR, Foxe JJ, Engel AK, (2008) Crossmodal binding through neural coherence: Implications for multisensory processing. Trends Neurosci 31: 401-409.
52. Singer W, Gray CM, (1995) Visual feature integration and the temporal correlation hypothesis. Ann Rev Neurosci 18: 555-586.
53. Schroeder CE, Lakatos P, Kajikawa Y, Partan S, Puce A, (2008) Neuronal oscillations and visual amplification of speech. Trends in Cog Sci 12: 106-113.