Unconscious presentation of fearful face modulates electrophysiological responses to emotional prosody

Cerebral Cortex

Volumn 25, Issue 3, 2015, Pages 817-832

  Doi, Hirokazu ^a   Shinohara, Kazuyuki ^a  

^a NAGASAKI UNIVERSITY GRADUATE SCHOOL OF BIOMEDICAL SCIENCES   (Japan)

Author keywords

Cross modal integration; facial expression; fear; prosody; unconscious perception

Indexed keywords

ACCURACY; ADULT; ARTICLE; AWARENESS; EGO DEVELOPMENT; ELECTROENCEPHALOGRAM; EMOTIONAL PROSODY; EMOTIONALITY; EVENT RELATED POTENTIAL; FACIAL EXPRESSION; FEAR; FEMALE; FUSIFORM GYRUS; HUMAN; HUMAN EXPERIMENT; LATENT PERIOD; LAUGHTER; MALE; NORMAL HUMAN; PRIORITY JOURNAL; BRAIN; ELECTROENCEPHALOGRAPHY; EMOTION; EVOKED RESPONSE; HEARING; PERCEPTIVE THRESHOLD; PHYSIOLOGY; RECOGNITION; VISION; YOUNG ADULT;

ADULT; AUDITORY PERCEPTION; BRAIN; ELECTROENCEPHALOGRAPHY; EMOTIONS; EVOKED POTENTIALS; FACIAL EXPRESSION; FEAR; FEMALE; HUMANS; LAUGHTER; MALE; RECOGNITION (PSYCHOLOGY); SUBLIMINAL STIMULATION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84925420198     PISSN: 10473211     EISSN: 14602199     Source Type: Journal    
DOI: 10.1093/cercor/bht282     Document Type: Article

References (108)

1. Anderson AK, Christoff K, Stappen I, Panitz D, Ghahremani DG, Glover G, Gabrieli JDE, Sobel N. 2003. Dissociated neural representations of intensity and valence in human olfaction. Nat Neurosci. 6:196-202
2. Armony JL, Dolan RJ. 2002. Modulation of spatial attention by fearconditioned stimuli: an event-related fMRI study. Neuropsychologia. 40:817-826
3. Belin P, Fillion-Bilodeau S, Gosselin F. 2008. The Montreal Affective Voices: A validated set of nonverbal affect bursts for research on auditory affective processing. Behav Res Methods. 40:531-539
4. Blank H, Anwander A, von Kriegstein K. 2011. Direct structural connections between voice-and face-recognition areas. J Neurosci. 31:12906-12915
5. Brefczynski-Lewis J, Lowitszch S, Parsons M, Lemieux S, Puce A. 2009. Audiovisual non-verbal dynamic faces elicit converging fMRI and ERP responses. Brain Topogr. 21:193-206
6. Bridge H, Thomas O, Jbabdi S, Cowey A. 2008. Changes in connectivity after visual cortical brain damage underlie altered visual function. Brain. 131:1433-1444
7. Britton JC, Taylor SF, Sudheimer KD, Liberzon I. 2006. Facial expressions and complex IAPS pictures: common and differential networks. NeuroImage. 31:906-919
8. Campanella S, Belin P. 2007. Integrating face and voice in person perception. Trends Cogn Sci. 11:535-543
9. Cappe C, Thut G, Romei V, Murray MM. 2010. Auditory-visual multisensory interactions in humans: timing, topography, directionality, and sources. J Neurosci. 30:12572-12580
10. Carretié L, Hinojosa JA, Mercado F, Tapia M. 2005. Cortical response to subjectively unconscious danger. NeuroImage. 24:615-623
11. Collignon O, Girard S, Gosselin F, Roy S, Saint-Amour D, Lassonde M, Lepore F. 2008. Audio-visual integration of emotion expression. Brain Res. 1242:126-135
12. Cuthbert BN, Schupp HT, Bradley MM, Birbaumer N, Lang PJ. 2000. Brain potentials in affective picture processing: covariation with autonomic arousal and affective report. Biol Psychol. 52:95-111
13. de Gelder B, Bertelson P. 2003. Multisensory integration, perception and ecological validity. Trends Cogn Sci. 7:460-467
14. de Gelder B, Böcker KBE, Tuomainen J, Hensen M, Vroomen J. 1999. The combined perception of emotion from voice and face: early interaction revealed by human electric brain responses. Neurosci Lett. 260:133-136
15. de Gelder B, Morris JS, Dolan RJ. 2005. Unconscious fear influences emotional awareness of faces and voices. Proc Natl Acad Sci USA. 102:18682-18687
16. de Gelder B, Pourtois G, Weiskrantz L. 2002. Fear recognition in the voice is modulated by unconsciously recognized facial expressions but not by unconsciously recognized affective pictures. Proc Natl Acad Sci USA. 99:4121-4126
17. Dehaene S, Changeux JP, Naccache L, Sackur J, Sergent C. 2006. Conscious, preconscious, and subliminal processing: A testable taxonomy. Trends Cogn Sci. 10:204-211
18. Del Cul A, Baillet S, Dehaene S. 2007. Brain dynamics underlying the nonlinear threshold for access to consciousness. PLoS Biol. 5:2408-2423
19. Dimberg U, Thunberg M, Elmehed K. 2000. Unconscious facial reactions to emotional facial expressions. Psychol Sci. 11:86-89
20. Doehrmann O, Weigelt S, Altmann CF, Kaiser J, Naumer MJ. 2010. Audiovisual functional magnetic resonance imaging adaptation reveals multisensory integration effects in object-related sensory cortices. J Neurosci. 30:3370-3379
21. Doi H, Amamoto T, Okishige Y, Kato M, Shinohara K. 2010. The own-sex effect in facial expression recognition. NeuroReport. 21:564-568
22. Dolan RJ, Morris JS, de Gelder B. 2001. Crossmodal binding of fear in voice and face. Proc Natl Acad Sci USA. 98:10006-10010
23. Driver J, Noesselt T. 2008. Multisensory interplay reveals crossmodal influences on 'Sensory-Specific' brain regions, neural responses, and judgments. Neuron. 57:11-23
24. Eimer M, Kiss M, Holmes A. 2008. Links between rapid ERP responses to fearful faces and conscious awareness. J Neuropsychol. 2:165-181
25. Ethofer T, Anders S, Erb M, Droll C, Royen L, Saur R, Reiterer S, Grodd W, Wildgruber D. 2006. Impact of voice on emotional judgment of faces: an event-related fMRI study. Hum Brain Mapp. 27:707-714
26. Föcker J, Gondan M, Röder B. 2011. Preattentive processing of audiovisual emotional signals. Acta Psychol (Amst). 137:36-47
27. Fuchs M, Kastner J, Wagner M, Hawes S, Ebersole JS. 2002. A standardized boundary element method volume conductor model. Neurophysiol Clin. 113:702-712
28. Giard MH, Peronnet F. 1999. Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. J Cogn Neurosci. 11:473-490
29. Grill-Spector K, Knouf N, Kanwisher N. 2004. The fusiform face area subserves face perception, not generic within-category identification. Nat Neurosci. 7:555-562
30. Hamann S. 2003. Nosing in on the emotional brain. Nat Neurosci. 6:106-108
31. Holmes A, Green S, Vuilleumier P. 2005. The involvement of distinct visual channels in rapid attention towards fearful facial expressions. Cogn Emot. 19:899-922
32. Honey C, Kirchner H, VanRullen R. 2008. Faces in the cloud: Fourier power spectrum biases ultrarapid face detection. J Vis. 8:9
33. Jessen S, Kotz SA. 2011. The temporal dynamics of processing emotions from vocal, facial, and bodily expressions. NeuroImage. 58:665-674
34. Jessen S, Obleser J, Kotz SA. 2012. How bodies and voices interact in early emotion perception. PLos ONE. 7:e36070
35. Jiang Y, He S. 2006. Cortical responses to invisible faces: dissociating subsystems for facial-information processing. Curr Biol. 16:2023-2029
36. Joassin F, Maurage P, Bruyer R, Crommelinck M, Campanella S. 2004. When audition alters vision: an event-related potential study of the cross-modal interactions between faces and voices. Neurosci Lett. 369:132-137
37. Kayser C, Logothetis NK. 2007. Do early sensory cortices integrate cross-modal information? Brain Struct Funct. 212:121-132
38. Killgore WDS, Yurgelun-Todd DA. 2004. Activation of the amygdala and anterior cingulate during nonconscious processing of sad versus happy faces. NeuroImage. 21:1215-1223
39. Kiss M, Eimer M. 2008. ERPs reveal subliminal processing of fearful faces. Psychophysiol. 45:318-326
40. Klasen M, Chen Y-H, Mathiak K. 2012. Multisensory emotions: perception, combination and underlying neural processes. Rev Neurosci. 23:381-392
41. Klasen M, Kenworthy CA, Mathiak KA, Kircher TTJ, Mathiak K. 2011. Supramodal representation of emotions. J Neurosci. 31: 13635-13643
42. Koenig T, Kottlow M, Stein M, Melie-García L. 2011. Ragu: A free tool for the analysis of EEG and MEG event-related scalp field data using global randomization statistics. Comput Intell Neurosci. 2011:938925
43. Koenig T, Melie-García L. 2009. Statistical analysis of multichannel scalp field data. In: Michel CM, Koening T, Brandeis D, Gianotti LRR, Wackermann, editors. Electrical Neuroimaging. 1st ed. Cambridge: The Cambridge University Press. p. 169-190
44. Kreifelts B, Ethofer T, Grodd W, Erb M, Wildgruber D. 2007. Audiovisual integration of emotional signals in voice and face: an event-related fMRI study. NeuroImage. 37:1445-1456
45. Kujala T, Alho K, Huotilainen M, Ilmoniemi RJ, Lehtokoski A, Leinonen A, Rinne T, Salonen Oc, Sinkkonen J, Standertskjöld-Nordenstam CG et al. 1997. Electrophysiological evidence for cross-modal plasticity in humans with early-and late-onset blindness. Psychophysiol. 34:213-216
46. Lamme VAF, Roelfsema PR. 2000. The distinct modes of vision offered by feedforward and recurrent processing. Trends Neurosci. 23:571-579
47. Lang PJ, Bradley MM. 2010. Emotion and the motivational brain. Biol Psychol. 84:437-450
48. LeDoux J. 2003. The emotional brain, fear, and the amygdala. Cell Mol Neurobiol. 23:727-738
49. Lehmann D, Skrandies W. 1980. Reference-free identification of components of checkerboard-evoked multichannel potential fields. Electroencephalogr Clin Neurophysiol. 48:609-621
50. Levy I, Hasson U, Avidan G, Hendler T, Malach R. 2001. Centerperiphery organization of human object areas. Nat Neurosci. 4:533-539
51. Lewis R, Noppeney U. 2010. Audiovisual synchrony improves motion discrimination via enhanced connectivity between early visual and auditory areas. J Neurosci. 30:12329-12339
52. Liddell BJ, Brown KJ, Kemp AH, Barton MJ, Das P, Peduto A, Gordon E, Williams LM. 2005. A direct brainstem-amygdala-cortical 'alarm' system for subliminal signals of fear. NeuroImage. 24:235-243
53. Liddell BJ, Williams LM, Rathjen J, Shevrin H, Gordon E. 2004. A temporal dissociation of subliminal versus supraliminal fear perception: an event-related potential study. J Cogn Neurosci. 16:479-486
54. Li W, Zinbarg RE, Boehm SG, Paller KA. 2008. Neural and behavioral evidence for affective priming from unconsciously perceived emotional facial expressions and the influence of trait anxiety. J Cogn Neurosci. 20:95-107
55. Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, Kennedy WA, Ledden PJ, Brady TJ, Ronsen BR, Tootell RBH. 1995. Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc Natl Acad Sci USA. 92:8135-8139
56. Maris E, Oostenveld R. 2007. Nonparametric statistical testing of EEGand MEG-data. J Neurosci Methods. 164:177-190
57. Massaro DW, Egan PB. 1996. Perceiving affect from the voice and the face. Psychon Bull Rev. 3:215-221
58. Mazziotta J, Toga A, Evans A, Fox P, Lancaster J, Zilles K, Woods R, Paus T, Simpson G, Pike B et al. 2001. A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM). Philos Trans R Soc Lond B Biol Sci. 356:1293-1322
59. Merikle PM, Smilek D, Eastwood JD. 2001. Perception without awareness: perspectives from cognitive psychology. Cognition. 79: 115-134
60. Moradi F, Koch C, Shimojo S. 2005. Face adaptation depends on seeing the face. Neuron. 45:169-175
61. Morris JP, Pelphrey KA, McCarthy G. 2007. Face processing without awareness in the right fusiform gyrus. Neuropsychologia. 45: 3087-3091
62. Morris JS, Öhman A, Dolan RJ. 1999. A subcortical pathway to the right amygdala mediating "unseen" fear. Proc Natl Acad Sci USA. 96:1680-1685
63. Müller VI, Cieslik EC, Turetsky BI, Eickhoff SB. 2012. Crossmodal interactions in audiovisual emotion processing. NeuroImage. 60: 553-561
64. Müller VI, Habel U, Derntl B, Schneider F, Zilles K, Turetsky BI, Eickhoff SB. 2011. Incongruence effects in crossmodal emotional integration. NeuroImage. 54:2257-2266
65. Murray MM, Brunet D, Michel CM. 2008. Topographic ERP analyses: A step-by-step tutorial review. Brain Topogr. 20:249-264
66. Noesselt T, Rieger JW, Schoenfeld MA, Kanowski M, Hinrichs H, Heinze HJ, Driver J. 2007. Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J Neurosci. 27:11431-11441
67. Öhman A, Flykt A, Esteves F. 2001. Emotion drives attention: detecting the snake in the grass. J Exp Psychol Gen. 130:466-478
68. Olofsson JK, Polich J. 2007. Affective visual event-related potentials: arousal, repetition, and time-on-task. Biol Psychol. 75:101-108
69. Oostenveld R, Fries P, Maris E, Schoffelen JM. 2011. FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput Intell Neurosci. 2011:156869
70. Pascual-Marqui RD. 2002. Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. Methods Find Exp Clin Pharmacol. 24:5-12
71. Paulmann S, Jessen S, Kotz SA. 2009. Investigating the multimodal nature of human communication: insights from ERPs. J Psychophysiol. 23:63-76
72. Pegna AJ, Landis T, Khateb A. 2008. Electrophysiological evidence for early non-conscious processing of fearful facial expressions. Int J Psychophysiol. 70:127-136
73. Pessoa L, Japee S, Sturman D, Ungerleider LG. 2006. Target visibility and visual awareness modulate amygdala responses to fearful faces. Cereb Cortex. 16:366-375
74. Pessoa L, McKenna M, Gutierrez E, Ungerleider LG. 2002. Neural processing of emotional faces requires attention. Proc Natl Acad Sci USA. 99:1458-11463
75. Phelps EA, LeDoux JE. 2005. Contributions of the amygdala to emotion processing: from animal models to human behavior. Neuron. 48:175-187
76. Pourtois G, Debatisse D, Despland P-A, de Gelder B. 2002. Facial expressions modulate the time course of long latency auditory brain potentials. Cogn Brain Res. 14:99-105
77. Pourtois G, de Gelder B, Vroomen J, Rossion B, Crommelinck M. 2000. The time-course of intermodal binding between seeing and hearing affective information. NeuroReport. 11:1329-1333
78. Pourtois G, Schwartz S, Seghier ML, Lazeyras F, Vuilleumier P. 2006. Neural systems for orienting attention to the location of threat signals: an event-related fMRI study. NeuroImage. 31:920-933
79. Pourtois G, Thut G, De Peralta RG, Michel C, Vuilleumier P. 2005. Two electrophysiological stages of spatial orienting towards fearful faces: early temporo-parietal activation preceding gain control in extrastriate visual cortex. NeuroImage. 26:149-163
80. Puce A, Epling JA, Thompson JC, Carrick OK. 2007. Neural responses elicited to face motion and vocalization pairings. Neuropsychologia. 45:93-106
81. Robins DL, Hunyadi E, Schultz RT. 2009. Superior temporal activation in response to dynamic audio-visual emotional cues. Brain Cogn. 69:269-278
82. Rossignol M, Philippot P, Douilliez C, Crommelinck M, Campanella S. 2005. The perception of fearful and happy facial expression is modulated by anxiety: an event-related potential study. Neurosci Lett. 377:115-120
83. Sabatinelli D, Bradley MM, Lang PJ, Costa VD, Versace F. 2007. Pleasure rather than salience activates human nucleus accumbens and medial prefrontal cortex. J Neurophysiol. 98:1374-1379
84. Schroeder CE, Foxe J. 2005. Multisensory contributions to low-level, 'unisensory' processing. Curr Opin Neurobiol. 15:454-458
85. Schulz E, Maurer U, van der Mark S, Bucher K, Brem S, Martin E, Brandeis D. 2008. Impaired semantic processing during sentence reading in children with dyslexia: combined fMRI and ERP evidence. NeuroImage. 41:153-168
86. SchuppHT, Stockburger J, CodispotiM, JunghöferM, WeikeAI, HammAO. 2007. Selective visual attention to emotion. J Neurosci. 27:1082-1089
87. Shah R, Lewis MB. 2003. Locating the neutral expression in the facial-emotion space. Vis Cogn. 10:549-566
88. Shimizu H, Imae K. 1981. Nihongoban state-trait anxiety inventory no Sakusei. Jpn J Educ Psychol. 29:348-353 (in Japanese)
89. Smith E, Weinberg A, Moran T, Hajcak G. 2013. Electrocortical responses to NIMSTIM facial expressions of emotion. Int J Psychophysiol. 88:17-25
90. Smith ML. 2012. Rapid processing of emotional expressions without conscious awareness. Cereb Cortex. 22:1748-1760
91. Snodgrass M, Shevrin H. 2006. Unconscious inhibition and facilitation at the objective detection threshold: replicable and qualitatively different unconscious perceptual effects. Cognition. 101:43-79
92. Spielberger CD, Gorsuch RL, Luchene RE. 1970. Manual for the State-Trait Anxiety Inventory (Self-Evaluation Questionnaire). Palo Alto California: Consulting Psychologists Press
93. Stekelenburg JJ, Vroomen J. 2007. Neural correlates of multisensory integration of ecologically valid audiovisual events. J Cogn Neurosci. 19:1964-1973
94. Stienen BMC, Tanaka A, de Gelder B. 2011. Emotional voice and emotional body postures influence each other independently of visual awareness. PLoS ONE. 6:e25517
95. Tamietto M, Castelli L, Vighetti S, Perozzo P, Geminiani G, Weiskrantzf L, de Gelder B. 2009. Unseen facial and bodily expressions trigger fast emotional reactions. Proc Natl Acad Sci USA. 106:17661-17666
96. Tamietto M, de Gelder B. 2010. Neural bases of the non-conscious perception of emotional signals. Nat Rev Neurosci. 11:697-709
97. Taylor MJ. 2002. Non-spatial attentional effects on P1. Clin Neurophysiol. 113:1903-1908
98. Tsuchiya N, Koch C. 2005. Continuous flash suppression reduces negative afterimages. Nat Neurosci. 8:1096-1101
99. Tsuchiya N, Koch C, Gilroy LA, Blake R. 2006. Depth of interocular suppression associated with continuous flash suppression, flash suppression, and binocular rivalry. J Vis. 6:6
100. Vlamings PHJM, Goffaux V, Kemner C. 2009. Is the early modulation of brain activity by fearful facial expressions primarily mediated by coarse low spatial frequency information? J Vis. 9:12
101. Voss P, Gougoux F, Lassonde M, Zatorre RJ, Lepore F. 2006. A positron emission tomography study during auditory localization by late-onset blind individuals. NeuroReport. 17:383-388
102. Vroomen J, Driver J, de Gelder B. 2001. Is cross-modal integration of emotional expressions independent of attentional resources? Cogn Affect Behav Neurosci. 1:382-387
103. Vuilleumier P. 2005. How brains beware: neural mechanisms of emotional attention. Trends Cogn Sci. 9:585-594
104. Vuilleumier P, Schwartz S. 2001. Emotional facial expressions capture attention. Neurology. 56:153-158
105. Willenbockel V, Sadr J, Fiset D, Horne GO, Gosselin F, Tanaka JW. 2010. Controlling low-level image properties: the SHINE toolbox. Behav Res Methods. 42:671-684
106. Williams MA, Morris AP, McGlone F, Abbott DF, Mattingley JB. 2004. Amygdala responses to fearful and happy facial expressions under conditions of binocular suppression. J Neurosci. 24:2898-2904
107. Winston JS, Vuilleumier P, Dolan RJ. 2003. Effects of low-spatial frequency components of fearful faces on fusiform cortex activity. Curr Biol. 13:1824-1829
108. Yang E, Zald D H, Blake R. 2007. Fearful expressions gain preferential access to awareness during continuous flash suppression. Emotion. 7:882-886.