Preferential decoding of emotion from human non-linguistic vocalizations versus speech prosody

Biological Psychology

Volumn 111, Issue , 2015, Pages 14-25

  Pell, M D ^a,b   Rothermich, K ^a   Liu, P ^a   Paulmann, S ^c   Sethi, S ^a   Rigoulot, S ^b  

^a MCGILL UNIVERSITY   (Canada)

^b International Laboratory for Brain Music and Sound (BRAMS)   (Canada)

^c UNIVERSITY OF ESSEX   (United Kingdom)

Author keywords

Anxiety; Emotional communication; ERPs; Speech prosody; Vocal expression

Indexed keywords

ADULT; ANGER; ARTICLE; CONTROLLED STUDY; ELECTROENCEPHALOGRAPHY; EMOTION; EMOTIONALITY; EVENT RELATED POTENTIAL; EXTRAVERSION; FEMALE; HAPPINESS; HUMAN; HUMAN EXPERIMENT; LINGUISTICS; MALE; NEUROPHYSIOLOGY; NEUROSIS; PRIORITY JOURNAL; SADNESS; SPEECH PROSODY; VOCALIZATION; BRAIN; EVOKED RESPONSE; HEARING; PHYSIOLOGY; VERBAL BEHAVIOR; VOICE; YOUNG ADULT;

ADULT; AUDITORY PERCEPTION; BRAIN; ELECTROENCEPHALOGRAPHY; EMOTIONS; EVOKED POTENTIALS; FEMALE; HUMANS; LINGUISTICS; MALE; VERBAL BEHAVIOR; VOICE; YOUNG ADULT;

EID: 84942066200     PISSN: 03010511     EISSN: 18736246     Source Type: Journal    
DOI: 10.1016/j.biopsycho.2015.08.008     Document Type: Article

References (92)

1. Bachorowski J.A., Owren M.J. Not all laughs are alike: voiced but not unvoiced laughter readily elicits positive affect. Psychological Science: A Journal of the American Psychological Society/APS 2001, 12:252-257. 10.1111/1467-9280.00346.
2. Banse R., Scherer K.R. Acoustic profiles in vocal emotion expression. Journal of Personality and Social Psychology 1996, 70(3):614-636.
3. Bar-Haim Y., Lamy D., Pergamin L., Bakermans-Kranenburg M.J., van IJzendoorn M.H. Threat-related attentional bias in anxious and nonanxious individuals: a meta-analytic study. Psychol. Bull. 2007, 133(1):1-24. 10.1037/0033-2909.133.1.1.
4. Beauchemin M., De Beaumont L., Vannasing P., Turcotte A., Arcand C., Belin P., Lassonde M. Electrophysiological markers of voice familiarity. European Journal of Neuroscience 2006, 23(April):3081-3086. 10.1111/j.1460- 9568.2006.04856.x.
5. Belin P., Fillion- Bilodeau S., Gosselin F. The montreal affective voices: a validated set of nonverbal affect bursts for research on auditory affective processing. Behavior Research Methods 2008, 40(2):531-539. 10.3758/BRM.40.2.531.
6. Belin P., Zatorre R.J., Lafaille P., Ahad P., Pike B. Voice-selective areas in human auditory cortex. Nature 2000, 403:309-312. 10.1038/35002078.
7. Bolinger D. Intonation across languages. Universals of human language. Phonology 1978, Vol. 2:471-524. Stanford University Press, Stanford.
8. Bostanov V., Kotchoubey B. Recognition of affective prosody: continuous wavelet measures of event-related brain potentials to emotional exclamations. Psychophysiology 2004, 41:259-268. 10.1111/j.1469-8986.2003.00142.x.
9. Brosch T., Grandjean D., Sander D., Scherer K.R. Cross-modal emotional attention: emotional voices modulate early stages of visual processing. Journal of Cognitive Neuroscience 2008, 21:1670-1679. 10.1162/jocn.2009.21110.
10. Brück C., Kreifelts B., Kaza E., Lotze M., Wildgruber D. Impact of personality on the cerebral processing of emotional prosody. Neuroimage 2011, 58(1):259-268. 10.1016/j.neuroimage.2011.06.005.
11. Bryant G.A., Aktipis C.A. The animal nature of spontaneous human laughter. Evolution and Human Behavior 2014, 35(4):327-335. 10.1016/j.evolhumbehav.2014.03.003.
12. Charest I., Pernet C.R., Rousselet G.A., Quiñones I., Latinus M., Fillion- Bilodeau S., et al. Electrophysiological evidence for an early processing of human voices. BMC Neuroscience 2009, 10:127. 10.1186/1471-2202-10-127.
13. Costa P.T., McCrae R.R. Bridging the gap with the five-factor model. Personality Disorders: Theory, Research, and Treatment 2010, 1(2):127-130. 10.1037/a0020264.
14. Crystal D. Prosodic systems and intonation in English 1969, Cambridge University Press, London.
15. Delorme A., Makeig S. EEGLAB: an open source toolbox for analysis of single- trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods 2004, 134:9-21. 10.1016/j.jneumeth.2003.10.009.
16. Fecteau S., Armony J.L., Joanette Y., Belin P. Priming of non-speech vocalizations in male adults: the influence of the speaker's gender. Brain Cognition 2004, 55:300-302. 10.1016/j.bandc.2004.02.024.
17. Fecteau S., Belin P., Joanette Y., Armony J.L. Amygdala responses to nonlinguistic emotional vocalizations. Neuroimage 2007, 36(2):480-487. 10.1016/j.neuroimage.2007.02.043.
18. Formisano E., De Martino F., Bonte M., Goebel R. Who is saying what? Brain-based decoding of human voice and speech. Science 2008, 322(7 November):970-973.
19. Frühholz S., Grandjean D. Towards a fronto-temporal neural network for the decoding of angry vocal expressions. Neuroimage 2012, 62(3):1658-1666. 10.1016/j.neuroimage.2012.06.015.
20. Frühholz S., Trost W., Grandjean D. The role of the medial temporal limbic system in processing emotions in voice and music. Progress in Neurobiology 2014, 123:1-17. 10.1016/j.pneurobio.2014.09.003.
21. Gädeke J.C., Föcker J., Röder B. Is the processing of affective prosody influenced by spatial attention? An ERP study. BMC Neuroscience 2013, 14:14. 10.1186/1471-2202-14-14.
22. Hawk S.T., van Kleef G.A., Fischer A.H., van der Schalk J. Worth a thousand words: absolute and relative decoding of nonlinguistic affect vocalizations. Emotion 2009, 9(3):293-305. 10.1037/a0015178.
23. Hinojosa J.A., Carretié L., Méndez-Bértolo C., Míguez A., Pozo M.A. Arousal contributions to affective priming: electrophysiological correlates. Emotion 2009, 9(2):164-171. 10.1037/a0014680.
24. Ishii Y., Morita K., Shouji Y., Nakashima Y., Uchimura N. Effects of emotionally charged sounds in schizophrenia patients using exploratory eye movements: comparison with healthy subjects. Psychiatry Clinical Neurosciences 2010, 64:10-18. 10.1111/j.1440-1819.2009.02043.x.
25. Jablonka E., Ginsburg S., Dor D. The co-evolution of language and emotions. Philosophical Transactions of the Royal Society B: Biological Sciences 2012, 367:2152-2159. 10.1098/rstb.2012.0117.
26. Jaywant A., Pell M.D. Categorical processing of negative emotions from speech prosody. Speech Communication 2012, 54(1):1-10.
27. Jessen S., Kotz S.A. The temporal dynamics of processing emotions from vocal, facial, and bodily expressions. NeuroImage 2011, 58(2):665-674. 10.1016/j.neuroimage.2011.06.035.
28. Jiang X., Paulmann S., Robin J., Pell M.D. More than accuracy: nonverbal dialects modulate the time course of vocal emotion recognition across cultures. Journal of Experimental Psychology: Human Perception and Performance 2015, 41(3):597-612. http://dx.doi.org/10.1037/xhp0000043.
29. Jiang X., Pell M.D. On how the brain decodes vocal cues about speaker confidence. Cortex 2015, 66:9-34. http://dx.doi.org/10.1016/j.cortex.2015.02.002.
30. Juslin P.N., Laukka P. Communication of emotions in vocal expression and music performance: different channels, same code?. Psychological Bulletin 2003, 129(5):770-814. 10.1037/0033- 2909.129.5.770.
31. Kaganovich N., Kim J., Herring C., Schumaker J., Macpherson M., Weber- Fox C. Musicians show general enhancement of complex sound encoding and better inhibition of irrelevant auditory change in music: an ERP study. European Journal of Neuroscience 2013, 37:1295-1307. 10.1111/ejn.12110.
32. Kanske P., Kotz S.A. Concreteness in emotional words: ERP evidence from a hemifield study. Brain Research 2007, 1148:138-148. 10.1016/j.brainres.2007.02.044.
33. Koster E.H.W., Crombez G., Van Damme S., Verschuere B., De Houwer J. Does imminent threat capture and hold attention?. Emotion (Washington, D. C.) 2004, 4(3):312-317. 10.1037/1528- 3542.4.3.312.
34. Koster E.H.W., Crombez G., Verschuere B., Van Damme S., Wiersema J.R. Components of attentional bias to threat in high trait anxiety: facilitated engagement, impaired disengagement, and attentional avoidance. Behaviour Research and Therapy 2006, 44:1757-1771. 10.1016/j.brat.2005.12.011.
35. Kotz S.A., Paulmann S. Emotion, language, and the brain. Language and Linguistics Compass 2011, 5(3):108-125.
36. Kreifelts B., Brück C., Ritter J., Ethofer T., Domin M., Lotze M., et al. They are laughing at me: cerebral mediation of cognitive biases in social anxiety. PLoS One 2014, 9(6). 10.1371/journal.pone.0099815.
37. Lima C.F., Castro S.L., Scott S.K. When voices get emotional: a corpus of nonverbal vocalizations for research on emotion processing. Behavior Research Methods 2013, 45:1234-1245. 10.3758/s13428-013-0324-3.
38. Liu P., Pell M.D. Recognizing vocal emotions in Mandarin Chinese: a validated database of Chinese vocal emotional stimuli. Behavior Research Methods 2012, 1042-1051. 10.3758/s13428-012-0203-3.
39. Liu P., Rigoulot S., Pell M.D. Culture modulates the brain response to human expressions of emotion: electrophysiological evidence. Neuropsychologia 2015, 67:1-13.
40. Liu T., Pinheiro A.P., Deng G., Nestor P.G., McCarley R.W., Niznikiewicz M.A. Electrophysiological insights into processing nonverbal emotional vocalizations. Neuroreport 2012, 23:108-112. 10.1097/WNR.0b013e32834ea757.
41. Martin-Soelch C., Stöcklin M., Dammann G., Opwis K., Seifritz E. Anxiety trait modulates psychophysiological reactions, but not habituation processes related to affective auditory stimuli. International Journal of Psychophysiology 2006, 61:87-97. 10.1016/j.ijpsycho.2005.07.009.
42. Meyer M., Elmer S., Baumann S., Jancke L. Short-term plasticity in the auditory system: differential neural responses to perception and imagery of speech and music. Restorative Neurology and Neuroscience 2007, 25:411-431.
43. Mogg K., Bradley B.P. A cognitive-motivational analysis of anxiety. Behavior Research and Therapy 1998, 36:809-848. 10.1016/S0005-7967(98)63- 1.
44. Mogg K., Philippot P., Bradley B.P. Selective attention to angry faces in clinical social phobia. Journal of Abnormal Psychology 2004, 113(1):160-165. 10.1037/0021-843X.113.1.160.
45. Näätänen R., Picton T. The N1 wave of the human electric and magnetic response to sound: a review and an analysis of the component structure. Psychophysiology 1987, 24(4):375-423.
46. Obleser J., Kotz S.A. Multiple brain signatures of integration in the comprehension of degraded speech. Neuroimage 2011, 55(2):713-723.
47. Öhman A. The psychophysiology of emotion: an evolutionary-cognitive perspective. Advances in Psychophysiology 1987, 2:79-127.
48. Owren M.J., Amoss R.T., Rendall D. Two organizing principles of vocal production: implications for nonhuman and human primates. American Journal of Primatology 2011, 73:530-544. 10.1002/ajp.20913.
49. Paulmann S., Bleichner M., Kotz S.A. Valence, arousal, and task effects in emotional prosody processing. Frontiers in Psychology 2013, 4(June):345. 10.3389/fpsyg.2013.00345.
50. Paulmann S., Kotz S.A. Early emotional prosody perception based on different speaker voices. Neuroreport 2008, 19:209-213.
51. Paulmann S., Ott D.V.M., Kotz S.A. Emotional speech perception unfolding in time: the role of the basal ganglia. PLoS One 2011, 6(3):13-17. 10.1371/journal.pone.0017694.
52. Paulmann S., Pell M.D. Contextual influences of emotional speech prosody on face processing: how much is enough?. Cognitive, Affective & Behavioral Neuroscience 2010, 10(2):230-242. 10.3758/CABN.10.2.230.
53. Paulmann S., Titone D., Pell M.D. How emotional prosody guides your way: evidence from eye movements. Speech Communication 2012, 54(1):92-107. 10.1016/j.specom.2011.07.004.
54. Pell M.D. Evaluation of nonverbal emotion in face and voice: some preliminary findings on a new battery of tests. Brain Cognition 2002, 48:499-504.
55. Pell M.D. Nonverbal emotion priming: evidence from the facial affect decision task. Journal of Nonverbal Behavior 2005, 29(1):45-73. 10.1007/s10919-004-0889-8.
56. Pell M.D. Prosody-face interactions in emotional processing as revealed by the facial affect decision task. Journal of Nonverbal Behavior 2005, 29(4):193-215. 10.1007/s10919-005-7720-z.
57. Pell M.D., Kotz S.A. On the time course of vocal emotion recognition. PLoS One 2011, 6(11). 10.1371/journal.pone.0027256.
58. Pell M.D., Paulmann S., Dara C., Alasseri A., Kotz S.A. Factors in the recognition of vocally expressed emotions: a comparison of four languages. Journal of Phonetics 2009, 37:417-435. 10.1016/j.wocn.2009.07.005.
59. Pell M.D., Skorup V. Implicit processing of emotional prosody in a foreign versus native language. Speech Communication 2008, 50:519-530. 10.1016/j.specom.2008.03.006.
60. Peschard V., Maurage P., Philippot P. Towards a cross-modal perspective of emotional perception in social anxiety: review and future directions. Frontiers in Human Neuroscience 2014, 8(May):322. 10.3389/fnhum.2014.00322.
61. Phillips M.L., Young A.W., Scott S.K., Calder A.J., Andrew C., Giampietro V., Gray J.A. Neural responses to facial and vocal expressions of fear and disgust. Proceedings, Biological Sciences/The Royal Society 1998, 265(July):1809-1817. 10.1098/rspb.1998.0506.
62. Quadflieg S., Wendt B., Mohr A., Miltner W.H.R., Straube T. Recognition and evaluation of emotional prosody in individuals with generalized social phobia: a pilot study. Behaviour Research and Therapy 2007, 45:3096-3103. 10.1016/j.brat.2007.08.003.
63. Rigoulot S., Fish K., Pell M.D. Neural correlates of inferring speaker sincerity from white lies: an event-related potential source localization study. Brain Research 2014, 1565:48-62. 10.1016/j.brainres.2014.04.022.
64. Rigoulot S., Pell M.D. Emotion in the voice influences the way we scan emotional faces. Speech Communication 2014, 65(June):36-49. 10.1016/j.specom.2014.05.006.
65. Rigoulot S., Pell M.D., Armony J.L. Time course of the influence of musical expertise on the processing of vocal and musical sounds. Neuroscience 2015, 290:175-184. 10.1016/j.neuroscience.2015.01.033.
66. Rigoulot S., Wassiliwizky E., Pell M.D. Feeling backwards? How temporal order in speech affects the time course of vocal emotion recognition. Frontiers in Psychology 2013, 4(June):367. 10.3389/fpsyg.2013.00367.
67. Sander D., Grandjean D., Pourtois G., Schwartz S., Seghier M.L., Scherer K.R., et al. Emotion and attention interactions in social cognition: brain regions involved in processing anger prosody. Neuroimage 2005, 28:848-858. 10.1016/j.neuroimage.2005.06.023.
68. Sander K., Scheich H. Left auditory cortex and amygdala, but right insula dominance for human laughing and crying. Journal of Cognitive Neuroscience 2005, 17:1519-1531. 10.1162/089892905774597227.
69. Sauter D.A., Eimer M. Rapid detection of emotion from human vocalizations. Journal of Cognitive Neuroscience 2009, 22:474-481. 10.1162/jocn.2009.21215.
70. Scherer K.R. Vocal affect expression: a review and a model for future research. Psychological Bulletin 1986, 99(2):143-165. 10.1037/0033-2909.99.2.143.
71. Scherer K.R. Affect bursts 1994, 161-193. Erlbaum, Hillsdale. S.H.M. van Goozen, N.E. van de Poll, J.A. Sergeant (Eds.).
72. Scherer K.R., Banse R., Wallbott H.G. Emotion Inferences from vocal expression correlate across languages and cultures. Journal of Cross-Cultural Psychology 2001, 32:76-92. 10.1177/0022022101032001009.
73. Scherer K.R., Banse R., Wallbott H.G., Goldbeck T. Vocal cues in emotion encoding and decoding. Motivation and Emotion 1991, 15(2):123-148. 10.1007/BF00995674.
74. Schirmer A., Chen C.-B., Ching A., Tan L., Hong R.Y. Vocal emotions influence verbal memory: neural correlates and interindividual differences. Cognitive, Affective & Behavioral Neuroscience 2013, 13:80-93. 10.3758/s13415-012-0132-8.
75. Schirmer A., Escoffier N. Emotional MMN: anxiety and heart rate correlate with the ERP signature for auditory change detection. Clinical Neurophysiology 2010, 121(1):53-59. 10.1016/j.clinph.2009.09.029.
76. Schirmer A., Fox P.M., Grandjean D. On the spatial organization of sound processing in the human temporal lobe: a meta- analysis. Neuroimage 2012, 63(1):137-147. 10.1016/j.neuroimage.2012.06.025.
77. Schirmer A., Kotz S.A. ERP evidence for a sex-specific Stroop effect in emotional speech. Journal of Cognitive Neuroscience 2003, 15:1135-1148. 10.1162/089892903322598102.
78. Schirmer A., Kotz S.A. Beyond the right hemisphere: Brain mechanisms mediating vocal emotional processing. Trends in Cognitive Science 2006, 10(1):24-30. 10.1016/j.tics.2005.11.009.
79. Schirmer A., Kotz S.A., Friederici A.D. On the role of attention for the processing of emotions in speech: sex differences revisited. Cognitive Brain Research 2005, 24:442-452. 10.1016/j.cogbrainres.2005.02.022.
80. Scott S.K., Young A.W., Calder A.J., Hellawell D.J., Aggleton J.P., Johnson M. Impaired auditory recognition of fear and anger following bilateral amygdala lesions. Nature 1997, 10.1038/385254a0.
81. Siegman A.W., Anderson R.A., Berger T. The angry voice: its effects on the experience of anger and cardiovascular reactivity. Psychosomatic Medicine 1990, 52:631-643.
82. Spielberger C., Jacobs G., Russell S., Crane R. Assessment of anger: the state-trait anger scale. Advances in personality assesment 1983, Vol 2:161-189. Erlbaum, Hillsdale.
83. Stekelenburg J.J., Vroomen J. Neural correlates of multisensory integration of ecologically valid audiovisual events. Journal of Cognitive Neuroscience 2007, 19:1964-1973. 10.1162/jocn.2007.91213.
84. Szameitat D.P., Alter K., Szameitat A.J., Darwin C.J., Wildgruber D., Dietrich S., Sterr A. Differentiation of emotions in laughter at the behavioral level. Emotion (Washington, D. C.) 2009, 9(3):397-405. 10.1037/a0015692.
85. Szameitat D.P., Kreifelts B., Alter K., Szameitat A.J., Sterr A., Grodd W., Wildgruber D. It is not always tickling: distinct cerebral responses during perception of different laughter types. Neuroimage 2010, 53(4):1264-1271. 10.1016/j.neuroimage.2010.06.028.
86. Trouvain J. Laughing, breathing, clicking-the prosody of nonverbal vocalisations. Speech Prosody 2014, 2014:598-602.
87. Ulrich R., Miller J. Using the jackknife-based scoring method for measuring LRP onset effects in factorial designs. Psychophysiology 2001, 38:816-827.
88. Wang X., Kadia S.C. Differential representation of species-specific primate vocalizations in the auditory cortices of marmoset and cat. Journal of Neurophysiology 2001, 86:2616-2620.
89. Wildgruber D., Ethofer T., Grandjean D., Kreifelts B. A cerebral network model of speech prosody comprehension. International Journal of Speech-Language Pathology 2009, 11(4):277-281. 10.1080/17549500902943043.
90. Wildgruber D., Szameitat D.P., Ethofer T., Brück C., Alter K., Grodd W., Kreifelts B. Different types of laughter modulate connectivity within distinct parts of the laughter perception network. PLoS One 2013, 8(5):10-13. 10.1371/journal.pone.0063441.
91. Woldorff M.G., Gallen C., Hampson S.A., Hillyard S.A., Pantev C., Sobel D., et al. Modulation of early sensory processing in human auditory cortex during auditory selective attention. Proceedings of the National Academy of Science 1993, 90:8722-8726.
92. Zvielli A., Bernstein A., Koster E.H.W. Dynamics of attentional bias to threat in anxious adults: bias towards and/or away?. PLoS One 2014, 9(8):e104025. 10.1371/journal.pone.0104025.