Predicting vocal emotion expressions from the human brain

Human Brain Mapping

Volumn 34, Issue 8, 2013, Pages 1971-1981

  Kotz, Sonja A ^a   Kalberlah, Christian ^a,b   Bahlmann, Jörg ^a,c   Friederici, Angela D ^a,d   Haynes, John D ^a,b,d  

^a MAX PLANCK INSTITUTE FOR HUMAN COGNITIVE AND BRAIN SCIENCES   (Germany)

^b CHARITÉ UNIVERSITÄTSMEDIZIN BERLIN   (Germany)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d HUMBOLDT UNIVERSITY   (Germany)

Author keywords

Emotion; FMRI; Multivariate pattern analysis; Prosody; Vocal expressions

Indexed keywords

ADULT; ARTICLE; BRAIN; CONTROLLED STUDY; ELECTROENCEPHALOGRAM; EMOTION; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUNCTIONAL NEUROIMAGING; HUMAN; HUMAN EXPERIMENT; MALE; PRIORITY JOURNAL; TEMPORAL LOBE; VOCALIZATION;

EMOTION; FMRI; MULTIVARIATE PATTERN ANALYSIS; PROSODY; VOCAL EXPRESSIONS;

ADULT; AUDITORY PERCEPTION; BRAIN; BRAIN MAPPING; EMOTIONS; FEMALE; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; SPEECH; SUPPORT VECTOR MACHINES; VOICE; YOUNG ADULT;

EID: 84880134647     PISSN: 10659471     EISSN: 10970193     Source Type: Journal    
DOI: 10.1002/hbm.22041     Document Type: Article

References (83)

1. Ackermann H, Riecker A, Mathiak K, Erb M, Grodd W, Wildgruber D ( 2001): Rate-dependent activation of a prefrontal-insular-cerebellar network during passive listening to trains of click stimuli: An fMRI study. Neuroreport 18: 4087-4092.
2. Adolphs R ( 2002): Neural systems for recognizing emotion. Curr Opin Neurobiol 12: 169-177.
3. Adolphs R, Tranel D, Damasio H ( 2001): Emotion recognition from faces and prosody following temporal lobectomy. Neuropsychology 15: 396-404.
4. Baillieux H, De Smet HJ, Paquier PF, De Deyn PP, Mariën P ( 2008): Cerebellar neurocognition: Insights into the bottom of the brain. Clin Neurol Neurosurg 110: 763-773.
5. Bamiou DE, Musiek FE, Luxon LM ( 2003): The insula (Island of Reil) and its role in auditory processing. Literature review. Brain Res Brain Res Rev 42: 143-154.
6. Banse R, Scherer K ( 1996): Acoustic profiles in vocal emotion expression. J Pers Soc Psychol 70: 614-636.
7. Beaucousin V, Lacheret A, Turbelin MR, Morel M, Mazoyer B, Tzourio-Mazoyer N ( 2007): FMRI study of emotional speech comprehension. Cereb Cortex 17: 339-352.
8. Belin P, Fecteau S, Bédard C ( 2004): Thinking the voice: Neural correlates of voice perception. Trends Cogn Sci 8: 129-135.
9. Borod JC, Bloom RL, Brickman AM, Nakhutina L, Curko EA ( 2002): Emotion processing deficits in individuals with unilateral brain damage. Appl Neuropsychol 9: 23-36.
10. Calder AJ, Lawrence AD, Young AW ( 2001): Neuropsychology of fear and loathing. Nat Neurosci Rev 2: 352-363.
11. Cancelliere AE, Kertesz A ( 1990): Lesion localization in acquired deficits of emotional expression and comprehension. Brain Cogn 13: 133-147.
12. Chandrasekhar PV, Capra CM, Moore S, Noussair C, Berns GS ( 2008): Neurobiological regret and rejoice functions for aversive outcomes. Neuroimage 39: 1472-1484.
13. Craig AD ( 2002): How do you feel? Interoception: The sense of the physiological condition of the body. Nat Rev Neurosci 3: 655-666.
14. Craig AD ( 2004): Human feelings: Why are some more aware than others? Trends Cogn Sci 8: 239-241.
15. Craig AD ( 2005): Forebrain emotional asymmetry: A neuroanatomical basis? Trends Cogn Sci 9: 566-571.
16. Critchley HD ( 2005): Neural mechanisms of autonomic, affective, and cognitive integration. J Comp Neurol 493: 154-166.
17. Critchley H, Daly E, Phillips M, Brammer M, Bullmore E, Williams S, Van Amelsvoort T, Robertson D, David A, Murphy D ( 2000): Explicit and implicit neural mechanisms for processing of social information from facial expressions: A functional magnetic resonance imaging study. Hum Brain Mapp 9: 93-105.
18. Crottaz-Herbette S, Menon V ( 2006): Where and when the anterior cingulate cortex modulates attentional response: Combined fMRI and ERP evidence. J Cogn Neurosci 18: 766-780.
19. Davidson RJ ( 2000): Affective style, psychopathology, and resilience: Brain mechanisms and plasticity. Am Psychol 55: 1196-1214.
20. Davidson R, Irwin W. 1999. The functional neuroanatomy of emotion and affective style. Trends Cogn Sci 3: 11-21.
21. Desai R, Liebenthal E, Waldron E, Binder JF ( 2008): Left posterior temporal regions are sensitive to auditory categorization. J Cogn Neurosci 20: 1174-1188.
22. Ekman P ( 1992): An argument for basic emotions. Cogn Emot 6: 169-200.
23. Ekman P, Friesen WV ( 1976): Measuring facial movement. Environ Psychol Nonverb Behav 1: 56-75.
24. Engelien A, Silbersweig D, Stern E, Huber W, Döring W, Frith C, Frackowiak RS ( 1995): The functional anatomy of recovers from auditory agnosia. A PET study of sound categorization in a neurological patient and normal controls. Brain 118: 1395-1409.
25. Ethofer T, Anders S, Erb M, Herbert C, Wiethoff S, Kissler J, Grodd W, Wildgruber D ( 2006): Cerebral pathways in processing of affective prosody: A dynamic causal modeling study. Neuroimage 30: 580-587.
26. Ethofer T, Van De Ville D, Scherer K, Vuilleumier P ( 2009): Decoding of emotional information in voice-sensitive cortices. Curr Biol 19: 1028-1033.
27. Fecteau S, Belin P, Joanette Y, Armony JL ( 2007): Amygdala responses to nonlinguistic emotional vocalizations. Neuroimage 36: 480-487.
28. Formisano E, De Martino YF, Bonte M, Goebel R ( 2008): "Who" is saying "what"? Brain-based decoding of human voice and speech. Science 322: 970-973.
29. Fusar-Poli P, Placentino A, Carletti F, Landi P, Allen P, Surguladze S, Benedetti F, Abbamonte M, Gasparotti R, Barale F, Perez J, McGuire P, Politi P ( 2009): Functional atlas of emotional faces processing: A voxel-based meta-analysis of 105 functional magnetic resonance imaging studies. J Psychiatry Neurosci 34: 418-432.
30. Gallese V, Keysers C, Rizzolatti G ( 2004): A unifying view of the basis of social cognition. Trends Cogn Sci 8: 396-403.
31. Grandjean D, Sander D, Pourtois G, Schwartz S, Seghier ML, Scherer KR, Vuilleumier P ( 2005): The voices of wrath: Brain responses to angry prosody in meaningless speech. Nat Neurosci 8: 145-146.
32. Griffiths TD, Rees A, Witton C, Cross PM, Shakir RA, Green GG ( 1997): Spatial and temporal auditory processing deficits following right hemisphere infarction. A psychophysical study. Brain 120: 785-794.
33. Habib M, Daquin G, Milandre L, Royere ML, Rey M, Lanteri A, Salamon G, Khalil R ( 1995): Mutism and auditory agnosia due to bilateral insular damage--role of the insula in human communication. Neuropsychologia 33: 327-339.
34. Haynes JD, Rees G ( 2005): Predicting the stream of consciousness from activity in human visual cortex. Curr Biol 15: 1301-1307.
35. Haynes JD, Rees G ( 2006): Decoding mental states from brain activity in humans. Nat Rev Neurosci 7: 523-534.
36. Haynes JD, Sakai K, Rees G, Gilbert S, Frith C, Passingham RE ( 2007): Reading hidden intentions in the human brain. Curr Biol 20: 323-328.
37. Heilman KM, Bowers D, Speedie L, Coslett HB ( 1984): Comprehension of affective and nonaffective prosody. Neurology 34: 917-921.
38. Hickok G, Poeppel D ( 2007): The cortical organization of speech processing. Nat Rev Neurosci 8: 393-402.
39. Imaizumi S, Mori K, Kiritani S, Kawashima R, Sugiura M, Fukuda H, Itoh K, Kato T, Nakamura A, Hatano K, Kojima S, Nakamura K ( 1997): Vocal identification of speaker and emotion activates different brain regions. Neuroreport 8: 2809-2812.
40. Jabbi M, Swart M, Keysers C ( 2007): Empathy for positive and negative emotions in the gustatory cortex. Neuroimage 34: 1744-1753.
41. Kalberlah C, Chen Y, Heinzle J, Haynes JD ( 2011): Beyond topographic representation: Decoding visuospatial attention from local activity patterns in the human frontal cortex. Int J Imag Syst Tech 21: 201-210.
42. Kamitani Y, Tong F ( 2005): Decoding the visual and subjective contents of the human brain. Nat Neurosci 8: 679-685.
43. Kanske P, Kotz SA ( 2011): Emotion speeds up conflict resolution: A new role for the ventral anterior cingulate cortex? Cereb Cortex 21: 911-919.
44. Koelsch S, Fritz T, von Cramon DY, Müller K, Friederici AD ( 2006): Investigating emotion with music: An fMRI study. Hum Brain Mapp 27: 239-250.
45. Kotz SA, Meyer M, Alter K, Besson M, von Cramon DY, Friederici AD ( 2003): On the lateralization of emotional prosody: An event-related functional MR investigation. Brain Lang 86: 366-376.
46. Kotz SA, Meyer M, Paulmann S ( 2006): Lateralization of emotional prosody in the brain: An overview and synopsis on the impact of study design. Prog Brain Res 156: 285-294.
47. Kriegeskorte N, Goebel R, Bandettini P ( 2006): Information-based functional brain mapping. Proc Natl Acad Sci USA 103: 3863-3868.
48. Kriegstein KV, Giraud AL ( 2004): Distinct functional substrates along the right superior temporal sulcus for the processing of voices. Neuroimage 22: 948-955.
49. Lattner S, Meyer ME, Friederici AD ( 2005): Voice perception: Sex, pitch, and the right hemisphere. Hum Brain Mapp 24: 11-20.
50. LeDoux J ( 2000): Emotion circuits in the brain. Annu Rev Neurosci 23: 155-184.
51. Liebenthal E, Binder JR, Spitzer SM, Possing ET, Medler DA ( 2005): Neural substrates of phonemic perception. Cereb Cortex 15: 1621-1631.
52. Meyer M, Alter K, Friederici AD, Lohmann G, von Cramon DY ( 2002): FMRI reveals brain regions mediating slow prosodic modulations in spoken sentences. Hum Brain Mapp 17: 73-88.
53. Morris JS, Scott SK, Dolan RJ ( 1999): Saying it with feeling: Neural responses to emotional vocalizations. Neuropsychologia 37: 1155-1163.
54. Olsson A, Phelps EA ( 2007): Social learning of fear. Nat Neurosci 10: 1095-1102.
55. Paquier PF, Marien P ( 2005): A synthesis of the role of the cerebellum in cognition. Aphasiology 19: 3-19.
56. Peelen MV, Atkinson AP, Vuilleumier P ( 2010): Supramodal representations of perceived emotions in the human brain. J Neurosci 30: 10127-10134.
57. Perry DW, Zatorre RJ, Petrides M, Alivisatos B, Meyer E, Evans AC ( 1999): Localization of cerebral activity during simple singing. Neuroreport 10: 3979-3984.
58. Petkov CI, Kayser C, Steudel T, Whittingstall K, Augath M, Logothetis NK ( 2008): A voice region in the monkey brain. Nat Neurosci 11: 367-374.
59. Pittam J, Scherer KR ( 1993): Vocal expression and communication of emotion. In: Lewis M, Haviland JM, editors. Handbook of Emotion, 3rd ed. New York: Guilford Press. pp 185-197.
60. Polyn SM, Natu VS, Cohen JD, Norman KA ( 2005): Category-specific cortical activity precedes retrieval during memory search. Science 310: 1963-1966.
61. Rauschecker JP ( 2005): Neural encoding and retrieval of sound sequences. Ann N Y Acad Sci 1060: 125-135.
62. Remedios R, Logothetis NK, Kayser C ( 2009): An auditory region in the primate insular cortex responding preferentially to vocal communication sounds. J Neurosci 29: 1034-1045.
63. Riecker A, Ackermann H, Wildgruber D, Dogil G, Grodd W ( 2000): Opposite hemispheric lateralization effects during speaking and singing at motor cortex, insula and cerebellum. Neuroreport 11: 1997-2000.
64. Romanski LM, Goldman-Rakic PS ( 2002): An auditory domain in primate prefrontal cortex. Nat Neurosci 5: 15-16.
65. Romanski LM, Averbeck BB, Diltz M ( 2005): Neural representation of vocalizations in the primate ventrolateral prefrontal cortex. J Neurophysiol 93: 734-747.
66. Russ BE, Ackelson AL, Baker AE, Cohen YE ( 2008): Coding of auditory-stimulus identity in the auditory non-spatial processing stream. J Neurophysiol 99: 87-95.
67. Sander K, Scheich H ( 2005): Left auditory cortex and amygdala, but right insula dominance for human laughing and crying. J Cogn Neurosci 17: 1519-1531.
68. Scherer KR ( 1995): Expression of emotion in voice and music. J Voice 9: 235-248.
69. Schirmer A, Kotz SA ( 2006): Beyond the right hemisphere: Brain mechanisms mediating vocal emotional processing. Trends Cogn Sci 10: 24-30.
70. Schmahmann JD ( 2004): Disorders of the cerebellum: Ataxia, dysmetria of thought, and the cerebellar cognitive affective syndrome. J Neuropsychiatry Clin Neurosci 16: 367-378.
71. Scott SK, Young AW, Calder AJ, Hellawell DJ, Aggleton JP, Johnson M ( 1997): Impaired auditory recognition of fear and anger following bilateral amygdala lesions. Nature 385: 254-257.
72. Singer T ( 2006): The neuronal basis and ontogeny of empathy and mind reading: Review of literature and implications for future research. Neurosci Biobehav Rev 30: 855-863.
73. Small DM, Gitelman MD, Nobre AC, Parish TB, Mesulam MM ( 2003): The posterior cingulate and medial prefrontal cortex mediate the anticipatory allocation of spatial attention. Neuroimage 18: 633-641.
74. Soon CS, Brass M, Heinze HJ, Haynes JD ( 2008): Unconscious determinants of free decisions in the human brain. Nat Neurosci 11: 543-545.
75. Stoodley CJ, Schmahmann JD ( 2010): Evidence for topographic organization in the cerebellum of motor control versus cognitive and affective processing. Cortex 46: 831-844.
76. Tian B, Reser D, Durham A, Kustov A, Rauschecker JP ( 2001): Functional specialization in rhesus monkey auditory cortex. Science 292: 290-293.
77. Wang X ( 2000): On cortical encoding of vocal communication sounds in primates. Proc Natl Acad Sci USA 97: 11843-11849.
78. Wicker B, Keysers C, Plailly J, Royet JP, Gallese V, Rizzolatti G ( 2003): Both of us disgusted in My insula: The common neural basis of seeing and feeling disgust. Neuron 40: 655-664.
79. Wildgruber D, Riecker A, Hertrich I, Erb M, Grodd W, Ethofer T, Ackermann H ( 2005): Identification of emotional intonation evaluated by fMRI. Neuroimage 24: 1233-1241.
80. Wildgruber D, Ackermann H, Kreifelts B, Ethofer T ( 2006): Cerebral processing of linguistic and emotional prosody: fMRI studies. Prog Brain Res 156: 249-268.
81. Wong PC, Parsons LM, Martinez M, Diehl RL ( 2004): The role of the insular cortex in pitch pattern perception: The effect of linguistic contexts. J Neurosci 24: 9153-9160.
82. Worsley KJ, Marrett S, Neelin P, Vandal AC, Friston KJ, Evans AC ( 1996): A unified statistical approach for determining significant signals in images of cerebral activation. Hum Brain Mapp 4: 58-73.
83. Zatorre RJ, Evans AC, Meyer E ( 1994): Neural mechanisms underlying melodic perception and memory for pitch. J Neurosci 14: 1908-1919.