The differential contribution of facial expressions, prosody, and speech content to empathy

Cognition and Emotion

Volumn 26, Issue 6, 2012, Pages 995-1014

  Regenbogen, Christina ^a,b   Schneider, Daniel A ^a,b   Finkelmeyer, Andreas ^b,c   Kohn, Nils ^a,b   Derntl, Birgit ^a,b,d   Kellermann, Thilo ^a,b   Gur, Raquel E ^e   Schneider, Frank ^a,b,e   Habel, Ute ^a,b  

^a RWTH AACHEN UNIVERSITY   (Germany)

^b JARA TRANSLATIONAL BRAIN MEDICINE   (Germany)

^c NEWCASTLE UNIVERSITY   (United Kingdom)

^d UNIVERSITY OF VIENNA   (Austria)

^e UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Empathy; Facial expressions; Galvanic skin conductance; Prosody; Speech content

Indexed keywords

ADULT; ARTICLE; ELECTRODERMAL RESPONSE; EMOTIONALITY; EMPATHY; FACIAL EXPRESSION; FEMALE; HUMAN; HUMAN EXPERIMENT; MALE; NORMAL HUMAN; PRIORITY JOURNAL; SKIN CONDUCTANCE; SPEECH; STORYTELLING;

ACOUSTIC STIMULATION; ADULT; EMOTIONS; EMPATHY; FACIAL EXPRESSION; FEMALE; GALVANIC SKIN RESPONSE; HUMANS; MALE; MIDDLE AGED; PHOTIC STIMULATION; RECOGNITION (PSYCHOLOGY); SPEECH; SPEECH ACOUSTICS; SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 84859432448     PISSN: 02699931     EISSN: 14640600     Source Type: Journal    
DOI: 10.1080/02699931.2011.631296     Document Type: Article

References (77)

1. Adolphs, R. 2002. Recognizing emotion from facial expressions: Psychological and neurological mechanisms. Behavioral and Cognitive Neuroscience Reviews, 1(1): 21-61.
2. Ambadar, Z., Schooler, J. W. and Cohn, J. F. 2005. Deciphering the enigmatic face: The importance of facial dynamics in interpreting subtle facial expressions. Psychological Science, 16(5): 403-410.
3. Argyle, M., Alkema, F. and Gilmour, R. 1971. The communication of friendly and hostile attitudes by verbal and non-verbal signals. European Journal of Social Psychology, 1(3): 385-402.
4. Aschenbrenner, S., Tucha, O. and Lange, K. W. 2000. Regensburger Word Fluency Test [Regensburger Wortflüssigkeits-Test, RWT], Göttingen, Germany: Hogrefe.
5. Bagby, R. M., Taylor, G. J. and Ryan, D. 1986. Toronto Alexithymia Scale-Relationship with personality and psychopathology measures. Psychotherapy and Psychosomatics, 45(4): 207-215.
6. Batson, C. D. 2009. "These things called empathy". In The Social Neuroscience of Empathy, Edited by: Decety, J. and Ickes, W. 3-16. Cambridge, MA: MIT Press.
7. Benedek, M. and Kaernbach, C. 2010. Decomposition of skin conductance data by means of nonnegative deconvolution. Psychophysiology, 47(4): 647-658.
8. Blasko, D. G. and Hall, M. D. 1998. Influence of prosodic boundaries on comprehension of spoken English sentences. Perceptual and Motor Skills, 87(1): 3-18.
9. Bostanov, V. and Kotchoubey, B. 2004. Recognition of affective prosody: Continuous wavelet measures of event-related brain potentials to emotional exclamations. Psychophysiology, 41(2): 259-268.
10. Buck, R., Miller, R. E. and Caul, W. F. 1974. Sex, personality, and physiological variables in the communication of affect via facial expression. Journal of Personality and Social Psychology, 30: 587-596.
11. Buck, R., Savin, V. J., Miller, R. E. and Caul, W. F. 1972. Communication of affect through facial expressions in humans. Journal of Personality and Social Psychology, 23: 362-371.
12. Cahill, L., Gorski, L., Belcher, A. and Huynh, Q. 2004. The influence of sex versus sex-related traits on long-term memory for gist and detail from an emotional story. Consciousness and Cognition, 13(2): 391-400.
13. Campanella, S. and Belin, P. 2007. Integrating face and voice in person perception. Trends in Cognitive Sciences, 11(12): 535-543.
14. Carton, J. S., Kessler, E. A. and Pape, C. L. 1999. Non-verbal decoding skills and relationship well-being in adults. Journal of Nonverbal Behavior, 23(1): 91-100.
15. Codispoti, M., Surcinelli, P. and Baldaro, B. 2008. Watching emotional movies: Affective reactions and gender differences. International Journal of Psychophysiology, 69: 90-95.
16. Collignon, O., Girard, S., Gosselin, F., Roy, S., Saint-Amour, D.Lassonde, M. 2008. Audio-visual integration of emotion expression. Brain Research, 1242: 126-135.
17. Collignon, O., Girard, S., Gosselin, F., Saint-Amour, D., Lepore, F. and Lassonde, M. 2010. Women process multisensory emotion expressions more efficiently than men. Neuropsychologia, 48(1): 220-225.
18. Cutler, A., Dahan, D. and van Donselaar, W. 1997. Prosody in the comprehension of spoken language: A literature review. Language and Speech, 40(2): 141-201.
19. Davis, M. H. 1980. A multidimensional approach to individual differences in empathy. JSAS (Journal Supplement Abstract Service) Catalog of Selected Documents in Psychology, 10: 85
20. de Gelder, B. and Vroomen, J. 2000. The perception of emotions by ear and by eye. Cognition and Emotion, 14: 289-311.
21. de Vignemont, F. and Singer, T. 2006. The empathic brain: How, when and why?. Trends in Cognitive Sciences, 10(10): 435-441.
22. de Waal, F. B. M. 2008. Putting the altruism back into altruism: The evolution of empathy. Annual Review of Psychology, 59: 279-300.
23. Decety, J. and Jackson, P. L. 2004. The functional architecture of human empathy. Behavioral and Cognitive Neuroscience Reviews, 3(2): 71-100.
24. Derntl, B., Finkelmeyer, A., Eickhoff, S., Kellermann, T., Falkenberg, D. I.Schneider, F. 2010. Multidimensional assessment of empathic abilities: Neural correlates and gender differences. Psychoneuroendocrinology, 35(1): 67-82.
25. Derntl, B., Finkelmeyer, A., Toygar, T. K., Hülsmann, A., Schneider, F.Falkenberg, D. I. 2009. Generalized deficit in all core components of empathy in schizophrenia. Schizophrenia Research, 108(1-3): 197-206.
26. Derntl, B., Habel, U., Windischberger, C., Robinson, S., Kryspin-Exner, I.Gur, R. C. 2009. General and specific responsiveness of the amygdala during explicit emotion recognition in females and males. BMC Neuroscience, 10: 91
27. Diekman, A. B. and Eagly, A. 2000. Stereotypes as dynamic constructs: Women and men of the past, present, and future. Personality and Social Psychology Bulletin, 26(10): 1171-1188.
28. Dolan, R. J., Morris, J. S. and de Gelder, B. 2001. Crossmodal binding of fear in voice and face. Proceedings of the National Academy of Sciences, 98(17): 10006-10010.
29. Ekman, P. and Friesen, W. V. 1976. Pictures of Facial Affect, Palo Alto, CA: Consulting Psychologists Press.
30. Etcoff, N. L. and Magee, J. J. 1992. Categorical perception of facial expressions. Cognition, 44(3): 227-240.
31. Ethofer, T., Anders, S., Erb, M., Droll, C., Royen, L.Saur, R. 2006. Impact of voice on emotional judgment of faces: An event-related fMRI study. Human Brain Mapping, 27(9): 707-714.
32. Ethofer, T., Van De Ville, D., Scherer, K. and Vuilleumier, P. 2009. Decoding of emotional information in voice-sensitive cortices. Current Biology, 19(12): 1028-1033.
33. Farrow, T. F., Zheng, Y., Wilkinson, I. D., Spence, S. A., Deakin, J. F.Tarrier, N. 2001. Investigating the functional anatomy of empathy and forgiveness. Neuroreport, 12(11): 2433-2438.
34. Friederici, A. D. 2002. Towards a neural basis of auditory sentence processing. Trends in Cognitive Sciences, 6(2): 78-84.
35. Friederici, A. D. and Alter, K. 2004. Lateralization of auditory language functions: A dynamic dual pathway model. Brain and Language, 89(2): 267-276.
36. Greenwald, A., Poehlman, T. A., Uhlmann, E. L. and Banaji, M. R. 2009. Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity. Journal of Personality and Social Psychology, 97(1): 17-41.
37. Grossman, M. and Wood, W. 1993. Sex differences in intensity of emotional experience: A social role interpretation. Journal of Psychology and Social Psychology, 65: 1010-1022.
38. Habel, U., Windischberger, C., Derntl, B., Robinson, S., Kryspin-Exner, I.Gur, R. C. 2007. Amygdala activation and facial expressions: Explicit emotion discrimination versus implicit emotion processing. Neuropsychologia, 45(10): 2369-2377.
39. Hall, J., Harris, J. M., Sprengelmeyer, R., Sprengelmeyer, A., Young, A. W.Santos, I. M. 2004. Social cognition and face processing in schizophrenia. British Journal of Psychiatry, 185: 169-170.
40. Hammerschmidt, K. and Jürgens, U. 2007. Acoustical correlates of affective prosody. Journal of Voice, 21(5): 531-540.
41. Humphries, C., Love, T., Swinney, D. and Hickok, G. 2005. Response of anterior temporal cortex to syntactic and prosodic manipulations during sentence processing. Human Brain Mapping, 26(2): 128-138.
42. Kilts, C. D., Egan, G., Gideon, D. A., Ely, T. D. and Hoffman, J. M. 2003. Dissociable neural pathways are involved in the recognition of emotion in static and dynamic facial expressions. NeuroImage, 18: 156-168.
43. Koch, K., Pauly, K., Kellermann, T., Seiferth, N. Y., Reske, M.Backes, V. 2007. Gender differences in the cognitive control of emotion: An fMRI study. Neuropsychologia, 45: 2744-2754.
44. Koelewijn, T., Bronkhorst, A. and Theeuwes, J. 2010. Attention and the multiple stages of multisensory integration: A review of audiovisual studies. Acta Psychologica, 134(3): 372-384.
45. Kotz, S. A., Meyer, M., Alter, K., Besson, M., von Cramon, D. Y. and Friederici, A. D. 2003. On the lateralization of emotional prosody: An event-related functional MR investigation. Brain and Language, 86(3): 366-376.
46. Krämer, U. M., Mohammadi, B., Doñamayor, N., Samii, A. and Münte, T. F. 2010. Emotional and cognitive aspects of empathy and their relation to social cognition-An fMRI-study. Brain Research, 1311: 110-120.
47. Kreifelts, B., Ethofer, T., Grodd, W., Erb, M. and Wildgruber, D. 2007. Audiovisual integration of emotional signals in voice and face: An event-related fMRI study. NeuroImage, 37(4): 1445-1456.
48. Kring, A. M. and Gordon, A. H. 1998. Sex differences in emotion: Expression, experience, and physiology. Journal of Personality and Social Psychology, 74(3): 686-703.
49. LaBar, K. S., Crupain, M. J., Voyvodic, J. T. and McCarthy, G. 2003. Dynamic perception of facial affect and identity in the human brain. Cerebral Cortex, 13(10): 1023-1033.
50. Leibetseder, M., Laireiter, A.-R. and Köller, T. 2007. Structural analysis of the E-scale. Personality and Individual Differences, 3: 547-561.
51. Massaro, D. W. 1987. Speech perception by eye and by ear: A paradigm for psychological inquiry, Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
52. Massaro, D. W. and Egan, P. B. 1996. Perceiving affect from the voice and the face. Psychonomic Bulletin and Review, 3: 215-221.
53. Mehrabian, A. 1981. Silent messages: Implicit communication of emotions and attitudes, Belmont, CA: Wadsworth.
54. Nater, U. M., Abbruzzese, E., Krebs, M. and Ehlert, U. 2006. Sex differences in emotional and psychophysiological responses to musical stimuli. International Journal of Psychophysiology, 62(2): 300-308.
55. Nygaard, L. C. and Queen, J. S. 2008. Communicating emotion: Linking affective prosody and word meaning. Journal of Experimental Psychology: Human Perception and Performance, 34(4): 1017-1030.
56. Oldfield, R. C. 1971. The assessment and analysis of handedness: The Edinburgh Inventory. Neuropsychologia, 9(1): 97-113.
57. Parker, J. D., Taylor, G. J. and Bagby, R. M. 2003. The 20-Item Toronto Alexithymia Scale. III. Reliability and factorial validity in a community population. Journal of Psychosomatic Research, 55(3): 269-275.
58. Paulmann, S. and Kotz, S. A. 2008. Early emotional prosody perception based on different speaker voices. Neuroreport, 19(2): 209-213.
59. Peelen, M. V., Atkinson, A. P. and Vuilleumier, P. 2010. Supramodal representations of perceived emotions in the human brain. The Journal of Neuroscience, 30(30): 10127-10134.
60. Penn, D. L., Sanna, L. J. and Roberts, D. L. 2008. Social cognition in schizophrenia: An overview. Schizophrenia Bulletin, 34(3): 408-411.
61. Roman, F., Garcia-Sanchez, F. A., Martinez-Selva, J. M., Gomez-Amor, J. and Carrillo, E. 1989. Sex differences and bilateral electrodermal activity: A replication. The Pavlovian Journal of Biological Science, 24(4): 150-155.
62. Sato, W. S., Kochiyama, T., Yoshikawa, S., Naito, E. and Matsumura, M. 2004. Enhanced neural activity in response to dynamic facial expressions of emotion: An fMRI study. Cognitive Brain Research, 20: 81-91.
63. Schirmer, A. 2010. Mark my words: Tone of voice changes affective word representations in memory. PLoS ONE, 5(2): e9080
64. Schmidt, K. H. and Metzler, P. 1992. Wortschatztest (WST), Weinheim, Germany: Beltz Test GmbH.
65. Seubert, J., Kellermann, T., Loughead, J., Boers, F., Brensinger, C.Schneider, F. 2010. Processing of disgusted faces is facilitated by odor primes: A functional MRI study. NeuroImage, 53(2): 746-756.
66. Simons, R. F., Detender, B. H., Roedema, T. M. and Reiss, J. E. 1999. Emotion processing in three systems: The medium and the message. Psychophysiology, 36: 619-627.
67. Singer, T. and Lamm, C. 2009. The social neuroscience of empathy. Annals of the New York Academy of Sciences, 1156: 81-96.
68. Stevenson, R. A., VanDerKlok, R. M., Pisoni, D. B. and James, T. W. 2011. Discrete neural substrates underlie complementary audiovisual speech integration processes. NeuroImage, 55(3): 1339-1345.
69. Summerfield, A. Q. 1979. Use of visual information on phonetic perception. Phonetica, 36: 314-331.
70. Thönnessen, H., Boers, F., Dammers, J., Chen, Y. H., Norra, C. and Mathiak, K. 2010. Early sensory encoding of affective prosody: Neuromagnetic tomography of emotional category changes. NeuroImage, 50(1): 250-259.
71. Trautmann, S. A., Fehr, T. and Herrmann, M. 2009. Emotions in motion: Dynamic compared to static facial expressions of disgust and happiness reveal more widespread emotion-specific activations. Brain Research, 1284: 100-115.
72. Venables, P. H., Gartshore, S. A. and O'Riordan, P. W. 1980. The function of skin conductance response recovery and rise time. Biological Psychology, 10(1): 1-6.
73. Weiss, U., Salloum, J. B. and Schneider, F. 1999. Correspondence of emotional self-rating with facial expression. Psychiatry Research, 86(2): 175-184.
74. Werner, S. and Noppeney, U. 2010. Distinct functional contributions of primary sensory and association areas to audiovisual integration in object categorization. Journal of Neuroscience, 30(7): 2662-2675.
75. Weyers, P., Mühlberger, A., Hefele, C. and Pauli, P. 2006. Electromyographic responses to static and dynamic avatar emotional facial expressions. Psychophysiology, 43: 450-453.
76. Wildgruber, D., Ackermann, H., Kreifelts, B. and Ethofer, T. 2006. Cerebral processing of linguistic and emotional prosody: fMRI studies. Progress in Brain Research, 156: 249-268.
77. Zeelenberg, R. and Bocanegra, B. R. 2010. Auditory emotional cues enhance visual perception. Cognition, 115(1): 202-206.