Emotional Speech Processing at the Intersection of Prosody and Semantics

PLoS ONE

Volumn 7, Issue 10, 2012, Pages

  Schwartz, Rachel ^a   Pell, Marc D ^a  

^a MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ACCURACY; ADULT; ARTICLE; EMOTION; EMOTIONAL STABILITY; FACIAL EXPRESSION; FEMALE; HUMAN; HUMAN EXPERIMENT; INFORMATION PROCESSING; LATENT PERIOD; MALE; NORMAL HUMAN; PROSODY; SEMANTICS; SENSORY STIMULATION; SPEECH; SPEECH DISCRIMINATION; TASK PERFORMANCE;

ADULT; BRAIN; CUES; ELECTROENCEPHALOGRAPHY; EMOTIONS; FEMALE; HUMANS; INTERPERSONAL RELATIONS; MALE; REACTION TIME; SEMANTICS; SPEECH; SPEECH PERCEPTION;

EID: 84868267660     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0047279     Document Type: Article

References (65)

1. Barrett LF, Lindquist KA, Gendron M, (2007) Language as context for the perception of emotion. Trends in Cognitive Sciences 11: 327-332.
2. Ekman P, (1993) Facial expression and emotion. American Psychologist 48: 384-392.
3. Etcoff NL, Magee JL, (1992) Categorical perception of facial expressions. Cognition 44: 227-240.
4. Pell MD, (2001) Influence of emotion and focus location on prosody in matched statements and questions. Journal of the Acoustical Society of America 109: 1668-1680.
5. Schirmer A, Kotz SA, Friederici AD, (2002) Sex differentiates the role of emotional prosody during word processing. Cognitive Brain Research 14: 228-233.
6. Schirmer A, Kotz SA, Friederici A, (2005) On the role of attention for the processing of emotions in speech: Sex differences revisited. Cognitive Brain Research 24: 442-452.
7. Wurm LH, Vakoch DA, Strasser MR, Calin-Jageman R, Ross SE, (2001) Speech perception and vocal expression of emotion. Cognition and Emotion 15: 831-852.
8. Borod JC, Pick LH, Hall S, Sliwinski M, Madigan N, et al. (2000) Relationships among facial, prosodic, and lexical channels of emotional perceptual processing. Cognition and Emotion 14: 193-211.
9. Efron D (1941) Gestures and Environment. New York: King's Crown.
10. Ekman P, Friesen W, (1969) The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica 1: 49-98.
11. de Gelder B, (2006) Toward a biological theory of emotional body language. Biological Theory 1: 130-132.
12. Edlund J, Heldner M, Hirschberg J (2009) Pause and gap length in face-to-face interaction. Proceedings of Interspeech 2009. Brighton, UK.
13. Giles H, Coupland N, Coupland J (1992) Accomodation theory: Communication, context and consequence. In: Giles, H, Coupland, N, Coupland J, editor. Contexts of Accommodation. Cambridge University Press. pp. 1-68.
14. Lakin JL, Jefferis VE, Cheng CM, Chartrand TL, (2003) The chameleon effect as social glue: Evidence for the evolutionary significance of nonconscious mimicry. Journal of Nonverbal Behavior 27: 145-162.
15. Welkowitz J, Kuc M, (1973) Interrelationships among warmth, genuineness, empathy and temporal speech patterns in interpersonal interaction. Journal of Consulting and Psychology 41: 472-473.
16. Rockwell P, (2000) Lower, slower, louder: Vocal cues of sarcasm. Journal of Psycholinguistic Research 29: 483-495.
17. Bryant G, Fox Tree J, (2002) Recognizing verbal irony in spontaneous speech. Metaphor and Symbol 17: 99-117.
18. Cheang HS, Pell MD, (2008) The sound of sarcasm. Speech Communication 50: 366-381.
19. Woodland J, Voyer D, (2011) Context and Intonation in the Perception of Sarcasm. Metaphor and Symbol 26: 227-239.
20. Pell MD, Kotz SA, (2011) On the time course of vocal emotion recognition. PLoS ONE 6: e27256.
21. Pell MD, (2005) Prosody-face interactions in emotional processing as revealed by the facial affect decision task. Journal of Nonverbal Behavior 29: 193-215.
22. Paulmann S, Kotz SA, (2008) Early emotional prosody perception based on different speaker voices. NeuroReport 19: 209-213.
23. Paulmann S, Pell MD, (2010) Contextual influences of emotional speech prosody on face processing: How much is enough? Cognitive, Affective, & Behavioral Neuroscience 10: 230-242.
24. Bostanov V, Kotchoubey B, (2004) Recognition of affective prosody: Continuous wavelet measures of event-related potentials to emotional exclamations. Psychophysiology 41: 259-268.
25. Pell MD, (2005) Nonverbal emotion priming: Evidence from the "facial affect decision task.". Journal of Nonverbal Behavior 29: 45-73.
26. Pell MD, Skorup V, (2008) Implicit processing of emotional prosody in a foreign versus native language. Speech Communication 50: 519-530.
27. Pell MD, Jaywant A, Monetta L, Kotz SA, (2011) Emotional speech processing: Disentangling the effects of prosody and semantic cues. Cognition and Emotion 25: 834-853.
28. Kotz SA, Paulmann S, (2007) When emotional prosody and semantics dance cheek to cheek: ERP evidence. Brain Research 115: 107-118.
29. Paulmann S, Pell MD, (2011) Is there an advantage for recognizing multi-modal emotional stimuli? Motivation and Emotion 35: 192-201.
30. Aguert M, Laval V, Le Bigot L, Bernicot J, (2010) Understanding expressive speech acts: the role of prosody and situational context in French-speaking 5- to 9-year-olds. Journal of speech, language, and hearing research: JSLHR 53: 1629-1641.
31. Shuster LI, (2009) The effect of sublexical and lexical frequency on speech production: An fMRI investigation. Brain and Language 111: 66-72.
32. Pell MD, (2005) Nonverbal emotion priming: Evidence from the "Facial Affect Decision Task.". Journal of Nonverbal Behavior 29: 45-73.
33. Collignon O, Girard S, Gosselin F, Roy S, Saint-Amour D, et al. (2008) Audio-visual integration of emotion expression. Brain Res 1242: 126-135.
34. de Gelder B, Vroomen J, (2000) The perception of emotions by ear and by eye. Cognition and Emotion 14: 289-311.
35. Kreifelts B, Ethofer T, Grodd W, Erb M, Wildgruber D, (2007) Audiovisual integration of emotional signals in voice and face: An event-related fMRI study. NeuroImage 37: 1445-1456.
36. Paulmann S, Jessen S, Kotz SA, (2009) Investigating the multimodal nature of human communication. Journal of Psychophysiology 23: 63-76.
37. Leppänen JM, Hietanen JK, (2004) Positive facial expressions are recognized faster than negative facial expressions, but why? Psychological research 69: 22-29.
38. Pell MD, Monetta L, Paulmann S, Kotz SA, (2009) Recognizing emotions in a foreign language. Journal of Nonverbal Behavior 33: 107-120.
39. Paulmann S, Pell MD, (2009) Facial expression decoding as a function of emotional meaning status: ERP evidence. NeuroReport 20: 1603-1608.
40. Jaywant A, Pell MD, (2012) Categorical processing of negative emotions from speech prosody. Speech Communication 54: 1-10.
41. Carroll NC, Young AW, (2005) Priming of emotion recognition. The Quarterly journal of experimental psychology A, Human experimental psychology 58: 1173-1197.
42. Hsu S-M, Hetrick WP, Pessoa L, (2008) Depth of facial expression processing depends on stimulus visibility: Behavioral and electrophysiological evidence of priming effects. Cognitive, Affective, & Behavioral Neuroscience 8: 282-292.
43. Niedenthal PM, (1990) Implicit perception of affective information. Journal of Experimental Social Psychology 26: 505-527.
44. Niedenthal PM, Halberstadt JB, Setterlund MB, (1997) Being happy and seeing "happy": Emotional state mediates visual word recognition. Cognition and Emotion 11: 403-432.
45. Kitayama S, Howard S (1994) Affective regulation of perception and comprehension: Amplification and sematic priming. In: Niedenthal PM, Kitayama S, editors. The heart's eye: Emotional influences in perception and attention. San Diego: Academic Press. pp. 41-65.
46. Bower GH, (1981) Mood and memory. American Psychologist 36: 129-148.
47. Beaucousin V, Lacheret A, Turbelin M-R, Morel M, Mazoyer B, et al. (2007) FMRI study of emotional speech comprehension. Cerebral Cortex 17: 339-352.
48. Nygaard LC, Queen JS, (2008) Communicating emotion: Linking affective prosody and word meaning. Journal of Experimental Psychology Human Perception and Performance 34: 1017-1030.
49. Techentin C, Voyer D, Klein RM, (2009) Between- and within-ear congruency and laterality effects in an auditory semantic/emotional prosody conflict task. Brain and Cognition 70: 201-208.
50. Wambacq IJA, Jerger JF, (2004) Processing of affective prosody and lexical-semantics in spoken utterances as differentiated by event-related potentials. Brain Res Cogn Brain Res 20: 427-437.
51. De Gelder B, Bertelson P, (2003) Multisensory integration, perception and ecological validity. Trends in Cognitive Sciences 7: 460-467.
52. Grimshaw GM, (1998) Integration and interference in the cerebral hemispheres: relations with hemispheric specialization. Brain and Cognition 36: 108-127.
53. Kitayama S, Ishii K, (2002) Word and voice: Spontaneous attention to emotional utterances in two languages. Cognition and Emotion 16: 29-59.
54. Schirmer A, Kotz SA, (2003) ERP evidence for a sex-specific Stroop effect in emotional speech. Journal of cognitive neuroscience 15: 1135-1148.
55. Astesano C, (2004) Brain potentials during semantic and prosodic processing in French. Cognitive Brain Research 18: 172-184.
56. Schirmer A, Kotz S, (2006) Beyond the right hemisphere: Brain mechanisms mediating vocal emotional processing. Trends in Cognitive Sciences 10: 24-30.
57. Laurienti PJ, Kraft RA, Maldjian JA, Burdette JH, Wallace MT, (2004) Semantic congruence is a critical factor in multisensory behavioral performance. Experimental brain research Experimentelle Hirnforschung Expérimentation cérébrale 158: 405-414.
58. Meyer M, Alter K, Friederici A, Lohmann G, von Cramon DY, (2002) FMRI reveals brain regions mediating slow prosodic modulations in spoken sentences. Human Brain Mapping 17: 73-88.
59. Kotz S, Meyer M, Alter K, Besson M, von Cramon Y, et al. (2003) On the lateralization of emotional prosody: An event-related functional MR investigation. Brain and Language 86: 366-376.
60. Regenbogen C, Schneider DA, Gur RE, Schneider F, Habel U, et al. (2012) Multimodal human communication - Targeting facial expressions, speech content and prosody. NeuroImage 60: 2346-2356.
61. Regenbogen C, Schneider DA, Finkelmeyer A, Kohn N, Derntl B, et al. (2012) The differential contribution of facial expressions, prosody, and speech content to empathy. Cognition & Emotion pp. 37-41.
62. de Gelder B, Vroomen J, Pourtois G (2004) Multisensory perception of emotion, its time course, and its neural basis. In: Stein B, et al., editor. Handbook of Multisensory Perception. MIT Press. pp. 581-596.
63. Britton BK, Glynn SM, Meyer BJ, Penland MJ, (1982) Effects of text structure on use of cognitive capacity during reading. Journal of Educational Psychology 74: 51-61.
64. Leppänen JM, Hietanen JK, (2004) Positive facial expressions are recognized faster than negative facial expressions, but why? Psychological Research 69: 22-29.
65. Tracy JL, Robins RW, (2008) The automaticity of emotion recognition. Emotion 8: 81-95.