How emotional prosody guides your way: Evidence from eye movements

Speech Communication

Volumn 54, Issue 1, 2012, Pages 92-107

  Paulmann, Silke ^a   Titone, Debra ^b   Pell, Marc D ^b  

^a UNIVERSITY OF ESSEX   (United Kingdom)

^b MCGILL UNIVERSITY   (Canada)

Author keywords

Affective prosody; Eye tracking; Gaze; Semantics; Speech processing

Indexed keywords

AFFECTIVE PROSODY; CROSS-MODAL; EMOTIONAL PROSODY; EXPLICIT SEMANTICS; EYE FIXATIONS; EYE-GAZE; EYE-TRACKING; GAZE; GAZE BEHAVIOR; LINGUISTIC INFORMATION; SEMANTIC CUES; SEMANTIC INFORMATION; SOCIAL INFORMATION PROCESSING; TASK RELEVANT; VISUAL SEARCH;

DATA PROCESSING; SEMANTICS; SPEECH PROCESSING;

EYE MOVEMENTS;

EID: 80052722398     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2011.07.004     Document Type: Article

References (84)

1. P.D. Allopenna, J.S. Magnuson, and M.K. Tanenhaus Tracking the time course of spoken word recognition: evidence for continuous mapping models Journal of Memory and Language 38 1998 419 439 (Pubitemid 128348607)
2. V. Ashley, P. Vuilleumier, and D. Swick Time-course and specificity of event-related potentials to emotional expressions Neuroreport 15 2004 211 215
3. R. Banse, and K.R. Scherer Acoustic profiles in vocal emotion expression Journal of Personality and Social Psychology 70 3 1996 614 636 (Pubitemid 126420699)
4. M. Batty, and M.J. Taylor Early processing of the six basic facial emotional expressions Brain Research: Cognitive Brain Research 17 2003 613 620 (Pubitemid 38373367)
5. J.M.J. Berman, C.G. Chambers, and S.A. Graham Preschooler's appreciation of speaker vocal affect as a cue to referential intent Journal of Experimental Child Psychology 107 2 2010 87 99
6. M. Besson, C. Magne, and D. Schön Emotional prosody: sex differences in sensitivity to speech melody Trends in Cognitive Science 6 2002 405 407 (Pubitemid 36160448)
7. Boersma, P.; Weenink, D.; 2009. Praat: doing phonetics by computer (Version 5.1.05) [Computer program].
8. J.C. Borod, L.H. Pick, S. Hall, M. Sliwinski, N. Madigan, L.K. Obler, J. Welkowitz, E. Canino, H.M. Erhan, M. Goral, C. Morrison, and M. Tabert Relationships among facial, prosodic, and lexical channels of emotional perceptual processing Cognition and Emotion 14 2000 193 211 (Pubitemid 30353598)
9. V. Bostanov, and B. Kotchoubey Recognition of affective prosody: continuous wavelet measures of event-related brain potentials to emotional exclamations Psychophysiology 41 2004 259 268 (Pubitemid 38524098)
10. G.H. Bower Mood and memory American Psychologist 36 1981 129 148
11. D. Bowers, R. Bower, and K. Heilman The nonverbal affect lexicon: theoretical perspectives from neuropsychological studies of affect perception Neuropsychology 7 1993 433 444
12. D. Bowers, H.B. Coslette, R. Bauer, L. Speedie, and K.M. Heilman Comprehension of emotional prosody following unilateral hemispheric lesions: processing defect versus distraction defect Neuropsychologia 25 1987 317 328 (Pubitemid 17080377)
13. T. Brosch, D. Grandjean, D. Sander, and K.R. Scherer Cross-modal emotional attention: emotional voices modulate early stages of visual processing Journal of Cognitive Neuroscience 21 2008 1670 1679
14. M.G. Calvo, and L. Nummenmaa Detection of emotional faces: salient physical features guide effective visual search Journal of Experimental Psychology: General 137 2008 471 494
15. M.G. Calvo, L. Nummenmaa, and P. Avero Visual search of emotional faces: eye-movement assessment of component processes Experimental Psychology 55 2008 359 370
16. J.M. Carroll, and J.A. Russell Do facial expressions signal specific emotions? Judging the face in context Journal of Personality and Social Psychology 70 1996 205 218 (Pubitemid 126420669)
17. N.C. Carroll, and A.W. Young Priming of emotion recognition The Quarterly Journal of Experimental Psychology 58A 2005 1173 1197 (Pubitemid 41539379)
18. D. Dahan, and M.K. Tanenhaus Looking at the rope when looking for the snake: conceptually mediated eye movements during spoken word recognition Psychonomic Bulletin & Review 12 2005 453 459 (Pubitemid 41382746)
19. B. DeGelder, and J. Vroomen The perception of emotions by ear and by eye Cognition and Emotion 14 2000 289 311 (Pubitemid 30367746)
20. J.D. Eastwood, D. Smilek, and P.M. Merikle Differential attentional guidance by unattended faces expressing positive and negative emotion Perception & Psychophysics 63 2001 1004 1013 (Pubitemid 33603946)
21. M. Eimer, and A. Holmes An ERP study on the time course of emotional face processing Neuroreport 13 2002 427 431 (Pubitemid 34293183)
22. M. Eimer, A. Holmes, and F.P. McGlone The role of spatial attention in the processing of facial expression: an ERP study of rapid brain responses to six basic emotions Cognitive, Affective, and Behavioral Neuroscience 3 2003 97 110 (Pubitemid 37517026)
23. P. Ekman An argument for basic emotions Cognition and Emotion 6 3/4 1992 169 200
24. P. Ekman, E.R. Sorenson, and W.V. Friesen Pancultural elements in facial displays of emotion Science 164 3875 1969 86 88
25. R.H. Fazio On the automatic activation of associated evaluations: an overview Cognition and Emotion 15 2001 115 141 (Pubitemid 32536958)
26. A. Frischen, J.D. Eastwood, and D. Smilek Visual search for faces with emotional expressions Psychological Bulletin 134 2008 662 676
27. G.M. Grimshaw Integration and interference in the cerebral hemispheres: relations with hemispheric specialization Brain and Cognition 36 1998 108 127 (Pubitemid 28164798)
28. C.H. Hansen, and R.D. Hansen Finding the face in the crowd: an anger superiority effect Journal of Personality & Social Psychology 54 1988 917 924
29. C.H. Hansen, and C.A. Shantz Emotion-specific priming: congruence effects on affect and recognition across negative emotions Journal of Personality and Social Psychology 21 1995 548 557
30. J.M. Henderson, and F. Ferreira Scene perception for psycholinguists J.M. Henderson, F. Ferreira, The Interface of Language, Vision, and Action: Eye Movements and the Visual World 2004 Psychology Press New York
31. U. Hess, A. Kappas, and K.R. Scherer Multichannel Communication of Emotion: Synthetic Signal Production 1988 Erlbaum Hillsdale
32. J.K. Hietanen, J.M. Leppänen, M. Illi, and V. Surakka Evidence for the integration of audiovisual emotional information at the perceptual level of processing European Journal of Cognitive Psychology 16 2004 769 790 (Pubitemid 39506861)
33. G. Horstmann Preattentive face processing: what do we learn from visual search experiments? Visual Cognition 15 2007 799 833 (Pubitemid 47456981)
34. A. Innes-Ker, and P. Niedenthal Emotion concepts and emotional states in social judgment and categorization Journal of Personality and Social Psychology 83 2002 804 816
35. D.M. Isaacowitz, K. Toner, D. Goren, and H.R. Wilson Looking while unhappy: mood congruent gaze in young adults, positive gaze in older adults Psychological Science 19 2008 843 853
36. K. Ito, and S.R. Speer Anticipatory effect of intonation: eye movements during instructed visual search Journal of Memory and Language 58 2008 541 573
37. T. Johnstone, and K.R. Scherer Vocal communication of emotion M. Lewis, J. Haviland, Handbook of Emotions second ed. 2000 Guilford Press New York 220 235
38. J. Kissler, and A. Keil Look - don't look! How emotional pictures affect pro- and anti-saccades Experimental Brain Research 188 2008 215 222
39. S. Kitayama, and K. Ishii Word and voice: spontaneous attention to emotional utterances in two languages Cognition and Emotion 16 2002 29 59 (Pubitemid 34097598)
40. S. Koelsch, and W. Siebel Towards a neural basis of music perception Trends in Cognitive Sciences 9 2005 578 584 (Pubitemid 41677490)
41. S.A. Kotz, and S. Paulmann When emotional prosody and semantics dance cheek to cheek: ERP evidence Brain Research 1151 2007 107 118 (Pubitemid 46709874)
42. D.W. Massaro, and P.B. Egan Perceiving affect from the voice and the face Psychonomic Bulletin Review 3 1996 215 221 (Pubitemid 126421515)
43. E. Matin, K.C. Shao, and K.R. Boff Saccadic overhead. Information-processing time with and without saccades Perception & Psychophysics 53 1993 372 380
44. P.M. Niedenthal, and J.B. Halberstadt The acquisition and structure of emotional response categories The Psychology of Learning and Motivation 33 1995 23 63
45. L. Nummenmaa, J. Hyönä, and M.G. Calvo Emotional scene content drives the saccade generation system reflexively Journal of Experimental Psychology: Human Perception and Performance 35 2009 305 323
46. L.C. Nygaard, and E.R. Lunders Resolution of lexical ambiguity by emotional tone of voice Memory & Cognition 30 2002 583 593 (Pubitemid 34832962)
47. A. Öhman, Flykt, and F. Esteves Emotion drives attention: detecting the snake in the grass Journal of Experimental Psychology: General 130 2001 466 478 (Pubitemid 33597705)
48. Paulmann, S.; Kotz, S.A.; 2006. Valence, arousal, and task effects on the P200 in emotional prosody processing. In: Proceedings of Architectures and Mechanisms for Language Processing 2006 (AMLaP 2006), Nijmegen, The Netherlands, p. 37.
49. S. Paulmann, and S.A. Kotz Early emotional prosody perception based on different speaker voices Neuroreport 19 2 2008 209 213
50. S. Paulmann, and S.A. Kotz An ERP investigation on the temporal dynamics of emotional prosody and emotional semantics in pseudo- and lexical sentence context Brain and Language 105 2008 59 69
51. S. Paulmann, and M.D. Pell Decoding emotional faces depends on their representational value: ERP evidence Neuroreport 20 2009 1603 1608
52. S. Paulmann, and M.D. Pell Contextual influences of speech tone on emotional processing: how much is enough? Cognitive, Affective, and Behavioral Neuroscience 10 2010 230 242
53. S. Paulmann, and M.D. Pell Is there an advantage for recognizing multi-modal emotional stimuli? Motivation and Emotion 35 2 2011 192 201
54. S. Paulmann, S. Jessen, and S.A. Kotz Investigating the multi-modal nature of human communication: insights from ERPs Journal of Psychophysiology 23 2009 63 76
55. S. Paulmann, M.D. Pell, and S.A. Kotz How aging affects the recognition of emotional speech Brain and Language 104 2008 262 269 (Pubitemid 351342698)
56. M.D. Pell Prosody face interactions in emotional processing as revealed by the facial affect decision task Journal of Nonverbal Behavior 29 4 2005 193 215 (Pubitemid 41807466)
57. M.D. Pell, and V. Skorup Implicit processing of emotional prosody in a foreign versus native language Speech Communication 50 6 2008 519 530
58. M.D. Pell, L. Monetta, S. Paulmann, and S.A. Kotz Recognizing emotions in a foreign language Journal of Nonverbal Behavior 33 2 2009 107 120
59. M.D. Pell, S. Paulmann, C. Dara, A. Alasseri, and S.A. Kotz Factors in the recognition of vocally expressed emotions: a comparison of four languages Journal of Phonetics 37 2009 417 435
60. Pell, M.D.; Jaywant, A.; Monetta, L.; Kotz, S.A.; in press. Emotional speech processing: disentangling the effects of prosody and semantic cues. Cognition and Emotion.
61. M.D. Pell Nonverbal emotion priming: evidence from the 'facial affect decision task' Journal of Nonverbal Behavior 29 1 2005 45 73 (Pubitemid 40366351)
62. M.D. Pell Cerebral mechanisms for understanding emotional prosody in speech Brain and Language 96 2 2006 221 234 (Pubitemid 43221748)
63. J. Pittam, and K.R. Scherer Vocal expression and communication of emotion M. Lewis, J.M. Haviland, Handbook of Emotions 1993 Guilford Press New York pp. 185-197
64. J. Russell, and G. Lemay Emotion Concepts M. Lewis, M. Haviland-Jones, Handbook of Emotion 2000 Guilford Press New York
65. D.A. Sauter, A.J. Calder, F. Eisner, and S.K. Scott Perceptual cues in non-verbal vocal expressions of emotion The Quarterly Journal of Experimental Psychology 63 11 2010 2251 2272
66. D. Sauter, and M. Eimer Rapid detection of emotion from human vocalizations Journal of Cognitive Neuroscience 22 2010 474 481
67. K.R. Scherer, R. Banse, and H. Wallbott Emotion inferences from vocal expression correlate across languages and cultures Journal of Cross-Cultural Psychology 32 1 2001 76 92
68. K.R. Scherer Vocal Measurement of Emotion 1989 Academic Press New York
69. A. Schirmer, and S.A. Kotz Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing Trends in Cognitive Sciences 10 2006 24 30 (Pubitemid 43049769)
70. A. Schirmer, S.A. Kotz, and A.D. Friederici Sex differentiates the role of emotional prosody during word processing Cognitive Brain Research 14 2002 228 233 (Pubitemid 34626951)
71. A. Schirmer, and S.A. Kotz ERP evidence for a sex-specific Stroop effect in emotional speech Journal of Cognitive Neuroscience 15 2003 1135 1148 (Pubitemid 37487388)
72. A. Schirmer, T. Striano, and A.D. Friederici Sex differences in the preattentive processing of vocal emotional expressions NeuroReport 16 2005 635 639 (Pubitemid 40586098)
73. A. Schirmer, S.A. Kotz, and A.D. Friederici On the role of attention for the processing of emotions in speech: sex differences revisited Cognitive Brain Research 24 2005 442 452 (Pubitemid 41133282)
74. M.J. Spivey, M.J. Tyler, K.M. Eberhard, and M.K. Tanenhaus Linguistically mediated visual search Psychological Science 12 2001 282 286 (Pubitemid 33653282)
75. Susan MuManus M.; 2009. Gaze Fixation During the Perception of Visual and Auditory Affective Cues. Psychology Theses. Paper 70. .
76. M.K. Tanenhaus, M.J. Spivey-Knowlton, K.M. Eberhard, and J.E. Sedivy Integration of visual and linguistic information in spoken language comprehension Science 268 1995 1632 1634
77. W.F. Thompson, F.A. Russo, and L. Quinto Audio-visual integration of emotional cues in song Cognition and Emotion 22 2008 1457 1470
78. J. Vroomen, and B. de Gelder Sound enhances visual perception: cross-modal effects of auditory organization on vision Journal of Experimental Psychology - Human Perception and Performance 26 5 2000 1583 1590
79. J. Vroomen, J. Driver, and B. de Gelder Is cross-modal integration of emotional expressions independent of attentional resources? Cognitive, Affective, & Behavioral Neuroscience 1 2001 382 387 (Pubitemid 38391259)
80. I.J. Wambacq, K.J. Shea-Miller, and A. Abubakr Non-voluntary and voluntary processing of emotional prosody: an event-related potentials study Neuroreport 15 2004 555 559 (Pubitemid 38328159)
81. A. Weber, M. Grice, and M.W. Crocker The role of prosody in the interpretation of structural ambiguities: a study of anticipatory eye movements Cognition 99 2006 B63 B72
82. D. Wilson, and T. Wharton Relevance and prosody Journal of Pragmatics 36 1 2006 1559 1579 (Pubitemid 44227824)
83. A.W. Young, D. Rowland, A.J. Calder, N.L. Etcoff, A. Seth, and D.I. Perrett Facial expression megamix: tests of dimensional and category accounts of emotion recognition Cognition 63 1997 271 313 (Pubitemid 127432166)
84. R. Zeelenberg, and B.R. Bocanegra Auditory emotional cues enhance visual perception Cognition 115 2010 202 206