Feeling backwards? How temporal order in speech affects the time course of vocal emotion recognition

Frontiers in Psychology

Volumn 4, Issue , 2013, Pages

  Rigoulot, Simon ^a   Wassiliwizky, Eugen ^a,b   Pell, Marc D ^a  

^a MCGILL UNIVERSITY   (Canada)

^b FREIE UNIVERSITÄT BERLIN   (Germany)

Author keywords

Acoustics; Auditory gating; Prosody; Speech perception; Vocal emotions

Indexed keywords

EID: 84904543081     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2013.00367     Document Type: Article

References (63)

1. Alter, K., Rank, E., Kotz, S. A., Toepel, U., Besson, M., Schirmer, A., et al. (2003). Affective encoding in the speech signal and in event-related brain potentials. Speech Commun. 40, 61–70. doi: 10.1016/S0167-6393(02)00075-4
2. Banse, R., and Scherer, K. R. (1996). Acoustic profiles in vocal emotion expression. J. Pers. Soc. Psychol. 70, 614–636. doi: 10.1037/0022-3514.70.3.614
3. Barkhuysen, P., Krahmer, E., and Swerts, M. (2010). Crossmodal and incremental perception of audiovisual cues to emotional speech. Lang. Speech 53, 1–30. doi: 10.1177/0023830909348993
4. Becker, D. V., Neel, R., Srinivasan, N., Neufeld, S., Kumar, D., and Fouse, S. (2012). The vividness of happiness in dynamic facial displays of emotion. PLoS ONE 7:e26551. doi: 10.1371/journal.pone.0026551
5. Boersma, P., and Weenink, D. (2012). Praat: Doing Phonetics by Computer [Computer Program]. Version 5.3.21.
6. Bolinger, D. (1978). “Intonation across languages,” in Universals of Human Language, vol. II. Phonology, ed J. Greenberg (Palo Alto, CA: Stanford University Press), 471–524.
7. Bolinger, D. (1972). Accent is predictable (if you’re a mind reader). Language 48, 633–644. doi: 10.2307/412039
8. Cacioppo, J. T., and Gardner, W. L. (1999). Emotion. Annu. Rev. Psychol. 50, 191–214. doi: 10.1146/annurev.psych.50.1.191
9. Calder, A. J., Keane, J., Lawrence, A. D., and Manes, F. (2004). Impaired recognition of anger following damage to the ventral striatum. Brain 127: 1958–1969. doi: 10.1093/brain/awh214
10. Calder, A. J., Lawrence, A. D., and Young, A. W. (2001). Neuropsychology of fear and loathing. Nat. Rev. Neurosci. 2, 352–363. doi: 10.1038/35072584
11. Calder, A. J., Keane, J., Young, A. W., Lawrence, A. D., Mason, S., and Barker, R. (2010). The relation between anger and different forms of disgust: implications for emotion recognition impairments in Huntington’s disease. Neuropsychologia 48, 2719–2729. doi: 10.1016/j.neuropsychologia. 2010.05.019
12. Calvo, M. G., and Nummenmaa, L. (2009). Eye-movement assessment of the time course in facial expression recognition: neurophysiological implications. Cogn. Affect. Behav. Neurosci. 9, 398–411. doi: 10.3758/CABN.9.4.398
13. Carretié, L., Mercado, F., Tapia, M., and Hinojosa, J. A. (2001). Emotion, attention, and the “neg-ativity bias”, studied through event-related potentials. Int. J. Psychophysiol. 41, 75–85. doi: 10.1016/S0167-8760(00)00195-1
14. Cornew, L., Carver, L., and Love, T. (2010). There’s more to emotion than meets the eye: a processing bias for neutral content in the domain of emotional prosody. Cogn. Emot. 24: 1133–1152. doi: 10.1080/02699930903247492
15. Cosmides, L. (1983). Invariances in the acoustic expression of emotion during speech. J. Exp. Psychol. Hum. Percept. Perform. 9 (6), 864–881. doi: 10.1037/0096-1523.9.6.864
16. Ekman, P. (1972). “Universal and cultural differences in facial expressions of emotions,” in Nebraska Symposium on Motivation, 1971, ed J. K. Cole (Lincoln: University of Nebraska Press), 207–283.
17. Ekman, P. (1992). An argument for basic emotions. Cogn. Emot. 6, 169–200. doi: 10.1080/02699939208411068
18. Ekman, P., Levenson, R. W., and Friesen, W. V. (1983). Autonomic nervous system activity distinguishes between emotions. Science 221, 1208–1210. doi: 10.1126/science.6612338
19. Grosjean, F. (1980). Spoken word recognition processes and the gating paradigm. Percept. Psychophys. 18, 267–283. doi: 10.3758/BF03204386
20. Grosjean, F. (1996) Gating. Lang. Cogn. Process. 11, 597–604. doi: 10.1080/016909696386999
21. Hess, U., Beaupré, M. G., and Cheung, N. (2002). “Who to whom and why – cultural differences and similarities in the function of smiles,” in An Empirical Reflection on the Smile, eds M. Abel and C. H. Ceia (Lewiston, NY: The Edwin Mellen Press), 187–216.
22. Izard, C. E. (1971). The Face of Emotion. New York, NY: Appleton-Century-Crofts.
23. Jaywant, A., and Pell, M. D. (2012). Categorical processing of negative emotions from speech prosody. Speech Commun. 54, 1–10. doi: 10.1016/j.specom.2011.05.011
24. Johnstone, T., and Scherer, K. R. (2000). “Vocal communication of emotion,” in Handbook of Emotions, 2nd Edn. eds M. Lewis and J. Haviland (New York, NY: Guilford Press), 220–235.
25. Juslin, P. N., and Laukka, P. (2003). Communication of emotions in vocal expression and music performance: different channels, same code? Psychol. Bull. 129, 770–814. doi: 10.1037/0033-2909.129.5.770
26. Ladd, D. R. (1996). Intonational Phonology. Cambridge, UK: Cambridge University Press.
27. Ladd, D. R., Silverman, K., Tolkmitt, F., Bergmann, G., and Scherer, K. R. (1985). Evidence for the independent function of intonation contour type, voice quality and F0 range in signaling speaker affect. J. Acoust. Soc. Am. 78, 435–444. doi: 10.1121/1.392466
28. Laukka, P. (2005). Categorical perception of vocal emotion expressions. Emotion 5, 277–295. doi: 10.1037/1528-3542.5.3.277
29. Laukka, P., Juslin, P., and Bresin, R. (2005). A dimensional approach to vocal expression of emotion. Cogn. Emot. 19, 633–653. doi: 10.1080/02699930441000445
30. Leinonen, L., Hiltunen, T., Linnankoski, I., and Laakso, M.-L. (1997). Expression of emotional-motivational connotations with a one-word utterance. J. Acoust. Soc. Am. 102, 1853–1863. doi: 10.1121/1.420109
31. Levenson, R. W. (1992). Autonomic nervous system differences among emotions. Psychol. Sci. 3, 23–27. doi: 10.1111/j.1467-9280.1992. tb00251.x
32. Levitt, E. A. (1964). “The relationship between abilities to express emotional meanings vocally and facially,” in The Communication of Emotional Meaning, ed J. R. Davitz (New York, NY: McGraw-Hill), 87–100.
33. Oller, D. K. (1973). The effect of position in utterance on speech segment duration in English. J. Acoust. Soci. Am. 54, 1235–1247. doi: 10.1121/1.1914393
34. Palermo, R., and Coltheart, M. (2004). Photographs of facial expression: accuracy, response times, and ratings of intensity. Behav. Res. Meth. Instrum. Comput. 36, 634–638. doi: 10.3758/BF03206544
35. Paulmann, S., and Kotz, S. A. (2008). Early emotional prosody perception based on different speaker voices. Neuroreport, 19, 209–213. doi: 10.1097/WNR.0b013e3282f454db
36. Paulmann, S., and Pell, M. D. (2010). Contextual influences of emotional speech prosody on face processing: how much is enough? Cogn., Affect. Behav. Neurosci. 10, 230–242. doi: 10.3758/CABN.10.2.230
37. Paulmann, S., and Pell, M. D. (2011). Is there an advantage for recognizing multi-modal emotional stimuli? Mot. Emot. 35, 192–201. doi: 10.1007/s11031-011-9206-0
38. Paulmann, S., Pell, M. D., and Kotz, S. A. (2008). Functional contributions of the basal ganglia to emotional prosody: evidence from ERPs. Brain Res. 1217, 171–178. doi: 10.1016/j.brainres.2008.04.032
39. Pell, M. D. (2001). Influence of emotion and focus location on prosody in matched statements and questions. J. Acoust. Soc. Am. 109, 1668–1680. doi: 10.1121/1.1352088
40. Pell, M. D., and Baum, S. R. (1997). Unilateral brain damage, prosodic comprehension deficits, and the acoustic cues to prosody. Brain Lang. 57, 195–214. doi: 10.1006/brln.1997.1736
41. Pell, M. D., and Kotz, S. (2011). On the time course of vocal emotion recognition. PLoS ONE 6:e27256. doi: 10.1371/journal.pone.0027256
42. Pell, M. D., Paulmann, S., Dara, C., Alasseri, A., and Kotz, S. A. (2009). Factors in the recognition of vocally expressed emotions: a comparison of four languages. J. Phonet. 37, 417–435. doi: 10.1016/j.wocn.2009. 07.005
43. Pell, M. D., and Skorup, V. (2008). Implicit processing of emotional prosody in a foreign versus native language. Speech Commun. 50, 519–530. doi: 10.1016/j.specom. 2008.03.006
44. Rigoulot, S., and Pell, M. D. (2012). Seeing emotion with your ears: emotional prosody implicitly guides visual attention to faces. PLoS ONE 7:e30740. doi: 10.1371/journal.pone.0030740
45. Rozin, P., Lowery, L., and Ebert, R. (1994). Varieties of disgust faces and the structure of disgust. J. Pers. Soc. Psychol. 66, 870–881. doi: 10.1037/0022-3514.66.5.870
46. Sauter, D., Eisner, F., Ekman, P., and Scott, S. K. (2010). Cross-cultural recognition of basic emotions through nonverbal emotional vocalizations. Proc. Natl. Acad. Sci. U.S.A. 107, 2408–2412. doi: 10.1073/pnas.0908239106
47. Sauter, D., and Scott, S. K. (2007). More than one kind of happiness: can we recognize vocal expressions of different positive states? Mot. Emot. 31, 192–199. doi: 10.1007/s11031-007-9065-x
48. Scherer, K. R. (1986) Vocal affect expression: a review and a model for future research. Psychol. Bull. 99, 143–165. doi: 10.1037/0033-2909.99.2.143
49. Scherer, K. R. (1988). On the symbolic functions of vocal affect expression. J. Lang. Soc. Psychol. 7, 79–100. doi: 10.1177/0261927X8800 700201
50. Scherer, K. R. (2009). Emotions are emergent processes: they require a dynamic computational architecture. Philos. Trans. R. Soc. B 364, 3459–3474. doi: 10.1098/rstb.2009.0141
51. Scherer, K. R., Banse, R., and Wallbott, H. G. (2001) Emotion inferences from vocal expression correlate across language and cultures. J. Cross Cult. Psychol. 32, 76–92. doi: 10.1177/0022022101032001009
52. Scherer, K. R., Banse, R., Wallbott, H. G., and Goldbeck, T. (1991) Vocal cues in emotion encoding and decoding. Mot. Emot. 15, 123–148. doi: 10.1007/BF00995674
53. Schirmer, A., and Kotz, S. A. (2006). Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing. Trends Cogn. Sci. 10, 24–30. doi: 10.1016/j.tics.2005.11.009
54. Simon-Thomas, E., Keltner, D., Sauter, D., Sinicropi-Yao, L., and Abramson, A. (2009) The voice conveys specific emotions: evidence from vocal burst displays. Emotion 9, 838–846. doi: 10.1037/a0017810
55. Sobin, C., and Alpert, M. (1999). Emotions in speech: the acoustic attributes of fear, anger, sadness and joy. J. Psycholinguist. Res. 28, 347–365. doi: 10.1023/A:1023237014909
56. Szameitat, D. P., Kreifelts, B., Alter, K., Szameitat, A. J., Sterr, A., Grodd, W., et al. (2010). It is not always tickling: distinct cerebral responses during perception of different laughter types. Neuroimage 53, 1264–1271. doi: 10.1016/j.neuroimage.2010. 06.028
57. Thompson, W. F., and Balkwill, L.-L. (2006). Decoding speech prosody in five languages. Semiotica 158, 407–24.
58. Tracy, J. L., and Robins, R. W. (2008). The automaticity of emotion recognition. Emotion 8, 81–95. doi: 10.1037/1528-3542.8.1.81
59. Van Bezooijen, R., Otto, S., and Heenan, T. (1983). Recognition of vocal expressions of emotion: a three-nation study to identify universal characteristics. J. Cross Cult. Psychol. 14, 387–406. doi: 10.1177/0022002183014004001
60. Wagner, H. L. (1993) On measuring performance in category judgment studies of nonverbal behavior. J. Nonverb. Behav. 17, 3–28. doi: 10.1007/BF00987006
61. Wallbott, H. G., and Scherer, K. R. (1986). Cues and channels in emotion recognition. J. Personal. Soc. Psychol. 51, 690–99. doi: 10.1037/0022-3514.51.4.690
62. Wilson, D., and Wharton, T. (2006). Relevance and prosody. J. Pragmat. 38, 1559–1579. doi: 10.1016/j.pragma.2005.04.012
63. Zuckerman, M., Lipets, M. S., Koivumaki, J. H., and Rosenthal, R. (1975). Encoding and decoding nonverbal cues of emotion. J. Pers. Soc. Psychol. 32, 1068–1076. doi: 10.1037/0022-3514.32.6.1068