On how the brain decodes vocal cues about speaker confidence

Cortex

Volumn 66, Issue , 2015, Pages 9-34

  Jiang, Xiaoming ^a   Pell, Marc D ^a  

^a MCGILL UNIVERSITY   (Canada)

Author keywords

Expressed confidence; Feeling of knowing; Late positivity; P200; Prosody

Indexed keywords

ADULT; ARTICLE; ASSOCIATION; BEHAVIOR; COMPREHENSION; CONTROLLED STUDY; ELECTROENCEPHALOGRAM; EMOTION; EVENT RELATED POTENTIAL; FEMALE; HEARING; HUMAN; HUMAN EXPERIMENT; MALE; NERVE POTENTIAL; NORMAL HUMAN; SEX DIFFERENCE; SPEAKER CONFIDENCE; SPEECH; SPEECH DISCRIMINATION; SPEECH PROCESSING; SPEECH RATE; TIME; VOCAL CUE; VOICE; WAVEFORM; ADOLESCENT; AUDITORY STIMULATION; BRAIN; ELECTROENCEPHALOGRAPHY; EVOKED AUDITORY RESPONSE; INTERPERSONAL COMMUNICATION; PERCEPTION; PHYSIOLOGY; REACTION TIME; SPEECH PERCEPTION; YOUNG ADULT;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; AUDITORY PERCEPTION; BRAIN; COMMUNICATION; CUES; ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS, AUDITORY; FEMALE; HUMANS; MALE; REACTION TIME; SOCIAL PERCEPTION; SPEECH PERCEPTION; VOICE; YOUNG ADULT;

EID: 84925012098     PISSN: 00109452     EISSN: 19738102     Source Type: Journal    
DOI: 10.1016/j.cortex.2015.02.002     Document Type: Article

References (87)

1. Alain C., Winkler I. Recording event-related potentials: application to study auditory perception. The human auditory cortex 2012, 69-96. Springer, New York. D. Poeppel, T. Overath, A. Popper, R. Fay (Eds.).
2. Allison C., Baron-Cohen S., Wheelwright S., Stone M., Muncer S. Psychometric analysis of the Empathy Quotient (EQ). Personality and Individual Differences 2011, 51:829-835.
3. Alter K., Rank E., Kotz S., Toepel U., Besson M., Schirmer A., et al. Affective encoding in the speech signal and in event-related brain potentials. Speech Communication 2003, 40:61-70.
4. Apple W., Streeter L., Krauss R. Effects of pitch and speech rate on personal attributes. Journal of Personality and Social Psychology 1979, 37:715-727.
5. Baayen R., Davidson D., Bates D. Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language 2008, 59:390-412.
6. Barr D. Paralinguistic correlates of conceptual structure. Psychonomic Bulletin & Review 2003, 10:462-467.
7. Belin P., Fecteau S., Bédard C. Thinking the voice: neural correlates of voice perception. Trends in Cognitive Sciences 2004, 8:129-135.
8. Bolinger D. Intonation across languages. Phonology 1978, Vol. 2:471-524. Stanford University Press, Stanford, CA. J.H. Greenberg, C.A. Ferguson, E.A. Moravcsik (Eds.).
9. Brennan S., Williams M. The feeling of another's knowing: prosody and filled pauses as cues to listeners about the metacognitive states of speakers. Journal of Language and Memory 1995, 34:383-398.
10. Brouwer H., Fitz H., Hoeks J. Getting real about semantic illusions: rethinking the functional role of the P600 in language comprehension. Brain Research 2012, 1446:127-143.
11. Brouwer H., Hoeks J. A time and place for language comprehension: mapping the N400 and the P600 to a minimal cortical network. Frontiers in Human Neuroscience 2013, 7:758.
12. Caffi C., Janney R. Toward a pragmatics of emotive communication. Journal of Pragmatics 1994, 22:325-373.
13. Charest I., Pernet C., Rousselet G., Quiñones I., Latinus M., Fillion-Bilodeau S., et al. Electrophysiological evidence for an early processing of human voices. BMC Neuroscience 2009, 10:127.
14. Comerchero C., Polich J. P3a and P3b from typical auditory and visual stimuli. Clinical Neurophysiology 1999, 110:24-30.
15. Delorme A., Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods 2004, 134:9-21.
16. DePaulo B., Stone J., Lassiter D. Telling ingratiating lies: effects of target sex and target attractiveness on verbal and nonverbal deceptive success. Journal of Personality and Social Psychology 1985, 48:1191-1203.
17. Dien J. Progressing towards a consensus on PCA of ERPs. Clinical Neurophysiology 2006, 117:699-702.
18. Dien J. Evaluating two-step PCA of ERP data with Geomin, Infomax, Oblimin, Promax, and Varimax rotations. Psychophysiology 2010, 47:170-183.
19. Dien J. Applying principal components analysis to event-related potentials: a tutorial. Developmental Neuropsychology 2012, 37:497-517.
20. Dien J., Michelson C., Franklin M. Separating the visual sentence N400 effect from the P400 sequential expectancy effect: cognitive and neuroanatomical implications. Brain Research 2010, 1355:126-140.
21. Foti D., Hajcak G., Dien J. Differentiating neural responses to emotional pictures: evidence from temporal-spatial PCA. Psychophysiology 2009, 46:521-530.
22. Hemsley G., Doob A. The effect of looking behavior on perceptions of a communicator's credibility. Journal of Applied Social Psychology 1978, 8:135-144.
23. Jessen S., Kotz S. The temporal dynamics of processing emotions from vocal, facial and bodily expressions. NeuroImage 2011, 58:665-674.
24. Jiang X., Li Y., Zhou X. Is it over-respectful or disrespectful? Differential patterns of brain activity in perceiving pragmatic violation of social status information during utterance comprehension. Neuropsychologia 2013, 51:2210-2223.
25. th International Conference in Speech Prosody (Social and Linguistic Speech Prosody) 2014, 573-576. ISSN: 2333-2042, Retrieved 04.12.14. http://fastnet.netsoc.ie/sp7/sp7book.pdf.
26. Kayser J., Tenke C. Optimizing PCA methodology for ERP component identification and measurement: theoretical rationale and empirical evaluation. Clinical Neurophysiology 2003, 114:2307-2325.
27. Kimble C., Seidel S. Vocal signs of confidence. Journal of Nonverbal Behavior 1991, 15:99-105.
28. Kolk H., Chwilla D. Late positivities in unusual situations. Brain and Language 2007, 100:257-261.
29. Kotz S., Paulmann S. When emotional prosody and semantics dance cheek to cheek: ERP evidence. Brain Research 2007, 1151:107-118.
30. Kotz S., Paulmann S. Emotion, language, and the brain. Language and Linguistics Compass 2011, 5:108-125.
31. Kotz S., Schwartze M. Cortical speech processing unplugged: a timely subcortico-cortical framework. Trends in Cognitive Sciences 2010, 14:392-399.
32. Kutas M., Federmeier K. Electrophysiology reveals semantic memory use in language comprehension. Trends in Cognitive Sciences 2000, 4:463-470.
33. Liu P., Pell D.M. Recognizing vocal emotions in Mandarin Chinese: a validated database of Chinese vocal emotional stimuli. Behavioral Research Method 2012, 44:1042-1051.
34. Liu T., Pinheiro A., Deng G., Nestor P., McCarley R., Niznikiewicz M. Electrophysiological insights into processing nonverbal emotional vocalizations. NeuroReport 2012, 23:108-112.
35. London H., McSeveney D., Tropper R. Confidence, overconfidence and persuasion. Human Relations 1971, 24:359-369.
36. London H., Meldman P., Lanckton A. The jury method: how the persuader persuades. Public Opinion Quarterly 1970, 34:171-183.
37. Lopez-Calderon J., Luck S. ERPLAB: an open-source toolbox for the analysis of event-related potentials. Frontiers in Human Neuroscience 2014, 8:213.
38. Luck S.J. Is it legitimate to compare conditions with different numbers of trials? 2010, Retrieved 04.12.14. http://erpinfo.org/Members/sjluck/Mean_Peak_Noise.pdf/view.
39. Makeig S., Bell A., Jung T., Sejnowski T. Independent component analysis of electroencephalographic data. Advances in Neural Information Processing Systems 8 1996, 145-151. MIT Press, Cambridge. D. Touretzky, M. Mozer, M. Hasselmo (Eds.).
40. Maslow C., Yoselson K., London H. Persuasiveness of confidence expressed via language and body language. British Journal of Clinical Psychology 1971, 10:234-240.
41. Mitchell R.L.C. Further characterisation of the functional neuroanatomy associated with prosodic emotion decoding. Cortex 2013, 49:1722-1732.
42. Mitchell R.L.C., Ross E. Attitudinal prosody: what we know and directions for future study. Neuroscience and Biobehavioral Reviews 2013, 37:471-479.
43. Molinaro N., Carreiras M. Electrophysiological evidence of interaction between contextual expectation and semantic integration during the processing of collocations. Biological Psychology 2010, 83:176-190.
44. Monetta L., Cheang H., Pell M.D. Understanding speaker attitudes from prosody by adults with Parkinson's disease. Journal of Neuropsychology 2008, 2:415-430.
45. Näätänen, R., Picton, T. The N1 wave of the human electric and magnetic response to sound. A review and analysis of the component structure 1987, 24:375-425.
46. Newman A., Tremblay A., Nichols E., Neville H., Ullman M. The influence of language proficiency on lexical semantic processing in native and late learners of English. Journal of Cognitive Neuroscience 2012, 24:1205-1223.
47. Pantev C., Elbert T., Ross B., Eulitz C., Terhardt E. Binaural fusion and the representation of virtual pitch in the human auditory cortex. Hearing Research 1996, 100:164-170.
48. Paulmann S., Bleichner M., Kotz S. Valence, arousal, and task effects in emotional prosody processing. Frontiers in Psychology 2013, 4:345.
49. Paulmann S., Kotz S. Early emotional prosody perception based on different speaker voices. NeuroReport 2008, 19:209-213.
50. Paulmann S., Kotz S. An ERP investigation on the temporal dynamics of emotional prosody and emotional semantics in pseudo-and lexical-sentence context. Brain and Language 2008, 105:59-69.
51. Paulmann S., Ott D., Kotz S. Emotional speech perception unfolding in time: the role of basal ganglia. PLoS ONE 2011, 6:e17694.
52. Pell M.D. Cerebral mechanisms for understanding emotional prosody in speech. Brain and Language 2006, 96:221-234.
53. Pell M.D. Reduced sensitivity to prosodic attitudes in adults with focal right hemisphere brain damage. Brain and Language 2007, 101:64-79.
54. Pell M.D., Leonard C.L. Processing emotional tone from speech in Parkinson's disease: a role for the basal ganglia. Cognitive, Affective, & Behavioral Neuroscience 2003, 3:275-288.
55. Picton T. The P300 wave of the human event-related potentials. Journal of Clinical Neurophysiology 1993, 9:456-479.
56. Pon-Barry H., Shieber S. Assessing self-awareness and transparency when classifying a speaker's level of certainty. Speech Prosody 2010.
57. Pourtois G., de Gelder B., Vroomen J., Rossion B., Crommelinck M. The time-course of intermodal binding between seeing and hearing affective information. NeuroReport 2000, 11:1329-1333.
58. Regel S., Coulson S., Gunter T. The communicative style of a speaker can affect language comprehension? ERP evidence from the comprehension of irony. Brain Research 2010, 1311:121-135.
59. Regel S., Gunter T., Friederici A. Isn't it ironic? An electrophysiological exploration of figurative language processing. Journal of Cognitive Neuroscience 2011, 23:277-293.
60. Regel S., Meyer L., Gunter T. Distinguishing neurocognitive processes reflected by P600 effects: evidence from ERPs and neural oscillations. PloS ONE 2014, 9:e96840.
61. Rigoulot S., Fish K., Pell M.D. Neural correlates of inferring speaker sincerity from white lies: an event-related potential source localization study. Brain Research 2014, 1565:48-62.
62. Sabbagh M., Baldwin D. Learning words from knowledgeable versus ignorant speakers: Links between preschoolers' theory of mind and semantic development. Child Development 2001, 72:1054-1070.
63. Sauter D.A., Eimer M. Rapid detection of emotion from human vocalizations. Journal of Cognitive Neuroscience 2010, 22:474-481.
64. Schact A., Sommer W. Time course and task dependence of emotion effects in word processing. Cognitive, Affective, & Behavioral Neuroscience 2009, 9:28-43.
65. Scherer K., London H., Wolf J. The voice of confidence: paralinguistic cues and audience evaluation. Journal of Research in Personality 1973, 7:31-44.
66. Schirmer A., Chen C., Ching A., Tan L., Hong R. Vocal emotions influence verbal memory: neural correlates and interindividual differences. Cognitive, Affective and Behavioral Neuroscience 2013, 13:80-93.
67. Schirmer A., Kotz S. Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing. Trends in Cognitive Sciences 2006, 10:25-30.
68. Schirmer A., Kotz S., Friederici A. Sex differentiates the role of emotional prosody during word processing. Cognitive Brain Research 2002, 14:228-233.
69. Schirmer A., Kotz S., Friederici A. On the role of attention for the processing of emotions in speech: sex differences revisited. Cognitive Brain Research 2005, 24:442-452.
70. Schirmer A., Lui M., Maess B., Escoffier N., Chan M., Penney T. Task and sex modulate the brain response to emotional incongruity in Asian listeners. Emotion 2006, 6:406-417.
71. Schirmer A., Simpson E., Escoffier N. Listen up! Processing of intensity change differs for vocal and nonvocal sounds. Brain Research 2007, 1176:103-112.
72. Schwartze M., Kotz S. A dual-pathway neural architecture for specific temporal prediction. Neuroscience and Biobehavioral Reviews 2013, 37:2587-2596.
73. Smith V., Clark H. On the course of answering questions. Journal of Memory and Language 1993, 32:25-38.
74. Spotorno N., Cheylus A., Van Der Henst J., Noveck I. What's behind a P600? Integration operations during irony processing. PloS ONE 2013, 8:1-10.
75. Spreckelmeyer K., Kutas M., Urbach T., Altenmüller E., Münte T. Neural processing of vocal emotion and identity. Brain and Cognition 2009, 69:121-126.
76. Strauß A., Kotz S., Obleser J. Narrowed expectancies under degraded speech: revisiting the N400. Journal of Cognitive Neuroscience 2013, 25:1384-1395.
77. Swerts M., Krahmer E. Audiovisual prosody and feeling of knowing. Journal of Memory and Language 2005, 53:81-94.
78. Tremblay A. LMERConvenienceFunctions: A suite of functions to back-fit fixed effects and forward-fit random effects, as well as other miscellaneous functions 2014, R-version 2.5. http://cran.r-project.org/web/packages/LMER%20ConvenienceFunctions/index.html.
79. Van Overwalle F., Van den Eede S., Baetens K., Vandekerckhove M. Trait inferences in goal-directed behavior: ERP timing and localization under spontaneous and intentional processing. Social, Cognitive and Affective Neuroscience 2009, 4:177-190.
80. Van den Brink D., Van Berkum J., Bastiaansen M., Tesink C., Kos M., Buitelaar J., et al. Empathy matters: ERP evidence for inter-individual differences in social language processing. Social, Cognitive & Affective Neuroscience 2012, 7:173-183.
81. Van der Cruyssen L., Van Duynslaeger M., Cortoos A., Van Overwalle F. ERP time course and brain areas of spontaneous and intentional goal inferences. Social Neuroscience 2009, 4:165-184.
82. Vespignani F., Canal P., Molinaro N., Fonda S., Cacciari C. Predictive mechanisms in idiom comprehension. Journal of Cognitive Neuroscience 2010, 22:1682-1700.
83. Walker M. The relative importance of verbal and nonverbal cues in the expression of confidence. Australian Journal of Psychology 1977, 29:45-57.
84. Wang L., Zhu Z., Bastiaansen M., Hagoort P., Yang Y. Recognizing the emotional valence of names: an ERP study. Brain and Language 2013, 125:118-127.
85. Wildgruber D., Ethofer T., Grandjean D., Kreifelts B. A cerebral network model of speech prosody. International Journal of Speech-Language Pathology 2009, 11:277-281.
86. Wilson D., Wharton T. Relevance and prosody. Journal of Pragmatics 2006, 38:1559-1579.
87. Zimmerman B., Ringle J. Effects of model persistence and statements of confidence on children's self-efficacy and problem solving. Journal of Educational Psychology 1981, 73:485-493.