Social cues, mentalizing and the neural processing of speech accompanied by gestures

Neuropsychologia

Volumn 48, Issue 2, 2010, Pages 382-393

  Straube, Benjamin ^a,b,c   Green, Antonia ^a,c   Jansen, Andreas ^c   Chatterjee, Anjan ^b   Kircher, Tilo ^c  

^a RWTH AACHEN UNIVERSITY   (Germany)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

^c PHILIPPS UNIVERSITY MARBURG   (Germany)

Author keywords

Body orientation; Emblematic gestures; Social cognition; Social relevance; Speech associated gestures

Indexed keywords

ADULT; ARTICLE; BODY MOVEMENT; BRAIN DEPTH STIMULATION; BRAIN FUNCTION; BRAIN REGION; COGNITION; CONTROLLED STUDY; FRONTAL LOBE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; GESTURE; HUMAN; HUMAN EXPERIMENT; IMAGE ANALYSIS; MALE; NORMAL HUMAN; OCCIPITAL LOBE; PARIETAL LOBE; PSYCHOMOTOR PERFORMANCE; SPEECH; TEMPORAL LOBE; VERBAL COMMUNICATION; VISUAL STIMULATION;

ADULT; ANALYSIS OF VARIANCE; BRAIN MAPPING; CEREBRAL CORTEX; CUES; FUNCTIONAL LATERALITY; GESTURES; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; ORIENTATION; OXYGEN; PHOTIC STIMULATION; SOCIAL PERCEPTION; SPEECH; YOUNG ADULT;

EID: 72449124627     PISSN: 00283932     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neuropsychologia.2009.09.025     Document Type: Article

References (59)

1. Amodio D.M., and Frith C.D. Meeting of minds: The medial frontal cortex and social cognition. Nature Reviews Neuroscience 7 (2006) 268-277
2. Baron-Cohen S., Ring H.A., Wheelwright S., Bullmore E.T., Brammer M.J., Simmons A., et al. Social intelligence in the normal and autistic brain: An fMRI study. European Journal of Neuroscience 11 (1999) 1891-1898
3. Bristow D., Rees G., and Frith C.D. Social interaction modifies neural response to gaze shifts. Social Cognitive and Affective Neuroscience 2 (2007) 52-61
4. Brunet E., Sarfati Y., Hardy-Bayle M.C., and Decety J. A PET investigation of the attribution of intentions with a nonverbal task. NeuroImage 11 (2000) 157-166
5. Castelli F., Happé F., Frith U., and Frith C. Movement and mind: A functional imaging study of perception and interpretation of complex intentional movement patterns. NeuroImage 12 (2000) 314-325
6. Dapretto M., Davies M.S., Pfeifer J.H., Scott A.A., Sigman M., Bookheimer S.Y., et al. Understanding emotions in others: Mirror neuron dysfunction in children with autism spectrum disorders. Nature Neuroscience 9 (2006) 28-30
7. Decety J., Jackson P.L., Sommerville J.A., Chaminade T., and Meltzoff A.N. The neural bases of cooperation and competition: An fMRI investigation. Neuroimage 23 (2004) 744-751
8. Dick A.S., Goldin-Meadow S., Hasson U., Skipper J.I., and Small S.L. Co-speech gestures influence neural activity in brain regions associated with processing semantic information. Human Brain Mapping (2009) 10.1002/hbm.20774
9. Efron, D. (1972). Gesture race and culture. The Hague: Mouton (Reprinted from Gesture and environment. New York: Kinǵs Crown Press, 1941).
10. Eickhoff S.B., Paus T., Caspers S., Grosbras M.H., Evans A.C., Zilles K., et al. Assignment of functional activations to probabilistic cytoarchitectonic areas revisited. Neuroimage 36 (2007) 511-521
11. Eickhoff S.B., Stephan K.E., Mohlberg H., Grefkes C., Fink G.R., Amunts K., et al. A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. Neuroimage 25 (2005) 1325-1335
12. Ekman P., and Friesen W. Nonverbal leakage and clues to deception. Psychiatry 32 (1969) 88-105
13. Fletcher P.C., Happé F., Frith U., Baker S.C., Dolan R.J., Frackowiak R.S., et al. Other minds in the brain: A functional imaging study of "theory of mind" in story comprehension. Cognition 57 (1995) 109-128
14. Friston K.J., Fletcher P., Josephs O., Holmes A., Rugg M.D., and Turner R. Event-related fMRI: Characterizing differential responses. Neuroimage 7 (1998) 30-40
15. Frith U. Mind blindness and the brain in autism. Neuron 32 (2001) 969-979
16. Frith U., and Frith C.D. Development and neurophysiology of mentalizing. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 358 1431 (2003) 459-473
17. Goldin-Meadow S. The role of gesture in communication and thinking. Trends in Cognitive Sciences 3 (1999) 419-429
18. Green A., Straube B., Weis S., Jansen A., Willmes K., Konrad K., et al. Neural integration of iconic and unrelated coverbal gestures: A functional MRI study. Human Brain Mapping 30 10 (2009) 3309-3324
19. Holle H., Gunter T.C., Rüschemeyer S.-A., Hennenlotter A., and Iacoboni M. Neural correlates of the processing of co-speech gestures. NeuroImage 39 (2008) 2010-2024
20. Holler J., and Beattie G. Pragmatic aspects of representational gestures: Do speakers use them to clarify verbal ambiguity for the listener?. Gesture 3 (2003) 127-154
21. Hubbard A.L., Wilson S.M., Callan D.E., and Dapretto M. Giving speech a hand: Gesture modulates activity in auditory cortex during speech perception. Human Brain Mapping 30 3 (2009) 1028-1037
22. Kampe K.K., Frith C.D., and Frith U. Hey John": Signals conveying communicative intention toward the self activate brain regions associated with "mentalizing," regardless of modality. Journal of Neuroscience 23 (2003) 5258-5263
23. Kelly, S. D., Creigh, P., & Bartolotti, J. (2009). Integrating speech and iconic gestures in a Stroop-like task: Evidence for automatic processing. Journal of Cognitive Neuroscience, doi:10.1162/jocn.2009.21254.
24. Kendon A. Gesture. Annual Review of Anthropology 26 (1997) 109-128
25. Kendon A. Some contrasts in gesticulation in Neapolitan speakers and speakers in Northamptonshire. In: Posner R., and Mueller C. (Eds). The semantics and pragmatics of everyday gesture (2004), WeidlerBuchverlag, Berlin 173-193
26. Kilner J.M., Marchant J.L., and Frith C.D. Modulation of the mirror system by social relevance. Social Cognitive and Affective Neuroscience 1 (2006) 143-148
27. Kircher T., Straube B., Leube D., Weis S., Sachs O., Willmes K., et al. Neural interaction of speech and gesture: Differential activations of metaphoric co-verbal gestures. Neuropsychologia 47 (2009) 169-179
28. Kircher T., Blumel I., Marjoram D., Lataster T., Krabbendam L., Weber J., et al. Online mentalising investigated with functional MRI. Neuroscience Letters 454 (2009) 176-181
29. Krach S., Blümel I., Marjoram D., Lataster T., Krabbendam L., Weber J., et al. Are women better mindreaders? Sex differences in neural correlates of mentalizing detected with functional MRI. BMC Neurosciences 10 (2009) 9
30. Krach S., Hegel F., Wrede B., Sagerer G., Binkofski F., and Kirchere T. Can machines think? Interaction and perspective taking with robots investigated via fMRI. PLoS ONE 3 7 (2008) e2597 10.1371/journal.pone.0002597
31. Lambert N.A., Swain M.A., Miller L.A., and Caine D. Exploring the neural organization of person-related knowledge: Lateralization of lesion, category specificity, and stimulus modality effects. Neuropsychology 20 (2006) 346-354
32. Lawrence E.J., Shaw P., Giampietro V.P., Surguladze S., Brammer M.J., and David A.S. The role of 'shared representations' in social perception and empathy: An fMRI study. Neuroimage 29 (2006) 1173-1184
33. Lieberman M.D. Social cognitive neuroscience: A review of core processes. Annu Annual Review of Psychology 58 (2007) 259-289
34. Lotze M., Heymans U., Birbaumer N., Veit R., Erb M., Flor H., et al. Differential cerebral activation during observation of expressive gestures and motor acts. Neuropsychologia 44 (2006) 1787-1795
35. Macmillan N.A., and Creelman C.D. Detection theory: A user's guide (1991), Cambridge University Press, New York
36. Mason M.F., Banfield J.F., and Macrae C.N. Thinking about actions: The neural substrates of person knowledge. Cerebral Cortex 14 (2004) 209-214
37. McNeill D. Hand and mind: What gestures reveal about thought (1992), University of Chicago Press, Chicago, IL
38. McNeill D. Gesture and thought (2005), University of Chicago Press, Chicago, IL
39. Mitchell J.P., Heatherton T.F., and Macrae C.N. Distinct neural systems subserve person and object knowledge. Proceedings of the National Academy of Sciences United States of America 99 (2002) 15238-15243
40. Montgomery K.J., Isenberg N., and Haxby J.V. Communicative hand gestures and object-directed handmovements activated the mirror neuron system. Social Cognitive and Affective Neuroscience 2 (2007) 114-122
41. Nakamura K., Oga T., Okada T., Sadato N., Takayama Y., Wydell T., et al. Hemispheric asymmetry emerges at distinct parts of the occipitotemporal cortex for objects, logograms and phonograms: A functional MRI study. NeuroImage 28 (2005) 521-528
42. Noordzij M.L., Newman-Norlund S.E., de Ruiter J.P., Hagoort P., Levinson S.C., and Toni I. Brain mechanisms underlying human communication. Frontiers in Human Neuroscience 3 (2009) 14 10.3389/neuro.09.014.2009
43. Oldfield R.C. The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia 9 (1971) 97-113
44. Oliver R.T., and Thompson-Schill S.L. Dorsal stream activation during retrieval of object size and shape. Cognitive and Affective Behavioural Neuroscience 3 4 (2003) 309-322
45. Olson I.R., Plotzker A., and Ezzyat Y. The Enigmatic temporal pole: A review of findings on social and emotional processing. Brain 130 (2007) 1718-1731
46. Özyürek A. Do speakers design their cospeech gestures for their addressees? The effects of addressee location on representational gestures. Journal of Memory and Language 46 (2002) 688-704
47. Pilgrim L.K., Moss H.E., and Tyler L.K. Semantic processing of living and nonliving concepts across the cerebral hemispheres. Brain and Language 94 (2005) 86-93
48. Rilling J.K., Sanfey A.G., Aronson J.A., Nystrom L.E., and Cohen J.D. The neural correlates of theory of mind within interpersonal interactions. Neuroimage 22 (2004) 1694-1703
49. Schulte-Ruther M., Markowitsch H.J., Fink G.R., and Piefke M. Mirror neuron and theory of mind mechanisms involved in face-to-face interactions: A functional magnetic resonance imaging approach to empathy. Journal of Cognitive Neuroscience 19 (2007) 1354-1372
50. Shen L., Hu X., Yacoub E., and Ugurbil K. Neural correlates of visual form and visuo-spatial processing. Human Brain Mapping 8 (1999) 60-71
51. Skipper J.I., Goldin-Meadow S., Nusbaum H.C., and Small S.L. Gestures orchestrate brain networks for language understanding. Current Biology 19 (2009) 661-667
52. Slotnick S.D., Moo L.R., Segal J.B., and Hart Jr. J. Distinct prefrontal cortex activity associated with item memory and source memory for visual shapes. Brain Research: Cognitive Brain Research 17 (2003) 75-82
53. Straube B., Green A., Weis S., Chatterjee A., and Kircher T. Memory effects of speech and gesture binding: Cortical and hippocampal activation in relation to subsequent memory performance. Journal of Cognitive Neuroscience 21 (2009) 821-836
54. Vogeley K., Bussfeld P., Newen A., Herrmann S., Happé F., Falkai P., et al. Mind reading: Neural mechanisms of theory of mind and self-perspective. NeuroImage 14 (2001) 170-181
55. Weiss P.H., Rahbari N.N., Lux S., Pietrzyk U., Noth J., and Fink G.R. Processing the spatial configuration of complex actions involves right posterior parietal cortex: An fMRI study with clinical implications. Human Brain Mapping 27 (2006) 1004-1014
56. Willems R.M., Ozyurek A., and Hagoort P. when language meets action: The neural integration of gesture and speech. Cerebral Cortex 17 (2007) 2322-2333
57. Willems R.M., Ozyurek A., and Hagoort P. Differential roles for left inferior frontal and superior temporal cortex in multimodal integration of action and language. Neuroimage 47 4 (2009) 1992-2004
58. Yoon C., Gutchess A.H., Feinberg F., and Polk T.A. A functional magnetic resonance imaging study of neural dissociations between brain and person judgments. Journal of Consumer Research 33 (2006) 31-40
59. Zahn R., Moll J., Krueger F., Huey E.D., Garrido G., and Grafman J. Social concepts are represented in the superior anterior temporal cortex. Proceedings of the National Academy of Sciences United States of America 104 (2007) 6430-6435