Links between gestures and multisensory processing: Individual differences suggest a compensation mechanism

Frontiers in Psychology

Volumn 8, Issue OCT, 2017, Pages

  Schmalenbach, Simon B ^a   Billino, Jutta ^b   Kircher, Tilo ^a   van Kemenade, Bianca M ^a   Straube, Benjamin ^a  

^a PHILIPPS UNIVERSITY MARBURG   (Germany)

^b JUSTUS LIEBIG UNIVERSITY   (Germany)

Author keywords

Action perception; Communication; Gesture; Individual differences; Multisensory processing

Indexed keywords

EID: 85031429313     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2017.01828     Document Type: Article

References (38)

1. Blakemore, S. J., Wolpert, D. M., and Frith, C. D. (1998). Central cancellation of self-produced tickle sensation. Nat. Neurosci. 1, 635-640. doi: 10.1038/2870
2. Chu, M., and Kita, S. (2011). The nature of gestures' beneficial role in spatial problem solving. J. Exp. Psychol. Gen. 140, 102-116. doi: 10.1037/a0021790
3. Chu, M., Meyer, A., Foulkes, L., and Kita, S. (2014). Individual differences in frequency and saliency of speech- accompanying gestures: the role of cognitive abilities and empathy. J. Exp. Psychol. Gen. 143, 694-709. doi: 10.1037/a0033861
4. Cullen, K. E. (2004). Sensory signals during active versus passive movement. Curr. Opin. Neurobiol. 14, 698-706. doi: 10.1016/j.conb.2004.10.002
5. Ernst, M. O., and Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429-433. doi: 10.1038/415429a
6. Ernst, M. O., and Bulthoff, H. H. (2004). Merging the senses into a robust percept. Trends Cogn. Sci. 8, 162-169. doi: 10.1016/j.tics.2004.02.002
7. Farrer, C., Bouchereau, M., Jeannerod, M., and Franck, N. (2008). Effect of distorted visual feedback on the sense of agency. Behav. Neurol. 19, 53-57. doi: 10.1155/2008/425267
8. Ford, J. M., Gray, M., Faustman, W. O., Heinks, T. H., and Mathalon, D. H. (2005). Reduced gamma-band coherence to distorted feedback during speech when what you say is not what you hear. Int. J. Psychophysiol. 57, 143-150. doi: 10.1016/j.ijpsycho.2005.03.002
9. Frund, I., Haenel, N. V., and Wichmann, F. A. (2011). Inference for psychometric functions in the presence of nonstationary behavior. J. Vis. 11:16. doi: 10.1167/11.6.16
10. Gallese, V., and Goldman, A. (1998). Mirror neurons and the simulation theory of mind-reading. Trends Cogn. Sci. 2, 493-501. doi: 10.1016/S1364-6613(98)01262-5
11. Goldin-Meadow, S., and Alibali, M. W. (2013). Gesture's role in speaking, learning, and creating language. Annu. Rev. Psychol. 64, 257-283. doi: 10.1146/annurev-psych-113011-143802
12. Goldin-Meadow, S., Nusbaum, H., Kelly, S. D., and Wagner, S. (2001). Explaining math: gesturing lightens the load. Psychol. Sci. 12, 516-522. doi: 10.1111/1467-9280.00395
13. Holle, H., Obermeier, C., Schmidt-Kassow, M., Friederici, A. D., Ward, J., and Gunter, T. C. (2012). Gesture facilitates the syntactic analysis of speech. Front. Psychol. 3:74. doi: 10.3389/fpsyg.2012.00074
14. Holle, H., Obleser, J., Rueschemeyer, S.-A., and Gunter, T. C. (2010). Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions. Neuroimage 49, 875-884. doi: 10.1016/j.neuroimage.2009.08.058
15. Hoover, A. E. N., and Harris, L. R. (2012). Detecting delay in visual feedback of an action as a monitor of self recognition. Exp. Brain Res. 222, 389-397. doi: 10.1007/s00221-012-3224-3
16. Hostetter, A. B., and Alibali, M. W. (2007). Raise your hand if you're spatial: Relations between verbal and spatial skills and gesture production. Gesture 7, 73-95. doi: 10.1075/gest.7.1.05hos
17. Hostetter, A. B., and Alibali, M. W. (2008). Visible embodiment: gestures as simulated action. Psychon. Bull. Rev. 15, 495-514. doi: 10.3758/PBR.15.3.495
18. Jeannerod, M. (2001). Neural simulation of action: a unifying mechanism for motor cognition. Neuroimage 14(1 Pt 2), S103-S109. doi: 10.1006/nimg.2001.0832
19. Kelly, S. D. (2001). Broadening the units of analysis in communication: speech and nonverbal behaviours in pragmatic comprehension. J. Child Lang. 28, 325-349. doi: 10.1017/S0305000901004664
20. Leube, D. T., Knoblich, G., Erb, M., Grodd, W., Bartels, M., and Kircher, T. T. J. (2003). The neural correlates of perceiving one's own movements. Neuroimage 20, 2084-2090. doi: 10.1016/j.neuroimage.2003.07.033
21. Marstaller, L., and Burianova, H. (2013). Individual differences in the gesture effect on working memory. Psychon. Bull. Rev. 20, 496-500. doi: 10.3758/s13423-012-0365-0
22. McDonald, J. J., Teder-Salejarvi, W. A., and Hillyard, S. A. (2000). Involuntary orienting to sound improves visual perception. Nature 407, 906-908. doi: 10.1038/35038085
23. Nagels, A., Kircher, T., Steines, M., Grosvald, M., and Straube, B. (2015). A brief self-rating scale for the assessment of individual differences in gesture perception and production. Learn. Individ. Differ. 39, 73-80. doi: 10.1016/j.lindif.2015.03.008
24. Obermeier, C., Dolk, T., and Gunter, T. C. (2012). The benefit of gestures during communication: evidence from hearing and hearing-impaired individuals. Cortex 48, 857-870. doi: 10.1016/j.cortex.2011.02.007
25. Pascual-Leone, A., Dang, N., Cohen, L. G., Brasil-Neto, J. P., Cammarota, A., and Hallett, M. (1995). Modulation of muscle responses evoked by transcranial magnetic stimulation during the acquisition of new fine motor skills. J. Neuophysiol. 74, 1037-1045.
26. Pouw, W., and Hostetter, A. B. (2016). Gesture as predictive action. Reti Saperi Lingauggi Ital. J. Cogn. Sci. 3, 57-80. doi: 10.12832/83918
27. Pouw, W. T. J. L., de Nooijer, J. A., van Gog, T., Zwaan, R. A., and Paas, F. (2014). Toward a more embedded/extended perspective on the cognitive function of gestures. Front. Psychol. 5:359. doi: 10.3389/fpsyg.2014.00359
28. Rohde, M., and Ernst, M. O. (2013). To lead and to lag - forward and backward recalibration of perceived visuo-motor simultaneity. Front. Psychol. 3:599. doi: 10.3389/fpsyg.2012.00599
29. Shimada, S., Qi, Y., and Hiraki, K. (2010). Detection of visual feedback delay in active and passive self-body movements. Exp. Brain Res. 201, 359-364. doi: 10.1007/s00221-009-2028-6
30. Straube, B., Green, A., Weis, S., Chatterjee, A., and Kircher, T. (2009). Memory effects of speech and gesture binding: cortical and hippocampal activation in relation to subsequent memory performance. J. Cogn. Neurosci. 21, 821-836. doi: 10.1162/jocn.2009.21053
31. Straube, B., van Kemenade, B. M., Arikan, B. E., Fiehler, K., Leube, D. T., Harris, L. R., et al. (2017). Predicting the multisensory consequences of one's own action: BOLD suppression in auditory and visual cortices. PLOS ONE 12:e0169131. doi: 10.1371/journal.pone.0169131
32. Valenzeno, L., Alibali, M. W., and Klatzky, R. (2003). Teachers' gestures facilitate students' learning: a lesson in symmetry. Contemp. Educ. Psychol. 28, 187-204. doi: 10.1016/S0361-476X(02)00007-3
33. van Kemenade, B. M., Arikan, B. E., Kircher, T., and Straube, B. (2016). Predicting the sensory consequences of one's own action: first evidence for multisensory facilitation. Attent. Percept. Psychophys. 78, 2515-2526. doi: 10.3758/s13414-016-1189-1
34. von Holst, E., and Mittelstaedt, H. (1950). Das reafferenzprinzip: wechselwirkungen zwischen zentralnervensystem und peripherie. Naturwissenschaften 37, 464-476. doi: 10.1007/BF00622503
35. Wang, J., Mathalon, D. H., Roach, B. J., Reilly, J., Keedy, S. K., Sweeney, J. A., et al. (2014). Action planning and predictive coding when speaking. Neuroimage 91, 91-98. doi: 10.1016/j.neuroimage.2014.01.003
36. Wolpert, D. M., and Flanagan, J. R. (2001). Motor prediction. Curr. Biol. 11, R729-R732. doi: 10.1016/S0960-9822(01)00432-8
37. Wolpert, D. M., Ghahramani, Z., and Jordan, M. I. (1995). An internal model for sensorimotor integration. Science 269, 1880-1882. doi: 10.1126/science.7569931
38. Wolpert, D. M., and Miall, R. C. (1996). Forward models for physiological motor control. Neural Netw. 9, 1265-1279. doi: 10.1016/S0893-6080(96)00035-4