Synchronization of speech and gesture: Evidence for interaction in action

Journal of Experimental Psychology: General

Volumn 143, Issue 4, 2014, Pages 1726-1741

  Chu, Mingyuan ^a   Hagoort, Peter ^a,b  

^a MAX PLANCK INSTITUTE FOR PSYCHOLINGUISTICS   (Netherlands)

^b RADBOUD UNIVERSITY   (Netherlands)

Author keywords

Forward model; Gesture speech synchronization; Interactive view; Pointing gesture; Virtual reality

Indexed keywords

ADOLESCENT; FEMALE; GESTURE; HUMAN; INTERPERSONAL COMMUNICATION; LANGUAGE; MALE; SPEECH; YOUNG ADULT;

ADOLESCENT; COMMUNICATION; FEMALE; GESTURES; HUMANS; LANGUAGE; MALE; SPEECH; YOUNG ADULT;

EID: 84905124753     PISSN: 00963445     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0036281     Document Type: Article

References (32)

1. Chu, M., & Kita, S. (2008). Spontaneous gestures during mental rotation tasks: Insights into the microdevelopment of the motor strategy. Journal of Experimental Psychology: General, 137, 706-723. doi:10.1037/ a0013157
2. Dell, G. S., Schwartz, M. F., Martin, N., Saffran, E. M., & Gagnon, D. A. (1997). Lexical access in aphasic and nonaphasic speakers. Psychological Review, 104, 801-838. doi:10.1037/0033-295X.104.4.801
3. De Ruiter, J. P. A. (1998). Gesture and speech production (Unpublished doctoral dissertation). University of Nijmegen, Nijmegen, the Netherlands.
4. Feyereisen, P. (1997). The competition between gesture and speech production in dual-task paradigms. Journal of Memory and Language, 36, 13-33. doi:10.1006/jmla.1995.2458
5. Feyereisen, P., & de Lannoy, J. (1991). Gestures and speech: Psychological investigations. New York, NY: Cambridge University Press.
6. Franklin, D. W., & Wolpert, D. M. (2011). Computational mechanisms of sensorimotor control. Neuron, 72, 425-442. doi:10.1016/j.neuron.2011 .10.006
7. Gazzaniga, M. S., Ivry, R. B., & Mangun, G. R. (2008). Cognitive neuroscience: The biology of the mind (3rd ed.). New York, NY: Norton.
8. Habets, B., Kita, S., Shao, Z., Ozyurek, A., & Hagoort, P. (2011). The role of synchrony and ambiguity in speech- gesture integration during comprehension. Journal of Cognitive Neuroscience, 23, 1845-1854. doi: 10.1162/jocn.2010.21462
9. Hagoort, P., & van Turennout, M. (1997). The electrophysiology of speaking: Possibilities of event-related potential research for speech production.In W. Hulstijn, H. Peters, & P. Van Lieshout (Eds.), Speech motor production and fluency disorders: Brain research in speech production (pp. 351-361). Amsterdam, the Netherlands: Elsevier.
10. Hickok, G., Houde, J., & Rong, F. (2011). Sensorimotor integration in speech processing: Computational basis and neural organization. Neuron, 69, 407-422. doi:10.1016/j.neuron.2011.01.019
11. Indefrey, P., & Levelt, W. J. M. (2004). The spatial and temporal signatures of word production components. Cognition, 92, 101-144. doi: 10.1016/j.cognition.2002.06.001
12. Iverson, J. M., & Goldin-Meadow, S. (2005). Gesture paves the way for language development. Psychological Science, 16, 367-371. doi: 10.1111/j.0956-7976.2005.01542.x
13. Iverson, J. M., & Thelen, E. (1999). Hand, mouth, and brain: The dynamic emergence of speech and gesture. Journal of Consciousness Studies, 6, 19-40.
14. Kandel, E., Schwartz, J., Jessell, T., Siegelbaum, S., & Hudspeth, A. (2012). Principles of neural science (5th ed.). New York, NY: McGraw Hill Medical.
15. Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge, United Kingdom: Cambridge University Press.
16. Kita, S. (2009). Cross-cultural variation of speech-accompanying gesture: A review. Language and Cognitive Processes, 24, 145-167. doi: 10.1080/01690960802586188
17. Kita, S., & Özyürek, A. (2003). What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language, 48, 16 -32. doi:10.1016/S0749-596X (02)00505-3
18. Krauss, R. M., Chen, Y., & Chawla, P. (1996). Nonverbal behaviour and nonverbal communication: What do conversational hand gestures tell us? Advances in Experimental Social Psychology, 28, 389-450. doi: 10.1016/S0065-2601(08)60241-5
19. Krauss, R. M., Chen, Y., & Gottesman, R. F. (2000). Lexical gestures and lexical access: A process model. In D. McNeill (Ed.), Language and gesture (pp. 261-283). Cambridge, United Kingdom: Cambridge University Press.
20. Levelt, W. J. M. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press.
21. Levelt, W. J. M., Richardson, G., & La Heij, W. (1985). Pointing and voicing in deictic expressions. Journal of Memory and Language, 24, 133-164. doi:10.1016/0749-596X(85)90021-X
22. Levelt, W. J. M., Roelofs, A., & Meyer, A. S. (1999). A theory of lexical access in speech production. Behavioral and Brain Sciences, 22, 1-38. doi:10.1017/S0140525X99001776
23. Loehr, D. (2007). Aspects of rhythm in gesture and speech. Gesture, 7, 179-214. doi:10.1075/gest.7.2.04loe
24. McClave, E. (1998). Pitch and manual gestures. Journal of Psycholinguistic Research, 27, 69-89. doi:10.1023/A:1023274823974
25. McNeill, D. (1992). Hand and mind. Chicago, IL: University of Chicago Press.
26. McNeill, D. (2005). Gesture and thought. Chicago, IL: University of Chicago Press.
27. McNeill, D. (2012). How language began: Gesture and speech in human evolution. Cambridge, United Kingdom: Cambridge University Press.
28. McNeill, D., & Duncan, S. D. (2000). Growth points in thinking-forspeaking. In D. McNeill (Ed.), Language and gesture (pp. 141-161). Cambridge, United Kingdom: Cambridge University Press.
29. Miall, R. C., Weir, D. J., Wolpert, D. M., & Stein, J. F. (1993). Is the cerebellum a Smith predictor? Journal of Motor Behaviour, 25, 203- 216. doi:10.1080/00222895.1993.9942050
30. Rapp, B., & Goldrick, M. (2000). Discreteness and interactivity in spoken word production. Psychological Review, 107, 460-499. doi:10.1037/ 0033-295X.107.3.460
31. Tuite, K. (1993). The production of gesture. Semiotica, 93, 83-105. doi: 10.1515/semi.1993.93.1-2.83
32. Welch, R. B., & Warren, D. H. (1986). Intersensory interactions. In K. R. Boff, L. Kaufman, & J. P. Thomas (Ed.), Handbook of perception and human performance: Vol. 1. Sensory processes and perception (pp. 25.1-25.36). New York, NY: Wiley.