Integrating speech and iconic gestures in a stroop-like task: Evidence for automatic processing

Journal of Cognitive Neuroscience

Volumn 22, Issue 4, 2010, Pages 683-694

  Kelly, Spencer D ^a   Creigh, Peter ^b   Bartolotti, James ^a  

^a Assistant Professor in the University Libraries   (United States)

^b UNIVERSITY OF PITTSBURGH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; BEHAVIOR; CONTROLLED STUDY; ELECTROPHYSIOLOGY; EVENT RELATED POTENTIAL; FEMALE; GESTURE; HUMAN; HUMAN EXPERIMENT; INFORMATION PROCESSING; LANGUAGE PROCESSING; MALE; PRIORITY JOURNAL; SPEECH PERCEPTION; TASK PERFORMANCE;

ANALYSIS OF VARIANCE; AUTOMATIC DATA PROCESSING; BRAIN; BRAIN MAPPING; ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS, AUDITORY; FEMALE; FUNCTIONAL LATERALITY; GESTURES; HUMANS; MALE; PHOTIC STIMULATION; PSYCHOLINGUISTICS; REACTION TIME; SEX FACTORS; SPEECH; SPEECH PERCEPTION; YOUNG ADULT;

EID: 77649204100     PISSN: 0898929X     EISSN: 15308898     Source Type: Journal    
DOI: 10.1162/jocn.2009.21254     Document Type: Article

References (61)

1. Armstrong, D. F., & Wilcox, S. E. (2007). The gestural origin of language. New York: Oxford University Press.
2. Bargh, J. A., & Morsella, E. (2008). The unconscious mind. Perspectives on Psychological Science, 3, 73-79.
3. Bates, E., Benigni, L., Bretherton, I., Camaioni, L., & Volterra, V. (1979). The emergence of symbols: Cognition and communication in infancy. New York: Academic Press.
4. Bates, E., & Dick, F. (2002). Language, gesture, and the developing brain. Developmental Psychobiology, 40, 293-310.
5. Beattie, G., & Shovelton, H. (1999). Do iconic hand gestures really contribute anything to the semantic information conveyed by speech? An experimental investigation. Semiotica, 123, 1-30.
6. Bernardis, P., Salillas, E., & Caramelli, N. (2008). Behavioural and neurophysiological evidence of semantic interaction between iconic gestures and words. Cognitive Neuropsychology, 25, 1114-1128.
7. Bessner, D., & Stolz, J. A. (1999). Unconsciously controlled processing: The Stroop effect reconsidered. Psychonomic Bulletin & Review, 6, 99-104.
8. Blakemore, S. J., & Decety, J. (2001). From the perception of action to the understanding of intention. Nature Reviews Neuroscience, 2, 561-567.
9. Brown, C., & Hagoort, P. (1993). The processing nature of the N400: Evidence from masked priming. Journal of Cognitive Neuroscience, 5, 34-44.
10. Cassell, J., McNeill, D., & McCullough, K. E. (1999). Speech-gesture mismatches: Evidence for one underlying representation of linguistic and nonlinguistic information. Pragmatics & Cognition, 7, 1-34.
11. Clark, H. H. (1996). Using language. Cambridge, England: Cambridge University Press.
12. Corballis, M. C. (2003). From hand to mouth: The origins of language. Princeton, NJ: Princeton University Press.
13. de Gelder, B. (2006). Towards the neurobiology of emotional body language. Nature Reviews Neuroscience, 7, 242-249.
14. Goldin-Meadow, S. (2003). Hearing gesture: How our hands help us think. Cambridge, MA: Belknap Press.
15. Goldin-Meadow, S., Wein, D., & Chang, C. (1992). Assessing knowledge through gesture: Using children's hands to read their minds. Cognition and Instruction, 9, 201-219.
16. Gratton, G., Coles, M. G. H., & Donchin, E. (1983). A new method for off-line removal of ocular artifacts. Electroencephalography and Clinical Neurophysiology, 55, 468-484.
17. Holcomb, P. (1993). Semantic priming and stimulus degradation: Implications for the role of the N400 in language processing. Psychophysiology, 30, 47-61.
18. Holle, H., & Gunter, T. C. (2007). The role of iconic gestures in speech disambiguation: ERP evidence. Journal of Cognitive Neuroscience, 19, 1175-1192.
19. Holle, H., Gunter, T., Rüschemeyer, S. A., Hennenlotter, A., & Iacoboni, M. (2008). Neural correlates of the processing of co-speech gestures. Neuroimage, 39, 2010-2024.
20. Howell, D. C. (2002). Statistical methods for psychology (5th ed.). Belmont, CA: Duxberry Press.
21. Hubbard, A. L., Wilson, S. M., Callan, D. E., & Dapretto, M. (2009). Giving speech a hand: Gesture modulates activity in auditory cortex during speech perception. Human Brain Mapping, 30, 1028-1037.
22. Iacoboni, M., Molnar-Szakacs, I., Gallese, V., Buccino, G., Mazziotta, J. C., & Rizzolatti, G. (2005). Grasping intentions of others with one's own mirror neuron system. PloS Biology, 3, 529-535.
23. Kelly, S. D., Barr, D., Church, R. B., & Lynch, K. (1999). Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory. Journal of Memory and Language, 40, 577-592.
24. Kelly, S. D., & Church, R. B. (1998). A comparison between children's and adults' ability to detect children's representational gestures. Child Development, 69, 85-93.
25. Kelly, S. D., Iverson, J., Terranova, J., Niego, J., Hopkins, M., & Goldsmith, L. (2002). Putting language back in the body: Speech and gesture on three timeframes. Developmental Neuropsychology, 22, 323-349.
26. Kelly, S. D., Kravitz, C., & Hopkins, M. (2004). Neural correlates of bimodal speech and gesture comprehension. Brain and Language, 89, 253-260.
27. Kelly, S. D., Ward, S., Creigh, P., & Bartolotti, J. (2007). An intentional stance modulates the integration of gesture and speech during comprehension. Brain and Language, 101, 222-233.
28. Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge: Cambridge University Press.
29. Kiefer, M., & Brendel, D. (2006). Attentional modulation of unconscious "automatic" processes: Evidence from event-related potentials in a masked priming paradigm. Journal of Cognitive Neuroscience, 18, 184-198.
30. Kita, S., & Özyürek, A. (2003). What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language, 48, 16-32.
31. Krauss, R. M. (1998). Why do we gesture when we speak? Current Directions in Psychological Science, 7, 54-59.
32. Krauss, R. M., Morrel-Samuels, P., & Colasante, C. (1991). Do conversational hand gestures communicate? Journal of Personality and Social Psychology, 61, 743-754.
33. Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207, 203-204.
34. Langton, S. R. H., & Bruce, V. (2000). You must see the point: Automatic processing of cues to the direction of social attention. Journal of Experimental Psychology: Human Perception and Performance, 26, 747-757.
35. Logan, G. D. (1978). Attention in character classification: Evidence for the automaticity of component stages. Journal of Experimental Psychology: General, 107, 32-63.
36. MacLeod, C. M. (1991). Half a century of research on the Stroop effect: An integrative review. Psychological Bulletin, 109, 163-203.
37. MacWhinney, B., & Bates, E. (1989). The crosslinguistic study of sentence processing. New York: Cambridge University Press.
38. McNeill, D. (1992). Hand and mind: What gesture reveals about thoughts. Chicago: University of Chicago Press.
39. McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press.
40. McPherson, W. B., & Holcomb, P. J. (1999). An electrophysiological investigation of semantic priming with pictures of real objects. Psychophysiology, 36, 53-65.
41. Miller, G. A., Gratton, G., & Yee, C. M. (1988). Generalized implementation of an eye movement correction procedure. Psychophysiology, 25, 241-243.
42. Montgomery, K. J., Isenberg, N., & Haxby, J. V. (2007). Communicative hand gestures and object-directed hand movements activated the mirror neuron system. Social Cognitive & Affective Neuroscience, 2, 114-122.
43. Nishitani, N., Schurmann, M., Amunts, K., & Hari, R. (2005). Broca's region: From action to language. Physiology, 20, 60-69.
44. Özyürek, A., & Kelly, S. D. (2007). Gesture, brain, and language. Brain and Language, 101, 181-184.
45. Özyürek, A., Willems, R. M., Kita, S., & Hagoort, P. (2007). On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience, 19, 605-616.
46. Posner, M. I., & Snyder, C. R. R. (1975). Attention and cognitive control. In R. L. Solso (Ed.), Information processing and cognition: The Loyola symposium (pp. 55-85). Hillsdale, NJ: Lawrence Erlbaum.
47. Pourtois, G., de Gelder, B., Vroomen, J., Rossion, B., & Crommelinck, M. (2000). The time-course of intermodal binding between hearing and seeing affective information. NeuroReport, 11, 1329-1333.
48. Rizzolatti, G., & Arbib, M. A. (1998). Language within our grasp. Trends in Neurosciences, 21, 188-194.
49. Rizzolatti, G., & Craighero, L. (2004). The mirror-neuron system. Annual Review of Neuroscience, 27, 169-192.
50. Ruz, M., Madrid, E., Lupiánez, J., & Tudela, P. (2003). High density ERP indices of conscious and unconscious semantic priming. Cognitive Brain Research, 17, 719-731.
51. Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information processing: Detection, search, and attention. Psychological Review, 84, 1-66.
52. Skipper, J. I., Goldin-Meadow, S., Nusbaum, H. C., & Small, S. L. (2007). Speech-associated gestures, Broca's area, and the human mirror system. Brain and Language, 101, 260-277.
53. Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18, 643-662.
54. Van Berkum, J. J. J., van den Brink, D., Tesink, C., Kos, M., & Hagoort, P. (2008). The neural integration of speaker and message. Journal of Cognitive Neuroscience, 20, 580-591.
55. Willems, R. M., & Hagoort, P. (2007). Neural evidence for the interplay between language, gesture, and action: A review. Brain and Language, 101, 278-289.
56. Willems, R. M., Özyürek, A., & Hagoort, P. (2007). When language meets action: The neural integration of gesture and speech. Cerebral Cortex, 17, 2322-2333.
57. Willems, R. M., Özyürek, A., & Hagoort, P. (2008). Seeing and hearing meaning: ERP and fMRI evidence of word versus picture integration into a sentence context. Journal of Cognitive Neuroscience, 20, 1235-1249.
58. Wilson, S. M., Molnar-Szakacs, I., & Iacoboni, M. (2008). Beyond superior temporal cortex: Intersubject correlations in narrative speech comprehension. Cerebral Cortex, 18, 230-242.
59. Winston, J. S., Strange, B. A., O?Doherty, J., & Dolan, R. J. (2002). Automatic and intentional brain responses during evaluation of trustworthiness of faces. Nature Neuroscience, 5, 277-283.
60. Wu, Y. C., & Coulson, S. (2007a). Iconic gestures prime related concepts: An ERP study. Psychonomic Bulletin & Review, 14, 57-63.
61. Wu, Y. C., & Coulson, S. (2007b). How iconic gestures enhance communication: An ERP study. Brain and Language, 101, 234-245.