The activation of modality-specific representations during discourse processing

Brain and Language

Volumn 126, Issue 3, 2013, Pages 338-349

  Kurby, Christopher A ^a   Zacks, Jeffrey M ^b  

^a GRAND VALLEY STATE UNIVERSITY   (United States)

^b WASHINGTON UNIVERSITY   (United States)

Author keywords

Discourse comprehension; Imagery; Language comprehension; Neuroimaging; Perceptual simulation

Indexed keywords

ADULT; ARTICLE; AUDITORY STIMULATION; BRAIN FUNCTION; BRAIN REGION; CENTRAL SULCUS; COMPREHENSION; CONTROLLED STUDY; DISCOURSE ANALYSIS; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; IMAGERY; LANGUAGE PROCESSING; MALE; MOTOR PERFORMANCE; NERVE STIMULATION; NEUROIMAGING; NORMAL HUMAN; POSTCENTRAL SULCUS; PRECENTRAL SULCUS; SENSORIMOTOR FUNCTION; STIMULUS RESPONSE; SUPRASYLVIAN GYRUS; TEMPORAL GYRUS; VISUAL STIMULATION;

DISCOURSE COMPREHENSION; IMAGERY; LANGUAGE COMPREHENSION; NEUROIMAGING; PERCEPTUAL SIMULATION;

ADULT; AUDITORY CORTEX; BRAIN MAPPING; COMPREHENSION; FEMALE; FRONTAL LOBE; HUMANS; IMAGINATION; MAGNETIC RESONANCE IMAGING; MALE; MOTOR CORTEX; READING;

EID: 84881119719     PISSN: 0093934X     EISSN: 10902155     Source Type: Journal    
DOI: 10.1016/j.bandl.2013.07.003     Document Type: Article

References (94)

1. Albrecht J.E., O'Brien E.J. Updating a mental model: Maintaining both local and global coherence. Journal of Experimental Psychology: Learning 1993, 19:1061-1070.
2. Allison T., Puce A., McCarthy G. Social perception from visual cues: Role of the STS region. Trends in Cognitive Sciences 2000, 4:267-278.
3. Anderson J.R., Lebiere C. The atomic components of thought 1998, Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US.
4. Aziz-Zadeh L., Wilson S., Rizzolatti G., Iacoboni M. Congruent embodied representations for visually presented actions and linguistic phrases describing actions. Current Biology 2006, 16:1818-1823.
5. Barker R.G., Wright H.S. One boy's day; a specimen record of behavior 1951, Harper, Oxford, England.
6. Barsalou L.W. Perceptual symbol systems. Behavioral and Brain Sciences 1999, 22:577-660.
7. Barsalou L.W. Grounded cognition. Annual Review of Psychology 2008, 59:617-645.
8. Barsalou L.W., Santos A., Simmons W.K., Wilson C.D. Language and simulation in conceptual processing. Symbols, embodiment, and meaning 2008, 245-283. Oxford University Press, Oxford, England. M. De Vega, A.M. Glenberg, A.C. Graesser (Eds.).
9. Bedny M., Caramazza A., Grossman E., Pascual-Leone A., Saxe R. Concepts are more than percepts: The case of action verbs. The Journal of Neuroscience 2008, 28:11347-11353.
10. Boynton G.M., Engel S.A., Glover G.H., Heeger D.J. Linear systems analysis of functional magnetic resonance imaging in human V1. Journal of Neuroscience 1996, 16(13):4207-4221.
11. Brunyé T.T., Ditman T., Mahoney C.R., Walters E.K., Taylor H.A. You heard it here first: Readers mentally simulate described sounds. Acta Psychologica 2010, 135(2):209-215. doi: http://dx.doi.org.ezproxy.gvsu.edu/10.1016/j.actpsy.2010.06.008.
12. Castiello U. The neuroscience of grasping. Nature Reviews Neuroscience 2005, 6:726-736.
13. Catani M., Jones D.K., Ffytche D.H. Perisylvian language networks of the human brain. Annals of Neurology 2005, 57:8-16.
14. Damasio H., Grabowski T.J., Tranel D., Pronto L.L., Hichwa R.D., Damasio A.R. Neural correlates of naming actions and of naming spatial relations. Neuroimage 2001, 13:1053-1064.
15. Desai R.H., Binder J.R., Conant L.L., Seidenberg M.S. Activation of sensory-motor areas in sentence comprehension. Cerebral Cortex 2010, 20:468-478.
16. D'Esposito M., Detre J.A., Aguirre G.K., Stallcup D., Alsop D.C., Tippett L.J., et al. A functional MRI study of mental image generation. Neuropsychologia 1997, 35:725-730.
17. Ehrsson H.H., Geyer S., Naito E. Imagery of voluntary movement of fingers, toes, and tongue activates corresponding body-part-specific motor representations. Journal of Neurophysiology 2003, 90:3304-3316.
18. Ferstl E.C., Neumann J., Bogler C., von Cramon D.Y. The extended language network: A meta-analysis of neuroimaging studies on text comprehension. Human Brain Mapping 2008, 29:581-593.
19. Fincher-Kiefer R. Perceptual components of situation models. Memory & Cognition 2001, 29:336-343.
20. Fischer M.H., Zwaan R.A. Embodied language: A review of the role of motor system in language comprehension. The Quarterly Journal of Experimental Psychology 2008, 61(6):825-850. doi: http://dx.doi.org.ezproxy.gvsu.edu/10.1080/17470210701623605.
21. Fodor J.A. The language of thought 1975, Harvard University Press, Cambride, MA.
22. Friese U., Rutschmann R., Raabe M., Schmalhofer F. Neural indicators of inference processes in text comprehension: An event-related functional magnetic resonance imaging study. Journal of Cognitive Neuroscience 2008, 20(11):2110-2124. doi: http://dx.doi.org.ezproxy.gvsu.edu/10.1162/jocn.2008.20141.
23. Gallese V., Lakoff G. The brain's concepts: The role of the sensory-motor system in conceptual knowledge. Cognitive Neuropsychology 2005, 22(3-4):455-479. doi: http://dx.doi.org.ezproxy.gvsu.edu/10.1080/02643290442000310.
24. Glenberg A.M. What memory is for. Behavioral and Brain Sciences 1997, 20:1-55.
25. Glenberg A.M., Gallese V. Action-based language: A theory of language acquisition, comprehension, and production. Cortex: A Journal Devoted to the Study of the Nervous System and Behavior 2012, 48(7):905-922. doi: http://dx.doi.org.ezproxy.gvsu.edu/10.1016/j.cortex.2011.04.010.
26. Glenberg A.M., Kaschak M.P. Grounding language in action. Psychonomic Bulletin & Review 2002, 9:558-565.
27. Graesser A.C., Millis K.K., Zwaan R.A. Discourse comprehension. Annual Review of Psychology 1997, 48:163-189.
28. Graesser A.C., Singer M., Trabasso T. Constructing inferences during narrative text comprehension. Psychological Review 1994, 101(3):371-395. doi: http://dx.doi.org.ezproxy.gvsu.edu/10.1037/0033-295X.101.3.371.
29. Grèzes J., Decety J. Functional anatomy of execution, mental simulation, observation, and verb generation of actions: A meta-analysis. Human Brain Mapping 2001, 12:1-19.
30. Grossman E., Donnelly M., Price R., Pickens D., Morgan V., Neighbor G., et al. Brain areas involved in perception of biological motion. Journal of Cognitive Neuroscience 2000, 12:711-720.
31. Halpern A.R., Zatorre R.J. When that tune runs through your head: A PET investigation of auditory imagery for familiar melodies. Cerebral Cortex 1999, 9:697-704.
32. Halpern A.R., Zatorre R.J., Bouffard M., Johnson J.A. Behavioral and neural correlates of perceived and imagined musical timbre. Neuropsychologia 2004, 42:1281-1292.
33. Hauk O., Johnsrude I., Pulvermüller F. Somatotopic representation of action words in human motor and premotor cortex. Neuron 2004, 41:301-307.
34. Hesslow G. Conscious thought as simulation of behaviour and perception. Trends in Cognitive Sciences 2002, 6:242-247.
35. Hubbard T.L. Auditory imagery: Empirical findings. Psychological Bulletin 2010, 136:302-329.
36. Johnson-Laird P.N. Mental models: Towards a cognitive science of language, inference, and consciousness 1983, Harvard University Press, Cambride, MA.
37. Kable J.W., Chatterjee A. Specificity of action representations in the lateral occipitotemporal cortex. Journal of Cognitive Neuroscience 2006, 18:1498-1517.
38. Kable J.W., Lease-Spellmeyer J., Chatterjee A. Neural substrates of action event knowledge. Journal of Cognitive Neuroscience 2002, 14:795-805.
39. Kanwisher N., Downing P., Epstein R., Kourtzi Z. Functional neuroimaging of human visual recognition. The handbook of functional neuroimaging 2001, 109-152. MIT Press, Cambride, MA. R. Cabeza, A. Kingstone (Eds.).
40. Kaschak M.P., Madden C.J., Therriault D.J., Yaxley R.H., Aveyard M., Blanchard A.A., et al. Perception of motion affects language processing. Cognition 2005, 94:B79-B89.
41. Kellenbach M.L., Brett M., Patterson K. Large, colorful, or noisy? Attribute- and modality-specific activations during retrieval of perceptual attribute knowledge. Cognitive, Affective & Behavioral Neuroscience 2001, 1:207-221.
42. Kemmerer D., Gonzalez-Castillo J. The two-level theory of verb meaning: An approach to integrating the semantics of action with the mirror neuron system. Brain and Language 2010, 112(1):54-76. doi: http://dx.doi.org.ezproxy.gvsu.edu/10.1016/j.bandl.2008.09.010.
43. Kiefer M., Sim E.-J., Herrnberger B., Grothe J., Hoenig K. The sound of concepts: Four markers for a link between auditory and conceptual brain systems. The Journal of Neuroscience 2008, 28(47):12224-12230. doi:http://dx.doi.org.ezproxy.gvsu.edu/10.1523/JNEUROSCI.3579-08.2008.
44. Kintsch W. Comprehension: A paradigm for cognition 1998, Cambridge University Press, New York, NY, US.
45. Kurby C.A., Magliano J.P., Rapp D.N. Those voices in your head: Activation of auditory images during reading. Cognition 2009, 112:457-461.
46. Louwerse M.M. Embodied relations are encoded in language. Psychonomic Bulletin & Review 2008, 15:838-844.
47. Louwerse M.M., Jeuniaux P. The linguistic and embodied nature of conceptual processing. Cognition 2010, 114:96-104.
48. Mahon B.Z., Caramazza A. A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. Journal of Physiology - Paris 2008, 102:59-70.
49. Mar R.A., Oatley K. The function of fiction is the abstraction and simulation of social experience. Perspectives on Psychological Science 2008, 3:173-192.
50. Markman A.B., Brendl C.M. Constraining theories of embodied cognition. Psychological Science 2005, 16:6-10.
51. Marks D.F. New directions for mental imagery research. Journal of Mental Imagery 1995, 19(3-4):153-167.
52. McAvoy M., Ollinger J.M., Buckner R.L. Cluster size thresholds for assessment of significant activation in fMRI. Neuroimage 2001, 13:S198.
53. McGuire P.K., Silbersweig D.A., Murray R.M., David A.S., Frackowiak R.S.J., Frith C.D. Functional neuroanatomy of inner speech and auditory verbal imagery. Psychological Medicine 1996, 26:29-38.
54. Michelon P., Vettel J.M., Zacks J.M. Lateral somatotopic organization during imagined and prepared movements. Journal of Neurophysiology 2006, 95:811-822.
55. Orban G.A., Van Essen D., Vanduffel W. Comparative mapping of higher visual areas in monkeys and humans. Trends in Cognitive Sciences 2004, 8:315-324.
56. Pecher D., Zeelenberg R., Barsalou L.W. Verifying different-modality properties for concepts produces switching costs. Psychological Science 2003, 14:119-124.
57. Poldrack R.A. Can cognitive processes be inferred from neuroimaging data?. Trends in Cognitive Sciences 2006, 10:59-63.
58. Porro C.A., Francescato M.P., Cettolo V., Diamond M.E., et al. Primary motor and sensory cortex activation during motor performance and motor imagery: A functional magnetic resonance imaging study. The Journal of Neuroscience 1996, 16:7688-7698.
59. Postle N., McMahon K.L., Ashton R., Meredith M., de Zubicaray G.I. Action word meaning representations in cytoarchitectonically defined primary and premotor cortices. NeuroImage 2008, 43:634-644.
60. Pulvermüller F. Words in the brain's language. Behavioral and Brain Sciences 1999, 22(2):253-336. doi: http://dx.doi.org.ezproxy.gvsu.edu/10.1017/S0140525X9900182X.
61. Pulvermüller F., Hauk O., Nikulin V.V., Llmoniemi R.J. Functional links between motor and language systems. European Journal of Neuroscience 2005, 21:793-797.
62. Pylyshyn Z.W. The imagery debate: Analogue media versus tacit knowledge. Psychological Review 1981, 88:16-45.
63. Pylyshyn Z.W. Mental imagery: In search of a theory. Behavioral and Brain Sciences 2002, 25:157-238.
64. Saxe R., Kanwisher N. People thinking about thinking people. The role of the temporo-parietal junction in "theory of mind". NeuroImage 2003, 19(4). 10.1016/S1053-8119(03)00230-1.
65. Shergill S.S., Bullmore E.T., Brammer M.J., Williams S.C.R., Murray R.M., McGuire P.K. A functional study of auditory verbal imagery. Psychological Medicine 2001, 31:241-253.
66. Simmons W.K., Ramjee V., Beauchamp M.S., McRae K., Martin A., Barsalou L.W. A common neural substrate for perceiving and knowing about color. Neuropsychologia 2007, 45:2802-2810.
67. Speer N.K., Reynolds J.R., Swallow K.M., Zacks J.M. Reading stories activates neural representations of visual and motor experiences. Psychological Science 2009, 20:989-999.
68. Speer N.K., Zacks J.M., Reynolds J.R. Human brain activity time-locked to narrative event boundaries. Psychological Science 2007, 18:449-455.
69. Stanfield R.A., Zwaan R.A. The effect of implied orientation derived from verbal context on picture recognition. Psychological Science 2001, 12:153-156.
70. Stone V.E., Baron-Cohen S., Knight R.T. Frontal lobe contributions to theory of mind. Journal of Cognitive Neuroscience 1998, 10(5). 10.1162/089892998562942.
71. Stuss D.T., Gallup G.G.J., Alexander M.P. The frontal lobes are necessary for "theory of mind". Brain: A Journal of Neurology 2001, 124(Pt 2). 10.1093/brain/124.2.279.
72. Talairach J., Tournoux P. Co-planar stereotaxic atlas of the human brain: 3-dimensional proportional system, an approach to cerebral imaging 1988, G. Thieme, Stuttgart.
73. Tettamanti M., Buccino G., Saccuman M.C., Gallese V., Danna M., Scifo P., et al. Listening to action-related sentences activates Fronto-parietal motor circuits. Journal of Cognitive Neuroscience 2005, 17:273-281.
74. Thompson-Schill S.L., Aguirre G.K., D'Esposito M., Farah M.J. A neural basis for category and modality specificity of semantic knowledge. Neuropsychologia 1999, 37:671-676.
75. Van Berkum J.J.A., Hagoort P., Brown C.M. Semantic integration in sentences and discourse: Evidence from the N400. Journal of Cognitive Neuroscience 1999, 11:657-671.
76. Van Dijk T.A., Kintsch W. Strategies in discourse comprehension 1983, Academic Press, New York, NY, US.
77. Van Essen D.C. A population-average, landmark- and surface-based (PALS) atlas of human cerebral cortex. Neuroimage 2005, 28:635-662.
78. Van Essen D.C., Drury H.A., Dickson J., Harwell J., Hanlon D., Anderson C.H. An integrated software suite for surface-based analyses of cerebral cortex. Journal of American Medical Informatics Association 2001, 8:443-459.
79. Van Horn J.D., Poldrack R.A. Functional MRI at the crossroads. International Journal of Psychophysiology 2009, 73:3-9.
80. Wheeler M.E., Petersen S.E., Buckner R.L. Memory's echo: Vivid remembering reactivates sensory-specific cortex. Proceedings of the National Academy of Sciences 2000, 97:11125-11129.
81. Willems R.M., Hagoort P., Casasanto D. Body-specific representations of action verbs: Neural evidence from right- and left-handers. Psychological Science 2010, 21:67-74.
82. Willems R.M., Labruna L., D'Esposito M., Ivry R., Casasanto D. A functional role for the motor system in language understanding: Evidence from theta-burst transcranial magnetic stimulation. Psychological Science 2011, 22:849-854.
83. Willems R.M., Toni I., Hagoort P., Casasanto D. Neural dissociations between action verb understanding and motor imagery. Journal of Cognitive Neuroscience 2010, 22(10). 10.1162/jocn.2009.21386.
84. Yao B., Belin P., Scheepers C. Silent reading of direct versus indirect speech activates voice-selective areas in the auditory cortex. Journal of Cognitive Neuroscience 2011, 23:3146-3152. doi: http://dx.doi.org.ezproxy.gvsu.edu/10.1162/jocn_a_00022.
85. Yarkoni T., Speer N.K., Zacks J.M. Neural substrates of narrative comprehension and memory. NeuroImage 2008, 41:1408-1425.
86. Yoo S.-S., Lee C.U., Choi B.G. Human brain mapping of auditory imagery: Event-related functional MRI study. NeuroReport 2001, 12:3045-3049.
87. Zacks J.M., Braver T.S., Sheridan M.A., Donaldson D.I., Snyder A.Z., Ollinger J.M., et al. Human brain activity time-locked to perceptual event boundaries. Nature Neuroscience 2001, 4:651-655.
88. Zatorre R.J., Halpern A.R., Perry D.W., Meyer E., Evans A.C. Hearing in the mind's ear: A PET investigation of musical imagery and perception. Journal of Cognitive Neuroscience 1996, 8:29-46.
89. Zwaan, R. A. (2004). The immersed experiencer: Toward an embodied theory of language comprehension. In Ross, Brian H. (Ed.), The psychology of learning and motivation: Advances in research and theory (vol. 44, Academic Press, New York, pp. 35-62).
90. Zwaan R.A., Radvansky G.A. Situation models in language comprehension and memory. Psychological Bulletin 1998, 123:162-185.
91. Zwaan R.A., Madden C.J., Yaxley R.H., Aveyard M.E. Moving words: Dynamic representations in language comprehension. Cognitive Science 2004, 28:611-619.
92. Zwaan R.A., Stanfield R.A., Yaxley R.H. Language comprehenders mentally represent the shape of objects. Psychological Science 2002, 13:168-171.
93. Zwaan R.A., Taylor L.J. Seeing, acting, understanding: Motor resonance in language comprehension. Journal of Experimental Psychology: General 2006, 135:1-11.
94. Zwaan R.A., Yaxley R.H. Spatial iconicity affects semantic relatedness judgments. Psychonomic Bulletin & Review 2003, 10:954-958.