Spatiotemporal dynamics of argument retrieval and reordering: An fMRI and EEG study on sentence processing

Frontiers in Psychology

Volumn 3, Issue DEC, 2012, Pages

  Meyer, Lars ^a   Obleser, Jonas ^a   Kiebel, Stefan J ^a   Friederici, Angela D ^a  

^a MAX PLANCK INSTITUTE FOR HUMAN COGNITIVE AND BRAIN SCIENCES   (Germany)

Author keywords

Argument verb dependency; Dipole time course; Inferior frontal gyrus; Inferior parietal cortex; Source localization; Syntax; Working memory

Indexed keywords

EID: 84874558752     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2012.00523     Document Type: Article

References (115)

1. Andrade, A., Paradis, A. L., Rou-quette, S., and Poline, J. B. (1999). Ambiguous results in functional neuroimaging data analysis due to covariate correlation. Neuroimage 10, 483-486.
2. Aoshima, S., Phillips, C., and Wein-berg, A. (2004). Processing filler-gap dependencies in a head-final language. J. Mem. Lang. 51, 23-54.
3. Baayen, H., Gulikers, L., and Piepen-brock, R. (1995). The CELEX Lexi-cal Database. Philadelphia: Linguis-tic Data Consortium.
4. Babyonyshev, M.,andGibson, E.(1999). The complexity of nested struc-tures in Japanese. Language 75, 423-450.
5. Baddeley, A. (2012). Working mem-ory: theories, models, and con-troversies. Annu. Rev. Psychol. 63, 1-29.
6. Baddeley, A. D., and Hitch, G. J. (1974). "Working memory," in The Psychol-ogy of Learning and Motivation, ed. G. H. Bower (New York: Academic Press), 47-89.
7. Benjamini, Y., and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. Series B Stat. Methodol. 57, 289-300.
8. Ben-Shachar, M., Hendler, T., Kahn, I., Ben-Bashat, D., and Grodzinsky, Y. (2003). The neural reality of syntac-tic transformations. Psychol. Sci. 14, 433-440.
9. Besl, P. J., and McKay, N. D. (1992). A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 239-256.
10. Biemann, C., Bordag, S., Quasthoff, U., and Wolff, C. (2004). "Web services for language resources and language technology applications,"in Proceed-ings of the Fourth International Con-ference on Language Resources and Evaluation, Lisbon.
11. Birbaumer, N., Elbert, T., Canavan, A., and Rockstroh, B. (1990). Slow potentials of the cerebral cortex and behavior. Physiol. Rev. 70, 1.
12. Boersma, P., and Weenink, D. (2001). Praat, a system for doing pho-netics by computer. Glot Int. 5, 341-345.
13. Bornkessel, I., Schlesewsky, M., and Friederici, A. (2002). Grammar overrides frequency: Evidence from the online processing of flexible word order. Cognition 85, B21-B30.
14. Buchsbaum, B. R., Baldo, J., Okada, K., Berman, K. F., Dronkers, N., D'E-sposito, M., et al. (2011). Conduc-tion aphasia,sensory-motor integra-tion, and phonological short-term memory -an aggregate analysis of lesion and fMRI data. Brain Lang. 119, 119-128.
15. Buchsbaum, B. R., Hickok, G., and Humphries, C. (2001). Role of left posterior superior temporal gyrus in phonological processing for speech perception and production. Cogn. Sci. 25, 663-678.
16. Caplan, D., Alpert, N., Waters, G., and Olivieri, A. (2000). Activation of Broca's area by syntactic process-ing under conditions of concurrent articulation. Hum. Brain Mapp. 9, 65-71.
17. Caplan, D., and Waters, G. (1999). Ver-bal working memory and sentence comprehension. Behav. Brain Sci. 22, 77-94.
18. Chomsky, N. (1955). The Logical Struc-ture of Linguistic Theory. New York: Plenum press.
19. Clerget, E., Badets, A., Duqué, J., and Olivier, E. (2011). Role of Broca's area in motor sequence program-ming: a cTBS study. Neuroreport 22, 965-969.
20. Clifton, C., and Frazier, L. (1988). "Comprehending sentences with long-distance dependencies," in Lin-guistic Structure in Language Pro-cessing, eds G. Carlson and M. K. Tanenhaus (Dordrecht: Reidel Press), 273-317.
21. Clifton, C., Frazier, L., and Connine, C. (1984). Lexical expectations in sen-tence comprehension. J. Verb. Learn. Verb. Behav. 23, 696-708.
22. Daneman, M., and Carpenter, P. A. (1980). Individual differences in working memory and reading. J. Verb. Learn. Verb. Behav. 19, 450-466.
23. Daunizeau, J., Laufs, H., and Friston, K. J. (2010). "EEG-fMRI informa-tion fusion: biophysics and data analysis," in EEG-fMRI Informa-tion Fusion: Biophysics and Data Analysis, eds C. Mulert and L. Lemieux (Heidelberg: Springer), 511-526.
24. Donchin, E.,and Coles, M. (1988). Is the P300 component a manifestation of context updating? Behav. Brain Sci. 11, 357-427.
25. Edgar, J., Stewart, J., and Miller, G. (2005). "Digital filters in ERP research," in Event-Related Poten-tials: A Handbook, ed. T. C. Handy (Cambridge: MIT Press), 85-113.
26. Eickhoff, S., Stephan, K., Mohlberg, H., Grefkes, C., Fink, G., Amunts, K., et al. (2005). A new SPM toolbox for combining probabilis-tic cytoarchitectonic maps and func-tional imaging data. Neuroimage 25, 1325-1335.
27. Ergen, M., Yildirim, E., Uslu, A., Gurvit, H., and Demiralp, T. (2012). P3 response during short-term mem-ory retrieval revisited by a spatio-temporal analysis. Int. J. Psychophys-iol. 84, 205-210.
28. Fedorenko, E., Behr, M. K., and Kan-wisher, N. (2011). Functional speci-ficity for high-level linguistic pro-cessing in the human brain. Proc. Natl. Acad. Sci. U.S.A. 108, 106, 8362-8367.
29. Fedorenko, E., Gibson, E., and Rohde, D. (2006). The nature of working memory in linguistic, arithmetic and spatial integration processes. J. Mem. Lang. 56, 246-269.
30. Felser, C., Clahsen, H., and Münte, T. F. (2003). Storage and integration in the processing of filler-gap depen-dencies: an ERP study of topicaliza-tion and wh-movement in German. Brain Lang. 87, 345-354.
31. Ferreira, F., and Henderson, J. M. (1993). Reading processes dur-ing syntactic analysis and reanaly-sis. Can. J. Exp. Psychol. 47, 247-247.
32. Fiebach, C., Schlesewsky, M., Lohmann, G., von Cramon, D., and Friederici, A. D. (2005). Revisiting the role of Broca's area in sentence processing: syntactic integration versus syntac-tic working memory. Hum. Brain Mapp. 24, 79-91.
33. Fiebach, C. J., Schlesewsky, M., and Friederici, A. D. (2001). Syntactic working memory and the estab-lishment of filler-gap dependencies: insights from ERPs and fMRI. J. Psycholinguist. Res. 30, 321-338.
34. Fiebach, C. J., Schlesewsky, M., and Friederici, A. D. (2002). Separat-ing syntactic memory costs and syntactic integration costs during parsing: the processing of German WH-questions. J. Mem. Lang. 47, 250-272.
35. Fodor, J. (1978). Parsing strategies and constraints on transformations. Lin-guist. Inq. 9, 427-473.
36. Frazier, L., and Clifton, C. (1989). Suc-cessive cyclicity in the grammar and the parser. Lang. Cogn. Process. 4, 93-126.
37. Friederici, A. D., Fiebach, C. J., Schlesewsky, M., Bornkessel, I. D., and Von Cramon, D. Y. (2006). Processing linguistic com-plexity and grammaticality in the left frontal cortex. Cereb. Cortex 16, 1709-1717.
38. Friederici, A. D., and Frisch, S. (2000). Verb argument structure process-ing: the role of verb-specific and argument-specific information. J. Mem. Lang. 43, 476-507.
39. Friederici, A. D., Hahne, A., and Saddy, D. (2002). Distinct neurophysiolog-ical patterns reflecting aspects of syntactic complexity and syntac-tic repair. J. Psycholinguist. Res. 31, 45-63.
40. Friederici, A. D., and Mecklinger, A. (1996). Syntactic parsing as revealed by brain responses: first-pass and second-pass parsing processes. J. Psycholinguist. Res. 25, 157-176.
41. Friederici, A. D., Pfeifer, E., and Hahne, A. (1993). Event-related brain potentials during natural speech processing: effects of seman-tic, morphological and syntactic vio-lations. Brain Res. Cogn. Brain Res. 1, 183-192.
42. Friederici, A. D., Steinhauer, K., Mecklinger, A., and Meyer, M. (1998). Working memory con-straints on syntactic ambiguity resolution as revealed by electrical brain responses. Biol. Psychol. 47, 193-221.
43. Friederici, A. D., Wang, Y., Herrmann, C. S., Maess, B., and Oertel, U. (2000). Localization of early syntac-tic processes in frontal and temporal cortical areas: a magnetoencephalo-graphic study. Hum. Brain Mapp. 11, 1-11.
44. Friedman, D., and Johnson, R. (2000). Event-related potential (ERP) stud-ies of memory encoding and retrieval: a selective review. Microsc. Res. Tech. 51, 6-28.
45. Friedman, D., Simson, R., Ritter, W.,and Rapin, I. (1975). The late positive component (P300) and information processing in sentences. Glot Int. 38, 255-262.
46. Gerton, B., Brown, T., Meyer-Lindenberg, A., Kohn, P., Holt, J., Olsen, R., et al. (2004). Shared and distinct neurophysiological components of the digits forward and backward tasks as revealed by functional neuroimaging. Neuropsychologia 42, 1781-1787.
47. Gibson, E. (1998). Linguistic complex-ity: locality of syntactic dependen-cies. Cognition 68, 1-76.
48. Gibson, E. (2000). "The dependency locality theory: a distance-based theory of linguistic complexity," in Image, Language, Brain, eds Y. Miyashita, A. Marantz, and W. Oíneil (Cambridge, MA: MIT Press), 95-126.
49. Gordon, P. C., Hendrick, R., and John-son, M. (2001). Memory interfer-ence during language processing. J. Exp. Psychol. Learn. Mem. Cogn. 27, 1411-1423.
50. Grodner, D., and Gibson, E. (2005). Consequences of the serial nature of linguistic input for sente-nial complexity. Cogn. Sci. 29, 261-290.
51. Grossberg, S. (1984). Some psychophys-iological and pharmacological cor-relates of a developmental, cognitive and motivational theory. Ann. N. Y. Acad. Sci. 425, 58-151.
52. Grossman, M., Cooke, A., Devita, C., Alsop, D., Detre, J., Chen, W., et al. (2002). Age-related changes in working memory during sentence comprehension: an fMRI study. Neuroimage 15, 302-317.
53. Hagoort, P., Brown, C.,and Groothusen, J. (1993). The syntactic positive shift (SPS) as an ERP measure of syntac-tic processing. Lang. Cogn. Process. 8, 439-483.
54. Hagoort, P., Hald, L., Bastiaansen, M., and Petersson, K. M. (2004). Inte-gration of word meaning and world knowledge in language comprehen-sion. Science 304, 438-441.
55. Halgren, E., Dhond, R. P., Christensen, N., Van Petten, C., Marinkovic, K., Lewine, J. D., et al. (2002). N400-like magnetoencephalography responses modulated by semantic context, word frequency, and lexical class in sentences. Neuroimage 17, 1101-1116.
56. Henson, R. N., Burgess, N., and Frith, C. D. (1999). Recoding, storage, rehearsal and grouping in verbal short-term memory: an fMRI study. Neuropsychologia 38, 426-440.
57. Johnson, B. W., and Hamm, J. P. (2000). High-density mapping in an N400 paradigm: evidence for bilat-eral temporal lobe generators. Clin. Neurophysiol. 111, 532-545.
58. Just, M. A., and Carpenter, P. (1992). A capacity theory of compre-hension: individual differences in working memory. Psychol. Rev. 99, 122-149.
59. Kaan, E., Harris, A., Gibson, E., and Holcomb, P. (2000). The P600 as an index of syntactic integration difficulty. Lang. Cogn. Process. 15, 159-201.
60. Kaan, E.,and Swaab, T.Y. (2003). Repair, revision, and complexity in syntac-tic analysis: an electrophysiological differentiation. J. Cogn. Neurosci. 15, 98-110.
61. Kamide, Y., Altmann, G., and Hay-wood, S. (2000). "Predictive eye-movements in incremental process-ing of head-final structures," in Poster Presentation at the 20th CUNY Sentence Processing Conference, La Jolla.
62. Kamide, Y., Scheepers, C., and Alt-mann, G. (2003). Integration of syntactic and semantic informa-tion in predictive processing: cross-linguistic evidence from German andEnglish.J.Psycholinguist.Res.32, 37-55.
63. Kiebel, S. J., Daunizeau, J., Phillips, C., and Friston, K. J. (2008). Vari-ational Bayesian inversion of the equivalent current dipole model in EEG/MEG. Neuroimage 39, 728-741.
64. King, J., and Just, M. (1991). Indi-vidual differences in syntacticprocessing: the role of work-ing memory. J. Mem. Lang. 30, 580-602.
65. Kinno, R., Kawamura, M., Shioda, S., and Sakai, K. L. (2008). Neural correlates of noncanonical syntac-tic processing revealed by a picture sentence matching task. Hum. Brain Mapp. 29, 1015-1027.
66. Kluender, R., and Kutas, M. (1993). Subjacency as a processing phe-nomenon. Lang. Cogn. Process. 8, 573-633.
67. Konieczny, L. (2000). Locality and pars-ing complexity. J. Psycholinguist. Res. 29, 627-645.
68. Konieczny, L., and Döring, P. (2003). "Anticipation of clause-final heads: evidence from eye-tracking and SRNs,"in Proceedings of ICCS/ASCS, Sydney.
69. Kutas, M., and Van Petten, C. V. (1994). "Psycholinguistics electri-fied," in Handbook of Psycholin-guistics, ed. M. A. Gernsbacher (San Diego, CA: Academic Press), 83-143.
70. Lau, E. F., Phillips, C., and Poep-pel, D. (2008). A cortical net-work for semantics: (de) construct-ing the N400. Nat. Rev. Neurosci. 9, 920-933.
71. Levy, R. (2008). Expectation-based syntactic comprehension. Cognition 106, 1126-1177.
72. Lewis, R. L., Vasishth, S., and Van Dyke, J. A. (2006). Computa-tional principles of working mem-ory in sentence comprehension. Trends Cogn. Sci. (Regul. Ed.) 10, 447-454.
73. Macmillan, N. A., and Creelman, C. D. (2005). Detection Theory: A User's Guide. Mahwan: Lawrence Erlbaum Associates.
74. Maess, B., Herrmann, C. S., Hahne, A., Nakamura, A., and Friederici, A. D. (2006). Localizing the distributed language network responsible for the N400 measured by MEG during auditory sentence processing. Brain Res. 1096, 163-172.
75. Makuuchi, M., Bahlmann, J., Anwan-der, A., and Friederici, A. D. (2009). Segregating the core computational faculty of human language from working memory. Proc. Natl. Acad. Sci. U.S.A. 106, 8362-8367.
76. Makuuchi, M., Bahlmann, J., and Friederici, A. D. (2012).An approach to separating the levels of hierarchi-cal structure building in language and mathematics. Philos. Trans. R. Soc. Lond. B Biol. Sci. 367, 2033-2045.
77. McElree, B., Foraker, S., and Dyer, L. (2003). Memory structures that subserve sentence comprehension. J. Mem. Lang. 48, 67-91.
78. Mecklinger, A., Schriefers, H., Stein-hauer, K., and Friederici, A. (1995). Processing relative clauses vary-ing on syntactic and semantic dimensions: an analysis with event-related potentials. Mem. Cognit. 23, 477-494.
79. Meyer, L., Obleser, J., Anwander, A., and Friederici, A. D. (2012b). Linking ordering in Broca's area to storage in left temporo-parietal regions: the case of sentence processing. Neu-roimage. 62, 1987-1998.
80. Meyer, L., Obleser, J., and Friederici, A. D. (2012). Left parietal alpha enhancement during working memory-intensive sentence pro-cessing. Cortex. Available at: http://dx.doi.org/10.1016/j.cortex.2012.03.006
81. Meyer, L., Obleser, J., and Friederici, A.D. (2010). "Distinct brain-electric signatures for order and distance in sentence processing," in Poster Pre-sentation at the 16th AMLaP, New York.
82. Mugler, J. P. III, and Brookeman, J. R. (1990). Three dimensional magnetization prepared rapid gradient echo imaging (3D MP RAGE). Magn. Reson. Med. 15, 152-157.
83. Nakatani, K., and Gibson, E. (2008). Distinguishing theories of syn-tactic storage costs: evidence from Japanese. Linguistics 46, 63-87.
84. Nicol, J. (1993). "Reconsidering reac-tivation," in Cognitive Models of Speech Processing, eds A. Altmann and R. Shillcock (Hove: Lawrence Erlbaum), 321-347.
85. Novais-Santos, S., Gee, J., Shah, M., Troiani, V., Work, M., and Gross-man, M. (2007). Resolving sen-tence ambiguity with planning and working memory resources: evi-dence from fMRI. Neuroimage 37, 361-378.
86. Obleser, J., Meyer, L., and Friederici, A. D. (2011). Dynamic assignment of neural resources in auditory com-prehension of complex sentences. Neuroimage 4, 2310-2320.
87. Oldfield, R. (1971). The assessment and analysis of handedness: the Edin-burgh inventory. Neuropsychologia 9, 97-113.
88. Oostenveld, R., Fries, P., Maris, E., and Schoffelen, J. M. (2011). FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput. Intell Neurosci. 2011, 1.
89. Osterhout, L., and Holcomb, P. J. (1992). Event-related brain potentials elicited by syntactic anomaly. J. Mem. Lang. 31, 785-785.
90. Osterhout, L., and Holcomb, P. J. (1993). Event-related potentials and syntactic anomaly: evidence of anomaly detection during the perception of continuous speech. Lang. Cogn. Process. 8, 413-437.
91. Osterhout, L., and Swinney, D. A. (1993). On the temporal course of gap-filling during comprehension of verbal passives. J. Psycholinguist. Res. 22, 273-286.
92. Phillips, C., Kazanina, N., and Abada, S. H.(2005).ERPeffectsof theprocess-ing of syntactic long-distance depen-dencies. Cogn. Brain Res. Cogn. Brain Res. 22, 407-428.
93. Polich, J. (2007). Updating P300: an integrative theory of P3a and P3b. Clin. Neurophysiol. 118, 2128-2148.
94. Polich, J., Howard, L., and Starr, A. (1983). P300 latency correlates with digit span. Psychophysiology 20, 665-669.
95. Ravizza, S. M., Delgado, M. R., Chein, J. M., Becker, J. T., and Fiez, J. A. (2004). Functional dissociations within the inferior parietal cortex in verbal working memory. Neuroim-age 22, 562-573.
96. Ravizza, S. M., Hazeltine, E., Ruiz, S., and Zhu, D. C. (2011). Left TPJ activity in verbal working memory: implications for storage-and sensory-specific models of short term memory. Neuroimage 55, 1836-1846.
97. Rösler, F., Pechmann, T., Streb, J., Röder, B., and Hennighausen, E. (1998). Parsing of sentences in a language with varying word order: word-by-word variations of pro-cessing demands are revealed by event-related brain potentials. J. Mem. Lang. 38, 150-176.
98. Rugg, M. D., and Curran, T. (2007). Event-related potentials and recognition memory. Trends Cogn. Sci. (Regul. Ed.) 11, 251-257.
99. Santi, A., and Grodzinsky, Y. (2007). Working memory and syntax inter-act in Broca's area. Neuroimage 37, 8-17.
100. Santi, A., and Grodzinsky, Y. (2010). fMRI adaptation dis-sociates syntactic complexity dimensions. Neuroimage 51, 1285-1293.
101. Service, E., Helenius, P., Maury, S., and Salmelin, R. (2007). Local-ization of syntactic and semantic brain responses using magnetoen-cephalography. J. Cogn. Neurosci. 19, 1193-1205.
102. Silva-Pereyra, J., Rivera-Gaxiola, M., Aubert, E., Bosch, J., Galán, L., and Salazar, A. (2003). N400 during lexical decision tasks: a current source localization study. Clin. Neurophysiol. 114, 2469-2486.
103. Simos, P. G., Basile, L. F. H., and Papan-icolaou, A. C. (1997). Source local-ization of the N400 response in a sentence-reading paradigm using evoked magnetic fields and magnetic resonance imaging. Brain Res. 762, 29-39.
104. Spivey, M., Fitneva, S., Tabor, W., and Ajmani, S. (2002). "The time course of information integration in sentence processing," in The Lexical Basis of Sentence Processing: Formal, Computational and Experimental Issues, eds P. Merlo and S. Steven-son (Amsterdam: Benjamins), 207-232.
105. Tanenhaus, M. K., Carlson, G. N., and Seidenberg, M. S. (1985). "Do listeners compute linguistic representations," in Natural Lan-guage Parsing: Psychological, Com-putational, and Theoretical Perspec-tives, eds D. R. Dowty, L. Kart-tunen, and A. M. Zwicky (New York: Cambridge University Press), 359-408.
106. Trueswell, J., Tanenhaus, M., and Kello, C. (1993). Verb-specific con-straints in sentence processing: sep-arating effects of lexical prefer-ence from garden-paths. J. Exp. Psychol. Learn. Mem. Cogn. 19, 528-553.
107. Tsuzuki, T., Uchida, T., Yukihiro, R., Hisano, M., and Tsuzuki, K. (2004). Effects of syntactic information on semantic access of ambiguous verbs in spoken language comprehension: evidence from a cross-modal prim-ing experiment1. Jpn. Psychol. Res. 46, 31-43.
108. Ueno, M., and Kluender, R. (2003). Event-related brain indices of Japan-ese scrambling. Brain Lang. 86, 243-271.
109. Van Dyke, J. A. (2007). Interfer-ence effects from grammatically unavailable constituents dur-ing sentence processing. J. Exp. Psychol. Learn. Mem. Cogn. 33, 407-430.
110. Van Petten, C., and Luka, B. J. (2012). Prediction during language compre-hension: benefits, costs, and ERP components. Int. J. Psychophysiol. 83, 176-190.
111. Van Petten, C. V. (1993). A com-parison of lexical and sentence-level context effects in event-related potentials. Lang. Cogn. Process. 8, 485-531.
112. Vasishth, S. (2003). Working Memory in Sentence Comprehension: Processing Hindi Center Embeddings. New York: Routledge.
113. Vos, S. H., Gunter, T. C., Schriefers, H. J., and Friederici, A. D. (2001). Syn-tactic parsing and working memory: the effects of syntactic complexity, reading span, and concurrent load. Lang. Cogn. Process. 16, 65-103.
114. Vul, E., and Pashler, H. (2012). Voodoo and circularity errors. Neuroimage 62, 945-948.
115. Wingfield, A., and Butterworth, B. (1984). "Running memory for sen-tences and parts of sentences: syn-tactic parsing as a control function in working memory," in Attention and Performance X: Control of Lan-guage Processes, eds H. Bouma and D. Bouwhuis (London: Lawrence Erl-baum Associates), 351-364.