Learning first-pass structural attachment preferences with dynamic grammars and recursive neural networks

Cognition

Volumn 88, Issue 2, 2003, Pages 133-169

  Sturt, Patrick ^a   Costa, Fabrizio ^b   Lombardo, Vincenzo ^c   Frasconi, Paolo ^b  

^a UNIVERSITY OF GLASGOW   (United Kingdom)

^b UNIVERSITY OF FLORENCE   (Italy)

^c UNIVERSITY OF TURIN   (Italy)

Author keywords

Ambiguity resolution; Connectionism; Hybrid models; Parsing

Indexed keywords

CODES (SYMBOLS); COGNITIVE SYSTEMS; COMPUTER SIMULATION; HYBRID COMPUTERS; NATURAL LANGUAGE PROCESSING SYSTEMS; NEURAL NETWORKS; SYNTACTICS;

SYMBOLIC GRAMMERS;

COMPUTATIONAL GRAMMARS;

ARTICLE; HYBRID SYSTEM; LANGUAGE ABILITY; LEARNING; LINGUISTICS; MENTAL FUNCTION; MENTAL TASK; MENTAL TEST; MODEL; NERVE CELL NETWORK; PRIORITY JOURNAL; REPRODUCIBILITY; SYMBOLISM;

EID: 0038392958     PISSN: 00100277     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0010-0277(03)00026-X     Document Type: Article

References (62)

1. Altmann G., Steedman M. Interaction with context during human sentence processing. Cognition. 30:1988;191-238.
2. Altmann G.T.M., van Nice K.Y., Garnham A., Henstra J.-A. Late closure in context. Journal of Memory and Language. 38:1998;459-484.
3. Bader M., Lasser I. German verb-final clauses and sentence processing. Clifton C., Frazier L., Rayner K. Perspectives on sentence processing. 1994;Lawrence Erlbaum Associates, Hillsdale, NJ.
4. Bever T. The cognitive basis for linguistic structures. Cognition and the development of language. 1970;279-360 Wiley, New York.
5. Branigan H.P., Pickering M.J., Liversedge S.P., Stewart A.J., Urbach T.P. Syntactic priming - investigating the mental representation of language. Journal of Psycholinguistic Research. 24:1995;489-506.
6. Brants T., Crocker M.W. Probabilistic parsing and psychological plausibility. Proceedings of the 18th International Conference on Computational Linguistics. 2000;Morgan Kaufmann, San Francisco, CA.
7. Cleeremans A., Servan-Schreiber D., McClelland J.L. Finite state automata and simple recurrent networks. Neural Computation. 1:(3):1989;372-381.
8. Collins M. Three generative, lexicalised models for statistical parsing. Proceedings of the 35th annual meeting of the Association for Computational Linguistics. 1997;16-23 Morgan Kaufmann, San Francisco, CA.
9. Collins M.J. A new statistical parser based on bigram lexical dependencies. Proceedings of the 34th annual meeting of the Association for Computational Linguistics. 1996;Morgan Kaufmann, San Francisco, CA.
10. Costa, F., Frasconi, P., Lombardo, V., & Soda, G. (in press). Towards incremental parsing of natural language using recursive neural networks. Applied Intelligence.
11. Costa F., Frasconi P., Lombardo V., Sturt P., Soda G. Enhancing first-pass attachment prediction. Proceedings of the 15th European Conference on Artificial Intelligence, Lyons. 2002;509-512 IOS Press, Amsterdam.
12. Costa, F., Frasconi, P., Sturt, P., & Lombardo, V. (2002, March). Exploring the effect of experience on a recursive neural network model of structural preferences. Paper presented at the 15th annual CUNY Conference on Human Sentence Processing, New York.
13. Crocker M.W., Brants T. Wide coverage probabilistic sentence processing. Journal of Psycholinguistic Research. 29:2000;647-669.
14. Cuetos F., Mitchell D.C. Cross-linguistic differences in parsing: restrictions on the use of the late closure strategy in Spanish. Cognition. 30:1988;72-105.
15. Cuetos F., Mitchell D.C., Corley M.M.B. Parsing in different languages. Carreiras M., García-Albea J.E., Sebastián-Galles N. Language processing in Spanish. 1996;145-187 Erlbaum, Hillsdale, NJ.
16. Elman J.L. Finding structure in time. Cognitive Science. 14:1990;179-211.
17. Fodor J.D. Parsing to learn. Journal of Psycholinguistic Research. 27:1998;339-374.
18. Fodor J.D., Ferreira F. Reanalysis in sentence processing. 1998;Kluwer, Dordrecht.
19. Frasconi P., Gori M., Sperduti A. A general framework for adaptive processing of data structures. IEEE Transactions on Neural Networks. 9:1998;768-786.
20. Frazier, L. (1978). On comprehending sentences: syntactic parsing strategies. PhD thesis, University of Connecticut, Storrs, CT.
21. Frazier L. Sentence processing: a tutorial review. Coltheart M. Attention and performance. 1987;559-586 Erlbaum, Hillsdale, NJ.
22. Frazier L., Rayner K. Making and correcting errors during sentence comprehension: eye movements in the analysis of structurally ambiguous sentences. Cognitive Psychology. 14:1982;178-210.
23. Garnsey S.M., Pearlmutter N.J., Myers E., Lotocky M.A. The contributions of verb bias and plausibility to the comprehension of temporarily ambiguous sentences. Journal of Memory and Language. 37:1997;58-93.
24. Gibson E., Pearlmutter N., Canesco-Gonzalez E., Hickok G. Recency preference in the human sentence processing mechanism. Cognition. 59:(1):1996;23-59.
25. Gibson E., Schuetze C.T., Salomon A. The relationship between the frequency and the processing complexity of linguistic structure. Journal of Psycholinguistic Research. 25:1996;59-92.
26. Gibson, E. A. F. (1991). A computational theory of human linguistic processing: memory limitations and processing breakdown. PhD thesis, Carnegie Mellon University, Pittsburgh, PA.
27. Gibson E.A.F. Linguistic complexity: locality of syntactic dependencies. Cognition. 68:(1):1998;1-76.
28. Gilboy E., Sopena J.M., Clifton C. Jr., Frazier L. Argument structure and association preferences in Spanish and English compound NPs. Cognition. 54:1995;131-167.
29. Goller C., Kuechler A. Learning task-dependent distributed structure-representations by back-propagation through structure. Proceedings of the IEEE International Conference on Neural Networks. 1996;347-352 IEEE, Washington, DC.
30. Hindle D., Rooth M. Structural ambiguity and lexical relations. Computational Linguistics. 19:1993;103-120.
31. Jurafsky D. A probabilistic model of lexical and syntactic access and disambiguation. Cognitive Science. 20:1996;137-194.
32. Kamide Y., Mitchell D.C. Incremental pre-head attachment in Japanese parsing. Language and Cognitive Processes. 14:1999;631-632.
33. Kempson R., Meyer-Viol W., Gabbay D. Dynamic syntax: the flow of language understanding. 2000;Blackwell, Oxford.
34. Kennison S.M. Limitations on the use of verb information during sentence comprehension. Psychonomic Bulletin and Review. 8:2001;132-137.
35. Kimball J. Seven principles of surface structure parsing in natural language. Cognition. 2:(1):1973;15-47.
36. Konieczny L., Hemforth B., Scheepers C., Strube G. The role of lexical heads in parsing: evidence from German. Language and Cognitive Processes. 12:1997;307-348.
37. Lombardo V., Sturt P. Incrementality and lexicalism: a treebank study. Stevenson S., Merlo P. The lexical basis of sentence processing: formal, computational and experimental issues. 2002;137-155 John Benjamins, Philadelphia, PA.
38. Marcus M.P., Santorini B., Marcinkiewicz M.A. Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics. 19:1993;313-330.
39. Marslen-Wilson W. Linguistic structure and speech shadowing at very short latencies. Nature. 244:1973;522-533.
40. McRae K., Spivey-Knowlton M.J., Tanenhaus M.K. Modeling the influence of thematic fit (and other constraints) in on-line sentence comprehension. Journal of Memory and Language. 38:1998;283-312.
41. Milward D. Dynamic dependency grammar. Linguistics and Philosophy. 17:1994;561-605.
42. Mitchell D.C. Verb guidance and other lexical effects in parsing. Language and Cognitive Processes. 4:(3):1989;123-154.
43. Mitchell D.C., Cuetos F., Corley M.M.B., Brysbaert M. Exposure-based models of human parsing: evidence for the use of coarse-grained (nonlexical) statistical records. Journal of Psycholinguistic Research. 24:1995;469-488.
44. Phillips, C. (1996). Order and structure. PhD thesis, Department of Linguistics and Philosophy, MIT, Cambridge, MA.
45. Pickering M.J., Traxler M.J. Plausibility and recovery from garden paths: an eye-tracking study. Journal of Experimental Psychology: Learning, Memory, and Cognition. 24:(4):1998;940-961.
46. Pickering M.J., Traxler M.J., Crocker M.W. Ambiguity resolution in sentence processing: evidence against frequency-based accounts. Journal of Memory and Language. 43:2000;447-475.
47. Pollack J.B. Recursive distributed representations. Artificial Intelligence. 46:(1-2):1990;77-106.
48. Rayner K., Carlson M., Frazier L. The interaction of syntax and semantics during sentence processing: eye movements in the analysis of semantically biased sentences. Journal of Verbal Learning and Verbal Behavior. 22:1983;358-374.
49. Rayner K., Frazier L. Parsing temporarily ambiguous complements. Quarterly Journal of Psychology. 39A:1987;657-673.
50. Roark B., Johnson M. Efficient probabilistic top-down and left-corner parsing. Proceedings of the 37th annual meeting of the Association for Computational Linguistics. 1999;421-428 Morgan Kaufmann, San Francisco, CA.
51. Rohde, D. L. T. (2002). A connectionist model of sentence comprehension and production. PhD thesis, Carnegie Mellon University, Pittsburgh, PA.
52. Schuetze C., Gibson E.A. Argumenthood and English prepositional phrase attachment. Journal of Memory and Language. 40:(3):1999;409-431.
53. Sperduti A., Starita A. Supervised neural networks for the classification of structures. IEEE Transactions on Neural Networks. 8:1997;714-735.
54. Spivey M.J., Tanenhaus M.K. Syntactic ambiguity resolution in discourse: modeling the effects of referential context and lexical frequency. Journal of Experimental Psychology: Learning, Memory and Cognition. 24:(6):1998;1521-1543.
55. Spivey-Knowlton M., Sedivy J.C. Resolving attachment ambiguities with multiple constraints. Cognition. 55:1995;227-267.
56. Stabler E.P. The finite connectivity of linguistic structure. Clifton C., Frazier L., Rayner K. Perspectives on sentence processing. 1994;303-336 Lawrence Erlbaum, Hillsdale, NJ.
57. Steedman M.J. Grammar, interpretation and processing from the lexicon. Wilson W.M. Lexical representation and process. 1989;463-504 MIT Press, Cambridge, MA.
58. Stevenson S. Competition and recency in a hybrid network model of syntactic disambiguation. Journal of Psycholinguistic Research. 23:(4):1994;295-321.
59. Traxler M.J., Pickering M.J., Clifton C. Architectures and mechanisms that process prepositional phrases and relative clauses. Journal of Memory and Language. 39:1998;558-592.
60. Trueswell J.C., Tanenhaus M.K., Kello C. Verb-specific constraints on sentence processing: separating effects of lexical preference from garden paths. Journal of Experimental Psychology: Learning, Memory, and Cognition. 19:(3):1993;528-553.
61. Tugwell, D. (1998). Dynamic syntax. PhD thesis, Centre for Cognitive Science, University of Edinburgh, Edinburgh.
62. Vosse T., Kempen G. Syntactic structure assembly in human parsing: a computational model based on competitive inhibition and a lexicalist grammar. Cognition. 75:2000;105-143.