The effect of word predictability on reading time is logarithmic

Cognition

Volumn 128, Issue 3, 2013, Pages 302-319

  Smith, Nathaniel J ^a   Levy, Roger ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Expectation; Information theory; Probabilistic models of cognition; Psycholinguistics; Reading

Indexed keywords

ARTICLE; EXPECTATION; HUMAN; LANGUAGE; LANGUAGE PROCESSING; LINGUISTICS; PREDICTION; PRIORITY JOURNAL; READING; SEMANTICS; VISUAL DISCRIMINATION;

EXPECTATION; INFORMATION THEORY; PROBABILISTIC MODELS OF COGNITION; PSYCHOLINGUISTICS; READING;

ADULT; ATTENTION; COMPREHENSION; HUMANS; LANGUAGE; REACTION TIME; READING; RECOGNITION (PSYCHOLOGY); TIME FACTORS; VOCABULARY;

EID: 84878795351     PISSN: 00100277     EISSN: 18737838     Source Type: Journal    
DOI: 10.1016/j.cognition.2013.02.013     Document Type: Article

References (95)

1. Adelman J.S., Brown G.D.A., Quesada J.F. Contextual diversity, not word frequency, determines word-naming and lexical decision times. Psychological Science 2006, 17(9):814-823. 10.1111/j.1467-9280.2006.01787.x.
2. Altmann G.T.M., Kamide Y. Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition 1999, 73(3):247-264. 10.1016/S0010-0277(99)00059-1.
3. Atkinson, K. (2004). The VARCON database, version 4.1. http://wordlist.sourceforge.net/.
4. Aylett M., Turk A. The smooth signal redundancy hypothesis: A functional explanation for relationships between redundancy, prosodic prominence, and duration in spontaneous speech. Language & Speech 2004, 47(1):31-56.
5. Baayen R.H. Demythologizing the word frequency effect: A discriminative learning perspective. The Mental Lexicon 2010, 5(3):436-461. 10.1075/ml.5.3.10baa.
6. Baayen R.H. The directed compound graph of English: An exploration of lexical connectivity and its processing consequences. New impulses in word-formation 2010, Vol. 17:383-402. Buske, Hamburg.
7. Baayen R.H., Davidson D.J., Bates D.M. Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language 2008, 59(4):390-412. 10.1016/j.jml.2007.12.005.
8. Bar M. Predictions: A universal principle in the operation of the human brain. Philosophical Transactions of the Royal Society B: Biological Sciences 2009, 364(1521):1181-1182. 10.1098/rstb.2008.0321.
9. Barr D.J., Levy R., Scheepers C., Tily H.J. Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language 2013, 68(3):255-278. 10.1016/j.jml.2012.11.001.
10. Bates, D., Maechler, M., & Bolker, B. (2011). lme4: Linear mixed-effects models using S4 classes [Computer software]. (R package version 0.999375-42).
11. BNC Consortium. (2001). The British National Corpus (version 2, BNC World) [Text corpus]. http://www.natcorp.ox.ac.uk/.
12. Boston M.F., Hale J., Kliegl R., Patil U., Vasishth S. Parsing costs as predictors of reading difficulty: An evaluation using the Potsdam Sentence Corpus. Journal of Eye Movement Research 2008, 2:1-12.
13. Brown C., Hagoort P. The processing nature of the N400: Evidence from masked priming. Journal of Cognitive Neuroscience 1993, 5(1):34-44.
14. Carpenter R.H.S., Williams M.L.L. Neural computation of log likelihood in control of saccadic eye movements. Nature 1995, 377(6544):59-62. 10.1038/377059a0.
15. Chen, S. F., & Goodman, J. (1998). An empirical study of smoothing techniques for language modeling (Technical Report No. TR-10-98). Cambridge, MA: Computer Science Group, Harvard University. ftp://ftp.deas.harvard.edu/techreports/tr-10-98.ps.gz.
16. Clark H.H. The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. Journal of Verbal Learning and Verbal Behavior 1973, 12:335-359.
17. Dahan D., Tanenhaus M.K. Continuous mapping from sound to meaning in spoken-language comprehension: Immediate effects of verb-based thematic constraints. Journal of Experimental Psychology: Learning, Memory, and Cognition 2004, 30(2):498-513.
18. DeLong K.A., Urbach T.P., Kutas M. Probabilistic word pre-activation during language comprehension inferred from electrical brain activity. Nature Neuroscience 2005, 8:1117-1121.
19. Demberg V., Keller F. Data from eye-tracking corpora as evidence for theories of syntactic processing complexity. Cognition 2008, 109(2):193-210.
20. Ehrlich S.F., Rayner K. Contextual effects on word perception and eye movements during reading. Journal of Verbal Learning and Verbal Behavior 1981, 20(6):641-655.
21. Engbert R., Nuthmann A., Richter E.M., Kliegl R. SWIFT: A dynamical model of saccade generation in reading. Psychological Review 2005, 112(4):777-813.
22. Federmeier K.D., Kutas M. A rose by any other name: Long-term memory structure and sentence processing. Journal of Memory and Language 1999, 41:469-495.
23. Feng G. Time course and hazard function: A distributional analysis of fixation duration in reading. Journal of Eye Movement Research 2009, 3(2):1-22.
24. Ferreira F., Clifton C. The independence of syntactic processing. Journal of Memory and Language 1986, 25:348-368.
25. Forster K. Accessing the mental lexicon. New approaches to language mechanisms 1976, 257-287. North-Holland, Amsterdam. R.J. Wales, E.C.T. Walker (Eds.).
26. Foss D.J. A discourse on semantic priming. Cognitive Psychology 1982, 14(4):590-607. 10.1016/0010-0285(82)90020-2.
27. Fossum, V., & Levy, R. (2012). Sequential vs. hierarchical syntactic models of human incremental sentence processing. In Proceedings of the 3rd workshop on cognitive modeling and computational linguistics (CMCL 2012) (pp. 61-69). Montrèal, Canada: Association for Computational Linguistics. http://www.aclweb.org/anthology/W12-1706.
28. Frank S.L., Bod R. Insensitivity of the human sentence-processing system to hierarchical structure. Psychological Science 2011, 22(6):829-834. 10.1177/0956797611409589.
29. Frisson S., Rayner K., Pickering M.J. Effects of contextual predictability and transitional probability on eye movements during reading. Journal of Experimental Psychology: Learning, Memory, and Cognition 2005, 31(5):862-877.
30. Froehlich A.L., Herbranson W.T., Loper J.D., Wood D.M., Shimp C.P. Anticipating by pigeons depends on local statistical information in a serial response time task. Journal of Experimental Psychology: General 2004, 133(1):31-45.
31. Frost R. Toward a strong phonological theory of visual word recognition: True issues and false trails. Psychological Bulletin 1998, 123(1):71-99. (WOS:000071350100006, doi:10.1037//0033-2909.123.1.71).
32. Genzel, D., & Charniak, E. (2002). Entropy rate constancy in text. In Proceedings of ACL.
33. Gold J.I., Shadlen M.N. The neural basis of decision making. Annual Reviews of Neuroscience 2007, 30(1):535-574. 10.1146/annurev.neuro.29.051605.113038.
34. Goodman, J. T. (2001). A bit of progress in language modeling, extended version (Technical Report No. MSR-TR-2001-72). Redmond, WA: Microsoft Research.
35. Hagoort P., Baggio G., Willems R.M. Semantic unification. The new cognitive neurosciences 2009, Vol. 4.:819-836. MIT Press, Boston, MA. 4th ed.
36. Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic model. In Proceedings of NAACL-2001 (pp. 159-166). Stroudsburg, PA: Association for Computational Linguistics.
37. Hess D.J., Foss D.J., Carroll P. Effects of global and local context on lexical processing during language comprehension. Journal of Experimental Psychology: General 1995, 124(1):62-82. 10.1037/0096-3445.124.1.62.
38. Howes D.H., Solomon R.L. Visual duration threshold as a function of word-probability. Journal of Experimental Psychology 1951, 41:401-410.
39. Ilkin Z., Sturt P. Active prediction of syntactic information during sentence processing. Dialogue & Discourse 2011, 2(1):35-58.
40. Jaeger T.F. Redundancy and reduction: Speakers manage syntactic information density. Cognitive Psychology 2010, 61(1):23-62. 10.1016/j.cogpsych.2010.02.002.
41. Janssen P., Shadlen M.N. A representation of the hazard rate of elapsed time in macaque area LIP. Nature Neuroscience 2005, 8(2):234-241. 10.1038/nn1386.
42. Just M.A., Carpenter P.A., Woolley J.D. Paradigms and processes in reading comprehension. Journal of Experimental Psychology: General 1982, 111(2):228-238.
43. Kamide Y., Altmann G.T.M., Haywood S.L. The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements. Journal of Memory and Language 2003, 49(1):133-156. 10.1016/S0749-596X(03)00023-8.
44. Kamide Y., Scheepers C., Altmann G.T.M. Integration of syntactic and semantic information in predictive processing: Cross-linguistic evidence from German and English. Journal of Psycholinguistic Research 2003, 32(1):37-55. 10.1023/A:1021933015362.
45. Kennedy, A., Hill, R., & Pynte, J. (2003). The Dundee corpus. In Proceedings of the 12th European conference on eye movement.
46. Kennedy, A., Pynte, J., Murray, W. S., & Paul, S.-A. (in press). Frequency and predictability effects in the Dundee corpus: An eye movement analysis. The Quarterly Journal of Experimental Psychology. doi:10.1080/17470218.2012.676054.
47. Kliegl R., Nuthmann A., Engbert R. Tracking the mind during reading: The influence of past, present, and future words on fixation durations. Journal of Experimental Psychology: General 2006, 135(1):12-35.
48. Kneser, R., & Ney, H. (1995). Improved backing-off for M-gram language modeling. In Proc. ICASSP (pp. 181-184).
49. Knoeferle P., Crocker M.W., Scheepers C., Pickering M.J. The influence of the immediate visual context on incremental thematic role-assignment: Evidence from eye-movements in depicted events. Cognition 2005, 95(1):95-127. 10.1016/j.cognition.2004.03.002.
50. Kutas M., Federmeier K.D. Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review of Psychology 2011, 62(1):621-647. 10.1146/annurev.psych.093008.131123.
51. Kutas M., Hillyard S.A. Brain potentials reflect word expectancy and semantic association during reading. Nature 1984, 307:161-163.
52. Laming D.R.J. Information theory of choice-reaction times 1968, Academic Press, London.
53. Lau E., Stroud C., Plesch S., Phillips C. The role of structural prediction in rapid syntactic analysis. Brain & Language 2006, 98:74-88.
54. Levy R. Expectation-based syntactic comprehension. Cognition 2008, 106:1126-1177.
55. Levy R., Fedorenko E., Breen M., Gibson T. The processing of extraposed structures in English. Cognition 2012, 122(1):12-36. 10.1016/j.cognition.2011.07.012.
56. Levy R., Jaeger T.F. Speakers optimize information density through syntactic reduction. Advances in Neural Information Processing Systems 2007, 19:849.
57. Manning C.D., Schütze H. Foundations of statistical natural language processing 1999, MIT Press.
58. McClelland J.L., Elman J.L. The TRACE model of speech perception. Cognitive Psychology 1986, 18(1):1-86. 10.1016/0010-0285(86)90015-0.
59. McDonald S.A., Shillcock R.C. Eye movements reveal the online computation of lexical probabilities during reading. Psychological Science 2003, 14(6):648-652.
60. McDonald S.A., Shillcock R.C. Low-level predictive inference in reading: The influence of transitional probabilities on eye movements. Vision Research 2003, 43:1735-1751.
61. McRae K., Spivey-Knowlton M.J., Tanenhaus M.K. Modeling the influence of thematic fit (and other constraints) in on-line sentence comprehension. Journal of Memory and Language 1998, 38(3):283-312.
62. Mitchell D.C. An evaluation of subject-paced reading tasks and other methods for investigating immediate processes in reading. New methods in reading comprehension research 1984, Lawrence Erlbaum Associates, Hillsdale, New Jersey. D.E. Kieras, M.A. Just (Eds.).
63. Morrison C.M., Ellis A.W. Roles of word frequency and age of acquisition in word naming and lexical decision. Journal of Experimental Psychology: Learning, Memory, and Cognition 1995, 21(1):116-133. http://dx.doi.org/10.1037/0278-7393.21.1.116.
64. Morrison C.M., Hirsh K.W., Duggan G.B. Age of acquisition, ageing, and verb production: Normative and experimental data. The Quarterly Journal of Experimental Psychology. A - Human Experimental Psychology 2003, 56(4):705-730. 10.1080/02724980244000594.
65. Morton J. Interaction of information in word recognition. Psychological Review 1969, 76(2):165-178.
66. Murray W.S., Forster K.I. Serial mechanisms in lexical access: The rank hypothesis. Psychological Science 2004, 111(3):721-756.
67. Narayanan, S., & Jurafsky, D. (2004). A Bayesian model of human sentence processing. http://www.icsi.berkeley.edu/snarayan/newcog.pdf.
68. Ni W., Crain S., Shankweiler D. Sidestepping garden paths: Assessing the contributions of syntax, semantics and plausibility in resolving ambiguities. Language & Cognitive Processes 1996, 11(3):283-334.
69. Norris D. The Bayesian reader: Explaining word recognition as an optimal Bayesian decision process. Psychological Review 2006, 113(2):327-357.
70. Norris D. Putting it all together: A unified account of word recognition and reaction-time distributions. Psychological Review 2009, 116(1):207-219.
71. Pang K., Merkel F., Egeth H., Olton D.S. Expectancy and stimulus frequency: A comparative analysis in rats and humans. Perception & Psychophysics 1992, 51(6):607-615.
72. Piantadosi S.T., Tily H., Gibson E. Word lengths are optimized for efficient communication. Proceedings of the National Academy of Sciences 2011, 108(8):3526-3529.
73. Plaut D.C., McClelland J.L., Seidenberg M.S., Patterson K. Understanding normal and impaired word reading: Computational principles in quasi-regular domains. Psychological Review 1996, 103(1):56-115. 10.1037/0033-295X.103.1.56.
74. Pollatsek A., Juhasz B.J., Reichle E.D., Machacek D., Rayner K. Immediate and delayed effects of word frequency and word length on eye movements in reading: A reversed delayed effect of word length. Journal of Experimental Psychology: Human Perception & Performance 2008, 34(3):726-750.
75. Rayner K. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 1998, 124(3):372-422.
76. Rayner K., Well A.D. Effects of contextual constraint on eye movements in reading: A further examination. Psychonomic Bulletin & Review 1996, 3(4):504-509.
77. Reichle E.D., Pollatsek A., Fisher D.L., Rayner K. Toward a model of eye movement control in reading. Psychological Review 1998, 105(1):125-157.
78. Roark, B., Bachrach, A., Cardenas, C., & Pallier, C. (2009). Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing. In Proceedings of the 2009 conference on empirical methods in natural language processing (pp. 324-333).
79. Seidenberg M.S., McClelland J.L. A distributed, developmental model of word recognition and naming. Psychological Review 1989, 96(4):523-568.
80. Shannon C.E. Prediction and entropy of printed English. Bell Systems Technical Journal 1951, 30:50-64.
81. Smith N.J., Levy R. Optimal processing times in reading: A formal model and empirical investigation. Proceedings of the thirtieth annual conference of the Cognitive Science Society 2008, 595-600. Cognitive Science Society, Austin, TX. B.C. Love, K. McRae, V.M. Sloutsky (Eds.).
82. Spivey M.J. The continuity of mind 2007, Oxford University Press, New York.
83. Spivey M.J., Grosjean M., Knoblich G. Continuous attraction towards phonological competitors. Proceedings of the National Academy of Sciences 2005, 102:10393-10398.
84. Staub A., Clifton C. Syntactic prediction in language comprehension: Evidence from either ... or. Journal of Experimental Psychology: Learning, Memory, and Cognition 2006, 32(2):425-436. 10.1037/0278-7393.32.2.425.
85. Stolcke, A. (2002). SRILM-An extensible language modeling toolkit. In Proc. intl. conf. on spoken language processing (Vol. 2, pp. 901-904). Denver.
86. Stone M. Models for choice-reaction time. Psychometrika 1960, 25:251-260.
87. Tanenhaus M.K., Spivey-Knowlton M.J., Eberhard K.M., Sedivy J.C. Integration of visual and linguistic information in spoken language comprehension. Science 1995, 268:1632-1634.
88. Taylor W.L. "Cloze procedure": A new tool for measuring readability. Journalism Quarterly 1953, 30:415-433.
89. Traxler M.J., Foss D.J. Effects of sentence constraint on priming in natural language comprehension. Journal of Experimental Psychology: Learning, Memory, and Cognition 2000, 26(5):1266-1282. 10.1037/0278-7393.26.5.1266.
90. Van Berkum J.J.A., Brown C.M., Zwitserlood P., Kooijman V., Hagoort P. Anticipating upcoming words in discourse: Evidence from ERPs and reading times. Learning and Memory 2005, 31(3):443-467.
91. Wicha N.Y.Y., Bates E.A., Moreno E.M., Kutas M. Potato not pope: Human brain potentials to gender expectation and agreement in spanish spoken sentences. Neuroscience Letters 2003, 346(3):165-168. 10.1016/S0304-3940(03)00599-8.
92. Wood S.N. Stable and efficient multiple smoothing parameter estimation for generalized additive models. Journal of the American Statistical Association 2004, 99:673-686.
93. Wood S.N. Generalized additive models: An introduction with R 2006, Chapman and Hall, Boca Raton.
94. Zevin J.D., Seidenberg M.S. Age of acquisition effects in word reading and other tasks. Journal of Memory and Language 2002, 47(1):1-29. 10.1006/jmla.2001.2834.
95. Zorzi M., Houghton G., Butterworth B. Two routes or one in reading aloud? A connectionist dual-process model. Journal of Experimental Psychology: Human Perception and Performance 1998, 24(4):1131-1161. http://dx.doi.org/10.1037/0096-1523.24.4.1131.