GENERATING AUTOMATED TEXT COMPLEXITY CLASSIFICATIONS THAT ARE ALIGNED WITH TARGETED TEXT COMPLEXITY STANDARDS

ETS Research Report Series

Volumn 2010, Issue 2, 2010, Pages i-44

  Sheehan, Kathleen M ^a   Kostin, Irene ^a   Futagi, Yoko ^a   Flor, Michael ^a  

^a EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84928113874     PISSN: None     EISSN: 23308516     Source Type: Journal    
DOI: 10.1002/j.2333-8504.2010.tb02235.x     Document Type: Article

References (64)

1. ACT, Inc. (2006). Reading between the lines: What the ACT reveals about college readiness in reading. Iowa City, IA.
2. American Institutes for Research (2008). Reading framework for the 2009 National Assessment of Educational Progress. Washington, DC: National Assessment Governing Board.
3. Baretta, L., Tomitch, L.M.B., MacNair, N., Lim, V.K. & Waldie, K.E. (2009). Inference making while reading narrative and expository texts: An ERP study. Psychology & Neuroscience, 2(2), 137–145.
4. Beck, I. L., McKeown, M. G., & Worthy, J. (1995). Giving a text voice can improve students' understanding. Reading Research Quarterly, 30(2), 220–238.
5. Beck, I. L., McKeown, M. G., & Sinatra, G.M., & Loxterman, J.A. (1991). Revising social studies text from a text processing perspective: Evidence of improved comprehensibility. Reading Research Quarterly, 26, 251–276.
6. Best, Floyd, & McNamara (2004, April). Understanding the fourth-grade slump: Comprehension difficulties as a function of reader aptitudes and text genre. Paper presented at the meeting of the American Educational Research Association, San Diego, CA.
7. Biber, D. (1986). Spoken and written textual dimension in English: Resolving the contradictory findings. Language, 62, 394–414.
8. Biber, D. (1988). Variation across Speech and Writing. Cambridge, England: Cambridge University Press.
9. Biber, D., Conrad, S., Reppen, R., Byrd, P., Helt, M., Clark, V., et al., (2004). Representing Language use in the University: Analysis of the TOEFL 2000 Spoken and Written Academic Language Corpus. (TOEFL Monograph Series, MS-25). Princeton, NJ: ETS.
10. Biber, D. Johansson, S., Leech, G., Conrad, S. & Finegan, E. (1999) Longman grammar of spoken and written English. Essex, England: Pearson Education Limited.
11. Breland, H. M., Jones, R. J., & Jenkins, L. (1994). The College Board Vocabulary Study. New York, NY: College Entrance Examination Board.
12. Britton, B., & Gulgoz, S. (1991) Using Kintsch's computational model to improve instructional text: Effects of repairing inference calls on recall and cognitive structures. Journal of Educational Psychology, 83, 329–345.
13. Carroll, J. B., Davies, P., & Richmond, B. (1971). The American Heritage word frequency book. Boston, MA: Houghton Mifflin.
14. Carver, R.P. (1985) Measuring readability using DRP units. Journal of Reading Behavior, 17, 303–316.
15. Chall, J. S. & Dale, E. (1995). Readability Revisited - The new Dale-Chall readability formula. Cambridge, MA: Brookline Books.
16. Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. Journal of the American Statistical Association, 74, 829–836.
17. Coleman, M. and Liau, T.L. (1975). A computer readability formula designed for machine scoring, Journal of Applied Psychology, 60(2), 283–284.
18. Coltheart, M. (1981). The MRC psycholinguistic database, Quarterly Journal of Experimental Psychology, 33 A, 497–505.
19. Council of Chief State School Officers (2010, June). Common Core State Standards.
20. Coxhead, A. (2000) A new academic word list. TESOL Quarterly, 34, 213–238.
21. Crossley, S.A., Greenfield, J., & McNamara, D.S. (2008). Assessing text readability using cognitively-based indices. TESOL Quarterly, 42(3), 475–493.
22. Davidson, A. and Kantor, R. N. (1982). On the failure of readability formulas to define readable text: A case study from adaptations. Reading Research Quarterly, 17, 187–209.
23. Dale, E. & Chall, J.S. (1948). A Formula for Predicting Readability. Educational Research Bulletin, 27, 11–20.
24. DuBay, W.H. (2004). The Principles of Readability. Costa Mesa, CA: Impact Information. Retrieved from http://www.impact-information.com/impactinfo/readability02.pdf.
25. Finn, P. J. (1978). Word frequency, information theory, and cloze performance: A transfer theory of processing in reading. Reading Research Quarterly, 13, 508–537.
26. Flesch, Rudolf F. (1948). A new readability yardstick, Journal of Applied Psychology, 32, 221–233.
27. Fry, E. (1968). A readability formula that saves time. Journal of Reading, 11, 513–516.
28. Graesser, A.C., McNamara, D.S., & Louwerse, M. (2003). What readers need to learn in order to process coherence relations in narrative and expository text. In A.P. Sweet & C. E. Snow (Eds.), Rethinking reading comprehension (pp. 82–98). New York, NY: Guilford Press.
29. Graesser, A. C., McNamara, D.S., Louwerse, M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavioral Research, Methods, Instruments and Computers, 36, 193–202.
30. Gunning R. (1952). The technique of clear writing. New York, NY: McGraw-Hill International.
31. Heilman, M., Collins-Thompson, Callan, J. & Eskenazi, M. (2007) Combining lexical and grammatical features to improve readability measures for first and second language texts. Proceedings of NAACL HLT 2007, 460–467.
32. Heilman, M., Collins-Thompson, & Eskenazi, M. (June, 2008) An Analysis of Statistical Models and Features for Reading Difficulty Prediction Proceedings of the 46th Annual Meeting of the ACL, 71-79, Columbus, OH.
33. Heibert, E.H. (2009). Interpreting Lexiles in Online Contexts and with Informational Texts. Seattle, WA: Apex Learning, Inc.
34. Holland, V. M. (1981). Psycholinguistic Alternatives to Readability Formulas, Washington, DC: American Institutes for Research.
35. Kincaid, J.P., Fishburne, R.P, Rogers, R.L. & Chissom, B.S. (1975). Derivation of new readability formulas (automated readability index, Fog count and Flesch reading ease formula) for navy enlisted personnel. (Research Branch Report 8-75.) Millington, TN: Naval Technical Training, U. S. Naval Air Station, Memphis, TN.
36. Kintsch, W. (1988). The use of knowledge in discourse processing: A construction-integration model. Psychological Review, 95, 163–182.
37. Kintsch, W. & Yarbrough, J. C. (1982). Role of rhetorical structure in text comprehension. Journal of Educational Psychology, 74, 828–834.
38. Klare, G. (1984). Readability. In P. D. Pearson (Ed.) Handbook of reading research. (pp.681–744). New York, NY: Longman.
39. Kukan, L., & Beck, I. L. (1997). Thinking aloud and reading comprehension research: Inquiry, instruction, and social interaction. Review of Educational Research. 67, 271–299.
40. Lee, D. Y. W. (2001). Defining core vocabulary and tracking its distribution across spoken and written genres. Journal of English Linguistics. 29, 250–278.
41. Leys, M., Fielding, L., Herman, P., & Pearson, P. D. (1983). Does cloze measure intersentence comprehension? A modified replication of Shanahan, Kamil, and Tobin. In J. A. Niles & L. A. Harris (Eds.), Searches for meaning in reading/language processing and instruction: Thirty-second yearbook of the National Reading Conference (pp. 111–114). Rochester, NY: National Reading Conference.
42. Lu, X. (in press). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers' language development. TESOL Quarterly, 44(4)
43. McCarthy, P., Graesser, A. C., & McNamara, D. S. (2006, July). Distinguishing genre using Coh-Metrix indices of cohesion. Poster presented at the annual meetings of the Society for Text and Discourse, Minneapolis, MN.
44. McNamara, D.S., Louwerse, M.M., McCarthy, P.M. & Graesser, A.C. (2010). Coh-Metrix: Capturing linguistic features of cohesion, Discourse Processes, 47(4), 292–330.
45. Pearson, P. D, and Hamm, D. N., (2005). The assessment of reading comprehension: A review of practices—past, present and future. In S.G. Paris and S. A. Stahl (Eds.), Children's Reading Comprehension and Assessment (pp. 13–70). Mahwah, NJ: Lawrence Erlbaum Associates.
46. Powers, R.D., Sumner, W.A., and Kearl, B.E. (1958). A recalculation of four adult readability formulas, Journal of Educational Psychology, 49(2), 99–105.
47. Rasch, G. (1960). Probabilistic Models for Some Intelligence and Attainment Tests. Copenhagen, Denmark, The Danish Institute for Educational Research.
48. Reppen, R. (2001). Register variation in student and adult speech and writing. In S. Conrad and D. Biber (Eds.), Variation in English: Multi-dimensional studies (pp. 187–199). London, England: Longman.
49. Resnick, L., Stein, M.K. & Koon, S. (2008). Standards Based Reform: A Powerful Idea Unmoored. In. R.D. Kahlenberg (Ed.), Improving on No Child Left Behind. New York, NY: Century Foundation Press.
50. Sawyer, M. H. (1991). A review of research in revising instructional text. Journal of Reading Behavior, 23(3), 307–333.
51. Schmeiser, C.B. & Welch, C.J. (2006). Test development. In R. L. Brennan (Ed.) Educational measurement (pp. 307–354). Westport, CT: American Council on Education and Praeger Publishers.
52. Shanahan, T., Kamil, M. L., & Tobin, A. W. (1983). Cloze as a measure of intersentential comprehension. Reading Research Quarterly, 17, 229–255.
53. Sheehan, K.M., Kostin, I., & Futagi, Y. (2009). U.S. Patent No. 20090197225. Reading Level Assessment Method, System, and Computer Program Product for High Stakes Testing Applications. U.S. Patent and Trademark Office.
54. Sheehan, K.M., Kostin, I., & Futagi, Y. (2008a, July). When do standard approaches for measuring vocabulary difficulty, syntactic complexity and referential cohesion yield biased estimates of text difficulty? Proceedings of the 30th Annual Meeting of the Cognitive Science Society, Washington DC.
55. Sheehan, K.M., Kostin, I., & Futagi, Y., (2008b, July). Reading level assessment for high-stakes testing applications: A second look at variation due to differences in text genre. Paper presented at the Annual Meeting of the Society for Text and Discourse, Memphis, TN.
56. Sheehan, K.M., Kostin, I., & Futagi, Y. (2007, October). SourceFinder: A construct-driven approach for locating appropriately targeted reading comprehension source texts. Proceedings of the 2007 Workshop of the International Speech Communication Association, Special Interest Group on Speech and Language Technology in Education, Farmington, PA.
57. Stenner, A. J., Burdick, H., Sanford, E., & Burdick, D. (2006). How accurate are Lexile text measures? Journal of Applied Measurement, 7(3), 307–322.
58. Stenner, A. J. (1996, February) Measuring Reading Comprehension with The Lexile Framework, presented at the Fourth North American Conference on Adolescent/Adult Literacy, Washington, DC.
59. Stenner, A. J., Horabin, I., Smith, D. R., & Smith, M. (1988). Most comprehension tests do measure reading comprehension: A response to McLean and Goldstein. Phi Delta Kappan, 79(10), 765–767.
60. Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics. Boston, MA: Allyn & Bacon.
61. van den Broek, P., Everson, M., Virtue, S., Sung, Y., & Tzeng, Y. (2002). Comprehension and memory of science texts: Inferential processes and the construction of a mental representation. In J. Otero, J. Leon, & A. C. Graesser (Eds.), The psychology of science text comprehension (pp. 131–154). Mahwah, NJ: Lawrence Erlbaum Associates.
62. Williamson, G.L. (2006). Aligning the journey with a destination: A model for K-16 reading standards. Durham, NC: Metametrics, Inc.
63. Zabrucky, K. M. & Moore, D. (1999). Influence of text genre on adults' monitoring of understanding and recall. Educational Gerontology, 25, 691–710.
64. Zeno, S. M., Ivens, S. H., Millard, R. T., & Duvvuri, R. (1995). The educator's word frequency guide. Brewster, NY: Touchstone Applied Science Associates.