Not Read, but Nevertheless Solved? Three Experiments on PIRLS Multiple Choice Reading Comprehension Test Items

Educational Assessment

Volumn 17, Issue 4, 2012, Pages 214-232

  Sparfeldt, Jörn R ^a   Kimmel, Rumena ^b   Löwenkamp, Lena ^b   Steingräber, Antje ^b   Rost, Detlef H ^b  

^a SAARLAND UNIVERSITY   (Germany)

^b PHILIPPS UNIVERSITY MARBURG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84871110842     PISSN: 10627197     EISSN: None     Source Type: Journal    
DOI: 10.1080/10627197.2012.735921     Document Type: Article

References (40)

1. Best, R. M., Floyd, R. G. and McNamara, D. S. 2008. Differential competencies contributing to children's comprehension of narrative and expository texts. Reading Psychology, 29: 137-164.
2. Bos, W., Lankes, E.-M., Prenzel, M., Schwippert, K., Valtin, R., Voss, A. and Walther, G., eds. 2005. IGLU. Skalenhandbuch zur Dokumentation der Erhebungsinstrumente [IGLU. Scale handbook for the documentation of data collection instruments] Münster, Germany: Waxmann.
3. Brown, J. O., Fishco, V. V. and Hanna, G. 1993. Nelson-Denny Reading Test Chicago, IL: Riverside. (Forms G and H)
4. Cervetti, G. N., Bravo, M. A., Hiebert, E. H., Pearson, P. D. and Jaynes, C. A. 2009. Text genre and science content: Ease of reading, comprehension, and reader preference. Reading Psychology, 30: 487-511.
5. Cohen, J. 1988. Statistical power analysis for the behavioral sciences, 2nd ed. New York, NY: Psychology Press.
6. Coleman, C., Lindstrom, J., Nelson, J., Lindstrom, W. and Gregg, K. N. 2010. Passageless comprehension on the Nelson-Denny Reading Test: Well above chance for university students. Journal of Learning Disabilities, 13: 244-249.
7. Daneman, M. and Hannon, B. 2001. Using working memory theory to investigate the construct validity of multiple-choice reading comprehension tests such as the SAT. Journal of Experimental Psychology: General, 130: 208-223.
8. Drum, P. A., Calfee, R. C. and Cook, L. K. 1981. The effects of surface structure variables on performance in reading comprehension tests. Reading Research Quarterly, 16: 486-514.
9. Gronlund, N. E. and Waugh, C. K. 2009. Assessment of student achievement, 9th ed. Upper Saddle River, NJ: Pearson.
10. Haladyna, T. M. 2004. Developing and validating multiple-choice test items, 3rd ed. Mahwah, NJ: Erlbaum.
11. Israel, S. E. and Duffy, G. G., eds. 2009. Handbook of research on reading comprehension New York, NY: Routledge.
12. Johnston, P. H. 1983. Reading comprehension assessment: A cognitive basis Newark, DE: International Reading Association.
13. Jonckheere, A. R. 1954. A distribution-free k-sample test against ordered alternatives. Biometrika, 41: 133-145.
14. Katz, S., Blackburn, A. B. and Lautenschlager, G. J. 1991. Answering reading comprehension items without passages on the SAT when items are quasi-randomized. Educational and Psychological Measurement, 51: 747-754.
15. Katz, S. and Lautenschlager, G. J. 1995. The SAT reading task in question: Reply to Freedle and Kostin. Psychological Science, 6: 126-127.
16. Katz, S., Lautenschlager, G. J., Blackburn, A. B. and Harris, F. H. 1990. Answering reading comprehension items without passages on the SAT. Psychological Science, 1: 122-127.
17. Katz, S., Marsh, R. L., Johnson, C. and Pohl, E. 2001. Answering quasi-randomized reading items without the passages on the SAT-I. Journal of Educational Psychology, 93: 772-775.
18. Keenan, J. M. and Betjemann, R. S. 2006. Comprehending the Gray Oral Reading Test without reading it: Why comprehension tests should not include passage-independent items. Scientific Studies of Reading, 10: 363-380.
19. Leslie, L. and Caldwell, J. 2009. "Formal and informal measures of reading comprehension.". In Handbook of research on reading comprehension Edited by: Israel, S. E. and Duffy, G. G. 403-427. New York, NY: Routledge.
20. Lifson, S., Scruggs, T. E. and Bennion, K. 1984. Passage independence in reading achievement tests: A follow-up. Perceptual and Motor Skills, 58: 945-946.
21. Liu, O. L., Lee, H.-S. and Linn, M. C. 2011. An investigation of explanation multiple-choice items in science assessment. Educational Assessment, 16: 164-184.
22. Lukhele, R., Thissen, D. and Wainer, H. 1994. On the relative value of multiple-choice, constructed response, and examinee-selected items on two achievement tests. Journal of Educational Measurement, 31: 234-250.
23. Magliano, J. P., Millis, K., Ozuru, Y. and McNamara, D. S. 2007. "A multidimensional framework to evaluate reading assessment tools.". In Reading comprehension strategies: Theories, interventions, and technologies Edited by: McNamara, D. S. 107-136. New York, NY: Erlbaum.
24. Mullis, I. V. S., Martin, M. O., Gonzalez, E. J. and Kennedy, A. M. 2003. PIRLS 2001 international report: IEA's study of reading literacy achievement in primary schools in 35 countries Chestnut Hill, MA: Boston College.
25. Organization for Economic Cooperation and Development. 2001. Knowledge and skills for life: First results from PISA 2000 Paris, France: Author.
26. Pearson, P. D. and Hamm, D. N. 2005. "The assessment of reading comprehension: A review of practices-past, present, and future.". In Children's reading comprehension and assessment Edited by: Paris, S. G. and Stahl, S. A. 13-69. Mahwah, NJ: Erlbaum.
27. Powers, D. E. and Leung, S. W. 1995. Answering the new SAT reading comprehension questions without the passages. Journal of Educational Measurement, 32: 105-129.
28. Preston, R. C. 1964. Ability of students to identify correct responses before reading. Journal of Educational Research, 58: 181-183.
29. Pyrczak, F. 1972. Objective evaluation of the quality of multiple-choice test items designed to measure comprehension of reading passages. Reading Research Quarterly, 8: 62-71.
30. Pyrczak, F. 1974. Passage-dependence of items designed to measure the ability to identify the main ideas of paragraphs: Implications for validity. Educational and Psychological Measurement, 34: 343-348.
31. Pyrczak, F. 1976. Context-independence of items designed to measure the ability to derive the meanings of words from their context. Educational and Psychological Measurement, 36: 919-924.
32. Rodriguez, M. C. 2003. Construct equivalence of multiple-choice and constructed-response items: A random effects synthesis of correlations. Journal of Educational Measurement, 40: 163-184.
33. Rost, D. H. and Sparfeldt, J. R. 2007. Leseverständnis ohne Lesen? Zur Konstruktvalidität von multiple-choice-Leseverständnistestaufgaben [Reading comprehension without reading? On the construct validity of multiple-choice reading comprehension test items]. Zeitschrift für Pädagogische Psychologie, 21: 305-314.
34. Sainsbury, M. and Campbell, J. 2003. "Developing the PIRLS reading assessment.". In PIRLS-PIRLS 2001 technical report Edited by: Martin, M. O., Mullis, I. V. S. and Kennedy, A. M. 13-27. Boston: International Study Center, Lynch School of Education, Boston Colllege.
35. Samuels, S. J. 1968. Effect of word associations on reading speed, recall, and guessing behaviour on tests. Journal of Educational Psychology, 59: 12-15.
36. Thorndike, R. L. 1951. "Reliability.". In Educational measurement Edited by: Lindquist, E. F. 560-620. Washington, DC: American Council on Education.
37. Tian, S. 2006. Passage independency of reading comprehension items in the GEPT and the TOEFL. The Reading Matrix, 6: 66-84.
38. Tuinman, J. J. 1973-1974. Determining the passage dependency of comprehension questions in 5 major tests. Reading Research Quarterly, 9: 206-223.
39. van Dijk, T. A. and Kintsch, W. 1983. Strategies of discourse comprehension New York, NY: Academic Press.
40. Wolfe, M. B. W. and Mienko, J. A. 2007. Learning and memory of factual content from narrative and expository text. British Journal of Educational Psychology, 77: 541-564.