How assessing reading comprehension with multiple-choice questions shapes the construct: A cognitive processing perspective

Language Testing

Volumn 23, Issue 4, 2006, Pages 441-474

  Rupp, André A ^a   Ferne, Tracy ^b   Choi, Hyeran ^b  

^a HUMBOLDT UNIVERSITY   (Germany)

^b UNIVERSITY OF OTTAWA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33750582967     PISSN: 02655322     EISSN: 14770946     Source Type: Journal    
DOI: 10.1191/0265532206lt337oa     Document Type: Article

References (71)

1. Aebersold, J. and Field, M. 1997: From reader to reading teacher: issues and strategies for second language classrooms. Cambridge: Cambridge University Press.
2. Alderson, J.C. 2000: Assessing reading. Cambridge: Cambridge University Press.
3. Allan, A. 1992: Development and validation of a scale to measure test-wiseness in EFL/ESL reading test takers. Language Testing 9, 101-22.
4. Anderson, R. 1991: Individual differences in strategy use in second language reading and testing. Modern Language Journal 75, 460-72.
5. Anderson, R. and Pearson, P. 1984: A schema-theoretic view of basic processes in reading comprehension. In Pearson, P.D., editor, Handbook of reading research. New York: Longman, 255-91.
6. Barnett, M. 1989: More than meets the eye: foreign language reading theory and practice. Englewood Cliffs, NJ: CAL and Prentice-Hall.
7. Birch, B. 2002: English L2 reading: getting to the bottom. Mahwah, NJ: Erlbaum.
8. Block, E.L. 1986: The comprehension strategies of second language readers. TESOL Quarterly 20, 463-94.
9. Brownstein, S.C., Weiner, M., Green, S.W. and Hilbert, S. 1999: How to prepare for the GRE. Hauppauge, NY: Barron.
10. Buck, G., Tatsuoka, K. and Kostin, I. 1997: The subskills of reading: Rule-space analysis of a MC test of second language reading comprehension. Language Learning 47, 423-66.
11. Carrell, P. 1984: Schema theory and ESL reading: classroom implications and applications. The Modern Language Journal 68, 332-43.
12. Carver, R.P. 1997: Reading for one second, one minute, or one year from the perspective of rauding theory. Scientific Studies of Reading 1, 3-43.
13. Clarke, A. 1979: Reading in Spanish and English. Language Learning 29, 121-50.
14. Creswell, J.W. 1998: Qualitative inquiry and research design: choosing among five traditions. Thousand Oaks, CA: Sage.
15. Drum, P.A., Calfee, R.C. and Cook, L.K. 1981: The effects of surface structure variables on performance in reading comprehension tests. Reading Research Quarterly 16, 486-514.
16. Embretson, S.E. and Wetzel, D. 1987: Component latent trait models or paragraph comprehension tests. Applied Psychological Measurement 11, 175-93.
17. Enright, M.K., Grabe, W., Koda, K., Mosenthal, P., Mulcahy-Ernt, P. and Schedl, M. 2000: TOEFL 2000 Reading Framework: a working paper. TOEFL Monograph Series MS-17. Princeton, NJ: Educational Testing Service.
18. Farr, R., Pritchard, R. and Smitten, B. 1990: A description of what happens when an examinee takes a multiple-choice reading comprehension test. Journal of Educational Measurement 27, 209-26.
19. Freedle, R. and Kostin, I. 1991: The prediction of SAT reading comprehension item difficulty for expository prose passages. Research Report RR 91-29. Princeton, NJ: Educational Testing Service.
20. Freedle, R. and Kostin, I. 1992: The prediction of GRE reading comprehension item difficulty for expository prose passages for each of three item types: main Ideas, inferences, and explicit statements. Research Report RR 91-59. Princeton, NJ: Educational Testing Service.
21. Freedle, R. and Kostin, I. 1993: The prediction of TOEFL reading item difficulty: implications for construct validity. Language Testing 10, 133-70.
22. Gallagher, N. 2000: DELTA's key to the TOEFL test McHenry, IL: Delta.
23. Gernsbacher, M.A. 1990: Language comprehension as structure building. Hillsdale, NJ: Erlbaum.
24. Gernsbacher, M.A. 1997: Two decades of structure building. Discourse Processes 23, 265-304.
25. Goldman, S.R. 1997: Learning from text: Reflections on the past and suggestions for the future. Discourse Processes 23, 357-98.
26. Goodman, E., Ojserkis, R. and Verini, B. 2004: GRE exam. New York: Simon and Schuster.
27. Goodman, E., Ojserkis, R. and Verini, B. 2005: GRE exam. New York: Kaplan
28. Gorin, J.S. 2002: Cognitive and psychometric modeling of text-based reading comprehension GRE-V test items. Unpublished doctoral dissertation, University of Kansas.
29. Gorin, J.S. 2005: Manipulation of processing difficulty of reading comprehension test questions: the feasibility of verbal item generation. Journal of Educational Measurement 42, 351-73.
30. Gorin, J.S. and Embretson, S.E. in press: Predicting item properties without tryout: cognitive modeling of paragraph comprehension items. Applied Psychological Measurement.
31. Grabe, W. 2000: Developments in reading research and their implication for computer-adaptive reading assessment. In Chalhoub-Deville, M., editor, Issues in computer-adaptive tests of reading, Cambridge: Cambridge University Press, 11-47.
32. Grabe, W. 2002: Narrative and expository macro-genres. In Johns, A.M., editor, Genre in the classroom: multiple perspectives. Mahwah, NJ: Erlbaum, 249-67.
33. Green, S.W. and Wolf, I.K. 2000: How to prepare for the GRE. Hauppauge, NY: Barron.
34. Hartz, S.M. 2002: A Bayesian guide for the Unified Model for assessing cognitive abilities: blending theory with practicality. Unpublished doctoral dissertation, University of Illinois, Urbana-Champaign.
35. Hinkel, E. 2004: TOEFL test strategies. Hauppauge, NY: Barron.
36. Hosenfeld, C. 1977: A preliminary investigation of the reading strategies of successful and nonsuccessful second language learners. System 5, 110-23.
37. Junker, B.W. 1999: Some statistical models and computational methods that may be useful for cognitively-relevant assessment. Unpublished manuscript. Available online at http://www.stat.cmu.edu/~brian/nrc/cfa (June 2006).
38. Katz, S., Blackburn, A.B. and Lautenschlager, G. 1991: Answering reading comprehension items without passages on the SAT when the items are quasi-randomized. Educational and Psychological Measurement 51, 747-54.
39. Katz, S. and Lautenschlager, G.J. 1994: Answering reading comprehension questions without passages on the SAT-I, ACT, and GRE. Educational Assessment 2, 295-308.
40. Katz, S. and Lautenschlager, G.J. 2001: The contribution of passage and no-passage factors to item performance on the SAT reading task. Educational Assessment 7, 165-76.
41. Katz, S., Lautenschlager, G.J., Blackburn, A.B. and Harris, F. 1990: Answering reading comprehension items without passages on the SAT. Psychological Science 1, 122-27.
42. Kintsch, W. 1998: Comprehension: a paradigm for cognition. New York: Cambridge University Press.
43. Kitao, S.K. and Kitao, K. 2002: Testing reading. Asian Journal of English Language Teaching 12, 161-78.
44. Kobayashi, M. 2002: Method effects on reading comprehension test performance: Text organization and response format. Language Testing 19, 193-220.
45. Leighton, J.P. 2004: Avoiding misconception, misuse, and missed opportunities: the collection of verbal reports in educational achievement testing. Educational Measurement: Issues and Practice 23, 6-15.
46. Lougheed, L. 2003: How to prepare for the TOEFL test. Hauppauge, NY: Barron.
47. Lurie, K., Pecsenye, M. and Robinson, A. 2005: Cracking the GRE. New York: Random House.
48. Martinson, T.H. 2005. Master the GRE. Lawrenceville, NJ: Thomson Peterson's.
49. McNamara, D.S. and Kintsch, W. 1996. Learning from texts: effects of prior knowledge and text coherence. Discourse Processes 22, 247-88.
50. Nevo, N. 1989. Test-taking strategies on a MC test of reading comprehension. Language Testing 6, 199-215.
51. Nichols, P.D., Chipman, S.F. and Brennan, R.L. editors, 1995: Cognitively diagnostic assessment. Hillsdale, NJ: Erlbaum.
52. NVivo [Software Program] 2005: Doncaster, Victoria, Australia: QSR International.
53. Perfetti, C.A. 1985: Reading ability. New York: Oxford University Press.
54. Perfetti, C.A. 1997: Sentences, individual differences, and multiple texts: three issues in text comprehension. Discourse Processes 23, 337-55.
55. Powers, D.E. and Leung, S.W. 1995: Answering the new SAT reading comprehension questions without the passages. Journal of Educational Measurement 32, 105-29.
56. Rogers, B. 2005: TOEFL success. Lawrenceville, NJ: Thomson Peterson's.
57. Rumelhart, D. 1980: Schemata: the building blocks of cognition. In Spiro, R.J., Bruce, B.C. and Brewer, B.W., editors, Theoretical issues in reading comprehension. Hillsdale, NJ: Erlbaum, 33-58.
58. Rupp, A.A. in press: The answer is in the question: a guide to describing and investigating the conceptual foundations and statistical properties of cognitive psychometric models. International Journal of Testing.
59. Rupp, A.A., Garcia, P. and Jamieson, J. 2001: Combining multiple regression and CART to understand difficulty in second language reading and listening comprehension test items. International Journal of Testing 1, 185-216.
60. Rymniak, M. and Shanks, J. 2002: TOEFL CBT exam. New York: Kaplan.
61. Sarig, G. 1987: High-level reading in first and in the foreign language: some comparative process data. In Devine, J. Carrell, P.L. and Eskey, D.E. editors, Research in reading as a second language. Washington, DC: Teachers of English to Speakers of Other Language 2, 105-20.
62. Sheehan, K.M. and Ginther, A. 2001: What do passage-based MC verbal reasoning items really measure? An analysis of the cognitive skills underlying performance on the current TOEFL reading section. Paper presented at the 2000 Annual Meeting of the National Council of Measurement in Education.
63. Sheehan, K.M., Ginther, A. and Schedl, M. 1999: Development of a proficiency scale for the TOEFL reading comprehension section. TOEFL Research Report. Princeton, NJ: Educational Testing Service.
64. Shmailo, L. 2002: Mastering the TOEFL CBT. New York: Kaplan.
65. Stanovich, K. 1980: Toward an interactive-compensatory model of individual differences in the development of reading fluency. Reading Research Quarterly 16, 32-71.
66. Stanovich, K. 1986: Matthew effects in reading: Some consequences of individual differences in the acquisition of literacy. Reading Research Quarterly 21, 360-406.
67. Sullivan, P.N., Brenner, G.A. and Zhong, G.L.Q. 2004: Master the TOEFL 2005. Lawrenceville, NJ: Thomson/Peterson.
68. Sullivan, P.N., Zhong, G.L.Q. and Brenner, G.A. 2000: Everything you need to score high on the TOEFL. New York: Pearson Education.
69. Underwood, G. and Batt, V. 1996: Reading and understanding: an introduction to the psychology of reading. Cambridge, MA: Blackwell.
70. Van Dijk, T. 1985: Strategic discourse comprehension. In Ballmer, T., editor, Linguistic dynamics: Discourses, procedures, and evolution. Berlin: Walter de Gruyter, 30-61.
71. Wenden, A. and Rubin, J. editors. 1987: Learner strategies in language learning. London: Prentice Hall International.