Manipulating processing difficulty of reading comprehension questions: The feasibility of verbal item generation

Journal of Educational Measurement

Volumn 42, Issue 4, 2005, Pages 351-373

  Gorin, Joanna S ^a,b  

^a ARIZONA STATE UNIVERSITY   (United States)

^b Department of Measurement Statistics and Methodological Studies   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 30344484274     PISSN: 00220655     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.1745-3984.2005.00020.x     Document Type: Article

References (52)

1. Alderson, J. C. (1990). Testing reading comprehension skills: Part 2. Getting students to talk about taking a reading test (A pilot study). Reading in a Foreign Language, 6, 425-438.
2. Anderson, R. C. (1982). How to construct achievement tests to assess comprehension. Review of Educational Research. 42, 145-170.
3. Arrer, S. (1992). Unterschiede zwishen Computer- und Papier-Bleistift- Vorgabc des 3DW. [Differences between computerized and paper-pencil forms of the 3DW. In German] Unpublished master's thesis. University of Vienna, Vienna.
4. Bejar, I. I. (1993). A generative approach to psychological and educational measurement. In N. Fredriksen, R. J. Mislevy, & I. I. Bejar (Eds.), Test theory for a new generation of tests (pp. 323-359). Hillsdale, NJ: Erlbaum.
5. Bennet, R. E. (1999). Using new technology to improve assessment. Educational Measurement: Issues and Practice, 18, 5-12.
6. Bennett, R. E., & Bejar, I. I. (1998).Validity and automated scoring: It's not only the scoring. Educational Measurement: Issues and Practice, 17(4), 9-16.
7. Bovair, S., & Kieras, D. E. (1981). A guide to propositional analysis for research on technical prose. In B. Britton & J. Black (Eds.), Understanding expository text (Technical Report No. 8). Tucson. AZ: University of Arizona Press.
8. Buck, G., Tatsuoka, K., & Kostin, I. (1997). The subskills of reading: Rule-space analysis of a multiple-choice test of second language reading comprehension. Language Learning, 47(3), 423-466.
9. Carpenter, P. A., & Just, M. A. (1975). Sentence comprehension: A psycholinguistic processing model of verification. Psychological Review, 82, 45-73.
10. Craik, F. I., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11(6), 671-684.
11. Dunlap, J. W., & Cureton, E. E. (1930). On the analysis of causation. Journal of Educational Psychology, 27, 657-680.
12. Ebel, R. L. (1953). The use of item response time measurements in the construction of educational achievement tests. Educational and Psychological Measurement, 13, 391-401.
13. Ebel, R. L. (1956). Obtaining and reporting evidence on content validity. Educational and Psychological Measurement, 16, 269-282.
14. Embretson, S. E. (1998). A cognitive design system approach to generating valid tests: Application to abstract reasoning. Psychological Methods, 3, 300-396.
15. Embretson, S. E. (1999). Generating items during testing: Psychometric issues and models. Psychometrika, 64(4), 407-433.
16. Embretson, S. E. (2001). Generating assembling objects from cognitive specifications (Report No. SubPR98-11). Washington, DC: Human Resources Organization.
17. Embretson, S. E., & Gorin, J. S. (2001). Improving construct validity with cognitive psychology principles. Journal of Educational Measurement, 38(4), 343-368.
18. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.
19. Embretson, S. E., & Wetzel, C. D. (1987). Component latent trait models for paragraph comprehension. Applied Psychological Measurement, 11(2), 175-193.
20. Fischer, G. H. (1973). Linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359-374.
21. Fischer, G. H. (1996). Unidimensional linear logistic Rasch models. In W. J. van der Linden & R. K. Hambleton (Eds.) Handbook of modern item response theory. New York: Springer.
22. Freedle, R., & Kostin, I. (1991). The prediction of SAT reading comprehension item difficulty for expository prose passages (ETS Research Report # RR-91-29). Princeton, NJ: Educational Testing Service.
23. Freedle, R., & Kostin, I. (1992). The prediction of GRE reading comprehension item difficulty for expository prose passages for each of three item types: Main Ideas, inferences, and explicit statements (ETS Research Report RR-91-59). Princeton, NJ: Educational Testing Service.
24. Gorin, J. S., & Embretson, S. E. (2005). Cognitive and psychometric modeling of text-based multiple choice reading comprehension items on the GRE-V. Manuscript submitted for publication.
25. Green, K. E., & Smith, R. M. (1987). A comparison of two methods of decomposing item difficulties. Journal of Educational Statistics, 12, 369-381.
26. Huggins, A., & Adams, M. (1980). Syntactic aspects of syntactic comprehension. In R. J. Spiro, B. C. Bruce, & W. F. Brewer (Eds.), Theoretical issues in reading comprehension. Hillsdale, NJ: Erlbaum.
27. Irvine, S. H. (2002). Item generation for test development: An introduction. In S. H. Irvine & P. C. Kyllonen (Eds.), Item generation for test development. Mahwah, NJ: Erlbaum.
28. Jackson, N. E. (2005). Are university students' component reading skills related to their text comprehension and academic achievement? Learning and Individual Differences, 15, 113-139.
29. Just, M. A., & Carpenter, P. A. (1971). Comprehension of negation with quantification. Journal of Verbal Learning and Verbal Behavior, 10, 244-253.
30. Just, M. A., & Clark, H. H. (1973). Drawing inferences from the presuppositions and implications of affirmative and negative sentences. Journal of Verbal Learning and Verbal Behavior, 12, 21-31.
31. Kieras, D. E. (1981). Component processes in the comprehension of simple prose. Journal of Verbal Learning and Verbal Behavior, 20, 1-23.
32. Kintsch, W. (1988). The role of knowledge in discourse comprehension: A construction-integration model. Psychological Review. 95(2), 163-182.
33. Kintsch, W. (1994). Text comprehension, memory, and learning. American Psychologist 49(4), 294-303.
34. Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York: Cambridge University Press.
35. Kintsch, W., & Keenan, J. (1973). Reading rate and retention as a function of the number of propositions in the base structure of sentences. Cognitive Psychology, 5, 257-274.
36. Kintsch, W., & van Dijk, A. (1978). Toward a model of text comprehension and production. Psychological Review, 85, 363-394.
37. Landauer, T. K. (1998). Learning and representing verbal meaning: The latent semantic analysis theory. Current Directions in Psychological Science, 7(5), 161-164.
38. Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2), 211-240.
39. Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25, 359-384.
40. Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3, 635-694.
41. Meijer, R. R. (2003). Diagnosing item score patterns on a test using item response theory-based person-fit statistics. Psychological Methods, 8, 72-87.
42. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.
43. Mislevy, R. J., & Bock, R. D. (1990). BILOG-3: Item analysis and test scoring with binary logistic models (Computer software). Mooresville, IN: Scientific Software.
44. Mitchell, K. (1983). Cognitive processing determinants of item difficulty on the verbal subtexts of the Armed Services Vocational Aptitude Batteries and their relationships to success in Army Training. Unpublished doctoral dissertation, Cornell University.
45. Muraki, E., & Bock, R. D. (1993). PARSCALE: IRT-based test scoring and item analysis for graded open-ended exercises and performance tasks. Mooresville, IN: Scientific Software.
46. Pononcy, I. (2002). On the applicability of some IRT models for repeated measurement designs: Conditions, consequences, and goodness-of-fit tests. Methods of Psychological Research (Electronic Version), 21-40.
47. Rasch, G. (1966). An item analysis which takes individual differences into account. British Journal of Mathematical and Statistical Psychology, 4, 49-57.
48. Rost, J. (1990). Rasch model in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 14, 211 -282.
49. Sheehan, K. M., & Ginther, A. (2001). What do passage-based multiple-choice verbal reasoning items really measure? An analysis of the cognitive skills underlying performance on the current TOEFL reading section. Paper presented at the 2000 Annual Meeting of the National Council of Measurement in Education, New Orleans, LA.
50. Stone, C. A., & Zhang, B. (2003). Assessing goodness of fit of item response theory models: A comparison of traditional and alternative procedures. Journal of Educational Measurement, 40, 331-352.
51. Whitely, S. E., & Schneider, L. M. (1981). Information structure for geometric analogies: A test theory approach. Applied Psychological Measurement, 5, 383-397.
52. Whittaker, T. A., Fitzpatrick, S. J., Williams, N. J., & Dodd, B. G. (2003). IRTGEN: An SAS macro program to generate known trait scores and item responses for commonly used item response theory models. Applied Psychological Measurement, 27, 299-300.