A Comparison of Different Psychometric Approaches to Modeling Testlet Structures: An Example with C-Tests

Journal of Educational Measurement

Volumn 51, Issue 4, 2014, Pages 400-418

  Schroeders, Ulrich ^a   Robitzsch, Alexander ^b   Schipolowski, Stefan ^c  

^a UNIVERSITY OF BAMBERG   (Germany)

^b Federal Institute for Education Research Innovation and Development of the Austrian Schooling System (BIFIE Salzburg)   (Australia)

^c HUMBOLDT UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84915789335     PISSN: 00220655     EISSN: 17453984     Source Type: Journal    
DOI: 10.1111/jedm.12054     Document Type: Article

References (47)

1. Ackerman, P. L., Beier, M. E., & Bowen, K. R. (2000). Explorations of crystallized intelligence: Completion tests, cloze tests, and knowledge. Learning and Individual Differences, 12, 105-121.
2. Alderson, J. C. (1979). The cloze procedure and proficiency in English as a foreign language. TESOL Quarterly, 13, 219-227.
3. Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573.
4. Bollen, K. A., Kirby, J. B., Curran, P. J., Paxton, P., & Chen, F. (2007). Latent variable models under misspecification. Two-stage least squares (2SLS) and maximum likelihood (ML) estimators. Sociological Methods and Research, 36, 48-86.
5. Bradlow, E. T., Wainer, H., & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64, 153-168.
6. Braeken, J. (2011). A boundary mixture approach to violations of conditional independence. Psychometrika, 76, 57-76.
7. Braeken, J., Tuerlinckx, F., & Boeck, P. (2007). Copula functions for residual dependency. Psychometrika, 72, 393-411.
8. DeMars, C. E. (2006). Application of the bi-factor multidimensional item response theory model to testlet-based tests. Journal of Educational Measurement, 43, 145-168.
9. DeMars, C. E. (2012). Confirming testlet effects. Applied Psychological Measurement, 36, 104-121.
10. Ebbinghaus, H. (1897). Über eine neue Methode zur Prüfung geistiger Fähigkeiten und ihre Anwendung bei Schulkindern [On a new method of measuring mental abilities and its implementation among school children]. Zeitschrift für Psychologie und Physiologie der Sinnesorgane, 13, 401-459.
11. Eckes, T. (2006). Rasch-Modelle zur C-Test-Skalierung [Rasch models to modeling C-tests]. In R. Grotjahn (Ed.), Der C-Test: Theorie, empirie, anwendungen (pp. 1-44). Frankfurt am Main, Germany: Peter Lang.
12. Eckes, T. (2007). Konstruktion und Analyse von C-Tests mit Ratingskalen-Rasch-Modellen [Construction and analysis of C-tests with rating-scale Rasch models]. Diagnostica, 53, 68-82.
13. Eckes, T. (2011). Item banking for C-tests: A polytomous Rasch modeling approach. Psychological Test and Assessment Modeling, 53, 414-439.
14. Eckes, T., & Grotjahn, R. (2006). C-Tests als Anker für TestDaF: Rasch-Analysen mit dem kontinuierlichen Ratingskalen-Modell [C-tests as an anchor in TestDaF: Rasch-analyses with the continous rating scale model]. In R. Grotjahn (Ed.), Der C-Test: Theorie, empirie, anwendungen (pp. 167-193). Frankfurt am Main, Germany: Peter Lang.
15. Gibbons, R. D., & Hedeker, D. R. (1992). Full-information item bi-factor analysis. Psychometrika, 57, 423-436.
16. Gonzalez, E., & Rutkowski, L. (2010). Principles of multiple matrix booklet designs and parameter recovery in large-scale assessments. In D. Hastedt & D. von Davier (Eds.), Issues and methodologies in large-scale assessments (Vol. 3, pp. 125-156). Hamburg, Germany: IEA-ETS Research Institute.
17. Harsch, C., & Hartig, J. (2010). Empirische und inhaltliche Analyse lokaler Abhängigkeiten im C-Test [Empirical analysis of local and contextual dependencies in the C-test]. In R. Grotjahn (Ed.), Der C-Test: Beiträge aus der aktuellen Forschung (pp. 193-204). Frankfurt am Main, Germany: Peter Lang.
18. Harsch, C., & Schröder, K. (2007). Textrekonstruktion: C-Test [Text reconstruction: C-test]. In B. Beck & E. Klieme (Eds.), Sprachliche Kompetenzen: Konzepte und Messung. DESI-Studie (Deutsch Englisch Schülerleistungen International) (pp. 238-253). Weinheim, Germany: Beltz.
19. Heagerty, P. J., & Zeger, S. L. (2000). Marginalized multilevel models and likelihood inference. Statistical Science, 15, 1-26.
20. Huynh, H., Michaels, H. R., & Ferrara, S. (1995, April). A comparison of three statistical procedures to identify clusters of items with local dependency. Paper presented at the annual meeting of National Council on Measurement in Education, San Francisco, CA.
21. Ip, E. H. (2010). Empirically indistinguishable multidimensional IRT and locally dependent unidimensional item response models. British Journal of Mathematical and Statistical Psychology, 63, 395-416.
22. Jakschik, G., & Klemmert, H. (2006). Erste Erprobung eines Multiple Choice C-Tests [Pilot testing of a multiple choice C-tests]. In R. Grotjahn (Ed.), Der C-Test: Theorie, Empirie, Anwendungen (pp. 195-210). Frankfurt am Main, Germany: Peter Lang.
23. Kiefer, T., Robitzsch, A., & Wu, M. (2014). TAM: Test analysis modules. R package version 1.0-2. [Computer software]. Retrieved May 21, 2014, from http://CRAN.R-project.org/package=TAM
24. Klein-Braley, C. (1985). A cloze-up on the C-Test: A study in the construct validation of authentic tests. Language Testing, 2, 76-104.
25. Klein-Braley, C. (1997). C-tests in the context of reduced redundancy testing: An appraisal. Language Testing, 14, 47-84.
26. Klein-Braley, C., & Raatz, U. (1984). A survey of research on the C-test. Language Testing, 1, 134-146.
27. Li, Y., Bolt, D. M., & Fu, J. (2006). A comparison of alternative models for testlets. Applied Psychological Measurement, 30, 3-21.
28. Linacre, J. M. (2002) Understanding Rasch measurement: Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3, 85-106.
29. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
30. Mislevy, R. J., Beaton, A. E., Kaplan, B., & Sheehan, K. M. (1992). Estimating population characteristics from sparse matrix samples of item responses. Journal of Educational Measurement, 29, 133-161.
31. Müller, H. (1987). A Rasch model for continuous ratings. Psychometrika, 52, 165-181.
32. Noel, Y., & Dauvier, B. (2007). A beta item response model for continuous bounded responses. Applied Psychological Measurement, 31, 47-73.
33. R Development Core Team. (2014). R: A language and environment for statistical computing (version 3.0.3) [Computer software]. Retrieved May 21, 2014, from http://www.R-project.org
34. Raatz, U., & Klein-Braley, C. (1981). The C-test-A modification of the cloze procedure. In T. Culhane, C. Klein-Braley, & D. K. Stevenson (Eds.), Practice and problems in language testing (pp. 113-145). Colchester, UK: University of Essex.
35. Rijmen, F. (2009). Three multidimensional models for testlet-based tests: Formal relations and an empirical comparison (No. ETS RR-09-37). Princeton, NJ: Educational Testing Service.
36. Robitzsch, A. (2014). sirt: Supplementary item response theory models. R package version 0.43-70 [Computer software]. Retrieved May 21, 2014, from http://CRAN.R-project.org/package=sirt
37. Robitzsch, A., & Karius, I. (2009). Predicting item difficulties and item dependencies for C-tests. Unpublished manuscript. Available at https://sites.google.com/site/alexanderrobitzsch/publikationen
38. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17). Richmond, VA: Psychometric Society. Retrieved September 29, 2014, from https://psychometricsociety.org/sites/default/files/pdf/MN17.pdf
39. Sireci, S. G., Thissen, D., & Wainer, H. (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28, 237-247.
40. Spolsky, B. (1971). Reduced redundancy as a language testing tool. In G. E. Perren & J. L. M. Trim (Eds.), Applications of linguistics (pp. 383-390), Cambridge, UK: Cambridge University Press.
41. Taylor, W. L. (1953). Cloze procedure: A new tool for measuring readability. Journalism Quarterly, 30, 415-433.
42. Taylor, W. L. (1957). Cloze readability scores as indices of individual differences in comprehension and aptitude. Journal of Applied Psychology, 41, 19-26.
43. Thissen, D., Steinberg, L., & Mooney, J. A. (1989). Trace lines for testlets: A use of multiple-categorical-response models. Journal of Educational Measurement, 26, 247-260.
44. Tuerlinckx, F., & De Boeck, P. (2004). Models for residual dependencies. In P. De Boeck & M. Wilson (Eds.), Explanatory item response models: A generalized linear and nonlinear approach (pp. 289-316). New York, NY: Springer.
45. Wang, W.-C., & Wilson, M. (2005). The Rasch testlet model. Applied Psychological Measurement, 29, 126-149.
46. Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125-145.
47. Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187-213.