Explaining and controlling for the psychometric properties of computer-generated figural matrix items

Applied Psychological Measurement

Volumn 32, Issue 3, 2008, Pages 195-210

  Freund, Philipp Alexander ^a   Hofer, Stefan ^b   Holling, Heinz ^a  

^a UNIVERSITY OF MÜNSTER   (Germany)

^b Bundesagentur fur Arbeit   (Germany)

Author keywords

Automatic item generation; Figural matrix items; Item task parameters; LLTM; Rasch model

Indexed keywords

EID: 41549114185     PISSN: 01466216     EISSN: 15523497     Source Type: Journal    
DOI: 10.1177/0146621607306972     Document Type: Article

References (40)

1. Andersen, E.B. (1973). A goodness of fit test for the Rasch model. Psychometrika, 38, 123-140.
2. Arendasy, M. (2002). GeomGen-Ein Itemgenerator für Matrizentestaufgaben [GeomGen-An item generator for matrices]. Wien: Eigenverlag.
3. Arendasy, M., & Gittler, G. (2003). IRT-basierter Vergleich zweier Varianten automatisiert erstellter Matrizentestaufgaben [IRT-based comparison of automatically generated figural matrix items]. Zeitschrift für Differentielle und Diagnostische Psychologie, 24, 261-275.
4. Arendasy, M., & Sommer, M. (2005). The effect of different types of perceptual manipulations on the dimensionality of automatically generated figural matrices. Intelligence, 33, 307-324.
5. Arendasy, M., & Sommer, M. (in press). Using psychometric technology in educational assessment: The case of a schema-based isomorphic approach to the automatic generation of quantitative reasoning items. Learning and Individual Differences.
6. Bejar, I.I. (2002). Generative testing: From conception to implementation. In S. H. Irvine & P. C. Kyllonen (Eds.), Item generation for test development (pp. 199-217). Mahwah, NJ: Lawrence Erlbaum.
7. Bors, D.A., & Stokes, T.L. (1998). Raven's Advanced Progressive Matrices: Norms for firstyear university students and the development of a short form. Educational and Psychological Measurement, 58, 382-398.
8. Carpenter, P.A., Just, M.A., & Shell, P. (1990). What one intelligence test measures: A theoretical account of the processing in the Raven Progressive Matrices test. Psychological Review, 97, 404-431.
9. Dennis, I., Handley, S., Bradon, P., Evans, J., & Newstead, S. (2002). Approaches to modeling item-generative tests. In S. H. Irvine & P. C. Kyllonen (Eds.), Item generation for test development (pp. 53-71). Mahwah, NJ: Lawrence Erlbaum.
10. Dillon, R.F., Pohlmann, J.T., & Lohman, D.F. (1981). A factor analysis of Raven's Advanced Progressive Matrices freed of difficulty factors. Educational and Psychological Measurement, 41, 1295-1302.
11. Embretson, S.E. (1998). A cognitive design system approach to generating valid tests: Application to abstract reasoning. Psychological Methods, 3, 380-396.
12. Embretson, S.E. (1999). Generating items during testing: Psychometric issues and models. Psychometrika, 64, 407-433.
13. Embretson, S.E. (2002). Generating abstract reasoning items with cognitive theory. In S. H. Irvine & P. C. Kyllonen (Eds.), Item generation for test development (pp. 219-250). Mahwah, NJ: Lawrence Erlbaum.
14. Enright, M.K., & Sheehan, K.M. (2002). Modeling the difficulty of quantitative reasoning items: Implications for item generation. In S. H. Irvine & P. C. Kyllonen (Eds.), Item generation for test development (pp. 129-157). Mahwah, NJ: Lawrence Erlbaum.
15. Fischer, G.H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359-374.
16. Fischer, G.H. (1995 a). Derivations of the Rasch model. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications (pp. 15-38). New York: Springer.
17. Fischer, G.H. (1995 b). The linear logistic test model. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications (pp. 157-180). New York: Springer.
18. Fischer, G.H., & Ponocny-Seliger, E. (1998). LPCM-Win 1.0. Structural Rasch modeling [Computer software]. Groningen, Netherlands: ProGamma.
19. Formann, W., & Piswanger, J. (1979). Wiener Matrizentest (WMT) [Wiener Matrices Test]. Weinheim, Germany: Beltz.
20. Gorin, J.S. (2005). Manipulating processing difficulty of reading comprehension questions: The feasibility of verbal item generation. Journal of Educational Measurement, 42, 351-373.
21. Gorin, J.S., & Embretson, S.E. (2006). Item difficulty modeling of paragraph comprehension items. Applied Psychological Measurement, 30, 394-411.
22. Green, K.E., & Kluever, R.C. (2001). Components of item difficulty of Raven's matrices. Journal of General Psychology, 119, 189-199.
23. Hofer, S. (2004). MatrixDeveloper [Unpublished computer software]. Münster, Germany: Psychological Institute IV, Westfälische Wilhelms-Universität.
24. Hornke, L.F., & Habon, M.W. (1986). Rule-based item bank construction and evaluation within the linear logistic framework. Applied Psychological Measurement, 10, 369-380.
25. Irvine, S.H., & Kyllonen, P.C. (2002). Item generation for test development. Mahwah, NJ: Lawrence Erlbaum.
26. Kyllonen, P.C. (2002). Item generation for repeated testing of human performance. In S. H. Irvine & P. C. Kyllonen (Eds.), Item generation for test development (pp. 251-275). Mahwah, NJ: Lawrence Erlbaum.
27. Kyllonen, P.C., & Christal, R.E. (1990). Reasoning ability is (little more than) working memory capacity?! Intelligence, 14, 389-433.
28. Martin-Lf, P. (1973). Statistika modeler: Anteckningar fran seminarier lasret 1969-1970, utarbetade av Rolf Sundberg. Obetydligt ändrat nytryk, Oktober 1973 [Statistical models: Notes from seminars 1969-1970, prepared by Rolf Sundberg]. Stockholm, Sweden: Institutet fr Frsäkringsmatematik och Matematisk Statistisk vid Stockhokms Universitet.
29. Meo, M., Roberts, M.J., & Marucci, F.S. (2007). Element salience as a predictor of item difficulty for Raven's Progressive Matrices. Intelligence, 35, 359-368.
30. Müller, A. (2001). Variation Einfachbezug versus Mehrfachbezug und Training versus kein Training bei regelgeleitet konstruierten Matrizenaufgaben auf die Ergebnisleistung [Variation of single versus multiple rules per element and training versus no training in rule-based generated matrix items and performance]. Unpublished master's thesis, University of Münster, Germany.
31. Nevo, B. (1976). The effects of general practice, specific practice, and item familiarization on change in aptitude test scores. Measurement and Evaluation in Guidance, 9, 16-20.
32. Preckel, F. (2003). Diagnostik intellektueller Hochbegabung. Testentwicklung zur Erfassung der fluiden Intelligenz [Assessment of intellectual giftedness: Test development for the assessment of fluid intelligence]. Gttingen, Germany: Hogrefe.
33. Primi, R. (2001). Complexity of geometric inductive reasoning tasks: Contribution to the understanding of fluid intelligence. Intelligence, 30, 41-70.
34. Raven, J.C. (1962). Advanced Progressive Matrices. London: H. K. Lewis.
35. Raven, J.C. (1965). Guide to using the Coloured Progressive Matrices Sets A, Ab, B. Dumfries, Scotland: Grieve.
36. Raven, J.C. (1985). Standard Progressive Matrices Sets A, B, C, D, and E. London: H. K. Lewis.
37. Rost, J., & von Davier, M. (1994). A conditional item fit index for Rasch models. Applied Psychological Measurement, 18, 171-182.
38. van der Linden, W.J. (2005). Linear models for optimal test design. New York: Springer.
39. van der Ven, A.H.G.S., & Ellis, J.L. (2000). A Rasch analysis of Raven's Standard Progressive Matrices. Personality and Individual Differences, 29, 45-64.
40. Vigneau, F., & Bors, D.A. (2005). Items in context: Assessing the dimensionality of Raven's Advanced Progressive Matrices. Educational and Psychological Measurement, 65, 109-123.