Do Adaptive Representations of the Item-Position Effect in APM Improve Model Fit? A Simulation Study

Educational and Psychological Measurement

Volumn 77, Issue 5, 2017, Pages 743-765

  Zeller, Florian ^a   Krampen, Dorothea ^a   Reiß, Siegbert ^a   Schweizer, Karl ^a  

^a GOETHE UNIVERSITY   (Germany)

Author keywords

APM; confirmatory factor analysis; item position effect; simulation

Indexed keywords

EID: 85029532270     PISSN: 00131644     EISSN: 15523888     Source Type: Journal    
DOI: 10.1177/0013164416654946     Document Type: Article

References (37)

1. Bandalos D. L., Gagné P., (2012). Simulation methods in structural equation modeling. In Hoyle R. H., (Ed.), Handbook of structural equation modeling (pp. 92-108). New York, NY: Guilford.
2. Bock R. D., Gibbons R., Muraki E., (1988). Full-information item factor analysis. Applied Psychological Measurement, 12, 261-280. doi:10.1177/014662168801200305
3. Carpenter P. A., Just M. A., Shell P., (1990). What one intelligence test measures: A theoretical account of the processing in the Raven Progressive Matrices Test. Psychological Review, 97, 404-431. doi:10.1037/0033-295X.97.3.404
4. Cheung G. W., Rensvold R. B., (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9, 233-255. doi:10.1207/S15328007SEM0902_5
5. DiStefano C., Morgan G. B., (2014). A comparison of diagonal weighted least squares robust estimation techniques for ordinal data. Structural Equation Modeling: A Multidisciplinary Journal, 21, 425-438. doi:10.1080/10705511.2014.915373
6. Embretson S. E., (1991). A multidimensional latent trait model for measuring learning and change. Psychometrika, 56, 495-515. doi:10.1007/BF02294487
7. Finney S. J., DiStefano C., (2013). Nonnormal and categorical data in structural equation modeling. In Hancock G. R., Mueller R. O., (Eds.), Structural equation modeling: A second course (2nd ed., pp. 439-492). Charlotte, NC: Information Age.
8. Graham J. M., (2006). Congeneric and (essentially) tau-equivalent estimates of score reliability: What they are and how to use them. Educational and Psychological Measurement, 66, 930-944. doi:10.1177/0013164406288165
9. Hamilton J. C., Shuminsky T. R., (1990). Self-awareness mediates the relationship between serial position and item reliability. Journal of Personality and Social Psychology, 59, 1301-1307. doi:10.1037/0022-3514.59.6.1301
10. Hartig J., Buchholz J., (2012). A multilevel item response model for item position effects and individual persistence. Psychological Test and Assessment Modelling, 54, 418-431.
11. Hartig J., Hölzel B., Moosbrugger H., (2007). A confirmatory analysis of item reliability trends (CAIRT): Differentiating true score and error variance in the analysis of item context effects. Multivariate Behavioral Research, 42, 157-183.
12. Hohensinn C., Kubinger K. D., Reif M., Holocher-Ertl S., Khorramdel L., Frebort M., (2008). Examining item-position effects in large-scale assessment using the linear logistic test model. Psychology Science Quarterly, 50, 391-402.
13. Hu L., Bentler P. M., (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55. doi:10.1080/10705519909540118
14. Jöreskog K. G., Sörbom D., (2001). Interactive LISREL: User guide. Lincolnwood, IL: Scientific Software International.
15. Jöreskog K. G., Sörbom D., (2004). LISREL 8.70. Lincolnwood, IL: Scientific Software International.
16. Kline R. B., (2011). Principles and practice of structural equation modeling (3rd ed.). New York, England: Guilford.
17. Knowles E. S., (1988). Item context effects on personality scales: Measuring changes the measure. Journal of Personality and Social Psychology, 55, 312-320. doi:10.1037/0022-3514.55.2.312
18. Knowles E. S., Byers B., (1996). Reliability shifts in measurement reactivity: Driven by content engagement or self-engagement?Journal of Personality and Social Psychology, 70, 1080-1090. doi:10.1037//0022-3514.70.5.1080
19. Kubinger K. D., (2008). On the revival of the Rasch model-based LLTM: From constructing tests using item generating rules to measuring item administration effects. Psychology Science Quarterly, 50, 311-327.
20. Li C.-H., (2015). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods. Advance online publication. doi:10.3758/s13428-015-0619-7
21. Lord F. M., Novick M. R., (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
22. Lozano J. H., (2015). Are impulsivity and intelligence truly related constructs? Evidence based on the fixed-links model. Personality and Individual Differences, 85, 192-198. doi:10.1016/j.paid.2015.04.049
23. McCullagh P. J., Nelder J. A., (1984). Generalized linear Models (Monographs on Statistics and Applied Probability; Vol. 37). London, England: Chapman & Hall.
24. McDonald R. P., Ahlawat K. S., (1974). Difficulty factors in binary data. British Journal of Mathematical and Statistical Psychology, 27, 82-99. doi:10.1111/j.2044-8317.1974. tb00530.x
25. Mollenkopf W. G., (1950). An experimental study of the effects on item-analysis data of changing item placement and test time limit. Psychometrika, 15, 291-315. doi:10.1007/BF02289044
26. Muthén B., (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49, 115-132. doi:10.1007/BF02294210
27. Rasch G., (1980). Probabilistic models for some intelligence and attainment tests (Expended ed.). Chicago, IL: University of Chicago Press.
28. Raven J. C., Raven J., Court J. H., (1997). Raven’s Progressive Matrices and Vocabulary Scales. Edinburgh, England: J. C. Raven.
29. Ren X., Goldhammer F., Moosbrugger H., Schweizer K., (2012). How does attention relate to the ability-specific and position-specific components of reasoning measured by APM?Learning and Individual Differences, 22(1), 1-7. doi:10.1016/j.lindif.2011.09.009
30. Ren X., Wang T., Altmeyer M., Schweizer K., (2014). A learning-based account of fluid intelligence from the perspective of the position effect. Learning and Individual Differences, 31, 30-35. doi:10.1016/j.lindif.2014.01.002
31. Schweizer K., (2011). Scaling variances of latent variables by standardizing loadings: Applications to working memory and the position effect. Multivariate Behavioral Research, 46, 938-955. doi:10.1080/00273171.2011.625312
32. Schweizer K., (2012). The position effect in reasoning items considered from the CFA perspective. International Journal of Educational and Psychological Assessment, 11(2), 44-58.
33. Schweizer K., (2013). A threshold-free approach to the study of the structure of binary data. International Journal of Statistics and Probability, 2(2). doi:10.5539/ijsp.v2n2p67
34. Schweizer K., Ren X., Wang T., (2015). A comparison of confirmatory factor analysis of binary data on the basis of tetrachoric correlations and of probability-based covariances: A simulation study. In Millsap R. E., Bolt D. M., van der Ark L. A., Wang W.-C., (Eds.), Springer proceedings in mathematics & statistics. Quantitative psychology research (Vol. 89, pp. 273-292). Berlin, Germany: Springer.
35. Schweizer K., Schreiner M., Gold A., (2009). The confirmatory investigation of APM items with loadings as a function of the position and easiness of items: A two-dimensional model of APM. Psychology Science Quarterly, 51(1), 47-64.
36. Schweizer K., Troche S. J., Rammsayer T. H., (2011). On the special relationship between fluid and general intelligence: New evidence obtained by considering the position effect. Personality and Individual Differences, 50, 1249-1254. doi:10.1016/j.paid.2011.02.019
37. Verguts T., De Boeck P., (2000). A Rasch model for detecting learning while solving an intelligence test. Applied Psychological Measurement, 24, 151-162. doi:10.1177/014662 10022031589