Uncovering Substantive Patterns in Student Responses in International Large-Scale Assessments-Comparing a Latent Class to a Manifest DIF Approach

International Journal of Testing

Volumn 14, Issue 3, 2014, Pages 265-287

  Oliveri, María Elena ^a   Ercikan, Kadriye ^b   Zumbo, Bruno D ^b   Lawless, René ^a  

^a EDUCATIONAL TESTING SERVICE   (United States)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

differential item functioning; latent class analysis; Progress for International Reading Literacy Study

Indexed keywords

EID: 84905407078     PISSN: 15305058     EISSN: 15327574     Source Type: Journal    
DOI: 10.1080/15305058.2014.891223     Document Type: Article

References (39)

1. Allalouf, A., Hambleton, R.K., & Sireci, S.G. (1999). Identifying the causes of DIF in translated verbal items. Journal of Educational Measurement, 36, 185-198.
2. Brown, T.A. (2006). Confirmatory factor analysis for applied research. New York, NY: Guildford.
3. Burket, G.R. (1998). PARDUX (Version 1.02) [Computer Program]. Monterey, CA: CTB/McGraw-Hill.
4. Cohen, A.S., & Bolt, D.M. (2005). A mixture model analysis of differential item functioning. Journal of Educational Measurement, 42, 133-148.
5. De Ayala, R.J., Kim, S.-H., Stapleton, L.M., & Dayton, C.M. (2002). Differential item functioning: A mixture distribution conceptualization. International Journal of Testing, 2, 243-276.
6. Dorans, N.J., & Middleton, K. (2012). Addressing the extreme assumptions of presumed linkings. Journal of Educational Measurement, 49, 1-18.
7. Ercikan, K., Gierl, M.J., McCreith, T., Puhan, G., & Koh, K. (2004). Comparability of bilingual versions of assessments: Sources of incomparability of English and French versions of Canada's national achievement tests. Applied Measurement in Education, 17, 301-321.
8. Ercikan, K., & Oliveri, M.E. (2013). Is fairness research doing justice? A modest proposal for an alternative validation approach in differential item functioning (DIF) investigations. In M. Chatterji (Ed.), Validity and test use: An international dialogue on educational assessment, accountability and equity (pp. 69-86). Bingley, UK: Emerald Publishing.
9. Hambleton, R.K., Swaminathan, H., & Rogers, H.J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
10. Holland, P.W., & Thayer, D.T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Erlbaum.
11. International Association for the Evaluation of Educational Achievement. (2008). Progress in International Reading Literacy Study 2006. Retrieved from http://pirls.bc.edu/pirls2006/index.html
12. Jodoin, M.G., & Gierl, M.J. (2001). Evaluating Type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14, 329-349.
13. Kennedy, A.M., Mullis, I.V. S., Martin, M.O., & Trong, K.L. (2007). PIRLS 2006 encyclopedia: A guide to reading education in the forty PIRLS 2006 countries. Chestnut Hill, MA: Boston College.
14. Kennedy, A.M., & Trong, K.L. (2007). Reporting student achievement in reading. In M.O. Martin, I.V. S. Mullis, & A.M. Kennedy (Eds.), PIRLS 2006 technical report: IEA's progress in international reading literacy study (pp. 173-193). Boston, MA: TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College.
15. Linn, R.L., & Harnisch, D.L. (1981). Interactions between item content and group membership on achievement test items. Journal of Educational Measurement, 18, 109-118.
16. Lubke, G.H., & Neale, M.C. (2008). Distinguishing between latent classes and continuous factors with categorical outcomes: Class invariance of parameters of factor mixture models. Multivariate Behavioral Research, 43, 592-620.
17. Malak, B., & Trong, K.L. (2007). Translation and translation verification of the PIRLS reading assessment and questionnaires. In M.O. Martin, I.V. S. Mullis, & A.M. Kennedy (Eds.), PIRLS 2006 technical report (pp. 49-60). Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College.
18. Martin, M.O., Mullis, I.V. S., & Kennedy, A.M. (2007). PIRLS 2006 technical report. Chestnut Hill, MA: Boston College.
19. Mislevy, R.J., & Verhelst, N. (1990). Modeling item responses when different subjects employ different solution strategies. Psychometrika, 55, 195-215.
20. Muthén, L.K. (2010, February 11). Re: Model fit index WRMR [Discussion group comment]. Retrieved from http://www.statmodel.com/discussion/messages/9/5096.html?1292455350
21. Muthén, L.K., & Muthén, B. O. (1998-2007). Mplus user's guide (5th ed.). Los Angeles, CA: Muthén & Muthén.
22. Oliveri, M.E., & Ercikan, K. (2011). Do different approaches to examining construct comparability lead to similar conclusions? Applied Measurement in Education, 24, 1-18.
23. Oliveri, M.E., Ercikan, K., & Zumbo, B.D. (2013). Analysis of sources of latent class DIF in International Assessments. International Journal of Testing, 13, 272-293.
24. Oliveri, M.E., Ercikan, K., & Zumbo, B.D. (in press). Effects of population heterogeneity on accuracy of DIF detection. Special issue on Levels of Analysis in the Assessment of Linguistic Minority Students. Applied Measurement in Education.
25. Rutkowski, L., Gonzalez, E., Joncas, M., & von Davier, M. (2010). International large-scale assessment data: Issues in secondary analysis and reporting. Educational Researcher, 39, 142151.
26. Samuelsen, K.M. (2005). Examining differential item functioning from a latent class perspective. Unpublished PhD dissertation, University of Maryland, College Park.
27. Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461-464.
28. Sireci, S.G., & Berberoglu, G. (2000). Using bilingual respondents to evaluate translated-adapted items. Applied Measurement in Education, 35, 229-259.
29. Steiger, J.H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173-180.
30. Swaminathan, H., & Rogers, H.J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370.
31. Tucker, L.R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1-10.
32. von Davier, M. (2005). mdltm: Software for the general diagnostic model and for estimating mixtures of multidimensional discrete latent traits models [Computer software]. Princeton, NJ: Educational Testing Service.
33. von Davier, M. (2008). The mixture general diagnostic model. In G.R. Hancock & K.M. Samuelsen (Eds.), Advances in latent variable mixture models (pp. 1-24). Charlotte, NC: Information Age Publishing.
34. von Davier, M., & von Davier, A. (2007). A unified approach to IRT scale linkage and scale transformations. Methodology, 3, 115-124.
35. Yen, W.M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187-214.
36. Yu, C.Y. (2002). Evaluating cut-off criteria of model fit indices for latent variable models with binary and continuous outcomes. Unpublished PhD dissertation, University of California, Los Angeles.
37. Zhang, Y., Dorans, N.J., & Matthews-Lopez, J.L. (2005). Using DIF dissection method to assess effects of item deletion (College Board Rep. No. 2005-10). Princeton, NJ: Educational Testing Service.
38. Zumbo, B.D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense.
39. Zumbo, B.D. (2009). Validity as contextualized and pragmatic explanation, and its implications for validation practice. In R.W. Lissitz (Ed.), The concept of validity: Revisions, new directions and applications (pp. 65-82). Charlotte, NC: Information Age Publishing.