Analysis of Sources of Latent Class Differential Item Functioning in International Assessments

International Journal of Testing

Volumn 13, Issue 3, 2013, Pages 272-293

  Oliveri, Maria Elena ^a   Ercikan, Kadriye ^b   Zumbo, Bruno ^b  

^a EDUCATIONAL TESTING SERVICE   (United States)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

international assessments; latent class; PIRLS; sources of DIF

Indexed keywords

EID: 84879043706     PISSN: 15305058     EISSN: 15327574     Source Type: Journal    
DOI: 10.1080/15305058.2012.738266     Document Type: Article

References (57)

1. American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing. 1999. Standards for educational and psychological testing, Washington, DC: American Educational Research Association.
2. Angoff, W. H. 1993. " Perspective on differential item functioning methodology ". In Differential item functioning, Edited by: Holland, P. W. and Wainer, H. 3 - 24. Hillsdale, NJ: Erlbaum.
3. Bolt, D. M., Cohen, A. S. and Wollack, J. A. 2001. A mixture model for multiple choice data. Journal of Educational and Behavioral Statistics, 26: 381 - 409.
4. Brown, T. A. 2006. Confirmatory factor analysis for applied research, New York, NY: Guilford.
5. Cohen, A. S. and Bolt, D. M. 2005. A mixture model analysis of differential item functioning. Journal of Educational Measurement, 42: 133 - 148.
6. De Ayala, R. J., Kim, S.-H., Stapleton, L. M. and Dayton, C. M. 2002. Differential item functioning: A mixture distribution conceptualization. International Journal of Testing, 2: 243 - 276.
7. Enders, C. K. and Tofighi, D. 2007. Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12: 121 - 138.
8. Ercikan, K. 2008. " Limitations in sample to population generalizing ". In Generalizing in educational research, Edited by: Ercikan, K. and Roth, M.-W. 211 - 235. New York, NY: Routledge.
9. Ercikan, K., Arim, R. G., Law, D. M., Lacroix, S., Gagnon, F. and Domene, J. F. 2010. Application of think-aloud protocols in examining sources of differential item functioning. Educational Measurement: Issues and Practice, 29 (2): 24 - 35.
10. Ercikan, K., Gierl, M. J., McCreith, T., Puhan, G. and Koh, K. 2004. Comparability of bilingual versions of assessments: Sources of incomparability of English and French versions of Canada's national achievement tests. Applied Measurement in Education, 17: 301 - 321.
11. Ercikan, K. and Lyons-Thomas, J. 2013. " Adapting tests for use in other languages and cultures ". In APA handbook of testing and assessment in psychology, Edited by: Geisinger, K. Washington, DC: American Psychological Association. vol. 3, pp. 545-569
12. Foy, P. and Kennedy, A. M. 2008. PIRLS 2006 user guide for the international database, Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
13. Gierl, M. J. and Khaliq, S. N. 2001. Identifying sources of differential item and bundle functioning on translated achievement tests: A confirmatory analysis. Journal of Educational Measurement, 38: 164 - 187.
14. Grisay, A., Gonzalez, E. and Monseur, C. 2009. " Equivalence of item difficulties across national versions of the PIRLS and PISA reading assessments ". In IERI monograph series: Issues and methodologies in large scale assessments, Vol. 2, Edited by: von Davier, M. and Hastedt, D. 63 - 83. Princeton, NJ: IEA-ETS Research Institute.
15. Grisay, A. and Monseur, C. 2007. Measuring the equivalence of item difficulty in the various versions of an international test. Studies in Educational Evaluation, 33: 69 - 86.
16. Haertel, E. H. 1989. Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26: 301 - 321.
17. Hambleton, R. K., Swaminathan, H. and Rogers, H. J. 1991. Fundamentals of item response theory, Newberry Park, CA: Sage.
18. Hedeker, D. 2003. A mixed-effects multinomial logistic regression model. Statistics in Medicine, 22: 1433 - 1446.
19. Holland, P. W. and Thayer, D. T. 1988. " Differential item performance and the Mantel-Haenszel procedure ". In Test validity, Edited by: Wainer, H. and Braun, H. 129 - 145. Hillsdale, NJ: Erlbaum.
20. International Association for the Evaluation of Educational Achievement. 2008. Progress in International Reading Literacy Study 2006 Retrieved from http://pirls.bc.edu/pirls2006/index.html
21. Kennedy, A. M., Mullis, I. V. S., Martin, M. O. and Trong, K. L. 2007. PIRLS 2006 encyclopedia: A guide to reading education in the forty PIRLS 2006 countries, Chestnut Hill, MA: Boston College.
22. Lesaux, N. K. and Siegel, L. S. 2003. The development of reading in children who speak English as a second language. Developmental Psychology, 25: 1005 - 1019.
23. Lubke, G. H. and Neale, M. C. 2008. Distinguishing between latent classes and continuous factors with categorical outcomes: Class invariance of parameters of factor mixture models. Multivariate Behavioral Research, 43: 592 - 620.
24. Lubke, G. H. and Spies, J. 2008. " Choosing a "correct" factor mixture model: Power, limitations, and graphical data exploration ". In Advances in latent variable mixture model, Edited by: Hancock, G. R. and Samuelsen, K. M. 343 - 362. Charlotte, NC: Information Age Publishing.
25. Maas, C. J. and Hox, J. J. 2005. Sufficient sample sizes for multilevel modeling. Methodology, 1: 86 - 92.
26. Martin, M. O., Mullis, I. V. S. and Kennedy, A. M. 2007. PIRLS 2006 technical report, Chestnut Hill, MA: Boston College.
27. Mislevy, R. J. and Verhelst, N. 1990. Modeling item responses when different subjects employ different solution strategies. Psychometrika, 55: 195 - 215.
28. Mislevy, R., Wilson, M., Ercikan, K. and Chudowsky, N. 2002. " Psychometric principles in student evaluation ". In International handbook of educational evaluation, Edited by: Nevo, D. and Stufflebeam, D. 478 - 520. Dordrecht, The Netherlands: Kluwer.
29. Mullis, I. V. S., Martin, M. O. and Gonzalez, E. 2004. International achievement in the processes of reading comprehension: Results from PIRLS 2001 in 35 countries, Chestnut Hill, MA: Boston College.
30. Mullis, I. V. S., Martin, M. O., Kennedy, A. M. and Foy, P. 2007. PIRLS 2006 international report: IEA's Progress in International Reading Literacy Study in primary school in 40 countries, Chestnut Hill, MA: Boston College.
31. Muraki, E. 1992. A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16: 159 - 176.
32. Muthén, B. and Satorra, A. 1995. Technical aspects of Muthén's LISCOMP approach to estimation of latent variable relations with a comprehensive measurement model. Psychometrika, 60: 489 - 503.
33. Muthén, L. K. 2010. Model fit index WRMR. February 11 Retrieved from http://www.statmodel.com/discussion/messages/9/5096.html?1292455350
34. Muthén, L. K. and Muthén, B. O. 1998-2007. Mplus User's Guide (5th ed.), Los Angeles, CA: Authors.
35. Oliveri, M. E. 2012. Investigation of within group heterogeneity in measurement comparability research Vancouver, BC, Canada Unpublished PhD dissertation, University of British Columbia
36. Oliveri, M. E. and Ercikan, K. 2011. Do different approaches to examining construct comparability lead to similar conclusions?. Applied Measurement in Education, 24: 1 - 18.
37. Oliveri, M. E. and von Davier, M. 2011. Investigation of model fit and score scale comparability in international assessments. Journal of Psychological Test and Assessment Modeling, 53: 315 - 333.
38. Oliveri, M. E. and von Davier, M. Application of hierarchical mixture item response theory models to international large-scale assessment data. Paper presented at the meeting of the International Test Commission Conference, Amsterdam, The Netherlands, July
39. Peugh, J. L. 2010. A practical guide to multilevel modeling. Journal of School Psychology, 48: 85 - 112.
40. Rost, J. 1990. Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 14: 271 - 282.
41. Rutkowski, L., Gonzalez, E., Joncas, M. and von Davier, M. 2010. International large-scale assessment data: Issues in secondary analysis and reporting. Educational Researcher, 39: 142 - 151.
42. Samuelsen, K. M. 2005. Examining differential item functioning from a latent class perspective College Park Unpublished PhD dissertation, University of Maryland
43. Schwarz, G. 1978. Estimating the dimension of a model. The Annals of Statistics, 6: 461 - 464.
44. Shealy, R. and Stout, W. 1993. A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58: 159 - 194.
45. Siegel, L. S. 1993. Phonological processing deficits as the basis of a reading disability. Developmental Review, 13: 246 - 257.
46. Slavin, R. E., Cheung, A., Groff, C. and Lake, C. 2008. Effective reading programs for middle and high schools: A best-evidence synthesis. Reading Research Quarterly, 43: 290 - 322.
47. Stanovich, K. E. and Siegel, L. S. 1994. The phenotypic performance profile of reading-disabled children: A regression-based test of the phonological-core variable-difference model. Journal of Educational Psychology, 86: 24 - 53.
48. Steiger, J. H. 1990. Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25: 173 - 180.
49. Tabachnick, B. G. and Fidell, L. S. 2007. Using multivariate statistics (5th ed.), Boston, MA: Allyn and Bacon.
50. Tucker, L. R. and Lewis, C. 1973. A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38: 1 - 10.
51. von Davier, M. 2005. Mdltm: Software for the general diagnostic model and for estimating mixtures of multidimensional discrete latent traits models [Computer software], Princeton, NJ: Educational Testing Service.
52. von Davier, M. 2008. " The mixture general diagnostic model ". In Advances in latent variable mixture models, Edited by: Hancock, G. R. and Samuelsen, K. M. 1 - 24. Charlotte, NC: Information Age Publishing.
53. von Davier, M. 2010. Hierarchical mixtures of diagnostic models. Psychological Test and Assessment Modeling, 52: 8 - 28.
54. von Davier, M. and von Davier, A. 2007. A unified approach to IRT scale linkage and scale transformations. Methodology, 3: 115 - 124.
55. von Davier, M., Xu, X. and Carstensen, C. H. 2011. Measuring growth in a longitudinal large scale assessment with a general latent variable model. Psychometrika, 76: 318 - 336.
56. Webb, M. L., Cohen, A. S. and Schwanenflugel, P. J. 2008. Latent class analysis of differential item functioning on the Peabody Picture Vocabulary Test-III. Educational and Psychological Measurement, 68: 335 - 351.
57. Yu, C. Y. 2002. Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes, Los Angeles: University of California. Unpublished PhD dissertation