Effects of Population Heterogeneity on Accuracy of DIF Detection

Applied Measurement in Education

Volumn 27, Issue 4, 2014, Pages 286-300

  Oliveri, María Elena ^a   Ercikan, Kadriye ^b   Zumbo, Bruno D ^b  

^a EDUCATIONAL TESTING SERVICE   (United States)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84907470591     PISSN: 08957347     EISSN: None     Source Type: Journal    
DOI: 10.1080/08957347.2014.944305     Document Type: Article

References (35)

1. Abedi, J. (2004). The No Child Left Behind Act and English language learners: Assessment and accountability issues. Educational Researcher, 33(1), 4–14.
2. De Ayala, R. J., Kim, S. H., Stapleton, L. M., & Dayton, C. M. (2002). Differential item functioning: A mixture distribution conceptualization. International Journal of Testing, 2, 243–276.
3. Dorans, N. J., & Holland, P. W. (1993). DIF detection and description: Mantel-Haenszel and standardization. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 35–66). Hillsdale, NJ: Erlbaum.
4. Ercikan, K. (2006). Developments in assessment of student learning and achievement. In P. A. Alexander & P. H. Winne (Eds.), American Psychological Association, Division 15, Handbook of educational psychology (2nd ed.) (pp. 929–953). New York, NY: Routledge.
5. Ercikan, K., Arim, R. G., Law, D. M., Lacroix, S., Gagnon, F., & Domene, J. F. (2010). Application of think-aloud protocols in examining sources of differential item functioning. Educational Measurement: Issues and Practice, 29(2), 24–35.
6. Ercikan, K., Gierl, M. J., McCreith, T., Puhan, G., & Koh, K. (2004). Comparability of bilingual versions of assessments: Sources of incomparability of English and French versions of Canada’s national achievement tests, Applied Measurement in Education, 17, 301–321.
7. Ercikan, K., & Lyons-Thomas, J. (2013). Adapting tests for use in other languages and cultures. In K. Geisinger (Ed.), APA handbook testing and assessment in psychology (Vol. 3; pp. 545–569). Washington, DC: American Psychological Association.
8. Ercikan, K., & Oliveri, M. E. (2013). Is fairness research doing justice? A modest proposal for an alternative validation approach in differential item functioning (DIF) investigations. In M. Chatterji (Ed.), Validity and test use: An international dialogue on educational assessment, accountability, and equity (pp. 69–86). Bingley, UK: Emerald Group Publishing.
9. Ercikan, K., Roth, W.-M., Simon, M., Sandilands, D., & Lyons-Thomas, J. (2014). Inconsistencies in DIF Detection for Sub-Groups in Heterogeneous Language Groups. Applied Measurement in Education, 27, 273–285.
10. French, B. F., & Maller, S. J. (2007). Iterative purification and effect size use with logistic regression for differential item functioning detection. Educational and Psychological Measurement, 67, 373–393.
11. Hambleton, R. K., & Rogers, H. J. (1989). Detecting potentially biased items: Comparison of IRT area and Mantel-Haenszel methods. Applied Measurement in Education, 2, 313–334.
12. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newberry Park, CA: Sage Publications.
13. Han, K. T. (2007). WinGen: Windows software that generates IRT parameters and item responses. Applied Psychological Measurement, 31, 457–459.
14. Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. Braun (Eds.), Test validity (pp. 129–145). Hillsdale, NJ: Erlbaum.
15. Jodoin, M. G., & Gierl, M. J. (2001). Evaluating Type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14, 329–349.
16. Ministry of Education. (2007). English language learners: ESL and ELD programs and services. Toronto, ON: Queen’s Printer for Toronto.
17. Narayanan, P., & Swaminathan, H. (1994). Performance of the Mantel-Haenszel and simultaneous item bias procedures for detecting differential item functioning. Applied Psychological Measurement, 18, 315–328.
18. Navas-Ara, M. J., & Gómez-Benito, J. (2002). Effects of ability scale purification on the identification of DIF. European Journal of Psychological Assessment, 18, 9–15.
19. Oliveri, M. E. (2012). Investigation of within group heterogeneity in measurement comparability research. Unpublished doctoral dissertation. University of British Columbia, Vancouver, BC, Canada.
20. Oliveri, M. E., & Ercikan, K. (2011). Do different approaches to examining construct comparability lead to similar conclusions? Applied Measurement in Education, 24, 1–18.
21. Oliveri, M. E., Ercikan, K., Lyons-Thomas, J., & Holtzman, S. (in press). Heterogeneity in linguistic groups and measurement comparability research. Applied Measurement in Education.
22. Oliveri, M. E., Ercikan, K., & Zumbo, B. D. (2013). Analysis of sources of latent class DIF in international assessments. International Journal of Testing, 13, 272–293.
23. Oliveri, M. E., Ercikan, K., Zumbo, B. D., & Lawless, R. R. (2014). Uncovering substantive patterns in student responses in international large-scale assessments – Comparing a latent class to a manifest DIF approach. International Journal of Testing, 14, 265–287.
24. Penfield, R. D. (2001). Assessing differential item functioning among multiple groups: A comparison of three Mantel-Haenszel procedures. Applied Measurement in Education, 14, 235–259.
25. Samuelsen, K. M. (2005). Examining differential item functioning from a latent class perspective. Unpublished doctoral dissertation, University of Maryland, College Park, MD.
26. Solano-Flores, G. (2008). Who is given tests in what language by whom, when, and where? The need for probabilistic views of language in the testing of English language learners. Educational Researcher, 37, 189–199.
27. Solano-Flores, G. (2014). Probabilistic approaches to examining linguistic features of test items and their effect on the performance of English language learners. Applied Measurement in Education, 27, 236–247.
28. Solano-Flores, G., & Li, M. (2009). Language variation and score variation in the testing of English language learners, native Spanish speakers. Educational Assessment, 14(3–4), 180–194.
29. Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361–370.
30. von Davier, A. A., & Oliveri, M. E. (2013). Psychometrics in support of a valid assessment of linguistic subgroups: Implications for the test and sampling designs. Paper presented at the meeting of the National Council on Measurement in Education, San Francisco, CA.
31. von Davier, M. (2008). The mixture general diagnostic model. In G. R. Hancock & K. M. Samuelsen (Eds.), Advances in latent variable mixture models (pp. 1–24). Charlotte, NC: Information Age Publishing.
32. Zenisky, A. L., Hambleton, R. K., & Robin, F. (2003). Detection of differential item functioning in large-scale state assessments: A study evaluating a two-stage approach. Educational and Psychological Measurement, 63, 51–64.
33. Zhang, Y., Dorans, N. J., & Matthews-López, J. L. (2005). Using DIF dissection method to assess effects of item deletion (College Board Research Report No. 2005-10). New York, NY: The College Board.
34. Zumbo, B. D., & Gelin, M. N. (2005). A matter of test bias in educational policy research: Bringing the context into picture by investigating sociological/community moderated (or mediated) test and item bias. Journal of Educational Research and Policy Studies, 5, 1–23.
35. Zwick, R., & Ercikan, K. E. (1989). Analysis of differential item functioning in the NAEP history assessment. Journal of Educational Measurement, 26, 55–66.