Inconsistencies in DIF Detection for Sub-Groups in Heterogeneous Language Groups

Applied Measurement in Education

Volumn 27, Issue 4, 2014, Pages 273-285

  Ercikan, Kadriye ^a   Roth, Wolff Michael ^b   Simon, Marielle ^c   Sandilands, Debra ^a   Lyons Thomas, Juliette ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^b UNIVERSITY OF VICTORIA   (Canada)

^c UNIVERSITY OF OTTAWA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84907473598     PISSN: 08957347     EISSN: None     Source Type: Journal    
DOI: 10.1080/08957347.2014.944306     Document Type: Article

References (42)

1. Abedi, J. (2004). The No Child Left behind Act and English language learners: Assessment and accountability issues. Educational Researcher, 33(1), 4–14.
2. Allen, M., & Cartwright, F. (2004). Minority language school systems: A profile of students, schools and communities. Education Quarterly Review, 9, 9–48.
3. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (AERA, APA, & NCME) (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
4. Bernardo, A. B. I. (2002). Language and mathematical problem solving among bilinguals. Journal of Psychology, 136, 283–297.
5. Council of Ministers of Education Canada (CMEC) (2005). SAIP 2004: Report on Science III. Toronto, Canada: Council of Ministers of Education Canada.
6. CTB/McGraw-Hill. (1991). PARDUX. [Computer software]. Monterey, CA; CTB/McGraw-Hill.
7. Duran, R. P. (1989). Testing of linguistic minorities. In R. Linn (Ed.), Educational measurement (3rd ed.) (pp. 573–587). New York, NY: Macmillan.
8. Ercikan, K. (2002). Disentangling sources of differential item functioning in multi-language assessments. International Journal of Testing, 2, 199–215.
9. Ercikan, K. (2006). Developments in assessment of student learning and achievement. In P.A. Alexander & P. H. Winne (Eds.), American Psychological Association, Division 15, Handbook of educational psychology (2nd ed.) (pp. 929–953). Mahwah, NJ: Lawrence Erlbaum Associates.
10. Ercikan, K. (2009). Limitations in sample to population generalizing. In K. Ercikan & M-W. Roth (Eds.), Generalizing in educational research: Beyond qualitative and quantitative polarization (pp. 211–235), New York, NY: Routledge.
11. Ercikan, K., Arim, R. G., Law, D. M., Lacroix, S., Gagnon, F., & Domene, J. F. (2010). Application of think-aloud protocols in examining sources of differential item functioning. Educational Measurement: Issues and Practice, 29, 24–35.
12. Ercikan, K., & Barclay-McKeown, S. (2007). Design and development issues in large-scale assessments: Designing assessments to provide useful information to guide policy and practice. Canadian Journal of Program Evaluation, 22, 53–71.
13. Ercikan, K., & Lyons-Thomas, J. (2013). Adapting tests for use in other languages and cultures. In K. Geisinger (Ed.), APA Handbook of Testing and Assessment in Psychology, Volume 3, (pp. 545–569). Washington, DC: American Psychological Association.
14. Ercikan, K., & McCreith, T. (2002). Effects of adaptations on comparability of test items and test scores. In D. Robitaille & A. Beaton (Eds.), Secondary analysis of the TIMSS results: A synthesis of current research (pp. 391–407). Dordrecht, The Netherlands: Kluwer Academic Publishers.
15. Ercikan, K. & Oliveri, M. E. (2013). Is fairness research doing justice? A modest proposal for an alternative validation approach in differential item functioning (DIF) investigations. In M. Chatterji (Ed.), Validity, fairness and testing of individuals in high stakes decision-making context (pp. 69–86). Bingley, UK: Emerald Publishing.
16. Ercikan, K., Oliveri, M. E, & Sandilands, D. (2013). Large scale assessments of achievement in Canada. In J. A. C. Hattie & E. M. Anderman (Eds.), The international handbook of student achievement (pp. 456–459). New York, NY: Routledge.
17. Ercikan, K., & Roth, W.-M. (2009). Generalizing from educational research: Beyond qualitative and quantitative polarization. New York, NY: Routledge.
18. Ercikan, K., & Roth, W.-M. (2014). Limits of generalizing in education research: Why criteria for research generalization should include population heterogeneity and users of knowledge claims. Teachers College Record, 116(5), 1–28.
19. Farmer, D. (2008). My mother is from Russia, My father is from Rwanda. Canadian Issues, 3, 124–127.
20. Farmer, D., & Labrie, N. (2008). Immigration et francophonie dans les écoles ontariennes: Comment se structurent les rapports entre les institutions, les parents et le monde communautaire? Revue des sciences de l’éducation, 34(2), 377–398. Retrieved from http://id.erudit.org/iderudit/019686ar
21. Hambleton, R. K., Swaminathan, H., & Rogers, J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
22. Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129–145). Hillsdale, NJ: Lawrence Erlbaum Associates.
23. Linn, R. L., & Harnisch, D. L. (1981). Interactions between item content and group measurement on achievement test items. Journal of Educational Measurement, 18, 109–118.
24. Mendes-Barnett, S., & Ercikan, K. (2006). Examining sources of gender DIF in mathematics assessments using a confirmatory multidimensional model approach. Applied Measurement in Education, 19, 289–304.
25. Oliveri, M. E., Ercikan, K., & Zumbo, B. D. (2013). Analysis of sources of latent class DIF in international assessments. International Journal of Testing, 13, 272–293.
26. Oliveri, M. E., Ercikan, K., & Zumbo, B. D. (2014). Effects of population heterogeneity on accuracy of DIF detection. Applied Measurement in Education, 27, 286–300.
27. Oliveri, M. E., Ercikan, K., Zumbo, B., & Lawless, R. (in press). Uncovering substantive patterns in student responses in international large-scale assessments – Comparing a latent class to a manifest DIF approach. International Journal of Testing, 14, 265–287.
28. Ontario Ministry of Education. (2009). Policy statement and guidelines on the admission, Welcoming, and support of students in french-language schools in Ontario. Toronto, Canada: Queen’s Printer for Ontario.
29. Organisation for Economic Co-operation and Development. (2002). PISA 2000 technical report. Paris, France: Author.
30. Organisation for Economic Co-operation and Development. (2005). PISA 2003 technical report. Paris, France: Author.
31. Organisation for Economic Co-operation and Development. (2006). Assessing scientific, reading and mathematical literacy: A framework for PISA 2006. Paris, France: Author.
32. Pellegrino, J., Chudowsky, N., & Glaser, R. (Eds.). (2001). Knowing what students know: The science and design of educational assessment (National Research Council’s Committee on the Foundations of Assessment). Washington, DC: National Academy Press.
33. Roth, W.-M., & Bowen, G. M. (2003). When are graphs ten thousand words worth? An expert/expert study. Cognition and Instruction, 21, 429–473.
34. Roth, W.-M., Oliveri, M. E., Sandilands, D., Lyons-Thomas, J., & Ercikan, K. (2013). Investigating sources of differential item functioning using expert think-aloud protocols. International Journal of Science Education, 35, 546–576.
35. Rutkowski, L., Gonzalez, E., Joncas, M., & von Davier, M. (2010). International large-scale assessment data. Educational Researcher, 39(2), 142–151.
36. Solano-Flores, G. (2008). Who is given tests in what language by whom, when, and where? The need for probabilistic views of language in the testing of English language learners. Educational Researcher, 37(4), 189–199.
37. Solano-Flores, G. (2014). Probabilistic approaches to examining linguistic features of test items and their effect on the performance of English language learners. Applied Measurement in Education, 27, 236–247.
38. Solano-Flores, G., & Trumbull, E. (2003). Examining language in context: The need for new research and practice paradigms in the testing of English-language learners. Educational Researcher, 32(2), 3–13.
39. Statistics Canada. (2006). Population by mother tongue, by census metropolitan area. Retrieved from http://www.statcan.gc.ca/tables-tableaux/sum-som/l01/cst01/demo12b-eng.htm
40. Statistics Canada. (2007). The evolving linguistic portrait, 2006 Census. Ottawa, Canada: Author.
41. Vygotsky, L. S. (1989). Concrete human psychology. Soviet Psychology, 27(2), 53–77.
42. Zwick, R., & Ercikan, K. (1989). Differential item functioning in NAEP. Journal of Educational of Measurement, 26, 55–66.