Analysis of Nonequivalent Assessments Across Different Linguistic Groups Using a Mixed Methods Approach: Understanding the Causes of Differential Item Functioning by Cognitive Interviewing

Journal of Mixed Methods Research

Volumn 8, Issue 1, 2014, Pages 52-68

  Benítez, Isabel ^a   Padilla, José Luis ^a  

^a UNIVERSITY OF GRANADA   (Spain)

Author keywords

cognitive interviews; cross lingual assessments; differential item functioning causes; mixed methods; Programme for International Student Assessment

Indexed keywords

EID: 84891068449     PISSN: 15586898     EISSN: 15586901     Source Type: Journal    
DOI: 10.1177/1558689813488245     Document Type: Article

References (44)

1. Allalouf A.,Hambleton R. K.,Sireci S.Identifying the causes of DIF in translated verbal items.Journal of Educational Measurement. 1999;36:185-198.
2. Beatty P.,Willis G. B.Research synthesis: The practice of cognitive interviewing.Public Opinion Quarterly. 2007;71:287-311.
3. Bryman A.Quantity and quality in social research. London, England: Unwin Hyman; 1988:.
4. Castillo-Díaz M.,Padilla J-L.How cognitive interviewing can provide validity evidence of the response processes to scale items.Social Indicator Research. 2012;:.
5. Clarke P. N.,Yaros P. S.Research blenders: Commentary and response.Nursing Science Quarterly. 1988;1:147-149.
6. Creswell J. W.Research design: Qualitative and quantitative approaches. Thousand Oaks. CA: Sage; 1995:.
7. Edmeades J.,Nyblade L.,Malhotra A.,MacQuarrie K.,Parasuraman S.,Walia S.Methodological innovation in studying abortion in developing countries: A narrative quantitative survey in Madhya Pradesh, India.Journal of Mixed Methods Research. 2010;4 (3): 176-198.
8. Elosua P.,López-Jaúregui A.Potential sources of differential item functioning in the adaptation of tests.International Journal of Testing. 2007;7:39-52.
9. Ercikan K.,Arim R.,Law D.,Domene J.,Gagnon F.,Lacroix S.Application of think aloud protocols for examining and confirming sources of differential item functioning identified by expert reviews.Educational Measurement: Issues and Practice. 2010;29 (2): 24-35.
10. Ferne T.,Rupp A. A.A synthesis of 15 years of research on DIF in language testing: Methodological advances, challenges, and recommendations.Language Assessment Quarterly. 2007;4:113-148.
11. Gadermann A. M.,Guhn M.,Zumbo B. D.Investigating the substantive aspect of construct validity for the satisfaction with life scale adapted for children: A focus on cognitive processes.Social Indicator Research. 2011;100:37-60.
12. Gierl M. J.,Khaliq S. N.Identifying sources of differential item and bundle functioning on translated achievement tests: A confirmatory analysis.Journal of Educational Measurement. 2001;38:164-187.
13. Hambleton R. K.Good practices for identifying differential item functioning.Medical Care. 2006;44:S182-S188.
14. Harkness J. A.Cross-cultural survey methods. Harkness J. A.van de Vijver F.Mohler P. Ph, ed. New York, NY: Wiley; 2003:19-34.
15. Hidalgo M. D.,Gómez-Benito J.International encyclopedia of education. 2010). Education measurement: Differential item functioning. In PetersonP.BakerE.McGawB. (Eds.), International encyclopedia of education (3rd ed., pp. 36-44). New York, NY: Elsevier Science.Peterson P.Baker E.McGaw B., ed. New York, NY: Elsevier Science; 2010:36-44.
16. Progress in International Reading Literacy Study (PIRLS). Alexandria, VA: National Center for Education Statistics; 2011:.
17. International Test Commission. (2010). International test commission guidelines for translating and adapting tests.Retrieved from http://www.intestcom.org.
18. Johnson T. P.Methods and frameworks for crosscultural measurement.Medical Care, 44. 2006;:S17-S20.
19. Linn R. L.,Harnisch D. L.Interactions between item content and group measurement on achievement test items.Journal of Educational Measurement. 1981;18:109-118.
20. Miller K.Design and analysis of cognitive interviews for cross-national testing. Paper presented at the European Survey Research Association annual meeting; 2007Prague, Czech Republic; 2007.
21. Miller T. R.,Spray J. A.Logistic discriminant function analysis for DIF identification of polytomously scored items.Journal of Educational Measurement. 1993;30:107-122.
22. Millsap R. E.,Everson H. T.Methodology review: Statistical approaches for assessing measurement bias.Applied Psychological Measurement. 1993;17:297-334.
23. National Center for Health Statistics. Q-notes software. Retrieved from https://www.cdc.gov/qnotes/login.aspx.
24. Organisation for Economic Co-operation and Development. (2006). PISA 2006 database.Retrieved from http://pisa2006.acer.edu.au/downloads.php.
25. Programme for International Student Assessment (PISA). Paris, France: Author; 2009:.
26. Paris, France: Author; 2011:.
27. Penfield R. D.DIFAS: Differential item functioning analysis system.Applied Psychological Measurement. 2005;29:150-151.
28. Penfield R. D.Distinguishing between net and global DIF in polytomous items.Journal of Educational Measurement. 2010;47:129-149.
29. Penfield R. D.,Alvarez K.,Lee O.Using a taxonomy of differential step functioning to improve the interpretation of dif in polytomous items: An illustration.Applied Measurement in Education. 2009;22:61-78.
30. Penfield R. D.,Gattamorta K.,Childs R. A.An NCME instructional module on using differential step functioning to refine the analysis of DIF in polytomous items.Educational Measurement: Issues and Practice. 2009;28:38-49.
31. Presser S.,Couper M. P.,Lessler J. T.,Martin E.,Martin J.,Rothgeb J. M.,Singer E.Methods for testing and evaluating survey questions.Public Opinion Quarterly. 2004;68:109-130.
32. Roussos L. A.,Stout W. F.A multidimensionality-based DIF analysis paradigm.Applied Psychological Measurement. 1996;20:355-371.
33. Schmeiser C. B.Handbook of methods for detecting test bias. Berk R. A., ed. Baltimore, MD: Johns Hopkins University Press; 1982:64-96.
34. Shealy R.,Stout W. F.Differential item functioning. Holland P. W.Wainer H., ed. Hillsdale, NJ: Lawrence Erlbaum; 1993:197-239.
35. Sireci S. G.Problems and issues in linking tests across languages.Educational Measurement: Issues and Practice. 1997;16:12-19.
36. Sireci S. G.,Patsula L.,Hambleton R. K.Adapting educational and psychological tests for cross-cultural assessment. Hambleton R. K.Merenda P. F.Spielberger C. D., ed. Hillsdale, NJ: Lawrence Erlbaum; 2005:93-115.
37. SPSS-16 user's guide. Chicago, IL: Author; 2007:.
38. Swanson D. B.,Clauser B. E.,Case S. M.,Nungester R. J.,Featherman C.Analysis of differential item functioning (DIF) using hierarchical logistic regression models.Journal of Educational and Behavioral Statistics. 2002;27:53-75.
39. Tashakkori A.,Teddlie C.Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage; 1998:.
40. van de,Vijver F. J. R.,Poortinga Y. H.Adapting educational and psychological tests for cross-cultural assessment. Hambleton R. K.Merenda P. F.Spielberger C. D., ed. Hillsdale, NJ: Lawrence Erlbaum; 2005:39-63.
41. Willis G. B.Cognitive interviewing. Thousand Oaks: Sage; 2005:.
42. Zieky M.Differential item functioning. Holland P. W.Wainer H., ed. Hillsdale, NJ: Lawrence Erlbaum; 1993:337-347.
43. Zumbo B. D.A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, Ontario, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense; 1999:.
44. Zumbo B. D.The concept of validity: Revisions, new directions and applications. Lissitz R. W., ed. Charlotte, NC: Information Age; 2009:65-82.