Evaluating test and survey items for bias across languages and cultures

Cross-Cultural Research Methods in Psychology

Volumn , Issue , 2010, Pages 216-240

  Sireci, Stephen G ^a  

^a UNIVERSITY OF MASSACHUSETTS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84925670872     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/CBO9780511779381.011     Document Type: Chapter

References (57)

1. Allalouf, A., Hambleton, R. K., & Sireci, S. G. (1999). Identifying the sources of differential item functioning in translated verbal items. Journal of Educational Measurement, 36, 185–198.
2. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
3. Angoff, W. H. (1972). Use of difficulty and discrimination indices for detecting item bias. In R. A. Berk (Ed.), Handbook of methods for detecting test bias (96–116). Baltimore: Johns Hopkins University Press.
4. Angoff, W. H., & Cook, L. L. (1988). Equating the scores of the Prueba de Aptitud Academica and the Scholastic Aptitude Test (Report No. 88-2). New York: College Entrance Examination Board.
5. Angoff, W. H., & Modu, C. C. (1973). Equating the scores of the Prueba de Aptitud Academica and the Scholastic Aptitude Test (Research Report No. 3). New York: College Entrance Examination Board.
6. Brislin, R. W. (1970). Back-translation for cross-cultural research. Journal of Cross- Cultural Psychology, 1, 185–216.
7. Budgell, G., Raju, N., & Quartetti, D. (1995). Analysis of differential item functioning in translated assessment instruments. Applied Psychological Measurement, 19, 309–321.
8. Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage.
9. Clauser, B. E., & Mazor, K. M. (1998). Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice, 17, 31–44.
10. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.
11. Day, S. X., & Rounds, J. (1998). Universality of vocational interest structure among racial and ethnic minorities. American Psychologist, 53, 728–736.
12. Dorans, N. J., & Holland, P. W. (1993). DIF detection and description: Mantel- Haenszel and standardization. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (35–66). Hillsdale, NJ: Erlbaum.
13. Dorans, N. J., & Kulick E. (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the Scholastic Aptitude Test. Journal of Educational Measurement, 23, 355–368.
14. Fischer, G. (1993). Notes on the Mantel-Haenszel procedure and another chi- squared test for the assessment of DIF. Methodika, 7, 88–100.
15. Geisinger, K. F. (1994). Cross-cultural normative assessment: Translation and adaptation issues influencing the normative interpretation of assessment instruments. Psychological Assessment, 6, 304–312.
16. Gierl, M. J., & Khaliq, S. N. (2001). Identifying sources of differential item and bundle functioning on translated achievement tests: A confirmatory analysis. Journal of Educational Measurement, 38, 164–187.
17. Hambleton, R. K. (1993). Translating achievement tests for use in cross-national studies. European Journal of Psychological Assessment, 9, 57–68.
18. Hambleton, R. K. (1994). Guidelines for adapting educational and psychological tests: A progress report. European Journal of Psychological Assessment, 10, 229–244.
19. Hambleton, R. K. (2005). Issues, designs, and technical guidelines for adapting tests into multiple languages and cultures. In R. K. Hambleton, P. Merenda, & C. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (3–38). Hillsdale, NJ: Erlbaum.
20. Hambleton, R. K., Sireci, S. G., & Robin, F. (1999). Adapting credentialing exams for use in multiple languages. CLEAR Exam Review, 10, 24–28.
21. Hauger, J. B., & Sireci, S. G. (2008). Detecting differential item functioning across examinees tested in their dominant language and examinees tested in a second language. International Journal of Testing, 8, 237–250.
22. Holland, P. W., & Thayer, D. T. (1988). Differential item functioning and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity(129–145). Hillsdale, NJ: Erlbaum.
23. Holland, P. W., & Wainer, H. (Eds.). (1993). Differential item functioning. Hillsdale, NJ: Erlbaum.
24. International Test Commission. (2001). International Test Commission guidelines for test adaptation. London: Author.
25. Jodoin, M. G., & Gierl, M. J. (2001). Evaluating power and Type I error rates using an effect size with the logistic regression procedure for DIF. Applied Measurement in Education, 14, 329–349.
26. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
27. Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 19–48.
28. Millsap, R. E., & Everson, H.T. (1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 297–334.
29. Muniz, J., Hambleton, R. K., & Xing, D. (2001). Small sample studies to detect flaws in test translation. International Journal of Testing, 1, 115–135.
30. Penfield, R. D. (2005). DIFAS: Differential item functioning analysis system. Applied Psychological Measurement, 29, 150–151.
31. Potenza, M. T., & Dorans, N. J. (1995). DIF assessment for polytomously scored items: A framework for classification and evaluation. Applied Psychological Measurement, 19, 23–37.
32. Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53, 495–502.
33. Raju, N. S. (1990). Determining the significance of estimated signed and unsigned areas between two item response functions. Applied Psychological Measurement, 14,197–207.
34. Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552–566.
35. Robin, F. (1999). SDDIF: Standardization and delta DIF analyses [computer program]. Amherst: University of Massachusetts, Laboratory of Psychometric and Evaluative Research.
36. Robin, F., Sireci, S. G., & Hambleton, R. K. (2003). Evaluating the equivalence of different language versions of a credentialing exam. International Journal of Testing, 3, 1–20.
37. Rogers, H. J., & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17, 105–116.
38. Shealy, R., & Stout, W. (1993). A model-based standardization differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159–194.
39. Sireci, S. G. (1997). Problems and issues in linking tests across languages. Educational Measurement: Issues and Practice, 16, 12–19.
40. Sireci, S. G. (2005). Using bilinguals to evaluate the comparability of different language versions of a test. In R. K. Hambleton, P. Merenda, & C. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment(117–138). Hillsdale, NJ: Erlbaum.
41. Sireci, S. G., Bastari, B., & Allalouf, A. (1998, August). Evaluating construct equivalence across adapted tests. Invited paper presented at the annual meeting of the American Psychological Association (Division 5), San Francisco, CA.
42. Sireci, S. G., & Berberoglu, G. (2000). Evaluating translation DIF using bilinguals. Applied Measurement in Education, 13, 229–248.
43. Sireci, S. G., Fitzgerald, C., & Xing, D. (1998). Adapting credentialing examinations for international uses. Laboratory of Psychometric and Evaluative Research Report No. 329. Amherst, MA: University of Massachusetts, School of Education.
44. Sireci, S. G., Harter, J., Yang, Y., & Bhola, D. (2003). Evaluating the equivalence of an employee attitude survey across languages, cultures, and administration formats. International Journal of Testing, 3,129–150.
45. Sireci, S. G., Patsula, L., & Hambleton, R. K. (2005). Statistical methods for identifying flawed items in the test adaptations process. In R. K. Hambleton, P. Merenda, &C. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (93–115). Hillsdale, NJ: Erlbaum.
46. Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361–370.
47. Thissen, D. (2001). IRTLRDIF v2.0b: Software for the computation of the statistics involved in item response theory likelihood-ratio tests for differential item functioning. Retrieved from http://www.unc.edu/~dthissen/dl.html.
48. Thissen, D., Steinberg, L., & Wainer, H. (1988). Use of item response theory in the study of group differences in trace lines. In H. Wainer & H. I. Braun (Eds.), Test validity (147–169). Hillsdale, NJ: Erlbaum.
49. Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (67–114). Mahwah, NJ: Erlbaum.
50. Van de Vijver, F. J. R., & Poortinga, Y. H. (1997). Towards an integrated analysis of bias in cross-cultural assessment. European Journal of Psychological Assessment, 13, 29–37.
51. Van de Vijver, F. J. R., & Poortinga, Y. H. (2005). Conceptual and methodological issues in adapting tests. In R. K. Hambleton, P. Merenda, & C. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 39–63). Hillsdale, NJ: Erlbaum.
52. Van de Vijver, F., & Tanzer, N. K. (1997). Bias and equivalence in cross-cultural assessment. European Review of Applied Psychology, 47, 263–279.
53. Wainer, H., & Sireci, S. G. (2005). Item and test bias. Encyclopedia of social measurement (Vol. 2,365–371). San Diego, CA: Elsevier.
54. Wainer, H., Sireci, S. G., & Thissen, D. (1991). Differential testlet functioning: Definitions and detection. Journal of Educational Measurement, 28,197–219.
55. Waller, N. G. (1998). EZDIF: The detection of uniform and non-uniform differential item functioning with the Mantel–Haenszel and logistic regression procedures. Applied Psychological Measurement, 22, 391.
56. Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense.
57. Zwick, R., Donoghue, J. R., & Grima, A. (1993). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30, 233–251.