How do we go about investigating test fairness?

Language Testing

Volumn 27, Issue 2, 2010, Pages 147-170

  Xi, Xiaoming ^a  

^a EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

Comparable validity; Fairness argument; Test fairness; Test validity; Validity; Validity argument

Indexed keywords

EID: 77952898170     PISSN: 02655322     EISSN: 14770946     Source Type: Journal    
DOI: 10.1177/0265532209349465     Document Type: Article

References (55)

1. Alderson, J.C. & Urquhart, A.H. (1985a). The effect of students' academic discipline on their performance in ESP reading tests. Language Testing, 2, 192-204.
2. Alderson, J.C. & Urquhart, A.H. (1985b). This test is unfair: I'm not an economist. In Hauptman, P. C., LeBlanc, R. & Wesche, M. B. (Eds.), Second language performance testing (pp. 25-44). Ottawa: University of Ottawa Press.
3. Angoff, W.H. (1989). Context bias in the Test of English as a Foreign Language (TOEFL Research Report. RR-29). Princeton, NJ: Educational Testing Service.
4. Association of Language Testers in Europe (1994). The Association of Language Testers of Europe Code of Practice. http://www.alte.org.
5. American Educational Research Association, American Psychological Association and National Council on Measurement in Education (1985). Standards for educational and psychological testing. Washington, DC: Author.
6. American Educational Research Association, American Psychological Association and National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, DC: Author.
7. Bachman, L.F. (2005). Building and supporting a case for test use. Language Assessment Quarterly, 2, 1-34.
8. Breland, H., Lee, Y-W., Najarian, M. & Muraki, E. (2004). An analysis of TOEFL- CBT writing prompt difficulty and comparability for different gender groups (TOEFL Research Report. RR-76). Princeton, NJ: Educational Testing Service.
9. Brown, A. (2003). Interviewer variation and the co-construction of speaking proficiency. Language Testing, 20, 1-25.
10. Brown, J.D. (1999). Relative importance of persons, items, subtests and languages to TOEFL test variance. Language Testing, 16, 216-237.
11. Chapelle, C. A., Enright, M. K. & Jamieson, J. M. (Eds.) (2008). Building a validity argument for the Test of English as a Foreign Language™. Mahwah, NJ: Lawrence Erlbaum.
12. Cheng, L. (2008). Washback, impact and consequences. In Shohamy, E. & Hornberger, N. H. (Eds.), Encyclopedia of language and education. Volume 7: Language testing and assessment, 2nd edn. (pp. 349-364). New York: Springer Science and Business Media LLC.
13. Clapham, C. (1998). The effect of language proficiency and background knowledge on EAP students' reading comprehension. In Kunnan, A. J. (Ed.), Validation in language assessment (pp. 141-168). Mahwah, NJ: Lawrence Erlbaum.
14. Cronbach, L.J. (1988). Five perspectives on the validity argument. In Wainer, H. & Braun, H. I. (Eds.), Test validity (pp. 3-17). Hillsdale, NJ: Lawrence Erlbaum.
15. Cronbach, L.J. & Meehl, P.E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.
16. Cureton, E.E. (1951).Validity. In Lindquist, E. F., (Ed.), Educational measurement, 1st edn (pp. 621-694). Washington, DC: American Council on Education.
17. Davies, A. guest editor(1997). Ethics (Special issue). Language Testing, 14.
18. Davies, A. guest editor(2004). Ethics (Special issue). Language Assessment Quarterly, 4.
19. Educational Testing Service (2002). ETS standards for quality and fairness. Princeton, NJ: Author.
20. Educational Testing Service (2003). ETS fairness review guidelines. Princeton, NJ: Author.
21. Ferne, T. & Rupp, A. (2007). A synthesis of research on DIF in language testing: methodological advances, challenges, and recommendations. Language Assessment Quarterly, 4, 113-148.
22. Fulcher, G. & Davidson, F. (2007). Language testing and assessment. London and New York: Routledge.
23. Ginther, A. & Stevens, J. (1998). Language background, ethnicity, and the internal construct validity of the Advanced Placement Spanish language examination. In Kunnan, J. (Ed.), Validation in language assessment (pp. 169-194). Mahwah, NJ: Lawrence Erlbaum.
24. Hale, G. (1988). Student major field and text content: Interactive effects on reading comprehension in the TOEFL. Language Testing, 5, 49-61.
25. Hale, G.A., Rock, D.A. & Jirele, T.J. (1989). Confirmatory factor analysis of the Test of English as a Foreign Language (TOEFL Research Report. RR-32). Princeton, NJ: Educational Testing Service.
26. International Language Testing Association (2000). Code of Ethics for ILTA, Retrieved February 18, 2008 from http://www.iltaonline.com/code.pdf.
27. International Language Testing Association (2005). Draft Code of Practice, Version 3, Retrieved February 18, 2008 from http://www.iltaonline.com/ILTA-COP-ver3-21Jun2006.pdf.
28. Jensen, H.R. (1980). Bias in mental testing. New York: Free Press.
29. Joint Committee on Testing Practices (1988). Code of fair testing practices in education. Washington, DC: Author.
30. Joint Committee on Testing Practices (2004). Code of fair testing practices in education. Washington, DC: Author.
31. Kane, M.T. (1992). An argument-based approach to validity. Psychological Bulletin, 112, 527-535.
32. Kane, M.T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38, 319-342.
33. Kane, M.T. (2002). Validating high-stakes testing programs. Educational Measurement: Issues and Practice, 21, 31-41.
34. Kane, M.T. (2004). Certification testing as an illustration of argument-based validation. Measurement: Interdisciplinary Research and Perspectives, 2, 135-170.
35. Kane, M.T. (2006). Validation. In Brennan, R. L. (Ed.), Educational measurement, 4th edn. (pp.18-64). Washington, DC: American Council on Education/Praeger.
36. Kane, M., Crooks, T. & Cohen, A. (1999). Validating measures of performance. Educational Measurement: Issues and Practice, 18, 5-17.
37. Kunnan, A.J. (1995). Test taker characteristics and performance: A structural modeling approach. Cambridge, UK: Cambridge University Press.
38. Kunnan, A.J. (2000). Fairness and justice for all. In A. J. Kunnan (Ed.), Fairness and validation in language assessment (pp. 1-14). Cambridge, UK: Cambridge University Press.
39. Kunnan, A.J. (2004). Test fairness. In Milanovic, M. & Weir C., (Eds.), European language testing in a global context: Proceedings of the ALTE Barcelona Conference (pp. 27-48). Cambridge, UK: Cambridge University Press.
40. McNamara, T.F. & Roever, C. (2006). Language testing: The social dimension. Oxford: Blackwell.
41. Messick, S. (1989). Validity. In Linn, R. L. (Ed.), Educational measurement, 3rd edn. (pp. 13-103). New York: American Council on Education and Macmillan.
42. O'Loughlin, K. (2002). The impact of gender in oral proficiency testing. Language Testing, 19, 169-192.
43. Oltman, P., Stricker L. & Barrows, T. (1990). Analyzing test structure by multidimensional scaling. Journal of Applied Psychology, 75, 21-27.
44. Shohamy, E. (2000). Fairness in language testing. In Kunnan, A. J. (Ed.), Fairness and validation in language assessment (pp. 15-19). Cambridge, UK: Cambridge University Press.
45. Shohamy, E. (2001). The power of tests: A critical perspective of the uses of language tests. London: Longman.
46. Swinton, S.S. & Powers, D.E. (1980). Factor analysis of the TOEFL® test for several language groups (TOEFL Research Report. RR-06). Princeton, NJ: Educational Testing Service.
47. Stricker, L.J., Rock, D.A. & Lee, Y.W. (2005). Factor structure of the LanguEdgeTM Test across language groups (TOEFL Monograph Series MS-32). Princeton, NJ: Educational Testing Service.
48. Taylor, C., Jamieson, J., Eignor, D. & Kirsch, I. (1998). The relationship between computer familiarity and performance on computer-based TOEFL test tasks (TOEFL Research Report. RR-61). Princeton, NJ: Educational Testing Service.
49. Toulmin, S., Rieke, R. & Janik, A. (1984). An introduction to reasoning, 2nd edn. New York: Macmillan.
50. Webster's Ninth New Collegiate Dictionary (1988). Springfield, MA: Merriam-Webster.
51. Willingham, W.W. (1999). A systemic view of test fairness. In Messick S. (Ed.), Assessment in higher education: Issues in access, quality, student development, and public policy (pp. 213-242). Mahwah, NJ: Lawrence Erlbaum.
52. Willingham, W.W. & Cole, N. (1997). Gender and fair assessment. Mahwah, NJ: Lawrence Erlbaum.
53. Zeidner, M. (1986). Are English language aptitude tests biased towards culturally different minority groups? Some Israeli findings. Language Testing, 3, 80-95.
54. Zeidner, M. (1987). A comparison of ethnic, sex and age biases in the predictive validity of English language aptitude tests: Some Israeli data. Language Testing, 4, 55-71.
55. Zwick, R.J. & Thayer, D.T. (1995). A comparison of the performance of graduate and undergraduate school applicants on the Test of Written English (TOEFL Research Report. RR-50). Princeton, NJ: Educational Testing Service.