Exploring the various interpretations of "test bias"

Cultural Diversity and Ethnic Minority Psychology

Volumn 20, Issue 4, 2014, Pages 570-582

  Warne, Russell T ^a   Yoon, Myeongsun ^b   Price, Chris J ^a  

^a UTAH VALLEY UNIVERSITY   (United States)

^b TEXAS A AND M UNIVERSITY   (United States)

Author keywords

Differential item functioning; Group differences; Item bias; Standardized tests; Test bias

Indexed keywords

ARTICLE; EMPIRICAL RESEARCH; HUMAN; POPULATION DYNAMICS; PREDICTIVE VALIDITY; SOCIAL ASPECT; STANDARDIZATION; STATISTICAL BIAS; TEST BIAS; THEORETICAL STUDY; AWARENESS; EDUCATION; SOCIOECONOMICS; STANDARDS;

AWARENESS; BIAS (EPIDEMIOLOGY); EDUCATIONAL MEASUREMENT; HUMANS; SOCIOECONOMIC FACTORS;

EID: 84920513939     PISSN: 10999809     EISSN: 19390106     Source Type: Journal    
DOI: 10.1037/a0036503     Document Type: Article

References (97)

1. ACT. (2007). The ACT technical manual. Retrieved from http://www.act.org/aap/pdf/ACT_Technical_Manual.pdf
2. Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Long Grove, IL: Waveland Press.
3. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
4. Andrich, D., & Hagquist, C. (2012). Real and artificial differential item functioning. Journal of Educational and Behavioral Statistics, 37, 387-416. doi:10.3102/1076998611411913
5. Beaujean, A. A., McGlaughlin, S. M., & Margulies, A. S. (2009). Factorial validity of the Reynolds Intellectual Assessment Scales for referred students. Psychology in the Schools, 46, 932-950. doi:10.1002/pits.20435
6. Benson, N., Hulac, D. M., & Kranzler, J. H. (2010). Independent examination of the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV): What does the WAIS-IV measure? Psychological Assessment, 22, 121-130. doi:10.1037/a0017767
7. Bersoff, D. N. (1981). Testing and the law. American Psychologist, 36, 1047-1056. doi:10.1037/0003-066X.36.10.1047
8. Borsboom, D. (2006). The attack of the psychometricians. Psychometrika, 71, 425-440. doi:10.1007/s11336-006-1447-6
9. Buckendahl, C. W., & Hunt, R. (2005). Whose rules? The relation between the "rules" and "law" of testing. In R. P. Phelps (Ed.), Defending standardized testing (pp. 147-158). Mahwah, NJ: Erlbaum.
10. Camara, W. J. (2009). College admission testing: Myths and realities in an age of admissions hype. In R. P. Phelps (Ed.), Correcting fallacies about educational and psychological testing (pp. 147-180). Washington, DC: American Psychological Association. doi:10.1037/11861-004
11. Camilli, G. (2006). Test fairness. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 221-256). Westport, CT: Praeger.
12. Carlson, J. F., & Geisinger, K. F. (2009). Psychological diagnostic testing: Addressing challenges in clinical applications of testing. In R. P. Phelps (Ed.), Correcting fallacies about educational and psychological testing (pp. 67-88). Washington, DC: American Psychological Association. doi:10.1037/11861-002
13. Carroll, L. (1871/1917). Through the looking-glass and what Alice found there. New York, NY: Rand McNally.
14. Carter, R. T., & Goodwin, A. L. (1994). Racial identity and education. Review of Research in Education, 20, 291-336. doi:10.2307/1167387
15. Cizek, G. J., Fitzgerald, S. M., & Rachor, R. A. (1995). Teachers' assessment practices: Preparation, isolation, and the kitchen sink. Educational Assessment, 3, 159-179. doi:10.1207/s15326977ea0302_3
16. Clarizio, H. F. (1979). In defense of the IQ test. School Psychology Review, 8, 79-88.
17. Cleary, T. A. (1968). Test bias: Prediction of Negro and White students in integrated colleges. Journal of Educational Measurement, 5, 115-124. doi:10.1111/j.1745-3984.1968.tb00613.x
18. Cleary, T. A., & Hilton, T. L. (1968). An investigation of item bias. Educational and Psychological Measurement, 28, 61-75. doi:10.1177/001316446802800106
19. Cleary, T. A., Humphreys, L. G., Kendrick, S. A., & Wesman, A. (1975). Educational uses of tests with disadvantaged students. American Psychologist, 30, 15-41. doi:10.1037/0003-066X.30.1.15
20. Crocker, L., & Algina, J. (2008). Introduction to classical and modern test theory. Mason, OH: Cengage Learning.
21. Cronbach, L. J. (1975). Five decades of public controversy over mental testing. American Psychologist, 30, 1-14. doi:10.1037/0003-066X.30.1.1
22. Cronbach, L. J. (1980). Selection theory for a political world. Public Personnel Management, 9, 37-50.
23. Dolan, C. V. (2000). Investigating Spearman's hypothesis by means of multi-group confirmatory factor analysis. Multivariate Behavioral Research, 35, 21-50. doi:10.1207/S15327906MBR3501_2
24. Elliott, R. (1987). Litigating intelligence: IQ tests, special education, and social science in the courtroom. Dover, MA: Auburn House.
25. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.
26. Flaugher, R. L. (1978). The many definitions of test bias. American Psychologist, 33, 671-679. doi:10.1037/0003-066X.33.7.67
27. Flowers, L. A. (2008). Racial differences in the impact of participating in Advanced Placement programs on educational and labor market outcomes. Educational Foundations, 22, 121-132.
28. Ford, D. Y. (2003). Equity and excellence: Culturally diverse students in gifted education. In N. Colangelo & G. A. Davis (Eds.), Handbook of gifted education (pp. 506-520). Boston, MA: Allyn & Bacon.
29. Forsyth, R. A., Ansley, T. N., Feldt, L. S., & Alnot, S. D. (2003). Iowa Tests of Educational Development guide to research and development. Itasca, IL: Riverside Publishing.
30. Goodman, D., & Hambleton, R. K. (2005). Some misconceptions about large-scale educational assessments. In R. P. Phelps (Ed.), Defending standardized testing (pp. 91-110). Mahwah, NJ: Erlbaum.
31. Gorsuch, R. L. (2003). Factor analysis. In J. A. Schinka & W. F. Velicer (Eds.), Handbook of psychology: Vol. 2. Research methods in psychology (pp. 143-164). Hoboken, NJ: Wiley.
32. Gottfredson, L. S. (1994). Egalitarian fiction and collective fraud. Society, 31, 53-59. doi:10.1007/BF02693231
33. Gottfredson, L. S. (1997). Why g matters: The complexity of everyday life. Intelligence, 24, 79-132. doi:10.1016/S0160-2896(97)90014-3
34. Gottfredson, L. S. (2009). Logical fallacies used to dismiss the evidence on intelligence testing. In R. P. Phelps (Ed.), Correcting fallacies about educational and psychological testing (pp. 11-65). Washington, DC: American Psychological Association. doi:10.1037/11861-001
35. Gould S. J. (1981). The mismeasure of man. New York, NY: Norton.
36. Hoover, H. D., Dunbar, S. B., Frisbie, D. A., Oberley, K. R., Ordman, V. L., Naylor, R. J.,. Shannon, G. P. (2003). Iowa Tests of Basic Skills guide to research and development. Itasca, IL: Riverside Publishing.
37. Hunter, J. E., & Schmidt, F. L. (2000). Racial and gender bias in ability and achievement tests: Resolving the apparent paradox. Psychology, Public Policy, and Law, 6, 151-158. doi:10.1037/1076-8971.6.1.151
38. Jensen, A. R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger.
39. Johansen, K. F. (1998). A history of ancient philosophy from the beginnings to Augustine. New York, NY: Routledge.
40. Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 17-64). Westport, CT: Praeger.
41. Kane, M. T., & Mroch, A. A. (2010). Modeling group differences in OLS and orthogonal regression: Implications for differential validity studies. Applied Measurement in Education, 23, 215-241. doi:10.1080/08957347.2010.485990
42. Kaplan, R. M., & Saccuzzo, D. P. (2009). Psychological testing: Principles, applications, and issues (7th ed.). Belmont, CA: Wadsworth.
43. Koh, T.-h., Abbatiello, A., & McLoughlin, C. S. (1984). Cultural bias in WISC subtest items: A response to Judge Grady's suggestion in relation to the PASE case. School Psychology Review, 13, 89-94.
44. Koretz, D. (2003). Using multiple measures to address perverse incentives and score inflation. Educational Measurement: Issues and Practice, 22, 18-26. doi:10.1111/j.1745-3992.2003.tb00124.x
45. Lane, S., Zumbo, B. D., Abedi, J., Benson, J., Dossey, J., Elliott, S. N.,. Willhoft, J. (2009). Prologue: An introduction to the evaluation of NAEP. Applied Measurement in Education, 22, 309-316. doi:10.1080/08957340903221436
46. Lee, J. (2002). Racial and ethnic achievement gap trends: Reversing the progress toward equity? Educational Researcher, 31, 3-12. doi:10.3102/0013189X031001003
47. Li, H., Lee, D., Pfeiffer, S. I., Kamata, A., Kumtepe, A. T., & Rosado, J. (2009). Measurement invariance of the Gifted Rating Scales-School Form across five cultural groups. School Psychology Quarterly, 24, 186-198. doi:10.1037/a0017382
48. Linn, R. L., & Drasgow, F. (1987). Implications of the Golden Rule settlement for test construction. Educational Measurement: Issues and Practice, 6, 13-17. doi:10.1111/j.1745-3992.1987.tb00405.x
49. Lohman, D. F. (2005). Review of Naglieri and Ford (2003): Does the Naglieri Nonverbal Ability Test identify equal proportions of highscoring White, Black, and Hispanic students? Gifted Child Quarterly, 49, 19-28. doi:10.1177/001698620504900103
50. Larry P. v. Riles, 495 F. Supp. 926 (N. D. Cal. 1979).
51. Maruyama, G. (2012). Assessing college readiness: Should we be satisfied with ACT or other threshold scores? Educational Researcher, 41, 252-261. doi:10.3102/0013189X12455095
52. Mattern, K. D., Shaw, E. J., & Kobrin, J. L. (2011). An alternative presentation of incremental validity: Discrepant SAT and HSGPA performance. Educational and Psychological Measurement, 71, 638-662. doi:10.1177/0013164410383563
53. McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
54. Mensch, E., & Mensch, H. (1991). The IQ mythology: Class, race, gender, and inequality. Carbondale, IL: Southern Illinois University Press.
55. Mercer, J. R. (1979). In defense of racially and culturally nondiscriminatory assessment. School Psychology Review, 8, 89-115.
56. Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525-543. doi:10.1007/BF02294825
57. Millsap, R. E., & Everson, H. T. (1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 297-334. doi:10.1177/014662169301700401
58. Millsap, R. E., & Kwok, O.-M. (2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9, 93-115. doi:10.1037/1082-989X.9.1.93
59. Morgan, P. L., Farkas, G., Hillemeier, M. M., & Maczuga, S. (2012). Are minority children disproportionately represented in early intervention and early childhood special education? Educational Researcher, 41, 339-351. doi:10.3102/0013189X12459678
60. Moss, G. (2008). Diversity study circles in teacher education practice: An experiential learning project. Teaching and Teacher Education, 24, 216-224. doi:10.1016/j.tate.2006.10.010
61. National Center for Educational Statistics. (2009). NAEP 2008 trends in academic progress (NCES Report 2009-479). Retrieved from http://nces.ed.gov/nationsreportcard/pdf/main2008/2009479.pdf
62. Navas-Ara, M. J., & Gómez-Benito, J. (2002). Effects of ability scale purification on the identification of dif. European Journal of Psychological Assessment, 18, 9-15. doi:10.1027//1015-5759.18.1.9
63. Neisser, U., Boodoo, G., Bouchard, T. J., Boykin, A. W., Brody, N., Ceci, S. J.,. Urbina, S. (1996). Intelligence: Knowns and unknowns. American Psychologist, 51, 77-101. doi:10.1037/0003-066X.51.2.77
64. Nichols, J. D. (2003). Prediction indicators for students failing the State of Indiana high school graduation exam. Preventing School Failure, 47, 112-120. doi:10.1080/10459880309604439
65. O'Boyle, E. H., Jr., & McDaniel, M. A. (2009). Criticisms of employment testing: A commentary. In R. P. Phelps (Ed.), Correcting fallacies about educational and psychological testing (pp. 181-197). Washington, DC: American Psychological Association. doi:10.1037/11861-005
66. Olszewski-Kubilius, P., & Lee, S.-Y. (2011). Gender and other group differences in performance on off-level tests: Changes in the 21st century. Gifted Child Quarterly, 55, 54-73. doi:10.1177/0016986210382574
67. Phelps, R. P. (2003). Kill the messenger: The war on standardized testing. New Brunswick, NJ: Transaction Publishers.
68. Phillips, S. E. (2000). GI Forum v. Texas Education Agency: Psychometric evidence. Applied Measurement in Education, 13, 343-385. doi: 10.1207/S15324818AME1304_04
69. Phillips, S. E., & Camara, W. J. (2006). Legal and ethical issues. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 733-755). Westport, CT: Praeger.
70. Popham, W. J. (1997). Consequential validity: Right concern-wrong concept. Educational Measurement: Issues and Practice, 16, 9-13. doi: 10.1111/j.1745-3992.1997.tb00586.x
71. Posselt, J. R., Jaquette, O., Bielby, R., & Bastedo, M. N. (2012). Access without equity: Longitudinal analyses of institutional stratification by race and ethnicity, 1972-2004. American Educational Research Journal, 49, 1074-1111. doi:10.3102/0002831212439456
72. Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552-566. doi: 10.1037/0033-2909.114.3.552
73. Reschly, D. J. (1980). Psychological evidence in the Larry P. opinion: A case of right problem-wrong solution? School Psychology Review, 9, 123-135.
74. Reschly, D. J., & Sabers, D. L. (1979). Analysis of test bias in four groups with the regression definition. Journal of Educational Measurement, 16, 1-9. doi:10.1111/j.1745-3984.1979.tb00080.x
75. Reynolds, C. R. (1980). An examination of bias in a preschool battery across race and sex. Journal of Educational Measurement, 17, 137-146. doi:10.1111/j.1745-3984.1980.tb00822.x
76. Reynolds, C. R. (2000). Why is psychometric research on bias in mental testing so often ignored? Psychology, Public Policy, and Law, 6, 144-150. doi:10.1037/1076-8971.6.1.144
77. Reynolds, C. R., & Lowe, P. A. (2009). The problem of bias in psychological assessment. In C. R. Reynolds & T. B. Gutkin (Eds.), The handbook of school psychology (pp. 332-374). New York, NY: Wiley.
78. Richert, E. S. (2003). Excellence with justice in identification and programming. In N. Colangelo & G. A. Davis (Eds.), Handbook of gifted education (3rd ed., pp. 146-158). Boston, MA: Allyn & Bacon.
79. Roid, G. H. (2003). Stanford-Binet Intelligence Scales, fifth edition, technical manual. Itasca, IL: Riverside Publishing.
80. Rushton, J. P., & Jensen, A. R. (2005). Thirty years of research on race differences in cognitive ability. Psychology, Public Policy, and Law, 11, 235-294. doi:10.1037/1076-8971.11.2.235
81. Salend, S. J., Garrick Duhaney, L. M., & Montgomery, W. (2002). A comprehensive approach to identifying and addressing issues of disproportionate representation. Remedial and Special Education, 23, 289-299. doi:10.1177/07419325020230050401
82. Santor, D. A., Ramsay, J. O., & Zuroff, D. C. (1994). Nonparametric item analyses of the Beck Depression Inventory: Evaluating gender item bias and response option weights. Psychological Assessment, 6, 255-270. doi:10.1037/1040-3590.6.3.255
83. Schafer, W. D. (2000). GI Forum v. Texas Education Agency: Observations for states. Applied Measurement in Education, 13, 411-418. doi: 10.1207/S15324818AME1304_07
84. Schmitt, A. P., & Dorans, N. J. (1990). Differential item functioning for minority examinees on the SAT. Journal of Educational Measurement, 27, 67-81. doi:10.1111/j.1745-3984.1990.tb00735.x
85. Schmitt, N., & Kuljanin, G. (2008). Measurement invariance: Review of practice and implications. Human Resource Management Review, 18, 210-222. doi:10.1016/j.hrmr.2008.03.003
86. Sireci, S. G. (2005). The most frequently unasked questions about testing. In R. P. Phelps (Ed.), Defending standardized testing (pp. 111-121). Mahwah, NJ: Erlbaum.
87. Smith, R. A. (2003). Race, poverty, & special education: Apprenticeships for prison work. Poverty & Race, 12, 1-4.
88. Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370. doi:10.1111/j.1745-3984.1990.tb00754.x
89. Terman, L. M. (1928). The influence of nature and nurture upon intelligence scores: An evaluation of the evidence in Part I of the 1928 Yearbook of the National Society for the Study of Education. Journal of Educational Psychology, 19, 362-373. doi:10.1037/h0071466
90. Ward, C. A. (2000). GI Forum v. Texas Education Agency: Implications for state assessment programs. Applied Measurement in Education, 13, 419-426. doi:10.1207/S15324818AME1304_08_1
91. Warne, R. T. (2011). An investigation of measurement invariance across genders on the Overexcitability Questionnaire-Two. Journal of Advanced Academics, 22, 578-593. doi:10.1177/1932202X11414821
92. Woodford, F. P. (1967). Sounder thinking through clearer writing. Science, 156, 743-745. doi:10.1126/science.156.3776.743
93. Yoakum, C. S., & Yerkes, R. M. (1920). Army mental tests. New York, NY: Henry Holt and Company. doi:10.1037/11054-000
94. Young, J. W., Kane, M., Monfils, L., Li, C., & Ezzo, C. (2013, May). Investigating differential validity and differential prediction using orthogonal regression analysis. Paper presented at annual meeting of the American Educational Research Association, San Francisco, CA.
95. Young, J. W. (2004). Differential validity and prediction: Race and sex differences in college admissions testing. In R. Zwick (Ed.), Rethinking the SAT: The future of standardized testing in university admissions (pp. 289-302). New York, NY: Rouledge-Falmer. doi:10.4324/9780203463932_Differential_Validity_and_Prediction
96. Zwick, R. (2006). Higher education admissions testing. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 647-679). Westport, CT: Praeger.
97. Zwick, R. (2007). College admission testing. Retrieved from http://www.nacacnet.org/research/PublicationsResources/Marketplace/Documents/TestingWhitePaper.pdf