Racial/Ethnic Differences in the Criterion-Related Validity of Cognitive Ability Tests: A Qualitative and Quantitative Review

Journal of Applied Psychology

Volumn 96, Issue 5, 2011, Pages 881-906

  Berry, Christopher M ^a   Clark, Malissa A ^b   McClure, Tara K ^c  

^a TEXAS A AND M UNIVERSITY   (United States)

^b AUBURN UNIVERSITY   (United States)

^c WAYNE STATE UNIVERSITY   (United States)

Author keywords

Cognitive ability tests; Differential validity; Performance; Racial ethnic subgroups

Indexed keywords

APTITUDE TEST; ARTICLE; COGNITION; COMPARATIVE STUDY; ETHNIC GROUP; HUMAN; INSTRUMENTATION; META ANALYSIS; PHYSIOLOGY; PSYCHOLOGICAL ASPECT; PSYCHOMETRY; RACE; REPRODUCIBILITY; STANDARD;

APTITUDE TESTS; COGNITION; CONTINENTAL POPULATION GROUPS; ETHNIC GROUPS; HUMANS; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS;

EID: 80052869529     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0023222     Document Type: Article

References (113)

1. Aguinis H., Culpepper S.A., Pierce C.A. Revival of test bias research in preemployment testing. Journal of Applied Psychology 2010, 95:648-680. 10.1037/a0018714.
2. Aguinis H., Smith M.A. Understanding the impact of test validity and bias on selection errors and adverse impact in human resource selection. Personnel Psychology 2007, 60:165-199. 10.1111/j.1744-6570.2007.00069.x.
3. Academic achievement of White, Black, and Hispanic students in teacher education programs 1992, *, American Association of Colleges for Teacher EducationAmerican Association of Colleges for Teacher Education, Author, Washington, DC, (Report No. 0-89333-094-9).
4. Baggaley A.R. Academic prediction at an Ivy League college, moderated by demographic variables. Measurement and Evaluation in Guidance 1974, 6:232-235.
5. Beaujean A.A., Firmin M.W., Knoop A.J., Michonski J.D., Berry T.P., Lowrie R.E. Validation of the Frey and Detterman (2004) IQ prediction equations using the Reynolds Intellectual Assessment Scales. Personality and Individual Differences 2006, 41:353-357.
6. Berry, C. M. (2007). Toward an understanding of evidence of differential validity of cognitive ability tests for racial/ethnic subgroups. (Unpublished doctoral dissertation). University of Minnesota.
7. Berry, C. M., & Sackett, P. R. (2008a, April). Black-White differences in the properties of grades. Poster session presented at the meeting of the Society for Industrial and Organizational Psychology, San Francisco, CA.
8. Berry, C. M., & Sackett, P. R. (2008b, April). Toward understanding race differences in validity of cognitive ability tests. Poster session presented at the meeting of the Society for Industrial and Organizational Psychology, San Francisco, CA.
9. Boehm V.R. Negro-White differences in validity of employment and training procedures: Summary of research evidence. Journal of Applied Psychology 1972, 56:33-39. 10.1037/h0032130.
10. Breland H.M. Population validity and college entrance measures 1979, Educational Testing Service, Princeton, NJ, (College Board Research and Development Report RDR 78-79 No. 2).
11. Breland H., Griswold P. Group comparisons for basic skills measures 1981, *, College Entrance Examination Board, New York, NY, (College Board Report No. 81-6).
12. Bridgeman B., McCamley-Jenkins L., Ervin N. Predictions of freshman grade-point average from the revised and recentered SAT I: Reasoning test 2000, *, College Entrance Examination Board, New York, NY, (Research Report No. 2000-1).
13. Campbell J.T., Crooks L.A., Mahoney M.H., Rock D.A. An investigation of sources of bias in the prediction of job performance: A six year study 1973, *, Educational Testing Service, Princeton, NJ, (Final Report No. PR 73-37).
14. Carretta T.R. Group differences on U.S. Air Force pilot selection tests. International Journal of Selection and Assessment 1997, 5:115-127. 10.1111/1468-2389.00051.
15. Carretta T.R., Ree M.J. Near identity of cognitive structure in sex and ethnic groups. Personality and Individual Differences 1995, 19:149-155. 10.1016/0191-8869(95)00031-Z.
16. Cleary T.A. Test bias: Prediction of grades of Negro and White students in integrated colleges. Journal of Educational Measurement 1968, 5:115-124. 10.1111/j.1745-3984.1968.tb00613.x.
17. Crawford, P. L., Alferink, D. M., & Spencer, J. L. (1986). Postdictions of college GPAs from ACT composite scores and high school GPAs: Comparisons by race and gender. West Virginia State College (ERIC Document Reproduction Service No. ED 326541).
18. Davis J.A., Kerner S.E. The validity of tests and achievement in high school for predicting initial performance in the public universities of North Carolina with special attention to black students 1971, *, Educational Testing Service, Princeton, NJ, (Publication No. 1973-03-00).
19. Distefano M.K., Pryer M.W., Craig S.H. Predictive validity of general ability tests with Black and White psychiatric attendants. Personnel Psychology 1976, 29:197-204. 10.1111/j.1744-6570.1976.tb00408.x.
20. Dittmar, N. (1977). A comparative investigation of the predictive validity of admissions criteria for Anglos, Blacks, and Mexican Americans. (Unpublished doctoral dissertation). University of Texas at Austin.
21. Dolan C.V. Investigating Spearman's hypothesis by means of multi-group confirmatory factor analysis. Multivariate Behavioral Research 2000, 35:21-50. 10.1207/S15327906MBR3501_2.
22. Domino G., Morales A. Reliability and validity of the D-48 with Mexican American college students. Hispanic Journal of Behavioral Sciences 2000, 22:382-389. 10.1177/0739986300223007.
23. Drasgow F. Intelligence and the workplace. Handbook of psychology: Vol. 12. Industrial and organizational psychology 2003, 107-130. Wiley, Hoboken, NJ. W.C. Borman, D.R. Ilgen, R.J. Klimoski (Eds.).
24. Drasgow F., Nye C.D., Carretta T.R., Ree M.J. Factor structure of the Air Force Officer Qualifying Test Form S: Analysis and comparison with previous forms. Military Psychology 2010, 22:68-85. 10.1080/08995600903249255.
25. Duran R.P. Hispanics' education and background: Predictors of college achievement 1983, *, College Board, New York, NY.
26. Elliott R., Strenta A.C. Effects of improving the reliability of the GPA on prediction generally and on comparative predictions for gender and race particularly. Journal of Educational Measurement 1988, 25:333-347. 10.1111/j.1745-3984.1988.tb00312.x.
27. Farr J.L., O'Leary B.S., Pfeiffer C.M., Goldstein F.L., Bartlett C.J. Ethnic group membership as a moderator in the prediction of job performance: An examination of some less traditional procedures 1971, *, American Institutes for Research, Silver Spring, MD, (Rep. No. 151-277).
28. Fox H., Lefkowitz J. Differential validity: Ethnic group as a moderator in predicting job performance. Personnel Psychology 1974, 27:209-223. 10.1111/j.1744-6570.1974.tb01529.x.
29. Frey M.C., Detterman D.K. Scholastic achievement or g? The relationship between the Scholastic Assessment Test and general cognitive ability. Psychological Science 2004, 15:373-378.
30. Gael S., Grant D.L. Employment test validation for minority and nonminority telephone company service representatives. Journal of Applied Psychology 1972, 56:135-139. 10.1037/h0032659.
31. Gael S., Grant D.L., Ritchie R.J. Employment test validation for minority and nonminority clerks with work sample criteria. Journal of Applied Psychology 1975, 60:420-426. 10.1037/h0076908.
32. Gael S., Grant D.L., Ritchie R.J. Employment test validation for minority and nonminority telephone operators. Journal of Applied Psychology 1975, 60:411-419. 10.1037/h0076909.
33. Gardner D., Deadrick D.L. Underprediction of performance for U.S. minorities using cognitive ability measures. Equal Opportunities International 2008, 27:455-464. 10.1108/02610150810882305.
34. Ghiselli E.E., Campbell J.P., Zedeck S. Measurement theory for the behavioral sciences 1981, Freeman, San Francisco, CA.
35. Goldman R.D., Hewitt B.N. An investigation of test bias for Mexican-American college students. Journal of Educational Measurement 1975, 12:187-196. 10.1111/j.1745-3984.1975.tb01021.x.
36. Goldman R.D., Hewitt B.N. Predicting the success of Black, Chicano, Oriental, and White college students. Journal of Educational Measurement 1976, 13:107-117. 10.1111/j.1745-3984.1976.tb00002.x.
37. Goldman R.D., Richards R. The SAT prediction of grades for Mexican-American versus Anglo-American students at the University of California, Riverside. Journal of Educational Measurement 1974, 11:129-135. 10.1111/j.1745-3984.1974.tb00983.x.
38. Grant D.L., Bray D.W. Validation of employment tests for telephone company installation and repair occupations. Journal of Applied Psychology 1970, 54:7-14. 10.1037/h0028648.
39. Haney R., Michael W.B., Martois J. The prediction of success of three ethnic groups in the academic components of a nursing-training program at a large metropolitan hospital. Educational and Psychological Measurement 1976, 36:421-431. 10.1177/001316447603600222.
40. Hartigan J.A., Wigdor A.K. Differential validity and differential prediction. Fairness in employment testing: Validity generalization, minority issues, and the General Aptitude Test Battery 1989, 172-188. National Academy Press, Washington, DC. J.A. Hartigan, A.K. Wigdor (Eds.).
41. Herrnstein R.J., Murray C. The bell curve 1994, Free Press, New York, NY.
42. Hough L.M., Oswald F.L., Ployhart R.E. Determinants, detection and amelioration of adverse impact in personnel selection procedures: Issues, evidence and lessons learned. International Journal of Selection and Assessment 2001, 9:152-194. 10.1111/1468-2389.00171.
43. Houston W.M., Novick M.R. Race-based differential prediction in Air Force technical training programs. Journal of Educational Measurement 1987, 24:309-320. 10.1111/j.1745-3984.1987.tb00282.x.
44. Humphreys L.G. An analysis and evaluation of test and item bias in the prediction context. Journal of Applied Psychology 1986, 71:327-333. 10.1037/0021-9010.71.2.327.
45. Hunter J.E. Validity generalization for 12,000 jobs: An application of synthetic validity and validity generalization to the General Aptitude Test Battery (GATB) 1980, U.S. Department of Labor, Employment Service, Washington, DC.
46. Hunter J.E., Schmidt F.L. Differential and single-group validity of employment tests by race: A critical analysis of three recent studies. Journal of Applied Psychology 1978, 63:1-11. 10.1037/0021-9010.63.1.1.
47. Hunter J.E., Schmidt F.L. Methods of meta-analysis: Correcting error and bias in research findings 2004, Sage, New York, NY. 2nd ed.
48. Hunter J.E., Schmidt F.L., Hunter R. Differential validity of employment tests by race: A comprehensive review and analysis. Psychological Bulletin 1979, 86:721-735. 10.1037/0033-2909.86.4.721.
49. Hunter J.E., Schmidt F.L., Judiesch M. Individual differences in output variability as a function of job complexity. Journal of Applied Psychology 1990, 75:28-42. 10.1037/0021-9010.75.1.28.
50. Jencks C., Phillips M. The Black-White test score gap 1998, Brookings Institution Press, Washington, DC.
51. Jensen A.R. An examination of culture bias in the Wonderlic Personnel Test. Intelligence 1977, 1:51-64. 10.1016/0160-2896(77)90026-5.
52. Jensen A.R. Bias in mental testing 1980, Free Press, New York, NY.
53. Kallingal A. The prediction of grades for Black and White students at Michigan State University. Journal of Educational Measurement 1971, 8:263-265. 10.1111/j.1745-3984.1971.tb00935.x.
54. Kirkpatrick J.J., Ewen R.B., Barrett R.S., Katzell R.A. Testing and fair employment 1968, *, New York University, New York, NY.
55. Kraiger K., Ford J.K. The relation of job knowledge, job performance, and supervisory ratings as a function of ratee race. Human Performance 1990, 3:269-279. 10.1207/s15327043hup0304_4.
56. Kuncel N.R., Hezlett S.A., Ones D.S. Academic performance, career potential, creativity, and job performance: Can one construct predict them all?. Journal of Personality and Social Psychology 2004, 86:148-161.
57. Lichtman, C. (2008). ACT scores and college grades at a large inner-city university. (Unpublished technical report). Wayne State University.
58. Linn R.L. Single-group validity, differential validity, and differential predictions. Journal of Applied Psychology 1978, 63:507-512. 10.1037/0021-9010.63.4.507.
59. Linn R.L. Ability testing: Individual differences, prediction, and differential prediction. Ability testing: Uses, consequences, and controversies 1982, 335-388. National Academy of Sciences, Washington, DC. A.K. Wigdor, W.R. Garner (Eds.).
60. Lopez F.M. Current problems in test performance of job applicants: I. Personnel Psychology 1966, 19:10-18. 10.1111/j.1744-6570.1966.tb02430.x.
61. Mattern K.D., Patterson B.F., Shaw E.J., Kobrin J.L., Barbuti S.M. Differential validity and prediction of the SAT 2008, *, College Board, New York, NY, (College Board Research Report No. 2008-4).
62. McCornack R.L. Bias in the validity of predicted college grades in four ethnic minority groups. Educational and Psychological Measurement 1983, 43:517-522. 10.1177/001316448304300220.
63. McLaughlin D.H., Rossmeissl P.G., Wise L.L., Brandt D.A., Wang M. Validation of current Armed Services Vocational Aptitude Battery (ASVAB) area composites, based on training and Skill Qualification Test (SQT) information in fiscal year 1981 and 1982 1984, *, Army Research Institute, Alexandria, VA, (Report No. ARI-TR-651, AD-A156 807).
64. Mendoza J.L., Mumford M. Corrections for attenuation and range restriction on the predictor. Journal of Educational Statistics 1987, 12:282-293. 10.2307/1164688.
65. Moffatt, G. K. (1993, February). The validity of the SAT as a predictor of grade point average for nontraditional college students. Paper presented at the meeting of the Eastern Educational Research Association, Clearwater Beach, FL.
66. Morgan R. Analyses of predictive validity within student categorizations. Predicting college grades: An analysis of institutional trends over two decades 1990, 225-238. *, Educational Testing Service, Princeton, NJ. W.W. Willingham, C. Lewis, R. Morgan, L. Ramist (Eds.).
67. Noble J., Crouse J., Shulz M. Differential prediction/impact in course placement for ethnic and gender group 1996, *, ACT, Iowa City, IA, (ACT Research Report No. 96-8).
68. O'Connor E.J., Wexley K.N., Alexander R.A. Single-group validity: Fact or fallacy?. Journal of Applied Psychology 1975, 60:352-355. 10.1037/h0076635.
69. O'Leary B.S., Farr J.L., Bartlett C.J. Ethnic group membership as a moderator of job performance 1970, *, American Institutes for Research, Washington, DC, (Tech. Rep. No. 1).
70. O'Neill K.A., McPeek W.M. Item and test characteristics that are associated with differential item functioning. Differential item functioning 1993, 255-276. Erlbaum, Hillsdale, NJ. P.W. Holland, H. Wainer (Eds.).
71. Oppler S.H., Campbell J.P., Pulakos E.D., Borman W.C. Three approaches to the investigation of subgroup bias in performance measurement: Review, results, and conclusions. Journal of Applied Psychology 1992, 77:201-217. 10.1037/0021-9010.77.2.201.
72. Pandey R.E. The SCAT and race. Psychological Reports 1971, 28:459-462.
73. Pandolfi, V. (1997). Assessment of factor models underlying the WISC-III in White, Black, and Hispanic subgroups of the standardization sample. (Unpublished doctoral dissertation). Hofstra University.
74. Patterson B.F., Mattern K.D., Kobrin J.L. Validity of the SAT for predicting FYGPA: 2007 SAT validity sample http://professionals.collegeboard.com/profdownload/pdf/2007_Replication_Admissions_and_Differential.pdf.
75. Pearson B.Z. Predictive validity of the Scholastic Aptitude Test for Hispanic bilingual students. Hispanic Journal of Behavioral Sciences 1993, 15:342-356.
76. Pennock-Román M. Test validity and language background: A study of Hispanic American students at six universities 1990, *, College Board, New York, NY.
77. Pfeifer C.M., Sedlacek W.E. The validity of academic predictors for Black and White students at a predominantly White university. Journal of Educational Measurement 1971, 8:253-261.
78. Pulakos E.D., White L.A., Oppler S.H., Borman W.C. Examination of race and sex effects on performance ratings. Journal of Applied Psychology 1989, 74:770-780.
79. Ramist L., Lewis C., McCamley L. Implications of using freshman GPA as the criterion for the predictive validity of the SAT. Predicting college grades: An analysis of institutional trends over two decades 1990, 253-288. Educational Testing Service, Princeton, NJ. W.W. Willingham, C. Lewis, R. Morgan, L. Ramist (Eds.).
80. Ree M.J., Carretta T.R. Group differences in aptitude factor structure on the ASVAB. Educational and Psychological Measurement 1995, 55:268-277.
81. Reed, C. L. (2000). An investigation of measurement invariance in the WISC-III: Examining a sample of referred African American and Caucasian students. (Unpublished doctoral dissertation). University of South Florida.
82. Roberts H.E., Skinner J. Gender and racial equity of the Air Force Officer Qualifying Test in officer training school selection decisions. Military Psychology 1996, 8:95-113.
83. Roth P.L., Bevier C.A., Bobko P., Switzer F.S., Tyler P. Ethnic group differences in cognitive ability in employment and educational settings: A meta-analysis. Personnel Psychology 2001, 54:297-330.
84. Roth P.L., Bobko P., Mabon H. Utility analysis: A review and analysis at the turn of the century. Handbook of industrial, work, and organizational psychology 2001, 363-384. Sage, London, England. N. Anderson, D.S. Ones, H.K. Sinangil, C. Viswesveran (Eds.).
85. Rotundo M., Sackett P.R. Effect of rater race on conclusions regarding differential prediction in cognitive ability tests. Journal of Applied Psychology 1999, 84:815-822.
86. Sackett P.R., Borneman M.J., Connelly B.S. High stakes testing in higher education and employment: Appraising the evidence for validity and fairness. American Psychologist 2008, 63:215-227.
87. Sackett P.R., DuBois C.L.Z. Rater-ratee race effects on performance evaluation: Challenging meta-analytic conclusions. Journal of Applied Psychology 1991, 76:873-877.
88. Schmidt F.L. The problem of group differences in ability test scores in employment selection. Journal of Vocational Behavior 1988, 33:272-292.
89. Schmidt F.L., Berner J.G., Hunter J.E. Racial differences in validity of employment tests: Reality or illusion. Journal of Applied Psychology 1973, 58:5-9.
90. Schmidt F.L., Hunter J.E. Employment testing: Old theories and new research findings. American Psychologist 1981, 36:1128-1137.
91. Schmidt F.L., Hunter J.E. Development of a causal model of processes determining job performance. Current Directions in Psychological Science 1992, 1:89-92.
92. Schmidt F.L., Hunter J.E., McKenzie R.C., Muldrow T.W. Impact of valid selection procedures on work-force productivity. Journal of Applied Psychology 1979, 64:609-626.
93. Schmidt F.L., Pearlman K., Hunter J.E. The validity and fairness of employment and educational tests for Hispanic Americans: A review and analysis. Personnel Psychology 1980, 33:705-724.
94. Scott, C. C. (1976). Longer-term predictive validity of college admission tests for Anglo, Black, and Mexican American students. (Unpublished doctoral dissertation). University of New Mexico.
95. Stark S., Chernyshenko O.S., Drasgow F. Examining the effects of differential item (functioning and differential) test functioning on selection decisions: When are statistically significant effects practically important?. Journal of Applied Psychology 2004, 89:497-508.
96. Steele C.M., Aronson J. Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology 1995, 69:797-811.
97. Stricker L.J., Rock D.A., Burton N.W. Sex differences in prediction of college grades from Scholastic Aptitude Test scores. Journal of Educational Psychology 1993, 85:710-718.
98. Sue S., Abe J. Predictors of academic achievement among Asian American and White students 1988, *, College Board, New York, NY, (Research Report No. 88-11).
99. Synk D.J., Swarthout D. Comparison of Black and nonminority validities for the General Aptitude Test battery 1987, *, U.S. Department of Labor, Washington, DC, (Research Report No. 51).
100. Temp G. Validity of the SAT for Blacks and Whites in thirteen integrated institutions. Journal of Educational Measurement 1971, 8:245-251.
101. Tracey T.J., Sedlacek W.E. The relationship of noncognitive variables to academic success: A longitudinal comparison by race. Journal of College Student Personnel 1985, 26:405-410.
102. Development of USTES aptitude test battery for drafter 1974, *, U.S. Department of Labor, Manpower AdministrationU.S. Department of Labor, Manpower Administration, Author, Washington, DC, (Tech. Rep. No. S-266R74).
103. Development of USTES aptitude test battery for airframe-and-powerplant mechanic 1985, *, U.S. Department of Labor, Manpower AdministrationU.S. Department of Labor, Manpower Administration, Author, Washington, DC, (Tech. Rep. No. S-111R84).
104. Valentine L.D. Prediction of Air Force technical training success from ASVAB and educational background 1977, *, Air Force Human Resources Laboratory, Personnel Research Division, Lackland AFB, TX, (Report No. AFHRL-TR-77-18).
105. Whitener E.M. Confusion of confidence intervals and credibility intervals in meta-analysis. Journal of Applied Psychology 1990, 75:315-321.
106. Wicherts J.M., Dolan C.V., Hessen D.J. Stereotype threat and group differences in test performance: A question of measurement invariance. Journal of Personality and Social Psychology 2005, 89:696-716.
107. Wightman L.F. Beyond FYA: Analysis of the utility of LSAT scores and UGPA for predicting academic success in law school 2000, *, Law School Admission Council, Newtown, PA, (LSAC Research Report No. 99-06).
108. Wightman L.F., Muller D.G. An analysis of differential validity and differential prediction for Black, Mexican American, Hispanic, and White law school students 1990, *, Law School Admission Council, Newtown, PA, (LSAC Research Report No. 90-03).
109. Wilbourn J.M., Valentine L.D., Ree M.J. Relationships of the Armed Services Vocational Aptitude Battery (ASVAB) Forms 8, 9, and 10 to Air Force technical school final grades 1984, *, Air Force Human Resources Laboratory, Manpower and Personnel Division, Brooks Air Force Base, TX.
110. Willingham W.W., Pollack J.M., Lewis C. Grades and test scores: Accounting for observed differences 2000, Educational Testing Service, Princeton, NJ, (Educational Testing Service Research Report No. 00-15).
111. Wynne, W. D. (2003). An investigation of ethnic and gender intercept bias in the SAT's prediction of college freshman academic performance. (Unpublished doctoral dissertation). University of Texas at Austin.
112. Young J.W. Differential prediction of college grades by gender and by ethnicity: A replication study. Educational and Psychological Measurement 1994, 54:1022-1029.
113. Young J.W., Kobrin J.L. Differential validity, differential prediction, and college admission testing: A comprehensive review and analysis 2001, College Board, New York, NY, (College Board Research Report No. 2001-6).