Differential validity for cognitive ability tests in employment and educational settings: Not much more than range restriction?

Journal of Applied Psychology

Volumn 99, Issue 1, 2014, Pages 1-20

  Roth, Philip L ^a   Le, Huy ^b   Oh, In Sue ^c   Van Iddekinge, Chad H ^d   Buster, Maury A ^e   Robbins, Steve B ^f   Campion, Michael A ^g  

^a CLEMSON UNIVERSITY   (United States)

^b UNIVERSITY OF NEVADA   (United States)

^c TEMPLE UNIVERSITY   (United States)

^d FLORIDA STATE UNIVERSITY   (United States)

^e Alabama State Personnel Department   (United States)

^f EDUCATIONAL TESTING SERVICE   (United States)

^g PURDUE UNIVERSITY   (United States)

Author keywords

Adverse impact; Differential validity; Personnel selection

Indexed keywords

APTITUDE TEST; BLACK PERSON; CAUCASIAN; EDUCATION; EMPLOYMENT; HUMAN; PERSONNEL MANAGEMENT; PSYCHOMETRY; REFERENCE VALUE; REPRODUCIBILITY; STANDARDS; STATISTICS AND NUMERICAL DATA;

AFRICAN CONTINENTAL ANCESTRY GROUP; APTITUDE TESTS; EDUCATIONAL MEASUREMENT; EMPLOYMENT; EUROPEAN CONTINENTAL ANCESTRY GROUP; HUMANS; PERSONNEL SELECTION; PSYCHOMETRICS; REFERENCE VALUES; REPRODUCIBILITY OF RESULTS;

EID: 84892853164     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0034377     Document Type: Article

References (60)

1. Aguinis, H., Culpepper, S. A., & Pierce, C. A. (2010). Revival of test bias research in pre-employment testing. Journal of Applied Psychology, 95, 648-680. doi:10.1037/a0018714.
2. Aguinis, H., Dalton, D. R., Bosco, F. A., Pierce, C. A., & Dalton, C. M. (2011). Meta-analytic choices and judgment calls: Implications for theory building and testing, obtained effect sizes, and scholarly impact. Journal of Management, 37, 5-38. doi:10.1177/0149206310377113.
3. Aguinis, H., Pierce, C. A., Bosco, F. A., Dalton, D. R., & Dalton, C. M. (2011). Debunking myths and urban legends about meta-analysis. Organizational Research Methods, 14, 306-331. doi:10.1177/1094428110375720.
4. Aguinis, H., & Smith, M. A. (2007). Understanding the impact of test validity and bias on selection errors and adverse impact. Personnel Psychology, 60, 165-199. doi:10.1111/j.1744-6570.2007.00069.x.
5. Aguinis, H., & Stone-Romero, E. F. (1997). Methodological artifacts in moderated multiple regression and their effects on statistical power. Journal of Applied Psychology, 82, 192-206. doi:10.1037/0021-9010.82.1.192.
6. Arvey, R. D. (1979). Unfair discrimination in the employment interview: Legal and psychological aspects. Psychological Bulletin, 86, 736-765. doi:10.1037/0033-2909.86.4.736.
7. Arvey, R. D., & Faley, R. H. (1988). Fairness in selecting employees (2nd ed.). Reading, MA: Addison Wesley.
8. Bartlett, C. J., Bobko, P., Mosier, S. B., & Hannan, R. L. (1978). Testing for fairness with a moderated multiple regression strategy: An alternative to differential analysis. Personnel Psychology, 31, 233-241. doi: 10.1111/j.1744-6570.1978.tb00442.x.
9. Bartlett, C. J., Bobko, P., & Pine, S. M. (1977). Single-group validity: Fallacy of the facts? Journal of Applied Psychology, 62, 155-157. doi:10.1037/0021-9010.62.2.155.
10. Berry, C. M., Clark, M. A., & McClure, T. K. (2011). Racial/ethnic differences in the criterion-related validity of cognitive ability tests: A qualitative and quantitative review. Journal of Applied Psychology, 96, 881-906. doi:10.1037/a0023222.
11. Berry, C. M., Sackett, P. R., & Landers, R. N. (2007). Revisiting interview-cognitive ability relationships: Attending to specific range restriction mechanisms in meta-analysis. Personnel Psychology, 60, 837-874. doi:10.1111/j.1744-6570.2007.00093.x.
12. Bobko, P., & Bartlett, C. J. (1978). Subgroup validities: Differential definitions and differential prediction. Journal of Applied Psychology, 63, 12-14. doi:10.1037/0021-9010.63.1.12.
13. Bobko, P., Roth, P. L., & Bobko, C. (2001). Correcting the effect size of d for range restriction and unreliability. Organizational Research Methods, 4, 46-61. doi:10.1177/109442810141003.
14. Bobko, P., & Schemmer, M. (1980). Note on standardized regression estimates. Psychological Bulletin, 88, 233-236. doi:10.1037/0033-2909.88.1.233.
15. Boehm, V. R. (1977). Differential prediction: A methodological artifact? Journal of Applied Psychology, 62, 146-154. doi:10.1037/0021-9010.62.2.146.
16. Boehm, V. R. (1978). Populations, preselection, and practicalities: A reply to Hunter and Schmidt. Journal of Applied Psychology, 63, 15-18. doi:10.1037/0021-9010.63.1.15.
17. De Corte, W. (1999). Weighing job performance predictors to both maximize the quality of the selected workforce and control the level of adverse impact. Journal of Applied Psychology, 84, 695-702. doi: 10.1037/0021-9010.84.5.695.
18. De Corte, W., Lievens, F., & Sackett, P. R. (2006). Predicting adverse impact and mean criterion performance in multistage selection. Journal of Applied Psychology, 91, 523-537. doi:10.1037/0021-9010.91.3.523.
19. De Corte, W., Lievens, F., & Sackett, P. R. (2007). Combining predictors to achieve optimal trade-offs between selection quality and adverse impact. Journal of Applied Psychology, 92, 1380-1393. doi:10.1037/0021-9010.92.5.1380.
20. Finch, D. M., Edwards, B. D., & Wallace, C. J. (2009). Multistage selection strategies: Simulating the effects on adverse impact and expected performance for various predictor combinations. Journal of Applied Psychology, 94, 318-340.
21. Fincher, C. (1975). Differential validity and test bias. Personnel Psychology, 28, 481-500. doi:10.1111/j.1744-6570.1975.tb01387.x.
22. Fox, H., & Lefkowitz, J. (1974). Differential validity: Ethnic group as a moderator in predicting job performance. Personnel Psychology, 27, 209-223. doi:10.1111/j.1744-6570.1974.tb01529.x.
23. Gardner, D. G., & Deadrick, D. L. (2012). Moderation of selection procedure validity by employee race. Journal of Managerial Psychology, 27, 365-382. doi:10.1108/02683941211220180.
24. Helms, J. E. (2012). A legacy of eugenics underlies racial-group comparisons in intelligence testing. Industrial and Organizational Psychology, 5, 176-179. doi:10.1111/j.1754-9434.2012.01426.x.
25. Hotelling, H. (1953). New light on the correlation coefficient and its transforms. Journal of the Royal Statistical Society, Series B, 15, 193-225.
26. Hunter, J. E. (1986). Cognitive ability, cognitive aptitudes, job knowledge, and job performance. Journal of Vocational Behavior, 29, 340-362. doi:10.1016/0001-8791(86)90013-8.
27. Hunter, J. E., & Schmidt, F. L. (1978). Differential and single-group validity of employment tests by race: A critical analysis of three recent studies. Journal of Applied Psychology, 63, 1-11. doi:10.1037/0021-9010.63.1.1.
28. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting for error and bias in research findings (2nd ed). Newbury Park, NJ: Sage.
29. Hunter, J. E., Schmidt, F. L., & Hunter, R. (1979). Differential validity of employment tests by race: A comprehensive review and analysis. Psychological Bulletin, 86, 721-735. doi:10.1037/0033-2909.86.4.721.
30. Katzell, R. A., & Dyer, F. J. (1977). Differential validity revived. Journal of Applied Psychology, 62, 137-145. doi:10.1037/0021-9010.62.2.137.
31. Katzell, R. A., & Dyer, F. J. (1978). On differential validity and bias. Journal of Applied Psychology, 63, 19-21. doi:10.1037/0021-9010.63.1.19.
32. Kirchner, W. K. (1975). Some questions about "Differential validity: Ethnic group as a moderator in predicting job performance". Personnel Psychology, 28, 341-343. doi:10.1111/j.1744-6570.1975.tb01541.x.
33. Lawshe, C. H. (1987). Adverse impact: Is it a viable concept? Professional Psychology: Research and Practice, 18, 492-497. doi:10.1037/0735-7028.18.5.492.
34. Lefkowitz, J., & Fox, H. (1975). Some answers to "Some questions about 'Differential validity: Ethnic group as a moderator in predicting job performance.'" Personnel Psychology, 28, 345-349. doi:10.1111/j.1744-6570.1975.tb01542.x.
35. Linn, R. L. (1978). Single-group validity, differential validity, and differential prediction. Journal of Applied Psychology, 63, 507-512. doi: 10.1037/0021-9010.63.4.507.
36. Mattern, K. D., & Patterson, B. F. (2013). Test of slope and intercept bias in college admissions: A response to Aguinis, Culpepper, and Pierce (2010). Journal of Applied Psychology, 98, 134-147. doi:10.1037/a0030610.
37. McDaniel, M. A., Rothstein, H. R., & Whetzel, D. L. (2006). Publication bias: A case study of four test vendors. Personnel Psychology, 59, 927-953. doi:10.1111/j.1744-6570.2006.00059.x.
38. Ployhart, R. E., & Holtz, B. C. (2008). The diversity-validity dilemma: Strategies for reducing racioethnic and sex subgroup differences and adverse impact in selection. Personnel Psychology, 61, 153-172. doi: 10.1111/j.1744-6570.2008.00109.x.
39. Potosky, D. P., Bobko, P., & Roth, P. L. (2005). Forming composites of cognitive ability and alternative measures to predict job performance and reduce adverse impact: Corrected estimates and realistic expectations. International Journal of Selection and Assessment, 13, 304-315. doi: 10.1111/j.1468-2389.2005.00327.x.
40. Reilly, R. R., & Warech, M. A. (1993). The validity and fairness of alternatives to cognitive ability tests. In L. Wing & B. Gifford (Eds.), Policy issues in employment testing (pp. 131-224). Boston, MA: Kluwer.
41. Roth, P. L., Bevier, C. A., Bobko, P., Switzer, F. S., III, & Tyler, P. (2001). Ethnic group differences in cognitive ability in employment and educational settings: A meta-analysis. Personnel Psychology, 54, 297-330. doi:10.1111/j.1744-6570.2001.tb00094.x.
42. Roth, P. L., Switzer, F. S., Van Iddekinge, C. H., & Oh, I.-S. (2011). Toward better meta-analytic matrices: How input values can affect research conclusions in human resource management simulations. Personnel Psychology, 64, 899-935. doi:10.1111/j.1744-6570.2011.01231.x.
43. Sackett, P. R., De Corte, W., & Lievens, F. (2010). Decision aids for addressing the validity-adverse impact trade-off. In J. Outtz (Ed.), Adverse impact: Implications for organizational staffing and high stakes selection (pp. 453-472). New York, NY: Routledge.
44. Sackett, P. R., & Roth, L. (1996). Multi-stage selection strategies: A Monte Carlo investigation of effects on performance and minority hiring. Personnel Psychology, 49, 549-572. doi:10.1111/j.1744-6570.1996.tb01584.x.
45. Sackett, P. R., Schmitt, N., Ellingson, J. E., & Kabin, M. E. (2001). High-stakes testing in employment, credentialing, and higher education: Prospects in a post-affirmative-action world. American Psychologist, 56, 302-318. doi:10.1037/0003-066X.56.4.302.
46. Sackett, P. R., & Shen, W. (2010). Subgroup differences on cognitive tests in contexts other than personnel selection. In J. Outtz (Ed.), Adverse impact: Implications for organizational staffing and high stakes selection (pp. 323-346). New York, NY: Routledge.
47. Sackett, P. R., & Yang, H. (2000). Correction for range restriction: An expanded typology. Journal of Applied Psychology, 85, 112-118. doi: 10.1037/0021-9010.85.1.112.
48. Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., & de Fruyt, F. (2003). International validity generalization of GMA and cognitive abilities: A European community meta-analysis. Personnel Psychology, 56, 573-605. doi:10.1111/j.1744-6570.2003.tb00751.x.
49. Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., de Fruyt, F., & Rolland, J. P. (2003). A meta-analytic study of general mental ability validity for different occupations in the European Community. Journal of Applied Psychology, 88, 1068-1081. doi:10.1037/0021-9010.88.6.1068.
50. Schmidt, F. L., Berner, J. G., & Hunter, J. E. (1973). Racial differences in validity of employment tests: Reality or illusion? Journal of Applied Psychology, 58, 5-9. doi:10.1037/h0035408.
51. Schmidt, F. L., & Hunter, J. E. (1980). The future of criterion-related validity. Personnel Psychology, 33, 41-60. doi:10.1111/j.1744-6570.1980.tb02163x.
52. Schmidt, F. L., & Hunter, J. E. (1981). Employment testing: Old theories and new research findings. American Psychologist, 36, 1128-1137. doi:10.1037/0003-066X.36.10.1128.
53. Schmidt, F. L., Oh, I.-S., & Le, H. (2006). Increasing the accuracy of corrections for range restriction: Implications for selection procedure validities and other research results. Personnel Psychology, 59, 281-305.
54. Schmidt, F. L., Shaffer, J. A., & Oh, I.-S. (2008). Increased accuracy for range restriction corrections: Implications for the role of personality and general mental ability in job and training performance. Personnel Psychology, 61, 827-868. doi:10.1111/j.1744-6570.2008.00132.x.
55. Schmitt, N., Rogers, W., Chan, D., Sheppard, L., & Jennings, D. (1997). Adverse impact and predictive efficiency of various predictor combinations. Journal of Applied Psychology, 82, 719-730. doi:10.1037/0021-9010.82.5.719.
56. Society for Industrial and Organizational Psychology. (2003). Principles for the validation and use of personnel selection procedures. Bowling Green, KY: Author.
57. Synk, D. J., & Swarthout, D. (1987). Comparison of Black and nonminority validities for the General Aptitude Test battery (Research Report No. 51). Washington, DC: U.S. Department of Labor.
58. Tonowski, R. F. (2011). The Uniform Guidelines and personnel selection: Identify and fix the right problem. Industrial and Organizational Psychology, 4, 521-525. doi:10.1111/j.1754-9434.2011.01384.x.
59. Whetzel, D., McDaniel, M., & Nguyen, N. (2008). Subgroup differences in situational judgment test performance: A meta-analysis. Human Performance, 21, 291-308.
60. Young, J. W. (1994). Differential prediction of college grades by gender and by ethnicity: A replication study. Educational and Psychological Measurement, 54, 1022-1029. doi:10.1177/0013164494054004019.