Updating the Trainability Tests Literature on Black-White Subgroup Differences and Reconsidering Criterion-Related Validity

Journal of Applied Psychology

Volumn 96, Issue 1, 2011, Pages 34-45

  Roth, Philip L ^a   Buster, Maury A ^b   Bobko, Philip ^c  

^a CLEMSON UNIVERSITY   (United States)

^b Alabama State Personnel Department   (United States)

^c GETTYSBURG COLLEGE   (United States)

Author keywords

Ethnic group differences; Standardized ethnic group differences; Trainability; Trainability tests

Indexed keywords

APTITUDE TEST; ARTICLE; CAUCASIAN; ENGINEERING; ETHNIC GROUP; FEMALE; HUMAN; MALE; MANPOWER; METHODOLOGY; NEGRO; PERSONNEL MANAGEMENT; PSYCHOLOGICAL ASPECT; REPRODUCIBILITY; STANDARD; VIDEORECORDING;

AFRICAN CONTINENTAL ANCESTRY GROUP; APTITUDE TESTS; ENGINEERING; ETHNIC GROUPS; EUROPEAN CONTINENTAL ANCESTRY GROUP; FEMALE; HUMANS; MALE; PERSONNEL SELECTION; REPRODUCIBILITY OF RESULTS; VIDEO RECORDING;

EID: 79251484980     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0020923     Document Type: Article

References (62)

1. Aramburu-Zabala L., Casals M. Prediction of training performance for diesel mechanics. Swiss Journal of Psychology 2003, 62:233-240. doi: 10.1024/1421-0185.62.4.233.
2. Aramburu-Zabala Higuera L., Casals Riera M. Validation of a trainability test for young apprentices. European Psychologist 2004, 9:56-63. doi: 10.1027/1016-9040.9.1.56.
3. Arthur W., Villado A.J. The importance of distinguishing between constructs and methods when comparing predictors in personnel selection and research in practice. Journal of Applied Psychology 2008, 93:435-442. doi: 10.1037/0021-9010.93.2.435.
4. Bobko P., Roth P.L., Potosky D. Derivation and implications of a meta-analytic matrix incorporating cognitive ability, alternative predictors and job performance. Personnel Psychology 1999, 52:561-589. doi: 10.1111/j.1744-6570.1999.tb00172.x.
5. Borman W.C., White L.A., Pulakos E.D., Oppler S.H. Models of supervisory job performance ratings. Journal of Applied Psychology 1991, 76:863-872. doi: 10.1037/0021-9010.76.6.863.
6. Callinan M., Robertson I.T. Work sample testing. International Journal of Selection and Assessment 2000, 8:248-260. doi: 10.1111/1468-2389.00154.
7. Campbell J.P., Gasser M.B., Oswald R.L. The substantive nature of job performance variability. Individual differences and behavior in organizations 1996, 258-299. Jossey-Bass, San Francisco, CA. K. Murphy (Ed.).
8. Campbell J.P., McCloy R.A., Oppler S.H., Sager C.E. A theory of performance. Personnel selection in organizations 1993, 35-70. Jossey-Bass, San Francisco, CA. N. Schmitt, W. Borman (Eds.).
9. Cascio W.F. Managing human resources: Productivity, quality of work life, profits 2006, McGraw-Hill, Boston, MA. 7th ed.
10. Cascio W.F., Aguinis H. Research in industrial and organizational psychology from 1963 to 2007: Changes, choices, and trends. Journal of Applied Psychology 2008, 93:1062-1081. doi: 10.1037/0021-9010.93.5.1062.
11. Cascio W.F., Phillips N. Performance testing: A rose among thorns?. Personnel Psychology 1979, 32:751-766. doi: 10.1111/j.1744-6570.1979.tb02345.x.
12. Ceci S.J., Papierno P.B. The rhetoric and reality of gap closing: When the " have-nots" gain but the " haves" gain even more. American Psychologist 2005, 60:149-160. doi: 10.1037/0003-066X.60.2.149.
13. Edwards B.D., Arthur W. An examination of factors contributing to a reduction in subgroup differences on a constructed-response and paper-and-pencil test of scholastic achievement. Journal of Applied Psychology 2007, 92:794-801. doi: 10.1037/0021-9010.92.3.794.
14. Goldstein H.W., Yusko K.P., Braverman E.P., Smith D.B., Chung B. The role of cognitive ability in the subgroup differences and incremental validity of assessment center exercises. Personnel Psychology 1998, 51:357-374. doi: 10.1111/j.1744-6570.1998.tb00729.x.
15. Goldstein H.W., Yusko K.P., Nicolopoulos V. Exploring black-white subgroup differences of managerial competencies. Personnel Psychology 2001, 54:783-807. doi: 10.1111/j.1744-6570.2001.tb00232.x.
16. Grissom R.J., Kim J.J. Effect sizes for research: A broad practical approach 2005, Erlbaum, Mahwah, NJ.
17. Guion R.M. Assessment, measurement, and prediction for personnel decisions 1998, Erlbaum, Mahwah, NJ.
18. Harris, P. A., & MacLane, C. N. (1988, August). The Miniature Training and Evaluation Test. Paper presented at the meeting of the American Psychological Association, Atlanta, GA.
19. Hough L.M., Oswald F.L., Ployhart R.E. Determinants, detection and amelioration of adverse impact in personnel selection procedures: Issues, evidence and lessons learned. International Journal of Selection and Assessment 2001, 9:152-194. doi: 10.1111/1468-2389.00171.
20. Howard A. Work samples and simulations in competency evaluation. Professional Psychology: Research and Practice 1983, 14:780-796. doi: 10.1037/0735-7028.14.6.780.
21. Huffcutt A.I., Arthur W. Hunter and Hunter (1984) revisited: Interview validity for entry-level jobs. Journal of Applied Psychology 1994, 79:184-190. doi: 10.1037/0021-9010.79.2.184.
22. Hunter J.E. A causal analysis of cognitive ability, job knowledge, and job performance, and supervisory ratings. Performance measurement and theory 1983, 257-266. Erlbaum, Hillsdale, NJ. F. Landy, S. Zedeck, J. Cleveland (Eds.).
23. Hunter J.E., Hunter R.F. Validity and utility of alternative predictors of job performance. Psychological Bulletin 1984, 96:72-98. doi: 10.1037/0033-2909.96.1.72.
24. Hunter J.E., Schmidt F.L. Methods of meta-analysis: Correcting for error and bias in research findings 2004, Sage, Newbury Park, NJ. 2nd ed.
25. Jones R.G., Sanchez J.I., Parameswaran G., Phelps J., Shoptaugh C., Williams M., White S. Selection or training? A two-fold test of the validity of job-analytic ratings of trainability. Journal of Business and Psychology 2001, 15:363-389. doi: 10.1023/A:1007804815480.
26. Kanfer R., Ackerman P.L. Motivation and cognitive abilities: An integrative/aptitude-treatment interaction approach to skill acquisition. Journal of Applied Psychology 1989, 74:657-690. doi: 10.1037/0021-9010.74.4.657.
27. King R.E., Manning C.A., Drechsler G.K. Operational use of the Air Traffic Selection and Training Battery. International Journal of Applied Aviation Studies 2006, 6:207-217.
28. LaHuis D.M., MacLane C.N., Schlessman B.R. Do applicant's perceptions matter? Investigating reapplication behavior using fairness theory. International Journal of Selection and Assessment 2007, 15:383-393. doi: 10.1111/j.1468-2389.2007.00397.x.
29. Law K.S., Schmidt F.L., Hunter J.E. A test of two refinements in procedures for meta-analysis. Journal of Applied Psychology 1994, 79:978-986. doi: 10.1037/0021-9010.79.6.978.
30. Lubinski D. Applied individual differences research and its quantitative methods. Psychology, Public Policy, and Law 1996, 2:187-203. doi: 10.1037/1076-8971.2.2.187.
31. Lubinski D. Scientific and social significance of assessing individual differences: " Sinking shafts at a few critical points." Annual Review of Psychology 2000, 51:405-444. doi: 10.1146/annurev.psych.51.1.405.
32. Lubinski D. Introduction to the special section on cognitive abilities: 100 years after Spearman's (1904) general intelligence, objectively determined and measured. Journal of Personality and Social Psychology 2004, 86:96-111. doi: 10.1037/0022-3514.86.1.96.
33. Lubinski D., Dawis R.V. Aptitudes, skills, and proficiencies. Handbook of industrial and organizational psychology 1992, 3:1-59. Consulting Psychologists Press, Palo Alto, CA. 2nd ed. M.D. Dunnette, L.M. Hough (Eds.).
34. McCloy R.A., Campbell J.P., Cudeck R. A confirmatory test of a model of performance determinants. Journal of Applied Psychology 1994, 79:493-505. doi: 10.1037/0021-9010.79.4.493.
35. Murphy K.R., Myors B. Statistical power analysis: A simple and general model for traditional and modern hypothesis tests 2004, Erlbaum, Mahwah, NJ. 2nd ed.
36. A final report on the Miniature Training and Evaluation Test 1987, Office of Personnel Management, Office of Personnel Management, Office of Examination Development, Washington, DC.
37. O'Leary L.R.F. Fair employment, sound psychometric practice, and reality: A dilemma and a partial solution. American Psychologist 1973, 28:147-150. doi: 10.1037/h0034246.
38. Ostroff C., Harrison D.A. Meta-analysis, level of analysis, and best estimates of population correlations: Cautions for interpreting meta-analytic results in organizational behavior. Journal of Applied Psychology 1999, 84:260-270. doi: 10.1037/0021-9010.84.2.260.
39. Papierno P.B., Ceci S.J. Beyond the American context. American Psychologist 2005, 60:1042-1043. doi: 10.1037/0003-066X.60.9.1042.
40. Ployhart R.E., Holtz B.C. The diversity-validity dilemma: Strategies for reducing racioethnic and sex subgroup differences and adverse impact in selection. Personnel Psychology 2008, 61:153-172. doi: 10.1111/j.1744-6570.2008.00109.x.
41. Ployhart R.E., Schneider B., Schmitt N. Staffing organizations: Contemporary practice and research 2006, Erlbaum, Mahwah, NJ.
42. Pulakos E.D. Selection assessment methods: A guide to implementing formal assessments to build a high-quality workforce 2005, Society for Human Resource Management, Alexandria, VA.
43. Reilly R.R., Israelski E.W. Development and validation of minicourses in the telecommunication industry. Journal of Applied Psychology 1988, 73:721-726. doi: 10.1037/0021-9010.73.4.721.
44. Reilly R.R., Warech M.A. The validity and fairness of alternatives to cognitive ability tests. Policy issues in employment testing 1993, 131-224. Kluwer, Boston, MA. L. Wing, B. Gifford (Eds.).
45. Robertson I.T., Downs S. Learning and the prediction of performance: Development of trainability testing in the United Kingdom. Journal of Applied Psychology 1979, 64:42-50. doi: 10.1037/0021-9010.64.1.42.
46. Robertson I.T., Downs S. Work sample tests of trainability: A meta-analysis. Journal of Applied Psychology 1989, 74:402-410. doi: 10.1037/0021-9010.74.3.402.
47. Robertson I.T., Kandola R.S. Work sample tests: Validity, adverse impact, and applicant reaction. Journal of Occupational Psychology 1982, 55:171-183.
48. Roth P.L., Bevier C.A., Bobko P., Switzer F.S., Tyler P. Ethnic group differences in cognitive ability in employment and educational settings: A meta-analysis. Personnel Psychology 2001, 54:297-330. doi: 10.1111/j.1744-6570.2001.tb00094.x.
49. Roth P.L., Bobko P., McFarland L.A., Buster M.A. Work sample test ethnic group differences in personnel selection: A meta-analysis of overall and exercise scores. Personnel Psychology 2008, 61:637-662. doi: 10.1111/j.1744-6570.2008.00125.x.
50. Sackett P.R. Personnel selection. Encyclopedia of psychology 2000, 6:152-156. Oxford University Press, New York, NY. A.E. Kazdin (Ed.).
51. Sackett P.R., Ellingson J.E. The effects of forming multi-predictor composites on group differences and adverse impact. Personnel Psychology 1997, 50:707-721. doi: 10.1111/j.1744-6570.1997.tb00711.x.
52. Sackett P.R., Schmitt N., Ellingson J.E., Kabin M.E. High-stakes testing in employment, credentialing, and higher education: Prospects in a post-affirmative-action world. American Psychologist 2001, 56:302-318. doi: 10.1037/0003-066X.56.4.302.
53. Sanchez J.I., Levine E.L. The analysis of work in the 20th and 21st centuries. Handbook of industrial, work and organizational psychology: Volume 1. Personnel psychology 2002, 71-89. Sage, Thousand Oaks, CA. N. Anderson, D. Ones, H. Sinangil, C. Viswesvaran (Eds.).
54. Schmidt F.L., Greenthal A.L., Hunter J.E., Berner J.G., Seaton F.W. Job sample vs. paper-and-pencil trades and technical tests: Adverse impact and examinee attitudes. Personnel Psychology 1977, 30:187-197. doi: 10.1111/j.1744-6570.1977.tb02088.x.
55. Schmidt F.L., Hunter J.E., Outerbridge A.N. Impact of job experience and ability on job knowledge, work sample performance, and supervisory ratings of job performance. Journal of Applied Psychology 1986, 71:432-439. doi: 10.1037/0021-9010.71.3.432.
56. Schmidt F.L., Hunter J.E., Outerbridge A.N., Goff S. Joint relation of experience and ability with job performance: A test of three hypotheses. Journal of Applied Psychology 1988, 73:46-57. doi: 10.1037/0021-9010.73.1.46.
57. Schmitt N., Clause C.S., Pulakos E.D. Subgroup differences associated with different measures of some common job-relevant constructs. International review of industrial and organizational psychology 1996, 11:115-139. Wiley, New York, NY. C.L. Cooper, I.T. Robertson (Eds.).
58. Schmitt N., Quinn A. Reductions in measured subgroup mean differences: What is possible?. Adverse impact: Implications for organizational staffing and high-stakes selection 2010, 425-451. Taylor & Francis, New York, NY. J. Outtz (Ed.).
59. Siegel A.I. The miniature job training and evaluation approach: Additional findings. Personnel Psychology 1983, 36:41-56. doi: 10.1111/j.1744-6570.1983.tb00501.x.
60. Siegel A.I., Bergman B.A. A job learning approach to performance prediction. Personnel Psychology 1975, 28:325-339. doi: 10.1111/j.1744-6570.1975.tb01540.x.
61. Synk D.J., Swarthout D. Comparison of Black and nonminority validities for the General Aptitude Test Battery 1987, U.S. Department of Labor, Division of Planning and Operations, Employment and Training Administration, Washington, DC.
62. Whetzel D.L., McDaniel M.A., Nguyen N.T. Subgroup differences in situational judgment test performance: A meta-analysis. Human Performance 2008, 21:291-309. doi: 10.1080/08959280802137820.