Positive Manifold Limits the Relevance of Content-Matching Strategies for Validating Selection Test Batteries

Journal of Applied Psychology

Volumn 94, Issue 4, 2009, Pages 1018-1031

  Murphy, Kevin R ^a   Dzieweczynski, Jessica L ^a   Zhang, Yang ^a  

^a PENNSYLVANIA STATE UNIVERSITY   (United States)

Author keywords

content validity; personnel selection; positive manifold; test batteries

Indexed keywords

APTITUDE TEST; ARTICLE; HUMAN; PERSONNEL MANAGEMENT; PSYCHOMETRY; PSYCHOMOTOR PERFORMANCE; REPRODUCIBILITY; STATISTICS; WORK;

APTITUDE TESTS; EMPLOYEE PERFORMANCE APPRAISAL; HUMANS; JOB DESCRIPTION; PERSONNEL SELECTION; PSYCHOMETRICS; PSYCHOMOTOR PERFORMANCE; REPRODUCIBILITY OF RESULTS; STATISTICS AS TOPIC;

EID: 67651071280     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0014075     Document Type: Article

References (75)

1. Ackerman P.L., and Humphreys L.G. Individual differences theory in industrial and organizational psychology. In: Dunnette M., and Hough L. (Eds). Handbook of industrial and organizational psychology. 2nd ed. 1 (1990), Consulting Psychologists Press, Palo Alto, CA 223-282
2. Allinger G.M. Do zero correlations really exist among measures of different cognitive abilities?. Educational and Psychological Measurement 48 (1988) 275-280
3. Berry C.M., Sackett P.R., and Landers R.N. Revisiting interview-cognitive ability relationships: Attending to specific range restriction mechanisms in meta-analysis. Personnel Psychology 60 (2007) 837-874
4. Bobko P., Roth P.L., and Potosky D. Derivation and implications of a meta- analytic matrix incorporating cognitive ability, alternative predictors, and job performance. Personnel Psychology 52 (1999) 561-589
5. Brogden H.E. Efficiency of classification as a function of number of jobs, percent rejected and validity and intercorrelation of job performance estimates. Educational and Psychological Measurement 19 (1959) 181-190
6. Brown K.G., Le H., and Schmidt F.L. Specific aptitude theory revisited: Is there incremental validity for training performance?. International Journal of Selection and Assessment 14 (2006) 87-100
7. Carrier M.R., Dalessio A.T., and Brown S.H. Correspondence between estimates of content and criterion-related validity values. Personnel Psychology 43 (1990) 85-100
8. Carroll J.B. Human cognitive abilities: A survey of factor-analytic studies (1993), Cambridge University Press, Cambridge, United Kingdom
9. Cascio W.F. Applied psychology in human resource management. 5th ed. (1998), Prentice-Hall, Upper Saddle River, NJ
10. Cooper-Hakim A., and Viswesvaran C. The construct of work commitment: Testing an integrative framework. Psychological Bulletin 131 (2005) 241-249
11. Cortina J.M., Goldstein N.B., Payne S.C., Davison H.K., and Gilliland S.W. The incremental validity of interview scores over and above cognitive ability and conscientiousness scores. Personnel Psychology 53 (2000) 325-351
12. Cucina J.M., Vasilopoulos N.L., and Sehgal K.G. Personality-based job analysis and the self-serving bias. Journal of Business and Psychology 20 (2005) 275-290
13. Dalessio A.T., and Silverhart T.A. Combining biodata test and interview information: Predicting decisions and performance criteria. Personnel Psychology 47 (1994) 303-315
14. Dawes R.M., and Corrigan B. Linear models in decision making. Psychological Bulletin 81 (1974) 95-106
15. Deller J., Ones D.S., and Viswesvaran C. Personality in personnel selection and assessment [Special issue]. International Journal of Selection and Assessment 15 1-2 (2007)
16. Dudley N.M., Orvis K.A., Lebiecki J.E., and Cortina J.M. A meta-analytic investigation of conscientiousness in the prediction of job performance: Examining the intercorrelations and the incremental validity of narrow traits. Journal of Applied Psychology 91 (2006) 40-57
17. Dye D.A., Reck M., and McDaniel M.A. The validity of job knowledge measures. International Journal of Selection and Assessment 1 (1993) 153-157
18. Earles J.A., and Ree M.J. The predictive validity of the ASVAB for training grades. Educational and Psychological Measurement 52 (1992) 721-725
19. Fairbanks B.A., Welsh J.R., and Sawin L.L. Armed Services Vocational Aptitude Battery (ASVAB): Validity of ASVAB Form 14 for predicting high school course grades (Interim report for period May 1988 to May 1990 AFHRL-TR-90-48) (1990), Air Force Human Resources Laboratory, Brooks Air Force Base, TX
20. Fleishman E.A., and Mumford M.D. Evaluating classifications of job behavior: A construct validation of ability requirement scales. Personnel Psychology 44 (1991) 523-575
21. Goldstein I.L., and Zedeck S. Content validation. In: Barrett R. (Ed). Fair employment strategies in human resource management (1996), Quorum Books, Westport, CT 27-37
22. Goldstein I.L., Zedeck S., and Schneider B. An exploration of the job analysis-content validity process. In: Schmitt N., and Borman W.C. (Eds). Personnel selection in organizations (1993), Jossey-Bass, San Francisco, CA
23. Gorsuch R.L. Factor analysis. 2nd ed. (1983), Erlbaum, Hillsdale, NJ
24. Guion R.M. Content validity-The source of my discontent. Applied Psychological Measurement 1 (1977) 1-10
25. Guion R.M. "Content validity" in moderation. Personal Psychology 31 (1978) 205-213
26. Guion R.M. Scoring content domain samples: The problem of fairness. Journal of Applied Psychology 63 (1978) 499-506
27. Guion R.M. Assessment, measurement, and prediction for personnel selection (1998), Erlbaum, Mahwah, NJ
28. Guttman L., and Levy S. Two structural laws for intelligence tests. Intelligence 15 (1991) 79-103
29. Harmon H. Modern factor analysis. 3rd ed. (1976), University of Chicago Press, Chicago
30. Hartigan J.A., and Wigdor A.K. Fairness in employment testing: Validity generalization, minority issues, and the General Aptitude Test Battery (1989), National Academy Press, Washington, DC
31. Hogan J., and Holland B. Using theory to evaluate personality and job- performance relations: A socioanalytic perspective. Journal of Applied Psychology 88 (2003) 100-112
32. Huffcutt A.I., Roth P.L., and McDaniel M.A. A meta-analytic investigation of cognitive ability employment interview evaluations: Moderating characteristics and implications for incremental validity. Journal of Applied Psychology 81 (1996) 459-473
33. Humphreys L.G. The construct of general intelligence. Intelligence 3 (1979) 105-120
34. Hunter, J. E. (1982, June 7). What is the validity of a content valid test? Paper presented at meeting of International Personnel Management Association, Minneapolis, MN.
35. Hunter J.E. Cognitive ability, cognitive aptitudes, job knowledge and job performance. Journal of Vocational Behavior 29 (1986) 340-362
36. Hunter J.E., and Hirsh H.R. Applications of meta-analysis. In: Cooper C.L., and Robertson I.T. (Eds). International review of industrial and organizational psychology (1987), Wiley, Chichester, United Kingdom 321-357
37. Hunter J.E., and Hunter R.F. Validity and utility of alternative predictors of job performance. Psychological Bulletin 96 (1984) 72-98
38. Jeanneret P.R., and Strong M.H. Linking O*NET job information to job requirement predictors: An O*NET application. Personnel Psychology 56 (2003) 465-492
39. Jenkins M., and Griffith R. Using personality constructs to predict performance: Narrow or broad bandwidth. Journal of Business and Psychology 19 (2004) 255-269
40. Jensen A.R. Bias in mental testing (1980), Free Press, New York
41. Jones G.E., and Ree M.J. Aptitude test score validity: No moderating effect due to job ability requirement differences. Educational and Psychological Measurement 58 (1998) 284-294
42. McDaniel M.A., Morgeson F.P., Finnegan E.B., Campion M.A., and Braverman E.P. Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology 86 (2001) 730-740
43. McHenry J.J., Hough L.M., Toquam J.L., Hanson M.A., and Ashworth S. Project A validity results: The relationship between predictor and criterion domains. Personnel Psychology 43 (1990) 335-354
44. Mount M.K., Witt L.A., and Barrick M.R. Incremental validity of empirically keyed biodata scales over GMA and the five factor personality constructs. Personnel Psychology 53 (2000) 299-323
45. Murphy K.R., Cleveland J.N., and Ross W.T. Evaluating the Marine Corps recruiting effort (Report to the Marine Corps Recruiting Command) (2003), Marine Corps Research University, Quantico, VA
46. Murphy K.R., and Davidshofer C.O. Psychological testing: Principles and applications. 6th ed. (2005), Prentice Hall, Upper Saddle River, NJ
47. Murphy K.R., and Dzieweczynski J.L. Why don't measures of broad dimensions of personality perform better as predictors of job performance?. Human Performance 18 (2005) 343-357
48. Orme D.R., Brehm W., and Ree M.J. Armed Forces Qualification Test as a measure of premorbid intelligence. Military Psychology 13 (2001) 187-197
49. Overstreet Jr. G.G., and Bradley Jr. E.L. Applicability of generic linear scoring models in the U.S. credit-union environment. IMA Journal of Management Mathematics 7 (1996) 291-311
50. Overstreet Jr. G.G., Bradley Jr. E.L., and Kemp R.S. The flat maximum effect and generic linear scoring models: A test. IMA Journal of Management Mathematics 4 (1992) 97-109
51. Pearlman K., Schmidt F.L., and Hunter J.E. Validity generalization results for tests used to predict job proficiency and training success in clerical occupations. Journal of Applied Psychology 65 (1980) 373-406
52. Peterson N.G., Wise L.L., Arabian J., and Hoffman R.G. Synthetic validation and validity generalization when empirical validation is not possible. In: Campbell J., and Knapp D. (Eds). Exploring the upper limits of personnel selection and classification (2001), Erlbaum, Mahwah, NJ 411-451
53. Potosky D., Bobko P., and Roth P.L. Forming composites of cognitive ability and alternate measures to predict job performance and reduce adverse impact: Corrected estimated and realistic expectations. International Journal of Selection and Assessment 13 (2005) 304-315
54. Raymark P.H., Schmit M.J., and Guion R.M. Identifying potentially useful personality constructs for employee selection. Personnel Psychology 50 (1997) 723-736
55. Ree M.J., and Carretta T.R. Expanding the nexus of cognitive and psychomotor abilities. International Journal of Selection and Assessment 3 (1997) 149-158
56. Ree M.J., and Carretta T.R. G2k. Human Performance 15 (2002) 3-23
57. Ree M.J., Carretta T.R., and Earles J.A. In top-down decisions, weighting variables does not matter: A consequence of Wilks' theorem. Organizational Research Methods 1 (1998) 407-420
58. Ree M.J., and Earles J.A. Predicting training success: Not much more than g. Personnel Psychology 44 (1991) 321-332
59. Ree M.J., and Earles J.A. Intelligence is the best predictor of job performance. Current Directions in Psychological Science 1 (1992) 86-89
60. Ree M.J., and Earles J.A. The ubiquitous predictiveness of g. In: Rumsey M.G., Walker C.B., and Harris J.H. (Eds). Personnel selection and classification (1994), Erlbaum, Hillsdale, NJ 127-136
61. Ree M.J., Earles J.A., and Teachout M.S. Predicting job performance: Not much more than g. Journal of Applied Psychology 79 (1994) 518-524
62. Reilly R.R., and Chao G.T. Validity and fairness of some alternate employee selection procedures. Personnel Psychology 35 (1982) 1-67
63. Riggio R.E., and Taylor S.J. Personality and communication skills as predictors of hospice nurse performance. Journal of Business and Psychology 15 (2000) 351-359
64. Roth P.L., Bobko P., and McFarland L.A. A meta-analysis of work sample test validity: Updating and integrating some classic literature. Personnel Psychology 58 (2005) 1009-1038
65. Sackett P.R. Assessment centers and content validity: Some neglected issues. Personnel Psychology 40 (1987) 13-25
66. Schmidt F.L., and Hunter J.E. The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin 124 (1998) 262-274
67. Schmidt F.L., Hunter J.E., and Pearlman K. Task differences as moderators of aptitude test validity in selection: A red herring. Journal of Applied Psychology 66 (1981) 166-185
68. Schmitt N., Gooding R.Z., Noe R.D., and Kirsch M. Metaanalyses of validity studies published between 1964 and 1982 and the investigation of study characteristics. Personnel Psychology 37 (1984) 407-422
69. Schmitt N., Rogers W., Chan D., Sheppard L., and Jennings D. Adverse impact and predictive efficiency of various predictor combinations. Journal of Applied Psychology 82 (1997) 719-730
70. Society for Industrial and Organizational Psychology. Principles for the validation and use of personnel selection procedures. 4th ed. (2003), Author, Bowling Green, OH
71. Sumer H.C., Sumer N., Demirutku K., and Cifci O.S. Using a personality- oriented job analysis to identify attributes to be assessed in officer selection. Military Psychology 13 (2001) 129-146
72. Tett R.P., Jackson D.N., and Rothstein M. Personality measures as predictors of job performance: A meta-analytic review. Personnel Psychology 44 (1991) 703-742
73. Uniform Guidelines on Employee Selection Procedures, 29 C.F.R. 1607 (1978).
74. Viswesvaran C., and Ones D.S. Perspectives on models of job performance. International Journal of Selection and Assessment 8 (2000) 216-226
75. Wainer H. Estimating coefficients in linear models: It don't make no never mind. Psychological Bulletin 83 (1976) 213-217