Cognitive Tests Used in Selection Can Have Content Validity as Well as Criterion Validity: A broader research review and implications for practice

International Journal of Selection and Assessment

Volumn 20, Issue 1, 2012, Pages 1-13

  Schmidt, Frank L ^a  

^a UNIVERSITY OF IOWA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84857334515     PISSN: 0965075X     EISSN: 14682389     Source Type: Journal    
DOI: 10.1111/j.1468-2389.2012.00573.x     Document Type: Review

References (86)

1. American Psychological Association. (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association, American Educational Research Association, National Council on Measurement in Education.
2. Bloom, B. S. (1976). Human characteristics and human learning. New York: McGraw-Hill.
3. Borman, W. C., White, L. A., Pulakos, E. D., & Oppler, S. H. (1991). Models evaluating the effects of ratee ability, knowledge, proficiency, temperament, awards, and problem behavior on supervisory ratings. Journal of Applied Psychology, 76, 863-872.
4. Bownas, D. A., & Heckman, R. W. (1976). Job analysis of the entry level firefighter position. Washington, DC: US Office of Personnel Management, Personnel Research and Development Center.
5. Brannick, M. T., Levine, E. L., & Morgeson, F. P. (2007). Job and work analysis (2nd ed.) Los Angeles, CA: Sage Publications.
6. Breland, H. M. (1977). Multiple choice tests assess writing ability: Findings. Psychometric Monographs, Vol. IV, No. 1. Chicago, IL: University of Chicago Press.
7. Brown, K., Le, H., & Schmidt, F. L. (2006). Specific aptitude theory revisited: Is there incremental validity for training performance? International Journal of Selection and Assessment, 14, 87-100.
8. Campbell, J. P. (1990). An overview of the army selection and classification projects (Project A). Personnel Psychology, 43, 231-239.
9. Carroll, J. B. (1986). A critical synthesis of knowledge about cognitive abilities. Final Technical Report, NSF Grant BNS 8212486, University of North Carolina, Chapel Hill, NC.
10. Carroll, J. B. (1993). Human cognitive abilities. Cambridge: Cambridge University Press.
11. City of Milwaukee. (1978). Documentation for revisions and content validity of the written examination for the position of firefighter. Technical Report, Supplement # 1, Fire and Police Commission, City of Milwaukee, Milwaukee.
12. Cornelius, E. T., Schmidt, F. L., & Carron, T. J. (1984). Job classification approaches and the implementation of validity generalization and meta-analysis results. Personnel Psychology, 37, 247-260.
13. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.
14. Dewey, J. (1896). The reflex arc concept in psychology. Psychological Review, 3, 357-370.
15. Ekstrom, R. B. (1973). Cognitive factors: Some recent literature. Princeton, NJ: Educational Testing Service.
16. Ekstrom, R. B., French, J. W., Harman, H. H., & Dermon, D. (1976). Kit of factor-referenced cognitive tests. Princeton, NJ: Educational Testing Service.
17. Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, & Department of Justice. (1978). Uniform Guidelines on employee selection procedures. Federal Register, 43, 38290-38315.
18. Fleishman, E. A., & Quaintance, M. K. (1984). Taxonomies of human performance. Orlando, FL: Academic Press.
19. French, J. W. (1951). The description of aptitude and achievement factors in terms of rotated factors. Psychometric Monographs, Vol. IV, No. 5. Chicago, IL: University of Chicago Press.
20. French, J. W. (1954). Manual of selected tests for reference aptitude and achievement factors. Princeton, NJ: Educational Testing Service.
21. French, J. W., Ekstrom, R. B., & Price, L. A. (1963). Kit of reference tests for cognitive factors. Princeton, NJ: Educational Testing Service.
22. Gustafsson, J.-E. (1984). A unifying model of the structure of intellectual abilities. Intelligence, 8, 179-203.
23. Gustafsson, J.-E. (2002). Measurement from a hierarchical point of view. In H. I. Braun, D. N. Jackson, & D. E. Wiley (Eds.), The role of constructs in psychological and educational measurement (pp. 72-97). Mahwah, NJ: Erlbaum.
24. Hartigan, J. A., & Wigdor, A. K. (1989). Fairness in employment testing: Validity generalization, minority issues, and the general aptitude test battery. Washington, DC: National Academy of Science.
25. Hirsh, H. R., Northrup, L. C., & Schmidt, F. L. (1986). Validity generalization results for law enforcement occupations. Personnel Psychology, 39, 399-420.
26. Holzinger, K. J. (1935). Introduction to bifactor theory. Preliminary reports on Spearman-Holzinger unitary trait study, Number 5. Chicago: University of Chicago Press.
27. Holzinger, K. J. (1944). A simple method of factor analysis. Psychometrika, 9, 257-262.
28. Hunter, J. E. (1980a). The dimensionality of the General Aptitude Test Battery (GATB) and the dominance of general factors over specific factors in the prediction of job performance. Washington, DC: US Employment Service.
29. Hunter, J. E. (1980b). Test validation for 12, 000 jobs: An application of synthetic validity and validity generalization to the General Aptitude Test Battery (GATB). Washington, DC: US Employment Service.
30. Hunter, J. E. (1983a). A causal analysis of cognitive ability, job knowledge, job performance, and supervisor ratings. In F. Landy, S. Zedeck, & J. Cleveland (Eds.), Performance measurement theory (pp. 257-266). Hillsdale, NJ: Erlbaum.
31. Hunter, J. E. (1983b). Validity generalization of the ASVAB: Preliminary report. Rockville, MD: Research Applications, Inc.
32. Hunter, J. E. (1983c). Validity generalization of the ASVAB: Higher validity for factor analytic composites. Rockville, MD: Research Applications, Inc.
33. Hunter, J. E. (1984a). The prediction of job performance in the civilian sector using the ASVAB. Rockville, MD: Research Applications, Inc.
34. Hunter, J. E. (1984b). The validity of the Armed Services Vocational Aptitude Battery (ASVAB) high school composites. Rockville, MD: Research Applications, Inc.
35. Hunter, J. E. (1986). Cognitive ability, cognitive aptitudes, job knowledge, and job performance. Journal of Vocational Behavior, 29, 340-362.
36. Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72-98.
37. Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks, CA: Sage Publications.
38. Hunter, J. E., & Schmidt, F. L. (1996). Intelligence and job performance: Economic and social implications. Psychology, Public Policy, and Law, 2, 447-472.
39. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.) Thousand Oaks, CA: Sage publications.
40. Hunter, J. E., Crossen, J. J., & Friedman, D. H. (1985). The validity of the Armed Services Vocational Aptitude Battery (ASVAB). Rockville, MD: Research Applications, Inc.
41. Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982). Meta-analysis. Beverly Hills, CA: Sage.
42. Hunter, J. E., Schmidt, F. L., & Le, H. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. Journal of Applied Psychology, 91, 594-612.
43. International Personnel Management Association. (1989). Development of the IPMA entry level firefighter examinations. Alexandria, VA: Morris and McDaniel, Inc.
44. Jensen, A. R. (1984). Test validity: G versus the specificity doctrine. Journal of Social and Biological Structures, 7, 93-118.
45. Le, H., & Schmidt, F. L. (2006). Correcting for indirect range restriction in meta-analysis: Testing a new meta-analytic procedure. Psychological Methods, 11, 416-438.
46. McCormick, E. J. (1979). Job analysis: Methods and applications. New York: AMACOM.
47. McDaniel, M. A. (1985). The evaluation of a causal model of job performance: The interrelationships of general mental ability, job experience, and job performance. Dissertation, George Washington University, Washington, DC.
48. McHenry, J. J., Hough, L. M., Toquam, J. L., Hanson, M. L., & Ashworth, S. (1990). Project a validity results: The Relationship between predictor and criterion domains. Personnel Psychology, 43, 335-354.
49. Merit Employment Assessment Services. (1980). Entry level firefighter test (MEAS 116.1). Washington, DC: Merit Employment Assessment Services.
50. Morris, D. M. (1990). The development and validation of a firefighter selections examination for the city of Denver. Alexandria, VA: Morris and McDaniel, Inc.
51. Murdy, L. B., & Norton, R. P. (1972). Firefighter test validation study: City of Fort Worth fire department. Fort Worth, TX: Texas Christian University.
52. Murphy, K. R. (2009). Content validation is useful for many things but validity is not one of them. Industrial and Organizational Psychology: Perspectives on Science and Practice, 2, 453-464.
53. Murphy, K. R., Dzieweczynski, J. L., & Zhang, Y. (2009). Positive manifold limits the relevance of content-matching strategies for validating selection test batteries. Journal of Applied Psychology, 94, 1018-1031.
54. Northrup, L. (1979). Psychometric support for test item type linkages to six ability constructs measured in the entrance examination for the D.C. Fire Department. Washington, DC: US Office of Personnel Management, Personnel Research and Development Center.
55. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.) New York: McGraw-Hill.
56. Olea, M. M., & Ree, M. J. (1994). Predicting pilot and navigator criteria: Not much more than g. Journal of Applied Psychology, 79, 845-851.
57. Payne, S. S., & Van Rijn, P. (1978). Development of a written test of cognitive abilities for entry into the D.C. Fire Department: The task-ability-test linkage procedure. Washington, DC: US Office of Personnel Management, Personnel Research and Development Center.
58. Pearlman, K. (1979). The validity of tests and item types used to select clerical personnel: A comprehensive summary and evaluation. Washington, DC: US Office of Personnel Management, Personnel Research and Development Center.
59. Pearlman, K. (1980). Job families: A review and discussion of their implications for personnel selection. Psychological Bulletin, 87, 1-28.
60. Pearlman, K., & Sanchez, J. I. (2010). Work analysis. In J. L. Farr & N. T. Tippins (Eds.), Handbook of employee selection (pp. 73-98). New York: Routledge.
61. Ployhart, R. E., Schneider, B., & Schmidt, N. (2006). Staffing organizations: Contemporary practice and theory (3rd ed.) Mahwah, NJ: Erlbaum.
62. Ree, M. H., Earles, J. A., & Teachout, M. (1994). Predicting job performance: Not much more than g. Journal of Applied Psychology, 79, 518-524.
63. Ree, M. J., & Earles, J. A. (1991). Predicting training success: Not much more than g. Personnel Psychology, 44, 321-332.
64. Ree, M. J., & Earles, J. A. (1992). Intelligence is the best predictor of job performance. Current Directions in Psychological Science, 1, 86-89.
65. Russell, C. J., Settoon, R. P., McGrath, R. N., Blanton, A. E., Kidwell, R. E., Lohrke, F. T., etal. (1994). Investigatory characteristics as moderators of personnel selection research: A meta-analysis. Journal of Applied Psychology, 79, 163-170.
66. Sackett, P. R. (2003). The status of validity generalization research: Key issues in drawing inferences from cumulative research findings. In K. R. Murphy (Ed.), Validity generalization: A critical review (pp. 91-114). Mahwah, NJ: Lawrence Erlbaum Associates.
67. Salgado, J. F. (1998). Sample size in validity studies of personnel selection. Journal of Occupational and Organizational Psychology, 71, 161-164.
68. Schmidt, F. L. (2002). The role of general cognitive ability in job performance: Why there cannot be a debate. Human Performance, 15, 187-210.
69. Schmidt, F. L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529-540.
70. Schmidt, F. L., & Hunter, J. E. (1978). Moderator research and the law of small numbers. Personnel Psychology, 31, 215-232.
71. Schmidt, F. L., & Hunter, J. E. (1992). Causal modeling of processes determining job performance. Current Directions in Psychological Science, 1, 89-92.
72. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.
73. Schmidt, F. L., & Hunter, J. E. (2004). General mental ability in the world of work: Occupational attainment and job performance. Journal of Personality and Social Psychology, 86, 162-174.
74. Schmidt, F. L., Hunter, J. E., & Caplan, J. R. (1981). Validity generalization results for two job groups in the petroleum industry. Journal of Applied Psychology, 66, 261-273.
75. Schmidt, F. L., Hunter, J. E., & Outerbridge, A. N. (1986). The impact of job experience and ability on job knowledge, work sample performance, and supervisory ratings of job performance. Journal of Applied Psychology, 71, 432-439.
76. Schmidt, F. L., Hunter, J. E., & Urry, V. E. (1976). Statistical power in criterion-related validation studies. Journal of Applied Psychology, 61, 473-485.
77. Schmidt, F. L., Oh, I.-S., & Le, H. (2006). Increasing the accuracy of corrections for range restriction: Implications for selection procedure validities and other research results. Personnel Psychology, 59, 281-305.
78. Schmidt, F. L., Ones, D., & Hunter, J. E. (1992). Personnel selection. Annual Review of Psychology, 43, 627-670.
79. Schmidt, F. L., Shaffer, J. A., & Oh, I.-S. (2008). Increased accuracy for range restriction corrections: Implications for the role of personality and general mental ability in job and training performance. Personnel Psychology, 61, 827-868.
80. Society for Industrial and Organizational Psychology. (2003). Principles for the validation and use of personnel and use of personnel selection procedures (4th ed.) Bowling Green, OH: The Society for Industrial and Organizational Psychology.
81. Spearman, C. (1904). General intelligence, objectively determined and measured. American Journal of Psychology, 15, 201-293.
82. Spearman, C. (1927). The abilities of man. New York: Macmillan.
83. Thorndike, R. L. (1985). The role of general ability in prediction. Multivariate Behavioral Research, 20, 241-254.
84. Thorndike, R. L. (1986). The role of general ability in prediction. Journal of Vocational Behavior, 29, 332-339.
85. Trattner, M. H. (1988). The validity of aptitude and ability tests used to select professional personnel. Washington, DC: US Office of Personnel Management, Personnel Research and Development Center.
86. Vernon, P. E. (1960). The structure of human abilities (rev. ed.) London: Methuen.