Use of situational judgment tests to predict job performance: A clarification of the literature

Journal of Applied Psychology

Volumn 86, Issue 4, 2001, Pages 730-740

  McDaniel, Michael A ^c   Finnegan, Elizabeth Bruhn ^a   Morgeson, Frederick P ^d   Campion, Michael A ^d   Braverman, Eric P ^b  

^a Ameren   (United States)

^b AT AND T LABS RESEARCH   (United States)

^c VIRGINIA COMMONWEALTH UNIVERSITY   (United States)

^d NONE

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CLINICAL TRIAL; FORECASTING; HUMAN; META ANALYSIS; PERSONNEL MANAGEMENT; PREDICTION AND FORECASTING; PROBLEM SOLVING; PROFESSIONAL COMPETENCE; PSYCHOLOGY; PSYCHOMETRY; TASK PERFORMANCE;

EMPLOYEE PERFORMANCE APPRAISAL; FORECASTING; HUMANS; PREDICTIVE VALUE OF TESTS; PROBLEM SOLVING; PROFESSIONAL COMPETENCE; PSYCHOLOGY, INDUSTRIAL; PSYCHOMETRICS; TASK PERFORMANCE AND ANALYSIS;

EID: 85047683400     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/0021-9010.86.4.730     Document Type: Article

References (104)

1. * Anonymous. (1954). Validity information exchange (No. 7-065). Personnel Psychology, 7, 301.
2. * Bass, B. M., Karstendiek, B., McCullough, G., & Pruitt, R. C. (1954). Validity information exchange (No. 7-024). Personnel Psychology, 7, 159-160.
3. * Bruce, M. M. (1953). The prediction of effectiveness as a factory foreman. Psychological Monographs, 67(12, Whole No. 362).
4. * Bruce, M. M. (1965). Examiner's manual: Business Judgment Test. Larchmont, NY: Author.
5. * Bruce, M. M. (1974). Examiner's manual: Supervisory Practices Test (Rev. ed.). Larchmont, NY: Author.
6. * Bruce, M. M., & Friesen, E. P. (1956). Validity information exchange (No. 9-35). Personnel Psychology, 9, 380.
7. * Bruce, M. M., & Learner, D. B. (1958). A supervisory practices test. Personnel Psychology, 11, 207-216.
8. * Campbell, J. P., Dunnette, M. D., Lawler, E. E., & Weick, K. E. (1970). Managerial behavior, performance and effectiveness. New York: McGraw-Hill.
9. Campion, M. A., Campion, J. E., & Hudson, J. P. (1994). Structured interviewing: A note on incremental validity and alternative question types. Journal of Applied Psychology, 79, 998-1002.
10. * Canter, R. R. (1951). A human relations training program. Journal of Applied Psychology, 35, 38-45.
11. Cardall, A. J. (1942). Preliminary manual for the Test of Practical Judgment. Chicago: Science Research.
12. * Carrington, D. H. (1949). Note on the Cardall Practical Judgment Test. Journal of Applied Psychology, 33, 29-30.
13. * Carter, G. C. (1952). Measurement of supervisory ability. Journal of Applied Psychology, 36, 393-395.
14. * Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82, 143-159.
15. * Clevenger, J. P., & Haaland, D. E. (2000, April). The relationship between job knowledge and situational judgment test performance. Paper presented at the 15th Annual Convention of the Society for Industrial and Organizational Psychology, New Orleans, LA.
16. * Clevenger, J. P., Jockin, T., Morris, S., & Anselmi, T. (1999, April). A situational judgment test for engineers: Construct and criterion-related validity of a less adverse alternative. Paper presented at the 14th Annual Convention of the Society for Industrial and Organizational Psychology, Atlanta, GA.
17. * Corts, D. B. (1980). Development and validation of a test for the ranking of applicants for promotion to first-line federal trades and labor supervisory positions (Report No. PRR-80-30). Washington, DC: U.S. Office of Personnel Management, Personnel Research and Development Center.
18. Dalessio, A. T. (1994). Predicting insurance agent turnover using a videobased situational judgment test. Journal of Business and Psychology, 9, 23-32.
19. * Decker, R. L. (1956). An item analysis of How Supervise? using both internal and external criteria. Journal of Applied Psychology, 40, 406-411.
20. * Dicken, C. F., & Black, J. D. (1965). Predictive validity of psychometric evaluations of supervisors. Journal of Applied Psychology, 49, 34-47.
21. * Dulsky, S. G., & Krout, M. H. (1950). Predicting promotion potential on the basis of psychological tests. Personnel Psychology, 3, 345-351.
22. File, Q. W. (1945). The measurement of supervisory quality in industry. Journal of Applied Psychology, 29, 381-387.
23. File, Q. W., & Remmers, H. H. (1948). How Supervise? manual 1948 revision. New York: Psychological Corporation.
24. File, Q. W., & Remmers, H. H. (1971). How Supervise? manual 1971 revision. Cleveland, OH: Psychological Corporation.
25. Gandy, J. A. (1986). Job complexity, aggregated subsamples, and aptitude test validity: Meta-analysis of the GATB data base. Paper prepared for the U.S. Office of Personnel Management, Office of Personnel Research and Development, Washington, DC.
26. Gaugler, B. B., Rosenthal, D. B., Thornton, G. C., & Benson, C. (1987). Meta-analysis of assessment center validity. Journal of Applied Psychology, 72, 493-511.
27. * Gekoski, N., & Schwartz, S. L. (1966). Supervisory Index. Chicago: Science Research Associates.
28. * Greenberg, S. H. (1963). Supervisory Judgment Test manual. Washington, DC: U.S. Civil Service Commission.
29. Gutenberg, R. L., Arvey, R. D., Osburn, H. G., & Jeanneret, P. R. (1983). Moderating effects of decision-making/information-processing demands on test validities. Journal of Applied Psychology, 68, 602-608.
30. * Hanson, M. A. (1995). Development and construct validation of a situational judgment test of supervisory effectiveness for first-line supervisors in the U.S. Army (Doctoral dissertation, University of Minnesota. 1994). Dissertation Abstracts International, 56(2-B), 1138.
31. Hanson, M. A., & Borman, W. C. (1989. April). Development of a situational judgment test to be used as a job performance measure for first-line supervisors in the U.S. Army. Paper presented at the 4th Annual Conference of the Society for Industrial and Organizational Psychology, Boston, MA.
32. * Hill, A. M. (1950). An evaluation of the Cardall Test of Practical Judgment in industrial supervisory selection. Unpublished master's thesis, University of Toronto, Toronto, Ontario, Canada.
33. * Hilton, A. C., Bolin, S. F., Parker, J. W., Taylor. E. K., & Walker, W. B. (1955). The validity of personnel assessments by professional psychologists. Journal of Applied Psychology, 39, 287-293.
34. * Holmes, F. J. (1950). Validity of tests for insurance office personnel. Personnel Psychology, 3, 57-69.
35. Huffcutt, A. I., & Roth, P. L. (1998). Racial group differences in employment interview evaluations. Journal of Applied Psychology, 83, 179-189.
36. Hunter, J. E. (1983). Test validation for 12,000 jobs: An application of job classification and validity generalization analysis to the General Aptitude Test Battery (U.S. Employment Service Test Research Rep., No. 45), Washington, DC: U.S. Department of Labor.
37. Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72-98.
38. Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
39. * Jagmin, N. (1986). Individual differences in perceptual/cognitive constructions of job-relevant situations as a predictor of assessment center success (Doctoral dissertation, University of Maryland, 1985). Dissertation Abstracts International, 47(4-B), 1768-1769.
40. * Johnson, R. J. (1954). Validity information exchange (No. 7-090). Personnel Psychology, 7, 567.
41. Jones, C., & Decotiis, T. A. (1986). Video-assisted selection of hospitality employees. The Cornell H.R.A. Quarterly, 27, 68-73.
42. * Jones, M. W., Dwight, S. A., & Nouryan, T. R. (1999, April). Exploration of the construct validity of a situational judgment test used for managerial assessment. Paper presented at the 14th Annual Convention of the Society for Industrial and Organizational Psychology, Atlanta, GA.
43. * Jurgensen, C. E. (1959). Supervisory Practices Test. In O. K. Burros (Ed.), The fifth mental measurements yearbook (pp. 946-947). Highland Park, NJ: Gryphon Press.
44. * Kirkpatrick, D. L., & Planty, E. (1960). Supervisory Inventory on Human Relations. Chicago: Science Research Associates.
45. Latham, G. P., Saari, L. M., Pursell, E. D., & Campion, M. A. (1980). The situational interview. Journal of Applied Psychology, 65, 422-427.
46. * Leaman, J. A., Vasilopoulos, N. L., & Usala, P. D. (1996, August). Beyond integrity testing: Screening border patrol applicants for counterproductive behaviors. Paper presented at the 104th Annual Convention of the American Psychological Association, Toronto, Ontario, Canada.
47. * Lobenz, R. E., & Morris, S. B. (1999, April). Is tacit knowledge distinct from g, personality, and social knowledge? Paper presented at the 14th Annual Convention of the Society for Industrial and Organizational Psychology, Atlanta, GA.
48. Mandell, M. M. (1953). How Supervise? In O. K. Buros (Ed.), The fourth mental measurements yearbook (pp. 774-775). Highland Park, NJ: Gryphon Press.
49. * McCormick, E. J., & Middaugh, R. W. (1956). The development of a tailor-made scoring key for the How Supervise? test. Personnel Psychology, 9, 27-37.
50. McDaniel, M. A. (1986a). Computer programs for calculating meta-analysis statistics. Educational and Psychological Measurement, 46, 175-177.
51. McDaniel, M. A. (1986b). The evaluation of a causal model of job performance: The interrelationships of general mental ability, job experience, and job performance (Doctoral dissertation, George Washington University, 1986). Dissertation Abstracts International, 47(3-A), 839-840.
52. McDaniel, M. A., Schmidt, F. L., & Hunter, J. E. (1988). A meta-analysis of methods for rating training and experience in personnel selection. Personnel Psychology, 41, 283-314,
53. McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599-616.
54. * Meyer, H. H. (1951). Factors related to success in the human relations aspect of work group leadership. Psychological Monographs, 65(3, Whole No. 320).
55. * Meyer, H. H. (1956). An evaluation of a supervisory selection program. Personnel Psychology, 9, 499-513.
56. * Millard, K. A. (1952). Is How Supervise? an intelligence test? Journal of Applied Psychology, 36, 221-224.
57. Moss, F. A. (1926). Do you know how to get along with people? Why some people get ahead in the world while others do not. Scientific American, 135, 26-27.
58. * Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75, 640-647.
59. * Motowidlo, S. J., & Tippins, N. (1993). Further studies of the low-fidelity simulation in the form of a situational inventory. Journal of Occupational and Organizational Psychology, 66, 337-344.
60. * Mowry, H. W. (1957). A measure of supervisory quality. Journal of Applied Psychology, 41, 405-408.
61. Northrop, L. C. (1989). The psychometric history of selected ability constructs. Washington, DC: J.S. Office of Personnel Management.
62. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
63. Oliverio, M. E. (1959). Supervisory Practices Test. In O. K. Buros (Ed.), The fifth mental measurements yearbook (pp. 954-955). Highland Park, NJ: Gryphon Press.
64. Olson-Buchanan, J. B., Drasgow, F., Moberg, P. J., Mead, A. D., Keenan, P. A., & Donovan, M. A. (1998). Interactive video assessment of conflict resolution skills. Personnel Psychology, 51, 1-24.
65. * Parry, M. E. (1968). Ability of psychologists to estimate validities of personnel tests. Personnel Psychology, 21, 139-147.
66. Pearlman, K. (1979). The validity of tests used to select clerical personnel: A comprehensive summary and evaluation (Test Rep. No. TS-79-1). Washington, DC: Office of Personnel Management, Personnel Research and Development Center.
67. * Pereira, G. M., & Harvey, V. S. (1999, April). Situational judgment tests: Do they measure ability, personality, or both? Paper presented at the 14th Annual Convention of the Society for Industrial and Organizational Psychology, Atlanta, GA.
68. * Phillips, J. F. (1992). Predicting sales skills. Journal of Business and Psychology, 7, 151-160.
69. * Phillips, J. F. (1993). Predicting negotiation skills. Journal of Business and Psychology, 7, 403-411.
70. * Ployhart, R. E., & Ryan, A. M. (2000). A construct-oriented approach for developing situational judgment tests in a service context. Unpublished manuscript.
71. * Pulakos, E. D., & Schmitt, N. (1996). An evaluation of two strategies for reducing adverse impact and their effects on criterion-related validity. Human Performance, 9, 241-258.
72. * Richardson, Bellows, Henry, & Co., Inc. (1949). Test of Supervisor Judgment: Form S. Washington, DC: Richardson, Bellows, and Henry.
73. * Richardson, Bellows, Henry, & Co., Inc. (1963). Test of Supervisory Judgment: Form T. Washington, DC: Richardson, Bellows, and Henry.
74. * Richards'on, Bellows, Henry, & Co., Inc. (1988a). The Manager Profile Record. Minneapolis, MN: National Computer Systems.
75. * Richardson, Bellows, Henry, & Co., Inc. (1988b). The Supervisor Profile Record. Minneapolis, MN: National Computer Systems.
76. Rosen, N. A. (1961). How Supervise? - 1943-1960. Personnel Psychology, 14, 87-99.
77. Rothstein, H. R. (1990). Interrater reliability of job performance ratings: Growth to asymptote level with increasing opportunity to observe. Journal of Applied Psychology, 75, 322-327.
78. Rothstein, H. R., Schmidt, F. L., Erwin, F. W., Owens, W. A., & Sparks, C. P. (1990). Biographical data in employment selection: Can validities be made generalizable? Journal of Applied Psychology, 75, 175-184.
79. * Rushmore, J. T. (1958). A note on the "Test of Practical Judgment." Personnel Psychology, 11-37.
80. * Sartian, A. Q. (1946). Relation between scores on certain standard tests and supervisory success in an aircraft factory. Journal of Applied Psychology, 29, 328-332.
81. * Schippman, J. S., & Prien, E. P. (1985). The Ghiselli Self-Description Inventory: A psychometric appraisal. Psychological Reports, 57, 1171-1177.
82. Schmidt, F. L., Law, K., Hunter, J. E., Rothstein, H. R., Pearlman, K., & McDaniel, M. A. (1993). Refinements in validity generalization methods: Implications for the situational specificity hypothesis. Journal of Applied Psychology, 78, 3-12.
83. * Smith, K. C., & McDaniel, M. A. (1998, April). Criterion and construct validity evidence for a situational judgment measure. Paper presented at the 13th Annual Convention of the Society for Industrial and Organizational Psychology, Dallas, TX.
84. * Spitzer, M. E., & McNamara, W. J. (1964). A managerial selection study. Personnel Psychology, 17, 19-40.
85. * Sternberg, R. J., Wagner, R. K., & Okagaski, L. (1993). Practical intelligence: The nature and role of tacit knowledge in work and at school. In J. M. Puckett & H. W. Reese (Eds.), Mechanisms of everyday cognition (pp. 205-227). Hillsdale, NJ: Erlbaum.
86. Sternberg, R. J., Wagner, R. K., Williams, W. M., & Horvath, J. A. (1995). Testing common sense. American Psychologist, 50, 912-927.
87. * Stevens, M. J., & Campion, M. A. (1999). Staffing work teams: Development and validation of a selection test for teamwork settings. Journal of Management, 25, 207-208.
88. * Super, D. E. (1949). Appraising vocational fitness by means of psychological tests. New York: Harper.
89. Taylor, H. R. (1949a). Social Intelligence Test: George Washington University series. In O. K. Buros (Ed.), The third mental measurements yearbook (pp. 96-97). New Brunswick, NJ: Rutgers University Press.
90. Taylor, H. R. (1949b). Test of Practical Judgment. In O. K. Buros (Ed.), The third mental measurements yearbook (pp. 694-695). New Brunswick, NJ: Rutgers University Press.
91. Tett, R. P., Jackson, D. N., & Rothstein, M. (1991). Personality measures as predictors of job performance: A meta-analytic review. Personnel Psychology, 44, 703-742.
92. Thorndike, R. L. (1941). Social Intelligence Test. In O. K. Buros (Ed.), The 1940 mental measurements yearbook (p. 1253). Highland Park, NJ: Mental Measurements Yearbook.
93. Thorndike, R. L., & Stein, S. (1937). An evaluation of the attempts to measure social intelligence. Psychological Bulletin, 34, 275-285.
94. * Thumin, F. J., & Page, D. S. (1966). A comparative study of two tests of supervisory knowledge. Psychological Reports, 18, 535-538.
95. * Timmreck, C. W. (1981). Moderating effect of tasks on the validity of selection tests (Doctoral dissertation. University of Houston, 1981). Dissertation Abstracts International, 42(3-B), 1221.
96. Wagner, R. K. (1987). Tacit knowledge in everyday intelligent behavior. Journal of Personality and Social Psychology, 52, 1236-1247.
97. * Wagner, R. K., & Sternberg, R. J. (1985). Practical intelligence in real-world pursuits: The role of tacit knowledge. Journal of Personality and Social Psychology, 48, 436-458.
98. * Wagner, R. K., & Sternberg, R. J. (1991). Tacit Knowledge Inventory for Managers: User manual. San Antonio, TX: Psychological Corporation.
99. * Weekley, J. A., & Jones, C. (1999). Further studies of situational tests. Personnel Psychology, 52, 679-700.
100. * Weitz, J., & Nuckols, R. C. (1953). A validation study of How Supervise? Journal of Applied Psychology, 36, 301-303.
101. Wernimont, P., & Campbell, J. P. (1968). Signs, samples, and criteria. Journal of Applied Psychology, 52, 372-376.
102. * Wickert, F. R. (1952). Relation between How Supervise?, intelligence, and education of a group of supervisory candidates in industry. Journal of Applied Psychology, 36, 301-303.
103. * Wiener, D. N. (1961). Evaluation of selection procedures for a management development program. Journal of Counseling Psychology, 8, 121-128.
104. Wiesner, W. H., & Cronshaw, S. F. (1988). The moderating impact of interview format and degree of structure on the validity of the employment interview. Journal of Occupational Psychology, 61, 275-290.