Good News: Work Sample Administrators' Global Performance Judgments are (about) as Valid as We've Suspected

Human Performance

Volumn 13, Issue 3, 2000, Pages 253-277

  Lance, Charles E ^a   Johnson, C Douglas ^b   Douthitt, Shane S ^b   Bennett Jr , Winston ^b   Harville, Donald L ^b  

^a University of Georgia ^*  (United States)

^b NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0034559304     PISSN: 08959285     EISSN: None     Source Type: Journal    
DOI: 10.1207/S15327043HUP1303_2     Document Type: Article

References (58)

1. Asher, J. T., & Sciarrino, J. A. (1974). Realistic work sample tests: A review. Personnel Psychology, 27, 519-533.
2. Borman, W. C. (1991). Job behavior, performance, and effectiveness. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 2, pp. 271-326). Palo Alto, CA: Consulting Psychologists Press.
3. Borman, W. C., & Hallam, G. L. (1991). Observation accuracy for assessors of work-sample performance: Consistency across task and individual-difference correlates. Journal of Applied Psychology, 76, 11-18.
4. Borman, W. C., & Motowidlo, S. J. (1993). Expanding the criterion domain to include elements of contextual performance. In N. Schmitt & W. C. Borman (Eds.), Personnel selection in organizations (pp. 71-98). San Francisco: Jossey-Bass.
5. Borman, W. C., & Motowidlo, S. J. (1997). Task performance and contextual performance: The meaning for personnel selection research. Human Performance, 10, 99-109.
6. Borman, W. C., White, L. A., & Dorsey, D. W. (1995). Effects of task performance and interpersonal factors on supervisor and peer performance ratings. Journal of Applied Psychology, 80, 168-177.
7. Borman, W. C., White, L. A., Pulakos, E. D., & Oppler, S. H. (1991). Models of supervisory job performance ratings. Journal of Applied Psychology, 76, 863-872.
8. Brugnoli, G. A., Campion, J. E., & Basen, J. A. (1979). Racial bias in the use of work samples for personnel selection. Journal of Applied Psychology, 64, 119-123.
9. Burtch, L. D., Lipscomb, M. S., & Wissman, D. J. (1982). Aptitude requirements based on task difficulty: Methodology for evaluation (AFHRL-TR-81-34) Brooks AFB, TX: Air Force Human Resources Laboratory, Manpower and Personnel Division.
10. Campbell, J. P. (1990). Modeling the performance prediction problem in industrial and organizational psychology. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 1, pp.687-732). Palo Alto, CA: Consulting Psychologists Press.
11. Campbell, J. P., McHenry, J. J., & Wise, L. L. (1990). Modeling job performance in a population of jobs. Personnel Psychology, 43, 313-333.
12. Campion, J. E. (1972). Work sampling for personnel selection. Journal of Applied Psychology, 56, 40-44.
13. Cooksey, R. W. (1996). Judgment analysis: Theory, methods, and applications. San Diego: Academic.
14. Department of Defense. (1984). Test manual for the armed services vocational aptitude battery. North Chicago, IL: U.S. Military Entrance Processing Command.
15. DuBois, C. L., Sackett, P. R., Zedeck, S., & Fogli, L. (1993). Further exploration of typical and maximum performance criteria: Definitional issues, prediction, and White-Black differences. Journal of Applied Psychology, 78, 205-211.
16. Ford, J. K., Kraiger, K., & Schechtman, S. L. (1986). Study of race effects in objective indices and subjective evaluations of performance: A meta-analysis of performance criteria. Psychological Bulletin, 99, 330-337.
17. Hamner, C. W., Kim, J. S., Baird, L., & Bigoness, W. J. (1974). Race and sex as determinants ratings by potential employers in a simulated work sample task. Journal of Applied Psychology, 59, 705-711.
18. Hedge, J. W., Dickinson, T. L., & Bierstedt, S. A. (1988). The use of videotape technology to train administrators of walk-through performance testing (AFHRL-TP-87-71). Brooks AFB, TX: Air Force Human Resources Laboratory, Training Systems Division.
19. Hedge, J. W., & Teachout, M. S. (1986). Job performance measurement: A systematic program of research and development (AFHRL-TP-86-37). Brooks AFB, TX: Air Force Human Resources Laboratory, Training Systems Division.
20. Hedge, J. W., & Teachout, M. S. (1992). An interview approach to work sample criterion measurement. Journal of Applied Psychology, 77, 453-461.
21. Hoffman, P. J. (1960). The paramorphic representation of clinical judgment. Psychological Bulletin, 57, 116-131.
22. Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72-98.
23. Ilgen, D. R., & Feldman, J. M. (1983). Performance appraisal: A process focus. Research in Organizational Behavior, 5, 141-197.
24. Kavanagh, M. J., Borman, W. C., Hedge, J. W., & Gould, R. B. (1987). Job performance measurement in the military: A classification scheme, literature review, and directions for research (AFHRL-TR-87-15). Brooks AFB, TX: Air Force Human Resources Laboratory, Training Systems Division.
25. Klimoski, R., & Brickner, M. (1987). Why do assessment centers work? The puzzle of assessment center validity. Personnel Psychology, 40, 243-260.
26. Lance, C. E., Hedge, J. W., & Alley, W. E. (1989). Joint relations of task proficiency with ability, experience, and task difficulty: A cross-level, interactional study. Human Performance, 2, 249-272.
27. Lance, C. E., LaPointe, J. A., & Stewart, A. M. (1994). A test of the context dependency of three causal models of halo rater error. Journal of Applied Psychology, 79, 332-340.
28. Lance, C. E., Parisi, A. G., Bennett, W., Jr., Teachout, M. S., Harville, D. L., & Welles, M. L. (1998). Moderators of skill retention interval/performance decrement relationships in eight U.S. Air Force enlisted specialties. Human Performance, 11, 103-123.
29. Lance, C. E., Teachout, M. S., & Donnelly, T. M. (1992). Specification of the criterion construct space: An application of hierarchical confirmatory factor analysis. Journal of Applied Psychology, 77, 437-452.
30. Lance, C. E., Woehr, D. J., & Fisicaro, S. A. (1991). Cognitive categorization processes in performance evaluation: Confirmatory tests of two models. Journal of Organizational Behavior, 12, 1-20.
31. Landy, F. J., & Fair, J. L. (1980). Performance rating. Psychological Bulletin, 87, 72-107.
32. Laue, F. J., Hedge, J. W., Wall, M. L., Pederson, L. A., & Bentley, B. A. (1992). Job performance measurement system development process (AL-TR-1992-0120). Brooks AFB, TX: Armstrong Laboratory, Human Resources Directorate, Technical Training Research Division.
33. Martell, R. F., Guzzo, R. A., & Willis, C. E. (1995). A methodological and substantive note on the performance-cue effect in ratings of work-group behavior. Journal of Applied Psychology, 80, 191-195.
34. Motowidlo, S. J., Borman, W. C., & Schmit, M. J. (1997). A theory of individual differences in task and contextual performance. Human Performance, 10, 71-83.
35. Motowidlo, S. J., & VanScotter, J. R. (1994). Evidence that task performance should be distinguished from contextual performance. Journal of Applied Psychology, 79, 475-480.
36. Mumford, M. D., Weeks, J. L., Harding, F. D., & Fleishman, E. A. (1987). Measuring occupational difficulty: A construct validation against training criteria. Journal of Applied Psychology, 72, 578-587.
37. Pulakos, E. D., White, L. A., Oppler, S. H., & Borman, W. C. (1989). Examination of race and sex effects on performance ratings. Journal of Applied Psychology, 74, 770-780.
38. Ree, M. J., Earles, J. A., & Teachout, M. S. (1994). Predicting job performance: Not much more than g. Journal of Applied Psychology, 79, 518-524.
39. Robertson, I. T., & Downs, S. (1989). Work-sample tests of trainability: A meta-analysis. Journal of Applied Psychology, 74, 402-410.
40. Robertson, I. T., & Kandola, R. S. (1982). Work sample tests: Validity, adverse impact, and applicant reaction. Journal of Occupational Psychology, 55, 171-183.
41. Saal, F. E., Downey, R. G., & Lahey, M. A. (1980). Rating the ratings: Assessing the psychometric quality of rating data. Psychological Bulletin, 88, 413-428.
42. Sackett, P. R., & DuBois, C. L. Z. (1991). Rater-ratee race effects on performance evaluation: Challenging meta-analytic conclusions. Journal of Applied Psychology, 76, 873-877.
43. Sackett, P. R., Zedeck, S., & Fogli, L. (1988). Relations between measures of typical and maximum job performance. Journal of Applied Psychology, 73, 482-486.
44. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.
45. Schmitt, N., Gooding, R. Z., Noe, R. A., & Kirsch, M. (1984). Meta-analyses of validity studies published between 1964 and 1982 and the investigation of study characteristics. Personnel Psychology, 37, 407-422.
46. Smith, F. D. (1991). Work samples as measures of performance. In A. K. Wigdor & B. F. Green Jr. (Eds.), Performance assessment for the workplace (Vol. 2, pp. 27-52). Washington, DC: National Academy Press.
47. Smith, M. (1994). A theory of the validity of predictors in selection. Journal of Occupational and Organizational Psychology, 67, 13-31.
48. Smith, P. C. (1976). Behavior, results, and organizational effectiveness: The problem of criteria. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology (pp.745-775). Chicago: Rand McNally.
49. Teachout, M. S., & Pellum, M. W. (1991). Air Force research to link standards for enlistment to on-the-job performance (AFHRL-TR-90-90). Brooks AFB, TX: Air Force Human Resources Laboratory, Training Systems Division.
50. Terpstra, D. E. (1996). The search for effective methods. HR Focus, 73(5), 16-17.
51. Thorndike, R. L. (1949). Personnel selection. New York: Wiley.
52. Thornton, G. C. (1992). Assessment centers in human resource management. Reading, MA: Addison Wesley.
53. Tosi, H. L., & Einbender, S. W. (1985). The effects of the type and amount of information in sex discrimination research: A meta-analysis. Academy of Management Journal, 28, 712-723.
54. Viswesvaran, C., Ones, D. S., & Schmidt, F. L. (1996). Comparative analysis of the reliability of job performance ratings. Journal of Applied Psychology, 81, 557-574.
55. Weeks, J. (1984). Occupational learning difficulty: A standard for determining the order of aptitude requirement minimums (AFHRL-SR-84-26). Brooks AFB, TX: Air Force Human Resources Laboratory, Manpower and Personnel Division.
56. Wherry, R. J., & Bartlett, C. J. (1982). The control of bias in ratings: A theory of rating. Personnel Psychology, 35, 521-551.
57. Wigdor, A. K., & Green, B. F., Jr. (Eds.), (1991a). Performance assessment for the workplace (Vol. 1). Washington, DC: National Academy Press.
58. Wigdor, A. K., & Green, B. F., Jr. (Eds.), (1991b). Performance assessment for the workplace (Vol. 2). Washington, DC: National Academy Press.