Predicting assessment center performance with 360-degree, top-down, and customer-based competency assessments

Human Resource Management

Volumn 45, Issue 3, 2006, Pages 357-390

  Hagan, Christine M ^a   Konopaske, Robert ^b   Bernardin, H John ^c,d   Tyler, Catherine L ^e  

^a UNIVERSITY OF MIAMI   (United States)

^b Applied Management Sciences Institute   (United States)

^c FLORIDA ATLANTIC UNIVERSITY   (United States)

^d Academy of Management   (Poland)

^e OAKLAND UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33749236486     PISSN: 00904848     EISSN: 1099050X     Source Type: Journal    
DOI: 10.1002/hrm.20117     Document Type: Article

References (84)

1. Anderson, C., & Bissell, P. (2004). Using semi covert research to evaluate an emergency hormonal contraception service. Pharmacy World & Science, 26(2), 102-106.
2. Arthur, W., Jr., Day, E. A., McNelly, T. L., & Edens, P. S. (2003). A meta-analysis of the criterion-related validity of assessment center dimensions. Personnel Psychology, 56, 125-154.
3. Arthur, W., Jr., Woehr, D. J., & Maldegen, R. (2000). Convergent and discriminant validity of assessment center dimensions: A conceptual and empirical re-examination of the assessment center construct-related validity paradox. Personnel Psychology, 26, 813-835.
4. Athey, T. R., & Orth, M. S. (1999). Emerging competency methods for the future. Human Resource Management, 38, 215-226.
5. Atkins, P. W. B., & Wood, R. E. (2002). Self- versus others' ratings as predictors of assessment center ratings: Validation evidence for 360-degree feedback programs. Personnel Psychology, 55, 871-904.
6. Barney, J. B. (1991). Firm resources and sustained competitive advantage. Journal of Management, 17(1), 99-120.
7. Becker, B. E., & Huselid, M. A. (1999). Overview: Strategic human resource management in five leading firms. Human Resource Management, 38, 287-301.
8. Becker, B. E., Huselid, M. A., & Ulrich, D. (2001). The HR scorecard: Linking people, strategy, and performance. Boston: Harvard Business School Press.
9. Bernardin, H. J. (1992). An 'analytic' framework for customer-based performance content development and appraisal. Human Resource Management Review, 2, 81-102.
10. Bernardin, H. J., & Beatty, R.W. (1984). Performance appraisal: Assessing human behavior at work. Boston: Kent-Wadsworth Publishing.
11. Bernardin, H. J., Buckley, M. R., Tyler, C. L., & Wiese, D. S. (2000). A consideration of strategies in rater training. In G. R. Ferris (Ed.), Research in personnel & human resources management (Vol. 18, pp. 221-274). Greenwich, CT: JAI Press.
12. Bernardin, H. J., Hagan, C. M., Kane, J. S., &Villanova, P. (1998). Effective performance management: A focus on precision, customers, and situational constraints. In J. W. Smither (Ed.), Performance appraisal: State of the art in practice (pp. 3-48). San Francisco: Jossey-Bass.
13. Bernardin, H. J., & Smith, P. C. (1981). A clarification of some issues regarding the development and use of behaviorally anchored rating scales. Journal of Applied Psychology, 66, 458-463.
14. Bernardin, H. J., & Tyler, C. L. (2001). Legal and ethical issues in multisource feedback. In D.W. Bracken, C. W.Timmreck, & A. H. Church (Eds.), The handbook of multisource feedback (pp. 447-462). San Francisco: Jossey-Bass.
15. Bernardin, H. J., Villanova, P. & Cooke, D. (2005). The relationship among the Big Five factors, rating discomfort and rating elevation. Manuscript under review.
16. Binning, J. F., & Barrett, G. V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74, 478-494.
17. Borman, W. C. (1997). 360 degree ratings: An analysis of assumptions and a research agenda for evaluating their validity. Human Resource Management Review, 7, 299-316.
18. Brogden, H. E., & Taylor, E. K. (1950). The theory and classification of criterion bias. Educational and Psychological Measurement, 10, 159-186.
19. Browne, M. W. (1975). Predictive validity of a linear regression equation. British Journal of Mathematical and Statistical Psychology, 28, 79-87.
20. Caldwell, C., Thornton, G. C., & Gruys, M. L. (2003). Ten classic assessment center errors: Challenges to selection validity. Public Personnel Management, 32, 73-88.
21. Campbell, D.T., & Fiske, D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.
22. Campbell, J. P. (1976). Psychometric theory. In M.D. Dunnette (Ed.), Handbook of industrial and organizational psychology (pp. 185-222). Chicago: Rand-McNally.
23. Cardy, R. L. (1998). Performance appraisal in a quality context: A new look at an old problem. In J. W. Smither (Ed.), Performance appraisal: State of the art in practice (pp. 132-162). San Francisco: Jossey-Bass.
24. Cascio, W. F., & Aguinis, H. (2005). Applied psychology in human resource management. Upper Saddle River, NJ: Prentice Hall.
25. Church, A. H. (1997). Do you see what I see? An expianation of congruence in ratings from multiple perspectives. Journal of Applied Social Psychology, 27, 983-1020.
26. Facteau, J. D., & Craig, S. B. (2001). Are performance appraisal ratings from different rating sources comparable? Journal of Applied Psychology 86. 215-227.
27. Fiedler, A. (2001). Adverse impact on Hispanic job applicants during assessment center evaluations. Hispanic Journal of Behavioral Sciences, 23, 102-110.
28. Finn, A., & Kayande, U. (1999). Unmasking a phantom: A psychometric assessment of mystery shopping. Journal of Retailing, 75(2), 195-217.
29. Fleenor, J. W., & Brutus, S. (2001). Multisource feedback for personnel decisions. In D.W. Bracken, C. W. Timmreck, & A. H. Church (Eds.),The handbook of multisource feedback (pp. 335-351). San Francisco: Jossey-Bass.
30. Fletcher, C., Baldry, C., & Cunningham-Snell, N. (1998). The psychometric properties of 360 degree feedback: An empirical study and a cautionary tale. International Journal of Selection and Assessment, 6, 19-34.
31. Ganzach, Y., Kluger, A. N., & Klayman, N. (2000) Making decisions from an interview: Expert measurement and mechanical combination. Personnel Psychology, 53, 1-20.
32. Gatewood, R. D., & Feild, H. S. (2001). Human resource selection. Fort Worth, TX: The Dryden Press.
33. Gaugler, B. B., Rosenthal, D. B., Thornton, G. C., III, & Bentson, C. (1987). Meta-analysis of assessment center validity. Journal of Applied Psychology, 72, 493-511.
34. Goldstein, H.W., Yusko, K. P., Braverman, E. P., Smith, D. B., & Chung, B. (1998).The role of cognitive ability in the subgroup differences and incremental validity of assessment center exercises. Personnel Psychology, 51, 357-374.
35. Goldstein, H. W., Yusko, K. P., & Nicolopoulos, V. (2001). Exploring black-white subgroup differences of managerial competencies. Personnel Psychology, 54, 783-807.
36. Greguras, G. J., & Robie, C. (1998). A new look at within-source interrater reliability of 360 degree feedback ratings. Journal of Applied Psychology, 83, 960-968.
37. Greguras, G. J., Robie, C., Schleicher, D. J., & Goff, M. (2003). A field study of the effects of rating purpose on the quality of multisource ratings. Personnel Psychology, 56, 1-21.
38. Hagan, C. M., & Bernardin, H. J. (2003). Customer feedback as a critical performance dimension: Review and exploratory examination. In C. A. Schriesheim & L. L. Neider (Eds.), New directions in human resource management (pp. 1-27). Greenwich, CT: Information Age.
39. Hamel, G., & Prahalad, C. K. (1994). Competing for the future. Boston, MA: Harvard Business School Press.
40. Hennessey, H.W., & Bernardin, H. J. (2003). The relationship between performance appraisal criterion specificity and statistical evidence of discrimination. Human Resource Management, 42, 143-158.
41. Kane, J. S., Bernardin, H. J., Villanova, P., & Peyrefitte, J. (1995).The stability of rater leniency: Three studies. Academy of Management Journal, 38, 1036-1051.
42. Kirn, S. P., Rucci, A. J., Huselid, M. A., & Becker, B. E. (1999). Strategic human resource management at Sears. Human Resource Management, 38, 329-335.
43. Klemp, G. O. (Ed.). (1980).The assessment of occupational competence. Washington, DC: Report to the National Institute of Education.
44. Lawler, E. E., III. (1994). From job-based to competency-based organizations. Journal of Organization Behavior, 15, 3-15.
45. Lengnick-Hall, C. A. (1996). Customer contributions to quality: A different view of the customer-oriented firm. Academy of Management Review, 21, 791-824.
46. Lievens, F. (1998). Factors which improve the construct validity of assessment centers: A review. International Journal of Selection and Assessment, 6, 141-152.
47. Lievens, F., Sanchez, J. I., & De Corte,W. D. (2004), Easing the inferential leap in competency modeling: The effects of task-related information and subject matter expertise. Personnel Psychology, 57, 881-904.
48. London, M., & Beatty, R. W. (1993). 360-degree feedback as a competitive advantage. Human Resource Management, 32, 353-372.
49. London, M., & Smither, J.W. (1995). Can multi-source feedback change perceptions of goal accomplishment, self-evaluations, and performance related outcomes? Personnel Psychology, 48, 803-839.
50. Lucia, A. D., & Lepsinger, R. (1999). The art and science of competency models: Pinpointing critical success factors in organizations. San Francisco: Jossey-Bass.
51. McCowan, R. A., Bowen, U., Huselid, M. A., & Becker, B. E. (1999). Strategic human resource management at Herman Miller. Human Resource Management, 38, 303-308.
52. Milkovich, G. T., & Newman, J. M. (2005). Compensation. New York: McGraw-Hill Irwin.
53. Moriarty, H., McLeod, D., & Dowell, A. (2003). Mystery shopping in health service evaluation. British Journal of General Practice, 53, 942-946.
54. Mount, M. K., Judge, T. A., Scullen, S. E., Sytsma, M. R., & Hezlett, S. A. (1998). Trait, rater and level effects in 360-degree performance ratings, Personnel Psychology, 51, 557-576.
55. Murphy, K. M., & Cleveland, J. N. (1995). Understanding performance appraisal: Social, organizational and goal-based perspectives. Thousand Oaks, CA: Sage.
56. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). NewYork: McGraw-Hill.
57. Ostroff, C., Atwater, L. E., & Feinberg, B. J. (2004). Understanding self-other agreement: A look at rater and rate characteristics, context, and outcomes. Personnel Psychology, 57, 333-375.
58. Parry, S. B. (1996).The quest for competencies. Training, 33(7), 48-54.
59. Pitoniak, M. J., Sireci, S. G., & Leucht, R. M. (2002). A multitrait-multimethod investigation of scores from a professional licensure examination. Educational and Psychological Measurement, 62, 498-516.
60. Powell, G. N., & Butterfield, D. A. (1997). Effect of race on promotions to top management in a federal department. Academy of Management Journal, 40, 112-128.
61. Prahalad, C. K., & Hamel, G. (1990). The core competence of the corporation. Harvard Business Review, 67(3), 79-91.
62. Raju, N. S., Bilgic, R., Edwards, J. E., & Fleer, P. F. (1997). Methodology review: Estimation of population validity and cross validity and the use of equal weights in prediction. Applied Psychological Measurement, 21, 291-305.
63. Raju, N. S., Bilgic, R., Edwards, J. E., & Fleer, P. F. (1999). Accuracy of population validity and cross-validity estimation: An empirical comparison of formula-based, traditional empirical, and equal weights procedures. Applied Psychological Measurement, 23, 99-115.
64. Schippmann, J. S. (1999). Strategic job modeling: Working at the core of integrated human resources. Mahwah, NJ: Erlbaum.
65. Schippmann, J. S., Ash, R. A., Battista, M., Carr, L., Eyde, L. D., Hesketh, B. et al. (2000). The practice of competency modeling. Personnel Psychology, 53, 703-740.
66. Schleicher, D. J., Day, D. V., Mayes, B. T., & Riggio, R. E. (2002). A new frame of reference training: Enhancing the construct validity of assessment centers. Journal of Applied Psychology, 87, 735-746.
67. Schmidt, F. L., & Hunter, J. E. (1981). Employment testing: Old theories and new research findings. American Psychologist, 36, 128-137.
68. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.
69. Scullen, S. E., Mount, M. K., & Goff, M. (2000). Understanding the latent structure of job performance ratings. Journal of Applied Psychology, 85, 956-970.
70. Scullen, S. E., Mount, M. K., & Judge, T. A. (2003). Evidence of the construct validity of developmental ratings of managerial performance. Journal of Applied Psychology, 88, 50-66.
71. Smither, J. W., London, M., & Reilly, R. R. (2005). Does performance improve following multisource feedback? A theoretical model, meta-analysis, and review of empirical findings. Personnel Psychology, 58, 33-66.
72. Spencer, L. M., Jr., & Spencer, S. M. (1993). Competence at work: Models for superior performance. New York: Wiley.
73. Spychalski, A. C., Quinones, M. A., Gaugler, B. B., & Pohley, K. (1997). A survey of assessment center practices in organizations in the United States. Personnel Psychology, 50, 71-90.
74. Tett, R. P., Guterman, H. A., Bleier, A., & Murphy, P. J. (2000). Development and content validation of a 'hyperdimensional' taxonomy of managerial competence. Human Relations, 13, 205-251.
75. Thornton, G. C., III. (1992). Assessment centers in human resource management. Reading, MA: Addison-Wesley.
76. Tornow, W. W. (1993). Editor's note: Introduction to special issue on 360-degree feedback. Human Resource Management, 32, 211-219.
77. Ulrich, D. (1997). Human resource champions: The next agenda for adding value and delivering results. Boston, MA: Harvard Business School Press.
78. Villanova, P. (1992). A customer-based model for developing job performance criteria. Human Resource Management Review, 2, 103-114.
79. Waldman, D. A., Atwater, L. A., & Antonioni, D. (1998). Has 360 feedback gone amok? Academy of Management Executive, 12(2), 86-94.
80. Wernerfelt, B. (1984). A resource-based view of the firm. Strategic Management Journal, 5(1), 171-180.
81. Wilson, A. M. (2002). Attitudes towards customer satisfaction measurement in the retail sector. International Journal of Market Research, 44, 213-222.
82. Woehr, D. J., & Arthur, W., Jr. (2003).The constructrelated validity of assessment center ratings: A review and meta-analysis of the role of methodological factors. Journal of Management, 29, 231-258.
83. Woehr, D. J., Sheehan, K. M., & Bennett, W. (2005). Assessing measurement equivalence across rating sources: A multitrait-multimethod approach. Journal of Applied Psychology, 90, 592-600.
84. Yammarino, F. J., & Atwater, L. E. (1993). Understanding self-perception accuracy: Implications for human resource management. Human Resource Management, 32, 231-247.