Predictive criterion-related validity of observer ratings of personality and job-related competencies using multiple raters and multiple performance criteria

Human Performance

Volumn 23, Issue 4, 2010, Pages 361-378

  Zimmerman, Ryan D ^a   Triana, Maríadel Carmen ^b   Barrick, Murray R ^c  

^a TEXAS A AND M UNIVERSITY   (United States)

^b UNIVERSITY OF WISCONSIN MADISON   (United States)

^c TEXAS A AND M UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77956852563     PISSN: 08959285     EISSN: None     Source Type: Journal    
DOI: 10.1080/08959285.2010.501049     Document Type: Article

References (58)

1. Aamodt, M. G. (2007). Industrial/organizational psychology: An applied approach (5th ed.). Pacific Grove, CA: Wadsworth.
2. Aamodt, M. G., & Williams, F. (2005, April). Reliability, validity, and adverse impact of references and letters of recommendation. Paper presented at the 20th Conference of the Society for Industrial-Organizational Psychology, Los Angeles, CA.
3. Atwater, L. E., Ostroff, C., Yammarino, F. J., & Fleenor, J.W. (1998). Self-other agreement: Does it really matter? Personnel Psychology, 51, 577-598.
4. Baldwin, T. T., Bedell, M. D., & Johnson, J. L. (1997). The social fabric of a team-based M.B.A. program: Network effects on student satisfaction and performance. Academy of Management Journal, 40, 1369-1397.
5. Barrick, M. R., & Mount, M. K. (1991). The big five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1-26.
6. Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality and performance at the beginning if the new millennium: What do we know and where do we go next? International Journal of Selection and Assessment, 9, 9-29.
7. Barrick, M. R., Mount, M. K., & Wonderlic, Inc. (2006). Wonderlic Productivity Index. Libertyville, IL: Wonderlic, Inc.
8. Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In K. J. Klein & S. W. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations (pp. 349-381). San Francisco: Jossey-Bass.
9. Bratko, D., Chamorro-Premuzic, T., & Saks, Z. (2006). Personality and school performance: Incremental validity of selfand peer-ratings over intelligence. Personality and Individual Differences, 41, 131-142.
10. Busato, V. V., Prins, F. J., Elshout, J. J., & Hamaker, C. (2000). Intellectual ability, learning style, personality, achievement motivation and academic success of psychology students in higher education. Personality & Individual Differences, 29, 1057-1068.
11. Campion, M. A., Palmer, D. K., & Campion, J. E. (1997). A review of structure in the selection interview. Personnel Psychology, 50, 655-702.
12. Carless, S. A., Wearing, A. J., & Mann, L. (2000). A short measure of transformational leadership. Journal of Business and Psychology, 14, 389-405.
13. Chamorro-Premuzic, T., & Furnham, A. (2003). Personality predicts academic performance: Evidence from two longitudinal university samples. Journal of Research in Personality, 37, 319-338.
14. Chapman, D. S., & Zweig, D. I. (2005). Developing a nomological network for interview structure: Antecedents and consequences of the structured selection interview. Personnel Psychology, 58, 673-702.
15. Connolly, J. J., Kavanagh, E. J., & Viswesvaran, C. (2007). The convergent validity between self and observer ratings of personality: A meta-analytic review. International Journal of Selection and Assessment, 15, 110-117.
16. Conway, J. M., Jako, R. A., & Goodman, D. F. (1995). A meta-analysis of interrater and internal consistency reliability of selection interviews. Journal of Applied Psychology, 80, 565-579.
17. Conway, J. M., Lombardo, K., & Sanders, K. C. (2001). A meta-analysis of incremental validity and nomological networks for subordinate and peer ratings. Human Performance, 14, 267-303.
18. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
19. Daniel, D. (1990). Validity of a standardized reference checklist. Applied H.R.M. Research, 1, 51-66.
20. Gatewood, R. D., Feild, H. S., & Barrick, M. (2008). Human resource selection (6th ed.). Mason, OH: Thomson South-Western.
21. Gilman, R., & Anderman, E. M. (2006). The relationship between relative levels of motivation and intrapersonal, interpersonal, and academic functioning among older adolescents. Journal of School Psychology, 44, 375-391.
22. Goff, M., & Ackerman, P. L. (1992). Personality-intelligence relations: Assessing typical intellectual engagement. Journal of Educational Psychology, 84, 537-552.
23. Hogan, R. T. (1996). A socioanalytic perspective on the five-factor model. In J. S.Wiggins (Ed.), The five-factor model of personality: Theoretical perspectives (pp. 163-179). New York: Guilford.
24. Huffcutt, A. I., Conway, J. M., Roth, P. L., & Stone, N. J. (2001). Identification and meta-analytic assessment of psychological constructs measured in employment interviews. Journal of Applied Psychology, 86, 897-913.
25. Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72-98.
26. James, L. R. (1982). Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology, 67, 219-229.
27. Judge, T. A., Bono, J. E., & Locke, E. A. (2000). Personality and job satisfaction: The mediating role of job characteristics. Journal of Applied Psychology, 85, 237-249.
28. Judge, T. A., Colbert, A. E., & Ilies, R. (2004). Intelligence and leadership: A quantitative review and test of theoretical propositions. Journal of Applied Psychology, 89, 542-552.
29. Judge, T. A., & Ferris, G. R. (1993). Social context of performance evaluation decisions. Academy of Management Journal, 35, 80-105.
30. Judge, T. A., Piccolo, R. F., & Ilies, R. (2004). The forgotten ones? The validity of consideration and initiating structure in leadership research. Journal of Applied Psychology, 89, 36-51.
31. Kolar, D. W., Funder, D. C., & Colvin, C. R. (1996). Comparing the accuracy of personality judgments by the self and knowledgeable others. Journal of Personality, 64, 311-337.
32. Kristof-Brown, A. L., & Stevens, C. K. (2001). Goal congruence in project teams: Does the fit between members'personal mastery and performance goals matter? Journal of Applied Psychology, 86, 1083-1095.
33. Lounsbury, J. W., Huffstetler, B. C., Leong, F. T., & Gibson, L. W. (2005). Sense of identity and collegiate academic achievement. Journal of College Student Development, 46, 501-514.
34. McCarthy, J. M., & Goffin, R. D. (2001). Improving the validity of letters of recommendation: An investigation of three standardized reference forms. Military Psychology, 13, 199-222.
35. McCrae, R. R.,&Costa, P. T. (1989). The structure of interpersonal traits:Wiggins's circumplex and the five-factor model. Journal of Personality and Social Psychology, 56, 586-595.
36. McCrae, R. R., & Weiss, A. (2007). Observer ratings of personality. In R.W. Robins, R. C. Fraley, & R. F. Krueger (Eds.), Handbook of research methods in personality psychology (pp. 259-272). New York: Guilford.
37. McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599-616.
38. Mohammed, S., Mathieu, J. E., & Bartlett, A. L. (2002). Technical-administrative task performance, leadership task performance, and contextual performance: Considering the influence of team- and task-related composition variables. Journal of Organizational Behavior, 23, 795-814.
39. Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollenbeck, J. R., Murphy, K., & Schmitt, N. (2007). Reconsidering the use of personality tests in personnel selection contexts. Personnel Psychology, 60, 683-729.
40. Morgeson, F. P., Reider, M. H., & Campion, M. A. (2005). Selecting individuals in team settings: The importance of social skills, personality characteristics, and teamwork knowledge. Personnel Psychology, 58, 583-611.
41. Mount, M. K., Barrick, M. R., & Strauss, J. P. (1994). Validity of observer ratings of the big five personality factors. Journal of Applied Psychology, 79, 272-280.
42. Norman, W. T. (1963). Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings. Journal of Abnormal and Social Psychology, 66, 574-583.
43. Ones, D.S. (1993). The construct validity of integrity tests. Unpublished doctoral dissertation.
44. Ones, D. S., Dilchert, S., Viswesvaran, C., & Judge, T. A. (2007). In support of personality assessment in organizational settings. Personnel Psychology, 60, 995-1027.
45. Paulhus, D. L. (1984). Two-component models of socially desirable responding. Journal of Personality and Social Psychology, 46, 598-609.
46. Paunonen, S. V., & Ashton, M. C. (2001). Big Five predictors of academic achievement. Journal of Research in Personality, 35, 78-90.
47. Reilly, R. R., & Chao, G. T. (1982). Validity and fairness of some alternate employee selection procedures. Personnel Psychology, 35, 1-62.
48. Salgado, J. F. (1997). The five factor model of personality and job performance in the European Community. Journal of Applied Psychology, 82, 30-43.
49. Schmidt, F. L, & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.
50. Schmidt, F. L., & Zimmerman, R. D. (2004). A counterintuitive hypothesis about employment interview validity and some supporting evidence. Journal of Applied Psychology, 89, 553-561.
51. Small, E. E., & Diefendorff, J. M. (2006). The impact of contextual self-ratings and observer ratings of personality on the personality-performance relationship. Journal of Applied Social Psychology, 36, 297-320.
52. Society for Human Resource Management. (2005). Reference and background checking survey report. Alexandria, VA: Author.
53. Taylor, P. J., Pajo, K., Cheung, G.W., & Stringfield, P. (2004). Dimensionality and validity of a structured telephone reference check procedure. Personnel Psychology, 57, 745-772.
54. Tupes, E. C., & Christal, R. E. (1961). Recurrent personality factors based on trait ratings. (Tech. Rep. No. ASD-TR 61-97). Lackland Air Force Base, TX: U.S. Air For ce.
55. Viswesvaran, C., Ones, D. S, & Schmidt, F. L. (1996). Comparative analysis of the reliability of job performance ratings. Journal of Applied Psychology, 81, 557-574.
56. Wernimont, P. F., & Campbell, J. P. (1968). Signs, samples, and criteria. Journal of Applied Psychology, 52, 372-376.
57. Zimmerman, R. D. (2008). Understanding the impact of personality traits on individuals' turnover decisions: A meta-analytic path model. Personnel Psychology, 61, 309-348.
58. Zimmerman, R. D., Mount, M. K., & Goff, M., III. (2008). Multisource feedback ratings and leaders' goal performance: Moderating effects of rating purpose, rater perspective, and performance dimension. International Journal of Selection and Assessment, 16, 121-134.