Lack of Consensus Among Competency Ratings of the Same Occupation: Noise or Substance?

Journal of Applied Psychology

Volumn 95, Issue 3, 2010, Pages 562-571

  Lievens, Filip ^a   Sanchez, Juan I ^b   Bartram, Dave ^c   Brown, Anna ^c  

^a GHENT UNIVERSITY   (Belgium)

^b FLORIDA INTERNATIONAL UNIVERSITY   (United States)

^c SHL Group Ltd   (United Kingdom)

Author keywords

Competency modeling; Consensus; Job crafting; Role theory; Work analysis

Indexed keywords

ADULT; ARTICLE; CONSENSUS; FEMALE; HUMAN; MALE; OCCUPATION; PERSONNEL MANAGEMENT; PROFESSIONAL COMPETENCE; QUESTIONNAIRE;

ADULT; CONSENSUS; EMPLOYEE PERFORMANCE APPRAISAL; FEMALE; HUMANS; MALE; OCCUPATIONS; PROFESSIONAL COMPETENCE; QUESTIONNAIRES;

EID: 77954793620     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0018035     Document Type: Article

References (62)

1. Amabile T.M. Effects of external evaluation on artistic creativity. Journal of Personality and Social Psychology 1979, 37:221-233.
2. Ashforth B.E., Humphrey R.H. Emotional labor in service roles: The influence of identity. Academy of Management Review 1993, 18:88-115.
3. Baranowski L.E., Anderson L.E. Examining rating source variation in work behavior to KSA linkages. Personnel Psychology 2005, 58:1041-1054.
4. Barrett, G. V., Callahan C. M. (1997, April). Competencies: The Madison Avenue approach to professional practice. In R. C. Page (Chair), Competency models: What are they and do they work? Practitioner Forum presented at the 12th Annual Conference of the Society for Industrial and Organizational Psychology, St. Louis, MO.
5. Barrett G.V., Depinet R.L. A reconsideration of testing for competence rather than for intelligence. American Psychologist 1991, 46:1012-1024.
6. Barrick M.R., Mount M.K. Autonomy as a moderator of the relationship between the Big Five personality dimensions and job performance. Journal of Applied Psychology 1993, 78:111-118.
7. Bartram D. The great eight competencies: A criterion-centric approach to validation. Journal of Applied Psychology 2005, 90:1185-1203.
8. Bond C.F., Titus L.J. Social facilitation: A meta-analysis of 241 studies. Psychological Bulletin 1983, 94:265-292.
9. Brennan R.L. Elements of generalizability theory 1992, American College Testing Program, Iowa City, IA.
10. Childs R.A., Peterson N.G., Mumford M.D. Occupational descriptor covariates: Potential sources of variance in O*NET ratings. An occupational information system for the 21st century: The development of O*NET 1999, 237-245. American Psychological Association, Washington, DC. N.G. Peterson, M.D. Mumford, W.C. Borman, P.R. Jeanneret, E.A. Fleishman (Eds.).
11. Cronbach L.J., Gleser G.C., Nanda H., Rajaratnam N. The dependability of behavioral measurements: Theory of generalizability for scores and profiles 1972, Wiley, New York.
12. Cronshaw S.F. Job analysis: Changing nature of work. Canadian Psychology/Psychologie Canadienne 1998, 39:5-13.
13. Dierdorff E.C., Morgeson F.P. Consensus in work role requirements: The influence of discrete occupational context on role expectations. Journal of Applied Psychology 2007, 92:1228-1241.
14. Dierdorff E.C., Rubin R.S., Morgeson F.P. The milieu of managerial work: An integrative framework linking work context to role requirements. Journal of Applied Psychology 2009, 94:972-988.
15. Edwards A.L. An introduction to linear regression and correlation 1984, Freeman, New York.
16. Farr J.L. Facilitating individual role innovation. Innovation and creativity at work 1990, 1-13. Wiley, New York. M.A. West, J.L. Farr (Eds.).
17. Geen R.G., Gange J.J. Drive theory of social facilitation: Twelve years of theory and research. Psychological Bulletin 1977, 84:1267-1288.
18. Gottfredson L.S. Where and why g matters: Not a mystery. Human Performance 2002, 15:25-46.
19. Grant A.M. Relational job design and the motivation to make a prosocial difference. Academy of Management Review 2007, 32:393-417.
20. Hollenbeck J.R., Ilgen D.R., Sego D.J. Repeated measures regression and mediational tests: Enhancing the power of leadership research. Leadership Quarterly 1994, 5:3-23.
21. House R.J., Shane S.A., Herold D.M. Rumors of the death of dispositional research are vastly exaggerated. Academy of Management Review 1996, 21:203-224.
22. Jackson S.E. Measurement of commitment to role identities. Journal of Personality and Social Psychology 1981, 40:138-146.
23. Kohn M.L., Schooler C. Work and personality: An inquiry into the impact of social stratification 1983, Ablex, Norwood, NJ.
24. Kotter J. What effective general managers really do. Harvard Business Review 1982, 60:157-169.
25. Kozlowski S.W.J., Hattrup K. A disagreement about within-group agreement: Disentangling issues of consistency versus consensus. Journal of Applied Psychology 1992, 77:161-167.
26. Landy F.J., Vasey J. Job analysis: The composition of SME samples. Personnel Psychology 1991, 44:27-50.
27. Lawler E. Competencies: A poor foundation for the new pay. Compensation and Benefits Review 1996, 28:20-22.
28. Lievens F., Sanchez J.I. Can training improve the quality of inferences made by raters in competency modeling? A quasi-experiment. Journal of Applied Psychology 2007, 92:812-819.
29. Lievens F., Sanchez J.I., De Corte W. Easing the inferential leap in competency modeling: The effects of task-related information and subject matter expertise. Personnel Psychology 2004, 57:881-904.
30. The 2004 LOMA competency survey 2005, LOMA, Retrieved from ftp://www.loma.org/CompetencySurveyReport2004.pdf.
31. Lyons P. The crafting of jobs and individual differences. Journal of Business and Psychology 2008, 23:25-36.
32. Mintzberg H. Crafting strategy. Harvard Business Review 1987, 65:66-75.
33. Mintzberg H. Managers not MBAs: A hard look at the soft practice of managing and management development 2004, Berrett-Koehler, San Francisco.
34. Mischel W. Personality and assessment 1968, Wiley, New York.
35. Morgeson F.P., Delaney-Klinger K., Hemingway M.A. The importance of job autonomy, cognitive ability, and job-related skill for predicting role breadth and job performance. Journal of Applied Psychology 2005, 90:399-406.
36. Morgeson F.P., Delaney-Klinger K., Mayfield M.S., Ferrara P., Campion M.A. Self-presentation processes in job analysis: A field experiment investigating inflation in abilities, tasks, and competencies. Journal of Applied Psychology 2004, 89:674-686.
37. Morris J.A., Feldman D.C. The dimensions, antecedents, and consequences of emotional labor. Academy of Management Review 1996, 21:986-1010.
38. Oldham G.R., Cummings A. Employee creativity: Personal and contextual factors at work. Academy of Management Journal 1996, 39:607-634.
39. Ouchi W.G. The relationship between organizational structure and organizational control. Administrative Science Quarterly 1977, 22:95-113.
40. Parker S.K. " That is my job": How employees' role orientation affects their job performance. Human Relations 2007, 60:403-434.
41. Parker S.K., Wall T.D., Jackson P.R. " That's not my job": Developing flexible employee work orientations. Academy of Management Journal 1997, 40:899-929.
42. Pearlman, K. (1997, April). Competencies: Issues in their application. In R. C. Page (Chair), Competency models: What are they and do they work? Practitioner forum presented at the 12th Annual Conference of the Society for Industrial and Organizational Psychology, St. Louis, MO.
43. Pedhazur E.J. Multiple regression in behavioral research 1982, Holt, Rinehart & Winston, New York.
44. Peterson N.G., Mumford M.D., Borman W.C., Jeanneret P.R., Fleishman E.A. An occupational information system for the 21st century: The development of O*NET 1999, American Psychological Association, Washington, DC.
45. Roberts L.M., Dutton J.E., Spreitzer G.M., Heaphy E.D., Quinn R.E. Composing the reflected best-self portrait: Building pathways for becoming extraordinary in work organizations. Academy of Management Review 2005, 30:712-736.
46. Sanchez J.I. From documentation to innovation: Reshaping job analysis to meet emerging business needs. Human Resource Management Review 1994, 4:51-74.
47. Sanchez J.I., Levine E.L. Accuracy or consequential validity: Which is the better standard for job analysis data?. Journal of Organizational Behavior 2000, 21:809-818.
48. Sanchez J.I., Levine E.L. What is (or should be) the difference between competency modeling and traditional job analysis?. Human Resource Management Review 2009, 19:53-63.
49. Sanchez J.I., Prager I., Wilson A., Viswesvaran C. Understanding within-job title variance in job-analytic ratings. Journal of Business and Psychology 1998, 12:407-419.
50. Sanchez J.I., Zamora A., Viswesvaran C. Moderators of agreement between incumbent and non-incumbent ratings of job characteristics. Journal of Occupational and Organizational Psychology 1997, 70:209-218.
51. Schippmann J.S., Ash R.A., Battista M., Carr L., Eyde L.D., Hesketh B., Kehoe J., Pearlman K., Prien E.P., Sanchez J.I. The practice of competency modeling. Personnel Psychology 2000, 53:703-740.
52. Scott S.G., Bruce R.A. Determinants of innovative behavior: A path model of individual innovation in the workplace. Academy of Management Journal 1994, 37:580-607.
53. Shalley C.E., Gilson L.L., Blum T.C. Matching creativity requirements and the work environment: Effects on satisfaction and intent to turnover. Academy of Management Journal 2000, 43:215-224.
54. Shalley C.E., Gilson L.L., Blum T.C. Interactive effects of growth need strength work context, and job complexity on self-reported creative performance. Academy of Management Journal 2009, 52:489-505.
55. SHL competency designer cards: User manual [Internal technical document] 2006, SHL Group Ltd, SHL Group Ltd., Thames Ditton, United Kingdom.
56. Strong M.H., Jeanneret P.R., McPhail S.M., Blakley B.R., D'Egidio E.L. Work context. An occupational information system for the 21st century: The development of O*NET 1999, 127-145. American Psychological Association, Washington, DC. N.G. Peterson, M.D. Mumford, W.C. Borman, P.R. Jeanneret, E.A. Fleishman (Eds.).
57. Tett R.P., Guterman H.A., Bleier A., Murphy P.J. Development and content validation of a " hyperdimensional" taxonomy of managerial competence. Human Performance 2000, 13:205-251.
58. Tierney P., Farmer S.M. Creative self-efficacy: Potential antecedents and relationship to creative performance. Academy of Management Journal 2002, 45:1137-1148.
59. Dictionary of occupational titles 1965, 1. U.S. Department of Labor, U.S. Government Printing Office, Washington, DC. 3rd ed.
60. Werbel J.D., DeMarie S.M. Aligning strategic human resource management and person-environment fit. Human Resource Management Review 2005, 15:247-262.
61. Whitley R. On the nature of managerial tasks and skills: Their distinguishing characteristics and organization. Journal of Management Studies 1989, 26:209-224.
62. Wrzesniewski A., Dutton J.E. Crafting a job: Revisioning employees as active crafters of their work. Academy of Management Review 2001, 26:179-201.