A field study of the effects of rating purpose on the quality of multisource ratings

Personnel Psychology

Volumn 56, Issue 1, 2003, Pages 1-21

  Greguras, Gary J ^b   Robie, Chet ^c   Schleicher, Deidra J ^a   Goff III, Maynard ^c  

^a UNIVERSITY OF TULSA   (United States)

^b LOUISIANA STATE UNIVERSITY   (United States)

^c NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0037356589     PISSN: 00315826     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.1744-6570.2003.tb00141.x     Document Type: Article

References (57)

1. Bernardin HJ. (1981, August). The effect of reciprocal leniency on the relationship between consideration scores from the LBDQ and performance ratings. Proceedings of the 40th annual meeting of the Academy of Management, 131-135, Detroit, MI.
2. Bernardin HJ, Beatty RW. (1984). Performance appraisal: Assessing human behavior at work. Boston: Kent.
3. Bernardin HJ, Dahmus SA, Redmon G. (1993). Attitudes of first-line supervisors toward subordinate appraisals. Human Resource Management, 32, 315-324.
4. Bohl DL. (1996). Minisurvey: 360-degree appraisals yield superior results, survey shows. Compensation and Benefits Review, 28(5), 16-20.
5. Brennan RL. (1992). Elements of generalizability theory. Iowa City, IA: American College Testing.
6. Brennan RL. (2000). Performance assessments from the perspective of generalizability theory. Applied Psychological Measurement, 24, 339-353.
7. Brutus S, Derayeh M. (2002). Multisource assessment programs in organizations: An insider's perspective. Human Resource Development Quarterly, 13, 187-203.
8. Church AH, Bracken DW (1997). Advancing the state of the art of 360-degree feedback. Group and Organization Management, 22, 149-161.
9. Clauser BE, Clyman SG, Swanson DB. (1999). Components of rater error in a complex performance assessment. Journal of Educational Measurement, 36, 29-45.
10. Cleveland JN, Murphy KR, Williams RE. (1989). Multiple uses of performance appraisal: Prevalence and correlates. Journal of Applied Psychology, 74, 130-135.
11. Conway JM. (1996). Analysis and design of multitrait-multirater performance appraisal studies. Journal of Management, 22, 139-162.
12. Conway JM, Huffcut AI. (1997). Psychometric properties os multisource performance ratings: A meta-analysis of subordinate, superior, peer, and self-ratings. Human Performance, 10, 331-360.
13. Cronbach LJ, Gleser GC, Nanda H, Rajaratnam N. (1972). The dependability of behavioral measurements. New York: Wiley.
14. Cronbach LJ, Rajaratnam N, Gleser B. (1963). Theory of generalizability: A liberalization of reliability theory. British Journal of Statistical Psychology, 16, 137-163.
15. Dalessio AT. (1998). Using multisource feedback for employee development and personnel decisions. In Smither JW (Ed.), Performance appraisal: State of the art in practice. San Francisco: Jossey-Bass.
16. Day DV, Silverman SB. (1992, August). Examining the generalizability of field performance ratings. Paper presented at the 100th Annual Convention of the American Psychological Association, Washington, D.C.
17. DeNisi AS, Cafferty T, Meglino B. (1984). A cognitive view of the performance appraisal process: A model and research propositions. Organizational Behavior and Human Performance, 33, 360-396.
18. Dobbins GH, Cardy RL, Truxillo DM. (1988). The effects of purpose of appraisal and individual differences in stereotypes of women on sex differences in performance ratings: A laboratory and field study. Journal of Applied Psychology, 73, 551-558.
19. Dorfman PW, Stephan WG, Loveland J. (1986). Performance appraisal behaviors: Supervisor perceptions and subordinate reaction. PERSONNEL PSYCHOLOGY, 39, 579-597.
20. Farh J, Cannella AA Jr, Bedeian AG. (1991). Peer ratings: The impact of purpose on rating quality and user acceptance. Group and Organizational Studies, 16, 367-386.
21. Farh J, Werbel JD. (1986). Effects of purpose of the appraisal and expectation of validation on self-appraisal leniency. Journal of Applied Psychology, 71, 527-529.
22. Fisher CD. (1989). Current and recurrent challenges in HRM. Journal of Management, 15, 157-180.
23. Gerhart B, Wright PM, McMahan GC, Snell SA. (2000). Measurement error in research on the human resources and firm performance: How much error is there and how does it influence effect size estimates? PERSONNEL PSYCHOLOGY, 53, 803-834.
24. Ghorpade J. (2000). Managing five paradoxes of 360-degree feedback. Academy of Management Executive, 14, 140-150.
25. Greguras GJ, Robie C. (1998). A new look at within-source interrater reliability of 360-degree feedback ratings. Journal of Applied Psychology, 83, 960-968.
26. Harris MM, Smith DE, Champagne D. (1995). A field study of performance appraisal purpose: Research versus administrative-based ratings. PERSONNEL PSYCHOLOGY, 48, 151-160.
27. Hoyt WT, Kerns M-D. (1999). Magnitude and moderators of bias in observer ratings: A meta-analysis. Psychological Methods, 4, 403-424.
28. Ilgen DR, Feldman JM. (1983). Performance appraisal: A process focus. In Staw BM, Cummings LL (Eds.), Research in organizational behavior, (Vol. 5, pp. 141-196). Greenwich, CT: JAI.
29. Jawahar IM, Williams CR. (1997). Where all the children are above average: The performance appraisal purpose effect. PERSONNEL PSYCHOLOGY, 50, 905-925.
30. Kraiger K, Teachout MS. (1990). Generalizability theory as construct-related evidence of the validity of job performance ratings. Human Performance, 3, 19-35.
31. Landy FJ, Farr J. (1980). Performance rating. Psychological Bulletin, 87, 72-107.
32. London M. (2001). The great debate: Should multisource feedback be used for administration or development only? In Bracken DW, Timmreck CW, Church AH (Eds.), The handbook of multisource feedback: The comprehensive resource for designing and implementing MSF processes, (pp. 368-385). San Francisco: Jossey-Bass.
33. London M, Smither JW. (1995). Can multisource feedback change self-awareness and behavior? Theoretical applications and directions for research. PERSONNEL PSYCHOLOGY, 48, 803-840.
34. Longenecker CO, Gioia DA, Sims HP. (1987). Behind the mask: The politics of employee appraisal. Academy of Management Executive, 1, 183-193.
35. McIntyre RM, Smith DE, Hassett CE. (1984). Accuracy of performance ratings as affected by rater training and perceived purpose of rating. Journal of Applied Psychology, 69, 147-156.
36. Mount MK, Judge TA, Scullen SE, Systma MR, Hezlett SA. (1998). Trait, rater, and level effects in 360-degree performance ratings. PERSONNEL PSYCHOLOGY, 57,557-576.
37. Murphy KR, Cleveland JN. (1995). Understanding performance appraisal: Social, organizational, and goal-based perspectives. Thousand Oaks, CA: Sage.
38. Murphy KR, DeShon R. (2000a). Interrater correlations do not estimate the reliability of job performance ratings. PERSONNEL PSYCHOLOGY, 53, 873-900.
39. Murphy KR, DeShon R. (2000b). Progress in psychometrics: Can industrial and organizational psychology catch up? PERSONNEL PSYCHOLOGY, 53, 913-924.
40. Nugent WR, Hankins JA. (1992). A comparison of classical, item response, and generalizability theories of measurement. Journal of Social Service Research, 16, 11-39.
41. Nunnally JC. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.
42. Scullen SE, Mount MK, Goff M. (2000). Understanding the latent structure of job performance ratings. Journal of Applied Psychology, 85, 956-970.
43. Shavelson RJ, Webb NM. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage.
44. Smither JW, London M, Vasilopoulos ML, Reilly RR, Millsap RE, Salvemini N. (1995). An examination of the effects of an upward feedback program over time. PERSONNEL PSYCHOLOGY, 48, 1-35.
45. Stephan WG, Dorfman PW. (1989). Administrative and developmental functions in performance appraisals: Conflict or synergy? Basic and Applied Social Psychology, 10, 27-41.
46. Sulsky LM, Keown L. (1998). Performance appraisal in the changing world of work: Implications for the meaning and measurement of work performance. Canadian Psychology, 39, 52-59.
47. Timmreck CW, Bracken DW. (1995, May). Upward feedback in the trenches: Challenges and realities. Paper presented at the 10th Annual Conference of the Society for Industrial and Organizational Psychology, Orlando, FL.
48. Timmreck CW, Bracken DW. (1996, April). Multisource assessment: Reinforcing the preferred "Means" to the end. Paper presented at the 11th Annual Conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
49. Timmreck CW, Bracken DW. (1997). Multisource feedback: A study of its use in decision making. Employment Relations Today, 12, 86-94.
50. Tornow WW. (1993). Editor's note: Introduction to special issue on 360-degree feedback. Human Resource Management, 32, 211-219.
51. Tornow W, London M. (1998). Maximizing the value of 360-degree feedback Aprocess for successful individual and organizational development. San Francisco: Jossey-Bass.
52. Viswesvaran C, Ones DS, Schmidt FL. (1996). Comparative analysis of the reliability of job performance ratings. Journal of Applied Psychology, 81, 557-574.
53. Waldman DA, Atwater LE, Antonioni D. (1998). Has 360-degree feedback gone amok? Academy of Management Executive, 72, 86-94.
54. Webb NM, Shavelson RJ, Kim K-S, Chen Z. (1989). Reliability (generalizability) of job performance measurements: Navy machinist mates. Journal of Military Psychology, 1, 91-110.
55. Wherry RJ. (1952). The control of bias in rating: A theory of rating. PRB Report No. 922, Contract No. DA-49-083 OSA 69, Department of the Army.
56. Wherry RJ Sr, Bartlett CJ. (1982). The control of bias in ratings: A theory of rating. PERSONNEL PSYCHOLOGY, 35, 521-551.
57. Williams KJ, DeNisi AS, Blencoe AG, Cafferty TP. (1985). The role of appraisal purpose: Effects of purpose information acquisition and utilization. Organizational Behavior and Human Decision Processes, 35, 314-339.