Performance appraisal and performance management: 100 years of progress?

Journal of Applied Psychology

Volumn 102, Issue 3, 2017, Pages 421-433

  Denisi, Angelo S ^a   Murphy, Kevin R ^b  

^a TULANE UNIVERSITY   (United States)

^b UNIVERSITY OF LIMERICK   (Ireland)

Author keywords

Appraisal; Criteria; Evaluation; Performance; Ratings

Indexed keywords

MODEL; PSYCINFO; HISTORY; HUMAN; PERSONNEL MANAGEMENT; PROCEDURES; PSYCHOLOGY; RESEARCH;

EMPLOYEE PERFORMANCE APPRAISAL; HISTORY, 20TH CENTURY; HISTORY, 21ST CENTURY; HUMANS; PERSONNEL MANAGEMENT; PSYCHOLOGY, APPLIED; RESEARCH;

EID: 85010281728     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/apl0000085     Document Type: Article

References (136)

1. Aguinis, H. (2013). Performance management (3rd ed.). Upper Saddle River, NJ: Pearson/Prentice Hall.
2. Aguinis, H., & Pierce, C. A. (2008). Enhancing the relevance of organizational behavior by embracing performance management research. Journal of Organizational Behavior, 29, 139-145. http://dx.doi.org/10 .1002/job.493
3. Aryee, S., Walumbwa, F. O., Seidu, E. Y., & Otaye, L. E. (2012). Impact of high-performance work systems on individual- and branch-level performance: Test of a multilevel model of intermediate linkages. Journal of Applied Psychology, 97, 287-300. http://dx.doi.org/10.1037/ a0025739
4. Athey, T. R., & McIntyre, R. M. (1987). Effect of rater training on rater accuracy: Levels-of-processing theory and social facilitation theory perspectives. Journal of Applied Psychology, 72, 567-572. http://dx.doi .org/10.1037/0021-9010.72.4.567
5. Barrett, M. A. (1914). A comparison of the order of merit method and the method of paired comparisons. Psychological Review, 21, 278-294. http://dx.doi.org/10.1037/h0075829
6. Bass, A. R., & Turner, J. N. (1973). Ethnic group differences in relationships among criteria of job performance. Journal of Applied Psychology, 57, 101-109. http://dx.doi.org/10.1037/h0037125
7. Bendig, A. W. (1953). The reliability of self-ratings as a function of the amount of verbal anchoring and the number of categories on the scale. Journal of Applied Psychology, 37, 38-41. http://dx.doi.org/10.1037/ h0057911
8. Berkshire, J. R., & Highland, R. W. (1953). Forced-choice performance rating: A methodological study. Personnel Psychology, 6, 355-378. http://dx.doi.org/10.1111/j.1744-6570.1953.tb01503.x
9. Bernardin, H. J., & Buckley, M. R. (1981). Strategies in rater training. Academy of Management Review, 6, 205-212.
10. Bernardin, H. J., Cardy, R. L., & Carlyle, J. J. (1982). Cognitive complexity and appraisal effectiveness: Back to the drawing board? Journal of Applied Psychology, 66, 151-160.
11. Bernardin, H. J., & Pence, E. C. (1980). Effects of rater training: Creating new response sets and decreasing accuracy. Journal of Applied Psychology, 65, 60-66. http://dx.doi.org/10.1037/0021-9010.65.1.60
12. Bernardin, H. J., & Smith, P. C. (1981). A clarification of some issues regarding the development and use of Behaviorally Anchored Ratings Scales (BARS). Journal of Applied Psychology, 66, 458-463. http://dx .doi.org/10.1037/0021-9010.66.4.458
13. Bernardin, H. J., & Walter, C. S. (1977). Effects of rater training and diary-keeping on psychometric error in ratings. Journal of Applied Psychology, 62, 64-69. http://dx.doi.org/10.1037/0021-9010.62.1.64
14. Bingham, W. V. (1939). Halo, invalid and valid. Journal of Applied Psychology, 23, 221-228. http://dx.doi.org/10.1037/h0060918
15. Blanz, F., & Ghiselli, E. E. (1972). The Mixed Standard Rating Scale: A new rating system. Personnel Psychology, 25, 185-199. http://dx.doi .org/10.1111/j.1744-6570.1972.tb01098.x
16. Borman, W. C. (1975). Effects of instructions to avoid halo error on reliability and validity of performance evaluation ratings. Journal of Applied Psychology, 60, 556-560. http://dx.doi.org/10.1037/0021-9010 .60.5.556
17. Borman, W. C. (1977). Consistency of rating accuracy and rating errors in the judgment of human performance. Organizational Behavior & Human Performance, 20, 238 -252. http://dx.doi.org/10.1016/0030- 5073(77)90004-6
18. Borman, W. C. (1979). Format and training effects on rating accuracy and rater errors. Journal of Applied Psychology, 64, 410-421. http://dx.doi .org/10.1037/0021-9010.64.4.410
19. Bowen, C., Swim, J. K., & Jacobs, R. (2000). Evaluating gender biases on actual job performance of real people: A meta-analysis. Journal of Applied Social Psychology, 30, 2194-2215.
20. Buckner, D. N. (1959). The predictability of ratings as a function of interrater agreement. Journal of Applied Psychology, 43, 60-64. http:// dx.doi.org/10.1037/h0047296
21. Butler, R. P., & Jaffee, C. L. (1974). Effects of incentive, feedback and manner of presenting the feedback on leader behavior. Journal of Applied Psychology, 59, 332-336. http://dx.doi.org/10.1037/h0036655
22. Camman, C., & Lawler, E. E. (1973). Employee reactions to a pay incentive plan. Journal of Applied Psychology, 58, 163-172.
23. Cattell, J. M. (1906). American men of science: A biographical dictionary. New York, NY: Science Press.
24. Cawley, B. D., Keeping, L. M., & Levy, P. E. (1998). Participation in the performance appraisal process and employee reactions: A meta-analytic review of field investigations. Journal of Applied Psychology, 83, 615-634. http://dx.doi.org/10.1037/0021-9010.83.4.615
25. Champney, H. (1941). The measurement of parent behavior. Child Development, 12, 131-166.
26. Cleveland, J. N., Murphy, K. R., & Williams, R. (1988). Multiple uses of performance appraisal: Prevalence and correlates. Journal of Applied Psychology, 74, 130 -135. http://dx.doi.org/10.1037/0021-9010.74.1 .130
27. Cooper, W. H. (1981). Conceptual similarity as a source of illusory halo in job performance ratings. Journal of Applied Psychology, 66, 302-307. http://dx.doi.org/10.1037/0021-9010.66.3.302
28. Cronbach, L. J. (1955). Processes affecting scores on understanding of others and assumed similarity. Psychological Bulletin, 52, 177-193. http://dx.doi.org/10.1037/h0044919
29. Crook, T. R., Todd, S. Y., Combs, J. G., Woehr, D. J., & Ketchen, D. J., Jr. (2011). Does human capital matter? A meta-analysis of the relationship between human capital and firm performance. Journal of Applied Psychology, 96, 443-456. http://dx.doi.org/10.1037/a0022147
30. Dejung, J. E., & Kaplan, H. (1962). Some differential effects of race of rater and rate on early peer ratings of combat aptitude. Journal of Applied Psychology, 46, 370-374. http://dx.doi.org/10.1037/h0048376 DeNisi, A. S. (2006). A cognitive approach to performance appraisal. New York, NY: Routledge.
31. DeNisi, A. S., Cafferty, T., & Meglino, B. (1984). A cognitive view of the performance appraisal process: A model and research propositions. Organizational Behavior & Human Performance, 33, 360-396. http:// dx.doi.org/10.1016/0030-5073(84)90029-1
32. DeNisi, A. S., & Kluger, A. N. (2000). Feedback effectiveness: Can 360-degreee appraisals be improved? The Academy of Management Executive, 14, 129-139.
33. DeNisi, A. S., & Peters, L. H. (1996). Organization of information in memory and the performance appraisal process: Evidence from the field. Journal of Applied Psychology, 81, 717-737. http://dx.doi.org/10.1037/ 0021-9010.81.6.717
34. DeNisi, A. S., & Pritchard, R. D. (2006). Improving individual performance: A motivational framework. Management and Organization Review, 2, 253-277. http://dx.doi.org/10.1111/j.1740-8784.2006.00042.x
35. DeNisi, A. S., Robbins, T., & Cafferty, T. P. (1989). Organization of information used for performance appraisals: Role of diary-keeping. Journal of Applied Psychology, 74, 124-129. http://dx.doi.org/10.1037/ 0021-9010.74.1.124
36. DeNisi, A. S., & Smith, C. E. (2014). Performance appraisal, performance management, and firm-level performance: A review, a proposed model, and new directions for future research. The Academy of Management Annals, 8, 127-179. http://dx.doi.org/10.1080/19416520.2014.873178
37. Dickinson, Z. C. (1937). Validity and independent criteria. Journal of Applied Psychology, 21, 522-527. http://dx.doi.org/10.1037/h0062095
38. Feldman, J. M. (1981). Beyond attribution theory: Cognitive processes in performance appraisal. Journal of Applied Psychology, 66, 127-148. http://dx.doi.org/10.1037/0021-9010.66.2.127
39. Ferguson, L. W. (1947). The development of a method of appraisal for assistant managers. Journal of Applied Psychology, 31, 306-311. http:// dx.doi.org/10.1037/h0057937
40. Fiske, D. W., & Cox, J. A. (1960). The consistency of ratings by peers. Journal of Applied Psychology, 44, 11-17. http://dx.doi.org/10.1037/ h0046278
41. Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51, 327-358. http://dx.doi.org/10.1037/h0061470
42. Folger, R., Konovsky, M. A., & Cropanzano, R. (1992). A due process metaphor for performance appraisal. Research in Organizational Behavior, 14, 127-148.
43. Gorman, C. A., & Rentsch, J. R. (2009). Evaluating frame-of-reference rater training effectiveness using performance schema accuracy. Journal of Applied Psychology, 94, 1336 -1344. http://dx.doi.org/10.1037/ a0016476
44. Greguras, G. J., & Robie, C. (1998). A new look at within-source interrater reliability of 360-degree feedback ratings. Journal of Applied Psychology, 83, 960-968. http://dx.doi.org/10.1037/0021-9010.83.6.960
45. Guilford, J. P., & Jorgensen, A. P. (1938). Some constant errors in ratings. Journal of Experimental Psychology, 22, 43-57. http://dx.doi.org/10 .1037/h0061118
46. Hamner, W. C., Kim, J. S., Baird, L., & Bigoness, W. J. (1974). Race and sex as determinants of ratings by potential employers in a simulated work-sampling task. Journal of Applied Psychology, 59, 705-711. http:// dx.doi.org/10.1037/h0037503
47. Hedge, J. W., & Kavanagh, M. J. (1988). Improving the accuracy of performance evaluations: Comparison of three methods of performance appraiser training. Journal of Applied Psychology, 73, 68-73. http://dx .doi.org/10.1037/0021-9010.73.1.68
48. Heidemeier, H., & Moser, K. (2009). Self-other agreement in job performance ratings: A meta-analytic test of a process model. Journal of Applied Psychology, 94, 353-370. http://dx.doi.org/10.1037/0021-9010 .94.2.353
49. Hollander, E. P. (1957). The reliability of peer nominations under various conditions of administration. Journal of Applied Psychology, 41, 85-90.
50. Hollander, E. P. (1965). Validity of peer nominations in predicting a distant performance criterion. Journal of Applied Psychology, 49, 434-438. http://dx.doi.org/10.1037/h0022805
51. Huselid, M. A. (1995). The impact of human resource practices on turnover, productivity, and corporate financial performance. Academy of Management Journal, 38, 635-672. http://dx.doi.org/10.2307/256741
52. Ilgen, D. R. (1993). Performance appraisal accuracy: An illusive and sometimes misguided goal. In H. Schuler, J. L. Farr, & M. Smith (Eds.), Personnel selection and assessment: Industrial and organizational perspectives (pp. 235-252). Hillsdale, NJ: Erlbaum.
53. Ilgen, D. R., Barnes-Farrell, J. L., & McKellin, D. B. (1993). Performance appraisal process research in the 1980s: What has it contributed to appraisals in use? Organizational Behavior and Human Decision Processes, 54, 321-368. http://dx.doi.org/10.1006/obhd.1993.1015
54. Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual feedback on behavior in organizations. Journal of Applied Psychology, 64, 349 -371. http://dx.doi.org/10.1037/0021-9010.64.4 .349
55. Ivancevich, J. M. (1979). Longitudinal study of the effects of rater training on psychometric error in ratings. Journal of Applied Psychology, 64, 502-508. http://dx.doi.org/10.1037/0021-9010.64.5.502
56. Jackson, S. E., Schuler, R. S., & Jiang, K. (2014). Strategic human resource management: A review and aspirational framework. Academy of Management Annals, 8, 1-56.
57. Kavanagh, M., MacKinney, A., & Wolins, L. (1971). Issues in managerial performance: Multitrait-multimethod analysis of ratings. Psychological Bulletin, 75, 34-49. http://dx.doi.org/10.1037/h0030412
58. Kay, B. R. (1959). The use of critical incidents in a forced-choice scale. Journal of Applied Psychology, 43, 269-270. http://dx.doi.org/10.1037/ h0045921
59. Kinicki, A. J., Hom, P. W., Trost, M. R., & Wade, K. J. (1995). Effects of category prototypes on performance-rating accuracy. Journal of Applied Psychology, 80, 354 -370. http://dx.doi.org/10.1037/0021-9010.80.3 .354
60. Kinicki, A. J., Jacobson, K. J., Peterson, S. J., & Prussia, G. E. (2013). Development and validation of a performance management behavior questionnaire. Personnel Psychology, 66, 1-45. http://dx.doi.org/10 .1111/peps.12013
61. Kluger, A. N., & DeNisi, A. S. (1996). The effects of feedback interventions on performance: A historical review, meta-analysis and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254-287. http://dx.doi.org/10.1037/0033-2909.119.2.254
62. Knauft, E. B. (1948). Construction and use of weighted check list rating scales for two industrial situations. Journal of Applied Psychology, 32, 63-70. http://dx.doi.org/10.1037/h0060388
63. Kozlowski, S. W., & Kirsch, M. P. (1987). The systematic distortion hypothesis, halo, and accuracy: An individual-level analysis. Journal of Applied Psychology, 72, 252-261. http://dx.doi.org/10.1037/0021-9010 .72.2.252
64. Kraiger, K., & Ford, J. K. (1985). A meta-analysis of ratee race effects in performance ratings. Journal of Applied Psychology, 70, 56-65. http:// dx.doi.org/10.1037/0021-9010.70.1.56
65. Landy, F. J., Barnes, J. L., & Murphy, K. R. (1978). Correlates of perceived fairness and accuracy of performance evaluations. Journal of Applied Psychology, 63, 751-754. http://dx.doi.org/10.1037/0021-9010 .63.6.751
66. Landy, F. J., Barnes-Farrell, J., & Cleveland, J. N. (1980). Perceived fairness and accuracy of performance evaluation: A follow-up. Journal of Applied Psychology, 65, 355-356. http://dx.doi.org/10.1037/0021- 9010.65.3.355
67. Landy, F. J., & Farr, J. L. (1980). Performance rating. Psychological Bulletin, 87, 72-107. http://dx.doi.org/10.1037/0033-2909.87.1.72
68. Landy, F. J., Vance, R. J., Barnes-Farrell, J. L., & Steele, J. W. (1980). Statistical control of halo error in performance ratings. Journal of Applied Psychology, 65, 501-506. http://dx.doi.org/10.1037/0021-9010 .65.5.501
69. Latham, G. P., & Wexley, K. N. (1977). Behavior observation scales for performance appraisal purposes. Personnel Psychology, 30, 255-268. http://dx.doi.org/10.1111/j.1744-6570.1977.tb02092.x
70. Lawler, E. E. (1967). The multitrait-multirater approach to studying managerial job performance. Journal of Applied Psychology, 51, 369-381. http://dx.doi.org/10.1037/h0025096
71. Lepkowski, J. R. (1963). Development of a forced-choice rating scale for engineer evaluations. Journal of Applied Psychology, 47, 87-88. http:// dx.doi.org/10.1037/h0044908
72. Levine, J., & Butler, J. (1952). Lecture vs. group decision in changing behavior. Journal of Applied Psychology, 36, 29-33. http://dx.doi.org/ 10.1037/h0053624
73. Levy, P. E., & Williams, J. R. (2004). The social context of performance appraisal: A review and framework for the future. Journal of Management, 30, 881-905. http://dx.doi.org/10.1016/j.jm.2004.06.005
74. London, M., & Smither, J. W. (1995). Can multi-source feedback change perceptions of goal accomplishment, self-evaluations, and performancerelated outcomes? Theory-based applications and directions for research. Personnel Psychology, 48, 803-839. http://dx.doi.org/10.1111/j.1744- 6570.1995.tb01782.x
75. Lord, R. G. (1985). Accuracy in behavioral measurement: An alternative definition based on raters' cognitive schema and signal detection theory. Journal of Applied Psychology, 70, 66-71. http://dx.doi.org/10.1037/ 0021-9010.70.1.66
76. McCormick, E. J., & Roberts, W. K. (1952). Paired comparison ratings: II. The reliability of ratings based on partial pairings. Journal of Applied Psychology, 36, 188-192. http://dx.doi.org/10.1037/h0055956
77. Meyer, H. H. (1951). Methods for scoring a check-list type of rating scale. Journal of Applied Psychology, 35, 46-49. http://dx.doi.org/10.1037/ h0055890
78. Meyer, H. H., Kay, E., & French, J. (1965). Split roles in performance appraisal. Harvard Business Review, 43, 123-129.
79. Mullins, C. J., & Force, R. C. (1962). Rater accuracy as a generalized ability. Journal of Applied Psychology, 46, 191-193. http://dx.doi.org/ 10.1037/h0044264
80. Murphy, K. R. (1982). Difficulties in the statistical control of halo. Journal of Applied Psychology, 67, 161-164. http://dx.doi.org/10.1037/0021- 9010.67.2.161
81. Murphy, K. R., & Balzer, W. K. (1986). Systematic distortions in memorybased behavior ratings and performance evaluations: Consequences for rating accuracy. Journal of Applied Psychology, 71, 39-44. http://dx .doi.org/10.1037/0021-9010.71.1.39
82. Murphy, K. R., & Balzer, W. K. (1989). Rater errors and rating accuracy. Journal of Applied Psychology, 74, 619-624. http://dx.doi.org/10.1037/ 0021-9010.74.4.619
83. Murphy, K. R., & Cleveland, J. N. (1991). Performance appraisal: An organizational perspective. Needham Heights, MA: Allyn & Bacon.
84. Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance appraisal: Social, organizational and goal-oriented perspectives. Newbury Park, CA: Sage.
85. Murphy, K. R., Cleveland, J. N., Skattebo, A. L., & Kinney, T. B. (2004). Raters who pursue different goals give different ratings. Journal of Applied Psychology, 89, 158-164. http://dx.doi.org/10.1037/0021-9010 .89.1.158
86. Murphy, K. R., & Constans, J. I. (1987). Behavioral anchors as a source of bias in rating. Journal of Applied Psychology, 72, 573-577. http://dx.doi .org/10.1037/0021-9010.72.4.573
87. Murphy, K. R., & DeNisi, A. (2008). A model of the performance appraisal process. In A. Varma, P. Budhwar, & A. DeNisi (Eds.), Performance management systems around the globe (pp. 81-96). London, UK: Routledge.
88. Murphy, K. R., Garcia, M., Kerkar, S., Martin, C., & Balzer, W. (1982). The relationship between observational accuracy and accuracy in evaluating performance. Journal of Applied Psychology, 67, 320-325. http:// dx.doi.org/10.1037/0021-9010.67.3.320
89. Murphy, K. R., Herr, B. M., Lockhart, M. C., & Maguire, E. (1986). Evaluating the performance of paper people. Journal of Applied Psychology, 71, 654-661. http://dx.doi.org/10.1037/0021-9010.71.4.654
90. Murphy, K. R., Martin, C., & Garcia, M. (1982). Do behavioral observation scales measure observation? Journal of Applied Psychology, 67, 562-567. http://dx.doi.org/10.1037/0021-9010.67.5.562
91. Nathan, B. R., & Lord, R. G. (1983). Cognitive categorization and dimensional schemata: A process approach to the study of halo in performance ratings. Journal of Applied Psychology, 68, 102-114. http://dx.doi.org/ 10.1037/0021-9010.68.1.102
92. Northcraft, G. B., Schmidt, A. M., & Ashford, S. J. (2011). Feedback and the rationing of time and effort among competing tasks. Journal of Applied Psychology, 96, 1076 -1086. http://dx.doi.org/10.1037/ a0023221
93. O'Boyle, E., Jr., & Aguinis, H. (2012). The best and the rest: Revisiting the norm of normality of individual performance. Personnel Psychology, 65, 79-119. http://dx.doi.org/10.1111/j.1744-6570.2011.01239.x
94. Obradovic, J. (1970). Modification of the forced choice method as a criterion for job proficiency. Journal of Applied Psychology, 54, 228-298. http://dx.doi.org/10.1037/h0029264
95. Paterson, D. G. (1922). The Scott Company Graphic Rating Scale. Journal of Personnel Research, 1, 361-376.
96. Ployhart, R. E., & Moliterno, T. P. (2011). Emergence of the human capital resource: A multilevel model. Academy of Management Review, 36, 127-150.
97. Podsakoff, P. M., Ahearne, M., & MacKenzie, S. B. (1997). Organizational citizenship behavior and the quantity and quality of work group performance. Journal of Applied Psychology, 82, 262-270. http://dx.doi.org/ 10.1037/0021-9010.82.2.262
98. Pritchard, R. D., Harrell, M. M., DiazGranados, D., & Guzman, M. J. (2008). The productivity measurement and enhancement system: A meta-analysis. Journal of Applied Psychology, 93, 540-567. http://dx .doi.org/10.1037/0021-9010.93.3.540
99. Pritchard, R. D., Jones, S. D., Roth, P. L., Stuebing, K. K., & Ekeberg, S. E. (1988). Effects of group feedback, goal setting, and incentives on organizational productivity. Journal of Applied Psychology, 73, 337-358. http://dx.doi.org/10.1037/0021-9010.73.2.337
100. Pulakos, E. D. (1984). A comparison of rater training programs: Error training and accuracy training. Journal of Applied Psychology, 69, 581-588. http://dx.doi.org/10.1037/0021-9010.69.4.581
101. Pulakos, E. D., Mueller-Hanson, R. A., O'Leary, R. S., & Meyrowitz, M. M. (2012). Building a high-performance culture: A fresh look at performance management. SHRM Foundation Effective Practice Guidelines Series. Alexandria, VA: SHRM Foundation.
102. Pulakos, E. D., White, L. A., Oppler, S. H., & Borman, W. C. (1989). Examination of race and sex effects on performance ratings. Journal of Applied Psychology, 74, 770-780. http://dx.doi.org/10.1037/0021-9010 .74.5.770
103. Ream, M. J. (1921). A statistical method for incomplete ordering of merit ratings. Journal of Applied Psychology, 5, 261-266. http://dx.doi.org/10 .1037/h0066886
104. Remmers, H. H. (1931). Reliability and halo effect of high school and college students' judgments of their teachers. Journal of Applied Psychology, 18, 619-630. http://dx.doi.org/10.1037/h0074783
105. Robbins, T. L., & DeNisi, A. S. (1994). A closer look at interpersonal affect as a distinct influence on cognitive processing in performance evaluations. Journal of Applied Psychology, 79, 341-353. http://dx.doi .org/10.1037/0021-9010.79.3.341
106. Ronan, W. W., Latham, G. P., & Kinne, S. B. (1973). Effects of goal setting and supervision on worker behavior in an industrial situation. Journal of Applied Psychology, 58, 302-307. http://dx.doi.org/10.1037/ h0036303
107. Rosen, B., & Jerdee, T. H. (1976). The nature of job-related age stereotypes. Journal of Applied Psychology, 61, 180-183. http://dx.doi.org/10 .1037/0021-9010.61.2.180
108. Rothe, H. C., & Nye, C. T. (1958). Output rates among coil winders. Journal of Applied Psychology, 42, 182-186. http://dx.doi.org/10.1037/ h0041121
109. Rugg, H. (1921). Is the rating of human character practicable? Journal of Educational Psychology, 12, 425-438, 485-501.
110. Saal, F. E. (1979). Mixed Standard Rating Scale: A consistent system for numerically coding inconsistent response combinations. Journal of Applied Psychology, 64, 422-428. http://dx.doi.org/10.1037/0021-9010.64 .4.422
111. Sackett, P. R., & DuBois, C. L. (1991). Rater-ratee race effects on performance evaluation: Challenging meta-analytic conclusions. Journal of Applied Psychology, 76, 873-877. Retrieved from http://psycnet.apa .org/journals/apl/76/6/873/
112. Sackett, P. R., DuBois, C. L., & Noe, A. W. (1991). Tokenism in performance evaluation: The effects of work group representation on malefemale and White-Black differences in performance ratings. Journal of Applied Psychology, 76, 263-267. Retrieved from http://psycnet.apa.org/ journals/apl/76/2/263/
113. Schmitt, N., & Lappin, M. (1980). Race and sex as determinants of the mean and variance of performance ratings. Journal of Applied Psychology, 65, 428-435. http://dx.doi.org/10.1037/0021-9010.65.4.428
114. Schwab, D. P., & Heneman, H. G. (1978). Age stereotyping in performance appraisal. Journal of Applied Psychology, 63, 573-578. http://dx .doi.org/10.1037/0021-9010.63.5.573
115. Scullen, S. E., Bergey, P. K., & Aiman-Smith, L. (2005). Forced distribution rating systems and the improvement of workforce potential. Personnel Psychology, 58, 1-32. http://dx.doi.org/10.1111/j.1744-6570 .2005.00361.x
116. Seifert, C. F., Yukl, G., & McDonald, R. A. (2003). Effects of multisource feedback and a feedback facilitator on the influence behavior of managers toward subordinates. Journal of Applied Psychology, 88, 561-569. http://dx.doi.org/10.1037/0021-9010.88.3.561
117. Sisson, E. D. (1948). Forced choice: The new rating. Personnel Psychology, 1, 365-381. http://dx.doi.org/10.1111/j.1744-6570.1948.tb01316.x
118. Smith, P. C., & Kendall, L. M. (1963). Retranlsation of expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Psychology, 47, 149-155. http://dx.doi.org/10.1037/ h0047060
119. Smither, J. W., London, M., & Reilly, R. R. (2005). Does performance improve following multisource feedback? A theoretical model, metaanalysis, and review of empirical findings. Personnel Psychology, 58, 33-66.
120. Springer, D. (1953). Ratings of candidates for promotion by co-workers and supervisors. Journal of Applied Psychology, 37, 347-351.
121. Sulsky, L. M., & Balzer, W. K. (1988). Meaning and measurement of performance rating accuracy: Some methodological and theoretical concerns. Journal of Applied Psychology, 73, 497-506.
122. Taylor, M. S., Tracy, K. B., Renard, M. B., Harrison, J. K., & Carroll, S. J. (1995). Due process in performance appraisal: A quasi-experiment in procedural justice. Administrative Science Quarterly, 40, 495-523. http://dx.doi.org/10.2307/2393795
123. Thorndike, E. L. (1920). A constant error in psychological ratings. Journal of Applied Psychology, 4, 25-29. http://dx.doi.org/10.1037/h0071663
124. Thornton, G. C., & Zorich, S. (1980). Training to improve observer accuracy. Journal of Applied Psychology, 65, 351-354. http://dx.doi.org/ 10.1037/0021-9010.65.3.351
125. Travers, R. M. (1951). A critical review of the validity and rationale of the forced-choice technique. Psychological Bulletin, 48, 62-70. http://dx.doi .org/10.1037/h0055263
126. Uggerslev, K. L., & Sulsky, L. M. (2008). Using frame-of-reference training to understand the implications of rater idiosyncrasy for rating accuracy. Journal of Applied Psychology, 93, 711-719. http://dx.doi.org/ 10.1037/0021-9010.93.3.711
127. Vancouver, J. B., & Tischner, E. C. (2004). The effect of feedback sign on task performance depends on self-concept discrepancies. Journal of Applied Psychology, 89, 1092-1098. http://dx.doi.org/10.1037/0021- 9010.89.6.1092
128. Varma, A., Budhwar, P., & DeNisi, A. (2008). Performance management systems around the globe. London, UK: Routledge.
129. Waldman, D. A., & Avolio, B. J. (1991). Race effects in performance evaluations: Controlling for ability, education, and experience. Journal of Applied Psychology, 76, 897-901. http://dx.doi.org/10.1037/0021- 9010.76.6.897
130. Wendelken, D. J., & Inn, A. (1981). Nonperformance influences on performance evaluations: A laboratory phenomenon? Journal of Applied Psychology, 66, 149 -158. http://dx.doi.org/10.1037/0021-9010.66.2 .149
131. Wexley, K. N., Singh, J. P., & Yukl, G. (1973). Subordinate personality as a moderator of the effects of participation in three types of appraisal interview. Journal of Applied Psychology, 58, 54-59. http://dx.doi.org/ 10.1037/h0035411
132. Wiley, L., & Jenkins, W. S. (1964). Selecting competent raters. Journal of Applied Psychology, 48, 215-217. http://dx.doi.org/10.1037/h0045601
133. Williams, K. J., Cafferty, T. P., & DeNisi, A. S. (1990). Effects of performance appraisal salience on recall and ratings. Organizational Behavior and Human Decision Processes, 46, 217-239.
134. Williams, K. J., DeNisi, A. S., Blencoe, A. G., & Cafferty, T. P. (1985). The effects of appraisal purpose on information acquisition and utilization. Organizational Behavior and Human Decision Processes, 36, 314-339. http://dx.doi.org/10.1016/0749-5978(85)90027-5
135. Woehr, D. J. (1994). Understanding frame-of-reference training: The impact of training on the recall of performance information. Journal of Applied Psychology, 79, 525-534. http://dx.doi.org/10.1037/0021-9010 .79.4.525
136. Zedeck, S., & Cascio, W. F. (1982). Performance appraisal decisions as a function of rater training and purpose of the appraisal. Journal of Applied Psychology, 67, 752-758. http://dx.doi.org/10.1037/0021-9010 .67.6.752