Discussion and consensus in rater groups: Implications for behavioral and rating accuracy

Human Performance

Volumn 19, Issue 2, 2006, Pages 91-115

  Roch, Sylvia G ^a  

^a STATE UNIVERSITY OF NEW YORK   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33751582706     PISSN: 08959285     EISSN: None     Source Type: Journal    
DOI: 10.1207/s15327043hup1902_1     Document Type: Article

References (77)

1. Banks, C. G., & Murphy, K. R. (1985). Toward narrowing the research-practice gap in performance evaluation. Personnel Psychology, 38, 335-345.
2. Bem, D. J., Wallach, M. A., & Kogan, N. (1965). Group decision making under risk of aversive consequences. Journal of Personality and Social Psychology, 1, 453-460.
3. Borman, W. C. (1982). Validity of behavioral assessment for predicting military recruiter performance. Journal of Applied Psychology, 67, 3-9.
4. Chalos, P., & Pickard, D. (1985). Information choice and cue use: An experiment in group information processing. Journal of Applied Psychology, 70, 634-641.
5. Clark, N. K., & Stephenson, G. M. (1989). Group remembering. In P. Paulus (Ed.), Psychology of group influence (2nd ed., pp. 357-391). Hillsdale, NJ: Lawrence Erbaum Associates, Inc.
6. Clark, S. E., Hori, A., Putnam, A., & Martin, T. P. (2000). Group collaboration in recognition memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26, 1578-1588.
7. Cleveland, J. N., Murphy, K. R., & Williams, R. E. (1989). Multiple uses of performance evaluation: Prevalence and correlates. Journal of Applied Psychology, 74, 130-135.
8. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale. NJ: Lawrence Erlbaum Associates, Inc.
9. Corrigan, D. K., & Hauenstein, N. M. A. (1996, April). Accountability effects on cognitive complexity and accuracy on performance ratings. Paper presented at the 11th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
10. Cronbach, L. (1955). Process affecting scores on "understanding of others" and "assumed similarity." Psychological Bulletin, 52, 177-193.
11. DeNisi, A. A., & Williams, K. J. (1988). Cognitive approaches to performance evaluation. In K. M. Rowland & G. R. Ferris (Eds.), Research in personnel and human resource management: Vol. 6 (pp. 109-155). Greenwich. CT: JAI.
12. Diehl, M. & Stroebe, W. (1987). Productivity loss in brainstorming groups: Toward the solution of the riddle. Journal of Personality and Social Psychology, 53, 497-509.
13. Dipboye, R. L (1990). Laboratory vs. field research in industrial and organizational psychology. In C. L. Cooper & I. T. Robertson (Eds.), International review of industrial and organizational psychology: Vol. 5 (pp. 1-34). Oxford, England: John Wiley & Sons.
14. Feltham, R. (1988). Assessment centre decision making: Judgmental vs. mechanical. Journal of Occupational Psychology, 61, 237-241.
15. Gilbert, P. J. (1981). An investigation of clinical and mechanical combinations of assessment center data. Journal of Assessment Center Technology, 4, 1-10.
16. Guzzo, R. A. (1986). Group decision making and group effectiveness in organizations. In P. Goodman (Ed), Designing effective work groups (pp. 34-71). San Francisco: Jossey-Bass.
17. Harris, M. M. (1994). Rater motivation in the performance evaluation context: A theoretical framework. Journal of Management, 20, 737-756.
18. Hartwick, J., Sheppard, B. H., & Davis. J. H. (1982). Group remembering: Research and implications. In R. A. Guzzo (Ed.), Improving group decision making in organizations (pp. 41-72). San Diego, CA: Academic.
19. Hastie, R., & Park, B. (1986). The relationship between memory and judgment depends on whether the judgment task is memory based or on-line. Psychological Review, 93, 256-268.
20. Hill. G. W. (1982). Group versus individual performance: Are N+1 heads better than one. Psychological Bulletin, 91, 677-686.
21. Hinsz, V. B. (1990). Cognitive and consensus processes in group recognition memory performance. Journal of Personality and Social Psychology, 59, 705-718.
22. Hinsz, V. B. (1992). Social influences on the goal choices of group members. Journal of Applied Social Psychology, 22, 1297-1317.
23. Hinsz, V. B., Tindale, R. S., & Vollrath, D. A. (1997). The emerging conceptualization of groups as information processors. Psychological Bulletin. 121, 43-64.
24. Hollowman, C. R., & Hendrick, H. W. (1972). Adequacy of group decisions as a function of the decision making process. Academy of Management Journal, 15, 175-184.
25. Ilgen, D. R., Barnes-Farrell, J. L., & McKellin, D. B. (1993). Performance appraisal process research in the 1980s: What has it contributed to appraisals in use? Organizational Behavior and Human Decision Processes, 54, 321-368.
26. Ilgen, D. R. & Feldman, J. M. (1983). Performance appraisal: A process focus. Research in Organizational Behavior, 5, 141-197.
27. Ilgen, D. R., & Knowlton, W. A., Jr. (1980). Performance attributional effects on feedback from superiors. Organizational Behavior and Human Performance, 25, 441-456.
28. Jawahar, I. M., & Williams, C. R. (1997). Where all the children are above average: The performance evaluation purpose effect. Personnel Psychology, 50, 905-925.
29. Jones, R. G. (1992). Attention allocation choices as an influence on accuracy and discriminant validity of observational ratings: When are we diligent? Unpublished doctoral dissertation, Ohio State University, Columbus, OH.
30. Kleiman, L. S., Lounsbury, J. W., & Faley, R. H. (1987). Performance evaluation: A case for the use of consensus ratings. Psychological Reports, 60, 407-413.
31. Klimoski, R., & Inks, L. (1990). Accountability forces in performance evaluation. Organizational Behavior and Human Decision Processes, 45, 194-208.
32. Lam, S. S. K., & Schaubroeck, J. (1999). Total quality management and performance evaluation: An experimental study of process versus results and group versus individual approaches. Journal of Organizational Behavior, 20, 445-457.
33. Lerner, J. S., & Tetlock, P. E. (1999). Accounting for the effects of accountability. Psychological Bulletin, 125, 255-275.
34. Longenecker, C. O., Sims, H. P, & Gioia, D. A. (1987). Behind the mask: The politics of employee appraisal. The Academy of Management Executive, 1, 183-193.
35. McCauley, D. P., Jago, I. A., Gore, B., Lance, C. E., Quartes, F. K., Sledge, L., et al. (1990, April). The development of more ecologically valid videotapes for use in performance evaluation research. Paper presented at the annual meeting of the Southeastern Psychological Association, Atlanta, GA.
36. McEvoy, G. M., Beatty, R. W., & Bernardin, H. J. (1987). Unanswered questions in assessment center research. Journal of Business and Psychology, 2, 97-111.
37. Martell, R. F., & Borg, M. R. (1993). A comparison of the behavioral rating accuracy of groups and individuals. Journal of Applied Psychology, 78, 43-50.
38. Martell, R. F., & Leavitt, K. N. (2002), Reducing the performance-cue bias in work behavior ratings: Can groups help. Journal of Applied Psychology, 87, 1032-1041.
39. Mero, N. P., & Motowidlo, S. J. (1995). Effects of rater accountability on the accuracy and favorability of performance ratings. Journal of Applied Psychology, 80, 517-524.
40. Miller, C. E. (1989). The social psychological effects of group decision rules. In P. Paulus (Ed.), The psychology of group influence (pp. 327-355). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
41. Mook, D. G. (1983). In defense of external invalidity. American Psychologist, 38, 379-387.
42. Mullen, B., Johnson, C., & Salas, E. (1991). Productivity loss in brainstorming groups: A meta-analytic integration. Basic and Applied Social Psychology, 12, 3-23.
43. Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance evaluation: Social, organizational, and goal-based perspectives. Thousand Oaks, CA: Sage.
44. Murphy, K. R., Garcia, M., Kerkar, S., Martin, C., & Balzer, W. K. (1982). Relationship between observational accuracy and accuracy in evaluating performance. Journal of Applied Psychology, 67, 320-325.
45. Nemeth, C. J., Personnaz., B., Personnaz, M., & Goncalo, J. A. (2004). The liberating role of conflict in group creativity: A study in two countries. European Journal of Social Psychology, 34, 365-374.
46. Paulus, P. B., & Murdoch, P. (1971). Anticipated evaluation and audience presence in the enhancement of dominant responses. Journal of Experimental Social Psychology, 7, 280-291.
47. Paulus, P. B., Larey, T. S., & Dzindolet, M. T. (2001). Creativity in groups and teams. In M. Turner (Ed), Groups at work: Theory and research (pp. 319-338). Mahwah, NJ: Lawrence Erbaum Associates, Inc.
48. Pulakos, E. D., Schmitt, N., Whitney, D., & Smith, M. (1996). Individual differences in interviewer ratings: The impact of standardization, consensus discussion, and sampling error on the validity of a structured interview. Personnel Psychology, 49, 85-102.
49. Pynes, J., & Bernardin, H. J. (1992). Mechanical vs. consensus derived assessment center ratings: A comparison of job performance validities. Public Personnel Management, 21, 17-28.
50. Pynes, J., Bernardin, J. J., Benton, A. L., & McEvoy, G. M. (1988). Should assessment center ratings be mechanically-derived? Journal of Business and Psychology, 2, 217-227.
51. Roach, D. W., & Gupta, N. (1992). A realistic simulation for assessing the relationship among components of rating accuracy. Journal of Applied Psychology, 77, 196-200.
52. Roch, S. G., Ayman, R., & McNall, L. (2003, April). Effect of rater audience, identifiability, and intrinsic motivation on ratings. Paper presented at the Society for Industrial and Organizational Psychology, Orlando, FL.
53. Roch, S. G., Ayman, R., Newhouse, N. K., & Harris, M. (2005). Effect of identifiability, rating audience, and conscientiousness on rating level. International Journal of Selection and Assessment, 13, 53-62.
54. Roch, S. G., & Woehr, D. J. (1997, August). The effect of rater motivation on the accuracy of performance evaluation: An NPI approach. Paper presented at the annual meeting of the American Psychological Association. Chicago, IL.
55. Rogers, D. A., & Lackey, J. N. (2000, April). Choice shift in performance evaluations: The validation of doubt hypothesis. Paper presented at the 15th Annual Meeting of the Society for Industrial and Organizational Psychology, New Orleans, LA.
56. Salvemini, N. J., Reilly, R. R., & Smither, J. W. (1993). The influence of rater motivation on assimilation effects and accuracy in performance ratings. Organizational Behavior and Human Decision Process, 55, 41-60.
57. Sanchez, J. I., & De La Torre, P. (1996). A second look at the relationship between rating and behavioral accuracy in performance evaluation. Journal of Applied Psychology, 81, 3-10.
58. Simonson, I., & Nye, P. (1992). The effect of accountability on susceptibility to decision error. Organizational Behavior and Human Decision Processes, 51, 416-446.
59. Snodgrass, J. G., & Corwin, J. (1988). Pragmatics of measuring recognition memory: Applications to dementia and amnesia. Journal of Experimental Psychology: General, 117, 34-50.
60. Spychalski, A. C., Quinones, M. A., Gaugler, B. B., & Pohley, K. (1997). A survey of assessment center practices in organizations in the United States. Personnel Psychology, 50, 71-92.
61. Steiner, I. D. (1972). Group process and productivity. San Diego, CA: Academic Process.
62. Steward, D. D., & Stasser, G. (1995). Expert role assignment and information sampling during collective recall and decision making. Journal of Personality and Social Psychology, 69, 619-628.
63. Sulsky, L. M., & Balzer, W. K. (1988). Meaning and measurement of performance rating accuracy: Some methodological and theoretical concerns. Journal of Applied Psychology, 73, 497-506.
64. Sulsky, L. M., & Day, D. V. (1992). Frame-of-reference training and cognitive categorization: An empirical investigation of rater memory issues. Journal of Applied Psychology, 77, 501-510.
65. Tetlock, P. E. (1983). Accountability and complexity of thought. Journal of Personality and Social Psychology, 45, 74-83.
66. Tetlock, P. E. (1985). Accountability: The neglected social context of judgment and choice. In B. M. Staw & L. Cummings (Eds.), Research in organizational behavior: Vol. 7(pp. 297-332). Greenwich, CT: JAI.
67. Tetlock, P. E., & Kim, J. I. (1987). Accountability and judgment processes in a personality prediction task. Journal of Personality and Social Psychology, 52, 700-709.
68. Thornton, G. C., III, & Byham, C. C. (1982). Assessment centers and managerial performance. New York: Academic.
69. Vollrath, D. A., Sheppard, B. H., Hinsz, V. B., & Davis, J. H. (1989). Memory performance by decision-making groups and individuals. Organizational Behavior and Human Decision Processes, 43, 289-300.
70. Waung, M., & Highhouse, S. (1997). Fear of conflict and empathic buffering: Two explanations for the inflation of performance feedback. Organizational Behavior and Human Decision Processes, 71(1), 37-54.
71. Weldon, M. S., & Bellinger, K. D. (1997). Collective memory: Collaborative and individual processes in remembering. Journal of Experimental Psychology, Learning, Memory, & Cognition, 23, 1160-1175.
72. Weldon, M. S., Blair, C., & Huebsch, P. D. (2000). Group remembering: Does social loafing underlie collaborative inhibition? Journal of Experimental Psychology, Learning, Memory, & Cognition, 26, 1568-1577.
73. Wiesner, W., & Cronshaw, S. (1988). A meta-analytic investigation of the impact of interview format and degree of structure on the validity of the employment interview. Journal of Occupational Psychology, 61, 275-290.
74. Wingrove, J., Jones, A., & Herriot, P. (1985). The predictive validity of pre- and post-discussion assessment centre ratings. Journal of Occupational Psychology, 58, 189-192.
75. Wittenbaum, G. M., Stasser, G., & Merry, C. J. (1996). Tacit coordination in anticipation of small group task completion. Journal of Experimental Social Psychology, 32, 129-152.
76. Woehr, D. J. (1994). Understanding frame-of-reference training: The impact of training on the recall of performance information. Journal of Applied Psychology, 79, 525-534.
77. Yarmey, D. A. (1992). The effects of dyadic discussion on earwitness recall. Basic and Applied Social Psychology, 75, 251-263.