Rating Leniency and Halo in Multisource Feedback Ratings: Testing Cultural Assumptions of Power Distance and Individualism-Collectivism

Journal of Applied Psychology

Volumn 96, Issue 5, 2011, Pages 1033-1044

  Ng, Kok Yee ^a   Koh, Christine ^a   Ang, Soon ^a   Kennedy, Jeffrey C ^a   Chan, Kim Yin ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Culture; Individualism collectivism; Multisource feedback; Power distance; Rating bias

Indexed keywords

ADULT; ARTICLE; ATTITUDE; COMPARATIVE STUDY; FEEDBACK SYSTEM; FEMALE; GROUP PROCESS; HUMAN; MALE; PEER GROUP; PERSONNEL MANAGEMENT; PSYCHOLOGICAL MODEL; SOCIAL DISTANCE; SOCIAL DOMINANCE; SOCIAL PSYCHOLOGY;

ADULT; ATTITUDE; EMPLOYEE PERFORMANCE APPRAISAL; FEEDBACK; FEMALE; GROUP PROCESSES; HIERARCHY, SOCIAL; HUMANS; MALE; MODELS, PSYCHOLOGICAL; PEER GROUP; SOCIAL DISTANCE; SOCIAL VALUES;

EID: 80052859635     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0023368     Document Type: Article

References (94)

1. Ang S., Van Dyne L., Koh S.K., Ng K.Y., Templer K.J., Tay C., Chandrasekar N.A. Cultural intelligence: An individual difference with effects on cultural judgment and decision making, cultural adaptation, and task performance. Management and Organization Review 2007, 3:335-371. 10.1111/j.1740-8784.2007.00082.x.
2. Antonakis J., Atwater L. Leader distance: A review and a proposed theory. Leadership Quarterly 2002, 13:673-704. 10.1016/S1048-9843(02)00155-8.
3. Antonioni D., Park H. The relationship between rater affect and three sources of 360-degree feedback ratings. Journal of Management 2001, 27:479-495. 10.1177/014920630102700405.
4. Antonioni D., Woehr D.J. Improving the quality of multisource rater performance. Handbook of multisource feedback 2001, 114-129. Jossey-Bass, San Francisco, CA. D. Bracken, C. Timmreck, A. Church (Eds.).
5. Atwater L.E., Wang M., Smither J.W., Fleenor J.W. Are cultural characteristics associated with a relationship between self and others' ratings of leadership?. Journal of Applied Psychology 2009, 94:876-886. 10.1037/a0014561.
6. Bass B.M. Leadership, psychology, and organizational behavior 1960, Harper, New York, NY.
7. Bernardin H.J., Cooke D.K., Villanova P. Conscientiousness and agreeableness as predictors of rating leniency. Journal of Applied Psychology 2000, 85:232-236. 10.1037/0021-9010.85.2.232.
8. Bettenhausen K.L., Fedor D.B. Peer and upward appraisals: A comparison of their benefits and problems. Group and Organization Management 1997, 22:236-263. 10.1177/1059601197222006.
9. Bliese P.D. Multilevel random coefficient modeling in organization research: Examples using SAS and S-PLUS. Measuring and analyzing behavior in organizations: Advances in measurement and data analysis 2002, 401-445. Jossey-Bass, San Francisco, CA. F. Drasgow, N. Schmitt (Eds.).
10. Bond M.H., Wan K., Leung K., Giacalone R.A. How are responses to verbal insult related to cultural collectivism and power distance?. Journal of Cross-Cultural Psychology 1985, 16:111-127. 10.1177/0022002185016001009.
11. Brett J.F., Atwater L.E. 360° feedback: Accuracy, reactions, and perceptions of usefulness. Journal of Applied Psychology 2001, 86:930-942. 10.1037/0021-9010.86.5.930.
12. Bryk A.S., Raudenbush S.W. Hierarchical linear models: Applications and data analysis methods 1992, Sage Publications, Newbury, CA.
13. Chan K.Y., Drasgow F. Toward a theory of individual differences and leadership: Understanding the motivation to lead. Journal of Applied Psychology 2001, 86:481-498. 10.1037/0021-9010.86.3.481.
14. Chen H., Ng S., Rao A.R. Cultural differences in consumer impatience. Journal of Marketing Research 2005, 42:291-301. 10.1509/jmkr.2005.42.3.291.
15. De Luque M.F.S., Sommer S.M. The impact of culture on feedback-seeking behavior: An integrated model and propositions. Academy of Management Review 2000, 25:829-849. 10.2307/259209.
16. de Nisi A.S., Mitchell J.L. An analysis of peer ratings as predictors and criterion measures and a proposed new application. Academy of Management Review 1978, 3:369-374. 10.2307/257678.
17. Dorfman P.W., Howell J.P. Dimensions of national culture and effective leadership patterns: Hofstede revisited. Advances in international comparative management 1988, Vol. 3:127-150. JAI Press, Stamford, CT. J.L.C. Cheng, R.B. Peterson (Eds.).
18. Drexler J.A., Beehr T.A., Stetz T.A. Peer appraisals: Differentiation of individual performance on group tasks. Human Resource Management 2001, 40:333-345. 10.1002/hrm.1023.
19. Earley P.C. East meets West meets Mideast: Further explorations of collectivistic and individualistic work groups. Academy of Management Journal 1993, 36:319-348. 10.2307/256525.
20. Earley P.C. Playing follow the leader: Status-determining traits in relation to collective efficacy across cultures. Organizational Behavior and Human Decision Processes 1999, 80:192-212. 10.1006/obhd.1999.2863.
21. Enders C.K., Tofighi D. Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods 2007, 12:121-138. 10.1037/1082-989X.12.2.121.
22. Entrekin L., Chung Y.W. Attitudes towards different sources of executive appraisal: A comparison of Hong Kong Chinese and American managers in Hong Kong. International Journal of Human Resource Management 2001, 12:965-987.
23. Erez M., Earley P.C. Culture, self-identity, and work 1993, Oxford University Press, New York, NY.
24. Farh J.-L., Cannella A.A., Bedeian A.G. The impact of purpose on rating quality and user acceptance. Group and Organization Management 1991, 16:367-386. 10.1177/105960119101600403.
25. Farh J.L., Hackett R.D., Liang J. Individual-level cultural values as moderators of perceived organizational support-employee outcome relationships in China: Comparing the effects of power distance and traditionality. Academy of Management Journal 2007, 50:715-729.
26. Fedor D.B., Bettenhausen K.L., Davis W. Peer reviews: Employees' dual roles as raters and recipients. Group and Organization Management 1999, 24:92-120. 10.1177/1059601199241006.
27. Fletcher C., Perry E.L. Performance appraisal and feedback: A consideration of national culture and a review of contemporary research and future trends. Handbook of industrial, work and organizational psychology 2001, Vol. 1:125-144. Sage, Thousand Oaks, CA. N. Anderson, D.S. Ones, H.K. Sinangil, C. Viswesvaran (Eds.).
28. French J.R.P., Raven B.H. The bases of social power. Studies in social power 1959, 150-167. Institute for Social Research, Ann Arbor, MI. D. Cartwright (Ed.).
29. Gelfand M.J., Erez M., Aycan Z. Cross-cultural organizational behavior. Annual Review of Psychology 2007, 58:479-514. 10.1146/annurev.psych.58.110405.085559.
30. Greguras G.J., Robie C. A new look at within-source interrater reliability of 360-degree feedback ratings. Journal of Applied Psychology 1998, 83:960-968. 10.1037/0021-9010.83.6.960.
31. Harris M.M. Rater motivation in the performance appraisal context: A theoretical framework. Journal of Management 1994, 20:737-756. 10.1016/0149-2063(94)90028-0.
32. Harris M.M., Schaubroeck J. A meta-analysis of self-boss, self-peer, and peer-boss ratings. Personnel Psychology 1988, 41:43-62. 10.1111/j.1744-6570.1988.tb00631.x.
33. Harris M.M., Smith D.E., Champagne D. A field study of performance appraisal purpose: Research versus administrative-based ratings. Personnel Psychology 1995, 48:151-160. 10.1111/j.1744-6570.1995.tb01751.x.
34. Heidemeier H., Moser K. Self-other agreement in job performance ratings: A meta-analytic test of a process model. Journal of Applied Psychology 2009, 94:353-370. 10.1037/0021-9010.94.2.353.
35. Hofmann D.A. An overview of the logic and rationale of hierarchical linear models. Journal of Management 1997, 23:723-744. 10.1177/014920639702300602.
36. Hofstede G. Culture's consequences 2001, Sage, Beverly Hills, CA.
37. Hogg M.A., Martin R., Epitropaki O., Mankad A., Svensson A., Weeden K. Effective leadership in salient groups: Revisiting leader-member exchange theory from the perspective of the social identity theory of leadership. Personality and Social Psychology Bulletin 2005, 31:991-1004. 10.1177/0146167204273098.
38. Holzbach R.L. Rater bias in performance ratings: Superior, self and peer ratings. Journal of Applied Psychology 1978, 63:579-588. 10.1037/0021-9010.63.5.579.
39. House R., Hanges P., Javidan M., Dorfman P., Gupta V. Culture, leadership, and organizations: The GLOBE study of 62 societies 2004, Sage, Thousand Oaks, CA.
40. Hox J.J. Multilevel analysis: Techniques and applications 2002, Lawrence Erlbaum, Mahwah, NJ.
41. Ilies R., Wagner D.T., Morgeson F. Explaining affective linkages in teams: Individual differences in susceptibility to contagion and individualism-collectivism. Journal of Applied Psychology 2007, 92:1140-1148. 10.1037/0021-9010.92.4.1140.
42. Jawahar I.M., Williams C.R. When all the children are above average: The performance appraisal purpose effect. Personnel Psychology 1997, 50:905-925. 10.1111/j.1744-6570.1997.tb01487.x.
43. Jöreskog K.G., Sörbom D. Windows LISREL 8.8 2006, Scientific Software, Chicago, IL.
44. Kim J.Y., Nam S.H. The concept and dynamics of face: Implications for organizational behavior in Asia. Organization Science 1998, 9:522-534. 10.1287/orsc.9.4.522.
45. Kirkman B.L., Chen G., Farh J., Chen Z.X. Individual power distance orientation and follower reactions to transformational leaders: A cross-level, cross-cultural examination. Academy of Management Journal 2009, 52:744-764.
46. Kirkman B.L., Lowe K.B., Gibson C.B. A quarter century of Culture's Consequences: A review of empirical research incorporating Hofstede's cultural values framework. Journal of International Business Studies 2006, 37:285-320. 10.1057/palgrave.jibs.8400202.
47. Klimoski R.J., London M. Role of the rater in performance appraisal. Journal of Applied Psychology 1974, 59:445-451. 10.1037/h0037332.
48. Kreft I., de Leeuw J. Introducing multilevel modeling 1998, Sage, Thousand Oaks, CA.
49. Kudisch J.D., Fortunato V.J., Smith A.F.R. Contextual and individual difference factors predicting individuals' desire to provide upward feedback. Group and Organization Management 2006, 31:503-529. 10.1177/1059601106286888.
50. Lance C.E. Test of a latent structure of performance ratings derived from Wherry's (1952) theory of ratings. Journal of Management 1994, 20:751-771.
51. Lawler E.E. The multitrait-multirater approach to measuring managerial job performance. Journal of Applied Psychology 1967, 51:369-381. 10.1037/h0025095.
52. Lebreton J.M., Burgess J.R.D., Kaiser R.B., Atchley E.K., James L.R. The restriction of variance hypothesis and interrater reliability and agreement: Are ratings from multiple sources really dissimilar?. Organizational Research Methods 2003, 6:80-128. 10.1177/1094428102239427.
53. Leslie J.B., Gryskiewicz N.D., Dalton M.A. Understanding cultural influences on 360-degree feedback process. Maximizing the value of 360-degree feedback: A process for successful individual and organizational development 1998, 196-216. Jossey-Bass, San Francisco, CA. W.W. Tornow, M. London (Eds.).
54. Leung K., Bond M.H., Reimel de Carrasquel S., Muñoz C., Hernández M., Murakami F., Singelis T.M. Social axioms: The search for universal dimensions of general beliefs about how the world functions. Journal of Cross-Cultural Psychology 2002, 33:286-302. 10.1177/0022022102033003005.
55. Levy P.E., Williams J.R. The social context of performance appraisal: A review and framework for the future. Journal of Management 2004, 30:881-905. 10.1016/j.jm.2004.06.005.
56. Li J., Ngin P.M., Teo A.C.Y. Culture and leadership in Singapore: Combination of the East and the West. Culture and leadership across the world: The GLOBE book of in-depth dtudies of 25 societies 2007, 947-968. Lawrence Erlbaum Associates, Mahwah, NJ. J. Chhokar, F.C. Brodbeck, R.J. House (Eds.).
57. London M. Job feedback: Giving, seeking, and using feedback for performance improvement 2003, Lawrence Erlbaum, Mahwah, NJ.
58. London M., Smither J.W. Can multisource feedback change perceptions of goal accomplishment, self-evaluations, and performance-related outcomes? Theory-based applications and direction for research. Personnel Psychology 1995, 48:803-839. 10.1111/j.1744-6570.1995.tb01782.x.
59. Markus H.R., Kitayama S. Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review 1991, 98:224-253. 10.1037/0033-295X.98.2.224.
60. Mero N.P., Motowidlo S.J. Effects of rater accountability on the accuracy and the favorability of performance ratings. Journal of Applied Psychology 1995, 80:517-524. 10.1037/0021-9010.80.4.517.
61. Morrison E.W., Chen Y.-R., Salgado S.R. Cultural differences in newcomer feedback seeking: A comparison of the United States and Hong Kong. Applied Psychology: An International Review 2004, 53:1-22. 10.1111/j.1464-0597.2004.00158.x.
62. Mount M.K. Psychometric properties of subordinate ratings of managerial performance. Personnel Psychology 1984, 37:687-702. 10.1111/j.1744-6570.1984.tb00533.x.
63. Mount M.K., Judge T.A., Scullen S.E., Sytsma M.R., Hezlett S.A. Trait, rater and level effects in 360-degree performance ratings. Personnel Psychology 1998, 51:557-576. 10.1111/j.1744-6570.1998.tb00251.x.
64. Mount M.K., Scullen S.E. Multisource feedback ratings: What do they really measure?. How people evaluate others in organizations 2001, 155-176. Lawrence Erlbaum, Mahwah, NJ. M. London (Ed.).
65. Murphy K.R., Balzer W.K. Rater errors and rating accuracy. Journal of Applied Psychology 1989, 74:619-624. 10.1037/0021-9010.74.4.619.
66. Murphy K.R., Cleveland J.N. Understanding performance appraisal: Social, organizational, and goal-based perspectives 1995, Sage, Thousand Oaks, CA.
67. Murphy K.R., Cleveland J.N., Skattebo A.L., Kinney T.B. Raters who pursue different goals give different ratings. Journal of Applied Psychology 2004, 89:158-164. 10.1037/0021-9010.89.1.158.
68. Ng K.Y., Van Dyne L. Individualism-collectivism as a boundary condition for effectiveness of minority influence in decision making. Organizational Behavior and Human Decision Processes 2001, 84:198-225. 10.1006/obhd.2000.2927.
69. Pfeffer J., Salancik G.R. Determinants of supervisory behaviors: A role set analysis. Human Relations 1975, 28:139-154. 10.1177/001872677502800203.
70. Ployhart R.E., Wiechmann D., Schmitt N., Sacco J.M., Rogg K. The cross-cultural equivalence of job performance ratings. Human Performance 2003, 16:49-79. 10.1207/S15327043HUP1601_3.
71. Prentice D.A., Miller D.T. When small effects are impressive. Psychological Bulletin 1992, 112:160-164. 10.1037/0033-2909.112.1.160.
72. Raudenbush S.W., Bryk A.S. Hierarchical linear models: Applications and data analysis methods 2002, Sage, Thousand Oaks, CA.
73. Reilly R.R., Smither J.W., Vasilopoulos N.L. A longitudinal study of upward feedback. Personnel Psychology 1996, 49:599-612. 10.1111/j.1744-6570.1996.tb01586.x.
74. Saal F.E., Downey R.G., Lahey M.A. Rating the ratings: Assessing the psychometric quality of rating data. Psychological Bulletin 1980, 88:413-428. 10.1037/0033-2909.88.2.413.
75. Schneier E.C. Operational utility and psychometric characteristics of behavioral expectations scales: A cognitive reinterpretation. Journal of Applied Psychology 1977, 62:541-548. 10.1037/0021-9010.62.5.541.
76. Schwartz S.M., Bilsky W. Toward a universal psychological structure of human values. Journal of Personality and Social Psychology 1987, 53:550-562. 10.1037/0022-3514.53.3.550.
77. Scullen S.E., Mount M.K., Goff M. Understanding the latent structure of job performance ratings. Journal of Applied Psychology 2000, 85:956-970. 10.1037/0021-9010.85.6.956.
78. Shipper F., Hoffman R.C., Rotondo D.M. Does the 360 feedback process create actionable knowledge equally across cultures?. Academy of Management Learning and Education 2007, 6:33-50.
79. Spence J.R., Keeping L.M. The impact of non-performance information on ratings of job performance: A policy-capturing approach. Journal of Organizational Behavior 2010, 31:587-608. 10.1002/job.648.
80. Springer D. Ratings of candidates for promotion by co-workers and supervisors. Journal of Applied Psychology 1953, 64:124-131.
81. Stone-Romero E.F., Stone D.L. Cross-cultural differences in responses to feedback: Implications for individual, group, and organizational effectiveness. Research in Personnel and Human Resources Management 2002, 21:275-331. 10.1016/S0742-7301(02)21007-5.
82. Triandis H.C. Individualism and collectivism 1995, Westview Press, Boulder, CO.
83. Tsui A.S. A role set analysis of managerial reputation. Organizational Behavior and Human Performance 1984, 36:64-96. 10.1016/0030-5073(84)90037-0.
84. Tsui A.S., Barry B. Interpersonal affect and rating errors. Academy of Management Journal 1986, 29:586-599. 10.2307/256225.
85. Tsui A., O'Reilly C.A. Beyond simple demographic effects: The importance of relational demography in superior-subordinate dyads. Academy of Management Journal 1989, 32:402-423. 10.2307/256368.
86. Vance R.J., Winne P.S., Wright E.S. A longitudinal examination of rater and ratee effects in performance ratings. Personnel Psychology 1983, 36:609-620. 10.1111/j.1744-6570.1983.tb02238.x.
87. Varela O.E., Premeaux S.F. Do cross-cultural values affect multisource feedback dynamics? The case of high power distance and collectivism in two Latin American countries. International Journal of Selection and Assessment 2008, 16:134-142. 10.1111/j.1468-2389.2008.00418.x.
88. Walker A.G., Smither J.W. A five-year study of upward feedback: What managers do with their results matter. Personnel Psychology 1999, 52:393-423. 10.1111/j.1744-6570.1999.tb00166.x.
89. Westerman J.W., Rosse J.G. Reducing the threat of rater nonparticipation in 360-degree feedback systems: An exploratory examination of antecedents to participation in upward ratings. Group and Organization Management 1997, 22:288-309. 10.1177/1059601197222008.
90. Wherry R.J., Bartlett C.J. The control of bias in ratings: A theory of rating. Personnel Psychology 1982, 35:521-551. 10.1111/j.1744-6570.1982.tb02208.x.
91. Wohlers A.J., Hall M.J., London M. Subordinates rating managers: Organizational and demographic correlates of self-subordinate agreement. Journal of Occupational and Organizational Psychology 1993, 66:263-275.
92. Wong K.F.E., Kwong J.Y.Y. Effects of rater goals on rating patterns: Evidence from an experimental field study. Journal of Applied Psychology 2007, 92:577-585. 10.1037/0021-9010.92.2.577.
93. Yukl G., Falbe C.M. Importance of different power sources in downward and lateral relations. Journal of Applied Psychology 1991, 76:416-423. 10.1037/0021-9010.76.3.416.
94. Zedeck S., Imparto N., Krausz M., Oleno T. Development of behaviorally anchored rating scales as a function of organizational level. Journal of Applied Psychology 1974, 59:249-252. 10.1037/h0036521.