"I don't see me like you see me, but is that a problem?" Cultural influences on rating discrepancy in 360-degree feedback instruments

European Journal of Work and Organizational Psychology

Volumn 19, Issue 3, 2010, Pages 259-278

  Eckert, Regina ^a   Ekelund, Bjørn Z ^b   Gentry, William A ^c   Dawson, Jeremy F ^d  

^a Center for Creative Leadership   (Belgium)

^b Oslo and Agder University ^*  (Norway)

^c CENTER FOR CREATIVE LEADERSHIP   (United States)

^d ASTON UNIVERSITY   (United Kingdom)

Author keywords

360 degree feedback; Cultural differences; Self observer discrepancy

Indexed keywords

EID: 77953927815     PISSN: 1359432X     EISSN: 14640643     Source Type: Journal    
DOI: 10.1080/13594320802678414     Document Type: Article

References (63)

1. Atwater, L. E., & Brett, J. F. (2005). Antecedents and consequences of reactions to developmental 3608 feedback. Journal of Vocational Behavior, 66, 532-548.
2. Atwater, L. E., Ostroff, C., Yammarino, F. J., & Fleenor, J. W. (1998). Self-other agreement: Does it really matter? Personnel Psychology, 51(3), 577-598.
3. Atwater, L., Waldman, D., Ostroff, C., Robie, C., & Johnson, K. M. (2005). Self-other agreement: Comparing its relationship with performance in the US and Europe. International Journal of Selection and Assessment, 13, 25-40.
4. Atwater, L. E., & Yammarino, F. (1997). Self-other rating agreement: A review and model. Research in Personnel and Human Resources Management, 15, 141-164.
5. Badjo, L. M., & Dickson, M. W. (2001). Perception of organizational culture and women's advancement in organizations: A cross-cultural examination. Sex Roles, 45(5/6), 399-414.
6. Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analyses. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 349-381). San Francisco: Jossey-Bass.
7. Bliese, P. D. (2002). Multilevel random coefficient modeling in organizational research: Examples using SAS and S-PLUS. In F. Drasgow & N. Schmitt (Eds.), Measuring and analyzing behavior in organizations: Advances in measurement and data analysis (pp. 401-445). San Francisco: Jossey-Bass.
8. Braddy, P. W., Fleenor, J. W., & Campbell, M. (2008). Methods for reducing the length of 360- degree instruments. Paper presented at the 23rd annual conference of the Society for Industrial and Organizational Psychology, San Francisco.
9. Brodbeck, F. C. (2000). Cultural variation of leadership prototypes across 22 European countries. Journal of Occupational and Organizational Psychology, 73(1), 1-29.
10. Brutus, S., Fleenor, J. W., & London, M. (1998). Does 360-degree feedback work in different industries? A between-industry comparison of the reliability and validity of multi-source performance ratings. Journal of Management Development, 17, 177-190.
11. Brutus, S., Fleenor, J. W., & McCauley, C. D. (1999). Demographic and personality predictors of congruence in multi-source ratings. Journal of Management Development, 18, 417-435.
12. Brutus, S., Leslie, J. B., & McDonald-Mann, D. (2001). Cross-cultural issues in multisource feedback. In D. Bracken, C. Timmreck, & A. Church (Eds.), Handbook of multisource feedback (pp. 433-446). San Francisco: Jossey-Bass.
13. Carl, D., Gupta, V., & Javidan, M. (2004). Power distance. In R. J. House, P. J. Hanges,
14. M. Javidan, P. W. Dorfman, & V. Gupta (Eds.), Culture, leadership, and organizations: The GLOBE study of 62 societies (pp. 513-563). Thousand Oaks, CA: Sage.
15. Carty, H. M. (2003). Review of BENCHMARKS® [revised]. In B. S. Plake, J. Impara, & R. A. Spies (Eds.), The fifteenth mental measurements yearbook (pp. 123-124). Lincoln, NE: Buros Institute of Mental Measurements.
16. Center for Creative Leadership. (2004). BENCHMARKS® facilitator's manual. Greensboro, NC: Center for Creative Leadership.
17. Conway, J. M. (2000). Managerial performance development constructs and personality correlates. Human Performance, 13, 23-46.
18. Dalessio, A. T. (1998). Using multi-source feedback for employee development and personnel decisions. In J. W. Smither (Ed.), Performance appraisal: State-of-the-art in practice (pp. 278-330). San Francisco: Jossey-Bass.
19. Dansereau, F., Graen, G., & Haga, W. J. (1975). A vertical dyad linkage approach to leadership within formal organizations: A longitudinal investigation of the role making process. Organizational Behavior and Human Performance, 13, 46-78.
20. Den Hartog, D. N., House, R. J., Hanges, P. J., Ruiz-Quintanilla, S. A., Dorfman, P. W., & GLOBE co-authors. (1999). Culture specific and cross-culturally generalizable implicit leadership theories: Are attributes of charismatic/transformational leadership universally endorsed? Leadership Quarterly, 10(2), 219-256.
21. Douglas, C. A. (2003). Key events and lessons for managers in a diverse workforce: A report on research and findings. Greensboro, NC: Center for Creative Leadership.
22. Edwards, J. R. (1994). The study of congruence in organizational behavior research: Critique and proposed alternative. Organizational Behavior and Human Decision Processes, 58, 51-100.
23. Edwards, J. R. (1995). Alternatives to difference scores as dependent variables in the study of congruence in organizational research. Organizational Behavior and Human Decision Processes, 64, 307-324.
24. Ensari, N., & Murphy, S. E. (2003). Cross-cultural variations in leadership perceptions and attribution of charisma to the leader. Organizational Behavior and Human Decision Processes, 92(1-2), 52-66.
25. Feldman, D. C., & Bolino, M. C. (1999). The impact of on-site mentoring on expatriate socialisation: A structural equation modelling approach. International Journal of Human Resource Management, 10(1), 54-71.
26. Fleenor, J. W., McCauley, C. D., & Brutus, S. (1996). Self-other rating agreement and leader effectiveness. Leadership Quarterly, 7, 487-506.
27. Gentry, W. A., Hannum, K. M., Ekelund, B. Z., & de Jong, A. (2007). A study of the discrepancy between self- and observer-ratings on managerial derailment characteristics of European managers. European Journal of Work and Organizational Psychology, 16, 295-325.
28. Gerstner, C. R., & Day, D. V. (1994). Cross-cultural comparison of leadership prototypes. Leadership Quarterly, 5(2), 121-134.
29. Greguras, G. J., & Robie, C. (1998). A new look at within-source interrater reliability of 360- degree feedback ratings. Journal of Applied Psychology, 83, 960-968.
30. Harris, M. M., & Schaubroeck, J. (1988). A meta-analysis of self-supervisor, self-peer, and peer supervisor ratings. Personnel Psychology, 41, 43-61.
31. Hofstede, G. (1984). Culture's consequences: International differences in work-related values, Vol. 2. Newbury Park, CA: Sage.
32. House, R. J., Hanges, P. J., Javidan, M., Dorfman, P. W., & Gupta V. (Eds.). (2004). Culture, leadership, and organizations: The GLOBE study of 62 societies. Thousand Oaks, CA: Sage.
33. James, L. R. (1988). Organizational climate: Another look at a potentially important construct. In S. G. Cole & R. G. Demaree (Eds.), Applications of interactionist psychology: Essays in honor of Saul B. Sells (pp. 253-282). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
34. Johns, G. (1981). Difference score measures of organizational behavior variables: A critique. Organizational Behavior and Human Performance, 27, 443-463.
35. Kulas, J. T., & Finkelstein, L. M. (2007). Content and reliability of discrepancy-defined self- awareness in multisource feedback. Organizational Research Methods, 10(3), 502-522.
36. Lindell, M. K., Brandt, C. J., & Whitney, D. J. (1999). A revised index of interrater agreement for multi-item ratings of a single target. Applied Psychological Measurement, 23, 127-135.
37. Lombardo, M. M., & McCauley, C. D. (1994). BENCHMARKS®: A manual and trainer's guide. Greensboro, NC: Center for Creative Leadership.
38. Lombardo, M. M., McCauley, C. D., McDonald-Mann, D., & Leslie, J. B. (1999). BENCHMARKS® developmental reference points. Greensboro, NC: Center for Creative Leadership.
39. London, M., & Smither, J. W. (1995). Can multi-source feedback change self-evaluations, skill development, and performance? Theory-based applications and directions for research. Personnel Psychology, 48, 375-390.
40. Lord, R. G., & Maher, K. J. (1991). Leadership and information processing: Linking perceptions and performance. London: Routledge.
41. McCauley, C., Lombardo, M., & Usher, C. (1989). Diagnosing management development needs: An instrument based on how managers develop. Journal of Management, 15, 389-403.
42. Merritt, A. C. (2000). Culture in the cockpit: Do Hofstede's dimensions replicate? Journal of Cross-Cultural Psychology, 31, 283-301.
43. Morgeson, F. P., Mumford, T. V., & Campion, C. A. (2005). Coming full circle: Using research and practice to address 27 questions about 360-degree feedback programs. Consulting Psychology Journal: Practice and Research, 57, 196-209.
44. Mount, M. K., Judge, T. A., Scullen, T. E., Sytsma, M. R., & Hezlett, S. A. (1998). Trait, rater, and level effects in 360-degree performance ratings. Personnel Psychology, 51, 557-576.
45. Muthen, L., & Muthen, B. (2006). Mplus (Version 4.0) [Computer software]. Los Angeles: Author.
46. Ostroff, C., Atwater, L. E., & Feinberg, B. J. (2004). Understanding self-other agreement: A look at rater and ratee characteristics, context, and outcomes. Personnel Psychology, 57, 333-375.
47. Sala, F. (2003). Executive blind spots: Discrepancies between self- and other-ratings. Consulting Psychology Journal, 55, 222-229.
48. Satorra, A. (2000). Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In R. D. H. Heimans, D. S. G. Pollock, & A. Satorra (Eds.), Innovations in multivariate statistical analysis (pp. 233-247). London: Kluwer Academic Publishers.
49. Schermerhorn, J. R., & Bond, M. H. (1997). Cross-cultural leadership dynamics in collectivism and high power distance settings. Leadership and Organization Development Journal, 18(4), 187-193.
50. Schyns, B. (2006). The role of implicit leadership theories in the performance appraisals and promotion recommendations of leaders. Equal Opportunities International, 25(3), 188-199.
51. Shipper, F., Hoffman, R. C. I., & Rotondo, D. M. (2007). Does the 360 feedback process create actionable knowledge equally across cultures? Academy of Management Learning and Education, 6(1), 33-50.
52. Spangler, M. (2003). Review of BENCHMARKS® [revised]. In B. S. Plake, J. Impara, & R. A. Spies (Eds.), The fifteenth mental measurements yearbook (pp. 124-126). Lincoln, NE: Buros Institute of Mental Measurements.
53. Spector, P. E., Cooper, C. L., & Sparks, K. (2001). An international study of the psychometric properties of the Hofstede Values Survey Module 1994: A comparison of individual and country/province level results. Applied Psychology: An International Review, 50(2), 269-281.
54. Stark, E., Hansen, J., & Thomas, L. T. (1998). A field study of accuracy, agreement, and outcomes of multi-source feedback: Implications across international boundaries. Paper presented at the international conference of the Academy of Business and Administrative Sciences.
55. Sullivan, D. M., Mitchell, M., & Uhl-Bien, M. (2003). The new conduct of business: How LMX can help capitalize on cultural diversity. In G. Graen (Ed.), Dealing with diversity (pp. 183-218). Charlotte, NC: Information Age.
56. Sully de Luque, M. F., & Sommer, S. M. (2000). The impact of culture on feedback-seeking behavior: An integrated model and propositions. Academy of Management Review, 25, 829-849.
57. Tornow, W. W. (1993). Perceptions or reality: Is multi-perspective measurement a means or an end? Human Resource Management, 32, 246-264.
58. Van der Vegt, G. S., van de Vliert, E., & Huang, X. (2005). Location-level links between diversity and innovative climate depend on national power distance. Academy of Management Journal, 48(6), 1171.
59. Van Velsor, E., & Leslie, J. B. (1991). Feedback to managers: Vol. 1. A guide to evaluating multirater feedback instruments. Greensboro, NC: Center for Creative Leadership.
60. Wohlers, A., & London, M. (1989). Ratings of managerial characteristics: Evaluation difficulty, co-worker agreement and self-awareness. Personnel Psychology, 42, 235-261.
61. Yammarino, F. (2003). Modern data analytic techniques for multisource feedback. Organiza- tional Research Methods, 6, 6-14.
62. Yammarino, F., & Atwater, L. (1993). Understanding self-perception accuracy: Implications for human resource management. Human Resource Management, 32(2/3), 231-247.
63. Zedeck, S. (1995). Review of BENCHMARKS®. In J. Conoley & J. Impara (Eds.), The twelfth mental measurements yearbook (Vol. 1, pp. 128-129). Lincoln, NE: Buros Institute of Mental Measurements.