When ratings from one source have been averaged, but ratings from another source have not: Problems and solutions

Journal of Applied Psychology

Volumn 82, Issue 6, 1997, Pages 880-888

  Scullen, Steven E ^a  

^a UNIVERSITY OF IOWA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0031316896     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/0021-9010.82.6.880     Document Type: Article

References (28)

1. Atwater, L. E., & Yammarino, F. Y. (1992). Does self-other agreement on leadership perceptions moderate the validity of leadership and performance predictions? Personnel Psychology, 45, 141-164.
2. Brennan, R. L. (1983). Essentials of generalizability theory. Iowa City, IA: American College Testing Program.
3. Campbell, J. P., Dunnette, M. D., Lawler, E. E., & Weick, K. E. (1970). Managerial behavior, performance, and effectiveness. New York: McGraw-Hill.
4. Cronhach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: Wiley.
5. Firebaugh, G. (1978). A rule for interpreting individuel relationships from aggregate data. American Sociological Review, 43, 522-557.
6. Gordon, L. V., & Medlund, F. F. (1965). The cross-group stability of peer ratings of leadership potential. Personnel Psychology, 18, 173-177.
7. Guilford, J. P. (1954). Psychometric methods. New York: McGraw-Hill.
8. Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
9. James, L. R. (1982). Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology, 67, 219-229.
10. Klein, K. J., Dansereau, F., & Hall, R. J. (1994). Levels issues in theory development, data collection, and analysis. Academy of Management Review, 19, 195-229.
11. Kraut, A. J. (1975). Prediction of managerial success by peer and training-staff ratings. Journal of Applied Psychology, 60, 14-19.
12. London, M., & Smither, J. W. (1995). Can multi-source feedback change perceptions of goal accomplishment, self-evaluations, and performance-related outcomes? Theory-based applications and directions for research. Personnel Psychology, 48, 803-839.
13. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
14. Mount, M. K. (1984). Psychometric properties of subordinate ratings of managerial performance. Personnel Psychology, 37, 687-702.
15. Mumford, M. D. (1983). Social comparison theory and the evaluation of peer evaluations: A review and some applied implications. Personnel Psychology, 36, 867-881.
16. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. New York: McGraw-Hill.
17. Ostroff, C. (1993a). Comparing correlations based on individual-level and aggregated data. Journal of Applied Psychology, 78, 569-582.
18. Ostroff, C. (1993b). The effects of climate and personal influences on individual behavior and attitudes in organizations. Organizational Behavior and Human Decision Processes, 56, 56-90.
19. Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15, 351-357.
20. Rousseau, D. M. (1985). Issues of level in organizational research: Multi-level and cross-level perspectives. In L. L. Cummings & B. M. Staw (Eds.), Research in organizational behavior (Vol. 7, pp. 1-37). Greenwich, CT: JAI Press.
21. Scullen, S. E., Mount, M. K., Sytsma, M. R., & Hezlett, S. A. (1996, April). Comparisons of self, peer, direct report, and boas ratings of managers' performance. Poster presented at the annual meeting of the Society for Industrial and Organizational Psychology, San Diego, CA.
22. Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage.
23. Shrout, P., & Fleiss, J. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428.
24. Tsui, A. S. (1983). Qualities of judgmental ratings by four rater sources. East Lansing, MI: National Center for Research on Teacher Learning. (ERIC Document Reproduction Service No. ED 237 913)
25. Tsui, A. S. (1990). A multiple-constituency model of effectiveness: An empirical examination at the human resource subunit level. Administrative Science Quarterly, 35, 458-483.
26. Tsui, A. S., & Ohlott, P. (1988). Multiple assessment of managerial effectiveness: Interrater agreement and consensus in effectiveness models. Personnel Psychology, 41, 779-803.
27. Wherry, R. J., & Fryer, D. H. (1949). Buddy ratings: Popularity contest or leadership criterion? Personnel Psychology, 2, 147-159.
28. Wohlers, A. J., Hall, M. J., & London, M. (1993). Subordinates rating managers: Organizational and demographic correlates of self/subordinate agreement. Journal of Occupational and Organizational Psychology, 66, 263-275.