Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer, and self-ratings

Human Performance

Volumn 10, Issue 4, 1997, Pages 331-360

  Conway, James M ^a   Huffcutt, Allen I ^b  

^a SETON HALL UNIVERSITY   (United States)

^b BRADLEY UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0031497749     PISSN: 08959285     EISSN: None     Source Type: Journal    
DOI: 10.1207/s15327043hup1004_2     Document Type: Article

References (189)

1. Albrecht, P. A., Glaser, E. M., & Marks, J. (1964). Validation of a multiple-assessment procedure for managerial personnel. Journal of Applied Psychology, 48, 351-360.
2. Atwater, L., Roush, P., & Fischthal, A. (1995). The influence of upward feedback on self- and follower ratings of leadership. Personnel Psychology, 48, 35-59.
3. Atwater, L. E., & Yammarino, F. J. (1992). Does self-other agreement on leadership perceptions moderate the validity of leadership and performance predictions? Personnel Psychology, 45, 141-164.
4. Baird, L. S. (1977). Self and superior ratings of performance: As related to self-esteem and satisfaction with supervision. Academy of Management Journal, 20, 291-300.
5. Baril, G. L., Ayman, R., & Palmiter, D. J., Jr. (1994). Measuring leader behavior: Moderators of discrepant self and subordinate descriptions. Journal of Applied Social Psychology, 24, 82-94.
6. Bass, B. M. (Ed.). (1981). Stogdill's handbook of leadership: A survey of theory and research. New York: Free Press.
7. Bass, A. R., & Turner, J. N. (1973). Ethnic group differences in relationships among criteria of job performance. Journal of Applied Psychology, 57, 101-109.
8. Bass, B. M., Valenzi, E. R., Farrow, D. L., & Solomon, R. J. (1975). Management styles associated with organizational, task, personal, and interpersonal contingencies. Journal of Applied Psychology, 60, 720-729.
9. Bass, B. M., & Wurster, C. R. (1953). Effects of company rank on LGD performance of oil refinery supervisors. Journal of Applied Psychology, 37, 100-104.
10. Bechtoldt, H. P, Maucker, J. W., & Stuit, D. B. (1947). The use of order-of-merit rankings. In G. A. Kelly (Ed.), New methods in applied psychology (pp. 26-33). College Park, MD: University of Maryland.
11. Becker, T. E., & Vance, R. J. (1993). Construct validity of three types of organizational citizenship behavior: An illustration of the direct product model with refinements. Journal of Management, 19, 663-682.
12. Berkeley, M. H. (1955). Comparison of supervisor, co-worker, and self-ratings of WAF job performance (AFPTRC-TN-55-25). Lackland Air Force Base, TX: Air Force Personnel and Training Research Center.
13. Bernardin, H. J., Dahmus, S. A., & Redmon, G. (1993). Attitudes of first-line supervisors toward subordinate appraisals. Human Resource Management, 32, 315-324.
14. Besco, R. O., & Lawshe, C. H. (1959). Foreman leadership as perceived by superiors and subordinates. Personnel Psychology, 12, 573-582.
15. Bommer, W. H., Johnson, J. L., Rich, G. A., Podsakoff, P. M., & MacKenzie, S. B. (1995). On the interchangeability of objective and subjective measures of employee performance: A meta-analysis. Personnel Psychology, 48, 587-605.
16. Borman, W. C. (1974). The rating of individuals in organizations: An alternate approach. Organizational Behavior and Human Performance, 12, 105-124.
17. Borman, W. C. (1991). Job behavior, performance, and effectiveness. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 2, pp. 271-326). Palo Alto, CA: Consulting Psychologists Press.
18. Borman, W. C., & Brush, D. H. (1993). More progress toward a taxonomy of managerial performance requirements. Human Performance, 6, 1-21.
19. Borman, W. C., & Vallon, W. R. (1974). A view of what can happen when behavioral expectation scales are developed in one setting and used in another. Journal of Applied Psychology, 59, 197-201.
20. Boruch, R. F., Larkin, J. D., Wolins, L., & MacKinney, A. C. (1970). Alternative methods of analysis: Multitrait-multimethod data. Educational and Psychological Measurement, 30, 833-853.
21. Bosshardt, M. J. (1994). Situational interviews versus behavior description interviews: A comparative validity study (Doctoral dissertation, University of Minnesota, 1994). Dissertation Abstracts International, 54, 3882.
22. Brief, A. P, Aldag, R. J., & Van Sell, M. (1977). Moderators of the relationship between self and superior evaluations of job performance. Journal of Occupational Psychology, 50, 129-134.
23. Brown, S. H. (1995). [Supervisor, peer, and subordinate ratings of life insurance agency field managers.] Unpublished raw data.
24. Brumback, G. B., & Howell, M. A. (1972). Rating the clinical effectiveness of employed physicians. Journal of Applied Psychology, 56, 241-244.
25. Brutus, S., Fleenor, J. W., & Taylor, S. (1996, April). Methodological issues in 360-degree feedback research. Paper presented at the 11th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
26. Buckner, D. N. (1959). The predictability of ratings as a function of interrater agreement. Journal of Applied Psychology, 43, 60-64.
27. Campbell, D. J. (n.d.). Self-appraisals from two perspectives: Esteem versus consistency influences. Unpublished manuscript, National University of Singapore.
28. Campbell, J. P. (Ed.). (1986). Improving the selection, classification, and utilization of army enlisted personnel: Annual report, 1986 fiscal year (ARI Technical Report 813101). Alexandria, VA: U. S. Army Research Institute for the Behavioral and Social Sciences.
29. Campbell, J. P., Dunnette, M. D., Arvey, R. D., & Hellervik, L. V. (1973). The development and evaluation of behaviorally based rating scales. Journal of Applied Psychology, 57, 15-22.
30. Carroll, S. J., & Schneier, C. E. (1982). Performance appraisal and review systems: The identification, measurement, and development of performance in organizations. Glenview, IL: Scott, Foresman.
31. Church, A. H., Javitch, M., & Burke, W. W. (1996, April). 450 degree multirater feedback: A comparison of performance ratings. Paper presented at the 11th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
32. Cleveland, J. N., & Shore, L. M. (1992). Self- and supervisory perspectives on age and work attitudes and performance. Journal of Applied Psychology, 77, 469-484.
33. Conway, J. M. (1990). [Self- and supervisor ratings of life insurance sales representatives]. Unpublished raw data.
34. Cooper, R. (1966). Leader's task relevance and subordinate behaviour in industrial work groups. Human Relations, 19, 57-84.
35. Cooper, R., & Payne, R. (1967). Extraversion and some aspects of work behavior. Personnel Psychology, 20, 45-57.
36. Cummins, R. C. (1971). Relationship of initiating structure and job performance as moderated by consideration. Journal of Applied Psychology, 55, 489-490.
37. Day, D. V., & Silverman, S. B. (1989). Personality and job performance: Evidence of incremental validity. Personnel Psychology, 42, 25-36.
38. Dicken, C. F., & Black, J. D. (1965). Predictive validity of psychometric evaluations of supervisors. Journal of Applied Psychology, 49, 34-47.
39. Dickinson, T. L., & Tice, T. E. (1973). A multitrait-multimethod analysis of scales developed by retranslation. Organizational Behavior and Human Performance, 9, 421-438.
40. Dunnette, M. D., Perry, D. K., & Mahoney, T. A. (1956). Criteria of executive effectiveness (AFPTRC-TN-56-73). Lackland Air Force Base, TX: Air Force Personnel and Training Research Center.
41. Edgerton, H. A., Feinberg, M. R., & Thomson, K. F. (1957). Prediction of the "human relations" effectiveness of industrial supervisors. Personnel Psychology, 10, 421-431.
42. Erisman, M. S., & Reilly, N. P. (1996, April). The 360 degree performance evaluation: Unique or redundant information ? Paper presented at the 11th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
43. Evans, M. G. (1972). Leadership behavior: Demographic factors and agreement between subordinate and self-descriptions. Personnel Psychology, 25, 649-653.
44. Farh, J.-L., Dobbins, G. H., & Cheng, B.-S. (1991). Cultural relativity in action: A comparison of self-ratings made by Chinese and U. S. workers. Personnel Psychology, 44, 129-147.
45. Ferris, G. R., Yates, V. L., Gilmore, D. C., & Rowland, K. M. (1985). The influence of subordinate age on performance ratings and causal attributions. Personnel Psychology, 38, 545-557.
46. Fleenor, J. W. (1995). [Subordinate feedback ratings for managers]. Unpublished raw data.
47. Fleenor, J. W. (1996). Constructs and developmental assessment centers: Further troubling empirical findings. Journal of Business and Psychology, 10, 319-335.
48. Fleishman, E. A., Harris, E. F., & Burtt, H. E. (1955). Leadership and supervision in industry: An evaluation of a supervisory training program [Bureau of Education Research Monograph No. 33]. Columbus: The Ohio State University.
49. Forehand, G. A. (1963). Assessments of innovative behavior: Partial criteria for the assessment of executive performance. Journal of Applied Psychology, 47, 206-213.
50. Furnham, A., & Stringfield, P. (1994). Congruence of self and subordinate ratings of managerial practices as a correlate of supervisor evaluation. Journal of Occupational and Organizational Psychology, 67, 57-67.
51. Gardner, D. G., Dunham, R. B., Cummings, L. L., & Pierce, J. L. (1989). Focus of attention at work: Construct definition and empirical validation. Journal of Occupational Psychology, 62, 61-77.
52. Greene, C. N. (1973a). Causal connections among managers' merit pay, job satisfaction, and performance. Journal of Applied Psychology, 58, 95-100.
53. Greene, C. N. (1973b, August). A longitudinal analysis of relationships among leader behavior and subordinate performance and satisfaction. Proceedings of the Academy of Management, 433-440.
54. Greene, C. N. (1979). Questions of causation in the path-goal theory of leadership. Academy of Management Journal, 22, 22-41.
55. Griffiths, R. D. P. (1973). A standardized assessment of the work behaviour of psychiatric patients. British Journal of Psychiatry, 123, 403-408.
56. Griffiths, R. D. P. (1975). The accuracy and correlates of psychiatric patients' self-assessment of their work behaviour. British Journal of Social and Clinical Psychology, 14, 181-189.
57. Gunderson, E. K. E., & Nelson, P. D. (1966). Criterion measures for extremely isolated groups. Personnel Psychology, 19, 67-80.
58. Gunderson, E. K. E., & Ryman, D. H. (1971). Convergent and discriminant validities of performance evaluations in extremely isolated groups. Personnel Psychology, 24, 715-724.
59. Hackman, J. R., & Lawler, E. E., III (1971). Employee reactions to job characteristics. Journal of Applied Psychology, 55, 259-286.
60. Hackman, J. R., & Porter, L. W. (1968). Expectancy theory predictions of work effectiveness. Organizational Behavior and Human Performance, 3, 417-426.
61. Halpin, A. W. (1957). The leader behavior and effectiveness of aircraft commanders. In R. M. Stogdill & A. E. Coons (Eds.), Leader behavior: Its description and measurement (pp. 52-64). Columbus: Bureau of Business Research Monograph No. 88, The Ohio State University.
62. Halpin, A. W., & Winer, B. J. (1957). A factorial study of the leader behavior descriptions. In R. M. Stogdill & A. E. Coons (Eds.), Leader behavior: Its description and measurement (pp. 39-51). Columbus: Bureau of Business Research Monograph No. 88, The Ohio State University.
63. Hand, H. H., Richards, M. D., & Slocum, J. W., Jr. (1973). Organizational climate and the effectiveness of a human relations training program. Academy of Management Journal, 16, 185-195.
64. Harris, M. M., & Schaubroeck, J. (1988). A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings. Personnel Psychology, 41, 43-62.
65. Harris, M. M., Smith, D. E., & Champagne, D. (1995). A field study of performance appraisal purpose: Research- versus administrative-based ratings. Personnel Psychology, 48, 151-160.
66. Hartigan, J. A., & Wigdor, A. K. (Eds.). (1989). Fairness in employment testing: Validity generalization, minority issues, and the General Aptitude Test Battery. Washington, DC: National Academy Press.
67. Hater, J. J., & Bass, B. M. (1988). Superiors' evaluations and subordinates' perceptions of transformational and transactional leadership. Journal of Applied Psychology, 73, 695-702.
68. Hausman, H. J., & Strupp, H. H. (1955). Non-technical factors in supervisors' ratings of job performance. Personnel Psychology, 8, 201-217.
69. Hay, E. N. (1954). Comparative validities in clerical testing. Journal of Applied Psychology, 38, 299-301.
70. Hazucha, J. F. Hezlett, S. A., & Schneider, R. J. (1993). The impact of 360-degree feedback on management skills development. Human Resource Management, 32, 325-351.
71. Heneman, H. G., III (1974). Comparisons of self- and superior ratings of managerial performance. Journal of Applied Psychology, 59, 638-642.
72. Heneman, R. L. (1986). The relationship between supervisory ratings and results-oriented measures of performance: A meta-analysis. Personnel Psychology, 39, 811-826.
73. Hicks, J. A., & Stone, J. B. (1962). The identification of traits related to managerial success. Journal of Applied Psychology, 46, 428-432.
74. Hilton, A. C., Bolin, S. F., Parker, J. W., Jr., Taylor, E. K., & Walker, W. B. (1955). The validity of personnel assessments by professional psychologists. Journal of Applied Psychology, 39, 287-293.
75. Hoffman, C. C., Nathan, B. R., & Holden, L. M. (1991). A comparison of validation criteria: Objective versus subjective performance measures and self- versus supervisor ratings. Personnel Psychology, 44, 601-619.
76. Hogan, J., Hogan, R., & Busch, C. M. (1984). How to measure service orientation. Journal of Applied Psychology, 69, 167-173.
77. Holland, J. L. (1996). Exploring careers with a typology: What we have learned and some new directions. American Psychologist, 51, 397-406.
78. Holzbach, R. L. (1978). Rater bias in performance ratings: Superior, self-, and peer ratings. Journal of Applied Psychology, 63, 579-588.
79. Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72-98.
80. Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
81. Hunter, J. E., Schmidt, F. L., & Judiesch, M. K. (1990). Individual differences in output variability as a function of job complexity. Journal of Applied Psychology, 75, 28-42.
82. Johnson, J. A., & Hogan, R. (1981). Vocational interests, personality and effective police performance. Personnel Psychology, 34, 49-53.
83. Jurgensen, C. E. (1950). Overall job success as a basis for employee ratings. Journal of Applied Psychology, 34, 333-337.
84. Kane, J. S., & Lawler, E. E., III (1978). Methods of peer assessment. Psychological Bulletin, 85, 555-586.
85. Kavanagh, M. J., MacKinney, A. C., & Wolins, L. (1971). Issues in managerial performance: Multitrait-multimethod analyses of ratings. Psychological Bulletin, 75, 34-49.
86. King, L. M., Hunter, J. E., & Schmidt, F. L. (1980). Halo in a multidimensional forced-choice performance evaluation scale. Journal of Applied Psychology, 65, 507-516.
87. Kirchner, W. K. (1965). Relationships between supervisory and subordinate ratings for technical personnel. Journal of Industrial Psychology, 3, 57-60.
88. Klieger, W. A., & Mosel, J. N. (1953). The effect of opportunity to observe and rater status on the reliability of performance ratings. Personnel Psychology, 6, 57-63.
89. Klimoski, R. J., & London, M. (1974). Role of the rater in performance appraisal. Journal of Applied Psychology, 59, 445-451.
90. Knauft, E. B. (1949). A selection battery for bake shop managers. Journal of Applied Psychology, 33, 304-315.
91. Kremer, J. F. (1990). Construct validity of multiple measures in teaching, research, and service and reliability of peer ratings. Journal of Educational Psychology, 82, 213-218.
92. Landy, F. J., & Farr, J. L. (1983). The measurement of work performance: Methods, theory, and applications. New York: Academic.
93. Landy, F. J., Farr, J. L., Saal, F. E., & Freytag, W. R. (1976). Behaviorally anchored scales for rating the performance of police officers. Journal of Applied Psychology, 61, 750-758.
94. Landy, F. J., & Guion, R. M. (1970). Development of scales for the measurement of work motivation. Organizational Behavior and Human Performance, 5, 93-103.
95. Lawler, E. E., III (1967). The multitrait-multirater approach to measuring managerial job performance. Journal of Applied Psychology, 51, 369-381.
96. Lawler, E. E., III (1968). A correlational-causal analysis of the relationship between expectancy attitudes and job performance. Journal of Applied Psychology, 52, 462-468.
97. Lawshe, C. H., & McGinley, A. D., Jr. (1951). Job performance criteria studies: I. The job performance of proofreaders. Journal of Applied Psychology, 35, 316-320.
98. Lee, R., Malone, M., & Greco, S. (1981). Multitrait-multimethod-multirater analysis of performance ratings for law enforcement personnel. Journal of Applied Psychology, 66, 625-632.
99. Lee, T. (1959). The selection of student nurses: A revised procedure. Occupational Psychology, 33, 209-216.
100. Levine, H. G., McGuire, C. H., & Nattress, L. W., Jr. (1970). The validity of multiple choice achievement tests as measures of competence in medicine. American Educational Research Journal, 7, 69-82.
101. London, M., & Smither, J. W. (1995). Can multi-source feedback change perceptions of goal accomplishment, self-evaluations, and performance-related outcomes? Theory-based applications and directions for research. Personnel Psychology, 48, 803-839.
102. London, M., & Wohlers, A. J. (1991). Agreement between subordinate and self-ratings in upward feedback. Personnel Psychology, 44, 375-390.
103. Love, K. G. (1981). Comparison of peer assessment methods: Reliability, validity, friendship bias, and user reaction. Journal of Applied Psychology, 66, 451-457.
104. Mabe, P. A., III, & West, S. G. (1982). Validity of self-evaluation of ability: A review and meta-analysis. Journal of Applied Psychology, 67, 280-296.
105. Malka, S. (1990). Application of the multitrait-multirater approach to performance appraisal in social service organizations. Evaluation and Program Planning, 13, 243-250.
106. Maslow, A. H., & Zimmerman, W. (1956). College teaching ability, scholarly activity, and personality. Journal of Educational Psychology, 47, 185-189.
107. Masztal, J. J., & Bourque, M. D. (1990). Criterion development for the validation of AIM: Assessment Inventory for Management (Tech. Rep. MRR7-1990). Farmington, CT: Life Insurance Marketing and Research Association.
108. McDermid, C. D. (1965). Some correlates of creativity in engineering personnel. Journal of Applied Psychology, 49, 14-19.
109. McEvoy, G. M., & Beatty, R. W. (1989). Assessment centers and subordinate appraisals of managers: A seven-year examination of predictive validity. Personnel Psychology, 42, 37-52.
110. Meritt-Haston, R. E. (1984). Explaining interrater reliability (Doctoral dissertation, Michigan State University, 1984). Dissertation Abstracts International, 45, 2723.
111. Miner, J. B. (1970). Psychological evaluations as predictors of consulting success. Personnel Psychology, 23, 393-405.
112. Miner, J. B. (1988). Development and application of the rated ranking technique in performance appraisal. Journal of Occupational Psychology, 61, 291-305.
113. Morgan, R. B. (1993). Self- and co-worker perceptions of ethics and their relationships to leadership and salary. Academy of Management Journal, 36, 200-214.
114. Motowidlo, S. J. (1982). Relationship between self-rated performance and pay satisfaction among sales representatives. Journal of Applied Psychology, 67, 209-213.
115. Mount, M. K. (1984). Psychometric properties of subordinate ratings of managerial performance. Personnel Psychology, 37, 687-702.
116. Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance appraisal: Social, organizational, and goal-based perspectives. Thousand Oaks, CA: Sage.
117. Naas, P. A., & Sachau, D. A. (1996, April). The relationship between self-subordinate performance rating congruence and subordinate job satisfaction. Paper presented at the 11th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
118. Nelson, P. D. (1964). Supervisor esteem and personnel evaluations. Journal of Applied Psychology, 48, 106-109.
119. Nilsen, D. (1996). [Ratings using the Campbell Leadership Inventory]. Unpublished raw data.
120. Norton, S. M. (1992). Peer assessments of performance and ability: An exploratory meta-analysis of statistical artifacts and contextual moderators. Journal of Business and Psychology, 6, 387-399.
121. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.
122. Oldham, G. R. (1976). The motivational strategies used by supervisors: Relationships to effectiveness indicators. Organizational Behavior and Human Performance, 15, 66-86.
123. Orpen, C. (1973). An empirical assessment of the job performance of high-level executives by means of a multitrait-multimethod matrix. Psychologia Africana, 15, 7-14.
124. Ostroff, C. (1991). Training effectiveness measures and scoring schemes: A comparison. Personnel Psychology, 44, 353-374.
125. Parker, J. W., Taylor, E. K., Barrett, R. S., & Martens, L. (1959). Rating scale content: III. Relationship between supervisory- and self-ratings. Personnel Psychology, 12, 49-63.
126. Piercy, F. P. (1974). Relationships among counselor effectiveness self-ratings, peer ratings, supervisor ratings, and client ratings. (ERIC Document Reproduction Service No. ED 130 188)
127. Piotrowski, M. J. (1993). [Ratings of human resource workers]. Unpublished raw data.
128. Pollak, F. (1994). [Team assessments]. Unpublished raw data.
129. Porter, L. W., & Lawler, E. E., III (1968). Managerial attitudes and performance. Homewood, IL: Irwin.
130. Prien, E. P., & Liske, R. E. (1962). Assessments of higher-level personnel: III. Rating criteria: A comparative analysis of supervisor ratings and incumbent self-ratings of job performance. Personnel Psychology, 15, 187-194.
131. Pulakos, E. D., & Barman, W. C. (Eds.). (1986). Development and field test of army-wide rating scales and the rater orientation and training program (Tech. Rep. No. 716). Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.
132. Pulakos, E. D., & Schmitt, N. (1995). Experience-based and situational interview questions: Studies of validity. Personnel Psychology, 48, 289-308.
133. Pym, D. L. A., & Auld, H. D. (1965). The self-rating as a measure of employee satisfactoriness. Occupational Psychology, 39, 103-113.
134. Riggio, R. E., & Cole, E. J. (1992). Agreement between subordinate and superior ratings of supervisory performance and effects on self and subordinate job satisfaction. Journal of Occupational and Organizational Psychology, 65, 151-158.
135. Robinson, D. D. (1970). Predicting police effectiveness from self reports of relative time spent in task performance. Personnel Psychology, 23, 327-345.
136. Rose, R. M., Helmreich, R. L., Fogg, L., & McFadden, T. J. (1993). Assessments of astronaut effectiveness. Aviation, Space, & Environmental Medicine, 64, 789-794.
137. Rosinger, G., Myers, L. B., Levy, G. W., Loar, M., Mohrman, S. A., & Stock, J. R. (1982). Development of a behaviorally based performance appraisal system. Personnel Psychology, 35, 75-88.
138. Rosse, J. G., & Kraut, A. I. (1983). Reconsidering the vertical dyad linkage model of leadership. Journal of Occupational Psychology, 56, 63-71.
139. Rothstein, H. R. (1990). Interrater reliability of job performance ratings: Growth to asymptote level with increasing opportunity to observe. Journal of Applied Psychology, 75, 322-327.
140. Rupe, J. C. (1951). When workers rate the boss. Personnel Psychology, 4, 271-289.
141. Rushmore, J. T. (1958). A note on the "Test of Practical Judgment." Personnel Psychology, 11, 37.
142. Russell, C. J., Mattson, J., Devlin, S. E., & Atwater, D. (1990). Predictive validity of biodata items generated from retrospective life experience essays. Journal of Applied Psychology, 75, 569-580.
143. Saal, F. E., & Landy, F. J. (1977). The mixed standard rating scale: An evaluation. Organizational Behavior and Human Performance, 18, 19-35.
144. Schmitt, N., Noe, R. A., Menu, R., & Fitzgerald, M. P. (1984). Validity of assessment center ratings for the prediction of performance ratings and school climate of school administrators. Journal of Applied Psychology, 69, 207-213.
145. Schmitt, N., & Saari, B. B. (1978). Behavior, situation, and rater variance in descriptions of leader behaviors. Multivariate Behavioral Research, 13, 483-495.
146. Scullen, S. E., Mount, M. K., & Sytsma, M. R. (1996, April). Comparisons of self, peer, direct report and boss ratings of managers' performance. Paper presented at the 11th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
147. Seeman, M. (1957). A comparison of general and specific leader behavior descriptions. In R. M. Stogdill & A. E. Coons (Eds.), Leader behavior: Its description and measurement (pp. 86-102). Columbus: Bureau of Business Research Monograph No. 88, The Ohio State University.
148. Seybolt, J. W., & Pavett, C. M. (1979). The prediction of effort and performance among hospital professionals: Moderating effects of feedback on expectancy theory formulations. Journal of Occupational Psychology, 52, 91-105.
149. Sheridan, J. E., Fairchild, T. J., & Kaas, M. (1983). Assessing the job performance of nursing home staff. Nursing Research, 32, 102-107.
150. Shore, L. M., & Bleicken, L. M. (1991). Effects of supervisor age and subordinate age on rating congruence. Human Relations, 44, 1093-1105.
151. Smith, C. A., Organ, D. W., & Near, J. P. (1983). Organizational citizenship behavior: Its nature and antecedents. Journal of Applied Psychology, 68, 653-663.
152. Smith, M. R., & Whitney, D. J. (1995, May). The validity of employment interviews: Does the interviewer make a difference? Paper presented at the 10th annual conference of the Society for Industrial and Organizational Psychology, Orlando, FL.
153. Smither, J. W., London, M., Vasilopoulos, N. L., Rally, R. R., Millsap, R. E., & Salvemini, N. (1995). An examination of the effects of an upward feedback program over time. Personnel Psychology, 48, 1-34.
154. Smither, J. W., Wohlers, A. J., & London, M. (1995). A field study of reactions to normative versus individualized upward feedback. Group & Organization Management, 20, 61-89.
155. Sorenson, W. W. (1966). Test of mechanical principles as a suppressor variable for the prediction of effectiveness on a mechanical repair job. Journal of Applied Psychology, 50, 348-352.
156. Spector, A. J. (1957). Human relations behavior on the job: The officer behavior description. Journal of Applied Psychology, 41, 110-113.
157. Sprecher, T. B. (1959). A study of engineers' criteria for creativity. Journal of Applied Psychology, 43, 141-148.
158. Stahl, M. J., & Steger, J. A. (1977). Measuring innovation and productivity - A peer rating approach. Research Management, 20, 35-38.
159. Steel, R. P., & Ovalle, N. K., II. (1984). Self-appraisal based upon supervisory feedback. Personnel Psychology, 37, 667-685.
160. Stogdill, R. M., Scott, E. L., & Jaynes, W. E. (1956). Leadership and role expectations [Bureau of Business Research Monograph No. 86]. Columbus: The Ohio State University.
161. Stogdill, R. M., & Shartle, C. L. (1955). Methods in the study of administrative leadership [Bureau of Business Research Monograph No. 80]. Columbus: The Ohio State University.
162. Stoltz, R. E. (1958). Development of a criterion of research productivity. Journal of Applied Psychology, 42, 308-310.
163. Sypher, B. D., & Sypher, H. E. (1984). Seeing ourselves as others see us: Convergence and divergence in assessments of communication behavior. Communication Research, 11, 97-115.
164. Thomson, H. A. (1970). Comparison of predictor and criterion judgments of managerial performance using the multitrait-multimethod approach. Journal of Applied Psychology, 54, 496-502.
165. Thornton, G. C. (1968). The relationship between supervisory- and self-appraisals of executive performance. Personnel Psychology, 21, 441-455.
166. Tornow, W. W. (1993). Perceptions or reality: Is multi-perspective measurement a means or an end? Human Resource Management, 32, 221-229.
167. Tsui, A. S., & McGregor, J. (1982). The multiple-rater approach to assessing managerial performance: Further empirical evidence. Proceedings of the 14th Annual Meeting of the American Institute for Decision Sciences, 508-510.
168. Tsui, A. S., & Ohlott, P. (1988). Multiple assessment of managerial effectiveness: Interrater agreement and consensus in effectiveness models. Personnel Psychology, 41, 779-803.
169. Tucker, M. F., Cline, V. B., & Schmitt, J. R. (1967). Prediction of creativity and other performance measures from biographical information among pharmaceutical scientists. Journal of Applied Psychology, 51, 131-138.
170. Validity Information Exchange, No. 11-10 (1958). Personnel Psychology, 11, 121-123.
171. Validity Information Exchange, No. 13-03 (1960). Personnel Psychology, 13, 449-450.
172. Van Velsor, E., & Leslie, J. B. (1991a). Feedback to managers, Volume I: A guide to evaluating multi-rater feedback instruments. Greensboro, NC: Center for Creative Leadership.
173. Van Velsor, E., & Leslie, J. B. (1991b). Feedback to managers, Volume II: A review and comparison of sixteen multi-rater feedback instruments. Greensboro, NC: Center for Creative Leadership.
174. Villanova, P., & Bernardin, H. J. (1990). Work behavior correlates of interviewer job compatibility. Journal of Business and Psychology, 5, 179-195.
175. Viswesvaran, C. (1996, April). Modeling job performance: Is there a general factor? Paper presented at the 11th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
176. Viswesvaran, C., Ones, D. S., & Schmidt, F. L. (1996). Comparative analysis of the reliability of job performance ratings. Journal of Applied Psychology, 81, 557-574.
177. Waag, W. L., & Houck, M. R. (1994). Tools for assessing situational awareness in an operational fighter environment. Aviation, Space. & Environmental Medicine, 65, A13-A19.
178. Warmke, D. L., & Billings, R. S. (1979). Comparison of training methods for improving the psychometric quality of experimental and administrative performance ratings. Journal of Applied Psychology, 64, 124-131.
179. Wheeler, A. E., & Knoop, H. R. (1982). Self, teacher and faculty assessments of student teaching performance. Journal of Educational Research, 75, 178-181.
180. Willemin, L. P. (1965). Criterion aspects of army research on the prediction of officer performance (ARI Research Study 65-6).
181. Williams, W. E., & Seiler, D. A. (1973). Relationship between measures of effort and job performance. Journal of Applied Psychology, 57, 49-54.
182. Wohlers, A. J., Hall, M.-J., & London, M. (1993). Subordinates rating managers: Organizational and demographic correlates of self-subordinate agreement. Journal of Occupational and Organizational Psychology, 66, 263-275.
183. Wohlers, A. J., & London, M. (1989). Ratings of managerial characteristics: Evaluation difficulty, co-worker agreement, and self-awareness. Personnel Psychology, 42, 235-261.
184. Woodmansee, J. J. (1978). Validation of the nurturance scale of the Edwards Personal Preference Schedule. Psychological Reports, 42, 495-498.
185. Worbois, G. M. (1975). Validation of externally developed assessment procedures for identification of supervisory potential. Personnel Psychology, 28, 77-91.
186. Yammarino, F. J., & Dubinsky, A. J. (1994). Transformational leadership theory: Using levels of analysis to determine boundary conditions. Personnel Psychology, 47, 787-811.
187. Yu, J., & Murphy, K. R. (1993). Modesty bias in self-ratings of performance: A test of the cultural relativity hypothesis. Personnel Psychology, 46, 357-363.
188. Yukl, G. A., & Nemeroff, W. F. (1979). Identification and measurement of specific categories of leadership behavior: A progress report. In J. G. Hunt & L. L. Larson (Eds.), Crosscurrents in leadership (pp. 164-200). Carbondale: Southern Illinois University Press.
189. Zalesny, M. D., & Kirsch, M. P. (1989). The effect of similarity on performance ratings and interrater agreement. Human Relations, 42, 81-96.