Understanding method variance in multitrait-multirater performance appraisal matrices: Examples using general impressions and interpersonal affect as measured method factors

Human Performance

Volumn 11, Issue 1, 1998, Pages 29-55

  Conway, James M ^a  

^a SETON HALL UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0032351449     PISSN: 08959285     EISSN: None     Source Type: Journal    
DOI: 10.1207/s15327043hup1101_2     Document Type: Article

References (39)

1. Balzer, W. K., & Sulsky, L. M. (1992). Halo and performance appraisal research: A critical examination. Journal of Applied Psychology, 77, 975-985.
2. Becker, T. E., & Cote, J. A. (1994). Additive and multiplicative method effects in applied psychological research: An empirical assessment of three models. Journal of Management, 20, 625-641.
3. Byrne, B. M. (1984). The general/academic self-concept nomological network: A review of construct validation research. Review of Educational Research, 54, 427-456.
4. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.
5. Cardy, R. L., & Dobbins, G. H. (1986). Affect and appraisal accuracy: Liking as an integral dimension in evaluating performance. Journal of Applied Psychology, 71, 672-678.
6. Conway, J. M. (1996). Analysis and design of multitrait-multirater performance appraisal studies. Journal of Management, 22, 139-162.
7. Conway, J. M. (in press). Estimation and uses of the proportion of method variance for multitrait-multimethod data. Organizational Research Methods.
8. Cooper, W. H. (1981). Ubiquitous halo. Psychological Bulletin, 90, 218-244.
9. Dansereau, F., Jr., Graen, G., & Haga, W. J. (1975). A vertical dyad linkage approach to leadership within formal organizations: A longitudinal investigation of the role making process. Organizational Behavior and Human Performance, 13, 46-78.
10. Ferris, G. R., & Judge, T. A. (1991). Personnel/human resources management: A political influence perspective. Journal of Management, 17, 447-488.
11. Fleenor, J. W. (1996). Constructs and developmental assessment centers: Further troubling empirical findings. Journal of Business and Psychology, 10, 319-335.
12. Hastie, R., & Park, B. (1986). The relationship between memory and judgment depends on whether the judgment task is memory-based or on-line. Psychological Review, 93, 258-268.
13. Holzbach, R. L. (1978). Rater bias in performance ratings: Superior, self-, and peer ratings. Journal of Applied Psychology, 63, 579-588.
14. Jöreskog, K. G., & Sörbom, D. (1993). LISREL 8 user's reference guide. Chicago: Scientific Software International.
15. Kenny, D. A., & Kashy, D. A. (1992). Analysis of the multitrait-multimethod matrix by confirmatory factor analysis. Psychological Bulletin, 112, 165-172.
16. King, L. M., Hunter, J. E., & Schmidt, F. L. (1980). Halo in a multidimensional forced-choice performance evaluation scale. Journal of Applied Psychology, 65, 507-516.
17. Lance, C. E., LaPointe, J. A., & Fisicaro, S. A. (1994). Tests of three causal models of halo rater error. Organizational Behavior and Human Decision Processes, 57, 83-96.
18. Lance, C. E., LaPointe, J. A., & Stewart, A. M. (1994). A test of the context dependency of three causal models of halo rater error. Journal of Applied Psychology, 79, 332-340.
19. Lance, C. E., Teachout, M. S., & Donnelly, T. M. (1992). Specification of the criterion construct space: An application of hierarchical confirmatory factor analysis. Journal of Applied Psychology, 77, 437-452.
20. Lawler, E. E., III. (1967). The Multitrait-Multi rater approach to measuring managerial job performance. Journal of Applied Psychology, 51, 369-381.
21. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130-149.
22. Marsh, H. W. (1989). Confirmatory factor analyses of multitrait-multimethod data: Many problems and a few solutions. Applied Psychological Measurement, 13, 335-361.
23. Murphy, K. R. (1991). Criterion issues in performance appraisal research: Behavioral accuracy versus classification accuracy. Organizational Behavior and Human Decision Processes, 50, 45-50.
24. Murphy, K. R., Jako, R. A., & Anhalt, R. L. (1993). Nature and consequences of halo error: A critical analysis. Journal of Applied Psychology, 78, 218-225.
25. Pulakos, E. D., Schmitt, N., & Chan, D. (1996). Models of job performance ratings: An examination of ratee race, ratee gender, and rater level effects. Human Performance, 9, 103-119.
26. Robbins, T. L., & DeNisi, A. S. (1994). A closer look at interpersonal affect as a distinct influence on cognitive processing in performance evaluations. Journal of Applied Psychology, 79, 341-353.
27. Schmitt, N. (1994). Method bias: The importance of theory and measurement. Journal of Organizational Behavior, 15, 393-398.
28. Schmitt, N., & Stults, D. M. (1986). Methodology review: Analysis of multitrait-multimethod matrices. Applied Psychological Measurement, 10, 1-22.
29. Srull, T. K., & Wyer, R. S., Jr. (1989). Person memory and judgment. Psychological Review, 96, 58-83.
30. Thorndike, E. L. (1920). A constant error in psychological ratings. Journal of Applied Psychology, 4, 25-29.
31. Tornow, W. W. (1993). Perceptions or reality: Is multi-perspective measurement a means or an end? Human Resource Management, 32, 221-229.
32. Tucker, M. F., Cline, V. B., & Schmitt, J. R. (1967). Prediction of creativity and other performance measures from biographical information among pharmaceutical scientists. Journal of Applied Psychology, 51, 131-138.
33. Varma, A., DeNisi, A. S., & Peters, L. H. (1996). Interpersonal affect and performance appraisal: A field study. Personnel Psychology, 49, 341-360.
34. Wherry, R. J., & Fryer, D. H. (1949). Buddy ratings: Popularity contest or leadership criteria? Personnel Psychology, 2, 147-159.
35. Widaman, K. F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9, 1-26.
36. Williams, L. J., & Anderson, S. E. (1994). An alternative approach to method effects by using latent-variable models: Applications in organizational behavior research. Journal of Applied Psychology, 79, 323-331.
37. Williams, L. J., Gavin, M. B., & Williams, M. L. (1996). Measurement and nonmeasurement processes with negative affectivity and employee attitudes. Journal of Applied Psychology, 81, 88-101.
38. Wyer, R. S., Jr., & Srull, T. K. (1986). Human cognition in its social context. Psychological Review, 93, 322-359.
39. Yukl, G. A. (1994). Leadership in organizations (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.