Are performance appraisal ratings from different rating sources comparable?

Journal of Applied Psychology

Volumn 86, Issue 2, 2001, Pages 215-227

  Facteau, Jeffrey D ^b   Craig, Bartholomew S ^a  

^a Kaplan DeVries Inc   (United States)

^b AUBURN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; COMPARATIVE STUDY; FACTORIAL ANALYSIS; FEEDBACK SYSTEM; HUMAN; OBSERVER VARIATION; PERSONNEL MANAGEMENT; STATISTICS;

EMPLOYEE PERFORMANCE APPRAISAL; FACTOR ANALYSIS, STATISTICAL; FEEDBACK; HUMANS; OBSERVER VARIATION;

EID: 85047682744     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/0021-9010.86.2.215     Document Type: Article

References (56)

1. Allen, R. L., & Oshagan, H. (1995). The UCLA Loneliness Scale: Invariance of social structural characteristics. Personality Individual Differences, 19, 185-195.
2. Baker, F. (1995). EQUATE 2.1: Computer program for equating two metrics in item response theory. Madison: University of Wisconsin, Laboratory of Experimental Design.
3. Banks, C. A., & Murphy, K. R. (1985). Toward narrowing the research-practice gap in performance appraisal. Personnel Psychology, 38, 335-345.
4. Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238-246.
5. Bentler, P. M. (1995). EQS structural equations program manual. Encino, CA: Multivariate Software, Inc.
6. Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588-606.
7. Bernardin, H. J., & Villanova, P. (1986). Performance appraisal. In E. Locke (Ed.), Generalizing from laboratory to field settings (pp. 43-62). Lexington, MA: Lexington Books.
8. Bollen, K. A. (1989). Structural equations modeling with latent variables. New York: Wiley.
9. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.
10. Byrne, B. (1989). A primer of LISREL: Basic applications and programming for confirmatory factor analytic models. New York: Springer-Verlag.
11. Byrne, B. (1993). The Maslach Burnout Inventory: Testing for factorial validity and invariance across elementary, intermediate and secondary teachers. Journal of Occupational and Organizational Psychology, 66, 197-212.
12. Byrne, B. M. (1994). Structural equation modeling with EQS and EQS/ Windows: Basic concepts, applications, and programming. Thousand Oaks, CA: Sage.
13. Byrne, B. M., Baron, P., & Campbell, T. L. (1993). Measuring adolescent depression: factorial validity and invariance of the Beck Depression Inventory across gender. Journal of Research on Adolescence, 3, 127-143.
14. Byrne, B. M., Shavelson, R. J., & Muthen, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456-466.
15. Campbell, D. J., & Lee, C. (1988). Self-appraisal in performance evaluation: Development versus evaluation. Academy of Management Review, 13, 302-314.
16. Candell, G. L., & Drasgow, F. (1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 12, 253-260.
17. Cardy, R. L., & Dobbins, G. H. (1994). Performance appraisal: Alternative perspectives. Cincinnati, OH: South-Western Publishing Co.
18. Cascio, W. F. (1991). Applied psychology in personnel management (4th ed.), Englewood Cliffs, NJ: Prentice-Hall.
19. Cleveland, J. N., Murphy, K. R., & Williams, R. E. (1989). Multiple uses of performance appraisal: Prevalence and correlates. Journal of Applied Psychology, 74, 130-135.
20. Conway, J. M., & Huffcutt, A. I. (1997). Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer, and self-ratings. Human Performance, 10, 331-360.
21. Drasgow, F. (1984). Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables are central issues. Psychological Bulletin, 95, 134-135.
22. Drasgow, F. (1987). Study of the measurement bias of two standardized psychological tests. Journal of Applied Psychology 72, 19-29.
23. Drasgow, F., & Kanfer, R. (1985). Equivalence of psychological measurement in heterogeneous populations. Journal of Applied Psychology, 70, 662-680.
24. Dunnette, M. D. (1993). My hammer or your hammer? Human Resource Management, 32, 373-384.
25. Farh, J. L., & Dobbins, G. H. (1989a). Effects of comparative performance information on the accuracy of self-ratings and agreement between self-and supervisor ratings. Journal of Applied Psychology, 74, 606-610.
26. Farh, J. L., & Dobbins, G. H. (1989b). Effects of self-esteem on leniency bias in self-reports of performance: A structural equation model analysis. Personnel Psychology, 42, 835-850.
27. Gerbing, D. W., & Anderson, J. C. (1984). On the meaning of within-factor correlated measurement errors. Journal of Consumer Research, 11, 572-580.
28. Harris, M. M., & Schaubroeck, J. (1988). A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings. Personnel Psychology, 41, 43-62.
29. Hoyle, R. H. (1991). Evaluating measurement models in clinical research: Covariance structure analysis of latent variable models of self-conception. Journal of Consulting and Clinical Psychology, 59, 67-76.
30. Hu, L., & Bentler, P. M. (1995). Evaluating model lit. In R. H. Hoyle (Ed.), Structural equation modeling (pp. 158-176). Thousand Oaks, CA: Sage.
31. Idaszak, J. R., Bottom, W. P., & Drasgow, F. (1988). A test of the measurement equivalence of the revised Job Diagnostic Survey: Past problems and current solutions. Journal of Applied Psychology, 73, 647-656.
32. Kline, R. B. (1998). Principles and practices of structural equation modeling. New York: Guilford Press.
33. Lance, C. E., & Bennett, W., Jr. (1997, April). Rater source differences in cognitive representations of performance information. Paper presented at the annual meeting of the Society for Industrial and Organizational Psychology, St. Louis, MO.
34. Lance, C. E., Teachout, M. S., & Donnelly, T. M. (1992). Specification of the criterion construct space: An application of hierarchical confirmatory factor analysis. Journal of Applied Psychology, 77, 437-452.
35. Lance, C. E., & Woehr, D. J. (1989). The validity of performance judgments: Normative accuracy model versus ecological perspectives. In D. F. Ray (Ed.), Proceedings of the Southern Management Association (pp. 115-117). Starkville, MS: Southern Management Association.
36. Landy, F. J., & Farr, J. L. (1980). Performance rating. Psychological Bulletin, 87, 72-107.
37. London, M., & Smither, J. W. (1995). Can multi-source feedback change perceptions of goal accomplishment, self-evaluations, and performance related outcomes? Theory-based applications and directions for research. Personnel Psychology, 48, 803-839.
38. Lord, F. M. (1952). A theory of test scores. Psychometric Monograph, 7.
39. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
40. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130-149.
41. MacCallum, R. C., & Tucker, L. R. (1991). Representing sources of error in common factor analysis: Implications for theory and practice. Psychological Bulletin, 109, 50-511.
42. Maurer, T. J., Raju, N. S., & Collins, W. C. (1998). Peer and subordinate performance appraisal measurement equivalence. Journal of Applied Psychology, 83, 693-702.
43. Mount, M. K. (1984). Psychometric properties of subordinate ratings of managerial performance. Personnel Psychology, 37, 687-702.
44. Mount, M. K., Judge, T. A., Scullen, S. E., Sytsma, M. R., & Hezlett, S. A. (1998). Trait, rater and level effects in 360-degree performance ratings. Personnel Psychology, 51, 557-576.
45. Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance appraisal: Social, organizational, and goal based perspectives. Thousand Oaks, CA: Sage.
46. Raju, N. S. (1999). DFIT4P: A computer program for analyzing differential item and test functioning. Chicago: Illinois Institute of Technology.
47. Raju, N. S., van der Linden, W., & Fleer, P. (1995). An IRT-based internal measure of test bias with applications for differential item functioning. Applied Psychological Measurement, 19, 353-368.
48. Reckase, M. D. (1979). Unifactor latent trait models applied to multi-factor tests: Results and implications. Journal of Educational Statistics, 4, 207-230.
49. Reddy, S. K. (1992). Effects of ignoring correlated measurement error in structural equation models. Educational and Psychological Measurement, 52, 549-570.
50. Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552-566.
51. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph, 17.
52. Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201-210.
53. Thissen, D. (1995). MULTILOG 6.3: A computer program for multiple, categorical item analysis and test scoring using item response theory. Chicago: Scientific Software, Inc.
54. Thornton, G. C., III (1980). Psychometric properties of self-appraisals of job performance. Personnel Psychology, 33, 263-271.
55. Tornow, W. W. (1993). Editor's note: Introduction to the special issue on 360-degree feedback. Human Resource Management, 32, 211-219.
56. Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1-10.