Assessing Information Technology Personnel: Toward a Behavioral Rating Scale

Data Base for Advances in Information Systems

Volumn 35, Issue 3, 2004, Pages 88-104

  Chilton, Michael A ^a   Hardgrave, Bill C ^b  

^a KANSAS STATE UNIVERSITY   (United States)

^b UNIVERSITY OF ARKANSAS   (United States)

Author keywords

assessment; behavior; behaviorally based rating scale; IT personnel; performance measures; performance ratings; productivity

Indexed keywords

EID: 28844504343     PISSN: 00950033     EISSN: None     Source Type: Journal    
DOI: 10.1145/1017114.1017122     Document Type: Article

References (51)

1. Anderson, J. C., and Gerbing, D. W. (1988). “Structural equation modeling in practice: A review and recommended two-step approach,” Psychological Bulletin, Vol. 103, pp. 411–423.
2. Ang, S., and Slaughter, S.A. (2001). “Work outcomes and job design for contract versus permanent information systems professionals on software development teams,” MIS Quarterly, Vol. 25, No. 3, pp. 321–350.
3. Arvey, R. D., and Hoyle, J. C. (1974). “A Guttman approach to the development of behaviorally based rating scales for systems analysts and programmer/analysts,” Journal of Applied Psychology, Vol. 59, No. 1), pp. 61–65.
4. Arvey, R. D., and Renz, G. L. (1992). “Fairness in the selection of employees,” Journal of Business Ethics, Vol. 11, pp. 331–340.
5. Bartol, K.M., and Martin, D.C. (1982). “Managing information systems personnel: A review of the literature and managerial implications,” MIS Quarterly, Vol. 6, Special Issue, pp. 49–70.
6. Beatty, R.W., Schneier, C.E., and Beatty, J.R. (1977). “An empirical investigation of perceptions of ratee behavior frequency and ratee behavior change using behavioral expectation scales,” Personnel Psychology, Vol. 30, No. 4, pp. 647–658.
7. Bernard, H.R. (2000). Social research methods: Qualitative and quantitative approaches. Sage Publications, Inc., Thousand Oaks, CA.
8. Bernardin, H.J., and Smith, P.C. (1981). “A clarification of some issues regarding the development and use of behaviorally anchored rating scales (BARS),” Journal of Applied Psychology, Vol. 66, No. 4, pp. 458–463.
9. Bollen, K. A. (1989). Structural Equations with Latent Variables. John Wiley and Sons, New York, NY.
10. Brett, J.F., and Atwater, L.E. (2001). “360 degree feedback: Accuracy, reactions, and perceptions of usefulness,” Journal of Applied Psychology, Vol. 86, No. 5, pp. 930–942.
11. Browne, M.W. and Cudeck, R. (1989). “Single sample cross-validation indices for covariance structures,” Multivariate Behavioral Research, Vol. 24, No. 4, pp. 445–455.
12. Browne, M.W. and Cudeck, R. (1993). “Alternative ways of assessing model fit,” in Testing Structural Equation Models, Bollen, K.A. and Long, J.S. (eds.), Sage Publications, Newbury Park, CA, pp. 136–162.
13. Campbell, J. P. (1990). “Modeling the performance prediction problem in industrial and organizational psychology,” in Handbook of Industrial and Organizational Psychology, M. D. Dunnette and L. M. Hough (eds.), Consulting Psychologists Press, Palo Alto, CA, pp. 687–732.
14. Campbell, J.P. (1994). “Models of measurement and their implications for research on the linkages between individual and organizational productivity,” in Organizational Linkages: Understanding the Productivity Paradox, Harris, D.H., (ed.), National Academies Press, Washington, D.C., pp. 193–213.
15. Campbell, J. P., Dunnette, M. D., Arvey, R. D., and Hellervik, L. V. (1973). “The development and evaluation of behaviorally based rating scales,” Journal of Applied Psychology Vol. 57, No. 4, pp. 15–22.
16. Carmines, E. G., and Zeller, R. A. (1979). Reliability and Validity Assessment, Sage Publications, Thousand Oaks, CA.
17. Cudeck, R. and Browne, M.W. (1982). “Cross validation of covariance structures,” Multivariate Behavioral Research, Vol. 18, pp. 147–167.
18. Ferratt, T.W., and Agarwal, R. (1994). “Guided self development: An innovative approach to management education and development applied to information systems supervisors,” Computer Personnel, Vol. 15, No. 3, pp. 3–20.
19. Ferratt, T. W., and Short, L. E. (1988). “Are information systems people different? An investigation of how they are and should be managed, ”MIS Quarterly, Vol. 12, No. 3, pp. 427 443.
20. Gomez-Mejia, L.R. (1988). “Evaluating employee performance: Does the appraisal instrument make a difference,” Journal of Organizational Behavior Management, Vol. 9, No. 2, pp. 155–172.
21. Green, G. I. (1989). “Perceived importance of system analysts’ job skills, roles, and non-salary incentives,” MIS Quarterly, Vol. 13, No. 2, pp. 114–133.
22. Greenhaus, J. H., Parasuraman, S., and Wormley, W. M. (1990). “Effects of race on organizational experiences, job performance evaluations, and career outcomes,” Academy of Management Journal, Vol. 33, No. 1, pp. 64–86.
23. Hatcher, L. (1994). A Step-by-step Approach to Using the SAS(r) System for Factor Analysis and Structural Equation Modeling, SAS(r) Institute, Cary, NC.
24. Hesketh, B., and Neal, A. (1999). “Technology and performance,” in Implications for Staffing, Motivation, and Development, Daniel Ilgen and Elain Pulakos, (eds), Jossey-Bass, San Francisco, CA, pp. 21–55.
25. Igbaria, M., and Baroudi, J. (1995). “The impact of job performance evaluations on career advancement prospects: An examination of gender differences in the IS workplace,” MIS Quarterly, Vol. 19, No. 1, pp. 107–123.
26. Igbaria, M., and Greenhaus, J.H. (1992). “The career advancement prospects of managers and professionals: Are MIS employees unique?” Decision Sciences, Vol. 23, No. 2, pp. 478–499.
27. Igbaria, M., and Wormley, W. (1992). “Organizational experiences and career success of MIS professionals and managers: An examination of race differences,” MIS Quarterly, Vol. 16, No. 4, pp. 507–529.
28. Jiang, J.J., Sobol, M.G., and Klein, G. (2000). “Performance ratings and importance of performance measures for IS staff: The different perceptions of IS users and IS staff,” IEEE Transactions on Engineering Management, Vol. 47, No. 4, pp. 424–434.
29. Joreskog, K. & Sorbom, D. (1993). LISREL 8: Structural Equation Modeling with the SIMPLIS Command Language. L. Erlbaum Associates, Hillsdale, NJ.
30. Khan, M.B., and Kukalis, S. (1990). “MIS professionals: Education and performance,” Information and Management, Vol. 19, pp. 249 255.
31. Kavanagh, M.J. (1971). “The content issue in performance appraisal: A review,” Personnel Psychology, Vol. 24, No. 4, pp. 653–668.
32. Ketler, K., and Smith, R.D. (1993). “Placement, performance and turnover of information systems professionals: Implications for HRIS,” Proceedings of the 1993 ACM SIGCPR Conference, April 1–3, St. Louis, MO, ACM Press, New York, NY, pp. 131–138.
33. Killingworth, B.L., Hayden, M.B., Crawford, D., and Schellenberger, R. (2001). “A model for motivating and measuring quality performance in information systems staff,” Information Systems Management, Vol. 18, No. 2, pp. 8–14.
34. Landy, F.J., and Farr, J.L. (1980). “Performance rating,” Psychological Bulletin, Vol. 87, No. 1, pp. 72–107.
35. Latham, G.P., and Wexley, K.N. (1977). “Behavioral observation scales for performance appraisal purposes,” Personnel Psychology, Vol. 30, No. 2, pp. 255–268.
36. Lee, D. M. S., Trauth, E. M., and Farwell, D. (1995). “Critical skills and knowledge requirements of IS professionals: A joint academic/industry investigation,” MIS Quarterly, Vol. 19, No. 3, pp. 313–340.
37. Leitheiser, R. L. (1992). “MIS skills for the 1990s: A survey of MIS managers’ perceptions,” Journal of Management Information Systems, Vol. 9, No. 1, pp. 69–91.
38. Murphy, K.R., and Constans, J.I. (1987). “Behavioral anchors as a source of bias in rating,” Journal of Applied Psychology, Vol. 72, No. 4, pp. 573–577.
39. Murphy, K.R., Balzer, W.K., Lockhart, M.C., and Eisenman, E.J. (1985). “Effects of previous performance on evaluations of present performance,” Journal of Applied Psychology, Vol. 70, no. 1, pp. 72–84.
40. Murphy, K.R., Martin, C., and Garcia, M. (1982). “Do behavioral observation scales measure observation?” Journal of Applied Psychology, Vol. 67, No. 5, pp. 562–567.
41. Pedhazur, E. J., and Schmelkin, L. P. (1991). Measurement, Design, and Analysis: An Integrated Approach, Lawrence Erlbaum Associates, Hillsdale, NJ.
42. Rasch, R.H., and Tosi, H.L. (1992). “Factors affecting software developers’ performance: An integrated approach,” MIS Quarterly, Vol. 16, No. 3, pp. 395–413.
43. Ruch, W. A. (1994). “Managing and measuring individual productivity,” in Organizational Linkages: Understanding the Productivity Paradox, Harris, D.H., (ed.), National Academies Press, Washington, D.C., pp. 105–130.
44. Smith, P. C., and Kendall, L. M. (1963). “Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales,” Journal of Applied Psychology, Vol. 47, No. 2, pp. 149–155.
45. Todd, P.A., McKeen, J.D., Gallupe, R.B. (1995). “The evolution of IS job skills: A content analysis of IS job advertisements from 1970 to 1990,” MIS Quarterly, Vol. 19, No. 1, pp. 1–27.
46. Touliatos, J., Bedeian, A. G., Mossholder, K. W., and Barkman, A. I. (1984). “Job related perceptions of male and female government, industrial, and public accountants,” Social Behavior and Personality, Vol. 12, No. 1, pp. 61–68.
47. Tziner, A., Joanis, C., and Murphy, K.R. (2000). “A comparison of three methods of performance appraisal with regard to goal properties, goal perception, and ratee satisfaction,” Group and Organization Management, Vol. 25, No. 2, pp. 175–190.
48. Van Dyne, L. (1993). In-Role and Extra-Role Behaviors: Cross Level and Longitudinal Effects of Individual Similarity to Other Group Members, Unpublished Ph.D. Dissertation, University of Minnesota, Minneapolis.
49. Vitalari, N.P. (1985). “Knowledge as a basis for expertise in systems analysis: An empirical study,” MIS Quarterly, Vol. 9, No. 3, pp. 221–241.
50. Wiersma, U.J., and Latham, G.P. (1986). “The practicality of behavioral observation scales, behavioral expectation scales, and trait scales,” Personnel Psychology, Vol. 39, No. 3, pp. 619 628.
51. Wiersma, U.J., Van Den Berg, P.T., and Latham, G.P. (1995). “Dutch reactions to behavioral observation, behavioral expectation, and trait scales,” Group and Organization Management, Vol. 20, No. 3, pp. 297–309.