Measuring the performance of information systems: A functional scorecard

Journal of Management Information Systems

Volumn 22, Issue 1, 2005, Pages 85-115

  Chang, Jerry Cha Jan ^a   King, William R ^b  

^a UNIVERSITY OF NEVADA   (United States)

^b UNIVERSITY OF PITTSBURGH   (United States)

Author keywords

Functional scorecard; Information systems performance measurement; Instrument development; Structural equation modeling

Indexed keywords

INFORMATION TECHNOLOGY; MATHEMATICAL MODELS; PERFORMANCE; SYSTEMS ANALYSIS;

FUNCTIONAL SCORECARD; INFORMATION SYSTEMS PERFORMANCE MEASUREMENT; INSTRUMENT DEVELOPMENT; STRUCTURAL EQUATION MODELING;

MANAGEMENT INFORMATION SYSTEMS;

EID: 23744432786     PISSN: 07421222     EISSN: None     Source Type: Journal    
DOI: 10.1080/07421222.2003.11045833     Document Type: Review

References (113)

1. Alavi, M.; Marakas, G.M.; and Yoo, Y. A comparative study of distributed learning environments on learning outcomes. Information Systems Research, 13, 4 (December 2002), 404-415.
2. Anderson, J.C. An approach for confirmatory measurement and structural equation modeling of organizational properties. Management Science, 33, 4 (April 1987), 525-541.
3. Anderson, J.C., and Gerging, D.W. Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 3 (May 1988), 411-423.
4. Armstrong, J.S., and Overton, T.S. Estimating nonresponse bias in mail surveys. Journal of Marketing Research, 14, 3 (August 1977), 396-402.
5. Bagozzi, R.P. An examination of the validity of two models of attitude. Multivariate Behavioral Research, 16, 3 (July 1981), 323-359.
6. Bailey, J.E., and Pearson, S.W. Development of a tool for measuring and analyzing computer user satisfaction. Management Science, 29, 5 (May 1983), 530-545.
7. Ballantine, J.; Bonner, M.; Levy, M.; Martin, A.; Monro, I.; and Powell, P.L. Developing a 3-D model of information systems success. In E.J. Garrity and G.L. Sanders (eds.), Information Systems Success Measurement. Hershey, PA: Idea Group, 1998, pp. 46-59.
8. Baroudi, J.J., and Orlikowski, W.J. A short-form measure of user information satisfaction: A psychometric evaluation and notes on use. Journal of Management Information Systems, 4, 4 (Spring 1988), 44-59.
9. Basu, A., and Kumar, A. Workflow management issues in e-business, Information Systems Research, 13, 1 (March 2002), 1-14.
10. Benaroch, M. Managing information technology investment risk: A real options perspective. Journal of Management Information Systems, 19 2 (Fall 2002), 43-84.
11. Bergeron, F.; Rivard, S.; and De Serre, L. Investigating the support role of the information center. MIS Quarterly, 14, 3 (September 1990), 247-260.
12. Brancheau, J.C., and Wetherbe, J.C. Key issues in information systems management. MIS Quarterly, 11, 1 (March 1987), 23-45.
13. Brancheau, J.C.; Janz, B.D.; and Wetherbe, J.C. Key issues in information systems management: 1994-95 SIM Delphi results. MIS Quarterly, 20, 2 (June 1996), 225-242.
14. Broadbent, M., and Weill, P. Management by Maxim: How business and IT managers can create IT infrastructers. Sloan Management Review, 38 3 (Spring 1997), 77-92.
15. Broadbent, M.; Weill, P.; O'Brien, T.; and Neo, B.N. Firm context and patterns of IT infrastructure capability. In J.I. DeGross, S.L. Jarvenpaa, and A. Srinivasan (eds.), Proceedings of Seventeenth International Conference on Information Systems. Atlanta: Association for Information Systems, 1996, pp. 174-194.
16. Brynjolfsson, E. The productivity paradox of information technology. Communications of the ACM, 36, 12 (December 1993), 67-77.
17. Byrd, T.A., and Turner, D.E. Measuring the flexibility of information technology infrastructure: Exploratory analysis of a construct. Journal of Management Information Systems, 17, 1 (Summer 2000), 167-208.
18. Cameron, K.S. A study of organizational effectiveness and its predictors. Management Science, 32, 1 (January 1986), 87-112.
19. Cameron, K.S., and Whetton, D.A. Some conclusions about organizational effectiveness. In K.S. Cameron and D.A. Whetton (eds.), Organizational Effectiveness: A Comparison of Multiple Models. New York: Academic Press, 1983, pp. 261-277.
20. Carr, C.L. Managing service quality at the IS help desk: Toward the development and testing of TECH-QUAL, a model of IS technical support service quality. Ph.D. dissertation, University of Minnesota, Minneapolis, 1999.
21. Chan, Y.E.; Huff, S.L.; Barclay, D.W.; and Copeland, D.G. Business strategic orientation, information systems strategic orientation, and strategic alignment, Information Systems Research, 8, 2 (June 1997), 125-150.
22. Chatterjee, D.; Grewal, R.; and Sambamurthy, V. Shaping up for e-commerce: Institutional enablers of the organizational assimilation of Web technologies. MIS Quarterly, 26, 2 (June 2002), 65-90.
23. Chatterjee, D.; Pacini, C.; and Sambamurthy, V. The shareholder-wealth and trading-volume effects of information technology infrastructure investments. Journal of Management Information Systems, 19, 2 (Fall 2002), 7-42.
24. Chircu, A.M., and Kauffman, R.J. Limits to value in electronic commerce-related IT investments. Journal of Management Information Systems, 17, 2 (Fall 2000), 59-80.
25. Churchill, G.A. A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16, 1 (February 1979), 64-73.
26. Cronbach, L.J., and Meehl, P.E. Construct validity in psychological tests. In D.M. Jackson and S. Messick (eds.), Problems in Human Assessment. New York: McGraw-Hill, 1967, pp. 57-77.
27. Cronin, J.J.J., and Taylor, S.A. SERVPERF versus SERVQUAL: Reconciling performance-based and perceptions-minus-expectations measurement of service quality. Journal of Marketing, 58, 1 (January 1994), 125-131.
28. Davenport, T.H. Process Innovation: Reengineering Work Through Information Technology. Boston: Harvard Business School Press, 1993.
29. Davis, F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 3 (September 1989), 319-340.
30. DeLone, W.H., and McLean, E.R. Information systems success: The quest for the dependent variable. Information Systems Research, 3, 1 (March 1992), 60-95.
31. DeLone, W.H., and McLean, E.R. The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19, 4 (Spring 2003), 9-30.
32. Dess, G.G., and Robinson, R.B.J. Measuring organizational performance in the absence of objective measures: The case of the privately-held firm and conglomerate business unit. Strategic Management Journal, 5, 3 (July-September 1984), 265-273.
33. Devaraj, S.; Fan, M.; and Kohli, R. Antecedents of B2C channel satisfaction and preference: Validating e-commerce metrics, Information Systems Research, 13, 3 (September 2002), 316-333.
34. Dickson, G.W.; Leitheiser, R.L.; Nechis, M.; and Wetherbe, J.C. Key information systems issues for the 1980s, MIS Quarterly, 8, 3 (September 1984), 135-148.
35. Doll, W.J., and Torkzadeh, G. The measurement of end-user computing satisfaction. MIS Quarterly, 12, 2 (June 1988), 259-274.
36. Duncan, N.B. Capturing flexibility of information technology infrastructure: A study of resource characteristics and their measure. Journal of Management Information Systems, 12, 2 (Fall 1995), 37-57.
37. Fan, M.; Stallaert, J.; and Whinston, A.B. Decentralized mechanism design for supply chain organizations using an auction market. Information Systems Research, 14, 1 (March 2003), 1-22.
38. Fitzgerald, L.; Johnston, R.; Brignall, S.; Silvestro, R.; and Voss, C. Performance Measurement in Service Businesses. London: Chartered Institute of Management Accountants, 1993.
39. Gefen, D., and Ridings, C.M. Implementation team responsiveness and user evaluation of customer relationship management: A quasi-experimental design study of social exchange theory. Journal of Management Information Systems, 19, 1 (Summer 2002), 47-70.
40. Gerbing, D.W., and Anderson, J.C. An updated paradigm for scale development incorporating unidimensionality and its assessment. Journal of Marketing Research, 25, 2 (May 1988), 186-192.
41. Glazer, R. Measuring the knower: Towards a theory of knowledge equity. California Management Review, 40, 3 (Spring 1998), 175-194.
42. Gold, A.H.; Malhotra, A.; and Segars, A.H. Knowledge management: An organizational capabilities perspective. Journal of Management Information Systems, 18, 1 (Summer 2001), 185-214.
43. Goodhue, D.L., and Thompson, R.L. Task-technology fit and individual performance. MIS Quarterly, 19, 2 (June 1995), 213-236.
44. Govindarajulu, C., and Reithel, B.J. Beyond the information center: An instrument to measure end-user computing support from multiple sources. Information & Management, 33, 5 (May 1998), 241-250.
45. Grover, V., and Davenport, T.H. General perspectives on knowledge management: Fostering a research agenda. Journal of Management Information Systems, 18, 1 (Summer 2001), 5-22.
46. Grover, V.; Jeong, S.R.; and Segars, A.H. Information systems effectiveness: The construct space and patterns of application. Information & Management, 31, 4 (December 1996), 177-191.
47. Guimaraes, T., and Igbaria, M. Exploring the relationship between IC success and company performance, Information & Management, 26, 3 (March 1994), 133-141.
48. Hair, J.F.J.; Anderson, R.E.; Tatham, R.L.; and Black, W.C. Multivariate Data Analysis, 5th ed. Upper Saddle River, NJ: Prentice Hall, 1998.
49. Hartog, C., and Herbert, M. 1985 opinion survey of MIS managers: Key issues. MIS Quarterly, 10, 4 (December 1986), 351-361.
50. Hattie, J. Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, 2 (June 1985), 139-164.
51. Hitt, L.M.; Wu, D.J.; and Zhou, X. Investment in enterprise resource planning: Business impact and productivity measures. Journal of Management Information Systems, 19, 1 (Summer 2002), 71-98.
52. Jiang, J.J.; Klein, G.; and Carr, C.L. Measuring information system service quality: SERVQUAL from the other side. MIS Quarterly, 26, 2 (June 2002), 145-166.
53. Jöreskog, K.G. Testing structural equation models. In K.A. Boland and L.S. Long (eds.), Testing Structural Equation Models. Newbury Park, CA: Sage, 1993, pp. 294-316.
54. Kerlinger, F.N. Foundations of Behavioral Research. New York: McGraw-Hill, 1978.
55. Kim, J.; Lee, J.; Han, K.; and Lee, M. Businesses as buildings: Metrics for the architectural quality of Internet businesses. Information Systems Research, 13, 3 (September 2002), 239-254.
56. King, W.R. Management information systems. In H. Bidgoli (ed.), Encyclopedia of Management Information Systems, vol. 3. New York: Academic Press, 2003.
57. King, W.R., and He, J. External validity, coverage and nonresponse errors in IS survey research. Katz Graduate School of Business, University of Pittsburgh, 2004.
58. Kohli, R., and Devaraj, S. Measuring information technology payoff: A meta-analysis of structural variables in firm-level empirical research. Information Systems Research, 14, 2 (June 2003), 127-145.
59. Kraemer, K.L.; Danziger, J.N.; Dunkle, D.E.; and King, J.L. The usefulness of computer-based information to public managers. MIS Quarterly, 17, 2 (June 1993), 129-148.
60. Kraut, R.; Dumais, S.; and Susan, K. Computerization, productivity, and quality of worklife. Communications of the ACM, 32, 2 (February 1989), 220-238.
61. Lacity, M., and Willcocks, L. Global Information Technology Outsourcing. Chichester, UK: Wiley, 2001.
62. Lambert, D.M., and Harrington, T.C. Measuring nonresponse bias in customer service mail surveys. Journal of Business Logistics, 11, 2 (1990), 5-25.
63. Larsen, K.R.T. A taxonomy of antecedents of information systems success: Variable analysis studies. Journal of Management Information Systems, 20, 2 (Fall 2003), 169-246.
64. Lee, H. Knowledge management enablers, processes, and organizational performance: An integrative view and empirical examination. Journal of Management Information Systems, 20, 1 (Summer 2003), 179-228.
65. Lee, H.; Kwak, W.; and Han, I. Developing a business performance evaluation system: An analytical hierarchical model. Engineering Economist, 40, 4 (Summer 1995), 343-357.
66. Magal, S.R.; Carr, H.H.; and Watson, H.J. Critical success factors for information center managers. MIS Quarterly, 12, 3 (September 1988), 314-425.
67. Mirani, R., and King, W.R. The development of a measure for end-user computing support. Decision Sciences, 25, 4 (July-August 1994), 481-498.
68. Moore, G.C., and Benbasat, I. Development of an instrument to measure the perceptions of adopting an information technology innovation. Information Systems Research, 2, 3 (September 1991), 192-222.
69. Myers, B.L.; Kappelman, L.A.; and Prybutok, V.R. A comprehensive model for assessing the quality and productivity of the information systems function: Toward a theory for information systems assessment. In E.J. Garrity and G.L. Sanders (eds.), Information Systems Success Measurement. Hershey, PA: Idea Group, 1998, pp. 94-121.
70. Nelson, K.M., and Cooprider, J.G. The contribution of shared knowledge to IS group performance. MIS Quarterly, 20, 4 (December 1996), 409-432.
71. Nelson, R.R., and Cheney, P.H. Training end users: An exploratory study. MIS Quarterly, 11, 4 (December 1987), 547-559.
72. Niederman, F.; Brancheau, J.C.; and Wetherbe, J.C. Information systems management issues for the 1990s. MIS Quarterly, 15, 4 (December 1991), 475-500.
73. Olian, J.D., and Rynes, S.L. Making total quality work: Aligning organizational processes, performance measures, and stakeholders. Human Resource Management, 30, 3 (Fall 1991), 303-333.
74. Parasuraman, A.; Zeithaml, V.A.; and Berry, L.L. Refinement and reassessment of the SERVQUAL scale. Journal of Retailing, 64, 4 (Winter 1991), 420-450.
75. Pitt, L.F.; Watson, R.T.; and Kavan, C.B. Service quality: A measure of information systems effectiveness. MIS Quarterly, 19, 2 (June 1995), 173-185.
76. Porter, M.E., and Millar, V.E. How information gives you competitive advantage. Harvard Business Review, 63, 4 (July-August 1985), 149-160.
77. Premkumar, G. Evaluation of Strategic Information Systems Planning: Empirical Validation of a Conceptual Model. Pittsburgh: University of Pittsburgh, 1989.
78. Raghunathan, B.; Raghunathan, T.S.; and Tu, Q. Dimensionality of the strategic grid framework: The construct and its measurement. Information Systems Research, 10, 4 (December 1999), 343-355.
79. Rai, A.; Lang, S.S.; and Welker, R.B. Assessing the validity of IS success models: An empirical test and theoretical analysis, Information Systems Research, 13, 1 (March 2002), 50-69.
80. Ryan, S.D., and Harrison, D.A. Considering social subsystem costs and benefit in information technology investment decisions: A view from the field on anticipated payoffs. Journal of Management Information Systems, 16, 4 (Spring 2000), 11-40.
81. Ryker, R., and Nath, R. An empirical examination of the impact of computer information systems on users. Information & Management, 29, 4 (1995), 207-214.
82. Saarinen, T. An expanded instrument for evaluating information system success. Information & Management, 31, 2 (1996), 103-118.
83. Santhanam, R., and Hartono, E. Issues in linking information technology capability to firm performance. MIS Quarterly, 27, 1 (March 2003), 125-154.
84. Saunders, C.S., and Jones, J.W. Measuring performance of the information systems function. Journal of Management Information Systems, 8, 4 (Spring 1992), 63-82.
85. Schwab, D.P. Construct validation in organizational behavior. In B.M. Staw and L.L. Cummings (eds.), Research in Organizational Behavior vol. 2. Greenwich, CT: JAI Press, 1980, pp. 3-43.
86. Scott, J.E. Facilitating interorganizational learning with information technology. Journal of Management Information Systems, 17, 2 (Fall 2000), 81-114.
87. Seddon, P.B. A respecification and extension of the DeLone and McLean model of IS success. Information Systems Research, 8, 3 (September 1997), 240-253.
88. Seddon, P.B.; Staples, S.; Patnayauni, R.; and Bowtell, M. Dimensions of information systems success. Communications of the AIS, 2 (November 1999), 2-39.
89. Segars, A.H. Assessing the unidimensionality of measurement: A paradigm and illustration within the context of information systems research. Omega, 25, 1 (February 1997), 107-121.
90. Segars, A.H., and Grover, V. Re-examining perceived ease of use and usefulness: A confirmatory factor analysis. MIS Quarterly, 17, 4 (December 1993), 517-525.
91. Segars, A.H., and Grover, V. Strategic information systems planning success: An investigation of the construct and its measurement. MIS Quarterly, 22, 2 (June 1998), 139-163.
92. Segars, A.H., and Hendrickson, A.R. Value, knowledge, and the human equation: Evolution of the information technology function in modern organizations. Journal of Labor Research, 21, 3 (Summer 2000), 431-445.
93. Sethi, V., and King, W.R. Development of measures to assess the extent to which an information technology application provides competitive advantage. Management Science, 40, 12 (December 1994), 1601-1627.
94. Sethi, V., and King, W.R. Organizational Transformation Through Business Process Re-Engineering. Upper Saddle River, NJ: Prentice Hall, 1998.
95. Steers, R.M. Problems in the measurement of organizational effectiveness. Administrative Science Quarterly, 20, 4 (December 1975), 546-558.
96. Straub, D.W.; Hoffman, D.L.; Weber, B.W.; and Steinfield, C. Measuring e-commerce in Net-enabled organizations: An introduction to the special issue. Information Systems Research, 13, 2 (June 2002), 115-124.
97. Sussman, S.W., and Siegal, W.S. Informational influence in organizations: An integrated approach to knowledge adoption. Information Systems Research, 14, 1 (March 2003), 247-65.
98. Tallon, P.P.; Kraemer, K.L.; and Gurbaxani, V. Executives' perceptions of the business value of information technology: A process-oriented approach. Journal of Management Information Systems, 16, 4 (Spring 2000), 145-173.
99. Tam, K.Y. The impact of information technology investment on firm performance and evaluation: Evidence from newly industrialized economies. Information Systems Research, 9, 1 (March 1998), 85-98.
100. Templeton, G.F.; Lewis, B.R.; and Snyder, C.A. Development of a measure for the organizational learning construct. Journal of Management Information Systems, 19, 2 (Fall 2002), 175-218.
101. Teo, T.S.H. Integration between business planning and information systems planning: Evolutionary-contingency perspectives. Ph.D. dissertation, University of Pittsburgh, 1994.
102. Torkzadeh, G., and Dhillon, G. Measuring factors that influence the success of Internet commerce. Information Systems Research, 13, 2 (June 2002), 187-204.
103. Torkzadeh, G., and Doll, W.J. The development of a tool for measuring the perceived impact of information technology on work. Omega, International Journal of Management Science, 27, 3 (June 1999), 327-339.
104. Van Dyke, T.P.; Kappelman, L.A.; and Prybutok, V.R. Measuring information systems service quality: Concerns on the use of the SERVQUAL questionnaire. MIS Quarterly, 21, 2 (June 1997), 195-208.
105. Venkatesh, V.; Morris, M.G.; Davis, G.B.; and Davis, F.D. User acceptance of information technology: Toward a unified view. MIS Quarterly, 27, 3 (September 2003), 425-478.
106. Venkatraman, N. Strategic orientation of business enterprises: The construct, dimensionality, and measurement. Management Science, 35 8 (August 1989), 942-962.
107. Venkatraman, N., and Ramanujam, V. Measurement of business economic performance: An examination of method convergence. Journal of Management, 13, 1 (1987), 109-122.
108. Wand, Y., and Wang, R.Y. Anchoring data quality dimensions in ontological foundations. Communications of the ACM, 39, 11 (November 1996), 86-95.
109. Wang, R.Y., and Strong, D.M. Beyond accuracy: What data quality means to data consumers. Journal of Management Information Systems, 12, 4 (Spring 1996), 5-34.
110. Watson, R.T.; Pitt, L.F.; and Kavan, C.B. Measuring information systems service quality: Lessons from two longitudinal case studies. MIS Quarterly, 22, 1 (March 1998), 61-79.
111. Weill, P. The relationship between investment in information technology and firm performance: A study of the valve manufacturing sector. Information Systems Research, 3, 4 (December 1992), 307-333.
112. Wells, C.E. Evaluation of the MIS function in an organization: An exploration of the measurement problem. Ph.D. dissertation, University of Minnesota, Minneapolis, 1987.
113. Xia, W. Dynamic capabilities and organizational impact of IT infrastructure: A research framework and empirical investigation. Ph.D. dissertation, University of Pittsburgh, 1998.