Methods for evaluating hospital information systems: A literature review

EuroMed Journal of Business

Volumn 3, Issue 1, 2008, Pages 99-118

  Aggelidis, Vassilios P ^a   Chatzoglou, Prodromos D ^a  

^a DEMOCRITUS UNIVERSITY OF THRACE   (Greece)

Author keywords

Customer satisfaction; Economic performance; Hospitals; Information systems

Indexed keywords

EID: 84864648205     PISSN: 14502194     EISSN: 1758888X     Source Type: Journal    
DOI: 10.1108/14502190810873849     Document Type: Article

References (98)

1. Agarwal, R. and Prasad, J. (1997), "The role of innovation characteristics and perceived voluntariness in the acceptance of information technologies", Decision Sciences, Vol. 28 No. 3, pp. 557-82.
2. Allan, J. and Englebright, J. (2000), "Patient-centered documentation: an effective and efficient use of clinical information systems", Journal of Nursing Administration, Vol. 30 No. 1, pp. 90-5.
3. Ammenwerth, E., Graber, S., Herrmann, G., Burkle, T. and Konig, J. (2003), "Evaluation of health information systems-problems and challenges", International Journal of Medical Informatics, Vol. 71 Nos 2/3, pp. 125-35.
4. Ammenwerth, E., Eichstadter, R., Haux, T., Pohl, U., Rebel, S. and Ziegler, S. (2001), "A randomized evaluation of a computer-based nursing documentation system", Methods of Information in Medicine, Vol. 40 No. 1, pp. 61-8.
5. Anderson, J. (1997), "Clearing the way for physicians' use of clinical information systems", Communications of the ACM, Vol. 40 No. 8, pp. 83-90.
6. Arenson, R.L. and London, J.W. (1979), "Comprehensive analysis of a radiology operations management computer system", Radiology, Vol. 2 No. 1, pp. 355-62.
7. Au, N., Ngai, E.W.T. and Cheng, T.C.E. (2002), "A critical review of end-user information system satisfaction research and a new research framework", Omega, Vol. 30 No. 6, pp. 451-78.
8. Bailey, J. and Pearson, S. (1983), "Development of a tool for measurement and analyzing computer user satisfaction", Management Science, Vol. 29 No. 5, pp. 530-45.
9. Bailey, J.E. (1990), "Development of an instrument for the management of computer user attitudes in hospitals", Methods of Information in Medicine, Vol. 29 No. 1, pp. 51-6.
10. Bates, D.W., Cohen, M., Leape, L.L., Overhage, J.M., Shabot, M.M. and Sheridan, T. (2001), "Reducing the frequency of errors in medicine using information technology", Journal of the American Medical Informatics Association, Vol. 8 No. 4, pp. 299-308.
11. Burkle, T., Ammenwerth, E., Prokosch, H. and Dudeck, J. (2001), "Evaluation of clinical information systems. What can be evaluated and what cannot?", Journal of Evaluation in Clinical Practice, Vol. 7 No. 4, pp. 373-85.
12. Bórkle, T., Kuch, R., Prokosch, H. and Dudeck, J. (1999), "Stepwise evaluation of information systems in an university hospital", Methods of Information in Medicine, Vol. 38 No. 1, pp. 9-15.
13. Chan, Y. (2000), "IT value: the great divide between qualitative and quantitative and individual and organizational measures", Journal of Management Information Systems, Vol. 16 No. 4, pp. 225-61.
14. Chin, H.L. and McClure, P. (1995), "Evaluating a comprehensive outpatient clinical information system: a case study and model for system evaluation", in Gardner, R.M. (Ed.), Proceedings of the 19th Annual Symposium on Computer Applications in Medical Care, Hanley & Belfus, Inc., Philadelphia, PA.
15. Chin, W.W. and Lee, M.K.O. (2000), "On the formation of end-user computing satisfaction: a proposed model and measurement instrument", in Orlikowski, W., Ang, S., Weill, P., Krcmar, H.C. and DeGross, J.I. (Eds), Proceedings of the 21st International Conference on Information Systems, Brisbane, pp. 553-63.
16. Cyert, J. and March, J.G. (1963), A Behavioral Theory of the Firm, Prentice-Hall, Englewood Cliffs, NJ.
17. Davis, F. (1989), "Perceived usefulness, perceived ease of use, and user acceptance of information technology", MIS Quarterly, Vol. 13 No. 3, pp. 319-40.
18. Degoulet, P. and Fieschi, M. (1997), Introduction to Clinical Informatics, Springer Verlag, New York, NY.
19. DeLone, W.H. and McLean, E.R. (1992), "Information systems success: the quest for the dependent variable", Information Systems Research, Vol. 3 No. 1, pp. 60-95.
20. DeLone, W. and McLean, E. (2003), "The DeLone and McLean model of information systems success: a ten-year update", Journal of Management Information Systems, Vol. 19 No. 4, pp. 9-30.
21. Ditsa, G.E.M. and MacGregor, R.C. (1997), "Our mousetrap's fine: so why aren't people beating a path to our door?", Information Resources Management Journal, Vol. 10 No. 3, pp. 28-39.
22. Doherty, N. and Mcaulay, L. (2002), "The relationship between the ex antes and ex posts information systems evaluation: reflections from the literature and the practice of evaluating e-commerce investments", in Brown, A. and Remenyi, D. (Eds), Proceedings of the 9th European Conference on Information Technology Evaluation (ECITE 2002), University Paris-Dauphine, Paris, 15-16 July, MCIL, Reading, pp. 129-36.
23. Doll, W. and Torkzadeh, G. (1988), "The measurement of end-user computing satisfaction", MIS Quarterly, Vol. 12 No. 2, pp. 259-74.
24. Doll, W.J., Hendrickson, A. and Deng, X. (1998), "Using Davis's perceived usefulness and ease of use instruments in decision making: a multigroup invariance analysis", Decision Sciences, Vol. 29 No. 4, pp. 839-70.
25. Drummond, M.F., Stoddart, G.L. and Torrance, G.W. (1987), Methods for the Economic Evaluation of Health Care Programmes, Oxford University Press, Oxford.
26. Drummond, M.F., O'Brien, B.J., Stoddart, G.L. and Torrance, G.W. (1997), Methods for the Economic Evaluation of Health Care Programmes, 2nd ed., Oxford University Press, New York, NY.
27. Dupuits, F.M.H.M. and Hasman, A. (1995), "User satisfaction of general practitioners with HIOS þ, a medical decision support system", Computer Methods and Programs in Biomedicine, Vol. 47 No. 2, pp. 183-8.
28. Franklin, I., Pain, D., Green, E. and Owen, J. (1992), "Job design within a human centred (system) design framework", Behavior and Information Technology, Vol. 11 No. 3, pp. 141-50.
29. Friedman, C. and Wyatt, J. (2000), Evaluation Methods in Medical Informatics, 3rd ed., Springer-Verlag, Ann Arbor, MI.
30. Friedman, C. and Wyatt, J.C. (1997), "Evaluation methods in medical informatics", Computers in Nursing, Vol. 15 No. 4, pp. 167-73.
31. Galletta, D.F. and Lederer, A.L. (1989), "Some cautions on the measurement of user information satisfaction", Decision Sciences, Vol. 20 No. 3, pp. 419-39.
32. Gardner, R.M. and Lundsgaarde, H.P. (1994), "Evaluation of user acceptance of a clinical expert System", Journal of the American Medical Informatics Association, Vol. 6 No. 1, pp. 428-38.
33. Gardner, R.M., Hulse, R.K. and Larsen, K.G. (1990), "Assessing the effectiveness of a computerized pharmacy system", Proceedings of the 14th Annual Symposium on Computer Applications in Medical Care, McGraw-Hill, New York, NY, pp. 668-72.
34. Garrity, E.J. and Sanders, G.L. (1998), Information Systems Success Measurement, Idea Group Publishing, London.
35. Gatian, A. (1994), "Is user satisfaction a valid measure of system effectiveness?", Information and Management, Vol. 26 No. 3, pp. 119-31.
36. Ginzberg, M.J. (1981), "Early diagnosis of MIS implementation failure: promising results and unanswered questions", Management Science, Vol. 27, April, pp. 459-78.
37. Gold, M.R., Siegel, J.E., Russell, L.B. and Weinstein, M.C. (1996), Cost-Effectiveness in Health and Medicine, Oxford University Press, New York, NY.
38. Goodhue, D.L. and Thompson, R.L. (1995), "Task-technology fit and individual performance", MIS Quarterly, Vol. 19 No. 2, pp. 213-33.
39. Grant, A., Plante, I. and Leblanc, F. (2003), "The TEAM methodology for the evaluation of information systems in biomedicine", Computers in Biology and Medicine, Vol. 32 No. 3, pp. 195-207.
40. Grover, V., Jeong, S.R. and Segars, A.H. (1996), "Information systems effectiveness: the construct space and patterns of application", Information & Management, Vol. 4 No. 1, pp. 117-91.
41. Grundın, K. (2002), "An evaluation model for CSCW systems", in Brown, A. and Remenyi, D. (Eds), Proceedings of the Ninth European Conference on Information Technology Evaluation (ECITE 2002), University Paris-Dauphine, Paris, 15-16 July, MCIL, Reading, pp. 181-7.
42. Hammond, J., Johnson, H., Ward, C., Varas, R., Dembicki, R. and Marcial, E. (1991), "Clinical evaluation of a computer-based patient monitoring and data management system", Heart Lung, Vol. 20 No. 1, pp. 119-24.
43. Hirschheim, R. and Smithson, S. (1988), "A critical analysis of information systems evaluation", in Bjorn Andersen, N. and Davis, G.B. (Eds), Information Systems Assessment: Issues and Challenges, North Holland, Amsterdam (also published as "Evaluation of information systems: a critical assessment" in Willcocks, L.P. and Lester, S. (Eds), Beyond the IT Productivity Paradox, Wiley, New York, NY, 1999).
44. Hogue, J. and Watson, H. (1983), "Management's role in the approval of decision support systems", MIS Quarterly, Vol. 7 No. 2, pp. 15-26.
45. Igbaria, M. and Tan, M. (1997), "The consequences of the information technology acceptance on subsequent individual performance", Information & Management, Vol. 32 No. 3, pp. 113-21.
46. Ives, B. and Olson, M. (1984), "User involvement and MIS success: a review of research", Management Science, Vol. 30 No. 5, pp. 586-603.
47. Ives, B., Olson, M.H. and Baroudi, J.J. (1983), "The measurement of user information satisfaction", Communications of the ACM, Vol. 26 No. 10, pp. 785-93.
48. Kaplan, B. (1997), "Addressing organizational issues into the evaluation of medical systems", Journal of the American Medical Informatics Association, Vol. 4 No. 2, pp. 94-101.
49. Kaplan, B. and Lundsgaarde, H. (1996), "Toward an evaluation of an integrated clinical imaging system: identifying clinical benefits", Methods of Information in Medicine, Vol. 35 No. 1, pp. 221-9.
50. Kim, C., Kunsoo, S. and Lee, J. (1998), "Utilization and user satisfaction in end-user computing: a task contingent model", Information Resources Management Journal, Vol. 11 No. 4, pp. 11-24.
51. Lee, F., Teich, J.M., Spurr, C.D. and Bates, D.W. (1996), "Implementation of physician order entry: user satisfaction and self-reported usage patterns", Journal of the American Medical Informatics Association, Vol. 3 No. 1, pp. 42-55.
52. Lehoux, P., Sicotte, C. and Denis, J. (1999), "Assessment of a computerized medical record system: disclosing scripts of use", Evaluation and Program Planning, Vol. 22 No. 1, pp. 439-53.
53. Lucas, H.C. (1978), "The evolution of an information system: from key-man to every person", Sloan Management Review, Vol. 19 No. 2, pp. 39-53.
54. McHaney, R., Hightower, R. and Pearson, J. (2002), "A validation of the end-user computing satisfaction instrument in Taiwan", Information and Management, Vol. 39 No. 6, pp. 503-11.
55. Mahmood, M.A., Burn, J.M., Gemoets, L.A. and Jacquez, C. (2000), "Variables affecting information technology end-user satisfaction: a meta-analysis of the empirical literature", International Journal of Human-Computer Studies, Vol. 52 No. 4, pp. 751-71.
56. Maish, A.M. (1979), "A user's behaviour toward his MIS", MIS Quarterly, Vol. 3 No. 1, pp. 39-52.
57. Marasovic, C., Kenney, C., Elliott, D. and Sindhusake, D. (1997), "A comparison of nursing activities associated with manual and automated documentation in an Australian intensive care unit", Computer Nursing, Vol. 15 No. 1, pp. 205-11.
58. Markus, M.L. and Keil, M. (1994), "If we build it, they will come: designing information systems that people want to use", Sloan Management Review, Vol. 35 No. 4, pp. 11-25.
59. Matlin, G. (1980), "What is the value of investment in information systems?", MIS Quarterly, Vol. 3 No. 3, pp. 5-34.
60. Melone, N. (1990), "The theoretical assessment of user satisfaction construct in information system research", Management Science, Vol. 36 No. 1, pp. 76-91.
61. Miller, J. (1989), "Information systems effectiveness: the fit between business needs and system capabilities", Proceedings of the 10th International Conference on Information Systems, pp. 273-88.
62. Money, A., Tromp, D. and Wegner, T. (1988), "The quantification of decision support benefits within the context of value analysis", MIS Quarterly, Vol. 12 No. 2, pp. 223-36.
63. Murphy, K.E. and Simon, S.J. (2002), "Intangible benefits valuation in ERP projects", Information Systems Journal, Vol. 12 No. 4, pp. 301-20.
64. Nevile, D.B., Keough, M., Barron, M., MacDonald, D., Gates, G., Tucker, S., Cotton, S., Farrell, G. and Hoekman, T. (2004), "Towards an evaluation framework for electronic health records initiatives: an inventory of electronic health records initiatives across Canada", available at: www.nlchi.nl.ca/pdf/Initiatives_mar04.pdf
65. Nykänen, P. (2000), "Decision support systems from a health informatics perspective", academic dissertation, Department of Computer and Information Sciences, University of Tampere, Tampere.
66. Ohmann, C., Boy, O. and Yang, Q. (1997a), "A systematic approach to the assessment of user satisfaction with health care systems: constructs, models and instruments", Studies in Health Technology and Informatics, Vol. 43, Part B, pp. 781-5.
67. Ohmann, C., Boy, O., Yang, Q. and Eich, H.P. (1997b), "Evaluierung der Benutzerzufriedenheit mit einem Krankenhausinformationssystem: theoretische Aspekte und klinische Anwendung", Medizinische Informatik, Biometrie und Epidemiologie, Vol. 82 No. 1, pp. 31-4.
68. Ostbye, T., Moen, A., Erikssen, G. and Hurlen, P. (1997), "Introducing a module for laboratory test order entry and reporting of results at a hospital ward: an evaluation study using a multi-method approach", Journal of Medical Systems, Vol. 21 No. 2, pp. 107-17.
69. Øvretveit, J. (1998), Evaluating Health Interventions, Open University Press, Buckingham.
70. Palvia, S., Sharma, R. and Conrath, A. (2001), "A sociotechnical framework for quality assessment of computer information systems", Industrial Management and Data Systems, Vol. 101 No. 5, pp. 237-51.
71. Pugh, G.E. and Tan, J.K.H. (1994), "Computerized databases for emergency care: what impact on patient care", Methods of Information in Medicine, Vol. 33 No. 5, pp. 503-17.
72. Ribiere, V., LaSalle, A.J., Khorramshahgol, R. and Gousty, Y. (1999), "Hospital information systems quality: a customer satisfaction assessment tool", Proceedings of the 32nd Hawaii International Conference on Systems Sciences, Maui, HI, January 5-8.
73. Rigby, D. (2001), "Integrated continence services", Nursing Standard, Vol. 16 No. 8, pp. 1-18.
74. Robey, D. (1979), "User attitudes and management information system use", Academy of Management Journal, Vol. 22 No. 3, pp. 527-38.
75. Saarinen, T. (1993), "Success of information systems-evaluation of development projects and the choice of procurement and implementation strategies", Acta Academiae Oeconomicae Helsingiensis, Series A:88, Helsinki School of Economics and Business Administration, Helsinki.
76. Saarinen, T. (1996), "An expanded instrument for evaluating information system success", Information and Management, Vol. 31 No. 2, pp. 103-18.
77. Sailly, J.C., Lebrun, T., Vincent, P., Eeckhoudt, L., Izard, D., Leclerc, H., Arbon, G. and Hassoun, A. (1985), "The medical and economic consequences of automation in bacteriology: a case study in a French university hospital", Social Science & Medicine, Vol. 10 No. 1, pp. 1163-6.
78. Salmela, H. and Turunen, P. (1997), "Evaluation of information systems in health care: framework and its application", in Berghout, E.W. and Remenyi, D.S.J. (Eds), Proceedings of the Fourth European Conference on the Evaluation of Information Technology, The Delft University Press, Delft, pp. 111-21.
79. Seddon, P.B. (1997), "A respecification and extension of the DeLone and McLean model of IS success", Information Systems Research, Vol. 8 No. 3, pp. 240-54.
80. Sethi, V. and King, R.C. (1998), "An application of the CUSP catastrophe model to user information satisfaction", Information and Management, Vol. 34 No. 1, pp. 41-53.
81. Sicotte, C., Denis, J., Lehoux, P. and Champagne, F. (1998), "The computer based patient record challenges towards timeless and space less medical practice", Journal of Medical Systems, Vol. 22 No. 1, pp. 237-56.
82. Straub, D.W., Limayem, M. and Karahanna, E. (1995), "Measuring system usage: implications for IS theory testing", Management Science, Vol. 41 No. 8, pp. 1328-42.
83. Szajna, B. (1996), "Empirical evaluation of the revised technology acceptance model", Management Science, Vol. 42 No. 1, pp. 85-92.
84. Szczepura, A.K. and Stilwell, J.A. (1988), "Information for decision makers at hospital laboratory level: an example of a graphical method of representing costs and effects for a replacement automated technology in a haematology laboratory", Social Science & Medicine, Vol. 7 No. 1, pp. 715-25.
85. Talmon, J. and Hasmann, A. (2002), "Medical informatics as a discipline at the beginning of the 21st century", Methods of Information in Medicine, Vol. 41 No. 1, pp. 4-7.
86. Trice, A.W. and Treacy, M.E. (1988), "Utilization as a dependent variable in MIS research", Data Base, Vol. 19 Nos 3/4, pp. 33-41.
87. Urschitz, M., Lorenz, S., Unterasinger, L., Metnitz, P., Preyer, K. and Popow, C. (1998), "Three years' experience with a patient data management system at a neonatal intensive care unit", Journal of Clinical Monitoring and Computing, Vol. 14 No. 1, pp. 119-25.
88. Vagelatos, A., Sofotassios, D., Papanikolaou, C. and Manolopoulos, C. (2002), "ICT penetration in public Greek hospitals", paper presented at the Medical Informatics in Europe Conference (MIE2002), Budapest, August.
89. van der Loo, R.P., van Gennip, E.M.S.J. and Baker, A.R. (1995), "Evaluation of automated information systems in health care: an approach to classifying evaluative studies", Computer Methods and Programs in Biomedicine, Vol. 48 No. 1, pp. 45-52.
90. van der Meijden, M.J., Tange, H.J. and Hasman, T.A. (2003), "Determinants of sucess of inpatient clinical information systems: a literature review", Journal of the American Medical Informatics Association, Vol. 20 No. 3, pp. 235-43.
91. Venkatesh, V., Speier, C. and Morris, M.G. (2002), "User acceptance enablers in individual decision making about technology: toward an integrated model", Decision Sciences, Vol. 33 No. 2, pp. 297-316.
92. Walton, R., Dovey, S., Harvey, E. and Freemantle, N. (1999), "Computer support for determining drug dose: systematic review and meta-analysis", British Medical Journal, Vol. 318 No. 7189, pp. 984-90.
93. Weiner, M., Gress, T., Thiemann, D.R. and Jenckes, M. (1999), "Contrasting views of physicians and nurses about an inpatient computer-based provider order-entry system", Journal of the American Medical Informatics Association, Vol. 6 No. 1, pp. 234-44.
94. Winter, A.F., Ammenwerth, E., Bott, O.J., Brigl, B., Buchauer, A., Gräber, S., Grant, A., Haber, A., Hasselbring, W., Haux, R., Heinrich, A., Janssen, H., Kock, I., Penger, O.-S., Prokosch, H.-U., Terstappen, A. and Winter, A. (2001), "Strategic information management plans: the basis for systematic information management in hospitals", International Journal of Medical Informatics, Vol. 64 No. 2, pp. 99-109.
95. Wyatt, J. and Spiegelhalter, D. (1992), "Field trials of medical decision-aids: potential problems and solutions", in Clayton, P. (Ed.), 15th Annual Symposium on Computer Applications in Medical Care, McGraw-Hill, New York, NY, pp. 3-7.
96. Wyatt, J.C. and Wyatt, S.M. (2003), "When and how to evaluate health information systems?", International Journal of Medical Informatics, Vol. 69 Nos 2/3, pp. 251-9.
97. Bokhari, R.H. (2005), "The relationship between system usage and user satisfaction: a meta-analysis", Journal of Enterprise Information Management, Vol. 18 No. 2, pp. 211-34.
98. Grover, V., Jing, S.R. and Sugars, A.H. (1996), "Information systems effectiveness: the construct space and patterns of application", Information & Management, Vol. 31 No. 4, pp. 117-91.