Computerised decision support systems in order communication for diagnostic, screening or monitoring test ordering: Systematic reviews of the effects and cost-effectiveness of systems

Health Technology Assessment

Volumn 14, Issue 48, 2010, Pages 1-227

  Main, C ^a   Moxham, T ^a   Wyatt, J C ^b   Kay, J ^c   Anderson, R ^a   Stein, K ^a  

^a Peninsula Technol Assessment Group   (United Kingdom)

^b UNIVERSITY OF DUNDEE   (United Kingdom)

^c OXFORD UNIVERSITY HOSPITALS NHS FOUNDATION TRUST   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

BLOOD EXAMINATION; CLINICAL DECISION MAKING; COMMUNICATION SOFTWARE; COST CONTROL; COST EFFECTIVENESS ANALYSIS; DECISION SUPPORT SYSTEM; DIAGNOSTIC TEST; HEALTH CARE UTILIZATION; LABORATORY TEST; NETHERLANDS; PATIENT MONITORING; PATIENT SAFETY; PATIENT SATISFACTION; PROGNOSIS; REVIEW; SCREENING TEST; SPAIN; SYSTEMATIC REVIEW; UNITED KINGDOM;

ALGORITHMS; BAYES THEOREM; COST-BENEFIT ANALYSIS; DECISION SUPPORT SYSTEMS, CLINICAL; DIAGNOSTIC TESTS, ROUTINE; EFFICIENCY; EFFICIENCY, ORGANIZATIONAL; GREAT BRITAIN; HUMANS; MASS SCREENING; STATE MEDICINE;

EID: 78149236070     PISSN: 13665278     EISSN: None     Source Type: Journal    
DOI: 10.3310/hta14480     Document Type: Review

References (219)

1. Whiting P, Toerien M, de Salis I, Sterne JAC, Dieppe P, Egger M, et al. A review identifies and classifies reasons for ordering diagnostic tests. J Clinl Epidemiol 2007;60:981-9.
2. Berger R, Kichak B. Computerized physician order entry: helpful or harmful? J Am Med Inform Assoc 2004;11:100-3.
3. Hwang JI, Park HA, Bakken S. Impact of a physician's order entry (POE) system on physicians' ordering patterns and patient length of stay. Int J Med Inform 2002;65:213-23.
4. Sittig D, Stead W. Computer-based physician order entry: state of the art. J Am Med Inform Assoc 1994;1:108-23.
5. Tierney WM. Improving clinical decisions and outcomes with information: a review. Int J Med Inform 2001;62:1-9.
6. Kuperman GJ, Gibson RF. Computer physician order entry: benefits, costs and issues. Ann Intern Med 2003;139:3-9.
7. Doolan DF, Bates DW, James BC. The use of computers for clinical care: a case series of advanced U.S. sites. J Am Med Inform Assoc 2003;10:94-107.
8. Tierney WM, Overhage JM, Takesue L, Harris M, Murray D, Vargo C. Computerizing guidelines to improve care and patients outcomes: example of heart failure. J Am Med Inform Assoc 1995;2:316-22.
9. Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 1998;280:1339-46.
10. Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review. JAMA 2005;293:1223-38.
11. Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome. A critical appraisal of research. Ann Intern Med 1994;120:135-42.
12. Danzon PM, Manning WG, Marquis MS. Factors affecting laboratory test use and prices. Health Care Financ Rev 1984;5:23-32.
13. Fawkes FG, Catford JC, Logan RF. Containing the use of laboratory tests. BMJ 1985;290:488-90.
14. Shawstack JA, Schroeder SA, Matsumoto MF. Changes in the use of medical technologies, 1972-1977: a study of 10 inpatient diagnoses. N Engl J Med 1982;306:706-12.
15. Commission for Healthcare Audit and Inspection. Getting results: Pathology services in acute and specialist trusts. 2007. London.
16. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005;330:765.
17. Wendt T, Knaup-Gregori P, Winter A. Decision support in medicine: a survey of problems of user acceptance. Stud Health Technol Inform 2000;77:852-6.
18. Wetter T. Lessons learnt from bringing knowledgebased decision support into routine use. Artif Intell Med 2002;24:195-203.
19. Aronsky D, Chan KJ, Haug PJ. Evaluation of a computerized diagnostic decision support system for patients with pneumonia: study design considerations. J Am Med Inform Assoc 2001;8:473-85.
20. Ramnarayan P, Britto J. Paediatric clinical decision support systems. Arch Dis Child 2002;87:361-2.
21. Boekeloo BO, Becker DM, Levine DM, Belitsos PC, Pearson TA. Strategies for increasing house staff management of cholesterol with inpatients. Am J Prev Med 1990;6(Suppl. 2):5-9.
22. Ryff-de-Leche A, Engler H, Nutzi E, Berger M, Berger W. Clinical applications of two computerized diabetes managment systems: comparison with the log-book method. Diabetes Res 1992;19:97-105.
23. Rossi RA, Every NA. A computerized intervention to decrease the use of calcium channel blockers in hypertension. J Gen Intern Med 1997;12:672-8.
24. Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of 'corollary orders' to prevent errors of omission. J Am Med Inform Assoc 1997;4:364-75.
25. Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW. Computer-based versus manual health maintenance tracking. A controlled trial. Arch Fam Med 1994;3:581-8.
26. Barnett GO, Winickoff RN, Morgan MM, Zielstorff RD. A computer-based monitoring system for follow-up of elevated blood pressure. Med Care 1983;21:400-9.
27. McPhee SJ, Bird JA, Fordham D, Rodnick JE, Osborn EH. Promoting cancer prevention activities by primary care physicians. Results of a randomized, controlled trial. JAMA 1991;266:538-44.
28. Tierney WM, McDonald CJ, Hui SL, Martin DK. Computer predictions of abnormal test results. Effects on outpatient testing. JAMA 1988;259:1194-8.
29. McDonald CJ, Wilson GA, McCabe GP, Jr. Physician response to computer reminders. JAMA 1980;244:1579-81.
30. Dexter PR, Wolinsky FD, Gramelspacher GP. Effectiveness of computer-generated reminders for increasing discussions about advance directives and completion of advance directive forms. A randomized, controlled trial. Ann Intern Med 1998;128:102-10.
31. Chambers CV, Balaban DJ, Carlson BL, Grasberger DM. The effect of microcomputergenerated reminders on influenza vaccination rates in a university-based family practice center. J Am Board Fam Pract 1991;4:19-26.
32. Hobbs FD, Delaney BC, Carson A, Kenkre JE. A prospective controlled trial of computerized decision support for lipid management in primary care. Fam Pract 1996;13:133-7.
33. Payne TH. Computer decision support systems. Chest 2000;118(Suppl. 2):47-52.
34. Shiffman RN, Brand CA, Liaw Y, Corb GJ. A design model for computer-based guideline implementation based on information management services. J Am Med Inform Assoc 1999;6:99-103.
35. Trivedi MH, Kern JK, Marcee A, Grannemann B, Kleiber B, Bettinger T. Development and implementation of computerized clinical guidelines: barriers and solutions. Methods Inf Med 2002;41:435-42.
36. Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003;10:523-30.
37. Hersh WR. Medical informatics: improving health care through information. JAMA 2002;288:1955-8.
38. Bodenheimer T, Grumbach K. Electronic technology: a spark to revitalize primary care? JAMA 2003;290:259-64.
39. Christakis DA, Zimmerman FJ, Wright JA, Garrison MM, Rivara FP, Daivs RL. A randomized controlled trial of point-of-care evidence to improve the antibiotic prescribing practices for otitis media in children. Pediatrics 2001;107:E15.
40. Lobach DF. Electronically distributed, computergenerated, individualized feedback enhances the use of a computerized practice guideline. Proceedings of the AMIA Annual Fall Symposium 1996;493-7.
41. Eccles M, McColl E, Steen N. Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ 2002;325:941.
42. Bennett JW, Glasziou PP. Computerised reminders and feedback in medication management: a systematic review of randomised controlled trials. Med J Aust 2003;178:217-22.
43. Lowensteyn I, Joseph L, Levinton C, Abrahamowicz M, Steinert Y, Grover S. Can computerized risk profiles help patients improve their coronary risk? The results of the coronary health assessment study (CHAS). Prev Med 1998;27:730-7.
44. Miller RA. Medical diagnostic decision support systems-past, present, and future: a threaded bibliography and brief commentary. J Am Med Inform Assoc 1994;1:8-27.
45. Morris AH. Academia and clinic. Developing and implementing computerized protocols for standardization of clinical decisions. Ann Intern Med 2001;132:373-83.
46. Lobach DF, Hammond WE. Computerized decision support based on a clinical practice guideline improves compliance with care standards. Am J Med 1997;102:89-98.
47. Litzelman DK, Dittus RS, Miller ME, Tierney WM. Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols. J Gen Intern Med 1993;8:311-17.
48. Overhage JM, Tierney WM, McDonald CJ. Computer reminders to implement preventive care guidelines for hospitalized patients. Arch Intern Med 1996;156:1551-6.
49. Meyer TJ, Van Kooten D, Marsh S, Prochazka AV. Reduction of polypharmacy by feedback to clinicians. J Gen Intern Med 1991;6:133-6.
50. Cobos A, Vilaseca J, Asenjo C, Pedro-Botet J, Sanchez E, Val A, et al. Cost effectiveness of a clinical decision support system based on the recommendations of the European Society of Cardiology and other societies for the management of hypercholesterolemia: Report of a clusterrandomized trial. Dis Manag Health Outcomes 2005;13:421-32.
51. Nicholls GM, Dissanayake AM, Hazell W. Notification of random hyperglycaemia to general practitioners by an emergency medicine team: impact of a simple intervention plan. Diabet Med 2008;25:751-4.
52. Kuperman GJ, Teich JM, Tanasijevic MJ, Ma'Luf N, Rittenberg E, Jha A et al. Improving response to critical laboratory results with automation: results of a randomized controlled trial. J Am Med Inform Assoc 1999;6:512-22.
53. Stock JL, Waud CE, Coderre JA, Overdoft JH, Janikas JS, Heiniluoma KM. Clinical reporting to primary care physicians leads to increased use and understanding of bone densitometry and affects the management of oesteoporosis. A randomized trial. Ann Intern Med 1998;128:996-9.
54. Harpole LH, Khorasani R, Fiskio J, Kuperman GJ, Bates DW. Automated evidence-based critiquing of orders for abdominal radiographs: impact on utilization and appropriateness. J Am Med Inform Assoc 1997;4:511-21.
55. Sim I, Gorman P, Greenes RA, Haynes RB, Kaplan B, Lehmann H. Clinical decision support systems for the practice of evidence-based medicine. J Am Med Inform Assoc 2001;8:527-34.
56. Ash JS, Stavri PZ, Kuperman GJ. A consensus statement on considerations for a successful CPOE implementation. J Am Med Inform Assoc 2003;10:229-34.
57. O'Connor PJ, Crain AL, Rush WA, Sperl-Hillen JM, Gutenkauf JJ, Duncan JE. Impact of an electronic medical record on diabetes quality of care. Ann Fam Med 2005;3:300-6.
58. Heathfield HA, Wyatt J. Philosophies for the design and development of clinical decision-support systems. Methods Inf Med 1989;32:1-8.
59. Frances CD, Alperin P, Adler JS, Grady D. Does a fixed physician reminder system improve the care of patients with coronary artery disease? A randomized controlled trial. West J Med 2001;175:165-6.
60. Belcher DW, Berg AO, Inui TS. Practical approaches to providing better preventive care: are physicians the problem or a solution? Am J Prev Med 1988;4.
61. Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care. Med Care 1986;24:659-66.
62. White P, Atherton A, Hewett G, Howells K. Using information from asthma patients: a trial of information feedback in primary care. BMJ 1995;311:1065-9.
63. Balas EA, Weingarten S, Garb CT, Blumenthal D, Boren SA, Brown GD. Improving preventive care by prompting physicians. Arch Intern Med 2000; 160:301-8.
64. McPhee SJ, Detmer WM. Office-based interventions to improve delivery of cancer prevention services by primary care physicians. Cancer 1993;72:1100-12.
65. McPhee SJ, Bird JA, Jenkins CN, Fordham D. Promoting cancer screening. A randomized, controlled trial of three interventions. Arch Intern Med 1989;149:1866-72.
66. Becker DM, Gomez EB, Kaiser DL, Yoshihasi A, Hodge RH. Improving preventive care at a medical clinic: how can the patient help? Am J Prev Med 1989;5:353-9.
67. Fordham D, McPhee SJ, Bird JA, Rodnick JE, Detmer WM. The Cancer Prevention Reminder System. MD Computing 1990;7:289-95.
68. Thompson W, Dodek PM, Norena M, Dodek J. Computerized physician order entry of diagnostic tests in an intensive care unit is associated with improved timeliness of service. Crit Care Med 2004;32:1306-9.
69. Strecher VJ, O'Malley MS, Villagra VG, Campbell EE, Gonzalez JJ, Irons TG. Can residents be trained to counsel patients about quitting smoking? Results from a randomized trial. J Gen Intern Med 1991;6:9-17.
70. McAlister NH, Covvey HD, Tong C, Lee A, Wigle ED. Randomised controlled trial of computer assisted management of hypertension in primary care. Br Med J 1986;293:670-4.
71. Buchsbaum DG, Buchanan RG, Lawton MJ, Elswick RK, Schnoll SH. A program of screening and prompting improves short-term physician References 106 counseling of dependent and nondependent harmful drinkers. Arch Intern Med 1993;153:1573-7.
72. Liu JLY, Wyatt J, Deeks JJ, Clamp S, Keen J, Verde P, et al. Systematic reviews of clinical decision tools for acute abdominal pain. Health Technol Assess 2006;10(47).
73. Spiegelhalter D, Knill-Jones R. Statistical and knowledge-based approaches to clinical decision support systems, with an application to gastroenterology. J R Stat Soc 1984;147:35-77.
74. Wyatt J. Decision support systems. J R Soc Med 2000;93:629-33.
75. Wyatt J, Spiegelhalter D. Evaluating medical expert systems: what to test and how? Med Inform 1990;15:205-17.
76. Balas EA, Boren SA. Clinical trials of information interventions. In: Berner ES, editor. Clinical decision support systems: theory and practice. Volume 1, issue 2. New York, NY: Springer; 1999. pp. 365-98
77. Ohmann C, Boy O, Yang Q. A systematic approach to the assessment of user satisfaction with health care systems: constructs, models and instruments. Stud Health Technol Inform 1997;43:781-5.
78. Ranger S (editor). NHS IT spending will reach 12.4 bn. 2009. URL: www.silicon.com/publicsector/0.3800010403.39159681n. Accessed 6 June 2009.
79. Khan K, Ter Riet G, Glanville J, Sowden A, Kleijnen, J. Undertaking systematic reviews of research on effectiveness: CRD's guidance for those carrying out or commissioning reviews. York, UK: Centre for Reviews and Dissemination; University of York; 2001.
80. De Dombal FT, Leaper DJ, Horrocks JC, Staniland JR, McCann AP. Human and computeraided diagnosis of abdominal pain: further report with emphasis on performance of clinicians. BMJ 1974;1:376-80.
81. Cohen JA. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37-46.
82. Cochrane Effective Practice and Organisation of Care Group. EPOC Methods paper: Including Interrupted Time Series (ITS) Designs in a EPOC Review; 2009.
83. Evers S, Goossens M, de Vet H, van Tulder M, Ament A. Criteria list for assessment of methodological quality of economic evaluations: Consensus on Health Economic Criteria. Int J Technol Assess Health Care 2005;21:240-5.
84. Popay J, Roberts HT, Sowden AP, Britten N, Arai L. Developing guidance on the conduct of narrative synthesis in systematic reviews. J Epidemiol Community Health 2005;59(Suppl. 1):A7.
85. Popay J, Roberts HT, Sowden AP, Arai L, Rodgers M. Guidance on the conduct of narrative synthesis in systematic reviews. 2006. ESRC Research Methods Programme.
86. Tierney WM, Miller ME, McDonald CJ. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med 1990;322:1499-1504.
87. Abboud PA, Ancheta R, McKibben M, Jacobs BR, Group CIOR. Impact of workflow-integrated corollary orders on aminoglycoside monitoring in children. Health Inform J 2006;12:187-98.
88. Solomon DH, Shmerling RH, Schur PH, Lew R, Fiskio J, Bates DW. A computer based intervention to reduce unnecessary serologic testing. J Rheumatol 1999;26:2578-84.
89. Bansal P, Aronsky D, Aronsky D, Talbert D, Miller RA. A computer based intervention on the appropriate use of arterial blood gas. Proceedings of the AMIA Annual Symposium 2001:6.
90. Palen TE, Raebel M, Lyons E, Magid DM. Evaluation of laboratory monitoring alerts within a computerized physician order entry system for medication orders. Am J Manag Care 2006;12:389-95.
91. Matheny ME, Sequist TD, Seger AC, Fiskio JM, Sperling M, Bugbee D, et al. A randomized trial of electronic clinical reminders to improve medication laboratory monitoring. J Am Med Inform Assoc 2008;15:424-9.
92. Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma'Luf N, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med 1999;106:144-50.
93. Carton M, Auvert B, Guerini H, Boulard JC, Heautot JF, Landre MF, et al. Assessment of radiological referral practice and effect of computer-based guidelines on radiological requests in two emergency departments. Clin Radiol 2002;57:123-8.
94. Steele AW, Eisert S, Witter J, Lyons P, Jones MA, Gabow P, et al. The effect of automated alerts on provider ordering behavior in an outpatient setting. PLoS Med 2005;2:255.
95. Rosenbloom ST, Chiu KW, Byrne DW, Talbert DA, Neilson EG, Miller RA. Interventions to regulate ordering of serum magnesium levels: report of an unintended consequence of decision support. J Am Med Inform Assoc 2005;12:546-53.
96. Poley MJ, Edelenbos KI, Mosseveld M, van Wijk MA, de Bakker DH, van der LJ, et al. Cost consequences of implementing an electronic decision support system for ordering laboratory tests in primary care: evidence from a controlled prospective study in the Netherlands. Clin Chem 2007;53:213-19.
97. Apkon M, Mattera JA, Lin Z, Herrin J, Bradley EH, Carbone M, et al. A randomized outpatient trial of a decision-support information technology tool. Arch Intern Med 2005;165:2388-94.
98. Bassa A, Del VM, Cobos A, Torremade E, Bergonon S, Crespo C, et al. Impact of a clinical decision support system on the management of patients with hypercholesterolemia in the primary healthcare setting. Dis Manag Health Outcomes 2005;13:65-72.
99. Nightingale PG, Peters M, Mutimer D, Neuberger JM. Effects of a computerised protocol management system on ordering of clinical tests. Qual Health Care 1994;3:23-8.
100. Boon-Falleur L, Sokal E, Peters M, Ketelslegers JM. A rule-based decision support application for laboratory investigations management. Proceedings of the Annual Symposium on Computer Applications in Medical Care 1995;314-18.
101. Bates DW, Kuperman GJ, Jha A, Teich JM, Orav EJ, Ma'Luf N, et al. Does the computerized display of charges affect inpatient ancillary test utilization? Arch Intern Med 1997;157:2501-8.
102. Subramanian U, Fihn SD, Weinberger M, Plue L, Smith FE, Udris EM, et al. A controlled trial of including symptom data in computer-based care suggestions for managing patients with chronic heart failure. Am J Med 2004;116:375-84.
103. Ash JS, Stavri PZ, Dykstra R, Fournier L. Implementing computerized physician order entry: the importance of special people. Int J Medl Inform 2003;69:235-50.
104. Sanders DL, Miller RA. The effects on clinician ordering patterns of a computerized decision support system for neuroradiology imaging studies. Proceedings of the AMIA Annual Symposium 2001:7.
105. Sittig DF, Krall MA, Dykstra RH, Russell A, Chin HL. A survey of factors affecting clinician acceptance of clinical decision support. BMC Med Inform Decis Mak 2006;6:6.
106. Suleebe G, Morcka K. Diagnostic and prognostic significance of different antinuclear antibodies in more than 1000 consecutive Albanian patients with rheumatic diseases. Clin Exp Rheumatol 1992;10:255-61.
107. Hochberg MC, Boyd RE, Ahearn JM. Systemic lupus erythematosus: a review of clinico-laboratory features and innunogenetic markers in 150 patients with emphasis on demographic subsets. Medicine 1985;64:285-95.
108. Reichlin M, Harley JB. Antinuclear antibodies: an overview. In Wallace DJ, Hahn BH, editors. Dubois' lumus erythematosus. Baltimore, MD: Williams and Wilkins; 1997. pp. 64-93.
109. Wolfe F, Cathey MA, Roberts FK. The latex test revisited: rheumatoid factor testing in 8287 rheumatic disease patients. Arthritis Rheum 1991;34:951-60.
110. Nimelstein SH, Brody S, McShane D, Holman HR. Mixed connective tissue disease: a subsequent evaluation of the original 25 patients. Medicine 1980;59:239-48.
111. Bloch KJ, Buchanan WW, Wohl MJ, Bunim JJ. Sjogren's syndrome: a clinical, pathological, and serological study of sixty-two cases. Medicine 1965;44:187-225.
112. Association. Subcommittee for scleroderma criteria for the American Rheumatism. Preliminary criteria for the classification of systemic sclerosis (scleroderma). Arthritis Rheum 1980;23:581-90.
113. US Preventitive Services Task Force. Guide to Clinical Preventive Services: An assessment of the Effectiveness of 169 interventions. Baltimore, MD: Williams and Wilkins; 1989.
114. Knoben J. P. A. Handbook of Clinical Drug Data. 6th Ed. Hamilton, NY: Drug Intelligence Publications; 1988.
115. American Diabetes Association. Standards of Medical Care in Diabetes. Diabetes Care 2005;28(Suppl. 1):4-36.
116. Henrich WL, Agoda LE, Barrett B, Bennett WM, Blantz RC, Buckalew VM. Analgesics and the kidney: summary and recommendations to the Scientific Advisory Board of the National Kidney Foundation from an Ad Hoc Committee of the Natinoal Kidney Foundation. Am J Kidney Dis 1996;27:162-5.
117. Chobanian AV, Bakris GL, Black HR, Cushman WC, Green LA, Izzo JL. The Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation and Treatment of High Blood Pressure: the JNC 7 report. JAMA 2003;289:2560-72.
118. Expert consensus guideline series. Optimizing pharmacologic treatment of psychotic disorders. J Clin Psych 2003;64(Suppl. 12):2-97.
119. Renshaw AA, Stelling JM, MH D. The lack of value of repeated Clostiridium difficile cytotoxity assays. Arch Pathol Lab Med 1996;120:49-52.
120. Bates DW, Kuperman GJ, Boyle DL, Rittenberg E, Menkin V, Tanasijevic M. Clinical laboratory tests: what proportion are redundant and potentially eliminable using simple computerized rules? Am J Med 1998;104:261-368.
121. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: develoment and validation. J Chroni Dis 1987;40:373-83.
122. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9- CM administrative databases. J Clin Epidemiol 1992;45:613-19.
123. Rush, WA, O'Connor PJ, Goodman MJ. Validation of a modified Charlson score for using health plan claims data. Minnesota Health Services Research Conference, Minneapolis, MN, 2000.
124. Second Joint Task Force of European and Other Societies on Coronary Prevention. Prevention of coronary heart disease in clinical practice. Eur Heart 1998;19:1434-503.
125. Lago F, bad Perez JJ, Alverez A. Recomendaciones semFYC-Dislipemias. 1997. Barcelona, SemFYC.
126. Cobos A, Jovell AJ, Garcoa A. Which statin is most efficient for the treatment of hypocholesterolemia? A cost-effectivneness analysis. Clin Therapy 1999;1924-36.
127. Oblikue Consulting. Base de Datos de Costes Sanitarios SOIKOS 2009. URL: www.oblikue.com/ bddcostes.
128. Ali NA, Mekhjian HS, Kuehn PL, Bentley TD, Kumar R, Ferketich AK, et al. Specificity of computerized physician order entry has a significant effect on the efficiency of workflow for critically ill patients. Crit Care Med 2005;33:110-14.
129. Beyond guidelines: tool arms physicians with critical knowledge at the point of care. Dis Manag Advis 2001;8:9-12.
130. Computerized system alerts docs to costs. Ed Management 1999;11:100-2.
131. Anton C, Nightingale PG, Adu D, Lipkin G, Ferner RE. Improving prescribing using a rule based prescribing system. Qual Saf Health Care 2004;13:186-90.
132. Augstein P, Vogt L, Kohnert KD, Freyse EJ, Heinke P, Salzsieder E. Outpatient assessment of Karlsburg Diabetes Management System-based decision support. Diabetes Care 2007;30:1704-8.
133. Ayanian JZ, Sequist TD, Zaslavsky AM, Johannes RS. Physician reminders to promote surveillance colonoscopy for colorectal adenomas: A randomized controlled trial. J Gen Intern Med 2008;23:762-7.
134. Bernstein RM, Hollingworth GR, Wood WE. Prompting physicians for cost-effective test ordering in the low prevalence conditions of family medicine. Proceedings of the Annual Symposium on Computer Applications in Medical Care 1994;824-8.
135. Berry DL, Wolpin SE, Lober WB, Ellis WJ, Russell KJ, Davison BJ. Actual use and perceived usefulness of a web-based, decision support program for men with prostate cancer. Stud Health Technol Inform 2006;122:781-2.
136. Bindels R, Hasman A, van Wersch JW, Talmon J, Winkens RA. Evaluation of an automated test ordering and feedback system for general practitioners in daily practice. Intern J Med Inform 2004;73:705-12.
137. Blaser R, Schnabel M, Biber C, Baumlein M, Heger O, Beyer M, et al. Improving pathway compliance and clinician performance by using information technology. Intern J Med Inform 2007;76:151-6.
138. Braham RL, Ruchlin HS. Physician practice profiles: a case study of the use of audit and feedback in an ambulatory care group practice. Health Care Manage Rev 1987;12:11-16.
139. Cannon-Wagner M, Soule DS, Walker DR, Vance RP. An independent practice association-supported disease management program: Impact on patients with coronary artery disease. Dis Manag Health Outcomes 2002;10:571-7.
140. Chambers CV, Balaban DJ, Carlson BL, Ungemack JA, Grasberger DM. Microcomputergenerated reminders. Improving the compliance of primary care physicians with mammography screening guidelines. J Fam Prac 1989;29:273-80.
141. Christensen-Szalanski JJ, Diehr PH, Wood RW, Tompkins RK. Phased trial of a proven algorithm at a new primary care clinic. Am J Pub Health 1982;72:16-21.
142. Chu S. Non-intrusive guideline-based electronic disease management programme: Principles and evaluation of a pilot. J Inform Technol Healthcare 2004;2:263-80.
143. Chu S. Computerised clinical pathway management systems and the implications. Collegian: 2001;8:19-24.
144. Colombet I, Bura-Riviere A, Chatila R, Chatellier G, Durieux P, et al. Personalized versus non-personalized computerized decision support system to increase therapeutic quality control of oral anticoagulant therapy: an alternating time series analysis. BMC Health Serv Res 2004;4:27.
145. Colombet I, Dart T, Leneveut L, Zunino S, Menard J, Chatellier G. Combining risks estimations and clinical practice guidelines in a computer decision aid: a pilot study of the EsPeR system. Stud Health Technol Inform 2003;95:525-30.
146. Connelly DP, Sielaff BH, Willard KE. The clinical workstation as a means of improving laboratory use. Clinica Chimica Acta 1996;248:51-64.
147. Connelly DP, Willard KE, Hallgren JH, Sielaff BH. Closing the clinical laboratory testing loop with information technology. Am J Clin Pathol 1996;105(Suppl. 1):Suppl. 7.
148. Connelly DP, Sielaff BH, Willard KE. A clinician's workstation for improving laboratory use. Integrated display of laboratory results. Am J Clin Pathol 1995;104:243-52.
149. Cordero L, Kuehn L, Kumar RR, Mekhjian HS. Impact of computerized physician order entry on clinical practice in a newborn intensive care unit. J Perinatol 2004;24:88-93.
150. De Wilde EJL, Pop P, Hasman A, Blom JA. OpenLabs services for ordering laboratory investigations. Comput Methods Programs Biomed 1996;50:135-41.
151. Demakis JG, Beauchamp C, Cull WL, Denwood R, Eisen SA, Lofgren R, et al. Improving residents' compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. JAMA 2000;284:1411-16.
152. Emerson JF, Emerson SS. The impact of requisition design on laboratory utilization. Am J Clin Pathol 2001;116:879-84.
153. Fihn SD, McDonell MB, Vermes D, Henikoff JG, Martin DC, Callahan CM, et al. A computerized intervention to improve timing of outpatient follow-up: a multicenter randomized trial in patients treated with warfarin. National Consortium of Anticoagulation Clinics. J Gen Internl Med 1994;9:131-9.
154. Fitzmaurice DA, Hobbs FD, Murray ET, Holder RL, Allan TF, Rose PE. Oral anticoagulation management in primary care with the use of computerized decision support and near-patient testing: a randomized, controlled trial. Arch Intern Med 2000;160:2343-8.
155. Fransen J, Twisk JW, Creemers MC, van Riel PL. Design and analysis of a randomized controlled trial testing the effects of clinical decision support on the management of rheumatoid arthritis. Arthritis Rheum 2004;51:124-7.
156. Georgiou A, Williamson M, Westbrook JI, Ray S. The impact of computerised physician order entry systems on pathology services: a systematic review. Intern J Med Inform 2007;76:514-29.
157. Glasgow RE, Nutting PA, King DK, Nelson CC, Cutter G, Gaglio B, et al. Randomized effectiveness trial of a computer-assisted intervention to improve diabetes care. Diabetes Care 2005;28:33-9.
158. Groopman DS, Powers RD. Effect of 'standard order' deletion on emergency department coagulation profile use. Ann Emerg Med 1992;21:524-7.
159. Guss DA, Chan TC, Killeen JP. The impact of a pneumatic tube and computerized physician order management on laboratory turnaround time. Ann Emerg Med 2008;51:181-5.
160. Harris RP, O'Malley MS, Fletcher SW, Knight BP. Prompting physicians for preventive procedures: a five-year study of manual and computer reminders. Am Journal Prev Med 1990;6:145-52.
161. Hasman A, Pop P, Winkens RA, Blom JL. To test or not to test, that is the question. Clinica Chimica Acta 1993;222:49-56.
162. Hetlevik I, Holmen J, Kruger O. Implementing clinical guidelines in the treatment of hypertension in general practice. Evaluation of patient outcome related to implementation of a computer-based clinical decision support system. Scand J Prim Health Care 1999;17:35-40.
163. Hetlevik I, Holmen J, Kruger O, Kristensen P, Iversen H. Implementing clinical guidelines in the treatment of hypertension in general practice. Blood Press 1998;7:270-6.
164. Holleman DR, Jr., Simel DL. Effectiveness of automatic diagnostic test result feedback on outpatient laboratory and radiology testing in veterans. A controlled trial. Med Care 1996;34:857-61.
165. Kern LM, Barron Y, Blair AJ, III, Salkowe J, Chambers D, Callahan MA, et al. Electronic result References 110 viewing and quality of care in small group practices. J Genl Intern Med 2008;23:405-10.
166. Kinney WC. Web-based clinical decision support system for triage of vestibular patients. Otolaryngol Head Neck Surg 2003;128:48-53.
167. Koide D, Ohe K, Kitamura K, Kitagawa M, Yoshihara H, Nagase T, et al. System for warning on excessive laboratory tests. Japan J Med Inform 1995;15:217-27.
168. Maass MC, Asikainen P, Maenpaa T, Wanne O, Suominen T. Usefulness of a Regional Health Care Information System in primary care: a case study. Comput Methods Programs Biomed 2008;91:175-81.
169. Mantha S, Roizen MF, Madduri J, Rajender Y, Shanti NK, Gayatri K. Usefulness of routine preoperative testing: a prospective single-observer study. J Clin Anesth 2005;17:51-7.
170. Martens JD, van der AA, Panis B, van der WT, Winkens RA, Severens JL. Design and evaluation of a computer reminder system to improve prescribing behaviour of GPs. Studies Health Technol Inform 2006;124:617-23.
171. Modai I, Sigler M. A computerized laboratory alerting system in a psychogeriatric unit. MD Computing 1998;15:95-9.
172. Mutimer D, McCauley B, Nightingale P, Ryan M, Peters M, Neuberger J. Computerised protocols for laboratory investigation and their effect on use of medical time and resources. J Clin Pathol 1992;45:572-4.
173. Nam HS, Han SW, Ahn SH, Lee JY, Choi HY, Park IC, et al. Improved time intervals by implementation of computerized physician order entry-based stroke team approach. Cerebrovasc Dis 2007;23:289-93.
174. Neilson EG, Johnson KB, Rosenbloom ST, Dupont WD, Talbert D, Giuse DA, et al. The impact of peer management on test-ordering behavior. Ann Intern Med 2004;141:196-204.
175. Nilasena DS, Lincoln MJ. A computer-generated reminder system improves physician compliance with diabetes preventive care guidelines. Proceedings of the Annual Symposium on Computer Applications in Medical Care 1995;640-5.
176. Ornstein SM, Garr DR, Jenkins RG, Musham C, Hamadeh G, Lancaster C. Implementation and evaluation of a computer-based preventive services system. Fam Med 1995;27:260-6.
177. Patkar V, Hurt C, Steele R, Love S, Purushotham A, Williams M, et al. Evidence-based guidelines and decision support services: A discussion and evaluation in triple assessment of suspected breast cancer. Br J Cancer 2006;95:1490-6.
178. Payne TH, Hoey PJ, Nichol P, Lovis C. Preparation and use of preconstructed orders, order sets, and order menus in a computerized provider order entry system. J Am Med Inform Assoc 2003;10:322-9.
179. Perkins J, Ambroson R, Kinsey K. Bring 'order' to diabetes management. Nurs Manage 2008;39:24-30.
180. Pham DQ, Pham AQ, Ullah E, McFarlane SI, Payne R. Evaluating the appropriateness of thromboprophylaxis in an acute care setting using a computerised reminder, through order-entry system. Int J Clin Pract 2008;62:134-7.
181. Piva E, Sciacovelli L, Zaninotto M, Laposata M, Plebani M. Evaluation of effectiveness of a computerized notification system for reporting critical values. Am J Clin Pathol 2009;131:432-41.
182. Rosenthal DI, Weilburg JB, Schultz T, Miller JC, Nixon V, Dreyer KJ, et al. Radiology order entry with decision support: initial clinical experience. J Am Coll Radiol 2006;3:799-806.
183. Stair TO. Reduction of redundant laboratory orders by access to computerized patient records. J Emerg Med 1998;16:895-7.
184. Studnicki J, Bradham DD, Marshburn J, Foulis PR, Straumfjord JV. A feedback system for reducing excessive laboratory tests. Arch Pathol Lab Med 1993;117:35-9.
185. Tierney WM, Miller ME, Overhage JM, McDonald CJ. Physician inpatient order writing on microcomputer workstations. Effects on resource utilization. JAMA 1993;269:379-83.
186. Valenstein PN, Howanitz PJ. Ordering accuracy: A College of American Pathologists Q-probes study of 577 institutions. Arch Pathol Lab Med 1995;119:117-22.
187. van Wijk MA, van der LJ, Mosseveld M, Bohnen AM, van Bemmel JH. Assessment of decision support for blood test ordering in primary care. A randomized trial. Ann Intern Med 2001;134:274-81.
188. Vashitz G, Meyer J, Gilutz H. General practitioners' adherence with clinical reminders for secondary prevention of dyslipidemia. Proceedings of the AMIA Annual Symposium 2007;766-70.
189. Aarts J, Berg M. A tale of two hospitals: a sociotechnical appraisal of the introduction of computerized physician order entry in two Dutch hospitals. Medinfo 2004;11(Pt 2):2-1002.
190. Ahearn MD, Kerr SJ. General practitioners' perceptions of the pharmaceutical decision-support tools in their prescribing software. Med J Aust 2003;179:34-7.
191. Alberdi E, Gilhooly K, Hunter J, Logie R, Lyon A, McIntosh N, et al. Computerisation and decision making in neonatal intensive care: A cognitive engineering investigation. J Clin Monit Comput 2000;16:85-94.
192. Asaro PV, Sheldahl AL, Char DM. Physician perspective on computerized order-sets with embedded guideline information in a commercial emergency department information system. Proceedings of the AMIA Annual Symposium 2005;10.
193. Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2007;14:415-23.
194. Ash JS, Gorman PN, Lavelle M, Stavri PZ, Lyman J, Fournier L, et al. Perceptions of physician order entry: results of a cross-site qualitative study. Methods Inf Med 2003; 42:313-23.
195. Aydin CE, Anderson JG, Rosen PN, Felitti VJ, Weng HC. Computers in the consulting room: a case study of clinician and patient perspectives. Health Care Manag Sci 1998;1:61-74.
196. Banet GA, Jeffe DB, Williams JA, Asaro PV. Effects of implementing computerized practitioner order entry and nursing documentation on nursing workflow in an emergency department. J Healthc Inf Manag 2006;20:45-54.
197. Bindels R, Hasman A, Derickx M, van Wersch JW, Winkens RA. User satisfaction with a real-time automated feedback system for general practitioners: a quantitative and qualitative study. Int J Qual Health Care 2003;15:501-8.
198. Bonnevie L, Thomsen T, Jorgensen T. The use of computerized decision support systems in preventive cardiology-principal results from the national PRECARD survey in Denmark. Eur J Cardiovasc Prev Rehabil 2005;12:52-5.
199. Bossen C. Evaluation of a computerized problemoriented medical record in a hospital department: does it support daily clinical practice? Int J Med Inform 2007;76:592-600.
200. Briggs JS, Fitch CJ. The ISABEL user survey. Med Inform Internet Med 2005;30:167-72.
201. Callen JL, Braithwaite J, Westbrook JI. Cultures in hospitals and their influence on attitudes to, and satisfaction with, the use of clinical information systems. Soc Sci Med 2007;65:635-9.
202. Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2006;13:547-56.
203. Fung CH, Tsai JS, Lulejian A, Glassman P, Patterson E, Doebbeling BN, et al. An evaluation of the Veterans Health Administration's clinical reminders system: a national survey of generalists. J Gen Intern Med 2008;23:392-8.
204. Gandhi TK, Poon EG, Sequist TD, Johnson R, Pizziferri L, Karson AS, et al. Primary care clinician attitudes towards ambulatory computerized physician order entry. Proceedings of the AMIA Annual Symposium 2005.
205. Grundmeier R, Johnson K. Housestaff attitudes toward computer-based clinical decision support. Proceedings of the AMIA Annual Symposium 1999;70.
206. Jaspers MWM, Peute LWP, Lauteslager A, Bakker PJM. Pre-post evaluation of physicians' satisfaction with a redesigned electronic medical record system. Studies Health Technol Inform 2008;136:303-8.
207. Kailajarvi M, Takala T, Gronroos P, Tryding N, Viikari J, Irjala K, et al. Reminders of drug effects on laboratory test results. Clin Chem 2000;46:1395-1400.
208. Krall MA, Sittig DF. Clinician's assessments of outpatient electronic medical record alert and reminder usability and usefulness requirements. Proceedings of the AMIA Annual Symposium 2002;4.
209. Krall MA, Sittig DF. Subjective assessment of usefulness and appropriate presentation mode of alerts and reminders in the outpatient setting. Proceedings of the AMIA Annual Symposium 2001;8.
210. Lee F, Teich JM, Spurr CD, Bates DW. Implementation of physician order entry: user satisfaction and self-reported usage patterns. J Am Med Inform Assoc 1996;3:42-55.
211. Murff HJ, Kannry J. Physician satisfaction with two order entry systems. J Am Med Inform Assoc 2001;8:499-509.
212. Rapley T, May C, Heaven B, Murtagh M, Graham R, Kaner EF, et al. Doctor-patient interaction in a randomised controlled trial of decision-support tools. Soc Sci Med 2006;62:2267-78.
213. Ridderikhoff J, van HB. Who is afraid of the system? Doctors' attitude towards diagnostic systems. Int J Med Inform 1999;53:91-100.
214. Rosenbloom ST, Talbert D, Aronsky D. Clinicians' perceptions of clinical decision support integrated into computerized provider order entry. Int J Med Inform 2004;73:433-41.
215. Short D, Frischer M, Bashford J. Barriers to the adoption of computerised decision support systems in general practice consultations: a qualitative study of GPs' perspectives. Int J Med Inform 2004;73:357-62.
216. Barnett PG, Rodgers JH. Use of the Decision Support System for VA cost-effectiveness research. Med Care 1999;37(Suppl. 4):Suppl-70.
217. Ohsfeldt RL, Ward MM, Schneider JE, Jaana M, Miller TR, Lei Y, et al. Implementation of hospital computerized physician order entry systems in a rural state: feasibility and financial impact. J Am Med Inform Assoc 2005;12:20-7.
218. Cobos A, Jovell AJ, Garcia-Altes A, Garcia-Closas R, Serra-Majem L. Which statin is most efficient for the treatment of hypocholesterolemia? A costeffectiveness analysis. Clin Ther 1999;21:1924-36.
219. Broughton PMG, Hogan TL. A new approach to the costing of clinical laboratory tests. Ann Clin Biochem 1981;18:330-42.