A Conceptual Framework for Evaluating Information Technologies and Decision Support Systems for Bioterrorism Preparedness and Response

Medical Decision Making

Volumn 24, Issue 2, 2004, Pages 192-206

  Bravata, Dena M ^a   McDonald, Kathryn M ^a   Szeto, Herbert ^b   Smith, Wendy M ^a   Rydzak, Chara ^a   Owens, Douglas K ^a,c,d  

^a STANFORD UNIVERSITY   (United States)

^b Kaiser Permanente ^*  (United States)

^c VA PALO ALTO HEALTH CARE SYSTEM   (United States)

^d STANFORD UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

Bioterrorism; Decision support techniques; Expert system; Information systems; Public health

Indexed keywords

ACCURACY; ARTICLE; BIOLOGICAL WARFARE; CONCEPT FORMATION; DECISION MAKING; DECISION SUPPORT SYSTEM; DECOMPOSITION; DIAGNOSTIC TEST; DISASTER PLANNING; EPIDEMIC; EVALUATION; HEALTH SURVEY; HUMAN; INFECTION CONTROL; INFORMATION PROCESSING; PUBLIC HEALTH; SENSITIVITY AND SPECIFICITY; TASK PERFORMANCE;

BIOTERRORISM; DISASTER PLANNING; HUMANS; INFORMATION SYSTEMS; PROGRAM EVALUATION; PUBLIC HEALTH; UNITED STATES;

EID: 1642380826     PISSN: 0272989X     EISSN: None     Source Type: Journal    
DOI: 10.1177/0272989X04263254     Document Type: Article

References (92)

1. Jernigan JA, Stephens DS, Ashford DA, et al. Bioterrorism-related inhalational anthrax: the first 10 cases reported in the United States. Emerg Infect Dis. 2001;7:933-44.
2. Update: investigation of bioterrorism-related inhalational anthrax-Connecticut, 2001. MMWR Morb Mortal Wkly Rep. 2001;50:1049-51.
3. Update: investigation of bioterrorism-related anthrax and interim guidelines for clinical evaluation of persons with possible anthrax. MMWR Morb Mortal Wkly Rep. 2001;50:941-8.
4. Update: investigation of bioterrorism-related anthrax and interim guidelines for exposure management and antimicrobial therapy, October 2001. MMWR Morb Mortal Wkly Rep. 2001;50:909-19.
5. Recognition of illness associated with the intentional release of a biologic agent. MMWR Morb Mortal Wkly Rep. 2001;50:893-7.
6. Update: investigation of anthrax associated with intentional exposure and interim public health guidelines, October 2001. MMWR Morb Mortal Wkly Rep. 2001;50:889-93.
7. Becker C. 20/20 hindsight: months after anthrax claimed the lives of several Americans, hospitals review their reaction to the event-and plan for future crises. Mod Healthc. 2002;32:8-9, 12.
8. Borio L, Frank D, Mani V, et al. Death due to bioterrorism-related inhalational anthrax: report of 2 patients. JAMA. 2001;286:2554-9.
9. Josefson D. Toll of anthrax cases reaches 15. BMJ. 2001;323:1022.
10. Mayer TA, Bersoff-Matcha S, Murphy C, et al. Clinical presentation of inhalational anthrax following bioterrorism exposure: report of 2 surviving patients. JAMA. 2001;286:2549-53.
11. Bravata DM, McDonald K, Owens DK, et al. Bioterrorism Preparedness and Response: Use of Information Technologies and Decision Support Systems (Evidence Report/Technology Assessment No. 59). Rockville (MD): Prepared by the UCSF-Stanford Evidence-Based Practice Center under contract No. 290-97-0013 for the Agency for Healthcare Research and Quality; June 2002.
12. Anderson JG, Aydin CE, Jay SJ, eds. Evaluating Health Care Information Systems: Methods and Applications. Thousand Oaks (CA): Sage; 1994.
13. Friedman CP, Wyatt JC. Evaluation Methods in Medical Informatics. New York: Springer-Verlag; 1997.
14. Owens DK, Bravata DM. Computer-based decision support: wishing on a star? Eff Clin Pract. 2001;4:34-8.
15. Kaplan B. Evaluating informatics applications-clinical decision support systems literature review. Int J Med Inf. 2001;64:15-37.
16. Ammenwerth E, Graber S, Herrmann G, Burkle T, Konig J. Evaluation of health information systems-problems and challenges. Int J Med Inf. 2003;71:125-35.
17. Goldratt EM. Theory of Constraints. Great Barrington (MA): North River Press; 1990.
18. Hunink MGM. In search of tools to aid logical thinking and communicating about medical decision making. Med Decis Making. 2001;21:267-77.
19. Friedman CP, Owens DK, Wyatt JC. Evaluation and technology assessment. In: Shortliffe EH, Perreault LE, Wiederhold G, Fagan LM, eds. Medical Informatics: Computer Applications in Health Care and Biomedicine. New York: Springer-Verlag; 2000.
20. Donabedian A. Exporations in Quality Assessment and Monitoring. Volume 2. Ann Arbor (MI): Health Administration Press; 1982.
21. Donabedian A. The quality of care: how can it be assessed? JAMA. 1988;260:1743-8.
22. Sim I, Owens DK, Lavori PW, Rennels GD. Electronic trial banks: a complementary method for reporting randomized trials. Med Decis Making. 2000;20:440-50.
23. Nease RF Jr, Owens DK. Use of influence diagrams to structure medical decisions. Med Decis Making. 1997;17:263-75.
24. Owens DK, Shachter RD, Nease RF. Representation and analysis of medical decision problems with influence diagrams. Med Decis Making. 1997;17:241-62.
25. Callan K. Preparing for a decision support system. Top Health Inf Manage. 2000;21:84-90.
26. Shortliffe EH. Computer programs to support clinical decision making. JAMA. 1987;258:61-6.
27. Inglesby T, Grossman R, O'Toole T. A Plague on Your City: Observations from TOPOFF. Center for Civilian Biodefense Studies, Johns Hopkins University. Available from: http://www.hopkins-biodefense.org/pages/news/quarter.html#Anchor-8398. Accessed 20 December 2000.
28. Hoffman RE, Norton JE. Lessons learned from a full-scale bioterrorism exercise. Emerg Infect Dis. 2000;6:652-3.
29. Dark Winter. Johns Hopkins Center for Civilian Biodefense, Center for Strategic and International Studies, ANSER, and Memorial Institute for the Prevention of Terrorism. Available from: http://www.hopkins-biodefense.org/participants.html. Accessed 8 November 2001.
30. MacKenzie WR, Schell WL, Blair KA, et al. Massive outbreak of waterborne cryptosporidium infection in Milwaukee, Wisconsin: recurrence of illness and risk of secondary transmission. Clin Infect Dis. 1995;21:57-62.
31. Outbreak of West Nile-like viral encephalitis-New York, 1999. MMWR Morb Mortal Wkly Rep. 1999;48):845-9.
32. Dennis DT, Inglesby TV, Henderson DA, et al. Tularemia as a biological weapon: medical and public health management. JAMA. 2001;285:2763-73.
33. Arnon SS, Schechter R, Inglesby TV, et al. Botulinum toxin as a biological weapon: medical and public health management. JAMA. 2001;285:1059-70.
34. Inglesby TV, Dennis DT, Henderson DA, et al. Plague as a biological weapon: medical and public health management. JAMA. 2001;285:1059-70.
35. Henderson DA, Inglesby TV, Bartlett JG, et al. Smallpox as a biological weapon: medical and public health management. JAMA. 1999;281:2127-37.
36. Inglesby TV, Henderson DA, Bartlett JG, et al. Anthrax as a biological weapon: medical and public health management. JAMA. 1999;281:1735-45.
37. Bioterrorism readiness plan-a template for healthcare facilities. Association for Professionals in Infection Control and Epidemiology Inc. and Centers for Disease Control and Prevention. ED Manag. 1999;11 Suppl:1-16.
38. Proceedings of the National Symposium on Medical and Public Health Response to Bioterrorism. Arlington, Virginia, USA. February 16-17, 1999. Emerg Infect Dis. 1999;5:491-592.
39. Smithson AE, Levy L. Ataxia: The Chemical and Biological Terrorism Threat and the U.S. Response. Washington (DC): Henry L. Stimson Center; 1999.
40. ReddiNet information binder. Los Angeles (CA): Healthcare Association of Southern California; 2001.
41. Centers for Disease Control and Prevention. Health Level Seven Specifications for Electronic Laboratory-Based Reporting of Public Health Information. Atlanta (GA): Centers for Disease Control and Prevention; 1997.
42. Electronic reporting of laboratory information for public health. Paper presented at the Conference on Electronic Reporting of Laboratory Information for Public Health, Centers for Disease Control and Prevention; 7-8 January 1999.
43. US Department of Health and Human Services. Public Health Conceptual Data Model. Atlanta (GA): US Department of Health and Human Services, Public Health Service; 2000.
44. Centers for Disease Control and Prevention. Secure Data Network Standards And Procedures. Atlanta (GA): Centers for Disease Control and Prevention, Agency for Toxic Substances and Disease Registry; 1999.
45. Owens DK, Nease RF. A normative analytic framework for development of practice guidelines for specific clinical populations. Med Decis Making. 1997;17:409-26.
46. Owens DK, Nease RF. Development of outcome-based practice guidelines: a method for structuring problems and synthesizing evidence. Jt Comm J Qual Improv. 1993;19:248-63.
47. Xiang Y, Poh KL. Time-critical dynamic decision modeling in medicine. Comput Biol Med. 2002;32:85-97.
48. Aliferis CF, Cooper GF, Pollack ME, Buchanan BG, Wagner MM. Representing and developing temporally abstracted knowledge as a means towards facilitating time modeling in medical decision-support systems. Comput Biol Med. 1997;27:411-34.
49. Hazen GB. Stochastic trees and the StoTree modeling environment: models and software for medical decision analysis. J Med Syst. 2002;26:399-413.
50. Hazen GB. Factored stochastic trees: a tool for solving complex temporal medical decision models. Med Decis Making. 1993;13:227-36.
51. Hazen GB. Stochastic trees: a new technique for temporal medical decision modeling. Med Decis Making. 1992;12:163-78.
52. Magni P, Quaglini S, Marchetti M, Barosi G. Deciding when to intervene: a Markov decision process approach. Int J Med Inf. 2000;60:237-53.
53. Sim I, Owens D, Lavori P, Rennels G. Electronic trial banks: a complementary method for reporting randomized trials. Med Decis Making. 2000;20:440-50.
54. Chandrasekaran B, Johnson T. Generic tasks and task structure: history, critique and new directions. In: David J, Krivine J, Simmons R, eds. Second-Generation Expert Systems. New York: Springer-Verlag; 1993. p 232-72.
55. Gruninger M, Fox M. Methodology for the design and evaluation of ontologies. Toronto (Canada): Department of Industrial Engineering; 1995.
56. Guidelines for evaluating surveillance systems. MMWR Morb Mortal Wkly Rep. 1988;37 Suppl 5:1-18.
57. Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998; 280:1339-46.
58. Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research. Ann Intern Med. 1994;120:135-42.
59. Hornberger J, Goldstein MK. Clinical decision-support systems: evaluating the evaluation. Med Decis Making. 2000;20:130-1.
60. Sox HC, Blatt MA, Higgins MC, Marton KI. Medical Decision Making. Boston: Butterworth-Heinemann; 1988.
61. Adlassnig KP, Scheithauer W. Performance evaluation of medical expert systems using ROC curves. Comput Biomed Res. 1989; 22:297-313.
62. Sosin DM. Draft framework for evaluating syndromic surveillance systems. J Urban Health. 2003;80 Suppl 1:I8-I13.
63. Berner ES, Webster GD, Shugerman AA, et al. Performance of four computer-based diagnostic systems. N Engl J Med. 1994;330:1792-6.
64. Bankowitz RA, Blumenfeld BH, Giuse BN. User variability in abstracting and entering printed case histories with Quick Medical Reference (QMR). Proceedings of the Annual Symposium on Computer Applications in Medical Care; 1987. p 68-73.
65. Bankowitz RA, Miller JK, Janosky J. A prospective analysis of inter-rater agreement between a physician and a physician's assistant in selecting QMR vocabulary terms. Proceedings of the Annual Symposium on Computer Applications in Medical Care; 1991. p 609-13.
66. Lemaire JB, Schaefer JP, Martin LA, Faris P, Ainslie MD, Hull RD. Effectiveness of the Quick Medical Reference as a diagnostic tool. CMAJ. 1999;161:725-8.
67. Bankowitz RA, McNeil MA, Challinor SM, Parker RC, Kapoor WN, Miller RA. A computer-assisted medical diagnostic consultation service. Implementation and prospective evaluation of a prototype. Ann Intern Med. 1989;110:824-32.
68. Bankowitz RA, McNeil MA, Challinor SM, Miller RA. Effect of a computer-assisted general medicine diagnostic consultation service on housestaff diagnostic strategy. Methods Inf Med. 1989;28:352-6.
69. Arene I, Ahmed W, Fox M, Barr CE, Fisher K. Evaluation of quick medical reference (QMR) as a teaching tool. MD Comput. 1998; 15:323-6.
70. Miller RA, McNeil MA, Challinor SM, Masarie FE Jr, Myers JD. The INTERNIST-1/Quick Medical Reference project-status report. West J Med. 1986;145:816-22.
71. Miller R, Masarie FE, Myers JD. Quick medical reference (QMR) for diagnostic assistance. MD Comput. 1986;3:34-48.
72. Bacchus CM, Quiston C, O'Rourke K, Detsky AS. A randomized crossover trial of QMR as a teaching tool for medical interns. J Gen Intern Med. 1994;9:616-21.
73. Berner ES, Maisiak RS. Influence of case and physician characteristics on perceptions of decision support systems. J Am Med Inform Assoc. 1999;6:428-34.
74. Berner ES, Maisiak RS, Cobbs CG, Taunton OD. Effects of a decision support system on physicians' diagnostic performance. J Am Med Inform Assoc. 1999;6:420-7.
75. Quick Medical Reference. First DataBank. Available from: http://www.firstdatabank.com. Accessed 14 November 2001.
76. Fiszman M, Haug PJ. Using medical language processing to support real-time evaluation of pneumonia guidelines. Proceedings of the AMIA Annual Symposium. 2000;27:235-9.
77. Aronsky D, Haug PJ. An integrated decision support system for diagnosing and managing patients with community-acquired pneumonia. Proceedings of the AMIA Annual Symposium; 1999. p 197-201.
78. Evans RS, Larsen RA, Burke JP, et al. Computer surveillance of hospital-acquired infections and antibiotic use. JAMA. 1986;256:1007-11.
79. Evans RS, Classen DC, Pestotnik SL, Lundsgaarde HP, Burke JP. Improving empiric antibiotic selection using computer decision support. Arch Intern Med. 1994;154:878-84.
80. Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted management program for antibiotics and other antiinfective agents. N Engl J Med. 1998;338:232-8.
81. Pestotnik SL, Evans RS, Burke JP, Gardner RM, Classen DC. Therapeutic antibiotic monitoring: surveillance using a computerized expert system. Am J Med. 1990;88:43-8.
82. Pestotnik SL, Classen DC, Evans RS, Burke JP. Implementing antibiotic practice guidelines through computer-assisted decision support: clinical and financial outcomes. Ann Intern Med. 1996; 124:884-90.
83. Hall B. Director, Rapid Health Electronic Alert, Communications, and Training System (RHEACTS). In: Bravata DM, McDonald K, Owens DK, et al. Bioterrorism Preparedness and Response: Use of Information Technologies and Decision Support Systems (Evidence Report/Technology AssessmentNo. 59). Rockville (MD): Prepared by the UCSF-Stanford Evidence-Based Practice Center under contract No. 290-97-0013 for the Agency for Healthcare Research and Quality; June 2002.
84. Ascher MS. In: Bravata DM, McDonald K, Owens DK, et al. Bioterrorism Preparedness and Response: Use of Information Technologies and Decision Support Systems (Evidence Report/Technology AssessmentNo. 59). Rockville (MD): Prepared by the UCSF-Stanford Evidence-Based Practice Center under contract No. 290-97-0013 for the Agency for Healthcare Research and Quality; June 2002.
85. California State Senate Committee on Health and Human Services. Bioterrorism and public health: assessing California's preparedness: background. Available from: http://www.sen.ca.gov/ftp/SEN/COMMITTEE/STANDING/HEALTH/_home/ BIOTERRORISM_BACKGROUND.DOC. Accessed 13 February 2002.
86. Pavlin JA, Kelley PW. ESSENCE: Electronic Surveillance System for the Early Notification of Community-Based Epidemics. Silver Spring (MD): US Department of Defense, Global Emerging Infections Surveillance and Response System; 2001.
87. Centers for Disease Control and Prevention. WONDER Logon. Available from: http://wonder.cdc.gov/. Accessed 5 October 2001.
88. Friede A, Reid JA, Ory HW. CDC WONDER: a comprehensive on-line public health information system of the Centers for Disease Control and Prevention. Am J Public Health. 1993;83:1289-94.
89. Friede A, Rosen DH, Reid JA. CDC WONDER: a cooperative processing architecture for public health. J Am Med Inform Assoc. 1994;1:303-12.
90. Ross JJ, Shapiro DS. Evaluation of the computer program GIDEON (Global Infectious Disease and Epidemiology Network) for the diagnosis of fever in patients admitted to a medical service. Clin Infect Dis. 1998;26:766-7.
91. International Society of Travel Medicine. GeoSentinel: surveillance strategy. Available from: http://www.istm.org/geosentinel/ surveill.html. Accessed 21 September 2001.
92. International Society of Travel Medicine. GeoSentinel: objectives. Available from: http://www.istm.org/geosentinel/objectiv. html. Accessed 21 September 2001.