Planning treatment of ischemic heart disease with partially observable Markov decision processes

Artificial Intelligence in Medicine

Volumn 18, Issue 3, 2000, Pages 221-244

  Hauskrecht, Milos ^a   Fraser, Hamish ^b  

^a Brown Univerity   (United States)

^b TUFTS MEDICAL CENTER   (United States)

Author keywords

Dynamic decision making; Ischemic heart disease; Medical therapy planning; Partially observable Markov decision process

Indexed keywords

DECISION MAKING; DISEASES; MARKOV PROCESSES; PATIENT TREATMENT;

MEDICAL THERAPY PLANNING; PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES (POMDP);

MEDICAL COMPUTING;

ANGIOPLASTY; ARTICLE; ARTIFICIAL INTELLIGENCE; CORONARY ARTERY BYPASS GRAFT; COST; DECISION MAKING; HUMAN; ISCHEMIC HEART DISEASE; MODEL; PRIORITY JOURNAL; TREATMENT PLANNING;

COST-BENEFIT ANALYSIS; DECISION MAKING; HEALTH CARE COSTS; HUMANS; MARKOV CHAINS; MYOCARDIAL ISCHEMIA; PATIENT CARE PLANNING;

EID: 0034160101     PISSN: 09333657     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0933-3657(99)00042-1     Document Type: Article

References (31)

1. Anderson K.M., Odell P.M., Wilson P.W.F., Kannel W.B. Cardiovascular disease risk profiles. Am. Heart J. 121:1990;293-298.
2. Astrom K.J. Optimal control of Markov decision processes with incomplete state estimation. J. Math. Anal. Applic. 10:1965;174-205.
3. Bellman R.E. Dynamic Programming. 1957;Princeton University Press, Princeton, NJ.
4. Bertsekas D.P. Dynamic Programming and Optimal Control. 1995;Athena Scientific, Belmont, MA.
5. Berzuini C, Bellazzi R, Spiegelhalter D. Bayesian networks applied to therapy monitoring. In Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, 1991, p. 35-43.
6. Boutilier C, Poole D. Exploiting structure in policy construction. In Proceedings of the Thirteenth National Conference on Artificial Intelligence, 1996, p. 1168-1175.
7. Dean T., Kanazawa K. A model for reasoning about persistence and causation. Comput. Intell. 5:1989;142-150.
8. Drake A. Observation of a Markov Process through a Noisy Channel. PhD thesis, Massachusetts Institute of Technology, 1962.
9. Hauskrecht M. Incremental methods for computing bounds in partially observable Markov decision processes. In Proceedings of the Fourteenth National Conference on Artificial Intelligence, 1997, p. 734-739.
10. Hauskrecht M. Planning and control in stochastic domains with imperfect information. PhD thesis, Massachusetts Institute of Technology, 1997.
11. Hauskrecht M, Fraser H. Modeling treatment of ischemic heart disease with partially observable Markov decision processes. In Proceedings of American Medical Informatics Association Anneal Symposium on Computer Applications in Health Care, 1998, p. 538-542.
12. Hauskrecht M, Fraser H. Planning medical therapy using partially observable Markov decision processes. In Proceedings of the Ninth International Workshop on Principles of Diagnosis (DX-98), 1998, p. 182-189.
13. Howard R.A. Dynamic Programming and Markov Processes. 1960;MIT Press, Cambridge.
14. Howard RA, Matheson JE. Influence diagrams. Principles and Applications of Decision Analysis 2, 1984.
15. Hu C., Lovejoy W.S., Shafer S.L. Comparison of some suboptimal control policies in medical drug therapy. Oper. Res. 44:1996;696-709.
16. Pack Kaelbling L, Littman ML, Cassandra AR. Planning and acting in partially observable stochastic domains. Artif. Intell. 1999:101.
17. Kjaerulff U. A computational scheme for reasoning in dynamic probabilistic networks. In Proceedings of the Eighth Conference on Uncertainty in Artificial Intelligence, 1992, p. 121-129.
18. Littman ML. Algorithms for Sequential Decision Making. PhD thesis, Brown University, 1996.
19. Lovejoy W.S. A survey of algorithmic methods for partially observed Markov decision processes. Ann. Oper. Res. 28:1991;47-66.
20. Mundhenk M, Goldsmith J, Lusena C, Allender E. Encyclopaedia of complexity results for finite-horizon Markov decision process problems. Technical Report, UK CS Dept TR 273-97, University of Kentucky, 1997.
21. Owens D.K., Sox H.C. Medical decision making: Probabilistic medical reasoning. Shortliffe E.H., Perreault L.E. Medical Informatics: Computer Applications in Health Care. 1990;Addison Wesley.
22. Papadimitriou C.H., Tsitsiklis J.N. The complexity of Markov decision processes. Math. Oper. Res. 12:1987;441-450.
23. Pearl J. Probabilistic Reasoning in Intelligent Systems. 1988;Kaufman.
24. Peek N. Explicit temporal models for decision-theoretic planning of clinical management. Artif. Intell. Med. 15:1999;135-154.
25. Platzman LK. Finite Memory Estimation and Control of Finite Probabilistic Systems. PhD thesis, Massachusetts Institute of Technology, 1977.
26. Puterman M.L. Markov Decision Processes: Discrete Stochastic Dynamic Programming. 1994;Wiley, New York.
27. Raiffa H. Decision Analysis. Introductory Lectures on Choices Under Uncertainty. 1970;Addison-Wesley.
28. Smallwood R.D., Sondik E.J. The optimal control of partially observable processes over a finite horizon. Oper. Res. 21:1973;1071-1088.
29. Sondik EJ. The Optimal Control of Partially Observable Markov Decision Processes. PhD thesis, Stanford University, 1971.
30. Tatman J.A., Schachter R.D. Dynamic programming and influence diagrams. IEEE Trans. Syst. Man Cyber. 20:1990;365-379.
31. Wong J.B., Sonnenberg F.A., Salem D.N., Pauker S.G. Myocardial revascularization for chronic stable angina. Ann. Intern. Med. 113:1990;852-871.