Dynamic decision making: A comparison of approaches

Journal of Multi-Criteria Decision Analysis

Volumn 9, Issue 6, 2000, Pages 243-262

  Da Costa, Paulo C G ^a   Buede, Dennis M ^a  

^a George Mason University   (United States)

Author keywords

Bayesian updating; DDN; Decision support; Dynamic decisions comparison; Multi criteria; POMDP

Indexed keywords

EID: 84975472936     PISSN: 10579214     EISSN: 10991360     Source Type: Journal    
DOI: 10.1002/1099-1360(200011)9:6<243::AID-MCDA281>3.0.CO;2-Z     Document Type: Article

References (61)

1. Astrom, AJ., 1965. Optimal control of Markov decision processes with incomplete state information. Journal of Mathematical Analysis and Applications 10: 174-205.
2. Bellman, R., 1957. Applied Dynamic Programming. Princeton University Press: Princeton, NJ.
3. Bertsekas, DP., 1995. Dynamic Programming and Optimal Control. Athena Scientific: Belmont, MA.
4. Booker, LB., Hota, N., 1988. Probabilistic reasoning about ship images. Uncertainty in Artificial Intelligence 2: 371-379.
5. Boutilier, C., Poole, D., 1996. Computing optimal policies for partially observable decision processes using compact representations. In Proceedings of the Thirteenth National Conference on Artificial Intelligence. AAAI Press/MIT Press; 1168-1175.
6. Buede, DM., 1999. Dynamic decision networks: an approach for solving the dual control problem. Paper presented at Spring INFORMS, Cincinnati, OH.
7. Buede, DM., Costa, PCG., 1999. A graph theoretic architecture for dual control: decision making in multisensor systems. Paper presented at Military Operation Research Society Symposium, West Point, NY.
8. Buede, DM., Ferrel, DO., 1993. Convergence in problem solving: a prelude to quantitative analysis. IEEE Transactions on Systems, Man, and Cybernetics 23: 746-765.
9. Buede, DM., Girardi, P., 1997. A target identification comparison of Bayesian and Dempster-Shafer multisensor fusion. IEEE Transactions on Systems, Man, and Cybernetics 27: 569-577.
10. Cassandra, AR., Kaelbling, LP., Litmann, ML., 1994. Acting optimically in partially observable stochastic domains. In Proceedings of the Twelfth National Conference on Artificial Intelligence; 1023-1028.
11. Cassandra, AR., Littman, ML., Zhang, NL., 1997. Incremental pruning: a simple, fast, exact method for partially observable Markov decision processes. In Proceedings of Thirteenth Conference on Uncertainty in Artificial Intelligence. AAAI Press/MIT Press; 54-61.
12. Charniak, E., 1991. Bayesian networks without fears. AI Magazine 12 (4): 50-63.
13. Chrisman, L., 1992. Reinforcement learning with perceptual aliasing: the perceptual distinctions approach. In Proceedings of the Tenth National Conference on Artificial Intelligence. AAAI Press: San Jose, CA; 183-188.
14. 3I, IT&E, George Mason University.
15. French, S., 1986. Decision Theory: An Introduction to the Mathematics of Rationality. John Wiley & Sons: Chichester, UK.
16. Hansen, EA., 1997. An improved polity iteration algorithm for partially observable MDPs. In Advances in Neural Information Processing Systems, vol. 10, MIT Press: Cambridge, MA.
17. Hansen, EA., 1998a. Finite-memory control of partially observable systems. PhD thesis, Department of Computer Science, University of Massachusetts at Amherst.
18. Hansen, EA., 1998b. Solving POMDPs by Searching in Policy Space. In Proceedings of the Eighth Conference on Uncertainty in Artificial Intelligence. Madison, WI; 211-219.
19. Hansom, O., Mayer, A., 1989. Heuristic search as evidential reasoning. In Proceedings of the Fifth Workshop on Uncertainty in Artificial Intelligence. Mountain View, CA.
20. Hilton, RW., 1981. The determinants of information value: synthesizing some general results. Management Science 27 (1): 57-64.
21. Howard, RA., 1960. Dynamic Programming and Markov Processes. MIT Press: Cambridge, MA.
22. Howard, RA., Matheson, JE., (eds). 1981. Influence diagrams. In The Principles and Applications of Decision Analysis, vol. II, Strategic Decisions Group: Menlo Park, CA; 719-762.
23. Howard, RA., 1971. Dynamic Probabilistic Systems, vol. I, John Wiley & Sons: New York.
24. Howard, RA., 1998. Transcript of Panel discussion: downstream decisions (options) and dynamic modeling, at 1998 Seattle Informs Meeting, Session SE01-Sunday, October 25th, 16:30-18:00, Seattle, WA.
25. Jensen, FV., 1996. An Introduction to Bayesian Networks. Springer: New York, NY.
26. Keeney, RL., Raiffa, H., 1976. Decisions with Multiple Objectives. John Wiley & Sons: New York, NY.
27. Keeney, RL., 1992. Value-Focused Thinking: A Path to Creative Decisionmaking. Harvard University Press: Cambridge, MA.
28. Keynes, JM., 1957. A Treatise on Probability. Macmillan: London, UK (originally published in 1921).
29. La Valle, I., 1968a. On cash equivalents and information evaluation in decisions under uncertainty: Part I. basic theory. Journal of the American Statistical Association 63: 252-276.
30. La Valle, I., 1968b. On cash equivalents and information evaluation in decisions under uncertainty: Part II. Incremental information decisions. Journal of the American Statistical Association 63: 277-284.
31. La Valle, I., 1968c. On cash equivalents and information evaluation in decisions under uncertainty: Part III. Exchanging partition-J for partition-K information. Journal of the American Statistical Association 63: 285-290.
32. Laplace, PS., 1951. A Philosophical Essay and Probabilities. Dover: New York, NY (originally published in 1820).
33. Matheson, JE., 1990. Using influence diagrams to value information and control. In Influence Diagrams, Belief Nets, and Decision Analysis, Oliver RM, Smith JQ (eds). John Wiley & Sons: Chichester, UK; 25-48.
34. Merkhofer, ML., 1988. Using influence diagrams in multiattribute utility analysis-improving effectiveness through improving communication. In Proceedings of the Conference on the Influence Diagrams for Decision Analysis, Inference and Prediction.
35. Miller, AC., Merkhofer, MM., Howard, RA., Matheson, JE., Rice, TR., 1976. Development of Automated Aids for Decision Analysis. Stanford Research Institute: Menlo Park, CA.
36. Monahan, GE., 1982. A survey of partially observable Markov decision processes: theory, models, and algorithms. Management Science 28 (1): 1-16.
37. Neapolitan, RE., 1990. Probabilistic Reasoning in Expert Systems: Theory and Algorithms. John Wiley & Sons: New York, NY.
38. Oliver, RM., Smith, JQ., 1990. Influence Diagrams, Belief Nets, and Decision Analysis. John Wiley & Sons: Chichester, UK.
39. Olmsted, SM., 1983. On representing and solving decision problems. PhD thesis, EES Department, Stanford University, Stanford, CA.
40. Pearl, J., 1988. Probabilistic Reasoning in Expert Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers: San Francisco, CA.
41. Putterman, ML., 1990. Markov decision processes. In Handbooks in OR & MS, vol. 2, Heyman DP, Sobel MJ (eds). Elsevier Science Publishers: Amsterdam; 331-434.
42. Putterman, ML., 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons: New York.
43. Savage, LJ., 1954. Foundations of Statistics. John Wiley & Sons: New York.
44. Savage, LJ., 1961. The foundations of statistics reconsidered. In Proceedings of the Fourth Berkeley Symposium on Mathematics and Probability, reprinted in Readings in Uncertain Reasoning, Shafer, Pearl (eds). Morgan Kaufmann: Palo Alto, CA.
45. Sawaki, K., Ichikawa, A., 1978. Optimal control for partially observable Markov decision processes over an infinite horizon. Journal of the Operations Research Society of Japan 21 (1): 1-14.
46. Shachter, RD., 1986. Evaluating influence diagrams. Operations Research 34: 871-882.
47. Shachter, RD., 1988. Probabilistic inference and influence diagrams. Operations Research 36: 589-604.
48. Shachter, RD., 1990. An ordered examination of influence diagrams. Networks 20: 535-563.
49. Smallwood, RD., Sondik, EJ., 1973. The optimal control of partially observable Markov decision processes over a finite horizon. Operations Research 21: 1071-1088.
50. Sondik, EJ., 1971. The optimal control of partially observable Markov decision processes. PhD dissertation, Stanford University, Stanford, CA.
51. Sondik, EJ., 1978. The optimal control of partially observable Markov decision processes over the infinite horizon. Operations Research 26.
52. Spiegelhalter, DJ., Franklin, R., Bull, K., 1989. Assessment criticism and improvement of imprecise probabilities for a medical expert system. In Proceedings of the Fifth Workshop on Uncertainty in Artificial Intelligence. Mountain View, CA; 335-342.
53. Tatman, JA., Shachter, RD., 1990. Dynamic programming and influence diagrams. IEEE Transactions on Systems, Man, and Cybernetics 20.
54. Von Newmann, J., Morgenstern, O., 1944. The Theory of Games and Economic Behavior. Princeton University Press: Princeton, NJ.
55. Watson, SR., Buede, DM., 1987. Decision Synthesis. Cambridge University Press: New York.
56. Weatherford, R., 1982. Philosophical Foundations of Probability Theory. Routledge & Kegan Paul: London.
57. Welman, MP., Heckerman, D., 1995. Real-world applications of Bayesian networks. Communications of the ACM 8: 24-30.
58. White III, CC., Scherer, WT., 1989. Solution procedures for partially observable Markov decision processes. Operations Research 37: 791-797.
59. White, DJ., 1992. Markov Decision Processes. John Wiley & Sons: New York.
60. Zang, NL., Liu, W., 1996. Planning in stochastic domains: problem characteristics and approximation. Technical Report HKUST-CS96-31, Department of Computer Science, Hong Kong University of Science and Technology.
61. Ziemba, WT., Butterworth, JE., 1975. Bounds on the value of information in uncertain decision problems. Stochastics 1: 361-378.