Partially observable Markov decision processes with imprecise parameters

Artificial Intelligence

Volumn 171, Issue 8-9, 2007, Pages 453-490

  Itoh, Hideaki ^a   Nakamura, Kiyohiko ^a  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

Parameter set; POMDP; Probability interval; Second order beliefs

Indexed keywords

ALGORITHMS; COMPUTATIONAL METHODS; COST EFFECTIVENESS; DECISION THEORY; OPTIMIZATION; PARAMETER ESTIMATION; PROBABILITY DISTRIBUTIONS;

PARAMETER SETS; PROBABILITY INTERVALS; SECOND-ORDER BELIEFS;

MARKOV PROCESSES;

EID: 34249672336     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.artint.2007.03.004     Document Type: Article

References (59)

1. D. Aberdeen, J. Baxter, Scaling internal-state policy-gradient methods for POMDPs, in: International Conference on Machine Learning (ICML-02), Sydney, Australia, July 2002, pp. 1-12
2. Aström K.J. Optimal control of Markov decision processes with incomplete state estimation. Journal of Mathematical Analysis and Applications 10 (1965) 174-205
3. T. Augustin, On the suboptimality of the generalized Bayes rule and robust Bayesian procedures from the decision theoretic point of view-a cautionary note on updating imprecise priors, in: Proceedings of 3rd International Symposium on Imprecise Probabilities and their Applications (ISIPTA-03), 2003
4. Bellman R. Dynamic Programming (1957), Princeton Univ. Press, Princeton, NJ
5. In: Bernard J.M., Seidenfeld T., and Zaffalon M. (Eds). Proceedings of the Third International Symposium in Imprecise Probabilities and its Applications (2003), Carleton Scientific
6. Bertsekas D.P. Dynamic Programming and Optimal Control, vol. 2. second ed. (2001), Athena Scientific, Belmont, MA
7. Bonet B. An epsilon-optimal grid-based algorithm for partially observable Markov decision processes. Proc. 19th International Conf. on Machine Learning (ICML-02) (2002), Morgan Kaufmann 51-58
8. Boutilier C., Dean T., and Hanks S. Decision-theoretic planning: Structural assumptions and computational leverage. Journal of Artificial Intelligence Research 11 (1999) 1-94
9. Boutilier C., and Poole D. Computing optimal policies for partially observable decision processes using compact representations. Proceedings of the Thirteenth National Conference on Artificial Intelligence (AAAI-96). Portland, OR (1996), AAAI Press/The MIT Press 1168-1175
10. Breese J., and Fertig K. Decision making with interval influence diagrams. Proceedings of the 6th Annual Conference on Uncertainty in Artificial Intelligence (UAI-91). New York (1991), Elsevier Science 467-478
11. Cassandra A., Littman M.L., and Zhang N.L. Incremental Pruning: A simple, fast, exact method for partially observable Markov decision processes. In: Geiger D., and Shenoy P.P. (Eds). Proceedings of the Thirteenth Annual Conference on Uncertainty in Artificial Intelligence (UAI-97). San Francisco, CA (1997), Morgan Kaufmann 54-61
12. Chrisman L. Independence with lower and upper probabilities. Proceedings of the 12th Annual Conference on Uncertainty in Artificial Intelligence (UAI-96). San Francisco, CA (1996), Morgan Kaufmann 169-177
13. Cozman F.G. Credal networks. Artificial Intelligence 120 (2000) 199-233
14. Cozman F.G., and Krotkov E. Quasi-Bayesian strategies for efficient plan generation: application to the 'planning to observe' problem. In: Horvitz E., and Jensen F.V. (Eds). Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence (UAI-96). San Francisco, CA (1996), Morgan Kaufmann 186-193
15. A. Drake, Observation of a Markov process through a noisy channel, PhD thesis, Massachusetts Institute of Technology, 1962
16. Z. Feng, E.A. Hansen, Approximate planning for factored POMDPs, in: Proceedings of the 6th European Conference on Planning (ECP-01), Toledo, Spain, September 2001
17. Fertig K., and Breese J. Interval influence diagrams. Proceedings of the 5th Annual Conference on Uncertainty in Artificial Intelligence (UAI-90) (1990), New York, Elsevier Science 149-161
18. Fertig K.W., and Breese J.S. Probability intervals over influence diagrams. IEEE Transactions on Pattern Analysis and Machine Intelligence 15 3 (1993) 280-286
19. Gaifman H. A theory of higher order probabilities. Proceedings of the 1986 Conference on Theoretical Aspects of Reasoning about Knowledge (1986), Morgan Kaufmann 275-292
20. Givan R., Leach S.M., and Dean T. Bounded-parameter Markov decision processes. Artificial Intelligence 122 1-2 (2000) 71-109
21. Good I.J. Good Thinking: The Foundations of Probability and its Applications (1983), University of Minnesota Press, Minneapolis
22. Grove A.J., and Halpern J.Y. Updating sets of probabilities. Proceedings of the 14th Annual Conference on Uncertainty in Artificial Intelligence (UAI-98). San Francisco, CA (1998), Morgan Kaufmann 173-182
23. Ha V., and Haddawy P. Theoretical foundations for abstraction-based probabilistic planning. Proceedings of the 12th Annual Conference on Uncertainty in Artificial Intelligence (UAI-96). San Francisco, CA (1996), Morgan Kaufmann 291-298
24. E.A. Hansen, Solving POMDPs by searching in policy space, in: Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence (UAI-98), 1998, pp. 211-219
25. E.A. Hansen, Z. Feng, Dynamic programming for POMDPs using a factored state representation, in: Artificial Intelligence Planning Systems (AIPS-00), 2000, pp. 130-139
26. E.A. Hansen, R. Zhou, Synthesis of hierarchical finite-state controllers for POMDPs, in: Thirteenth International Conference on Automated Planning and Scheduling (ICAPS-03), June 2003
27. Harmanec D. Generalizing Markov decision processes to imprecise probabilities. Journal of Statistical Planning and Inference 105 (2002) 199-213
28. Hauskrecht M. Value-function approximations for partially observable Markov decision processes. Journal of Artificial Intelligence Research 13 (2000) 33-94
29. Hauskrecht M., and Fraser H. Planning treatment of ischemic heart disease with partially observable Markov decision processes. Artificial Intelligence in Medicine 18 (2000) 221-244
30. Kaelbling L.P., Littman M.L., and Cassandra A.R. Planning and acting in partially observable stochastic domains. Artificial Intelligence 101 (1999) 99-134
31. Karmarkar N. A new polynomial-time algorithm for linear programming. Combinatorica 4 (1984) 373-395
32. Lehner P.E., Laskey K.B., and Dubois D. An introduction to issues in higher order uncertainty. IEEE Transactions on Systems, Man and Cybernetics, Part A 26 3 (1996) 289-293
33. Levi I. On indeterminate probabilities. Journal of Philosophy 71 (1974) 391-418
34. Levi I. The Enterprise of Knowledge (1980), MIT Press, Cambridge, MA
35. Lovejoy W.S. A survey of algorithmic methods for partially observed Markov decision processes. Annals of Operations Research 28 (1991) 47-66
36. Lusena C., Goldsmith J., and Mundhenk M. Nonapproximability results for partially observable Markov decision processes. Journal of Artificial Intelligence Research 14 (2001) 83-103
37. D.A. McAllester, S. Singh, Approximate planning for factored POMDPs using belief state simplification, in: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence (UAI-99), 1999, pp. 409-416
38. M. Montemerlo, J. Pineau, N. Roy, S. Thrun, V. Verma, Experiences with a mobile robotic guide for the elderly, in: Proceedings of the National Conference of Artificial Intelligence (AAAI-02), Edmonton, AB, July 2002, pp. 587-592
39. Nilim A., and El-Ghaoui L. Robustness in Markov decision problems with uncertain transition matrices. Advances in Neutral Information Processing Systems 16 (NIPS-03) (2004), MIT Press, Cambridge, MA
40. Nilim A., and El-Ghaoui L. Robust control of Markov decision processes with uncertain transition matrices. Operations Research 53 (2005) 780-798
41. Paaß G. Second order probabilities for uncertain and conflicting evidence. Proceedings of the 6th Annual Conference on Uncertainty in Artificial Intelligence (UAI-91). New York (1991), Elsevier Science 447-456
42. Papadimitriou C.H., and Tsitsiklis J.N. The complexity of Markov decision processes. Mathematics of Operations Research 12 3 (1987) 441-450
43. J. Pineau, Tractable planning under uncertainty: Exploiting structure, PhD thesis, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, 2004
44. Pineau J., Gordon G., and Thrun S. Point-based value iteration: An anytime algorithm for POMDPs. Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03) (2003), AAAI Press, Menlo Park, CA
45. P. Poupart, Exploiting structure to efficiently solve large scale partially observable Markov decision processes, PhD thesis, Department of Computer Science, University of Toronto, Toronto, Ontario, Canada, 2005
46. Poupart P., and Boutilier C. Bounded finite state controllers. Advances in Neural Information Processing Systems 16 (NIPS-03) (2004), MIT Press, Cambridge, MA
47. Poupart P., and Boutilier C. VDCBPI: An approximate scalable algorithm for large scale POMDPs. Advances in Neural Information Processing Systems 17 (NIPS-04) (2005), MIT Press, Cambridge, MA
48. Satia J.K., and Lave R.E. Markovian decision processes with uncertain transition probabilities. Operations Research 21 (1973) 728-740
49. Seidenfeld T., and Schervish M.J. Two perspectives on consensus for (Bayesian) inference and decisions. IEEE Transactions on Systems, Man and Cybernetics 20 2 (1990) 318-325
50. Shafer G. A Mathematical Theory of Evidence (1976), Princeton Univ. Press, Princeton, NJ
51. E.J. Sondik, The optimal control of partially observable Markov processes, PhD thesis, Stanford University, 1971
52. Spaan M.T.J., and Vlassis N. Perseus: Randomized point-based value iteration for POMDPs. Journal of Artificial Intelligence Research 24 (2005) 195-220
53. N. Vlassis, M.T.J. Spaan, A fast point-based algorithm for POMDPs, in: Benelearn 2004: Proceedings of the Annual Machine Learning Conference of Belgium and the Netherlands, Brussels, Belgium, 2004, pp. 170-176
54. Walley P. Statistical Reasoning with Imprecise Probabilities (1991), Chapman and Hall, London
55. White C.C., and Eldeib H.K. Parameter imprecision in finite state, finite action dynamic programs. Operations Research 34 (1986) 120-129
56. White C.C., and Eldeib H.K. Markov decision processes with imprecise transition probabilities. Operations Research 43 (1994) 739-749
57. Zhang N.L., and Zhang W. Speeding up the convergence of value iteration in partially observable Markov decision processes. Journal of Artificial Intelligence Research 14 (2001) 29-51
58. Zhang W., and Zhang N.L. Restricted value iteration: Theory and algorithms. Journal of Artificial Intelligence Research 23 (2005) 123-165
59. R. Zhou, E.A. Hansen, An improved grid-based approximation algorithm for POMDPs, in: Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI-01), 2001, pp. 707-716