A tutorial on partially observable Markov decision processes

Journal of Mathematical Psychology

Volumn 53, Issue 3, 2009, Pages 119-125

  Littman, Michael L ^a  

^a RUTGERS UNIVERSITY   (United States)

Author keywords

Markov processes; Planning under uncertainty; POMDP

Indexed keywords

EID: 67349193210     PISSN: 00222496     EISSN: 10960880     Source Type: Journal    
DOI: 10.1016/j.jmp.2009.01.005     Document Type: Article

References (28)

1. Aström K.J. Optimal control of Markov decision processes with incomplete state estimation. Journal of Mathematical Analysis and Applications 10 (1965) 174-205
2. Bellman R. Dynamic programming (1957), Princeton University Press, Princeton, NJ
3. Bernstein, D.S., Zilberstein, S., & Immerman, N. (2000).The complexity of decentralized control of markov decision processes. In Proceedings of the sixteenth conference on uncertainty in artificial intelligence, UAI (pp. 32-37)
4. Bernstein, D.S., Hansen, E.A., & Zilberstein, S. (2005). Bounded policy iteration for decentralized POMDPs. In Nineteenth international joint conference on artificial intelligence, IJCAI-05
5. Boger, J., Poupart, P., Hoey, J., Boutilier, C., Fernie, G., & Mihailidis, A. (2005). A decision-theoretic approach to task assistance for persons with dementia. In Proceedings of the international joint conference on artificial intelligence, IJCAI (pp. 1293-1299)
6. Cassandra A., Littman M.L., and Zhang N.L. Incremental pruning: A simple, fast, exact method for partially observable Markov decision processes. Proceedings of the thirteenth annual conference on uncertainty in artificial intelligence. UAI-97 (1997), Morgan Kaufmann Publishers, San Francisco, CA 54-61
7. Cassandra, A.R. (1998). Exact and approximate algorithms for partially observable markov decision problems. Ph.D. thesis. Department of Computer Science, Brown University
8. Hansen E.A. An improved policy iteration algorithm for partially observable MDPs. Advances in Neural Information Processing Systems 10 (1997)
9. Kaelbling L.P., Littman M.L., and Cassandra A.R. Planning and acting in partially observable stochastic domains. Artificial Intelligence 101 1-2 (1998) 99-134
10. Littman M.L., Sutton R.S., and Singh S. Predictive representations of state. Advances in Neural Information Processing Systems 14 (2002) 1555-1561
11. Lovejoy W.S. A survey of algorithmic methods for partially observable Markov decision processes. Annals of Operations Research 28 1 (1991) 47-65
12. Madani O., Hanks S., and Condon A. On the undecidability of probabilistic planning and infinite-horizon partially observable Markov decision problems. Proceedings of the sixteenth national conference on artificial intelligence (1999), The AAAI Press/The MIT Press 541-548
13. Monahan G.E. A survey of partially observable Markov decision processes: Theory, models, and algorithms. Management Science 28 1 (1982) 1-16
14. Murphy K.P. Neural Information Processing Systems (1999)
15. Pineau, J., Gordon, G., & Thrun, S. (2003). Point-based value iteration: An anytime algorithm for POMDPs. In International joint conference on artificial intelligence, IJCAI (pp. 1025-1032)
16. Poupart P., and Boutilier C. Bounded finite state controllers. Advances in Neural Information Processing Systems (NIPS) 16 (2003)
17. Puterman M.L. Markov decision processes-discrete stochastic dynamic programming (1994), John Wiley & Sons, Inc., New York, NY
18. Regan, K., Cohen, R., & Poupart, P. (2005).The advisor-POMDP: A principled approach to trust through reputation in electronic markets. In Proceedings of the third annual conference on privacy, security and trust, PST (pp.121-130)
19. Roy, N., Pineau, J., & Thrun, S. (2000).Spoken dialogue management using probabilistic reasoning. In Proceedings of the 38th annual meeting of the association for computational linguistics, ACL2000
20. Simmons, R., & Koenig, S. (1995).Probabilistic robot navigation in partially observable environments. In Proceedings of the international joint conference on artificial intelligence (pp. 1080-1087)
21. Singh, S., James, M.R., & Rudary, M.R. (2004). Predictive state representations: A new theory for modeling dynamical systems. In Uncertainty in artificial intelligence: Proceedings of the twentieth conference, UAI (pp. 512-519)
22. Smallwood R.D., and Sondik E.J. The optimal control of partially observable Markov processes over a finite horizon. Operations Research 21 (1973) 1071-1088
23. Sondik, E. (1971).The optimal control of partially observable markov processes. Ph.D. thesis. Stanford University
24. Sondik E.J. The optimal control of partially observable Markov processes over the infinite horizon: Discounted costs. Operations Research 26 2 (1978) 282-304
25. Spaan M.T.J., and Vlassis N. Perseus: Randomized point-based value iteration for POMDPs. Journal of Artificial Intelligence Research 24 (2005) 195-220
26. White III C.C., and Scherer W.T. Solution procedures for partially observed Markov decision processes. Operations Research 37 5 (1989) 791-797
27. White III C.C. Partially observed Markov decision processes: A survey. Annals of Operations Research 32 (1991)
28. Zilberstein S., Washington R., Bernstein D.S., and Mouaddib A. Decision-theoretic control of planetary rovers. In: Beetz M., et al. (Ed). Plan-based control of robotic agents. LNAI Vol. 2466 (2002) 270-289