Compact, convex upper bound iteration for approximate POMDP planning

Proceedings of the National Conference on Artificial Intelligence

Volumn 2, Issue , 2006, Pages 1245-1251

  Tao, Wang ^a   Poupart, Pascal ^b   Bowling, Michael ^a   Schuurmans, Dale ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

^b UNIVERSITY OF WATERLOO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION QUALITY; FUNCTION APPROXIMATOR; OPTIMAL VALUE FUNCTION; QUADRATICS;

APPROXIMATION THEORY; HEURISTIC METHODS; ITERATIVE METHODS; QUADRATIC PROGRAMMING; UNCERTAIN SYSTEMS;

MARKOV PROCESSES;

EID: 33750733906     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (33)

1. Amato, C.; Bernstein, D.; and Zilberstein, S. 2006. Solving POMDPs using quadratic ally constrained linear programs. In Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS).
2. Bertsekas, D. 1995. Dynamic Programming and Optimal Control, volume 2. Athena Scientific.
3. Boger, J.; Poupart, P.; Hoey, J.; Boutilier, C.; Fernie, G.; and Mihailidis, A. 2005, A decision-theoretic approach to task assistance for persons with dementia. In Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence (IJCAI).
4. Bonet, B. 2002. An e-optimal grid-based algorithm for partially observable Markov decision processes. In Proceedings of the Nineteenth International Conference on Machine Learning (ICML).
5. Boutilier, C., and Poole, D. 1996. Computing optimal policies for partially observable decision processes using compact representations. In Proceedings of the Thirteenth National Conference on Artificial Intelligence (AAAI).
6. Boyd, S., and Vandenberghe, L. 2004. Convex Optimization. Cambridge Univ. Press.
7. Cassandra, A.; Littman, M.; and Zhang, N. 1997. Incremental pruning: A simple, fast, exact method for partially observable Markov decision processes. In Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence (UAI).
8. Feng, Z., and Hansen, E. A. 2001. Approximate planning for factored POMDPs. In Proceedings of the Sixth European Conference on Planning.
9. Gelman, A.; Carlin, J.; Stern, H.; and Rubin, D. 1995. Bayesian Data Analysis. Chapman & Hall.
10. Gordon, G. 1995. Stable function approximation in dynamic programming. In Proceedings of the Twelfth International Conference on Machine Learning (ICML).
11. Hauskrecht, M. 1997. Incremental methods for computing bounds in partially observable markov decision processes. In Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI).
12. Hauskrecht, M. 2000. Value-function approximations for partially observable Markov decision processes. Journal of Artificial Intelligence Research 13:33-94.
13. Kearns, M.; Mansour, Y.; and Ng, A. 2002. A sparse sampling algorithm for near-optimal planning in large Markov decision processes. Machine Learning 49(2-3): 193-208.
14. Littman, M.; Cassandra, A.; and Kaelbling, L. 1995. Learning policies for partially observable environments: scaling up. In Proceedings of the Twelfth International Conference on Machine Learning (ICML).
15. Madani, O.; Hanks, S.; and Condon, A. 2003. On the undecidability of proabilistic planning and related stochastic optimization problems. Artificial Intelligence 147:5-34.
16. Mundhenk, M.; Goldsmith, J.; Lusena, C.; and Allender, E. 2000. Complexity of finite-horizon Markov decision processes. Journal of the Association for Computing Machinery 47(4):681-720.
17. Ng, A., and Jordan, M. 2000. Pegasus: A policy search method for large MDPs and POMDPs. In Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence (UAI).
18. Parr, R., and Russell, S. 1995. Approximating optimal policies for partially observable stochastic domains. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI).
19. Pineau, J.; Montemerlo, M.; Pollack, M.; Roy, N.; and Thrun, S. 2003. Towards robotic assistants in nursing homes: Challenges and results. Robotics and Autonomous Systems 42:271-281.
20. Pineau, J.; Gordon, G.; and Thrun, S. 2003. Point-based value iteration: An anytime algorithm for POMDPs. In Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI).
21. Poupart, P., and Boutilier, C. 2002. Value-directed compression of POMDPs. In Advances in Neural Information Processing Systems (NIPS 15).
22. Poupart, P., and Boutilier, C. 2003. Bounded finite state controllers. In Advances in Neural Information Processing Systems (NIPS 16).
23. Poupart, P., and Boutilier, C. 2004. VDCBPI: An approximate scalable algorithm for large POMDPs. In Advances in Neural Information Processing Systems (NIPS 17).
24. Poupart, P. 2005. Exploiting Structure to efficienty solve large scale partially observable Markov decision processes. Ph.D. Dissertation, Department of Computer Science, University of Toronto.
25. Roy, N.; Gordon, G.; and Thrun, S. 2005. Finding approximate POMDP solutions through belief compression. Journal of Artificial Intelligence Research 23:1-40.
26. Smith, T., and Simmons, R. 2005. Point-based POMDP algorithms: Improved analysis and implementation. In Pro-ceedings of the Twenty-first Conference on Uncertainty in Artificial Intelligence (UAI).
27. Sondik, E. 1978. The optimal control of partially observable Markov processes over the infinite horizon: Discounted costs. Operations Research 26:282-304.
28. Spaan, M., and Vlassis, N. 2005. Perseus: Randomized point-based value iteration for POMDPs. Journal of Artificial Intelligence Research 24:195-220.
29. Thrun, S.; Burgard, W.; and Fox, D. 2005. Probabilistic Robotics. MIT Press.
30. Thrun, S. 2000. Monte Carlo POMDPs. In Advances in Neural Information Processing Systems (NIPS 12).
31. Toh, K.; Todd, M.; and Tutuncu, R. 1999. SDPT3-a Matlab software package for semidefinite programming. Optimization Methods and Software 11.
32. Zhang, N., and Zhang, W. 2001. Speeding up the convergence of value iteration in partially observable Markov decision processes. Journal of Artificial Intelligence Research 14:29-51.
33. Zhou, R., and Hansen, E. 2001. An improved grid-based approximation algorithm for POMDPs. In Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI).