Solving POMDPs with continuous or large discrete observation spaces

IJCAI International Joint Conference on Artificial Intelligence

Volumn , Issue , 2005, Pages 1332-1338

  Hoey, Jesse ^a   Poupart, Pascal ^b  

^a UNIVERSITY OF TORONTO   (Canada)

^b UNIVERSITY OF WATERLOO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ASSISTED LIVING; DECISION PROBLEMS; DISCRETE OBSERVATIONS; DISCRETISATION; LOSSLESS; OBSERVATION SPACE; PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; RELEVANT FEATURES;

ARTIFICIAL INTELLIGENCE;

ALGORITHMS;

EID: 84880741298     PISSN: 10450823     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. K. J. Åström. Optimal control of Markov decision processes with incomplete state estimation. Journal of Mathematical Analysis and Applications, 10:174-205, 1965.
2. D. Aberdeen and J. Baxter. Scaling internal-state policy-gradient methods for POMDPs. In ICML, pages 3-10, Sydney, Australia, 2002.
3. D. P. Bertsekas and J. N. Tsitsiklis. Neuro-dynamic programming. Athena Scientific, Belmont, MA, 1996.
4. J. Boger, P. Poupart, J. Hoey, C. Boutilier, G. Fernie, and Alex Mihailidis. A decision-theoretic approach to task assistance for persons with dementia. In Proc. IJCAI, Edinburgh, 2005.
5. A. R. Cassandra, L. P. Kaelbling, and M. L. Littman. Acting optimally in partially observable stochastic domains. In AAAI, Seattle, WA, 1994.
6. H.-T. Cheng. Algorithms for Partially Observable Markov Decision Processes. PhD thesis, University of British Columbia, Vancouver, 1988.
7. T. Darrell and A. P. Pentland. Active gesture recognition using partially observable Markov decision processes. In IEEE Intl. Conf. on Pattern Recognition, Vienna, Austria, 1996.
8. Z. Feng and E. Hansen. Approximate planning for factored POMDPs. In Proc. ECP, Toledo, Spain, 2001.
9. W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13-30, 1963.
10. Jesse Hoey, Pascal Poupart, Craig Boutilier, and Alex Mihailidis. Semi-supervised learning of patient-caregiver interactions using partially observable Markov decision processes. Working paper, 2005.
11. M. Spaan J. Porta and N. Vlassis. Value iteration for continuous-state POMDPs. Technical Report IAS-UVA-04-04, Informatics Institute, University of Amsterdam, 2004.
12. Leslie Pack Kaelbling, Michael Littman, and Anthony R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101:99-134, 1998.
13. N. Meuleau, L. Peshkin, K.-E. Kim, and L. P. Kaelbling. Learning finite-state controllers for partially observable environments. In UAI, pages 427-436, Stockholm, 1999.
14. A. Mihailidis, B. Carmichael, and J. Boger. The use of computer vision in an intelligent environment to support aging-inplace, safety, and independence in the home. IEEE Trans. on Information Technology in Biomedicine (Spec. Issue on Pervasive Healthcare), 8(3):1-11, 2004.
15. M. Montemerlo, J. Pineau, N. Roy, S. Thrun, and V. Verma. Experiences with a mobile robotic guide for the elderly. In AAAI, pages 587-592, Edmonton, AB, 2002.
16. A. Y. Ng and M. Jordan. PEGASUS: A policy search method for large MDPs and POMDPs. In UAI, pages 406-415, Stanford, CA, 2000.
17. J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: an anytime algorithm for POMDPs. In IJCAI, Acapulco, Mexico, 2003.
18. P. Poupart and C. Boutilier. Bounded finite state controllers. In NIPS, Vancouver, BC, 2003.
19. P. Poupart and C. Boutilier. VDCBPI: an approximate scalable algorithm for large POMDPs. In NIPS, Vancouver, BC, 2004.
20. N. Roy and G. Gordon. Exponential family PCA for belief compression in POMDPs. In NIPS, pages 1635-1642, Vancouver, BC, 2002.
21. N. Roy, J. Pineau, and S. Thrun. Spoken dialog management using probabilistic reasoning. In ACL, Hong Kong, 2000.
22. R. Smallwood and E. Sondik. The optimal control of partially observable Markov processes over a finite horizon. Operations Research, 21:1071-1088, 1973.
23. S. Thrun. Monte Carlo POMDPs. In NIPS, pages 1064-1070, Denver, 1999.
24. N. Vlassis and M. T. J. Spaan. A fast point-based algorithm for POMDPs. In Proc. Belgian-Dutch Conference on Machine Learning, Brussels, Belgium, 2004.
25. J. Williams, P. Poupart, and S. Young. Using factored Markov decision processes with continuous observations for dialogue management. Technical Report CUED/F-INFEG/TR.520, Cambridge University, Engineering Department, 2005.
26. N. Zhang and W. Liu. Planning in stochastic domains: Problem characteristics and approximation. Technical Report HKUST-CS96-31, Hong Kong University of Science and Technology, 1996.
27. N. Zhang and W. Zhang. Speeding up the convergence of value-iteration in partially observable Markov decision processes. Journal of Artificial Intelligence Research, 14:29-51, 2001. (Pubitemid 33738058)