Finding approximate POMDP solutions through belief compression

Journal of Artificial Intelligence Research

Volumn 23, Issue , 2005, Pages 1-40

  Roy, Nicholas ^a   Gordon, Geoffrey ^b   Thrun, Sebastian ^c  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b Carnegie Mellon University   (United States)

^c Stanford University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA COMPRESSION; FEATURE EXTRACTION; MATHEMATICAL MODELS; MOBILE ROBOTS; OPTIMAL CONTROL SYSTEMS; PRINCIPAL COMPONENT ANALYSIS; PROBLEM SOLVING; PUBLIC POLICY; ROBOT APPLICATIONS;

EXPONENTIAL FAMILY PRINCIPAL COMPONENTS ANALYSIS; OPTIMAL POLICY; PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES (POMDPS); ROBOT NAVIGATION PLANNING;

ARTIFICIAL INTELLIGENCE;

EID: 27344443125     PISSN: 10769757     EISSN: 10769757     Source Type: Journal    
DOI: 10.1613/jair.1496     Document Type: Article

References (50)

1. Bagnell, J. A., & Schneider, J. (2001). Autonomous helicopter control using reinforcement learning policy search methods. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1615-1620, Seoul, South Korea. IEEE Press.
2. Bishop, C., Svensén, M., & Williams, C. (1998). GTM: the generative topographic mapping. Neural Computation, 10(1), 215-234.
3. Boutilier, C., & Poole, D. (1996). Computing optimal policies for partially observable Markov decision processes using compact representations. In Proceedings of the 13th National Conference on Artificial Intelligence (AAAI-96), pp. 1168-1175.
4. Boyen, X., & Koller, D. (1998). Tractable inference for complex stochastic processes. In Proceedings of the 14th Annual Conference on Uncertainty in AI (UAI), pp. 33-42, Madison, Wisconsin.
5. Brafman, R. I. (1997). A heuristic variable grid solution method for POMDPs. In Kuipers, B. K., & Webber, B. (Eds.), Proceedings of the 14th National Conference on Artificial Intelligence (AAAI), pp. 727-733, Providence, RI.
6. Cassandra, A. R., Kaelbling, L., & Kurien, J. A. (1996). Acting under uncertainty: Discrete Bayesian models for mobile-robot navigation. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems.
7. Chen, B. M. (2000). Robust and H-∞ Control. Springer-Verlag.
8. Cheng, H.-T. (1988). Algorithms for Partially Observable Markov Decision Processes. Ph.D. thesis, University of British Columbia, Vancouver, Canada.
9. Collins, M., Dasgupta, S., & Schapire, R. (2002). A generalization of principal components analysis to the exponential family. In Dietterich, T. G., Becker, S., & Ghahramani, Z. (Eds.), Advances in Neural Information Processing Systems 14 (NIPS), Cambridge, MA. MIT Press.
10. Cox, T., & Cox, M. (1994). Multidimensional Scaling. Chapman & Hall, London.
11. Fox, D., Burgard, W., & Thrun, S. (1999). Markov localization for mobile robots in dynamic environments. Journal of Artificial Intelligence Research, 11, 391-427.
12. Galassi, M., Davies, J., Theiler, J., Gough, B., Jungman, G., Booth, M., & Rossi, F. (2002). GNU Scientific Library Reference Manual (3rd Edition edition). http://www.gnu.org/software/gsl/.
13. Golub, G., & Reinsch, C. (1970). Singular value decomposition and least squares solutions. Numerische Mathematik, pp. 403-420.
14. Gordon, G. (1995). Stable function approximation in dynamic programming. In Prieditis, A., & Russell, S. (Eds.), Proceedings of the 12 International Conference on Machine Learning (ICML), pp. 261-268, San Francisco, CA. Morgan Kaufmann.
15. 2 models. In Becker, S., Thrun, S., & Obermayer, K. (Eds.), Advances in Neural Information Processing Systems 15 (NIPS). MIT Press.
16. Gutmann, J.-S., Burgard, W., Fox, D., & Konolige, K. (1998). An experimental comparison of localization methods. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Victoria, Canada.
17. Gutmann, J.-S., & Fox, D. (2002). An experimental comparison of localization methods continued. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Lausanne, Switzerland.
18. Hansen, E., & Feng, Z. (2000). Dynamic programming for POMDPs using a factored state representation. In Proceedings of the Fifth International Conference on Artificial Intelligence Planning and Scheduling (AIPS-00), Breckenridge, CO.
19. Hansen, E. (1998). Solving POMDPs by searching in policy space. In Proceedings of the 14th Conference on Uncertainty in Artifical Intelligence (UAI), pp. 211-219, Madison, WI.
20. Hauskrecht, M. (2000). Value-function approximations for partially observable Markov decision processes. Journal of Artificial Intelligence Research, 13, 33-94.
21. Hinton, G., & Roweis, S. (2003). Stochastic neighbor embedding. In Becker, S., Thrun, S., & Obermayer, K. (Eds.), Advances in Neural Information Processing Systems 15 (NIPS). MIT Press.
22. Howard, R. A. (1960). Dynamic Programming and Markov Processes. MIT.
23. Isard, M., & Blake, A. (1998). CONDENSATION - conditional density propagation for visual tracking. International Journal of Computer Vision, 29(1), 5-28.
24. Joliffe, I. T. (1986). Principal Component Analysis. Springer-Verlag.
25. Kanazawa, K., Koller, D., & Russell, S. (1995). Stochastic simulation algorithms for dynamic probabilistic networks. In Proceedings of the 11th Annual Conference on Uncertainty in AI (UAI), pp. 346-351, Montreal, Canada.
26. Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics, 48, 59-69.
27. Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401, 788-791.
28. Leonard, J., & Durrant-Whyte, H. (1991). Mobile robot localization by tracking geometric beacons. IEEE Transactions on Robotics and Automation, 7(3), 376-382.
29. Lovejoy, W. S. (1991). Computationally feasible bounds for partially observable Markov decison processes. Operations Research, 39, 192-175.
30. Mardia, K. V., & Jupp, P. E. (2000). Directional Statistics (2nd edition). Wiley, Chichester, NY.
31. McCullagh, P., & Nelder, J. A. (1983). Generalized Linear Models (2nd edition). Chapman and Hall, London.
32. Meuleau, N., Peshkin, L., Kim, K.-E., & Kaelbling, L. P. (1999). Learning finite-state controllers for partially observable environments. In Laskey, K. B., & Prade, H. (Eds.), Proceedings of the Fifteenth International Conference on Uncertainty in Artificial Intelligence, pp. 427-436, Stockholm, Sweden. Morgan Kaufmann.
33. Munos, R., & Moore, A. (1999). Variable resolution discretization for high-accuracy solutions of optimal control problems. In Dean, T. (Ed.), Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI), pp. 1348-1355, Stockholm Sweden. Morgan Kaufmann.
34. Munos, R., & Moore, A. (2002). Variable resolution discretization in optimal control. Machine Learning, 49(2-3), 291-323.
35. Nourbakhsh, I., Powers, R., & Birchfield, S. (1995). DERVISH an office-navigating robot. AI Magazine, 16(2), 53-60.
36. Olson, C. F. (2000). Probabilistic self-localization for mobile robots. IEEE Transactions on Robotics and Automation, 16(1), 55-66.
37. Pineau, J., Gordon, G., & Thrun, S. (2003a). Point-based value iteration: An anytime algorithm for POMDPs. In Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI 2003), Acapulco, Mexico.
38. Pineau, J., Gordon, G., & Thrun, S. (2003b). Policy-contingent abstraction for robust robot control. In Meek, C., & Kjælruff, U. (Eds.), Proceedings of the 19th Annual Conference on Uncertainty in Artificial Intelligence (UAI), Acapulco, Mexico.
39. Poupart, P., & Boutilier, C. (2002). Value-directed compression of POMDPs. In Becker, S., Thrun, S., & Obermayer, K. (Eds.), Advances in Neural Information Processing Systems 15 (NIPS), Vancouver, Canada. MIT Press.
40. Rockafellar, R. T. (1970). Convex Analysis. Princeton University Press, New Jersey.
41. Roweis, S., & Saul, L. (2000). Nonlinear dimensionality reduction by locally linear embedding.. Science, 290(5500), 2323-2326.
42. Roweis, S. T., Saul, L. K., & Hinton, G. E. (2002). Global coordination of local linear models. In Dietterich, T. G., Becker, S., & Ghahramani, Z. (Eds.), Advances in Neural Information Processing Systems, Vol. 14, Cambridge, MA. MIT Press.
43. Roy, N., & Thrun, S. (1999). Coastal navigation with mobile robots. In Solla, S. A., todd K. Leen, & Müller, K. R. (Eds.), Advances in Neural Processing Systems 12 (NIPS), pp. 1043-1049, Denver, CO. MIT Press.
44. Russell, S., & Norvig, P. (1995). Artificial Intelligence: A Modern Approach. Prentice Hall.
45. Shatkay, H., & Kaelbling, L. P. (2002). Learning geometrically- constrained hidden markov models for robot navigation: Bridging the geometrical-topological gap. Journal of AI Research.
46. Tenenbaum, J. B., de Silva, V., & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500), 2319-2323.
47. Thrun, S., Fox, D., Burgard, W., & Dellaert, F. (2000). Robust Monte Carlo localization for mobile robots. Artificial Intelligence, 128(1-2), 99-141.
48. Washington, R. (1997). BI-POMDP: Bounded, incremental partially-observable Markov-model planning. In Proceedings of the 4th European Conference on Planning (ECP).
49. Zhang, N. L., & Zhang, W. (2001). Speeding up the convergence of value iteration in partially observable Markov decision processes. Journal of Artificial Intelligence Research, 14, 1-28.
50. Zhou, R., & Hansen, E. (2001). An improved grid-based approximation algorithm for POMDPs. In Nebel, B. (Ed.), Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI), pp. 707-716, Seattle, Washington. Morgan Kaufmann.