Speeding up the convergence of value iteration in partially observable Markov decision processes

Journal of Artificial Intelligence Research

Volumn 14, Issue , 2001, Pages 29-51

  Zhang, Nevin L ^a   Zhang, Weihong ^a  

^a HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS; ITERATIVE METHODS; MARKOV PROCESSES; MATHEMATICAL MODELS;

PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES (POMDP);

DECISION MAKING;

EID: 0036374229     PISSN: 10769757     EISSN: None     Source Type: Journal    
DOI: 10.1613/jair.761     Document Type: Article

References (30)

1. Aström, K. J. (1965). Optimal control of Markov decision processes with the incomplete state estimation. Journal of Computer and System Sciences, 10, 174-205.
2. Brafman, R. I. (1997). A heuristic variable grid solution for POMDPs. In Proceedings of the Fourteenth National Conference on Artificial Intelligence(AAAI-97), 727-733.
3. Cassandra, A. R., Littman, M. L., and Zhang, N. L. (1997). Incremental pruning: A simple, fast, exact method for partially observable Markov decision processes. In Proceedings of Thirteenth Conference on Uncertainty in Artificial Intelligence, 54-61.
4. Cassandra, A. R. (1998a). Exact and approximate algorithms for partially observable Markov decision processes, PhD thesis, Department of Computer Science, Brown University.
5. Cassandra, A. R. (1998b). A survey of POMDP applications, in Working Notes of AAAI 1998 Fall Symposium on Planning with Partially Observable Markov Decision Processes, 17-24.
6. Denardo, E. V. (1982). Dynamic Programming: Models and Applications Prentice-Hall.
7. Eagle, J. N. (1984). The optimal search for a moving target when the search path is constrained. Operations Research, 32(5), 1107-1115.
8. Cheng, H. T. (1988). Algorithms for partially observable Markov decision processes. Ph D thesis, University of British Columbia.
9. Hansen, E. A. (1998). Solving POMDPs by searching in policy space. In Proceedings of Fourteenth Conference on Uncertainty in Artificial Intelligence, 211-219.
10. Hauskrecht, M. (1997a). Incremental methods for computing bounds in partially observable Markov decision processes, in Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI-97), 734-749.
11. Hauskrecht, M. (1997b). Planning and control in stochastic domains with imperfect information. PhD thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology.
12. Hauskrecht, M. (2000). Value function approximations for partially observable Markov decision processes, Journal of Artificial Intelligence Research, 13, 33-95.
13. Littman, M. L., Cassandra, A. R. and Kaelbling, L. P. (1995a). Efficient dynamic-programming updates in partially observable Markov decision processes. Technical Report CS-95-19, Brown University.
14. Littman, M. L., Cassandra, A. R. and Kaelbling, L. P. (1995b). Learning policies for partially observable environments, scaling up. In Proceedings of the Fifteenth Conference on Machine Learning, 362-370.
15. Littman, M. L. (1996). Algorithms for sequential decision making. Ph D thesis, Department of Computer Science, Brown University.
16. Kaelbling, L. P., Littman. M. L. and Cassandra, A. R.(1998). Planning and acting in partially observable stochastic domains, Artificial Intelligence, Vol 101.
17. Lovejoy, W. S. (1991). Computationally feasible bounds for partially observed Markov decision processes. Operations Research, 39, 192-175.
18. Lovejoy, W. S. (1993). Suboptimal policies with bounds for parameter adaptive decision processes. Operations Research, 41, 583-599.
19. Monahan, G. E. (1982). A survey of partially observable Markov decision processes: theory, models, and algorithms. Management Science, 28 (1), 1-16.
20. Parr, R., and Russell, S. (1995). Approximating optimal policies for partially observable stochastic domains. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence 1088-1094.
21. Papadimitriou, C. H., Tsitsiklis, J. N.(1987). The complexity of Markov decision processes. Mathematics of Operations Research, 12(3), 441-450.
22. Platzman, L. K.(1980). Optimal infinite-horizon undiscounted control of finite probabilistic systems. SIAM Journal of Control and Optimization, 18, 362-380.
23. Puterman, M. L. (1990), Markov decision processes, in D. P. Heyman and M. J. Sobel (eds.), Handbooks in OR & MS., Vol. 2, 331-434, Elsevier Science Publishers.
24. Smallwood, R. D. and Sondik, E. J. (1973). The optimal control of partially observable processes over a finite horizon. Operations Research, 21, 1071-1088.
25. Sondik, E. J. (1971). The optimal control of partially observable Markov processes. PhD thesis, Stanford University.
26. Sondik, E. J. (1978). The optimal control of partially observable Markov processes over the infinite horizon, Operations Research, 21, 1071-1088.
27. White, C. C. III and Scherer, W. T. (1989). Solution procedures for partially observed Markov decision processes, Operations Research, 37(5), 791-797.
28. Zhang, N. L., Lee, S. S., and Zhang, W. (1999). A method for speeding up value iteration in partially observable Markov decision processes, in Proc. of the 15th Conference on Uncertainties in Artificial Intelligence.
29. Zhang, N. L. and W. Liu (1997). A model approximation scheme for planning in stochastic domains, Journal of Artificial Intelligence Research, 7, 199-230.
30. Zubek, V. B. and Dietterich, T. G.(2000). A POMDP approximation algorithm that anticipates the need to observe. To appear in Proceedings of the Pacific Rim Conference on Artificial Intelligence (PRICAI-2000), Lecture Notes in Computer Science, New York: Springer-Verlag.