Nonapproximability results for partially observable Markov decision processes

Journal of Artificial Intelligence Research

Volumn 14, Issue , 2001, Pages 83-103

  Lusena, Christopher ^a   Goldsmith, Judy ^a   Mundhenk, Martin ^b  

^a UNIVERSITY OF KENTUCKY   (United States)

^b UNIVERSITY OF TRIER   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; DECISION MAKING; MARKOV PROCESSES; POLYNOMIALS; THEOREM PROVING;

MARKOV DECISION PROCESSES (MDP);

ARTIFICIAL INTELLIGENCE;

EID: 0036374190     PISSN: 10769757     EISSN: None     Source Type: Journal    
DOI: 10.1613/jair.714     Document Type: Article

References (38)

1. Bellman, R. (1957). Dynamic programming. Princeton University Press.
2. Blythe, J. (1999). Decision-theoretic planning. AI Magazine, 20(2), 37-54.
3. Boutilier, C., Dean, T. L., & Hanks, S. (1999). Decision-theoretic planning: Structural assumptions and computational leverage. Journal of AI Research, 11, 1-94.
4. Boutilier, C., Dearden, R., & Goldszmidt, M. (1995). Exploiting structure in policy construction. In Extending Theories of Action: Formal Theory & Practical Applications: Papers from the 1995 AAAI Spring Symposium (pp. 33-38). AAAI Press, Menlo Park, California.
5. Burago, D., de Rougemont, M., & Slissenko, A. (1996). On the complexity of partially observed Markov decision processes. Theoretical Computer Science, 157(2), 161-183.
6. Cassandra, A. (1998). Exact and approximate algorithms for partially observable Markov decision processes. Unpublished doctoral dissertation, Brown University.
7. Cassandra, A., Kaelbling, L., & Littman, M. L. (1995). Efficient dynamic-programming updates in partially observable Markov decision processes (Tech. Rep. No. CS-95-19). Providence, Rhode Island: Brown University.
8. Greenlaw, R., Hoover, H. J., & Ruzzo, W. L. (1995). Limits to parallel computation: P-completeness theory. Oxford University Press.
9. Hansen, E. (1998a). Finite-memory control of partially observable systems. Unpublished doctoral dissertation, Dept. of Computer Science, University of Massachussets at Amherst.
10. Hansen, E. (1998b). Solving POMDPs by searching in policy space. In Proceedings of the Conference Uncertainty in Artificial Intelligence (pp. 211-219).
11. Hansen, E., & Feng, Z. (2000). Dynamic programming for POMDPs using a factored state representation. In Proceedings of the 5th international conference on artificial intelligence planning and scheduling (pp. 130-149). Breckenridge, CO.
12. Hauskrecht, M. (1997). Planning and control in stochastic domains with imperfect information. Unpublished doctoral dissertation, Massachusetts Institute of Technology.
13. Kim, K.-E., Dean, T. L., & Meuleau, N. (2000). Approximate solutions to factored Markov decision processes via greedy search in the space of finite controllers. In Proceedings of the 5th international conference on artificial intelligence planning and scheduling (pp. 323-330). Breckenridge, CO.
14. Koller, D., & Parr, R. (2000). Policy iteration for factored MDPs. In Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence.
15. Littman, M. L. (1994). Memoryless policies: Theoretical limitations and practical results. In D. Cliff, P. Husbands, J.-A. Meyer, & S. W. Wilson (Eds.), From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior. MIT Press.
16. Littman, M. L., Dean, T. L., & Kaelbling, L. P. (1995). On the complexity of solving Markov decision problems. In Proceedings of the 11th Annual Conference on Uncertainty in Artificial Intelligence (pp. 394-402).
17. Lovejoy, W. S. (1991). Computationally feasible bounds for partially observed Markov decision process. Operations Research, 39(1), 192-175.
18. Lusena, C., Li, T., Sittinger, S., Wells, C., & Goldsmith, J. (1999). My brain is full: When more memory helps. In Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (pp. 374-381).
19. Madani, O., Hanks, S., & Condon, A. (1999). On the undecidability of probabilistic planning and infinite-horizon partially observable Markov decision problems. In Proceedings of the 16th National Conference on Artificial Intelligence (pp. 541-548).
20. McCallum, A. K. (1993). Overcoming incomplete perception with utile distinction memory. In Proceedings of the 10th International Conference on Machine Learning (pp. 190-196). Morgan Kaufmann Publishers.
21. Meuleau, N., Kim, K.-E., Kaelbling, L. P., & Cassandra, A. R. (1999). Solving POMDPs by searching the space of finite policies. In Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (pp. 417-426).
22. Meuleau, N., Peshkin, L., Kim, K.-E., & Kaelbling, L. P. (1999). Learning finite-state controllers for partially observable environments. In Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (pp. 427-436).
23. Mundhenk, M. (2000a). The complexity of optimal small policies. Mathematics of Operations Research, 25(1), 118-129.
24. Mundhenk, M. (2000b). The complexity of planning with partially-observable Markov decision processes (Tech. Rep. No. TR2000-376). Dartmouth College.
25. Mundhenk, M., Goldsmith, J., & Allender, E. (1997). The complexity of the policy existence problem for partially-observable finite-horizon Markov decision processes. In Proceedings 25th Mathematical Foundations of Computer Sciences conference (pp. 129-138). Springer-Verlag.
26. Mundhenk, M., Goldsmith, J., Lusena, C., & Allender, E. (2000). Complexity results for finite-horizon Markov decision process problems. Journal of the ACM, 47(4), 681-720.
27. Papadimitriou, C. H. (1994). Computational complexity. Addison-Wesley.
28. Papadimitriou, C. H., & Tsitsiklis, J. N. (1986). Intractable problems in control theory. SIAM Journal of Control and Optimization, 24(4), 639-654.
29. Papadimitriou, C. H., & Tsitsiklis, J. N. (1987). The complexity of Markov decision processes. Mathematics of Operations Research, 12(3), 441-450.
30. Peshkin, L., Meuleau, N., & Kaelbling, L. P. (1999). Learning policies with external memory. In Proceedings of the 16th International Conference on Machine Learning (pp. 307-314). Morgan Kaufmann, San Francisco, CA.
31. Platzman, L. K. (1977). Finite-memory estimation and control of finite probabilistic systems. Unpublished doctoral dissertation, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA.
32. Puterman, M. L. (1994). Markov decision processes. John Wiley & Sons, New York.
33. Shamir, A. (1992). IP = PSPACE. Journal of the ACM, 39(4), 869-877.
34. Smallwood, R. D., & Sondik, E. J. (1973). The optimal control of partially observed Markov processes over the finite horizon. Operations Research, 21, 1071-1088.
35. Sondik, E. J. (1971). The optimal control of partially observable Markov processes. Unpublished doctoral dissertation, Stanford University.
36. Tsitsiklis, J. N., & Van Roy, B. (1996). Feature-based methods for large scale dynamic programming. Machine Learning, 22, 59-94.
37. White, C. C., III. (1991). Partially observed Markov decision processes: A survey. Annals of Operations Research, 32, 215-230.
38. Zhang, N. L., Lee, S. S., & Zhang, W. (1999). A method for speeding up value iteration in partially obserable Markov decision processes. In Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (pp. 696-703).