On the undecidability of probabilistic planning and related stochastic optimization problems

Artificial Intelligence

Volumn 147, Issue 1-2, 2003, Pages 5-34

  Madani, Omid ^a   Hanks, Steve ^b   Condon, Anne ^c  

^a UNIVERSITY OF ALBERTA   (Canada)

^b UNIVERSITY OF WASHINGTON   (United States)

^c UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Computability; Computational complexity; Discounted; Infinity horizon; Markov decision processes; Partial observability; Probabilistic planning; Stochastic optimization; Undecidability; Unobservability

Indexed keywords

APPROXIMATION THEORY; COMPUTATIONAL COMPLEXITY; DECISION THEORY; MARKOV PROCESSES; MATHEMATICAL MODELS; PLANNING; PROBABILISTIC LOGICS; PROBLEM SOLVING;

PROBABILISTIC PLANNING;

ARTIFICIAL INTELLIGENCE;

EID: 0037840849     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0004-3702(02)00378-8     Document Type: Article

References (50)

1. Alur R., Couroubetis C., Yannakakis M. Distinguishing tests for nondeterministic and probabilistic machines. Proc. of 27th STOC. 1995;363-372.
2. Aoki M. Optimal control of partially observable Markovian systems. J. Franklin Inst. 280:1965;367-386.
3. Ästrom K.J. Optimal control of Markov processes with incomplete state information. J. Math. Anal. Appl. 10:1965;174-205.
4. Bertoni A. The solution to problems relative to probabilistic automata in the frame of the formal languages theory. Goos G., Hartmanis J. Vierte Jahrestagung der Gesellschaft für Informatik. 1975;107-112 Springer, Berlin.
5. Bertoni A., Mauri G., Torelli M. Some recursively unsolvable problems relating to isolated cutpoints in probabilistic automata. Automata, Languages and Programming. Lecture Notes in Computer Science. 1977;87-94 Springer, Berlin.
6. Bertsekas D.P. Dynamic Programming: Deterministic and Stochastic Models. 1987;Prentice-Hall, Englewood Cliffs, NJ.
7. V.D. Blondel, V. Canterini, Undecidable problems for probabilistic finite automata of fixed dimension, Technical Report, University of Louvain (Belgium), 2001. Available from: http://www.inma.ucl.ac.be/̃blondel/publications/.
8. Blondel V.D., Tsitsiklis J.N. A survey of computational complexity results in systems and control. Automatica. 36:(9):2000;1249-1274.
9. Boutilier C., Dean T., Hanks S. Decision theoretic planning: Structural assumptions and computational leverage. J. Artificial Intelligence Res. 11:2000;1-94.
10. Burago D., De Rougemont M., Slissenko A. On the complexity of partially observed Markov decision processes. Theoret. Comput. Sci. 1996;161-183.
11. Bylander T. The computational complexity of propositional STRIPS planning. Artificial Intelligence. 69:1994;161-204.
12. A.R. Cassandra, Exact and approximate algorithms for partially observable Markov decision processes, PhD Thesis, Brown University, 1998.
13. Chapman D. Planning for conjunctive goals. Artificial Intelligence. 32:(3):1987;333-377.
14. Condon A. The complexity of simple stochastic games. Inform. and Comput. 96:(2):1992;203-224.
15. Condon A., Lipton R. On the complexity of space bounded interactive proofs. Proc. 30th Annual Symposium on Foundations of Computer Science. 1989.
16. A.W. Drake, Observation of a Markov process through a noisy channel, PhD Thesis, Massachusetts Institute of Technology, 1962.
17. Draper D., Hanks S., Weld D. Probabilistic planning with information gathering and contingent execution. Proceedings of the Second International Conference on Artificial Intelligence Planning Systems. 1994;31-36 AAAI Press, Menlo Park, CA.
18. Fikes R.E., Nilsson N.J. STRIPS: A new approach to the application of theorem proving to problem solving. Artificial Intelligence. 2:(3-4):1971;189-208.
19. Freivalds R. Probabilistic two way machines. Proc. International Symposium on Mathemarical Foundations of Computer Science. 118:1981;33-45 Springer, Berlin.
20. Goldsmith J., Mundhenk M. Complexity issues in Markov decision processes. Proc. IEEE Conference on Computational Complexity. 1998;272-280.
21. E.A. Hansen, Finite memory control of partially observable systems, PhD Thesis, University of Massachusetts, Amherst, 1998.
22. Hopcroft J.E., Ullman J.D. Introduction to Automata Theory, Languages, and Computation. 1979;Addison-Wesley, Reading, MA.
23. Dean T.L., Kim K., Meuleau N. Approximate solutions to factored Markov decision processes via greedy search in the space of finite controllers. Proceedings of the Fifth International Conference on Artificial Intelligence Planning and Scheduling. 2000;323-330.
24. Kushmerick N., Hanks S., Weld D.S. An algorithm for probabilistic planning. Artificial Intelligence. 76:1995;239-286.
25. Moran S., Babai L. Arthur-Merlin games: A randomized proof system, and a hierarchy of complexity classes. J. Computer System Sci. 36:1988;254-276.
26. M.L. Littman, Algorithms for sequential decision making, PhD Thesis, Brown, 1996.
27. Littman M.L. Probabilistic propositional planning: Representations and complexity. Proceedings of the 14th National Conference on AI. 1997;AAAI Press.
28. Littman M.L., Dean T.L., Kaebling L.P. On the complexity of solving Markov decision problems. Proceedings of the Eleventh Annual Conference on Uncertainty in Artificial Intelligence. 1995.
29. Littman M.L., Goldsmith J., Mundhenk M. The computational complexity of probabilistic planning. J. Artificial Intelligence Res. 9:1998;1-36.
30. Lovejoy W. A survey of algorithmic methods for partially observable Markov decision processes. Ann. Oper. Res. 1991;47-66.
31. Ludwig W. A subexponential randomized algorithm for the simple stochastic game problem. Inform. and Comput. 117:1995;151-155.
32. Lusena C., Goldsmith J., Mundhenk M. Nonapproximability results for partially observable Markov decision processes. J. Artificial Intelligence Res. 14:2001;83-103.
33. O. Madani, Complexity results for infinite-horizon Markov decision processes, PhD Thesis, University of Washington, 2000.
34. Madani O., Hanks S., Condon A. On the computability of infinite-horizon partially observable Markov decision processes. Proc. of 16th National Conference in Artificial Intelligence. 1999;541-548.
35. Matiyasevich Y., Senizergues G. Decision problems for semi-thue systems with a few rules. Proc. 11th Annual IEEE Symposium on Logic in Computer Science. 1996;523-531.
36. Meuleau N., Peshkin L., Kim K., Kaelbling L.P. Learning finite-state controllers for partially observable environments. Laskey K.B., Prade H. Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (UAI-99). 1999;427-436 Morgan Kaufmann, San Francisco, CA.
37. Monahan G. A survey of partially observable Markov decision processes: Theory, models, and algorithms. Management Sci. 28:(1):1982;1-15.
38. Mundhenk M., Goldsmith J., Allender E. The complexity of policy evaluation for finite-horizon partially-observable Markov decision processes. Proc. 22nd Mathematical Foundations of Computer Science. 1997;129-138.
39. Mundhenk M., Goldsmith J., Lusena C., Allender E. Complexity results for finite-horizon Markov decision processs problems. J. ACM. 47:(4):2000.
40. Papadimitriou C.H., Tsitsiklis J.N. The complexity of Markov decision processes. Math. Oper. Res. 12:(3):1987;441-450.
41. Paz A. Introduction to Probabilistic Automata. 1971;Academic Press, New York.
42. Puterman M.L. Markov Decision Processes. 1994;Wiley Interscience.
43. Rabin M.O. Probabilistic automata. Inform. and Control. 6:(3):1963;230-245.
44. Rabin M.O. Lectures on classical and probabilistic automata. Caianiello E.R. Automata Theory. 1966;Academic Press, New York.
45. Rintanen J. Complexity of probabilistic planning under average rewards. Proc. IJCAI-01, Seattle, WA. 2001;503-508 Morgan Kaufmann, San Francisco, CA.
46. Smallwood R., Sondik E.J. The optimal control of partially observable Markov processes over a finite horizon. Oper. Res. 21:1973;1071-1088.
47. Sondik E.J. The optimal control of partially observable Markov processes over the infinite horizon: Discounted costs. Oper. Res. 26:(2):1978;282-304.
48. Tseng P. Solving H -horizon stationary Markov decision process in time proportional to log(H). Oper. Res. Lett. 9:(5):1990;287-297.
49. White D.J. Markov Decision Processes. 1993;Wiley, New York.
50. W. Zhang, Algorithms for partially observable Markov decision processes, PhD Thesis, Hong Kong University of Science and Technology, 2001.