The complexity of decentralized control of Markov decision processes

Mathematics of Operations Research

Volumn 27, Issue 4, 2002, Pages 819-840

  Bernstein, Daniel S ^a   Givan, Robert ^b   Immerman, Neil ^a   Zilberstein, Shlomo ^a  

^a Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

^b PURDUE UNIVERSITY   (United States)

Author keywords

Computational complexity; Decentralized control; Markov decision process

Indexed keywords

ALGORITHMS; BOUNDARY CONDITIONS; COMPUTATIONAL COMPLEXITY; DECENTRALIZED CONTROL; POLYNOMIALS;

MARKOV DECISION PROCESSES; NONDETERMINISTIC EXPONENTIAL TIME;

MARKOV PROCESSES;

EID: 0036874366     PISSN: 0364765X     EISSN: None     Source Type: Journal    
DOI: 10.1287/moor.27.4.819.297     Document Type: Article

References (24)

1. Altman, E. 2001. Applications of Markov decision processes in telecommunications: A survey. E. Feinberg, A. Shwartz, ed. Markov Decision Processes: Models, Methods, Directions, and Open Problems. Kluwer, New York.
2. Bobai, L., L. Fortnow, C. Lund. 1991. Non-deterministic exponential time has two-prover interactive protocols. Comput. Complexity 1 3-40.
3. Blondel, V. D., J. N. Tsitsiklis. 2000. A survey of computational complexity results in systems and control. Automatica 36(9) 1249-1274.
4. Cassandra, A., M. L. Littman, N. L. Zhang. 1997. Incremental pruning: A simple, fast, exact method for partially observable Markov decision processes. Proc. Thirteenth Ann. Conf. on Uncertainty in Artificial Intelligence, Providence, RI, 54-61.
5. Coradeschi, S., L. Karlsson, P. Stone, T. Balch, G. Kraetzschmar, M. Asada. 2000. Overview of RoboCup-99. AI Magazine 21(3) 11-18.
6. Hansen, E. 1998. Solving POMDPs by searching in policy space. In Proc. Fourteenth Ann. Conf. on Uncertainty in Artificial Intelligence, Madison, WI, 211-219.
7. Hsu, K., S. I. Marcus. 1982. Decentralized control of finite state Markov processes. IEEE Trans. Auto. Control AC-27(2) 426-431.
8. Jaakkola, T., S. P. Singh, M. I. Jordan. 1995. Reinforcement learning algorithm for partially observable Markov decision problems. Proc. Adv. in Neural Inform. Processing Systems 7 345-352.
9. Kaelbling, L. P., M. L. Littman, A. R. Cassandra. 1998. Planning and acting in partially observable stochastic domains. Artificial Intelligence 101(1-2), 99-134.
10. Lewis, H. 1978. Complexity of solvable cases of the decision problem for predicate calculus. Proc. Nineteenth Sympos. on the Foundations of Comput. Sci., Ann Arbor, MI, 35-47.
11. Lusena, C., J. Goldsmith, T. Li, S. Sittinger, C. Wells. 1999. My brain is full: When more memory helps. Proc. Fifteenth Conf. on Uncertainty in Artificial Intelligence, Stockholm, Sweden, 374-381.
12. Madani, O., S. Hanks, A. Condon. 1999. On the undecidability of probabilistic planning and infinite-horizon partially observable Markov decision problems. Proc. Sixteenth National Conf. on Artificial Intelligence, Orlando, FL, 541-548.
13. Meuleau, N., K.-E. Kim, L. Kaelbling, A. R. Cassandra. 1999. Solving POMPDs by searching the space of finite policies. Proc. Fifteenth Conf. on Uncertainty in Artificial Intelligence, Stockholm, Sweden, 417-426.
14. Mundhenk, M., J. Goldsmith, C. Lusena, E. Allender. 2000. Complexity of finite-horizon Markov decision process problems. J. ACM 47(4) 681-720.
15. Ooi, J. M., G. W. Wornell. 1996. Decentralized control of a multiple access broadcast channel: Performance bounds. In Proc. 35th Conf. on Decision and Control, Kobe, Japan, 293-298.
16. Papadimitriou, C. H. 1994. Computational Complexity. Addison-Wesley, Reading, MA.
17. ____, J. Tsitsiklis. 1982. On the complexity of designing distributed protocols. Inform. Control 53 211-218.
18. ____, ____. 1986. Intractable problems in control theory. SIAM J. Control Optim. 24(4) 639-654.
19. ____, ____. 1987. The complexity of Markov decision processes. Math. Oper. Res. 12(3) 441-450.
20. Peshkin, L., K.-E. Kim, N. Meuleau, L. P. Kaelbling. 2000. Learning to cooperate via policy search. Proc. Sixteenth Internat. Conf. on Uncertainty in Artificial Intelligence, Stanford, CA, 489-496.
21. Peterson, G. L., J. R. Reif. 1979. Multiple-person alternation. 20th Ann. Sympos. on Foundations of Comput. Sci., San Juan, PR, 348-363.
22. Puterman, M. L. 1994. Markov Decision Processes. Wiley & Sons, New York.
23. Schneider, J., W.-K. Wong, A. Moore, M. Riedmiller. 1999. Distributed value functions. Proc. Sixteenth Internat. Conf. on Machine Learning, Bled, Slovenia, 371-378.
24. Zhang, W. 2001. Algorithms for partially observable markov decision processes. Ph.D. thesis, Hong Kong University of Science and Technology, Kowloon, Hong Kong.