Communication-based decomposition mechanisms for decentralized MDPs

Journal of Artificial Intelligence Research

Volumn 32, Issue , 2008, Pages 169-202

  Goldman, Claudia V ^a   Zilberstein, Shlomo ^b  

^a Samsung Telecom Research Israel   (Israel)

^b Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DECISION MAKING; HEURISTIC ALGORITHMS; MARKOV PROCESSES; MULTI AGENT SYSTEMS; STOCHASTIC SYSTEMS;

DECOMPOSITION MECHANISM; GOAL-ORIENTED BEHAVIOR; HEURISTIC SEARCH ALGORITHMS; MARKOV DECISION PROBLEM; MULTI-ROBOT COORDINATION; POLYNOMIAL-TIME ALGORITHMS; SEMI-MARKOV DECISION PROCESS; STOCHASTIC ENVIRONMENT;

OPTIMIZATION;

EID: 52249090037     PISSN: None     EISSN: 10769757     Source Type: Journal    
DOI: 10.1613/jair.2466     Document Type: Article

References (27)

1. Becker, R., Zilberstein, S., Lesser, V., & Goldman, C. V. (2004). Solving transition independent decentralized MDPs. Journal of Artificial Intelligence Research, 22, 423-455.
2. Bererton, C., Gordon, G., Thrun, S., & Khosla, P. (2003). Auction mechanism design for multi-robot coordination. In Proceedings of the Seventeenth Annual Conference on Neural Information Processing Systems, Whistler, BC, Canada.
3. Bernstein, D., Givan, R., Immerman, N., & Zilberstein, S. (2002). The complexity of decentralized control of Markov decision processes. Mathematics of Operations Research, 27(4), 819-840.
4. Boutilier, C. (1999). Sequential optimality and coordination in multiagent systems. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, pp. 478-485, Stockholm, Sweden.
5. Ghavamzadeh, M., & Mahadevan, S. (2004). Learning to act and communicate in cooperative multiagent systems using hierarchical reinforcement learning. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multi-Agent Systems, pp. 1114-1121, New York City, NY.
6. Goldman, C. V., & Zilberstein, S. (2003). Optimizing information exchange in cooperative multi-agent systems. In Proceedings of the Second International Joint Conference on Autonomous Agents and Multi-Agent Systems, pp. 137-144, Melbourne, Australia.
7. Goldman, C. V., & Zilberstein, S. (2004a). Decentralized control of cooperative systems: Categorization and complexity analysis. Journal of Artificial Intelligence Research, 22, 143-174.
8. Goldman, C. V., & Zilberstein, S. (2004b). Goal-oriented Dec-MDPs with direct communication. Technical Report 04-44, University of Massachusetts at Amherst, Computer Science Department.
9. Guestrin, C., & Gordon, G. (2002). Distributed planning in hierarchical factored MDPs. In Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence, pp. 197-206, Edmonton, Canada.
10. Guestrin, C., Koller, D., & Parr, R. (2001). Multiagent planning with factored MDPs. In Advances in Neural Information Processing Systems, pp. 1523-1530, Vancouver, British Columbia.
11. Hansen, E. A. (1997). Markov decision processes with observation costs. Technical Report 97-01, University of Massachusetts at Amherst, Computer Science Department.
12. Hansen, E. A., & Zhou, R. (2003). Synthesis of hierarchical finite-state controllers for POMDPs. In Proceedings of the Thirteenth International Conference on Automated Planning and Scheduling, Trento, Italy.
13. Nair, R., Tambe, M., Roth, M., & Yokoo, M. (2004). Communication for improving policy computation in distributed POMDPs. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multi-Agent Systems, pp. 1098-1105, New York City, NY.
14. Nair, R., Tambe, M., Yokoo, M., Pynadath, D., & Marsella, S. (2003). Taming decentralized POMDPs: Towards efficient policy computation for multiagent settings. In Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, pp. 705711, Acapulco, Mexico.
15. Papadimitriou, C. H., & Tsitsiklis, J. (1987). The complexity of Markov decision processes. Mathematics of Operations Research, 12(3), 441-450.
16. Peshkin, L., Kim, K.-E., Meuleau, N., & Kaelbling, L. P. (2000). Learning to cooperate via policy search. In Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, pp. 489-496, Stanford, CA.
17. Petrik, M., & Zilberstein, S. (2007). Anytime coordination using separable bilinear programs. In Proceedings of the Twenty Second Conference on Artificial Intelligence, pp. 750-755, Vancouver, BC, Canada.
18. Puterman, M. L. (1994). Markov Decision Processes - Discrete Stochastic Dynamic Programming. Wiley & Sons, Inc., New York.
19. Pynadath, D. V., & Tambe, M. (2002). The communicative multiagent team decision problem: Analyzing teamwork theories and models. Journal of Artificial Intelligence Research, 16, 389-423.
20. Schneider, J., Wong, W.-K., Moore, A., & Riedmiller, M. (1999). Distributed value functions. In Proceedings of the Sixteenth International Conference on Machine Learning, pp. 371-378, Bled, Slovenia.
21. Seuken, S., & Zilberstein, S. (2007a). Improved memory-bounded dynamic programming for Dec-POMDPs. In Proceedings of the Twenty Third Conference on Uncertainty in Artificial Intelligence, Vancouver, BC, Canada.
22. Seuken, S., & Zilberstein, S. (2007b). Memory-bounded dynamic programming for DecPOMDPs. In Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, pp. 2009-2015, Hyderabad, India.
23. Seuken, S., & Zilberstein, S. (2008). Formal models and algorithms for decentralized decision making under uncertainty. Autonomous Agents and Multi-agent Systems. To appear.
24. Sutton, R. S., Precup, D., & Singh, S. (1999). Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112, 181-211.
25. Suzuki, L, & Yamashita, M. (1999). Distributed anonymous mobile robots: Formation of geometric patterns. SIAM Journal on Computing, 28(4), 1347-1363.
26. Wolpert, D. H., Wheeler, K. R., & Tumer, K. (1999). General principles of learning-based multi-agent systems. In Proceedings of the Third International Conference on Autonomous Agents, pp. 77-83, Seattle, Washington.
27. Xuan, P., Lesser, V., & Zilberstein, S. (2001). Communication decisions in multi-agent cooperation: Model and experiments. In Proceedings of the Fifth International Conference on Autonomous Agents, pp. 616-623, Montreal, Canada.