Online planning for multi-agent systems with bounded communication

Artificial Intelligence

Volumn 175, Issue 2, 2011, Pages 487-511

  Wu, Feng ^a,b   Zilberstein, Shlomo ^b   Chen, Xiaoping ^a  

^a UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA   (China)

^b Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

Author keywords

Communication in multi agent systems; Cooperation and collaboration; Decentralized POMDPs; Planning under uncertainty

Indexed keywords

COMMON KNOWLEDGE; COMMUNICATION CHANNEL; CONDITIONAL PLANS; COOPERATION AND COLLABORATION; COORDINATED BEHAVIOR; DECENTRALIZED POMDPS; DECISION-THEORETIC; FAST LOCAL SEARCHES; LOCAL POLICIES; MULTI-AGENT SETTING; OFF-LINE PLANNING; OFFLINE; ON-LINE ALGORITHMS; ON-LINE PLANNING; ONLINE METHODS; ONLINE OPERATIONS; PLANNING UNDER UNCERTAINTY;

ALGORITHMS; COMMUNICATION; COMPUTATIONAL COMPLEXITY; ONLINE SYSTEMS; OPTIMIZATION; PROBLEM SOLVING;

MULTI AGENT SYSTEMS;

EID: 78650942474     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.artint.2010.09.008     Document Type: Article

References (61)

1. I. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci Wireless sensor networks: a survey Computer Networks 38 4 2002 393 422
2. C. Amato, D.S. Bernstein, S. Zilberstein, Optimizing memory-bounded controllers for decentralized POMDPs, in: Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence, 2007, pp. 1-8.
3. C. Amato, J.S. Dibangoye, S. Zilberstein, Incremental policy generation for finite-horizon DEC-POMDPs, in: Proceedings of the 19th International Conference on Automated Planning and Scheduling, 2009, pp. 2-9.
4. C. Amato, S. Zilberstein, Achieving goals in decentralized POMDPs, in: Proceedings of the 8th International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2009, pp. 593-600.
5. R. Aras, A. Dutech, F. Charpillet, Mixed integer linear programming for exact finite-horizon planning in decentralized POMDPs, in: Proceedings of the 17th International Conference on Automated Planning and Scheduling, 2007, pp. 18-25.
6. R. Becker, A. Carlin, V. Lesser, and S. Zilberstein Analyzing myopic approaches for multi-agent communication Computational Intelligence 25 1 2009 31 50
7. R. Becker, V.R. Lesser, S. Zilberstein, Analyzing myopic approaches for multi-agent communication, in: Proceedings of the 2005 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, 2005, pp. 550-557.
8. R. Becker, S. Zilberstein, V.R. Lesser, Decentralized Markov decision processes with event-driven interactions, in: Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2004, pp. 302-309.
9. R. Becker, S. Zilberstein, V.R. Lesser, and C.V. Goldman Solving transition independent decentralized Markov decision processes Journal of Artificial Intelligence Research 22 2004 423 455
10. D.S. Bernstein, C. Amato, E.A. Hansen, and S. Zilberstein Policy iteration for decentralized control of Markov decision processes Journal of Artificial Intelligence Research 34 2009 89 132
11. D.S. Bernstein, E.A. Hansen, S. Zilberstein, Bounded policy iteration for decentralized POMDPs, in: Proceedings of the 19th International Joint Conference on Artificial Intelligence, 2005, pp. 1287-1292.
12. D.S. Bernstein, S. Zilberstein, N. Immerman, The complexity of decentralized control of Markov decision processes, in: Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, 2000, pp. 32-37.
13. A. Beynier, A. Mouaddib, A polynomial algorithm for decentralized Markov decision processes with temporal constraints, in: Proceedings of the 4th International Joint Conference on Autonomous Agents and Multiagent Systems, 2005, pp. 963-969.
14. A. Beynier, A. Mouaddib, An iterative algorithm for solving constrained decentralized Markov decision processes, in: Proceedings of the 21st National Conference on Artificial Intelligence, 2006, pp. 1089-1094.
15. C. Boutilier, Planning, learning and coordination in multiagent decision processes, in: Proceedings of the 6th Conference on Theoretical Aspects of Rationality and Knowledge, 1996, pp. 195-210.
16. L. Busoniu, R. Babuska, and B.D. Schutter A comprehensive survey of multiagent reinforcement learning IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 38 2 2008 156 172
17. A. Carlin, S. Zilberstein, Value-based observation compression for DEC-POMDPs, in: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2008, pp. 501-508.
18. A. Carlin, S. Zilberstein, Myopic and non-myopic communication under partial observability, in: Proceedings of the 2009 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, 2009, pp. 331-338.
19. J.S. Dibangoye, A. Mouaddib, B. Chaib-draa, Point-based incremental pruning heuristic for solving finite-horizon DEC-POMDPs, in: Proceedings of the 8th International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2009, pp. 569-576.
20. R. Emery-Montemerlo, Game-theoretic control for robot teams, Doctoral Dissertation, Robotics Institute, Carnegie Mellon University, August 2005.
21. R. Emery-Montemerlo, G.J. Gordon, J.G. Schneider, S. Thrun, Approximate solutions for partially observable stochastic games with common payoffs, in: Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2004, pp. 136-143.
22. R. Emery-Montemerlo, G.J. Gordon, J.G. Schneider, S. Thrun, Game theoretic control for robot teams, in: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005, pp. 1163-1169.
23. M. Ghavamzadeh, S. Mahadevan, Learning to communicate and act using hierarchical reinforcement learning, in: Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2004, pp. 1114-1121.
24. C.V. Goldman, M. Allen, and S. Zilberstein Learning to communicate in a decentralized environment Autonomous Agents and Multi-Agent Systems 15 1 2007 47 90
25. C.V. Goldman, S. Zilberstein, Optimizing information exchange in cooperative multi-agent systems, in: Proceedings of the 2nd International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2003, pp. 137-144.
26. C.V. Goldman, and S. Zilberstein Decentralized control of cooperative systems: Categorization and complexity analysis Journal of Artificial Intelligence Research 22 2004 143 174
27. E.A. Hansen, D.S. Bernstein, S. Zilberstein, Dynamic programming for partially observable stochastic games, in: Proceedings of the 19th National Conference on Artificial Intelligence, 2004, pp. 709-715.
28. M.L. Littman, A.R. Cassandra, L.P. Kaelbling, Learning policies for partially observable environments: Scaling up, in: Proceedings of the 12th International Conference on Machine Learning, 1995, pp. 362-370.
29. J. Marecki, M. Tambe, On opportunistic techniques for solving decentralized Markov decision processes with temporal constraints, in: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, 2007, pp. 825-832.
30. A.C. Morris, D. Ferguson, Z. Omohundro, D. Bradley, D. Silver, C. Baker, S. Thayer, W. Whittaker, and W.R.L. Whittaker Recent developments in subterranean robotics Journal of Field Robotics 23 1 2006 35 57
31. R. Nair, M. Tambe, M. Roth, M. Yokoo, Communication for improving policy computation in distributed POMDPs, in: Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multiagent Systems, 2004, pp. 1098-1105.
32. R. Nair, M. Tambe, M. Yokoo, D.V. Pynadath, S. Marsella, Taming decentralized POMDPs: Towards efficient policy computation for multiagent settings, in: Proceedings of the 18th International Joint Conference on Artificial Intelligence, 2003, pp. 705-711.
33. R. Nair, P. Varakantham, M. Tambe, M. Yokoo, Networked distributed POMDPs: A synthesis of distributed constraint optimization and POMDPs, in: Proceedings of the 20th National Conference on Artificial Intelligence, 2005, pp. 133-139.
34. F.A. Oliehoek, M.T.J. Spaan, N. Vlassis, Dec-POMDPs with delayed communication, in: The 2nd Workshop on Multi-agent Sequential Decision-Making in Uncertain Domains, 2007.
35. F.A. Oliehoek, M.T.J. Spaan, and N. Vlassis Optimal and approximate q-value functions for decentralized POMDPs Journal of Artificial Intelligence Research 32 2008 289 353
36. F.A. Oliehoek, M.T.J. Spaan, S. Whiteson, N.A. Vlassis, Exploiting locality of interaction in factored Dec-POMDPs, in: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems, 2008, pp. 517-524.
37. F.A. Oliehoek, N. Vlassis, Q-value functions for decentralized POMDPs, in: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, 2007, pp. 833-840.
38. F.A. Oliehoek, S. Whiteson, M.T.J. Spaan, Lossless clustering of histories in decentralized POMDPs, in: Proceedings of the 8th International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2009, pp. 577-584.
39. S. Paquet, L. Tobin, B. Chaib-draa, An online POMDP algorithm for complex multiagent environments, in: Proceedings of the 4th International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2005, pp. 970-977.
40. L. Peshkin, K.-E. Kim, N. Meuleau, L.P. Kaelbling, Learning to cooperate via policy search, in: Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, 2000, pp. 489-496.
41. D.V. Pynadath, and M. Tambe The communicative multiagent team decision problem: Analyzing teamwork theories and models Journal of Artificial Intelligence Research 16 2002 389 423 (Pubitemid 43057178)
42. M. Roth, Execution-time communication decisions for coordination of multi-agent teams, Ph.D. thesis, The Robotics Institute, Carnegie Mellon University, 2007.
43. M. Roth, R. Simmons, M. Veloso, What to communicate? Execution-time decision in multi-agent POMDPs, in: Proceedings of the 8th International Symposium on Distributed Autonomous Robotic Systems, 2006.
44. M. Roth, R.G. Simmons, M.M. Veloso, Reasoning about joint beliefs for execution-time communication decisions, in: Proceedings of the 4th International Joint Conference on Autonomous Agents and Multiagent Systems, 2005, pp. 786-793.
45. M. Roth, R.G. Simmons, M.M. Veloso, Exploiting factored representations for decentralized execution in multiagent teams, in: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, 2007, pp. 457-463.
46. S. Seuken, S. Zilberstein, Improved memory-bounded dynamic programming for decentralized POMDPs, in: Proceedings of the 23rd Conference in Uncertainty in Artificial Intelligence, 2007, pp. 344-351.
47. S. Seuken, S. Zilberstein, Memory-bounded dynamic programming for DEC-POMDPs, in: Proceedings of the 20th International Joint Conference on Artificial Intelligence, 2007, pp. 2009-2015.
48. S. Seuken, and S. Zilberstein Formal models and algorithms for decentralized decision making under uncertainty Journal of Autonomous Agents and Multi-Agent Systems 17 2 2008 190 250
49. J. Shen, V.R. Lesser, N. Carver, Minimizing communication cost in a distributed bayesian network using a decentralized mdp, in: Proceedings of the 2nd International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2003, pp. 678-685.
50. M.T.J. Spaan, G.J. Gordon, N. Vlassis, Decentralized planning under uncertainty for teams of communicating agents, in: Proceedings of the 5th International Joint Conference on Autonomous Agents and Multiagent Systems, 2006, pp. 249-256.
51. M.T.J. Spaan, F.S. Melo, Interaction-driven Markov games for decentralized multiagent planning under uncertainty, in: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems, 2008, pp. 525-532.
52. M.T.J. Spaan, F.A. Oliehoek, N. Vlassis, Multiagent planning under uncertainty with stochastic communication delays, in: Proceedings of the 18th International Conference on Automated Planning and Scheduling, 2008, pp. 338-345.
53. P. Stone, and M.M. Veloso Task decomposition, dynamic role assignment, and low-bandwidth communication for real-time strategic teamwork Artificial Intelligence 110 2 1999 241 273
54. D. Szer, F. Charpillet, Improving coordination with communication in multi-agent reinforcement learning, in: Proceedings of the 6th IEEE International Conference on Tools with Artificial Intelligence, 2004, pp. 436-440.
55. D. Szer, F. Charpillet, Point-based dynamic programming for DEC-POMDPs, in: Proceedings of the 21st National Conference on Artificial Intelligence, 2006, pp. 1233-1238.
56. D. Szer, F. Charpillet, S. Zilberstein, Maa*: A heuristic search algorithm for solving decentralized POMDPs, in: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence, 2005, pp. 576-590.
57. M. Tambe Towards flexible teamwork Journal of Artificial Intelligence Research 7 1997 83 124
58. J. Tsitsiklis, and M. Athans On the complexity of decentralized decision making and detection problems IEEE Transaction on Automatic Control 30 1985 440 446
59. S.A. Williamson, E.H. Gerding, N.R. Jennings, A principled information valuation for communication during multi-agent coordination, in: The 3rd Workshop on Multi-agent Sequential Decision-Making in Uncertain Domains, 2008.
60. S.A. Williamson, E.H. Gerding, N.R. Jennings, Reward shaping for valuing communications during multi-agent coordination, in: Proceedings of the 8th International Joint Conference on Autonomous Agents and Multiagent Systems, 2009, pp. 641-648.
61. P. Xuan, V. Lesser, S. Zilberstein, Communication decisions in multi-agent cooperation: Model and experiments, in: Proceedings of the 5th International Conference on Autonomous Agents, 2001, pp. 616-623.