Decentralized MDPs with sparse interactions

Artificial Intelligence

Volumn 175, Issue 11, 2011, Pages 1757-1789

  Melo, Francisco S ^a   Veloso, Manuela ^b  

^a Edifício IST ^*  (Portugal)

^b Carnegie Mellon University ^*  (United States)

Author keywords

Decentralized Markov decision processes; Multiagent coordination; Sparse interaction

Indexed keywords

DECISION-THEORETIC; INDEPENDENT AGENTS; LOCAL INTERACTIONS; MARKOV DECISION PROCESSES; MULTI-AGENT; MULTI-AGENT COORDINATIONS; NEW MODEL; SOLUTION METHODS; SPARSE INTERACTION; STATE SPACE; THEORETICAL ERRORS;

ERROR ANALYSIS; LEARNING ALGORITHMS; MARKOV PROCESSES;

MULTI AGENT SYSTEMS;

EID: 79955976414     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.artint.2011.05.001     Document Type: Article

References (51)

1. N. Meuleau, M. Hauskrecht, K. Kim, L. Peshkin, L. Kaelbling, T. Dean, C. Boutilier, Solving very large weakly coupled Markov decision processes, in: Proc. 15th AAAI Conf. Artificial Intelligence, 1998, pp. 165-172.
2. S. Singh, and D. Cohn How to dynamically merge Markov decision processes Advances in Neural Information Processing Systems 10 1998 1057 1063
3. R. Nair, and M. Tambe Hybrid BDI-POMDP framework for multiagent teaming Journal of Artificial Intelligence Research 23 2005 367 420 (Pubitemid 41525873)
4. P. Stone, M. Veloso, Team-partitioned, opaque-transition reinforcement learning, in: Proc. RoboCup-98, 1998, pp. 206-212.
5. M. Ghavamzadeh, S. Mahadevan, and R. Makar Hierarchical multiagent reinforcement learning Journal of Autonomous Agents and Multiagent Systems 13 2 2006 197 229
6. C. Guestrin, D. Koller, and R. Parr Multiagent planning with factored MDPs Advances in Neural Information Processing Systems 14 2001 1523 1530
7. J. Kok, P. Hoen, B. Bakker, N. Vlassis, Utile coordination: learning interdependencies among cooperative agents, in: IEEE Symp. Computational Intelligence and Games, 2005, pp. 61-68.
8. M. Roth, R. Simmons, M. Veloso, Exploiting factored representations for decentralized execution in multiagent teams, in: Proc. 6th Int. Conf. Autonomous Agents and Multiagent Systems, 2007, pp. 469-475.
9. M. Kearns, M. Littman, S. Singh, Graphical models for game theory, in: Proc. 17th Conf. Uncertainty in Artificial Intelligence, 2001, pp. 253-260.
10. A. Xin Jiang, K. Leyton-Brown, N. Bhat, Action-graph games, Tech. rep. TR-2008-13, Univ. British Columbia, 2008.
11. M. Allen, and S. Zilberstein Complexity of decentralized control: special cases Advances in Neural Information Processing Systems 22 2009 19 27
12. C. Goldman, and S. Zilberstein Decentralized control of cooperative systems: categorization and complexity analysis Journal of Artificial Intelligence Research 22 2004 143 174 (Pubitemid 41525885)
13. P. Varakantham, J. Kwak, M. Taylor, J. Marecki, P. Scerri, M. Tambe, Exploiting coordination locales in distributed POMDPs via social model shaping, in: Proc. 19th Int. Conf. Automated Planning and Scheduling, 2009, pp. 313-320.
14. M. Puterman, and Markov Decision Processes Discrete Stochastic Dynamic Programming 1994 John Wiley & Sons, Inc.
15. L. Kaelbling, M. Littman, and A. Cassandra Planning and acting in partially observable stochastic domains Artificial Intelligence 101 1998 99 134 (Pubitemid 128387390)
16. D. Bernstein, R. Givan, N. Immerman, and S. Zilberstein The complexity of decentralized control of Markov decision processes Mathematics of Operations Research 27 4 2002 819 840
17. C. Watkins, Learning from delayed rewards, Ph.D. thesis, King s College, Cambridge Univ., 1989.
18. D. Bertsekas, and J. Tsitsiklis Neuro-Dynamic Programming 1996 Athena Scientific
19. R. Smallwood, and E. Sondik The optimal control of partially observable Markov processes over a finite horizon Operations Research 21 5 1973 1071 1088
20. O. Madani, S. Hanks, A. Condon, On the undecidability of probabilistic planning and infinite-horizon partially observable Markov decision problems, in: Proc. 16th AAAI Conf. Artificial Intelligence, 1999, pp. 541-548.
21. A. Cassandra, Exact and approximate algorithms for partially observable Markov decision processes, Ph.D. thesis, Dept. Computer Sciences, Brown Univ., 1998.
22. D. Aberdeen, A (revised) survey of approximate methods for solving partially observable Markov decision processes, Tech. rep., National ICT Australia, 2003.
23. M. Littman, A. Cassandra, L. Kaelbling, Learning policies for partially observable environments: scaling up, in: Proc. 12th Int. Conf. Machine Learning, 1995, pp. 362-370.
24. F. Melo, M. Ribeiro, Transition entropy in partially observable Markov decision processes, in: Proc. 9th Int. Conf. Intelligent Autonomous Systems, 2006, pp. 282-289.
25. C. Papadimitriou, and J. Tsitsiklis The complexity of Markov decision processes Mathematics of Operations Research 12 3 1987 441 450
26. S. Seuken, and S. Zilberstein Formal models and algorithms for decentralized decision-making under uncertainty Journal of Autonomous Agents and Multiagent Systems 17 2 2008 190 250
27. R. Becker, S. Zilberstein, V. Lesser, and C. Goldman Solving transition independent decentralized Markov decision processes Journal of Artificial Intelligence Research 22 2004 423 455 (Pubitemid 41525892)
28. M. Allen, Interactions in decentralized environments, Ph.D. thesis, Univ. Massachusetts, Amherst, 2009.
29. A. Cassandra, Optimal policies for partially observable Markov decision processes, Tech. rep. CS-94-14, Dept. Computer Sciences, Brown Univ., 1994.
30. M. Spaan, F. Melo, Interaction-driven Markov games for decentralized multiagent planning under uncertainty, in: Proc. 7th Int. Conf. Autonomous Agents and Multiagent Systems, 2008, pp. 525-532.
31. C. Claus, C. Boutilier, The dynamics of reinforcement learning in cooperative multiagent systems, in: Proc. 15th AAAI Conf. Artificial Intelligence, 1998, pp. 746-752.
32. M. Tan Multi-agent reinforcement learning: independent vs. cooperative agents Readings in Agents 1997 Morgan Kaufman 487 494
33. D. Leslie, and E. Collins Generalised weakened fictitious play Games and Economic Behavior 56 2006 285 298 (Pubitemid 43812068)
34. R. Sutton, and A. Barto Reinforcement Learning: An Introduction 1998 MIT Press
35. L. Shapley Stochastic games Proceedings of the National Academy of Sciences 39 1953 1095 1100
36. C. Boutilier, Planning, learning and coordination in multiagent decision processes, in: Theoretical Aspects of Rationality and Knowledge, 1996, pp. 195-210.
37. D. Pynadath, and M. Tambe The communicative multiagent team decision problem: analyzing teamworktheories and models Journal of Artificial Intelligence Research 16 2002 389 423 (Pubitemid 43057178)
38. P. Gmytrasiewicz, and P. Doshi A framework for sequential planning in multiagent settings Journal of Artificial Intelligence Research 24 2005 49 79 (Pubitemid 43130932)
39. R. Emery-Montemerlo, G. Gordon, J. Schneider, S. Thrun, Approximate solutions for partially observable stochastic games with common payoffs, in: Proc. 3rd Int. Conf. Autonomous Agents and Multiagent Systems, 2004, pp. 136-143.
40. M. Roth, Execution-time communication decisions for coordination of multiagent teams, Ph.D. thesis, Carnegie Mellon University, August 2007.
41. R. Becker, S. Zilberstein, V. Lesser, C. Goldman, Transition-independent decentralized Markov decision processes, in: Proc. 2nd Int. Conf. Autonomous Agents and Multiagent Systems, 2003, pp. 41-48.
42. R. Makar, S. Mahadevan, Hierarchical multiagent reinforcement learning, in: Proc. 5th Int. Conf. Autonomous Agents, 2001, pp. 246-253.
43. C. Guestrin, S. Venkataraman, D. Koller, Context-specific multiagent coordination and planning with factored MDPs, in: Proc. 18th AAAI Conf. Artificial Intelligence, 2002, pp. 253-259.
44. J. Kok, N. Vlassis, Sparse cooperative Q-learning, in: Proc. 21st Int. Conf. Machine Learning, 2004, pp. 61-68.
45. N. Bhat, K. Leyton-Brown, Computing Nash equilibria of action-graph games, in: Proc. 20th Conf. Uncertainty in Artificial Intelligence, 2004, pp. 35-42.
46. A. Xin Jiang, K. Leyton-Brown, A polynomial-time algorithm for action-graph games, in: Proc. 21st AAAI Conf. Artificial Intelligence, 2006, pp. 679-684. (Pubitemid 44705361)
47. A. Xin Jiang, K. Leyton-Brown, Computing pure Nash equilibria in symmetric action-graph games, in: Proc. 22nd AAAI Conf. Artificial Intelligence, 2007, pp. 79-85. (Pubitemid 350149556)
48. R. Becker, V. Lesser, S. Zilberstein, Decentralized Markov decision processes with event-driven interactions, in: Proc. 3rd Int. Conf. Autonomous Agents and Multiagent Systems, 2004, pp. 302-309.
49. R. Becker, V. Lesser, S. Zilberstein, Analyzing myopic approaches for multiagent communication, in: Proc. IEEE Int. Conf. Intelligent Agent Technology, 2005, pp. 550-557. (Pubitemid 46116612)
50. R. Becker, A. Carlin, V. Lesser, and S. Zilberstein Analyzing myopic approaches for multiagent communications Computational Intelligence 25 1 2009 31 50
51. S. Simic On a global upper bound for Jensen s inequality Journal of Mathematical Analysis and Applications 343 2008 414 419