Coordinated learning by exploiting sparse interaction in multiagent systems

Concurrency and Computation: Practice and Experience

Volumn 26, Issue 1, 2014, Pages 51-70

  Yu, Chao ^a   Zhang, Minjie ^a   Ren, Fenghui ^a  

^a UNIVERSITY OF WOLLONGONG   (Australia)

Author keywords

coordination; multiagent learning; reinforcement learning; sparse interaction

Indexed keywords

LEARNING SYSTEMS; MULTI AGENT SYSTEMS; OBSERVABILITY; REINFORCEMENT LEARNING;

COORDINATED BEHAVIOR; COORDINATED LEARNING; COORDINATION; DISTRIBUTED LEARNING; DYNAMIC ENVIRONMENTS; MULTI-AGENT LEARNING; SPARSE INTERACTION; STATISTICAL INFORMATION;

AUTONOMOUS AGENTS;

EID: 84890565590     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.2947     Document Type: Article

References (42)

1. Busoniu L, Babuska R, De Schutter B,. A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on System Man Cybernetics: Part C 2008; 38 (2): 156-172. (Pubitemid 351404112)
2. Kalyanakrishnan S, Stone P, Learning complementary multiagent behaviors: a case study. In Proceedings of the 13th RoboCup International Symposium. Springer: Berlin/Heidelberg, 2010; 153-165.
3. Yang E, Gu D,. Multiagent reinforcement learning for multi-robot systems: a survey. Technical Report CSM-404, Department of Computer Science, Univervisty of Essex, Colchester, UK, 2004.
4. Zhang C, Lesser V, Abdallah S,. Self-organization for coordinating decentralized reinforcement learning. In Proceedings of 9th International Conference of Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems: Richland, SC, 2010; 739-746.
5. Goldman CV, Zilberstein S,. Decentralized control of cooperative systems: categorization and complexity analysis. Journal of Artificial Intelligence Research 2004; 22: 143-174. (Pubitemid 41525885)
6. Tesauro G, Jong NK, Das R, Bennani MN, A hybrid reinforcement learning approach to autonomic resource allocation. In 2006 IEEE International Conference on Autonomic Computing. IEEE Press: New York, 2006; 65-73. (Pubitemid 46666907)
7. Galstyan A, Czajkowski K, Lerman K,. Resource allocation in the grid with learning agents. Journal of Grid Computing 2005; 3 (1): 91-100. (Pubitemid 41762487)
8. Tesauro G, Kephart JO,. Pricing in agent economies using multi-agent Q-learning. Autonomous Agent and Multi-Agent Systems 2002; 5 (3): 289-304. (Pubitemid 37113883)
9. Sandholm T,. Perspectives on multiagent learning. Journal of Artificial Intelligence 2007; 171 (7): 382-391. (Pubitemid 46802424)
10. Stone P,. Multiagent learning is not the answer. It is the question. Journal of Artificial Intelligence 2007; 171 (7): 402-405. (Pubitemid 46802413)
11. Shoham Y, Powers B, Grenager T,. If multi-agent learning is the answer,what is the question?. Journal of Artificial Intelligence 2007; 171 (7): 365-377.
12. Tuyls K,. An overview of cooperative and competitive multiagent learning. First International Workshop on Learning and Adaptation in MAS(LAMAS), Utrecht, The Netherlands, 2005; 1-46.
13. Melo FS, Veloso M,. Decentralized MDPs with sparse interactions. Artifcial Intelligence 2011; 175: 1757-1789.
14. Kok JR, Hoen P, Bakker B, Vlassis N,. Utile coordination: learning interdependencies among cooperative agents. In Proceedings of Symposium on Computational Intelligence and Games. IEEE Press: New York, 2005; 29-36.
15. Kok JR, Vlassis N,. Sparse cooperative Q-learning. In Proceedings of 21st International Conference on Machine Learning. ACM Press: New York, 2004; 61-68.
16. De H Y M, Vrancx P, Nowé A,. Learning multi-agent state space representations. In Proceedings of 9th International Conference of Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems: Richland, SC, 2010; 715-722.
17. De Hauwere YM, Vrancx P, Nowé A,. Learning what to observe in multi-agent systems. In Proceedings of 20th Belgian-Netherlands Conference on Artificial Intelligence. University of Twente Publisher: Enschede, the Netherlands, 2009; 83-90.
18. Kok JR, Vlassis N,. Sparse tabular multiagent Q-learning. Annual Machine Learning Conference of Belgium and the Netherlands, Brussels, Belgium, 2004; 65-71.
19. Ghavamzadeh M, Mahadevan S, Makar R,. Hierarchical multi-agent reinforcement learning. Autonomous Agents and Multi-Agent Systems 2006; 13 (2): 197-229.
20. Ghavamzadeh M, Mahadevan S,. Hierarchical average reward reinforcement learning. The Journal of Machine Learning Research 2007; 8: 2629-2669. (Pubitemid 350241862)
21. Guestrin C, Lagoudakis M,. Coordinated reinforcement learning. In Proceedings of 19th International Conference on Machine Learning. Morgan Kaufmann Publishers: San Mateo, CA, 2002; 227-234.
22. Schneider J, Wong WK, Moore A, Riedmiller M,. Distributed value functions. In Proceedings of the 16th International Conference on Machine Learning. Morgan Kaufmann Publishers: San Mateo, CA, 1999; 371-378.
23. Zhang C, Lesser V,. Coordinated multi-agent reinforcement learning in networked distributed pomdps. In Proceedings of the 25th National Conference on Argificial Intelligence (AAAI). AAAI Press: Menlo Park, California, 2011; 764-770.
24. Guestrin C,. Planning under uncertainty in complex structured environments. PhD thesis, Computer Science Department, Stanford University, August 2003.
25. Spaan M, Melo FS,. Interaction-driven Markov games for decentralized multiagent planning under uncertainty. In Proceedings of 7th International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems: Richland, SC, 2008; 525-532.
26. Melo FS, Veloso M,. Learning of coordination: exploiting sparse interactions in multiagent systems. In Proceedings of 8th International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems: Richland, SC, 2009; 772-780.
27. Puterman ML,. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc.: Hoboken, New Jersey, 1994.
28. Sutton RS, Barto AG,. Reinforcement Learning: An Introduction. MIT Press: Cambridge, 1998.
29. Watkins CJCH, Dayan P,. Q-learning. Machine Learning 1992; 8 (3): 279-292.
30. Bernstein DS, Givan R, Immerman N, Zilberstein S,. The complexity of decentralized control of Markov decision processes. Mathematics of Operations Research 2002; 27 (4): 819-840.
31. Allen M, Zilberstein S,. Complexity of decentralized control: special cases. Advanced Neural Information Processing Systems 2009; 22: 19-27.
32. Roth M, Simmons R, Veloso M,. Exploiting factored representations for decentralized execution in multiagent teams. In Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems. ACM Press: New York, 2007; 469-475.
33. Seuken S, Zilberstein S,. Formal models and algorithms for decentralized decision making under uncertainty. Autonomous Agents and Multi-Agent Systems 2008; 17 (2): 190-250.
34. Pynadath DV, Tambe M,. Multiagent teamwork: analyzing the optimality and complexity of key theories and models. In Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems: Part 2. ACM Press: New York, 2002; 873-880. (Pubitemid 34975283)
35. Boutilier C,. Planning, learning and coordination in multiagent decision processes. In Proceedings of 6th Conference on Theoretical Aspects of Rationality and Knowledge. Morgan Kaufmann Publishers: San Mateo, CA, 1996; 195-210.
36. Boutilier C,. Sequential optimality and coordination in multiagent systems. In International Joint Conference on Artificial Intelligence. Morgan Kaufmann Publishers: San Mateo, CA, 1999; 478-485.
37. Becker R, Zilberstein S, Lesser V, Goldman CV,. Transition-independent decentralized Markov decision processes. In Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems. ACM Press: New York, 2003; 41-48.
38. Becker R, Zilberstein S, Lesser V, Goldman CV,. Solving transition independent decentralized Markov decision processes. Journal of Artificial Intelligence Research 2004; 22: 423-455. (Pubitemid 41525892)
39. Allen M, Petrik M, Zilberstein S,. Interaction structure and dimensionality in decentralized problem solving. In Conference on Artificial Intelligence (AAAI). ACM Press: New York, 2008; 1440-1441.
40. Claus C, Boutilier C,. The dynamics of reinforcement learning in cooperative multiagent systems. In Proceedings of National of Conference on Artificial Intelligence. AAAI Press: Menlo Park, California, 1998; 746-752.
41. Tan M,. IMulti-agent reinforcement learning: independent vs. cooperative agents. In Proceedings of the Tenth International Conference on Machine Learning. Morgan Kaufmann Publishers: San Mateo, CA, 1993; 1440-1441.
42. Sen S, Sekaran M, Hale J, Learning to coordinate without sharing information. In Proceedings of the National Conference on Artificial Intelligence. John Wiley & Sons, Inc.: Hoboken, New Jersey, 1994; 426-426.