Learning multiagent communication with backpropagation

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 2252-2260

  Sukhbaatar, Sainbayar ^a   Szlam, Arthur ^b   Fergus, Rob ^b  

^a POLYTECHNIC UNIVERSITY   (United States)

^b FACEBOOK AI RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BACKPROPAGATION; MULTI AGENT SYSTEMS;

COOPERATIVE TASKS; MULTI-AGENT COMMUNICATIONS; MULTIPLE AGENTS; NEURAL MODELING;

COOPERATIVE COMMUNICATION;

EID: 85018860957     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (37)

1. L. Busoniu, R. Babuska, and B. De Schutter. A comprehensive survey of multiagent reinforcement learning. Systems, Man, and Cybernetics, IEEE Transactions on, 38(2):156-172, 2008.
2. Y. Cao, W. Yu, W. Ren, and G. Chen. An overview of recent progress in the study of distributed multi-agent coordination. IEEE Transactions on Industrial Informatics, 1(9):427-438, 2013.
3. R. H. Crites and A. G. Barto. Elevator group control using multiple reinforcement learning agents. Machine Learning, 33(2):235-262, 1998.
4. J. N. Foerster, Y. M. Assael, N. de Freitas, and S. Whiteson. Learning to communicate to solve riddles with deep distributed recurrent Q-networks. arXiv, abs/1602.02672, 2016.
5. D. Fox, W. Burgard, H. Kruppa, and S. Thrun. Probabilistic approach to collaborative multi-robot localization. Autonomous Robots, 8(3):325-344, 2000.
6. C. L. Giles and K. C. Jim. Learning communication for multi-agent systems. In Innovative Concepts for Agent Based Systems, pages 377-390. Springer, 2002.
7. C. Guestrin, D. Koller, and R. Parr. Multiagent planning with factored MDPs. In NIPS, 2001.
8. X. Guo, S. Singh, H. Lee, R. L. Lewis, and X. Wang. Deep learning for real-time atari game play using offline monte-carlo tree search planning. In NIPS, 2014.
9. L. Kaiser and I. Sutskever. Neural gpus learn algorithms. In ICLR, 2016.
10. T. Kasai, H. Tenmoto, and A. Kamiya. Learning of communication codes in multi-agent reinforcement learning problem. IEEE Conference on Soft Computing in Industrial Applications, pages 1-6, 2008.
11. D. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
12. M. Lauer and M. A. Riedmiller. An algorithm for distributed reinforcement learning in cooperative multi-agent systems. In ICML, 2000.
13. S. Levine, C. Finn, T. Darrell, and P. Abbeel. End-to-end training of deep visuomotor policies. Journal of Machine Learning Research, 17(39):1-40, 2016.
14. Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel. Gated graph sequence neural networks. In ICLR, 2015.
15. M. L. Littman. Value-function reinforcement learning in markov games. Cognitive Systems Research, 2(1):55-66, 2001.
16. C. J. Maddison, A. Huang, I. Sutskever, and D. Silver. Move evaluation in go using deep convolutional neural networks. In ICLR, 2015.
17. D. Maravall, J. De Lope, and R. Domnguez. Coordination of communication in robot teams by reinforcement learning. Robotics and Autonomous Systems, 61(7):661-666, 2013.
18. M. Matari. Reinforcement learning in the multi-robot domain. Autonomous Robots, 4(1):73-83, 1997.
19. F. S. Melo, M. Spaan, and S. J. Witwicki. Querypomdp: Pomdp-based communication in multiagent systems. In Multi-Agent Systems, pages 189-204, 2011.
20. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, D. Wierstra, S. Legg, and D. Hassabis. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533, 2015.
21. R. Olfati-Saber, J. Fax, and R. Murray. Consensus and cooperation in networked multi-agent systems. Proceedings of the IEEE, 95(1):215-233, 2007.
22. J. Pearl. Reverend bayes on inference engines: A distributed hierarchical approach. In AAAI, 1982.
23. F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini. The graph neural network model. IEEE Trans. Neural Networks, 20(1):61-80, 2009.
24. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484-489, 2016.
25. P. Stone and M. Veloso. Towards collaborative and adversarial learning: A case study in robotic soccer. International Journal of Human Computer Studies, (48), 1998.
26. S. Sukhbaatar, A. Szlam, G. Synnaeve, S. Chintala, and R. Fergus. Mazebase: A sandbox for learning from games. CoRR, abs/1511.07401, 2015.
27. S. Sukhbaatar, A. Szlam, J. Weston, and R. Fergus. End-to-end memory networks. NIPS, 2015.
28. R. S. Sutton and A. G. Barto. Introduction to Reinforcement Learning. MIT Press, 1998.
29. A. Tampuu, T. Matiisen, D. Kodelja, I. Kuzovkin, K. Korjus, J. Aru, and R. Vicente. Multiagent cooperation and competition with deep reinforcement learning. arXiv:1511.08779, 2015.
30. M. Tan. Multi-agent reinforcement learning: Independent vs. cooperative agents. In ICML, 1993.
31. T. Tieleman and G. Hinton. Lecture 6.5-RmsProp: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, 2012.
32. P. Varshavskaya, L. P. Kaelbling, and D. Rus. Distributed Autonomous Robotic Systems 8, chapter Efficient Distributed Reinforcement Learning through Agreement, pages 367-378. 2009.
33. X. Wang and T. Sandholm. Reinforcement learning to play an optimal nash equilibrium in team markov games. In NIPS, pages 1571-1578, 2002.
34. J. Weston, A. Bordes, S. Chopra, and T. Mikolov. Towards ai-complete question answering: A set of prerequisite toy tasks. In ICLR, 2016.
35. R. J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. In Machine Learning, pages 229-256, 1992.
36. C. Xiong, S. Merity, and R. Socher. Dynamic memory networks for visual and textual question answering. ICML, 2016.
37. C. Zhang and V. Lesser. Coordinating multi-agent reinforcement learning with limited communication. In Proc. AAMAS, pages 1101-1108, 2013.