Multi-agent reinforcement learning as a rehearsal for decentralized planning

Neurocomputing

Volumn 190, Issue , 2016, Pages 82-94

  Kraemer, Landon ^a   Banerjee, Bikramjit ^a  

^a UNIVERSITY OF SOUTHERN MISSISSIPPI   (United States)

Author keywords

Decentralized planning; Multi agent reinforcement learning

Indexed keywords

BEHAVIORAL RESEARCH; BENCHMARKING; DECISION MAKING; FERTILIZERS; MARKOV PROCESSES; MULTI AGENT SYSTEMS; UNCERTAINTY ANALYSIS;

CENTRALIZED COMPUTATION; DECENTRALIZED PLANNING; DECISION MAKING UNDER UNCERTAINTY; DISTRIBUTED SOLUTIONS; MULTI-AGENT PLANNING; MULTI-AGENT REINFORCEMENT LEARNING; PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; SOLUTION TECHNIQUES;

REINFORCEMENT LEARNING;

ALGORITHM; ARTICLE; CONCEPTUAL FRAMEWORK; DECENTRALIZED PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; MACHINE LEARNING; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; MATHEMATICAL PARAMETERS; MULTI AGENT REINFORCEMENT LEARNING; PRIORITY JOURNAL; PROBABILITY;

EID: 84962082047     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2016.01.031     Document Type: Article

References (30)

1. Christopher Amato, Jilles Steeve Dibangoye, Shlomo Zilberstein, Incremental policy generation for finite-horizon Dec-POMDPs, in: Proceedings of the 19th International Conference on Automated Planning and Scheduling (ICAPS), Thessaloniki, Greece, September 19-23, AAAI, 2009.
2. Aras Raghav, Dutech Alain An investigation into mathematical programming for finite horizon decentralized POMDPs. J. Artif. Intell. Res. 2010, 37:329-396.
3. Auer Peter Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res. 2002, 3:397-422.
4. Bikramjit Banerjee, Jeremy Lyle, Landon Kraemer, Rajesh Yellamraju. Sample bounded distributed reinforcement learning for decentralized POMDPs, in: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI-12), Toronto, Canada, July 2012, pp. 1256-1262.
5. Bikramjit Banerjee, Peter Stone, General game learning using knowledge transfer, in: Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI-07), Hyderabad, India, 2007, pp. 672-677.
6. Bernstein Daniel S., Givan Robert, Immerman Neil, Zilberstein Shlomo The complexity of decentralized control of Markov decision processes. Math. Oper. Res. 2002, 27:819-840.
7. Craig Boutilier, Planning, learning and coordination in multiagent decision processes, in: Proceedings of 6th Conference on Theoretical Aspects of Rationality and Knowledge, 1996, pp. 195-210.
8. Lucian Busoniu, The MARL Toolbox version 1.3, 2010. http://busoniu.net/repository.php.
9. Busoniu Lucian., Babuska Robert., De Schutter Bart. A comprehensive survey of multiagent reinforcement learning. IEEE Trans. Syst. Man Cybern. Part C 2008, 38(2):156-172.
10. Jilles S. Dibangoye, Abdel-Illah Mouaddib, Brahim Chai-draa, Point-based incremental pruning heuristic for solving finite-horizon Dec-POMDPs, in: Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS-09), Budapest, Hungary, 2009, pp. 569-576.
11. Farahmand Amir Massoud Action-gap phenomenon in reinforcement learning. Advances in Neural Information Processing Systems 2011, vol. 24:172-180.
12. K-Team, Mobile robotics: Kilobots, 〈〉. http://www.k-team.com/mobile-robotics-products/kilobot.
13. W. Bradley Knox, Peter Stone, Combining manual feedback with subsequent MDP reward signals for reinforcement learning, in: Proceedings of 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2010), May 2010.
14. Landon Kraemer, Bikramjit Banerjee, Informed initial policies for learning in Dec-POMDPs, in: Proceedings of the AAMAS-12 Workshop on Adaptive Learning Agents (ALA-12), Valencia, Spain, June 2012, pp. 135-143.
15. Landon Kraemer, Bikramjit Banerjee, Concurrent reinforcement learning as a rehearsal for decentralized planning under uncertainty (extended abstract), in: Proceedings of the 12th International Conference on Autonomous Agents and Multi-agent Systems (AAMAS-13), St. Paul, MN, May 2013, pp. 1291-1292.
16. Yaxin Liu, Peter Stone, Value-function-based transfer for reinforcement learning using structure mapping, in: Proceedings of the Twenty-First National Conference on Artificial Intelligence, July 2006, pp. 415-420.
17. Mataric Maja J. Reinforcement learning in the multi-robot domain. Auton. Robots 1997, 4:73-83.
18. Andrew Y. Ng, Daishi Harada, Stuart Russell, Policy invariance under reward transformations: theory and application to reward shaping, in: Proceedings of 16th International Conference on Machine Learning, Morgan Kaufmann, Bled, Slovenia, 1999, pp. 278-287.
19. Frans A. Oliehoek, Matthijs T.J. Spaan, Jilles S. Dibangoye, Christopher Amato, Heuristic search for identical payoff Bayesian games, in: Proceedings of the Ninth International Conference on Autonomous Agents and Multiagent Systems (AAMAS-10), Toronto, Canada, 2010, pp. 1115-1122.
20. Oliehoek Frans A., Spaan Matthijs T.J., Vlassis Nikos Optimal and approximate Q-value functions for decentralized POMDPs. JAIR 2008, 32:289-353.
21. Frans A. Oliehoek, Nikos Vlassis, Q-value heuristics for approximate solutions of Dec-POMDPs, in: Proceedings of the AAAI Spring Symposium on Game Theoretic and Decision Theoretic Agents, March 2007, pp. 31-37.
22. Frans A. Oliehoek, Shimon Whiteson, Matthijs T.J. Spaan, Lossless clustering of histories in decentralized POMDPs, in: Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS-09), Budapest, Hungary, 2009, pp. 577-584.
23. Price Bob, Boutilier Craig Accelerating reinforcement learning through implicit imitation. J. Artif. Intell. Res. 2003, 19:569-629.
24. Sven Seuken, Shlomo Zilberstein, Memory-bounded dynamic programming for Dec-POMDPs, in: Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI-07), Hyderabad, India, 2007, pp. 2009-2015.
25. Matthijs Spaan, Dec-POMDP problem domains and format. 〈〉. http://masplan.org/.
26. Matthijs T.J. Spaan, Frans A. Oliehoek, Christopher Amato, Scaling up optimal heuristic search in Dec-POMDPs via incremental expansion, in: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence (IJCAI-11), Barcelona, Spain, 2011, pp. 2027-2032.
27. Sutton Richard S., Barto Andrew G. Reinforcement Learning: An Introduction 1998, MIT Press.
28. Daniel Szer, François Charpillet, Point-based dynamic programming for Dec-POMDPs, in: Proceedings of the 21st National Conference on Artificial Intelligence, Boston, MA, 2006, pp. 1233-1238.
29. Feng Wu, Shlomo Zilberstein, Xiaoping Chen, Rollout sampling policy iteration for decentralized POMDPs, in: Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI-10), 2010, pp. 666-673.
30. Chongjie Zhang, Victor Lesser, Coordinated multi-agent reinforcement learning in networked distributed POMDPs, in: Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI-11), San Francisco, CA, 2011.