The world of independent learners is not markovian

International Journal of Knowledge-Based and Intelligent Engineering Systems

Volumn 15, Issue 1, 2011, Pages 55-64

  Laurent, Guillaume J ^a   Matignon, Laëtitia ^a   Fort Piat, N Le ^a  

^a CNRS   (France)

Author keywords

machine learning; Multi agent system; reinforcement learning

Indexed keywords

FORMAL CONCEPTS; LEARNING AGENTS; LEARNING METHODS; LEARNING PATHS; MARKOVIAN; NON-MARKOVIAN; SINGLE-AGENT;

INTELLIGENT AGENTS; LEARNING ALGORITHMS; MULTI AGENT SYSTEMS;

LEARNING SYSTEMS;

EID: 80052079894     PISSN: 13272314     EISSN: 18758827     Source Type: Journal    
DOI: 10.3233/KES-2010-0206     Document Type: Article

References (41)

1. R. Bellman, A markov decision process, Journal of Mathematical Mechanics 6(1957), 679-684.
2. V. S. Borkar, Reinforcement learning in markovian evolutionary games, Advances in Complex Systems 5(1) (2002), 55-72.
3. M. Bowling, Multiagent Learning in the Presence of Agents with Limitations, PhD thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA, May 2003.
4. M. Bowling and M. Veloso, Multiagent learning using a variable learning rate, Artificial Intelligence 136(2002), 215-250. (Pubitemid 34232184)
5. M. Bowling and M. M. Veloso, An analysis of stochastic game theory for multiagent reinforcement learning, Technical Report CMU-CS-00-165, Computer Science Department, Carnegie Mellon University, 2000.
6. L. Busoniu, R. Babuska and B. De Schutter, A comprehensive survey of multiagent reinforcement learning, IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 3838(2) (2008), 156-172. (Pubitemid 351404112)
7. L. Busoniu, R. Babuska and B. De Schutter, Decentralized Reinforcement Learning Control of a Robotic Manipulator, In Proceedings of the 9th International Conference on Control, Automation, Robotics and Vision, pages 1347-1352, 2006.
8. C. Claus and C. Boutilier, The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems, In Proc. of the National Conference on Artificial Intelligence, pages 746-752. AAAI, 1998.
9. R. H. Crites and A. G. Barto, Improving Elevator Performance using Reinforcement Learning, In Proc. of Advances in Neural Information Processing Systems, Cambridge, MA, 1996. The MIT Press.
10. J. Dowling and S. Haridi, Decentralized reinforcement learning for the online optimization of distributed systems, in: Reinforcement Learning: Theory and Applications, C. Weber, M. Elshaw and N. M. Mayer, eds, I-TECH Education and Publishing, 2008, pp. 143-166.
11. N. Fulda and D. Ventura, Predicting and Preventing Coordination Problems in Cooperative Q-Learning Systems, In Proceedings of the International Joint Conference on Artificial Intelligence, 2007.
12. S. Kapetanakis and D. Kudenko, Improving on the Reinforcement Learning of Coordination in Cooperative Multi-Agent Systems, In Proc. of the Int. Conf. on Autonomous Agents and Multiagent Systems, Imperial College, London, April 2002.
13. J. R. Kok and N. Vlassis, Collaborative multiagent reinforcement learning by payoff propagation, Journal of Machine Learning Research 7(2006), 1789-1828. (Pubitemid 44373693)
14. M. Lauer and M. Riedmiller, An Algorithm for Distributed Reinforcement Learning in Cooperative Multi-Agent Systems, In Proc. of the Int. Conf. on Machine Learning, pages 535-542. Morgan Kaufmann, 2000.
15. M. Lauer and M. Riedmiller, Reinforcement Learning for Stochastic Cooperative Multi-Agent Systems, In Proc. of the Int. Conf. on Autonomous Agents and Multiagent Systems, pages 1516-1517, 2004.
16. M. L. Littman, Value-function reinforcement learning in markov games, Journal of Cognitive Systems Research 2(1) (2001), 55-66. (Pubitemid 33718550)
17. L. Matignon, G. J. Laurent and N. Le Fort-Piat, Hysteretic Q-Learning: An Algorithm for Decentralized Reinforcement Learning in Cooperative Multi-Agent Teams, In Proc. of the IEEE Int. Conf. on Intelligent Robots and Systems, pages 64-69, San Diego, CA, USA, Oct. 29-Nov. 2 2007.
18. L. Matignon, G. J. Laurent and N. Le Fort-Piat, Coordination of independent learners in cooperative markov games, Technical Report RR-2009-01, Institut FEMTO-ST/UFC-ENSMM-UTBMCNRS, Besançon, France, March 2009. http://hal.archives-ouvertes.fr/hal-00370889/fr/.
19. L. Matignon, G. J. Laurent, N. L. Fort-Piat and Y.-A. Chapuis, Designing decentralized controllers for distributed-airjet mems-based micromanipulators by reinforcement learning, Journal of Intelligent and Robotic Systems 59(2) (2010), 145-166.
20. F. S. Melo and M. C. Lopes, Convergence of independent adaptive learners. In Progress in Artificial Intelligence: 13th Portuguese Conf. on Artificial Intelligence, Lecture Notes in Artificial Intelligence, volume 4874, pages 555-567. Springer-Verlag, 2007.
21. J. E. Munoz, A. Lazaric and M. Restelli, Learning to Cooperate in Multi-Agent Social Dilemmas, In Proc. of the Int. Conf. on Autonomous Agents and Multiagent Systems, pages 783-785, 2006. (Pubitemid 46609551)
22. J. Von Neumann and O. Morgenstern, Theory of Games and Economic Behavior. John Wiley and Sons, 1944.
23. M. J. Osborne and A. Rubinstein, A Course in Game Theory. The MIT Press, 1994.
24. L. Panait and S. Luke, Cooperative multi-agent learning: The state of the art, Autonomous Agents and Multi-Agent Systems 11(3) (2005), 387-434. (Pubitemid 41425094)
25. L. Panait, K. Tuyls and S. Luke, Theoretical advantages of lenient learners: An evolutionary game theoretic perspective, Journal of Machine Learning Research 9(2008), 423-457. (Pubitemid 351469016)
26. T. Sandholm, Perspectives on multiagent learning, Artificial Intelligence 171(2007), 382-392.
27. I. Sen, M. Sekaran and J. Hale, Learning to Coordinate Without Sharing Information, In Proc. of the National Conference on Artificial Intelligence, pages 426-431. AAAI, 1994.
28. L. S. Shapley, Stochastic games, PNAS 39(1953), 1095-1100, Reprinted in (Kuhn, 1997).
29. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, The MIT Press, Cambridge, 1998.
30. M. Tan, Multiagent Reinforcement Learning: Independent vs. Cooperative Agents, In Proc. of the Int. Conf. on Machine Learning, pages 330-337, 1993.
31. G. Tesauro, Reinforcement learning in autonomic computing: A manifesto and case studies, IEEE Internet Computing 11(2) (2007), 22-30. (Pubitemid 46335538)
32. K. Tuyls, P. Jan, T. Hoen and B. Vanschoenwinkel, An evolutionary dynamical analysis of multi-agent learning in iterated games, Autonomous Agents and Multi-Agent Systems 12(2006), 115-153. (Pubitemid 43146342)
33. K. Verbeeck, A. Nowé, J. Parent and K. Tuyls, Exploring selfish reinforcement learning in repeated games with stochastic rewards, Proc of the Int Conf on Autonomous Agents and Multiagent Systems 14(3) (2007), 239-269.
34. K. Verbeeck, A. Nowé, M. Peeters and K. Tuyls, Multi-agent reinforcement learning in stochastic single and multi-stage games. Lecture Notes in Computer Science, Adaptive Agents and Multi-Agent Systems III 3394(2005), 275-294.
35. N. Vlassis, A Concise Introduction to Multiagent Systems and Distributed Artificial Intelligence, In Ronald Brachman and Thomas Dietterich, editors, Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool, 2007.
36. Y. Wang and C. W. de Silva, A machine-learning approach to multi-robot coordination, Engineering Applications of Artificial Intelligence 21(3) (2008), 470-484. (Pubitemid 351680683)
37. C. J. C. H. Watkins and P. Dayan, Technical note: Q-learning, Machine Learning 8(1992), 279-292.
38. M. Weinberg and J. Rosenschein, Best-Response Multiagent Learning in Non-Stationary Environments, In Proc. of the Int. Conf. on Autonomous Agents and Multiagent Systems, pages 506-513, 2004.
39. E. Yang and D. Gu, Multiagent reinforcement learning for multirobot systems: A survey. Technical report, Department of Computer Science, University of Essex, 2004.
40. H. P. Young, The possible and the impossible in multi-agent learning, Artificial Intelligence 171(2007), 429-433. (Pubitemid 46802420)
41. Z. Zheng, M. Shu-gen, C. Bing-gang, Z. Li-Ping and L. Bin, Multiagent Reinforcement Learning for a Planetary Exploration Multirobot System, In Proc. of the Int. Conf. on Autonomous Agents and Multiagent Systems, pages 339-350, 2006.