A survey on multiagent reinforcement learning towards multi-robot systems

IEEE 2005 Symposium on Computational Intelligence and Games, CIG'05

Volumn , Issue , 2005, Pages 292-299

  Yang, Erfu ^a   Gu, Dongbing ^a  

^a UNIVERSITY OF ESSEX   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

MULTI-AGENT REINFORCEMENT LEARNING; MULTI-ROBOT SYSTEMS; SCALING-UP; THEORETICAL RESEARCH;

INDUSTRIAL ROBOTS; INTELLIGENT ROBOTS; MULTI AGENT SYSTEMS; MULTIPURPOSE ROBOTS; REINFORCEMENT; ROBOT LEARNING; SURVEYS;

REINFORCEMENT LEARNING;

EID: 65149099581     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (63)

1. Y. U. Cao, A. S. Fukunaga, and A. B. Kahng, "Cooperative mobile robotics: antecedents and directions, " Auton. Robot., vol. 4, pp. 1-23, 1997.
2. M. J. Matarić, "Reinforcement learning in the multi-robot domain, " Auton. Robots, vol. 4, pp. 73-83, 1997. (Pubitemid 127508276)
3. F. Michaud and M. Matarić, "Learning from history for behavior-based mobile robots in non-stationary conditions, " Auton. Robots, vol. 5, pp. 335-354, 1998. (Pubitemid 128512026)
4. T. Balch and R. C. Arkin, "Behavior-based formation control for multirobot teams, " IEEE Trans. Robot. Automat., vol. 14, no. 6, pp. 926-939, 1998. (Pubitemid 128743571)
5. M. Asada, E. Uchibe, and K. Hosoda, "Co-operative behaviour acquisition for mobile robots in dynamically changing real worlds via vision-based reinforcement learning and development, " Art. Intel., vol. 110, pp. 275-292, 1999.
6. M.Wiering, R. Salustowicz, and J. Schmidhuber, "Reinforcement learning soccer teams with incomplete world models, " Auton. Robots, vol. 7, pp. 77-88, 1999.
7. S. V. Zwaan, J. A. A. Moreira, and P. U. Lima, "Cooperative learning and planning for multiple robots, " 2000. [Online]. Available: citeseer.nj.nec.com/299778.html.
8. C. F. Touzet, "Robot awareness in cooperative mobile robot learning, " Auton. Robots, vol. 8, pp. 87-97, 2000.
9. F. Fernandez and L. E. Parker, "Learning in large co-operative multi-robot domains, " Int. J. Robot. Automat., vol. 16, no. 4, pp. 217-226, 2001. (Pubitemid 34133699)
10. J. Liu and J. Wu, Multi-Agent Robotic Systems. CRC Press, 2001.
11. M. J. Matarić, "Learning in behavior-based multi-robot systems: policies, models, and other agents, " J. Cogn. Syst. Res., vol. 2, pp. 81-93, 2001. (Pubitemid 33718552)
12. M. Bowling and M. Veloso, "Simultaneous adversarial multi-robot learning, " in Proc. 8th Int. Joint Conf. Artificial Intelligence, August 2003.
13. I. H. Elhajj, A. Goradia, N. Xi, and et al, "Design and analysis of internet-based tele-coordinated multi-robot systems, " Auton. Robots, vol. 15, pp. 237-254, 2003.
14. L. Iocchi, D. Nardi, M. Piaggio, and et al, "Distributed coordination in heterogeneous multi-robot systems, " Auton. Robots, vol. 15, pp. 155-168, 2003.
15. M. J. Mataric, G. S. Sukhatme, and E. H. Østergaard, "Multi-robot task allocation in uncertain environments, " Auton. Robots, vol. 14, pp. 255-263, 2003.
16. C. F. Touzet, "Distributed lazy Q-learning for cooperative mobile robots, " Int. J. Advanced Robot. Syst., vol. 1, no. 1, pp. 5-13, 2004.
17. M. L. Littman, "Markov games as a framework for multi-agent learning, " in Proc. 11th Int. Conf. Machine Learning, San Francisco, 1994, pp. 157-163.
18. M. L. Littman and C. Szepesvári, "A generalized reinforcement-learning model: convergence and applications, " in Proc. 13th Int. Conf. Machine Learning, Bari, Italy, July 3-6 1996, pp. 310-318.
19. C. Szepesvári and M. L. Littman, "Generalized markov decision processes: dynamic-programming and reinforcement-learning algorithms, " Department of Computer Science, Brown University, Technical report CS-96-11, 1996.
20. C. Claus and C. Boutilier, "The dynamics of reinforcement learning in coopertive multiagent systems, " in Proc. 15th National Conf. Artificial Intelligence, Madison, WI, 1998, pp. 746-752.
21. J. Hu and M. P. Wellman, "Multiagent reinforcement learning in stochastic games, " 1999. [Online]. Availableciteseer.ist.psu.edu/ hu99multiagent.html.
22. R. Sun and D. Qi, "Rationality assumptions and optimality of co-learning, " Lecture Notes in Computer Science, vol. 1881. Springer, 2000, pp. 61-75.
23. C. Szepesvári and M. L. Littman, "A unified analysis of value-function-based reinforcement learning algorithms, " Neur. Comput., vol. 11, no. 8, pp. 2017- 2059, 1999.
24. P. Stone and M. Veloso, "Multiagent systems: a survey from a machine learning perspective, " Auton. Robots, vol. 8, pp. 345-383, 2000.
25. J. Hu and M. P.Wellman, "Learning about other agents in a dynamic multiagent system, " J. Cogn. Syst. Res., vol. 2, pp. 67-79, 2001. (Pubitemid 33718551)
26. M. L. Littman, "Value-function reinforcement learning in markov games, " J. Cogn. Syst. Res., vol. 2, pp. 55-66, 2001. (Pubitemid 33718550)
27. M. L. Littman and P. Stone, "Leading best-response strategies in repeated games, " in 17th Ann. Int. Joint Conf. Artificial Intelligence Work. Econ. Agents, Models, and Mechanism, 2001.
28. M. L. Littman, "Friend-or-foe Q-learning in general-sum games, " in Proc. 18th Int. Conf. Machine Learning, Morgan Kaufman, 2001, pp. 322-328.
29. F. A. Dahl, "The lagging anchor algorithm: reinforcement learning in twoplayer zero-sum games with imperfect information, " Mach. Learn., vol. 49, pp. 5-37, 2002. (Pubitemid 34325693)
30. M. N. Ahmadabali and M. Asadpour, "Expertness based cooperative Qlearning, " IEEE Trans. Syst., Man, and Cyber.-Part B: Cybernetics, vol. 32, no. 1, pp. 66-76, February 2002.
31. G. Chalkiadakis, "Multiagent reinforcement learning: stochastic games with multiple learning players, " Department of Computer Science, University of Toronto, " Technical report, 2003.
32. N. Suematsu and A. Hayashi, "A multiagent reinforcement learning algorithm using extended optimal response, " in Proc. 1st Int. Joint Conf. Auton. Agents & Multiagent Syst., Bologna, Italy, July 15-19 2002, pp. 370-377. (Pubitemid 34975488)
33. G. Chalkiadakis and C. Boutilier, "Multiagent reinforcement learning: theoretical framework and an algorithm, " in 2nd Int. Joint Conf. Auton. Agents & Multiagent Syst., Melbourne, Australia, July 14-18 2003, pp. 709-716.
34. J. Hu and M. P.Wellman, "Nash Q-learning for general-sum stochastic games, " J. Mach. Learn. Res., vol. 4, pp. 1039-1069, 2003.
35. M. Wahab, "Reinforcement learning in multiagent systems, " 2003. [Online]. Available: http://www.cs.mcgill.ca/̃mwahab/RL%20in%20MAS. pdf.
36. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, Massachusetts: MIT Press, 1998.
37. C. J. Watkins and P. Dayan, "Q-learning, " Mach. Learn., vol. 8, pp. 279-292, 1992.
38. T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory. London: Academic Press, 1982.
39. C. Boutilier, "Sequential optimality and coordination in multiagent systems, " in Proc. 16th Int. Joint Conf. Artificial Intelligence, 1999, pp. 478-485.
40. M. G. Lagoudakis and R. Parr, "Value function approximation in zero-sum markov games, " 2002. [Online]. Available: http://www.cs.duke.edu/ ̃mgl/papers/PDF/uai2002.pdf.
41. X. Li, "Refining basis functions in least-square approximation of zero-sum markov games, " 2003. [Online]. Available: http://www.xiaolei.org/ research/li03basis.pdf.
42. Y. Shoham and R. Powers, "Multi-agent reinforcement learning: a critical survey, " 2003. [Online]. Available: http://www.stanford.edu/ ̃grenager/MALearning?ACriticalSurvey?2003?0516.%pdf.
43. J. Hu and M. P. Wellman, "Multiagent reinforcement learning: theoretical framework and an algorithm, " in Proc. the 15th Int. Conf. Machine Learning, San Francisco, California, 1998, pp. 242-250.
44. M. Bowling, "Multiagent learning in the presence of agents with limitations, " Ph.D. dissertation, School of Computer Science, Carnegie Mellon University, Pittsburgh, May 2003.
45. M. H. Bowling and M. M. Veloso, "Multiagent learning using a variable learning rate, " Art. Intell., vol. 136, no. 2, pp. 215-250, 2002.
46. B. Banerjee and J. Peng, "Convergent gradient ascent in general-sum games, " in Proc. 13th Europ. Conf. Mach. Learn., August 13-19 2002, pp. 686-692.
47. B. Banerjee and J. Peng, "Adaptive policy gradient in multiagent learning, " in Proc. 2nd int. joint conf. Auton. agents and multiagent systems. ACM Press, 2003, pp. 686-692.
48. R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour, "Policy gradient methods for reinforcement learning with function approximation, " in Advanc. Neur. Inf. Proc. Syst.. MIT Press, 12, pp. 1057-1063.
49. E. F. Morales, "Scaling up reinforcement learning with a relational representation, " in Workshop Adaptabil. Multi-Agent Syst., Sydney, 2003.
50. D. Fudenberg and D. K. Levine, The Theory of Learning in Games. Cambridge, Massachusetts: MIT Press, 1999.
51. S. Sen and M. Sekaran, "Multiagent coordination with learning classifier systems, " [Online]. Available: citeseer.ist.psu.edu/ sen96multiagent.html.
52. S. Valluri and S. Babu, "Reinforcement learning for keepaway soccer problem, " 2002. [Online]. Available: http://www.cis.ksu.edu/̃babu/ final/html/ProjectReport.htm.
53. K. Kostiadis and H. Hu, "KaBaGe-RL: kanerva-based generalisation and reinforcement learning for possession football, " in Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, Hawaii, 2001.
54. K.-H. Park, Y.-J. Kim, and J.-H. Kim, "Modular Q-learning based multi-agent cooperation for robot soccer, " Robot. Auton. Syst., vol. 35, pp. 109-122, 2001. (Pubitemid 32427408)
55. H. R. Berenji and D. A. Vengerov, "Cooperation and coordination between fuzzy reinforcement learning agents in continuous-state partially observable markov decision processes, " in Proc. 8th IEEE Int. Conf. Fuzzy Systems, 2000.
56. H. R. Berenji and D. A. Vengerov, "Advantages of cooperation between reinforcement learning agents in difficult stochastic problems, " in Proc. 9th IEEE Int. Conf. Fuzzy Systems, 2000.
57. H. R. Berenji and D. Vengerov, "Generalized markov decision processes: dynamic-programming and reinforcement-learning algorithms, " [Online]. Available: http://www.iiscorp.com/projects/multi-agent/tech?rep?iis? 00?10.pdf.
58. I. Gültekin and A. Arslan, "Modular-fuzzy cooperative algorithm for multiagent systems, " Lecture Notes in Computer Science, vol. 2457. Springer, 2002, pp. 255-263.
59. A. Kilic and A. Arslan, "Minimax fuzzy Q-learning in cooperative multi-agent systems, " Lecture Notes in Computer Science, vol. 2457. Springer, 2002, pp. 264-272.
60. H. R. Berenji and D. Vengerov, "A convergent Actor-Critic-based FRL algorithm with application to power management of wireless transmitters, " IEEE Trans. Fuzz. Syst., vol. 11, no. 4, pp. 478-485, August 2003.
61. R. Munos, "A study of reinforcement learning in the continuous case by the means of viscosity solutions, " Mach. Learn., vol. 40, pp. 265-299, 2000.
62. O. Abul, F. Polat, and R. Alhajj, "Multiagent reinforcement learning using function approximation, " IEEE Trans. Syst., Man, and Cybern.-Part C: Appli.n and Rev., vol. 30, no. 4, pp. 485-497, 2000. (Pubitemid 32190905)
63. V. Tadić, "On the convergence of temporal-difference learning with linear function approximation, " Mach. Learn., vol. 42, pp. 241-267, 2001. (Pubitemid 32188797)