Learning from actions not taken in multiagent systems

Advances in Complex Systems

Volumn 12, Issue 4-5, 2009, Pages 455-473

  Tumer, Kagan ^a   Khani, Newsha ^a  

^a OREGON STATE UNIVERSITY   (United States)

Author keywords

Counterfactual reward; Difference reward; Multiagent learning

Indexed keywords

EID: 70349592320     PISSN: 02195259     EISSN: None     Source Type: Journal    
DOI: 10.1142/s0219525909002301     Document Type: Article

References (45)

1. Agogino, A. K. and Tumer, K., Handling communication restrictions and team formation in congestion games, J. Auton. Agents Multi Agent Syst. 13 (2006) 97-115.
2. Agogino, A. K. and Tumer, K., Analyzing and visualizing multiagent rewards in dynamic and stochastic environments, J. Auton. Agents Multi Agent Syst. 17 (2008) 320-338.
3. Agogino, A. K. and Tumer, K., Efficient evaluation functions for evolving coordination, Evol. Comput. 16 (2008) 257-288.
4. Arai, S., Sycara, K. and Payne, T., Multi-agent reinforcement learning for planning and scheduling multiple goals, in Proc. Fourth Int. Conf. on Multiagent Syst. (2000), pp. 359-360.
5. Arthur, W. B., Complexity in economic theory: Inductive reasoning and bounded rationality, Am. Econ. Rev. 84 (1994) 406-411.
6. Chalkiadakis, G. and Boutilier, C., Coordination in multiagent reinforcement learning: A bayesian approach, in Proc. Second Int. Joint Conf. on Autonomous Agents and Multiagent Systems (AAMAS-03) (Melbourne, Australia, 2003).
7. Challet, D. and Zhang, Y. C., On the minority game: Analytical and numerical studies, Physica A 256 (1998) 514.
8. Claus, C. and Boutilier, C., The dynamics of reinforcement learning cooperative multiagent systems, in Proc. Fifteenth National Conf. on Artificial Intelligence (Madison, WI, 1998), pp. 746-752.
9. de Cara, M. A. R., Pla, O. and Guinea, F., Competition, efficiency and collective behavior in the "El Farol" bar model, Eur. Phys. J. B 10 (1999) 187.
10. Dietterich, T. G., Hierarchical reinforcement learning with the MAXQ value function decomposition, J. Artif. Intell. 13 (2000) 227-303.
11. Guestrin, C., Lagoudakis, M. and Parr, R., Coordinated reinforcement learning, in Proc. 19th Int. Conf. on Machine Learning (2002).
12. Hu, J. andWellman, M. P., Multiagent reinforcement learning: Theoretical framework and an algorithm, in Proc. Fifteenth Int. Conf. on Machine Learning (1998), pp. 242- 250.
13. Hu, J. and Wellman, M. P., Online learning about other agents in a dynamic multiagent system, in Proc. Second Int. Conf. on Autonomous Agents (1998), pp. 239-246.
14. Jefferies, P., Hart, M. L. and Johnson, N. F., Deterministic dynamics in the minority game, Phys. Rev. E 65(016105) (2002).
15. Jennings, N. R., Sycara, K. and Wooldridge, M., A roadmap of agent research and development, Auton. Agents Multi-Agent Syst. 1 (1998) 7-38.
16. Kaelbling, L. P., Littman, M. L. and Moore, A.W., Reinforcement learning: A survey, J. Artif. Intell. Res. 4 (1996) 237-285.
17. Khani, N. and Tumer, K., Fast multiagent learning: Cashing in on team knowledge, in Artificial Neural Networks in Engineering (ASME, St. Louis, 2008), pp. 3-10.
18. Klügl, F., Bazzan, A. and Ossowski, S. (eds.), Applications of Agent Technology in Traffic and Transportation (Springer, 2005).
19. Mataric, M. J., Coordination and learning in multi-robot systems, in IEEE Intelligent Systems (1998), pp. 6-8.
20. McGlohon, M. and Sen, S., Learning to cooperate in multi-agent systems by combining Q-learning and evolutionary strategy, Int. J. Lateral Comput. 1 (2005) 58-64.
21. Narendra, K. S. and Thathachar, M. A. L., Learning Automata: An Introduction (Prentice Hall, 1989).
22. Panait, L., Tuyls, K. and Luke, S., Theoretical advantages of lenient learners: An evolutionary game theoretic perspective, J. Mach. Learn. Res. 9 (2008) 423-457.
23. Parkes, D., On learnable mechanism design, in Collectives and the Design of Complex Systems (Springer, 2004).
24. Pynadath, D. and Tambe, M., The communicative multiagent team decision problem: Analyzing teamwork theories and models, J. Artif. Intell. Res. 16 (2002) 389-423.
25. Sherstov, A. and Stone, P., Three automated stock-trading agents: A comparative study, in Agent Mediated Electronic Commerce VI: Theories for and Engineering of Distributed Mechanisms and Systems (AMEC 2004), Lecture Notes in Artificial Intelligence (Springer Verlag, Berlin, 2005), pp. 173-187.
26. Stone, P., Layered Learning in Multi-Agent Systems: A Winning Approach to Robotic Soccer (MIT Press, Cambridge, MA, 2000).
27. Stone, P., Sutton, R. S. and Kuhlmann, G., Reinforcement learning for RoboCupsoccer keepaway, Adapt. Behav. (2005).
28. Sutton, R. S. and Barto, A. G., Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA, 1998).
29. Tambe, M., Towards flexible teamwork, J. Artif. Intell. Res. 7 (1997) 83-124.
30. Taylor, M. E., Whiteson, S. and Stone, P., Comparing evolutionary and temporal difference methods for reinforcement learning, in Proc. Genetic and Evolutionary Computation Conf. (Seattle, WA, 2006), pp. 1321-1328.
31. Tesauro, G., Practical issues in temporal difference learning, in Advances in Neural Information Processing Systems, Vol.4, eds. Moody, J., Hanson, S. and Lippmann, R. (Morgan Kaufmann, 1992), pp. 259-266.
32. Tumer, K. and Agogino, A., Distributed agent-based air traffic flow management, in Proc. Sixth Int. Joint Conf. on Autonomous Agents and Multi-Agent Systems (Honolulu, HI, 2007), pp. 330-337.
33. Tumer, K., Agogino, A. and Wolpert, D., Learning sequences of actions in collectives of autonomous agents, in Proc. First Int. Joint Conf. on Autonomous Agents and Multi-Agent Systems (Bologna, Italy, 2002), pp. 378-385.
34. Tumer, K., Welch, Z. T. and Agogino, A., Aligning social welfare and agent preferences to alleviate traffic congestion, in Proc. Seventh Int. Joint Conf. on Autonomous Agents and Multi-Agent Systems (Estoril, Portugal, 2008).
35. Tumer, K. and Wolpert, D. (eds.), Collectives and the Design of Complex Systems (Springer, New York, 2004).
36. Tumer, K. and Wolpert, D. H., Collective intelligence and Braess' paradox, in Proc. Seventeenth National Conf. on Artificial Intelligence (Austin, TX, 2000), pp. 104- 109.
37. Tuyls, K. and Parsons, S., What evolutionary game theory tells us about multiagent learning, Artif. Intell. 171 (2007) 406-416.
38. Verbeeck, K., Nowe, A. and Tuyls, K., Coordinated exploration in multi-agent reinforcement learning: An application to load balancing, in Proc. Fourth Int. Joint Conf. on Autonomous Agents and Multi-Agent Systems (Utrecht, The Netherlands, 2005).
39. Verbeeck, K., Peeters, M., Nowe, A. and Tuyls, K., Reinforcement learning in stochastic single and multi-stage games, in Adaptive Agents and Multi-Agent Systems II, Lecture Notes in Artificial Intelligence (Springer Verlag, Berlin, 2005), pp. 275-294.
40. Vidal, J. M., Multiagent coordination using a distributed combinatorial auction, in AAAI Workshop on Auction Mechanism for Robot Coordination (2006).
41. Vidal, J. M. and Durfee, E. H., The moving target function problem in multi-agent learning, in Proc. Third Int. Conf. on Multi-Agent Systems (AAAI/MIT press, 1998), pp. 317-324.
42. Watkins, C. and Dayan, P., Q-learning, Mach. Learn. 8 (1992) 279-292.
43. Wellman, M. P., Cheng, S.-F., Reeves, D. M. and Lochne, K. M., Trading agents competing: Performance, progress, and market effectiveness, IEEE Intell. Syst. 18 (2003) 48-53.
44. Whiteson, S., Taylor, M. E. and Stone, P., Empirical studies in action selection for reinforcement learning, Adapt. Behav. 15 (2007).
45. Wolpert, D. H. and Tumer, K., Optimal reward functions for members of collectives, Adv. Complex Syst. 4 (2001) 265-279.