Perspectives on multiagent learning

Artificial Intelligence

Volumn 171, Issue 7, 2007, Pages 382-391

  Sandholm, Tuomas ^a  

^a Carnegie Mellon University   (United States)

Author keywords

Game theory; Learning in games; Multiagent learning; Reinforcement learning

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; GAME THEORY; REINFORCEMENT LEARNING;

LEARNING IN GAMES; MULTIAGENT LEARNING;

MULTI AGENT SYSTEMS;

EID: 34249045960     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.artint.2007.02.004     Document Type: Article

References (74)

1. T. Abbott, D. Kane, P. Valiant, On the complexity of two-player win-lose games, in: Symposium on Foundations of Computer Science, 2005
2. Auer P., Cesa-Bianchi N., Freund Y., and Schapire R.E. The nonstochastic multiarmed bandit problem. SIAM Journal of Computing 32 (2002) 48-77
3. B. Banerjee, J. Peng, Performance bounded reinforcement learning in strategic interactions, in: National Conf. on Artificial Intelligence, 2004
4. B. Banerjee, S. Sen, J. Peng, Fast concurrent reinforcement learners, in: Internat. Joint Conf. on Artificial Intelligence, 2001
5. M. Benisch, G. Davis, T. Sandholm, Algorithms for rationalizability and CURB sets, in: National Conf. on Artificial Intelligence, 2006
6. A. Blum, E. Even-Dar, K. Ligett, Routing without regret: On convergence to Nash equilibria of regret-minimizing algorithms in routing games, in: ACM Symposium on Principles of Distributed Computing, 2006
7. M. Bowling, Convergence and no-regret in multiagent learning, in: Conf. on Neural Information Processing Systems, 2005
8. Bowling M., and Veloso M. Multiagent learning using a variable learning rate. Artificial Intelligence 136 (2002) 215-250
9. Brafman R., and Tennenholtz M. A near-optimal polynomial time algorithm for learning in certain classes of stochastic games. Artificial Intelligence 121 (2000) 31-47
10. Brafman R., and Tennenholtz M. Learning to coordinate efficiently: A model-based approach. Journal of Artificial Intelligence Research 19 (2003) 11-23
11. Brafman R., and Tennenholtz M. R-max-a general polynomial time algorithm for near-optimal reinforcement learning. Journal of Machine Learning Research 3 (2003) 213-231
12. Brafman R., and Tennenholtz M. Efficient learning equilibrium. Artificial Intelligence 159 (2004) 27-47 Earlier version in NIPS-02
13. R. Brafman, M. Tennenholtz, Optimal efficient learning equilibrium: Imperfect monitoring in symmetric games, in: National Conf. on Artificial Intelligence, 2005
14. Y.-H. Chang, T. Ho, L. Kaelbling, Mobilized ad-hoc networks: A reinforcement learning approach, in: Internat. Conf. on Autonomic Computing, 2004
15. X. Chen, X. Deng, Settling the complexity of 2-player Nash equilibrium, in: Electronic Colloquium on Computational Complexity, Report No. 150, 2005
16. C. Claus, C. Boutilier, The dynamics of reinforcement learning in cooperative multiagent systems, in: National Conf. on Artificial Intelligence, 1998
17. V. Conitzer, T. Sandholm, BL-WoLF: A framework for loss-bounded learnability in zero-sum games, in: Internat. Conf. on Machine Learning, 2003
18. V. Conitzer, T. Sandholm, Complexity results about Nash equilibria, in: Internat. Joint Conf. on Artificial Intelligence, 2003
19. V. Conitzer, T. Sandholm, Communication complexity as a lower bound for learning in games, in: Internat. Conf. on Machine Learning, 2004
20. V. Conitzer, T. Sandholm, Complexity of (iterated) dominance, in: ACM Conf. on Electronic Commerce, 2005
21. V. Conitzer, T. Sandholm, A generalized strategy eliminability criterion and computational methods for applying it, in: National Conf. on Artificial Intelligence, 2005
22. Conitzer V., and Sandholm T. AWESOME: A general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents. Special issue on Learning and Computational Game Theory. Machine Learning 67 (2007) 23-43 Short version in ICML-03
23. V. Conitzer, T. Sandholm, Computing the optimal strategy to commit to, in: ACM Conf. on Electronic Commerce, 2006
24. A. Flaxman, A. Kalai, B. McMahan, Online convex optimization in the bandit setting: Gradient descent without a gradient, in: ACM-SIAM Symposium on Discrete Algorithms, 2005
25. Foster D.P., and Young H.P. On the impossibility of predicting the behavior of rational agents. Proceedings of the National Academy of Sciences 98 (2001) 12848-12853
26. Freund Y., and Schapire R. Adaptive game playing using multiplicative weights. Games and Economic Behavior 29 (1999) 79-103
27. Fudenberg D., and Levine D. The Theory of Learning in Games (1998), MIT Press
28. Fudenberg D., and Levine D.K. Consistency and cautious fictitious play. Journal of Economic Dynamics and Control 19 (1995) 1065-1089
29. Gilboa I., Kalai E., and Zemel E. The complexity of eliminating dominated strategies. Mathematics of Operation Research 18 (1993) 553-565
30. Gilboa I., and Zemel E. Nash and correlated equilibria: Some complexity considerations. Games and Economic Behavior 1 (1989) 80-93
31. A. Gilpin, S. Hoda, J. Peña, T. Sandholm, Gradient-based algorithms for finding Nash equilibria in extensive form games, Mimeo, 2007
32. A. Gilpin, T. Sandholm, Finding equilibria in large sequential games of imperfect information, in: ACM Conf. on Electronic Commerce, 2006
33. A. Greenwald, K. Hall, Correlated Q-learning, in: Internat. Conf. on Machine Learning, 2003
34. A. Greenwald, A. Jafari, A general class of no-regret learning algorithms and game-theoretic equilibria, in: Conf. on Learning Theory, 2003
35. S. Hart, Y. Mansour, The communication complexity of uncoupled Nash equilibrium procedures, 2006, Draft
36. Hart S., and Mas-Colell A. A simple adaptive procedure leading to correlated equilibrium. Econometrica 68 (2000) 1127-1150
37. Hart S., and Mas-Colell A. Uncoupled dynamics do not lead to Nash equilibrium. American Economic Review 93 (2003) 1830-1836
38. Hu J., and Wellman M.P. Nash Q-learning for general-sum stochastic games. Journal of Machine Learning Research 4 (2003) 1039-1069
39. A. Jafari, A. Greenwald, D. Gondek, G. Ercal, On no-regret learning, fictitious play, and Nash equilibrium, in: Internat. Conf. on Machine Learning, 2001
40. Kalai A., and Vempala S. Efficient algorithms for online decision problems. Journal of Computer and System Sciences 71 (2005) 291-307
41. Kalai E., and Lehrer E. Rational learning leads to Nash equilibrium. Econometrica 61 5 (1993) 1019-1045
42. M. Kearns, M. Littman, S. Singh, Graphical models for game theory, in: Conf. on Uncertainty in Artificial Intelligence, 2001
43. Knuth D.E., Papadimitriou C.H., and Tsitsiklis J.N. A note on strategy elimination in bimatrix games. Operations Research Letters 7 (1988) 103-107
44. Lemke C., and Howson J. Equilibrium points of bimatrix games. Journal of the Society of Industrial and Applied Mathematics 12 (1964) 413-423
45. Littlestone N., and Warmuth M.K. The weighted majority algorithm. Information and Computation 108 2 (1994) 212-261
46. M. Littman, Markov games as a framework for multi-agent reinforcement learning, in: Internat. Conf. on Machine Learning, 1994
47. M. Littman, Friend or foe Q-learning in general-sum Markov games, in: Internat. Conf. on Machine Learning, 2001
48. Littman M. Value-function reinforcement learning in Markov games. Journal of Cognitive Systems Research 2 (2001) 55-66
49. M. Littman, C. Szepesvári, A generalized reinforcement-learning model: Convergence and applications, in: Internat. Conf. on Machine Learning, 1996
50. Mannor S., and Shimkin N. The empirical Bayes envelope and regret minimization in competitive Markov decision processes. Mathematics of Operations Research 28 2 (2003) 327-345
51. P. McCracken, M. Bowling, Safe strategies for agent modelling in games, in: AAAI Fall Symposium on Artificial Multi-agent Learning, 2004
52. B. McMahan, A. Blum, Online geometric optimization in the bandit setting against an adaptive adversary, in: Conf. on Learning Theory, 2004
53. Nachbar J. Prediction, optimization, and learning in games. Econometrica 65 (1997) 275-309
54. Nachbar J. Bayesian learning in repeated games of incomplete information. Social Choice and Welfare 18 (2001) 303-326
55. Narendra K.S., and Thathachar M.A.L. Learning Automata: An Introduction (1989), Prentice Hall
56. C. Papadimitriou, T. Roughgarden, Computing equilibria in multi-player games, in: Symposium on Discrete Algorithms, 2005
57. K. Pivazyan, Y. Shoham, Polynomial-time reinforcement learning of near-optimal policies, in: National Conf. on Artificial Intelligence, 2002
58. R. Porter, E. Nudelman, Y. Shoham, Simple search methods for finding a Nash equilibrium, in: National Conf. on Artificial Intelligence, 2004
59. R. Powers, Y. Shoham, Learning against opponents with bounded memory, in: Internat. Joint Conf. on Artificial Intelligence, 2005
60. R. Powers, Y. Shoham, New criteria and a new algorithm for learning in multi-agent systems, in: Conf. on Neural Information Processing Systems, 2005
61. Sandholm T., and Crites R. Multiagent reinforcement learning in the iterated prisoner's dilemma. Biosystems 37 (1996) 147-166 special issue on the Prisoner's Dilemma. Early version in IJCAI-95 Workshop on Adaptation and Learning in Multiagent Systems
62. T. Sandholm, A. Gilpin, V. Conitzer, Mixed-integer programming methods for finding Nash equilibria, in: National Conf. on Artificial Intelligence, 2005
63. T. Sandholm, M.V. Nagendra Prasad, Learning pursuit strategies, Project for CmpSci 698 Machine Learning, Computer Science Department, University of Massachusetts at Amherst, Spring, 1993
64. J. Schaeffer, Y. Björnsson, N. Burch, A. Kishimoto, M. Müller, R. Lake, P. Lu, S. Sutphen, Solving checkers, in: Internat. Joint Conf. on Artificial Intelligence, 2005
65. S. Singh, M. Kearns, Y. Mansour, Nash convergence of gradient dynamics in general-sum games, in: Conf. on Uncertainty in Artificial Intelligence, 2000
66. F. Southey, M. Bowling, B. Larson, C. Piccione, N. Burch, D. Billings, C. Rayner, Bayes' bluff: Opponent modelling in poker, in: Conf. on Uncertainty in Artificial Intelligence, 2005
67. Stone P., and Veloso M. Towards collaborative and adversarial learning: A case study in robotic soccer. International Journal of Human Computer Studies 48 (1998)
68. M. Tan, Multi-agent reinforcement learning: Independent vs. cooperative agents, in: Internat. Conf. on Machine Learning, 1993
69. Tesauro G. Temporal difference learning and TD-gammon. Communications of the ACM 38 3 (1995)
70. Tsetlin M.L. Automaton Theory and the Modelling of Biological Systems (1973), Academic Press
71. D. Vickrey, D. Koller, Multi-agent algorithms for solving graphical games, in: National Conf. on Artificial Intelligence, 2002
72. X. Wang, T. Sandholm, Reinforcement learning to play an optimal Nash equilibrium in team Markov games, in: Conf. on Neural Information Processing Systems, 2002
73. X. Wang, T. Sandholm, Learning near-Pareto-optimal conventions in polynomial time, in: Conf. on Neural Information Processing Systems, 2003
74. M. Zinkevich, Online convex programming and generalized infinitesimal gradient ascent, in: Internat. Conf. on Machine Learning, 2003