Fictitious self-play in extensive-form games

32nd International Conference on Machine Learning, ICML 2015

Volumn 1, Issue , 2015, Pages 805-813

  Heinrich, Johannes ^a   Lanctot, Marc ^b   Silver, David ^b  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b DEEPMIND   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; GAME THEORY;

COMPUTATIONAL REQUIREMENTS; EXTENSIVE-FORM GAMES; FICTITIOUS PLAY; GAME-THEORETIC MODEL; IMPERFECT INFORMATION; LARGE PROBLEMS; LEARNING IN GAMES; NASH EQUILIBRIA;

LEARNING SYSTEMS;

EID: 84969504570     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. Benaim, Michel, Hofbauer, Josef, and Sorin, Sylvain. Stochastic approximations and differential inclusions. SIAM Journal on Control and Optimization, 44(l):328-348, 2005.
2. Bosansky, B, Kiekintveld, Christopher, Lisy, V, and Pechoucek, Michal. An exact double-oracle algorithm for zero-sum extensive-form games with imperfect information. Journal of Artificial Intelligence Research, pp. 829-866, 2014.
3. Bowling, Michael, Burch, Neil, Johanson, Michael, and Tam-melin, Oskari. Heads-up limit holdem poker is solved. Science, 347(6218):I45-I49, 2015.
4. Brown, George W. Iterative solution of games by fictitious play. Activity analysis of production and allocation, 13(1 ):374-376, 1951.'
5. Daskalakis, Constantinos and Pan, Qinxuan. A counter-example to Karlin's strong conjecture for fictitious play. In Foundations of Computer Science (FOCS), 2014 IEEE 55th Annual Symposium on, pp. 11-20. IEEE, 2014.
6. Ernst, Damien, Geurts, Pierre, and Wehenkel, Louis. Tree-based batch mode reinforcement learning. In Journal of Machine Learning Research, pp. 503-556, 2005.
7. Fudenberg, Drew. The theory of learning in games, volume 2. MIT press, 1998.
8. Ganzfried, Sam and Sandholm, Tuomas. Computing equilibria in multiplayer stochastic games of imperfect information. In Proceedings of the list International Joint Conference on Artifical Intelligence, pp. 140-146, 2009.
9. Gilpin, Andrew, Hoda, Samid, Pena, Javier, and Sandholm, Tuomas. Gradient-based algorithms for finding Nash equilibria in extensive form games. In Internet and Network Economics, pp. 57-69. Springer, 2007.
10. Greenwald, Amy, Li, Jiacui, Sodomka, Eric, and Littman, Michael. Solving for best responses in extensive-form games using reinforcement learning methods. The 1st Multidisci-plinarv Conference on Reinforcement Learning and Decision Making (RLDM), 2013.
11. Heinrich, Johannes and Silver, David. Smooth UCT search in computer poker. In Proceedings of the 24th International Joint Conference on Artifical Intelligence, 2015. In press.
12. Hendon, Ebbe, Jacobsen, Hans J0rgen, and Sloth, Birgitte. Fictitious play in extensive form games. Games and Economic Behavior, 15(2): 177-202, 1996.
13. Hofbauer, Josef and Sandholm, William H. On the global convergence of stochastic fictitious play. Econometrica, 70(6):2265-2294, 2002.
14. Karlin, Samuel. Mathematical methods and theory in games, programming and economics. Addison-Wesley, 1959.
15. Roller, Daphne, Megiddo, Nimrod, and Von Stengel, Bernhard. Fast algorithms for finding randomized strategies in game trees. In Proceedings of the 26th ACM Symposium on Theory of Computing, pp. 750-759. ACM, 1994.
16. Roller, Daphne, Megiddo, Nimrod, and Von Stengel, Bernhard. Efficient computation of equilibria for extensive two-person games. Games and Economic Behavior, 14(2):247-259, 1996.
17. Ruhn, Harold W. Extensive games and the problem of information. Contributions to the Theory of Games, 2(28): 193-216, 1953.
18. Lambert III, Theodore J, Epelman, Marina A, and Smith, Robert L. A fictitious play approach to large-scale optimization. Operations Research, 53(3):477-489, 2005.
19. Lanctot, Marc, Waugh, Revin, Zinkevich, Martin, and Bowling, Michael. Monte Carlo sampling for regret minimization in extensive games. In Advances in Neural Information Processing Systems 22, pp. 1078-1086, 2009.
20. Leslie, David S and Collins, Edmund J. Generalised weakened fictitious play. Games and Economic Behavior, 56(2):285-298, 2006.
21. McMahan, H Brendan and Gordon, Geoffrey J. A fast bundle-based anytime algorithm for poker and other convex games. In International Conference on Artificial Intelligence and Statistics, pp. 323-330, 2007.
22. Miltersen, Peter Bro and Sorensen, Troels Bjerre. Computing a quasi-perfect equilibrium of a two-player game. Economic Theory, 42(1): 175-192, 2010.
23. Myerson, Roger B. Game Theory: Analysis of Conflict. Harvard University Press, 1991.
24. Robinson, Julia. An iterative method of solving a game. Annals of Mathematics, pp. 296-301, 1951.
25. Sandholm, Tuomas. The state of solving large incomplete-information games, and application to poker. AI Magazine, 31(4):13-32, 2010.
26. Silver, David and Veness, Joel. Monte-Carlo planning in large POMDPs. In Advances in Neural Information Processing Systems, pp. 2164-2172, 2010.
27. Southey, Finnegan, Bowling, Michael, Larson, Bryce, Piccione, Carmelo, Burch, Neil, Billings, Darse, and Rayner, Chris. Bayes bluff: Opponent modelling in poker. In In Proceedings of the 21 st Annual Conference on Uncertainty in Artificial Intelligence (UAI, pp. 550-558, 2005.
28. Sutton, Richard S and Barto, Andrew G. Reinforcement learning: An introduction, volume 1. Cambridge Univ Press, 1998.
29. Von Stengel, Bernhard. Efficient computation of behavior strategies. Games and Economic Behavior, 14(2):220-246, 1996.
30. Watkins, Christopher JCH and Dayan, Peter. Q-learning. Machine learning, 8(3-4):279-292, 1992.
31. Zinkevich, Martin, Johanson, Michael, Bowling, Michael, and Piccione, Carmelo. Regret minimization in games with incomplete information. In Advances in Neural Infoimation Processing Systems, pp. 1729-1736, 2007.