Fast approximate max-n Monte Carlo tree search for Ms Pac-Man

IEEE Transactions on Computational Intelligence and AI in Games

Volumn 3, Issue 2, 2011, Pages 142-154

  Samothrakis, Spyridon ^a   Robles, David ^a   Lucas, Simon ^a  

^a UNIVERSITY OF ESSEX   (United Kingdom)

Author keywords

Max n; Monte Carlo; Monte Carlo tree search (MCTS); Pac Man

Indexed keywords

MAX-N; MONTE CARLO; ORDERS OF MAGNITUDE; PAC-MAN; REAL-TIME DECISION MAKING; TREE REPRESENTATION; TREE SEARCH;

DECISION MAKING; MONTE CARLO METHODS;

PLANT EXTRACTS;

EID: 79959242883     PISSN: 1943068X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCIAIG.2011.2144597     Document Type: Article

References (38)

1. R. Coulom, "Efficient selectivity and backup operators in Monte-Carlo tree search," in Proc. 5th Int. Conf. Comput. Games, 2006, pp. 72-83.
2. S. Gelly, Y. Wang, R. Munos, and O. Teytaud, "Modifications of UCT with patterns in Monte-Carlo Go," INRIA, Tech. Rep. 6062, 2006.
3. Y. Bjornsson and H. Finnsson, "CadiaPlayer: A simulation-based general game player," IEEE Trans. Comput. Intell. AI Games, vol. 1, no. 1, pp. 4-15, Mar. 2009.
4. B. Arneson, R. Hayward, and P. Henderson, "Monte Carlo tree search in Hex," IEEE Trans. Comput. Intell. AI Games, vol. 2, no. 4, pp. 251-258, Dec. 2010.
5. H. Matsumoto, T. Ashida, Y. Ozasa, T. Maruyama, and R. Thawonmas, "Ice Pambush 3," Ritsumeikan Univ., Tech. Rep., 2009.
6. J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection. Cambridge, MA: MIT Press, 1992.
7. I. Szita and A. Lorincz, "Learning to play using low-complexity rule-based policies," J. Artif. Intell. Res., vol. 30, no. 1, pp. 659-684, Dec. 2007.
8. J. S. D. Bonet and C. P. Stauffer, "Learning to play Pac-Man using incremental reinforcement learning," 1999 [Online]. Available: http://www.ai.mit.edu/people/stauffer/Projects/PacMan/
9. M. Gallagher and A. Ryan, "Learning to play Pac-Man: An evolutionary, rule-based approach," in Proc. IEEE Congr. Evol. Comput., 2003, pp. 2462-2469.
10. M. L. M. Gallagher, "Evolving Pac-Man players: Can we learn from raw input?," in Proc. IEEE Symp. Comput. Intell. Games, 2007, pp. 282-287.
11. N. Wirth and M. Gallagher, "An influence map model for playing Ms Pac-Man," in Proc. IEEE Symp. Comput. Intell. Games, 2008, pp. 228-233.
12. S. M. Lucas, "Evolving a neural network location evaluator to play Ms Pac-Man," in Proc. IEEE Symp. Comput. Intell. Games, 2005, pp. 203-210.
13. D. Robles and S. Lucas, "A simple tree search method for playing Ms Pac-Man," in Proc. 5th Int. Conf. Comput. Intell. Games, 2009, pp. 249-255.
14. P. Burrow and S. Lucas, "Evolution versus temporal difference learning for learning to play Ms Pac-Man," in Proc. 5th Int. Conf. Comput. Intell. Games, 2009, pp. 53-60.
15. A. M. Alhejali and S. Lucas, "Evolving diverse Ms Pac-Man playing agents using genetic programming," in Proc. U.K. Workshop Comput. Intell., 2010.
16. L. Kocsis and C. Szepesvári, "Bandit based Monte-Carlo planning," in Proc. 15th Eur. Conf. Mach. Learn., 2006, pp. 282-293. (Pubitemid 44618839)
17. E. Rasmusen, Games and Information: An Introduction to Game Theory. New York: Blackwell, 2007.
18. P. Auer, N. Cesa-Bianchi, and P. Fischer, "Finite-time analysis of the multiarmed bandit problem," Mach. Learn., vol. 47, no. 2, pp. 235-256, 2002. (Pubitemid 34126111)
19. S. Gelly and Y. Wang, "Exploration exploitation in Go: UCT for Monte-Carlo Go," in Proc. 20th Annu. Conf. Neural Inf. Process. Syst., 2006.
20. C. S. Lee, M. H. Wang, C. Chaslot, J. Hoock, A. Rimmel, O. Teytaud, S. R. Tsai, S. C. Hsu, and T. P. Hong, "The computational intelligence of MoGo revealed in Taiwan's computer Go tournaments," IEEE Trans. Comput. Intell. AI Games, vol. 1, no. 1, pp. 73-89, Mar. 2009.
21. S. Billouet, J. Hoock, C.-S. Lee, O. Teytaud, and S.-J. Yen, "9 9 Go as black with komi 7.5: At last some games won against top players in the disadvantageous situation," ICGA J., vol. 32, no. 3, pp. 241-246, 2009.
22. S.-J. Yen, C.-S. Lee, and O. Teytaud, "Human vs. Computer Go competition in FUZZ-IEEE 2009," ICGA J., vol. 32, no. 3, pp. 178-180, 2009.
23. R. Blincoe, "Go, going, gone?," The Guardian, 2006 [Online]. Available: http://www.guardian.co.uk/technology/2009/apr/30/games-soft-ware- mogo/print
24. R. Coulom, "Computing ELO ratings of move patterns in the game of Go," in Proc. Comput. Games Workshop, 2007.
25. G. Chaslot, M. Winands, H. Herik, J. Uiterwijk, and B. Bouzy, "Progressive strategies for Monte-Carlo tree search," New Math. Natural Comput., vol. 4, no. 3, p. 343, 2008.
26. M. Enzenberger, M. Muller, B. Arneson, and R. Segal, "FUEGO: An open-source framework for board games and go engine based on Monte-Carlo tree search," IEEE Trans. Comput. Intell. AI Games, vol. 2, no. 4, pp. 259-270, Dec. 2010.
27. F. Van Lishout, G. Chaslot, and J. Uiterwijk, "Monte-Carlo tree search in backgammon," in Proc. Comput. Games Workshop, 2007, pp. 175-184.
28. P. Hingston and M. Masek, "Experiments with Monte Carlo Othello," in Proc. IEEE Congr. Evol. Comput., 2007, pp. 4059-4064. (Pubitemid 351371926)
29. G. V. den Broeck, K. Driessens, and J. Ramon, "Monte-Carlo tree search in poker using expected reward distributions," in Proc. ACML, 2009, pp. 367-381.
30. S. Gelly and D. Silver, "Achieving master level play in 9 9 computer Go," in Proc. 23rd AAAI Conf. Artif. Intell., 2008, pp. 1537-1540.
31. M. Chung, M. Buro, and J. Schaeffer, "Monte Carlo planning in RTS games," in Proc. IEEE Symp. Comput. Intell. Games, 2005.
32. G. Chaslot, S. Bakkes, I. Szita, and P. Spronck, "Monte-Carlo tree search: A new framework for game AI," in Proc. 4th Artif. Intell. Interact. Digital Entertain. Conf., 2008, pp. 216-217.
33. R. Balla and A. Fern, "UCT for tactical assault planning in real-time strategy games," in Proc. 21st Int. Joint Conf. Artif. Intell., 2009, pp. 40-45.
34. D. Billings, A. Davidson, T. Schauenberg, N. Burch, M. Bowling, R. Holte, J. Schaeffer, and D. Szafron, "Game-tree search with adaptation in stochastic imperfect-information games," Lecture Notes Comput. Sci., vol. 3846, pp. 21-34, 2006. (Pubitemid 43900758)
35. C. Luckhardt and K. Irani, "An algorithmic solution of n-person games," in Proc. 5th Nat. Conf. Artif. Intell. (AAAI), 1986, pp. 158-162.
36. S. Samothrakis, D. Robles, and S. Lucas, "A UCT agent for Tron: Initial investigations," in Proc. 6th Int. Conf. Comput. Intell. Games, 2010.
37. T. Galaxies, Twin Galaxies-Ms Pac-Man High Scores @ON-LINE 2006 [Online]. Available: http://www.twingalaxies.com/index.aspx?c=22&pi=2&gi= 3162&vi=1386, [Online]. Available:
38. N. Ikehata and T. Ito, "Monte Carlo tree search in Ms Pac-Man," in Proc. 15th Game Program. Workshop, IPSJ Symposium Series Vol. 2010/12, 2010.