Reinforcement learning in first person shooter games

IEEE Transactions on Computational Intelligence and AI in Games

Volumn 3, Issue 1, 2011, Pages 43-56

  McPartland, Michelle ^a   Gallagher, Marcus ^a  

^a UNIVERSITY OF QUEENSLAND   (Australia)

Author keywords

Artificial intelligence (AI); computer games; reinforcement learning (RL)

Indexed keywords

COMPUTER GAMES; FIRST PERSON SHOOTER GAMES; INDUSTRY STANDARDS; MACHINE LEARNING TECHNIQUES; NAVIGATION CONTROLS; PATH-FINDING ALGORITHMS; RULE BASED;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SOFTWARE; HUMAN COMPUTER INTERACTION; NAVIGATION; REINFORCEMENT LEARNING; RESEARCH;

CONTROLLERS;

EID: 79952925038     PISSN: 1943068X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCIAIG.2010.2100395     Document Type: Article

References (24)

1. G. Tesauro, "Temporal difference learning and TD-gammon," Commun. ACM, vol. 38, no. 3, pp. 58-68, 1994.
2. N. Sturtevant and A. White, "Feature construction for reinforcement learning in hearts," in Proc. 5th Int. Conf. Comput. Games, 2006, pp. 122-134.
3. D. S.-C. Dalmau, Core Techniques and Algorithms in Game Programming. Indianapolis, IN: New Riders, 2003.
4. D. Isla, "Handling the complexity in Halo 2," in Proc. Game Developers Conf., 2005.
5. J. Jones, "Benefits of genetic algorithms in simulations for game designers," Master's thesis, Schl. Inf., Univ. Buffalo, Buffalo, NY, 2003.
6. C. A. Overholtzer and S. Levy, "Evolving AI opponents in a first-person-shooter video game," in Proc. 20th Nat. Conf. Artif. Intell., Pittsburgh, PA, 2005, pp. 1620-1621. (Pubitemid 43006511)
7. S. Bakkes, P. Spronck, and E. Postma, "TEAM: The team-oriented evolutionary adaptability mechanism," in Proc. Entertain. Comput., 2004, pp. 296-307.
8. M. McPartland and M. Gallagher, "Learning to be a bot: Reinforcement learning in shooter games," presented at the Artif. Intell. Interactive Digit. Entertain., Stanford, CA, 2008.
9. M. McPartland and M. Gallagher, "Creating a multi-purpose first person shooter bot with reinforcement learning," in IEEE Symp. Comput. Intell. Games, 2008, pp. 143-150.
10. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
11. J. Manslow, "Using reinforcement learning to solve AI control problems," in AIGame Programming Wisdom 2, S. Rabin, Ed. Hingham, MA: Charles River Media, 2004.
12. A. H. Tan and D. Xiao, "Self-organizing cognitive agents and reinforcement learning in multi-agent environment," in Proc. Int. Conf. Intell. Agent Technol., Compiegne, France, 2005, pp. 351-357. (Pubitemid 46116575)
13. J. Bradley and G. Hayes, "Group utility functions: Learning equilibria between groups of agents in computer games by modifying the reinforcement signal," in Proc. Congr. Evol. Comput., 2005, vol. 2, pp. 1914-1921. (Pubitemid 41496082)
14. S. Nason and J. E. Laird, "Soar-RL: Integrating reinforcement learning with soar," Cogn. Syst. Res., vol. 6, no. 1, pp. 51-59, 2005. (Pubitemid 39530523)
15. J. H. Lee, S. Y. Oh, and D. H. Choi, "TD based reinforcement learning using neural networks in control problems with continuous action space," in Proc. IEEE World Congr. Comput. Intell., Anchorage, AK, 1998, vol. 3, pp. 2028-2033.
16. K. Merrick and M. L. Maher, "Motivated reinforcement learning for non-player characters in persistent computer game worlds," in Proc. ACM SIGCHI Int. Conf. Adv. Comput. Entertain. Technol., Los Angeles, CA, 2006.
17. T. Graepel, R. Herbrich, and J. Gold, "Learning to fight," in Proc. Int. Conf. Comput. Games, Artif. Intell. Design Educ., 2004, pp. 193-200.
18. K. Merrick, "Modeling motivation for adaptive nonplayer characters in dynamic computer game worlds," ACM Comput. Entertain., vol. 5, no. 4, 2008, DOI: 10.1145/1324198.1324203. (Pubitemid 351438238)
19. S. Cohen, O. Maimon, and E. Khmlenitsky, "Reinforcement learning with hierarchical decision making," in Proc. 6th Int. Conf. Intell. Syst. Design Appl., 2006, vol. 3, pp. 177-182. (Pubitemid 47179253)
20. T. Watanabe and Y. Takahashi, "Hierarchical reinforcement learning using a modular Fuzzy model for multi-agent problem," in Proc. IEEE Int. Conf. Syst. Man Cybern., Montreal, QC, Canada, 2007, pp. 1681-1686. (Pubitemid 351409805)
21. M. Huber, "A hybrid architecture for hierarchical reinforcement learning," in Proc. IEEE Int. Conf. Robot. Autom., San Francisco, CA, 2000, vol. 4, pp. 3290-3295.
22. B. Blumberg, M. Downie, Y. A. Ivanov, M. Berlin, M. P. Johnson, and B. Tomlinson, "Integrated learning for interactive synthetic characters," ACM Trans. Graph., vol. 21, no. 3, pp. 417-426, 2002.
23. M. Vasta, S. Lee-Urban, and H. Munoz-Avila, "RETALIATE: Learning winning policies in first-person shooter games," in Proc. 17th Innovative Appl. Artif. Intell. Conf., 2007, pp. 1801-1806. (Pubitemid 350149828)
24. J. Laird, "It knows what you're going to do: Adding anticipation to a Quakebot," in Proc. 5th Int. Conf. Autonom. Agent, 2001, pp. 385-392. (Pubitemid 32866096)