Creating Autonomous Adaptive Agents in a Real-Time First-Person Shooter Computer Game

IEEE Transactions on Computational Intelligence and AI in Games

Volumn 7, Issue 2, 2015, Pages 123-138

  Wang, Di ^a   Tan, Ah Hwee ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Adaptive resonance theory operations; Real time computer game; Reinforcement learning; Temporal difference learning; Unreal tournament

Indexed keywords

AUTONOMOUS AGENTS; NEURAL NETWORKS; REINFORCEMENT LEARNING;

ADAPTIVE AGENTS; ADAPTIVE RESONANCE THEORY; FIRST PERSON SHOOTER; GAME ENVIRONMENT; REAL-TIME COMPUTER; SELF-ORGANIZING NEURAL NETWORK; TEMPORAL DIFFERENCE LEARNING; UNREAL TOURNAMENT;

COMPUTER GAMES;

EID: 84933053464     PISSN: 1943068X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCIAIG.2014.2336702     Document Type: Article

References (44)

1. Game vs. Movie Industry. [Online]. Available: http://mygaming.co. za/news/features/42864-game-vs-movie-industry.html (Accessible till May 2014)
2. J. E. Laird and M. Van Lent, "Human-level AI's killer application: Interactive computer games", AI Mag., vol. 22, no. 2, pp. 15-25, 2001.
3. D. B. Fogel, H. Timothy J, and D. R. Johnson, "A platform for evolving characters in competative games", in Proc. Congress E vol. Comput., 2004, pp. 1420-1426.
4. C. Pedersen, J. Togelius, and G. N. Yannakakis, "Modeling player experience for content creation", IEEE Trans. Comput. Intell. AI in Games, vol. 2, no. 1, pp. 54-67, Mar. 2010.
5. P. Hingston, "A Turing Test for computer game bots", IEEE Trans. Comput. Intell. AI in Games, vol. 1, no. 3, pp. 169-186, Sep. 2009.
6. D. Wang, B. Subagdja, and A.-H. Tan, "Creating human-like autonomous players in real-time first person shooter computer games", in Proc. Conf. Innovative Appl. Artif. Intell., 2009, pp. 173-178.
7. G. N. Yannakakis and J. Hallam, "Real-time game adaptation for optimizing player satisfaction", IEEE Trans. Comput. Intell. AI in Games, vol. 1, no. 2, pp. 121-133, Jun. 2009.
8. H. Wang, Y. Gao, and X. Chen, "RL-DOT: A reinforcement learning NPC team for playing domination games", IEEE Trans. Comput. Intell. AI in Games, vol. 2, no. 1, pp. 17-26, Mar. 2010.
9. A.-H. Tan, "FALCON: A fusion architecture for learning, cognition, and navigation", in Proc. Int. Joint Conf. Neural Netw., 2004, pp. 3297-3302.
10. G. A. Carpenter and S. Grossberg, "Adaptive resonance theory", in The Handbook of Brain Theory and neural Networks. Cambridge, MA, USA: MIT Press, 2003, pp. 87-90.
11. J. Gemrot, R. Kadlec, M. Bída, O. Burkert, R. Pibil, J. Havlicek, L. Zemcak, J. Simlovic, R. Vansa, and M. Stolba et al., "Pogamut 3 can assist developers in building AI (not only) for their videogame agents", Agents for Games Simul., pp. 1-15, 2009.
12. R. Lopes and R. Bidarra, "Adaptivity challenges in games and simulations: A survey", IEEE Trans. Comput. Intell. AI in Games, vol. 3, no. 2, pp. 1-14, Jun. 2011.
13. M. Ponsen and P. Spronck, "Improving adaptive game AI with evolutionary learning", Mach. Learn., pp. 389-396, 2004.
14. P. Spronck, M. Ponsen, I. Sprinkhuizen-Kuyper, and E. Postma, "Adaptive game AI with dynamic scripting", Mach. Learn., vol. 63, pp. 217-248, 2006.
15. D. J. Cook, L. B. Holder, and G. M. Youngblood, "Graph-based analysis of human transfer learing using a game testbed", IEEE Trans. Knowledge Data Eng., vol. 19, no. 11, pp. 1465-1478, Nov. 2007.
16. N. Cole, S. J. Louis, and C. Miles, "Using a genetic algorithm to tune first-person shooter bots", in Proc. Int. Congress E vol. Comput., 2004, pp. 139-145.
17. S. J. Louis and C. Miles, "Playing to learn: Case-injected genetic algorithms for learning to play computer games", IEEE Trans. E vol. Comput., vol. 9, no. 6, pp. 669-681, Dec. 2005.
18. K. O. Stanley, B. D. Bryant, and R. Miikkulainen, "Real-time neuroevolution in the nero video game", IEEE Trans. E vol. Comput., vol. 9, no. 6, pp. 653-668, Dec. 2005.
19. C. Miles and S. J. Louis, "Towards the co-evolution of influence map tree based strategy game players", in Proc. IEEE Symp. Comput. Intell. Games, 2006, pp. 75-82.
20. R. L. Hy, A. Arrigoni, P. Bessiere, and O. Lebetel, "Teaching Bayesian behaviours to video game characters", Robot. Autonom. Syst., vol. 47, pp. 177-185, 2004.
21. I. C. Kim, "UTBot: A virtual agent platform for teaching agent system design", J. Multimedia, vol. 2, no. 1, pp. 48-53, 2007.
22. M. Smith, S. Lee-Urban, and H. Munoz-Avila, "RETALIATE: Learning winning policies in first-person shooter games", in Proc. Nat. Conf. Innovative Appll. Artif. Intell., 2007, pp. 1801-1806.
23. J. Schrum, I. V. Karpov, and R. Miikkulainen, "UT2: Human-like behavior via neuroevolution of combat behavior and replay of human traces", in Proc. IEEE Conf. Comput. Intell. Games, 2011, pp. 329-336.
24. P. Hingston, Ed., Believable Bots: Can Computers Play Like People?. Berlin, Germany: Springer-Verlag, 2012.
25. D. Gamez, Z. Fountas, and A. K. Fidjeland, "A neurally controlled computer game avatar with humanlike behavior", IEEE Trans. Comput. Intell. AI in Games, vol. 5, no. 1, pp. 1-14, Mar. 2013.
26. G. Kaminka, M. Veloso, S. Schaffer, C. Sollitto, R. Adobbati, A. Marshall, A. Scholer, and S. Tejada, "Gamebots: The ever-challenging multi-agent research test-bed", Commun. ACM, vol. 45, no. 1, pp. 43-45, 2002.
27. L. D. Nguyen, K.-Y. Woon, and A.-H. Tan, "A self-organizing neural model for multimedia information fusion", in Proc. Conf. Inf. Fusion, 2008, pp. 1738-1744.
28. B. Subagdja and A.-H. Tan, "A self-organizing neural network architecture for intentional planning agents", in Proc. Int. Conf. Autonom. Agents Multiagent Syst., 2009, pp. 1081-1088.
29. Y. Feng, T.-H. Teng, and A.-H. Tan, "Modelling situation awareness for context-aware decision support", Expert Syst. Appl., vol. 36, no. 1, pp. 455-463, 2009.
30. A.-H. Tan and G.-W. Ng, "A biologically-inspired cognitive agent model integrating declarative knowledge and reinforcement learning", in Proc. Int. Conf. Web Intell. Intell. Agent Technol., 2010, pp. 248-251.
31. G. A. Carpenter, S. Grossberg, and D. B. Rosen, "Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system", Neural Netw., vol. 4, pp. 759-771, 1991.
32. A.-H. Tan, "Direct code access in self-organizing neural networks for reinforcement learning", in Proc. Int. Joint Conf. Artif. Intell., 2007, pp. 1071-1076.
33. G. A. Carpenter and S. Grossberg, "ART 2: Self-organization of stable category recognition codes for analog input patterns", Appl. Opt., vol. 26, no. 23, pp. 4919-4930, 1987.
34. G. A. Carpenter, S. Grossberg, and D. Rosen, "ART 2-A: An adaptive resonance algorithm for rapid category learning and recognition", Neural Netw., vol. 4, no. 4, pp. 493-504, 1991.
35. A.-H. Tan and D. Xiao, "Self-organizing cognitive agents and reinforcement learning in multi-agent environment", in Proc. Int. Conf. Intell. Agent Technol., 2005, pp. 351-357.
36. R. D. Smallwood and E. J. Sondik, "The optimal control of partially observable markov processes over a finite horizon", Operations Res., vol. 21, no. 5, pp. 1071-1088, 1973.
37. A.-H. Tan, N. Lu, and D. Xiao, "Integrating temporal difference methods and self-organizing neural networks for reinforcement learning with delayed evaluative feedback", IEEE Trans. Neural Netw., vol. 19, no. 2, pp. 230-244, Feb. 2008.
38. R. Small and C. B. Congdon, "Agent Smith: Towards an evolutionary rule-based agent for interactive dynamic games", in Proc. IEEE Congress E vol. Comput., 2009, pp. 660-666.
39. S. Feng and A.-H. Tan, "Self-organizing neural networks for behavior modeling in games", in Proc. Int. Joint Conf. Neural Netw., 2010, pp. 3649-3656.
40. A. J. F. Leiva and J. L. O. Barragan, "Decision tree-based algorithms for implementing bot AI in UT2004", in Foundations on Natural and Artificial Computation, Lecture Notes in Computer Science. Berlin, Germany: Springer-Verlag, 2011, vol. 6686, pp. 383-392.
41. R. A. Fisher, "The correlation between relatives on the supposition of mendelian inheritance", Philosoph. Trans. Roy. Soc. Edinburgh, vol. 52, pp. 399-433, 1918.
42. E. Tulving, Elements of Episodic Memory. Oxford, U. K.: Clarendon Press, 1983.
43. P. Auer, N. Cesa-Bianchi, and P. Fischer, "Finite-time analysis of the multiarmed bandit problem", Mach. Learn., vol. 47, no. 2-3, pp. 235-256, 2002.
44. T.-H. Teng, A.-H. Tan, and Y.-S. Tan, "Self-regulating action exploration in reinforcement learning", Proc. Comput. Sci., vol. 13, pp. 18-30, 2012.