Learning to play using low-complexity rule-based policies: Illustrations through Ms. Pac-Man

Journal of Artificial Intelligence Research

Volumn 30, Issue , 2007, Pages 659-684

  Szita, István ^a   Lorincz, András ^a  

^a EÖTVÖS LORÁND UNIVERSITY   (Hungary)

Author keywords

[No Author keywords available]

Indexed keywords

COMBINATORIAL MATHEMATICS; COMPUTATIONAL COMPLEXITY; GAME THEORY; OPTIMIZATION;

COMBINATORIAL REINFORCEMENT LEARNING; DECISION LISTS; ENTROPY OPTIMIZED POLICIES;

LEARNING SYSTEMS;

EID: 38349162555     PISSN: None     EISSN: 10769757     Source Type: Journal    
DOI: 10.1613/jair.2368     Document Type: Article

References (38)

1. Allon, G., Kroese, D. P., Raviv, T., & Rubinstein, R. Y. (2005). Application of the cross-entropy method to the buffer allocation problem in a simulation-based environment. Annals of Operations Research, 134, 137-151.
2. Baum, E. B. (1996). Toward a model of mind as a laissez-faire economy of idiots. In Proceedings of the 13rd International Conference on Machine Learning, pp. 28-36.
3. Baxter, J., Tridgell, A., & Weaver, L. (2001). Machines that learn to play games, chap. Reinforcement learning and chess, pp. 91-116. Nova Science Publishers, Inc.
4. Bertsekas, D. P., & Tsitsiklis, J. N. (1996). Neuro-Dynamic Programming. Athena Scientific.
5. Bonet, J. S. D., & Stauffer, C. P. (1999). Learning to play Pac-Man using incremental reinforcement learning.. [Online; accessed 09 October 2006].
6. Bull, L. (2004). Applications of Learning Classifier Systems, chap. Learning Classifier Systems: A Brief Introduction, pp. 3-13. Springer.
7. Bull, L., & Kovacs, T. (2005). Foundations of Learning Classifier Systems, chap. Foundations of Learning Classifier Systems: An Introduction, pp. 3-14. Springer.
8. Courtillat, P. (2001). NoN-SeNS Pacman 1.6 with C sourcecode.. [Online; accessed 09 October 2006].
9. Dambreville, F. (2006). Cross-entropic learning of a machine for the decision in a partially observable universe. Journal of Global Optimization. To appear.
10. de Boer, P.-T., Kroese, D. P., Mannor, S., & Rubinstein, R. Y. (2004). A tutorial on the cross-entropy method. Annals of Operations Research, 134, 19-67.
11. Gallagher, M., & Ryan, A. (2003). Learning to play pac-man: An evolutionary, rule-based approach. In et. al., R. S. (Ed.), Proc. Congress on Evolutionary Computation, pp. 2462-2469.
12. Holland, J. H. (1986). Escaping brittleness: The possibilities of general-purpose learning algorithms applied to parallel rule-based systems. In Mitchell, Michalski, & Carbonell (Eds.), Machine Learning, an Artificial Intelligence Approach. Volume II, chap. 20, pp. 593-623. Morgan Kaufmann.
13. Keith, J., & Kroese, D. P. (2002). Sequence alignment by rare event simulation. In Proceedings of the 2002 Winter Simulation Conference, pp. 320-327.
14. Koza, J. (1992). Genetic programming: on the programming of computers by means of natural selection. MIT Press.
15. Lucas, S. M. (2005). Evolving a neural network location evaluator to play Ms. Pac-Man. In IEEE Symposium on Computational Intelligence and Games, pp. 203-210.
16. Mannor, S., Rubinstein, R. Y., & Gat, Y. (2003). The cross-entropy method for fast policy search. In 20th International Conference on Machine Learning.
17. Margolin, L. (2004). On the convergence of the cross-entropy method. Annals of Operations Research, 134, 201-214.
18. Menache, I., Mannor, S., & Shimkin, N. (2005). Basis function adaptation in temporal difference reinforcement learning. Annals of Operations Research, 134(1), 215-238.
19. Muehlenbein, H. (1998). The equation for response to selection and its use for prediction. Evolutionary Computation, 5, 303-346.
20. Ponsen, M., & Spronck, P. (2004). Improving adaptive game AI with evolutionary learning. In Computer Games: Artificial Intelligence, Design and Education.
21. Rohanimanesh, K., & Mahadevan, S. (2001). Decision-theoretic planning with concurrent temporally extended actions. In Proceedings of the 17th Conference on Uncerainty in Artificial Intelligence, pp. 472-479.
22. Rubinstein, R. Y. (1999). The cross-entropy method for combinatorial and continuous optimization. Methodology and Computing in Applied Probability, 1, 127-190.
23. Samuel, A. L. (1959). Some studies in machine learning using the game of checkers. IBM Journal of Research and Development, 6, 211-229.
24. Schaul, T. (2005). Evolving a compact concept-based Sokoban solver. Master's thesis, École Polytechnique Fédérale de Lausanne.
25. Schmidhuber, J. (1997). A computer scientist's view of life, the universe, and everything. In Freksa, C., Jantzen, M., & Valk, R. (Eds.), Foundations of Computer Science: Potential - Theory - Cognition, Vol. 1337 of Lecture Notes in Computer Science, pp. 201-208. Springer, Berlin.
26. Spronck, P., Ponsen, M., Sprinkhuizen-Kuyper, I., & Postma, E. (2006). Adaptive game ai with dynamic scripting. Machine Learning, 63(3), 217-248.
27. Spronck, P., Sprinkhuizen-Kuyper, I., & Postma, E. (2003). Online adaptation of computer game opponent AI. In Proceedings of the 15th Belgium-Netherlands Conference on Artificial Intelligence, pp. 291-298.
28. Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press, Cambridge.
29. Szabó, Z., Póczos, B., & Lorincz, A. (2006). Cross-entropy optimization for independent process analysis. In ICA, pp. 909-916.
30. Szita, I. (2006). How to select the 100 voxels that are best for prediction - a simplistic approach. Tech. rep., Eötvös Loránd University, Hungary.
31. Szita, I., & Lorincz, A. (2006). Learning Tetris using the noisy cross-entropy method. Neural Computation, 18(12), 2936-2941.
32. Tesauro, G. (1994). TD-Gammon, a self-teaching backgammon program, achieves master-level play. Neural Computation, 6(2), 215-219.
33. Timuri, T., Spronck, P., & van den Herik, J. (2007). Automatic rule ordering for dynamic scripting. In The Third Artificial Intelligence and Interactive Digital Entertainment Conference, pp. 49-54.
34. Tiong, A. L. K. (2002). Rule set representation and fitness functions for an artificial pac man playing agent. Bachelor's thesis, Department of Information Technology and Electrical Engineering.
35. Ward, D. J., & MacKay, D. J. C. (2002). Fast hands-free writing by gaze direction. Nature, 418, 838-540.
36. Wikipedia (2006). Pac-Man -Wikipedia, the free encyclopedia. Wikipedia. [Online; accessed 20 May 2007].
37. Witten, I. A., Neal, R. M., & Cleary, J. G. (1987). Arithmetic coding for data compression. Communications of the ACM, 30, 520-540.
38. Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1, 67-82.