Application of reinforcement learning to the game of Othello

Computers and Operations Research

Volumn 35, Issue 6, 2008, Pages 1999-2017

  van Eck, Nees Jan ^a   van Wezel, Michiel ^a  

^a ERASMUS UNIVERSITY ROTTERDAM   (Netherlands)

Author keywords

Dynamic programming; Game playing; Markov decision processes; Multiagent learning; Neural networks; Othello; Q learning; Reinforcement learning

Indexed keywords

DECISION MAKING; DYNAMIC PROGRAMMING; FUNCTION EVALUATION; MANAGEMENT SCIENCE; MARKOV PROCESSES; NEURAL NETWORKS; OPERATIONS RESEARCH; PROBLEM SOLVING;

GAME PLAYING; MULTIAGENT LEARNING; OTHELLO; Q-LEARNING;

REINFORCEMENT LEARNING;

EID: 35349027192     PISSN: 03050548     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cor.2006.10.004     Document Type: Article

References (31)

1. White D.J. A survey of applications of Markov decision processes. The Journal of the Operational Research Society 44 11 (1993) 1073-1096
2. Bertsekas D.P., and Tsitsiklis J. Neuro-dynamic programming (1996), Athena Scientific, Belmont, MA, USA
3. Kaelbling L.P., Littman M.L., and Moore A.W. Reinforcement learning: a survey. Journal of Artificial Intelligence Research 4 (1996) 237-285
4. Sutton R.S., and Barto A.G. Reinforcement learning: an introduction (1998), MIT Press, Cambridge, MA, USA
5. Gosavi A. Simulation-based optimization: parametric optimization techniques and reinforcement learning (2003), Kluwer Academic Publishers, Boston, MA, USA
6. Tesauro G. TD-gammon a self-teaching backgammon program, achieves master-level play. Neural Computation 6 2 (1994) 215-219
7. Tesauro G. Temporal difference learning and TD-gammon. Communications of the ACM 38 3 (1995) 58-68
8. Tesauro G. Programming backgammon using self-teaching neural nets. Artificial Intelligence 134 1-2 (2002) 181-199
9. Gosavi A., Bandla N., and Das T.K. A reinforcement learning approach to a single leg airline revenue management problem with multiple fare classes and overbooking. IIE Transactions 34 9 (2002) 729-742
10. Crites R.H., and Barto A.G. Improving elevator performance using reinforcement learning. In: Touretzky D.S., Mozer M.C., and Hasselmo M.E. (Eds). Advances in neural information processing systems vol. 8 (1996), The MIT Press, Cambridge, MA 1017-1023
11. Crites R.H., and Barto A.G. Elevator group control using multiple reinforcement learning agents. Machine Learning 33 2 (1998) 235-262
12. Pednault E, Abe N, Zadrozny B. Sequential cost-sensitive decision-making with reinforcement learning. In: Proceedings of the eighth ACM SIGKDD international conference on knowledge discovery and data mining. Edmonton, Alberta, Canada: ACM Press; 2002. p. 259-68.
13. Das T.K., Gosavi A., Mahadevan S., and Marchalleck N. Solving semi-Markov decision problems using average reward reinforcement learning. Management Science 45 4 (1999) 560-574
14. Gosavi A. Reinforcement learning for long run average cost. European Journal of Operational Research 144 (2004) 654-674
15. Fürnkranz J. Machine learning in games: a survey. In: Fürnkranz J., and Kubat M. (Eds). Machines that learn to play games (2001), Nova Science Publishers, Huntington, NY, USA 11-59 [chapter 2]
16. Moriarty D.E., and Miikkulainen R. Discovering complex Othello strategies through evolutionary neural networks. Connection Science 7 3 (1995) 195-210
17. Chong S.Y., Tan M.K., and White J.D. Observing the evolution of neural networks learning to play the game of othello. IEEE Transactions on Evolutionary Computation 9 3 (2005) 240-251
18. Russell S., and Norvig P. Artificial intelligence-a modern approach. 2nd ed. (2003), Prentice-Hall, Englewood Cliffs, NJ, USA
19. Bellman R.E. Dynamic programming (1957), Princeton University Press, Princeton, NJ, USA
20. Watkins CJCH. Learning from delayed rewards. PhD thesis, Cambridge University, Cambridge, England; 1989.
21. Watkins C.J.C.H., and Dayan P. Q-learning. Machine Learning 8 3 (1992) 279-292
22. Allis LV. Searching for solutions in games and artificial intelligence. PhD thesis, University of Limburg, Maastricht, The Netherlands; 1994.
23. van der Herik H.J., Uiterwijk J.W.H.M., and van Rijswijck J. Games solved: now and in the future. Artificial Intelligence 134 1-2 (2002) 277-311
24. Bishop C.M. Neural networks for pattern recognition (1995), Oxford University Press, New York, NY, USA
25. Littman M.L. Markov games as a framework for multi-agent reinforcement learning. Proceedings of the eleventh international conference on machine learning (1994), Morgan Kaufmann, San Francisco, CA, USA 157-163
26. Littman M.L. Value-function reinforcement learning in Markov games. Journal of Cognitive Systems Research 2 1 (2001) 55-66
27. Hu J., and Wellman M.P. Nash Q-learning for general-sum stochastic games. Journal of Machine Learning Research 4 (2003) 1039-1069
28. le Comte M. Introduction to Othello; 2000.
29. Rose B. Othello: a minute to learn-a lifetime to master; 2005.
30. Doucette MJ. Wipeout: the engineering of an Othello program. Project report, Acadia University, Wolfville, NS, Canada; 1998.
31. Leouski AV, Utgoff PE. What a neural network can learn about Othello. Technical report UM-CS-1996-010, Computer Science Department, Lederle Graduate Research Center, University of Massachusetts, Amherst, MA, USA; 1996.