Multiagent reinforcement learning in the Iterated Prisoner's Dilemma

BioSystems

Volumn 37, Issue 1-2, 1996, Pages 147-166

  Sandholm, Tuomas W ^a   Crites, Robert H ^a  

^a The Manning College of Information and Computer Sciences   (United States)

Author keywords

Exploration; Machine learning; Multiagent learning; Prisoner's Dilemma; Recurrent neural network; Reinforcement learning

Indexed keywords

ALGORITHM; ARTICLE; COMPUTER PROGRAM; GAME; HUMAN; PRISON; PRISONER; REINFORCEMENT;

EID: 0030050933     PISSN: 03032647     EISSN: None     Source Type: Journal    
DOI: 10.1016/0303-2647(95)01551-5     Document Type: Article

References (40)

1. Ashlock, D., Smucker, M.D., Stanley, E.A. and Tesfatsion, L., 1995, Preferential partner selection in an evolutionary study of prisoner's dilemma. This Issue.
2. Axelrod, R., 1984, The Evolution of Cooperation (Basic Books, New York).
3. Barto, A.G., Sutton, R. and Anderson, C.W., 1983, Neuron-like adaptive elements that can solve difficult learning control problems. IEEE Trans. Sys., Man Cybern. 13, 834-846.
4. Barto, A., 1989, From chemotaxis to cooperativity: Abstracted exercises in neuronal learning strategies, in: The Computing Neuron, R. Durbin, C. Miall and G. Mitchison (eds.) (Addison-Wesley) pp. 73-98.
5. Barto, A.G., Bradtke, S.J. and Singh, S.P., 1995, Learning to act using real-time dynamic programming. Artif. Intell. 72, 81-138.
6. Barto, A.G. and Anandan, P., 1985, Pattern recognizing stochastic learning automata. IEEE Trans. Sys., Man Cybern. 15, 360-375.
7. Barto, A.G., 1985, Learning by statistical cooperation of self-interested neuron-like adaptive elements. Hum. Neurobiol. 4, 229-256.
8. Bertsekas, D.P. and Tsitsiklis, J.N., 1989, Parallel and Distributed Computation: Numerical Methods (Prentice-Hall, Englewood Cliffs, NJ).
9. Bradtke, S.J., 1993, Distributed Adaptive Optimal Control of Flexible Structures. Computer Science Department, University of Massachusetts at Amherst. Unpublished draft.
10. Crites, R., 1994, Multi-Agent Reinforcement Learning. PhD dissertation proposal. Computer Science Department, University of Massachusetts at Amherst.
11. Elman, J., 1990, Finding structure in time. Cognit. Sci. 14, 179-211.
12. Fudenberg, D. and Tirole, J., 1991, Game Theory (MIT Press, Cambridge, MA).
13. Haykin, S., 1994, Neural Networks: A Comprehensive Foundation (Macmillan, New York).
14. Hecht-Nielsen, R., 1991, Neurocomputing (Addison-Wesley, Reading, MA).
15. Kinney, M. and Tsatsoulis, C., 1993, Learning Communication Strategies in Distributed Agent Environments. Working paper WP-93-4. Intelligent Design Laboratory, University of Kansas.
16. Kreps, D., 1990, A Course in Microeconomic Theory (Princeton University Press, Princeton, NJ).
17. Lin, L.-J., 1993, Reinforcement Learning for Robots Using Neural Networks. Ph.D. dissertation, School of Computer Science, Carnegie Mellon University.
18. Littman, M., 1993, Markov games as a framework for multi-agent reinforcement learning, in: Machine Learning, Proceedings of the Eleventh International Conference (Rutgers University, NJ) pp. 157-163.
19. Littman, M. and Boyan, J., 1993, A Distributed Reinforcement Learning Scheme for Network Routing. Technical Report CMU-CS-93-165, Carnegie Mellon University.
20. Luce, D. and Raiffa, H., 1957, Games and Decisions. Reprint: 1989 (Dover Publications, New York).
21. Markey, K.L., 1993, Efficient learning of multiple degree-of-freedom control problems with quasi-independent Q-agents, in: Proceedings of the 1993 Connectionist Models Summer School (Erlbaum Associates, Hillsdale, NJ).
22. Narendra, K.S. and Thathachar, M.A.L., 1989, Learning Automata: An Introduction (Prentice-Hall, Englewood Cliffs, NJ).
23. Nowak, M. and Sigmund, K., 1993, A strategy of win-stay, lose-shift that outperforms Tit-For-Tat in the Prisoner's Dilemma game. Nature 364, 56-58.
24. Rumelhart, D.E., Hinton, G.E. and Williams, R.J., 1986, Learning internal representations by error propagation, in: Parallel Distributed Processing: Explorations in the Microstructure of Cognition 1, D.E. Rumelhart and J.L. Mc-Clellan (eds.) (MIT Press, Cambridge, MA) pp. 318-362.
25. Samuel, A.L., 1959, Some studies in machine learning using the game of checkers, IBM Journal on Research and Development, Reprinted: 1963, in: Computers and Thought, E.A. Feigenbaum and J. Feldman (eds.) (McGraw-Hill, New York) pp. 210-229
26. Sandholm, T., 1993, An implementation of the contract net protocol based on marginal cost calculations, in: Proceedings of the Eleventh National Conference on Artificial Intelligence (AAAI-93) (Washington DC) pp. 256-262.
27. Sandholm, T. and Lesser, V., 1994, Utility-based termination of anytime algorithms, in: Proceedings of the European Conference on Artificial Intelligence (ECAI-94) Workshop on Decision Theory for Distributed Artificial Intelligence Applications, pp. 88-99, Amsterdam, The Netherlands. Extended version: University of Massachusetts at Amherst, Computer Science Technical Report 94-54.
28. Sandholm, T. and Lesser, V., 1995a, Coalition formation among bounded rational agents, in: Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI-95) (Montreal, Canada).
29. Sandholm, T. and Lesser, V., 1995b, Issues in automated negotiation and electronic commerce: extending the contract net framework, in: Proceedings of the First International Conference on Multiagent Systems (ICMAS-95) (San Francisco, CA).
30. Sandholm, T. and Nagendraprasad, M., 1993, Learning Pursuit Strategies. Class project for CmpSci 689 Machine Learning. Computer Science Department, University of Massachusetts at Amherst, Spring 1993.
31. Sen, S., Sekaran, M. and Hale, J., 1994, Learning to coordinate without sharing information, in: Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI-94) (Seattle, WA) pp. 426-431.
32. Shoham, Y. and Tennenholtz, M., 1993, Co-Learning and the Evolution of Coordinated Multi-Agent Activity.
33. Sugawara, T. and Lesser, V., 1993, On-Line Learning of Coordination Plans. Computer Science Technical Report 93-27 (University of Massachusetts, Amherst).
34. Sutton, R.S., 1988, Learning to predict by the methods of temporal differences. Mach. Learning 3, 9-44.
35. Tan, M., 1993, Multi-agent reinforcement learning: independent vs. cooperative agents, in: Machine Learning, Proceedings of the Tenth International Conference (University of Massachusetts, Amherst), pp. 330-337.
36. Tesauro, G.J., 1992, Practical issues in temporal difference learning. Mach. Learning 8, 257-277.
37. Tsetlin, M.L., 1973, Automaton Theory and Modeling of Biological Systems (Academic Press, New York).
38. Watkins, C., 1989, Learning from delayed rewards. PhD Thesis, University of Cambridge, England.
39. Wei, G., 1993, Learning to coordinate actions in multi-agent systems, in: Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence (IJCAI-93) (Chambery, France) pp. 311-316.
40. Williams, R.J. and Zipser, D. 1989, A learning algorithm for continually running fully recurrent neural networks. Neural Computat. 1, 270-280.