Exploration Strategies for Model-based Learning in Multi-agent Systems

Autonomous Agents and Multi-Agent Systems

Volumn 2, Issue 2, 1999, Pages 141-172

  Carmel, David ^a,b   Markovitch, Shaul ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^b IBM HAIFA RESEARCH LAB   (Israel)

Author keywords

Exploration; Model based learning; Multi agent systems

Indexed keywords

EID: 0033423368     PISSN: 13872532     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1010007108196     Document Type: Article

References (42)

1. D. Angluin. "Learning regular sets from queries and counterexamples." Information and Computation, vol. 75 pp. 87-106, 1987.
2. R. Axelrod. The Evolution of Cooperation, Basic Books: New York, NY, 1984.
3. E. Ben-Porath. "The complexity of computing a best response automaton in repeated games with mixed strategies." Games and Economic Behavior, vol. 2 pp. 1-12, 1990.
4. D. A. Berry and B. Fristedt. Bandit Problems, Sequential Allocation and Experiments, Chapman and Hall: 1985.
5. D. Carmel and S. Markovitch. "Learning models of intelligent agents," in Proceedings of Thirteenth National Conference on Artificial Intelligence (AAAI 96), Portland, Oregon, pp. 62-67, August 1996.
6. D. Carmel and S. Markovitch. "Exploration and adaptation in multi-agent systems: A model-based approach," in Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence (IJCAI-97), Nagoya, Japan, pp. 606-611, August 1997.
7. D. Carmel and S. Markovitch. "Model-based learning of interaction strategies in multi-agent systems." Journal of Experimental and Theoretical Artificial Intelligence (JETAI), vol. 10 pp. 309-332, June 1998.
8. D. A. Cohn. "Neural network exploration using optimal experimental design," in J. D. Cowan, G. Tesauro, and J. Alspector, (Eds.), Advances in Neural Information Processing Systems 6, Morgan Kaufmann: pp. 679-686, 1994.
9. P. Dayan and T. J. Sejnowski. "Exploration bonuses and dual control." Machine Learning, vol. 25(1) pp. 5-22, 1996.
10. V. Fedorov. Theory of Optimal Experiments. Academic Press: New York, 1972.
11. L. Fortnow and D. Whang. "Optimality and domination in repeated games with bounded players," in Proceedings of the 25th Annual ACM Symposium on Theory and Computing, pp. 741-749, 1994.
12. Y. Freund, M. Kearns, D. Ron, R. Rubinfeld, R. E. Schapire, and Linda Sellie. "Efficient learning of typical finite automata from random walks," in Proceedings of the 25th Annual ACM Symposium on Theory and Computing, pp. 315-324, 1993.
13. Y. Freund, M. Kearns, Y. Mansour, D. Ron, R. Rubinfeled, and R. E. Schapire. "Efficient algorithms for learning to play repeated games against computationally bounded adversaries," in Proceedings of the Annual Symposium on the Foundations of Computer Science, pp. 332-341, 1995.
14. D. Fudenberg and D. Levine. "Steady state learning and nash equilibrium." Econometrica, vol. 61 pp. 547-574, 1993.
15. I. Gilboa and D. Samet. "Bounded versus unbounded rationality: The tyranny of the weak." Games and Economic Behavior, vol. 1 pp. 213-221, 1989.
16. I. Gilboa. "The complexity of computing best response automata in repeated games." Journal of Economic Theory, vol. 45 pp. 342-352, 1988.
17. J. C. Gittins. Multi-armed Bandit Allocation Indices. Wiley and Sons: New York, 1989.
18. J. E. Hopcroft and J. D. Ullman. Introduction to Automata Theory, Languages and Computation. Addison-Wesley: MA, 1979.
19. J. S. Jordan. "Bayesian learning in normal form games." Games and Economic Behavior, vol. 3 pp. 60-81, 1991.
20. J. S. Jordan. "The exponential convergence of bayesian learning in normal form games." Games and Economic Behavior, vol. 4 pp. 202-217, 1991.
21. L. P. Kaelbling, M. L. Littman, and A. W. Moore. "Reinforement learning: A survey." Journal of Artificial Intelligence Research, vol. 4 pp. 237-285, 1996.
22. L. P. Kaelbling. Learning in Embedded Systems. MIT Press: Cambridge, MA, 1993.
23. E. Kalai and E. Lehrer. "Rational learning leads to Nash equilibrium." Econometrica, vol. 61(5) pp. 1019-1045, September 1993.
24. E. Kalai. "Bounded rationality and strategic complexity in repeated games," in T Ichiishi, A. Neyman, and Y. Tauman, (Eds.), Game Theory and Applications, Academic Press: San Diego, pp. 131-157, 1990.
25. G. I. Karakoulas. "Probabilistic exploration in planning while learning," in Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence (UAI 95), pp. 352-361, July 1995.
26. M. L. Littman. "Markov games as a framework for multi-agent reinforcement learning," in Proceedings of the Eleventh International Conference on Machine Learning, pp. 157-163, July 1994.
27. R. D. Luce and H. Raiffa. Games and Decisions, Introduction and Critical Survey. John Wiley & Sons: 1957.
28. A. W. Moore and C. G. Atkeson. "Prioritized sweeping: Reinforcement learning with less data and less time." Machine Learning, vol. 13(1), 1993.
29. Y. Mor, C. V. Goldman, and J. S. Rosenschein. "Learn your opponent's strategy (in polynomial time)," in G. Weiß and S. Sen, (Eds.), Adaptation and Learning in Multi-agent Systems, Lecture Notes in AI. Springer-Verlag: 1996.
30. J. H. Nachbar. "Optimization and rational learning in games." Econometrica vol. 65(2), 1997.
31. C. H. Papadimitriou and J. N. Tsitsiklis. "The complexity of Markov decision processes." Mathematics of Operations Research, vol. 12(3) pp. 441-450, 1987.
32. C. H. Papadimitriou. "On players with a bounded number of states." Games and Economic Behavior, vol. 4 pp. 122-131, 1992.
33. D. Ron and R. Rubinfeled. "Exactly learning automata with small cover time," in Proceedings of the Seventh Annual ACM Conference on Computational Learning Theory, pp. 427-436, 1995.
34. A. Rubinstein. "Finite automata play the repeated Prisoner's Dilemma." Journal of Economic Theory, vol. 39 pp. 83-96, 1986.
35. T. W. Sandholm and R. H. Crites. "Multiagent reinforcement learning and the iterated Prisoner's Dilemma." Biosystems Journal, vol. 37 pp. 147-166, 1995.
36. M. Sato, K. Abe, and H. Takeda. "Learning control of finite Markov chains with an explicit trade-off between estimation and control," in IEEE Transactions on Systems, Man and Cybernetics, vol. 18(5), September 1991.
37. S. Sen, M. Sekaran, and J. Hale. "Learning to coordinate without sharing information," in Proceeding of the Twelfth National Conference on Artificial Intelligence (AAAI-94), Seattle, Washington, pp. 426-431, 1994.
38. S. Singh, T. Jaakkola, M. L. Littman, and C. Szpezvari. "Convergence results for single-step on-policy reinforcement-learning algorithms." Machine Learning Journal (to appear), 1998.
39. R. S. Sutton. "Integrated architectures for learning, planning, and reacting based on approximating dynamic programming," in Proceedings of the Seventh International Conference on Machine Learning, Morgan Kaufman: San Mateo, CA, pp. 216-224, 1990.
40. S. B. Thrun. "The role of exploration in learning control," in David A. White and Donald Sopfge, (Eds.), Handbook for Intelligent Control. Multiscience Press Inc.: 1992.
41. C. J. C. H. Watkins and P. Dayan. "Technical notes: Q-learning." Machine Learning, vol. 8 pp. 279-292, 1992.
42. G. Weiß and S. Sen. Adaptation and Learning in Multi-agent Systems, Lecture Notes in AI 1042. Springer-Verlag: 1996.