Learning and exploiting relative weaknesses of opponent agents

Autonomous Agents and Multi-Agent Systems

Volumn 10, Issue 2, 2005, Pages 103-130

  Markovitch, Shaul ^a   Reger, Ronit ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Machine learning; Multi agent systems; Opponent modelling

Indexed keywords

BOUNDEDLY RATIONAL AGENT; MODEL BASED LEARNING; MODEL-FREE LEARNING; OPPONENT MODELING;

ALGORITHMS; COMPUTATIONAL COMPLEXITY; DECISION SUPPORT SYSTEMS; FINITE AUTOMATA; LEARNING SYSTEMS; MATHEMATICAL MODELS; PROBLEM SOLVING;

MULTI AGENT SYSTEMS;

EID: 14744285085     PISSN: 13872532     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10458-004-6977-7     Document Type: Article

References (52)

1. V. Allis, "A knowledge-based approach of Connect-Four - the game is solved: White wins," Master's thesis, Department of mathematics and Computer Science, Vrije Universiteit, Amsterdam, The Netherlands, 1988.
2. D. Angluin, "On the complexity of minimum inference of regular sets," Information and Control vol. 39, pp. 337-350, 1978.
3. C. Atkeson, and J. Santamaria, "A comparison of direct and model-based reinforcement learning," 1997.
4. D. Billings, D. Papp, J. Schaeffer, and D. Szafron, "Opponent modeling in poker," in Proceedings of the Fifteenth National Conference on Artificial Intelligence, Madison, Wisconsin, pp. 493-499, 1998.
5. J. Bruce, M. Bowling, B. Browning, and M. Veloso, "Multi-robot team response to a multi-robot opponent team," in Proceedings of IROS-2002 workshop on Collaborative Rabots, 2002.
6. D. Carmel, and S. Markovitch, "Learning models of the opponent's strategy in game-playing," in, Proceedings of The AAAI Fall Symposium on Games: Planning and Learning, North Carolina, 1993.
7. D. Carmel, and S. Markovitch, "Incorporating Opponent Models into Adversary Search". in, Proceedings of the Thirteenth National Conference on Artificial Intelligence. Portland, Oregon, pp. 120-125.
8. D. Carmel, and S. Markovitch, "Learning and using opponent models in adversary search," Technical Report CIS9609, Technion, 1996b.
9. D. Carmel, and S. Markovitch, "Learning models of intelligent agents," in, Proceedings of the Thirteenth National Conference on Artificial Intelligence. Portland, Oregon, pp. 62-67, 1996c.
10. D. Carmel, and S. Markovitch, "Model-based learning of interaction strategies in multi-agent systems," Journal of Experimental and Theoretical Artificial Intelligence, vol.10, no. 3, pp.309-332, 1998.
11. D. Carmel, and S. Markovitch, "Exploration strategies for model-based learning in multiagent systems," Autonomous Agents and Multi-agent Systems, vol. 2, no. 2, pp. 141-172,1999.
12. H.Donkers, J. Uiterwijk, and H. van den Herik, "Probabilistic opponent-model search," Information Sciences vol. 135 no. 3-4, 123-149, 2001.
13. Y. Freund, M. Kearns, Y. Mansour, D. Ron, and R. Rubinfeld, "Efficient algorithms for learning to play repeated games against computationally bounded adversaries," in, Proceeding, of the 36th Annual Symposium on Foundations of Computer Science. IEEE Computer Society Press, Los Alamitos, CA, pp. 332-341, 1995.
14. X. Gao, H. Iida, J. W. Uiterwijk, and H. J. van den Herik, "Strategies anticipating a difference in search depth using opponent-model search," Theoretical Computer Science, vol. 252 no. 1-2, pp. 83-104, 2001.
15. X. Gao, H. Iida, J. W. H. M. Uiterwijk, and H. J. van den Herik, "Performance of (D,d)-OM search in othello," in, Proceedings of JSSST 14th Conference, Shikawa, Japan, pp. 229-232, 1997.
16. X. Gao, H. Iida, J. W. H. M. Uiterwijk, and H. J. Van den Herik, "A Speculative Strategy," Lecture Notes in Computer Science, vol. 1558, pp.74-92, 1999.
17. P. J. Gmytrasiewicz, and E. H. Durfee, "A rigorous, operational formalization of recursive modeling," in V. Lesser and L. Gasser (eds.), Proceedings of the First International Conference on Multi-Agent Systems (ICMAS-95). San Francisco, CA, USA, AAAI Press, 1995.
18. P. J. Gmytrasiewicz, S. Noh, and T. Kellogg, "Bayesian update of recursive agent models," User Modeling and User-Adapted Interaction, An International Journal, Special Issue on Learning for User Modeling, vol. 8, no. 1-2, pp. 49-69, 1998.
19. F.-H. Hsu, T. Ananthraman, M. Campbell, and A. Nowatzyk, "Deep thought," in T. Marsland and J. Schaeffer (eds.), Computers, Chess and Cognition. Springer New York, pp. 55-78, 1990.
20. Y.-J. Hu, and D. F. Kibler, "Generation of attributes for learning algorithms," in, Proceedings of the Thirteenth National Conference on Artificial Intelligence and the Eighth Innovative Applications of Artificial Intelligence Conference, Menlo Park, AAAI Press / MIT Press, pp. 806-811, 1996.
21. H. Iida, J. W. H. M. Uiterwijk, H. J. van den Herik, and I. S. Herschberg, "Potential applications of opponent-model search, Part I: The Domain of Applicability," ICCA Journal vol. 16 no. (4), pp. 201-208, 1993.
22. H. Iida, J. W. H. M. Uiterwijk, H. J. van den Herik, and I. S. Herschberg, "Potential applications of opponent-model search, Part II: Risks and strategies," ICCA Journal, vol. 17, no. 1, pp. 10-14, 1994.
23. P. J. Jansen, "Using knowledge about the opponent in game-tree search," Ph.D. thesis, Carnegie Mellon University, 1992.
24. A. Junghanns, and J. Schaeffer, "Search versus knowledge in game-playing programs revisited". in Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence (IJCAI-97). Nagoya, Japan, pp. 692-697, 1997.
25. L. P. Kaelbling, M. L. Littman, and A. P. Moore, "Reinforcement learning, a survey," Journal of Artificial Intelligence Research, vol. 4, pp. 237-285, 1996.
26. M. L. Littman, "Markov games as a framework for multi-agent reinforcement learning," in, Proceedings of the 11th International Conference on Machine Learning (ML-94), New Brunswick, NJ, Morgan Kaufmann, pp. 157-163, 1994.
27. S. Markovitch, and D. Rosenstein, "Feature generation using general constructor functions," Machine Learning, vol. 49, pp. 59-98, 2001.
28. S.Markovitch, and Y. Sella, "Learning of resource allocation strategies for game playing," Computational Intelligence, vol. 12 no. (1), pp. 88-105, 1996.
29. C. J. Matheus, and L. A. Rendell, "Constructive induction on decision trees," in, N. S. Sridharan (ed.), Proceedings of the 11th International Joint Conference on Artificial Intelligence, Detroit, MI, USA, Morgan Kaufmann, pp. 645-650, 1989.
30. A. W. Moore, and C. G. Atkeson, "Prioritized sweeping, reinforcement learning with less data and less time," Machine Learning, vol. 13 , pp.103-130, 1993.
31. Y. Mor, C. Goldman, and J. Rosenschein, "Learn your opponent's strategy (in polynomial time)!," in, G. Weiss and S. Sen (eds.), Adaptation and Learning in Multi-agent Systems, Lecture Notes in Artificial Intelligence, vol. 1042. Springer-Verlag, 1996.
32. D. S. Nau, "Pathology on game trees: A summary of results," in Proceedings of the First National Conference on Artificial Intelligence, Stanford, California, pp. 102-104, 1980.
33. D. S. Nau, "An investigation of the causes of pathology in games," Artificial Intelligence vol. 19, pp. 257-278, 1982.
34. G. Pagallo, and D. Haussler, "Boolean feature discovery in empirical learning," Machine Learning vol. 5 no. 1, pp. 71-99, 1990.
35. J. Pearl, "On the nature of pathology in game searching," Artificial Intelligence, vol. 20, pp. 427-453, 1983.
36. J. R.Quinlan, "Induction of decision trees," in Machine Learning, vol. 1, Morgan Kaufmann, pp. 81-106, 1986.
37. A. Reibman, and B. Ballard, "Non-minimax strategies for use against fallible opponents," in, Proceedings of the international conference on artificial intelligence AAAI-83, Los Altos, CA, William Kaufman,pp. 338-343, 1983.
38. S. Russell, and E. Wefald, Do the right thing: studies in limited rationality, Artificial Intelligence, Cambridge, Mass, MIT Press, 1991.
39. T. W. Sandholm, and R. H. Crites, "Multiagent reinforcement learning and the iterated Prisoner's Dilemma," Biosystems Journal vol. 37, 147-166, 1995.
40. R. Schapire, P. Stone, D. McAllester, M. Littman, and J. Csirik, "Modeling auction price uncertainty using boosting-based conditional density estimation," in Proceedings of the Nineteenth International Conference on Machine Learning, 2002.
41. S. Sen, and N. Arora, "Learning to take risks," in AAAI-97 Workshop on Multiagent Learning, pp. 59-64, 1997.
42. S. Sen, and G. Weiss, "Learning in multiagent systems," in, G. Weiss (ed.), Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence. Cambridge, Massachusetts, The MIT Press, Chapt. 6, pp. 259-298, 1999.
43. H. A. Simon, Models of Bounded Rationality, Volume 1. Cambridge, Massachusetts, The MIT Press, 1982.
44. P.Stone, P. Riley, and M. Veloso, "Defining and using ideal teammate and opponent agent models," in, Proceedings of the 7th Conference on Artificial Intelligence (AAAI-00) and of the 12th Conference on Innovative Applications of Artificial Intelligence (IAAI-00). Menlo Park, CA, AAAI Press, pp. 1040-1045, 2000.
45. R.Sutton, "Integrated architectures for learning, planning, and reacting based on approximating dynamic programming," in Proceedings of the Seventh International Conference on Machine Learning. pp. 216-224, 1990.
46. W. T. B. Uther, and M. M. Veloso, "Generalizing adversarial reinforcement learning," in Proceedings of the AAAI Fall Symposium on Model Directed Autonomous Systems, 1997.
47. J. M. Vidai, and E. H. Durfee, "The impact of nested agent models in an information economy," in, V. Lesser (ed.), Proceedings of the Second International Conference on Multi-Agent Systems (IC-MAS'96). Kyoto, Japan, The MIT Press, Cambridge, MA, USA, 1995.
48. J. M. Vidai, and E. H. Durfee, "Using recursive agent models effectively," in M. Wooldridge, J. P. Müller, and M. Tambe (eds.), Proceedings on the IJCAI Workshop on Intelligent Agents II: Agent Theories, Architectures, and Languages, vol. 1037 of LNAI. Springer-Verlag, Heidelberg, Germany, pp. 171-186, 1996.
49. C. J. Watkins, "Learning from delayed rewards," Ph.D. thesis, University of Cambridge, 1989.
50. C. J. Watkins, and P. Dayan, "Q-Learaing," Machine Learning vol. 8, pp. 279-292, 1992.
51. G. Weiss, and S. Sen, Adaptation and learning in multi-agent systems, Lectures Notes in Articial Intelligence, vol. 1042. Springer-Verlag, 1996.
52. S. Zilberstein,"Optimizing decision quality with contract algorithms". in, Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence. Montreal, Canada, pp. 1576-1582, 1995.