A study on expertise of agents and its effects on cooperative Q-learning

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 37, Issue 2, 2007, Pages 398-409

  Araabi, Babak Nadjar ^a   Mastoureshgh, Sahar ^a   Ahmadabadi, Majid Nili ^a,b  

^a UNIVERSITY OF TEHRAN   (Iran)

^b Warner Bros   (United States)

Author keywords

Area of expertise (AOE); Cooperative Q learning agents; Cooperative Q learning using AOE; Extraction of AOE; Multiagent (MASs)

Indexed keywords

ARTIFICIAL INTELLIGENCE; EXPERT SYSTEMS; LEARNING ALGORITHMS; LEARNING SYSTEMS;

Q-LEARNING; Q-TABLES; REINFORCEMENT-LEARNING HOMOGENEOUS AGENTS; STRATEGY SHARING (SS);

MULTI AGENT SYSTEMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; COOPERATION; DECISION SUPPORT SYSTEM; EXPERT SYSTEM; METHODOLOGY; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; COOPERATIVE BEHAVIOR; DECISION SUPPORT TECHNIQUES; EXPERT SYSTEMS; MODELS, THEORETICAL; PATTERN RECOGNITION, AUTOMATED;

EID: 34047122064     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2006.883264     Document Type: Article

References (24)

1. P. Stone and M. Veloso, "Multiagent systems: A survey from a machine learning perspective," Auton. Robots, vol. 8, no. 3, pp. 345-383, Jun. 2000.
2. G. Weiss, Multi Agent Systems: A Modern Approach to Distributed Artificial Intelligence, 2nd ed. Cambridge, MA: MIT Press, 2000.
3. Agent Platform Special Interest Group, Agent technology green paper, Sep. 2000, Needham, MA: Object Management Group (OMG).
4. M. N. Ahmadabadi and M. Asadpour, "Expertness-based cooperative Q-learning," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 32, no. 1, pp. 66-77, Feb. 2002.
5. M. Tan, "Multi-agent reinforcement learning: Independent vs. cooperative agents," in Proc. 10th Int. Conf. Mach. Learning, Amherst, MA, Jun. 1993, pp. 330-337.
6. C. F. Touzet, "Distributed lazy Q-1earning for cooperative mobile robots," Int. J. Adv. Robot. Syst., vol. 1, no. 1, pp. 5-13, Jan. 2004.
7. E. Curran, "SWARM: Cooperative reinforcement learning for routing in ad-hoc networks," M.S. thesis, Comput. Sci. Dept., Univ. Dublin, Dublin, Ireland, Sep. 2003.
8. J. R. Kok and N. Vlassis, "Sparse cooperative Q-learning," in Proc. 21st Int. Conf. Mach. Learning, Banff, AB, Canada, Jul. 4-8, 2004, pp. 481-488.
9. C. J. C. H. Watkins, "Learning with delayed rewards," Ph.D. dissertation, Psychol. Dept., Univ. Cambridge, Cambridge, U.K., 1989.
10. L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement learning: A survey," J. Artif. Intell. Res., vol. 4, pp. 237-285, 1996.
11. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
12. R. Bellman, Dynamic Programming. Princeton, NJ: Princeton Univ. Press, 1957.
13. P. Tangamchit, J. M. Dolan, and P. K. Khosla, "The necessity of average rewards in cooperative multirobot learning," in Proc. IEEE Int. Conf. Robot. Autom., Washington, DC, May 11-15, 2002, pp. 1296-1301.
14. L. Nunes and E. Oliveira, "Advice-exchange amongst heterogeneous learning agents: Experiments in the pursuit domain," in Proc. 2nd Int. Joint Conf. Auton. Agents and Multiagent Syst., Melbourne, Australia, Jul. 14-18, 2003, pp. 1084-1086.
15. L. Nunes and E. Oliveira, "Cooperative learning using advice exchange," in Lecture Notes in Artificial Intelligence, vol. 2636, E. Alonso, D. Kazakov, and D. Kudenko, Eds. New York: Springer-Verlag, Apr. 2003, pp. 33-48.
16. S. M. R. Mirfattah and M. N. Ahmadabadi, "Cooperative Q-learning with heterogeneity in actions," in Proc. IEEE Int. Conf. Syst. Man Cybern., Hammamet, Tunisia, Oct. 6-9, 2002.
17. C. Claus and C. Boutilier, "The dynamics of reinforcement learning in cooperative multiagent systems," in Proc. 15th Int. Conf. Artif. Intell., Menlo Park, CA, Aug. 1998, pp. 746-752.
18. J. Schmidhuber, "A general method for multi-agent reinforcement learning in unrestricted environments," AAAI, Menlo Park, CA, pp. 84-87, Tech. Rep. SS-96-01, Mar. 1996.
19. M. L. Littman, "Markov games as a framework for multi-agent reinforcement learning," in Proc. 11th Int. Conf. Mach. Learning, San Francisco, CA, 1994, pp. 157-163.
20. J. Hu and M. P. Wellman, "Multi-agent reinforcement learning: Theoretical framework and an algorithm," in Proc. 15th Int. Conf. Mach. Learning, Madison, WI, Jul. 1998, pp. 242-250.
21. S. Mastoureshgh, B. N. Araabi, and M. N. Ahmadabadi, "Cooperative Q-1earning through state transitions: A method for cooperation based on area of expertise," in Proc. 4th Asia-Pacific Conf. Simul. Evolution and Learning, Singapore, Nov. 2002, pp. 61-65.
22. A. Imanipour, M. Nili Ahmadabadi, B. N. Araabi, M. Asadpour, and R. Siegwart, "Knowledge-based extraction of area of expertise for cooperation in learning," in Proc. IEEE/RSJ Int. Conf. Intell. Robots and Syst. (IROS), Beijing, China, Oct. 2006, pp. 3700-3705.
23. D. W. McDonald and M. S. Ackerman, "Just talk to me: A field of expertise location," in Proc. ACM Conf. Comput.-Supported Cooperative Work, Seattle, WA, Nov. 14-18, 1998, pp. 315-324.
24. A. Harati, M. N. Ahmadabadi, and B. N. Araabi, "Knowledge based multiagent credit assignment: A study on task type and critic information," Intelligent Automation and Soft Computing (AutoSoft), vol. 13, no. 3, 2007, to be published.