Self-organizing neural architectures and cooperative learning in a multiagent environment

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 37, Issue 6, 2007, Pages 1567-1580

  Xiao, Dan ^a   Tan, Ah Hwee ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Learning (artificial intelligence); Multi agent systems; Multiagent cooperative learning; Neural net architecture; Reinforcement learning (RL); Self organizing neural architectures

Indexed keywords

BACKPROPAGATION ALGORITHMS; FEEDFORWARD NEURAL NETWORKS; REINFORCEMENT LEARNING; SELF ORGANIZING MAPS;

MULTIAGENT COOPERATIVE LEARNING; RESILIENT-PROPAGATION ALGORITHM; SELF-ORGANIZING NEURAL ARCHITECTURE;

MULTI AGENT SYSTEMS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; METHODOLOGY; THEORETICAL MODEL;

ALGORITHMS; COMPUTER SIMULATION; DECISION SUPPORT TECHNIQUES; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED;

EID: 36749092785     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2007.907040     Document Type: Article

References (50)

1. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction Cambridge, MA: MIT Press, 1998.
2. W. S. Lovejoy, "Computationally feasible bounds for partially observed Markov decision processes," Oper. Res., vol. 39, no. 1, pp. 162-175, Jan./Feb. 1991.
3. A. H. Tan, "FALCON: A fusion architecture for learning, cognition, and navigation," in Proc. IJCNN, Budapest, Hungary, 2004, pp. 3297-3302.
4. A.-H. Tan and D. Xiao, "Self-organizing cognitive agents and reinforcement learning in a multi-agent environment," in Proc. IEEE/ACM/WIC Int. Conf. Intell. Agent Technol., 2005, pp. 351-357.
5. A. H. Tan, N. Lu, and D. Xiao, "Integrating temporal difference methods and self-organizing neural networks for reinforcement learning with delayed evaluative feedback," IEEE Trans. Neural Netw. to be published.
6. D. Gordan and D. Subramanian, "A cognitive model of learning to navigate," in Proc. 19th Annu. Conf. Cognitive Sci. Soc., 1997, pp. 271-276.
7. R. Sun, E. Merrill, and T. Peterson, "From implicit skills to explicit knowledge: A bottom-up model of skill learning," Cogn. Sci., vol. 25, no. 2, pp. 203-244, Mar. 2001.
8. M. Riedmiller and H. Braun, "RPROP - A fast adaptive learning algorithm," Univ. Karlsruhe, Karlsruhe, Germany, 1992. Tech. Rep. (Also Proc. of ISCIS VII).
9. M. Riedmiller and H. Braun, "A direct adaptive method for faster back-propagation learning: The RPROP algorithm," in Proc. IEEE Int. Conf. Neural Netw., San Francisco, CA, 1993, pp. 586-591.
10. M. Brenda, V. Jagannathan, and R. Dodhiawala, "On optimal cooperation of knowledge sources - An empirical investigation," Boeing Adv. Technol. Center, Boeing Comput. Services, Seattle, WA, Tech. Rep. BCS-G2010-28, 1986.
11. L. Panait and S. Luke, "Cooperative multi-agent learning: The state of the art," George Mason Univ., Fairfax, VA, Tech. Rep. GMU-CS-TR-2003-1, 2003.
12. M. Dubreuil, C. Gagne, and M. Parizeau, "Analysis of a master-slave architecture for distributed evolutionary computations," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 36, no. 1, pp. 229-235, Feb. 2006.
13. H. P. Schwefel, Advantages and Disadvantages of Evolutionary Computation Over Other Approaches, Evolutionary Computation 1: Basic Algorithms and Operators. Bristol, PA: Inst. Phys. Publishing, 2000.
14. M. Kaya and R. Alhajj, "Modular fuzzy-reinforcement learning approach with internal model capabilities for multiagent systems" IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 34, no. 2, pp. 1210-1223, Apr. 2004.
15. M. Kaya and R. Alhajj, "Fuzzy OLAP association rules mining-based modular reinforcement learning approach for multiagent systems," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 35, no. 2, pp. 326-338, Apr. 2005.
16. C. J. C. H. Watkins and P. Dayan, "Q-learning," Mach. Learn., vol. 8, no. 3/4, pp. 279-292, 1992.
17. H. Kose, U. Tatlidede, C. Mericli, K. Kaplan, and H. L. Akin, "Q-learning based market-driven multi-agent collaboration in robot soccer," in Proc. Turkish Symp. Artif. Intell. Neural Netw., Izmir, Turkey, 2004, pp. 219-228.
18. V. Kononen, "Gradient descent for symmetric and asymmetric multiagent reinforcement learning," Web Intell. Agent Syst.: Int. J. (WIAS), vol. 3, no. 1, pp. 17-30, 2005.
19. E. F. Yang and D. B. Gu, "Multiagent reinforcement learning for multi-robot systems: A survey," Dep. Comput. Sci., Univ. Essex, Colchester, U.K., Tech. Rep. CSM-404, 2004.
20. K. S. Hwang, S. W. Tan, M. C. Hsiao, and C. S. Wu, "Cooperative multiagent congestion control for high-speed networks," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 35, no. 2, pp. 255-268, Apr. 2005.
21. L. Bull and T. C. Fogarty, "Evolving cooperative communicating classifier systems," in Proc. 4th Annu. Conf. Evol. Program., 1994, pp. 308-315.
22. T. Balch, "Reward and diversity in multirobot foraging," in Proc. Agents Learning About, From With Other Agents Workshop, Stockholm, Sweden, 1999.
23. J. M. Vidal and E. H. Durfee, "The moving target function problem in multi-agent learning," in Proc. 3rd Int. Conf. Multi-Agent Syst., Paris, France, Jul. 1998, pp. 317-324.
24. G. Chalkiadakis and C. Boutilier, "Coordination in multiagent reinforcement learning: A Bayesian approach," in Proc. 2nd Int. Joint Conf. Anton. Agents Multiagent Syst., 2003, pp. 709-716.
25. D. H. Wolpert and K. Turner, "Optimal payoff functions for members of collectives," Adv. Complex Systems, vol. 4, no. 2/3, pp. 265-279, 2001.
26. T. Balch, "Learning roles: Behavioural diversity in robot teams," Georgia Inst. Technol., Atlanta, GA, Tech. Rep. GIT-CC-97-12, 1997.
27. M. Mataric, "Learning to behave socially," in Proc. 3rd Int. Conf. Simul. Adaptive Behaviour, 1994, pp. 453-462.
28. C. Claus and C. Boutilier, "The dynamics of reinforcement learning in cooperative multiagent systems," in Proc. Nat. Conf. Artif. Intell. 1998, pp. 746-752.
29. M. Bowling and M. Veloso, "An analysis of stochastic game theory for multiagent reinforcement learning," Comput. Sci. Dept., Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-CS-00-165, 2000.
30. R. Mukherjee and S. Sen, "Towards a Pareto-optimal solution in general-sum games," in Proc. 2nd Int. Joint Conf. AAMAS, Melbourne, Australia, 2003, pp. 153-160.
31. M. Mundhe and S. Sen, "Evaluating concurrent reinforcement learners," in Proc. 4th ICMAS, Boston, MA, 2000, pp. 421-422.
32. S. Sen and M. Sekaran, "Individual learning of coordination knowledge," J. Exp. Theor. Artif. Intell., vol. 10, no. 3, pp. 333-356, Jul. 1998.
33. S. Sen M. Sekaran, and J. Hale, "Learning to coordinate without sharing information," in Proc. 12th Nat. Conf. Artif. Intell., Seattle, WA, 1994, pp. 426-431.
34. M. L. Littman, "Markov games as a framework for multiagent reinforcement learning," in Proc. 11th Int. Conf. Machine Learning San Francisco, CA, 1994, pp. 157-163.
35. M. Tan, "Multi-agent reinforcement learning: Independent vs. cooperative agents," in Proc. 10th Int. Conf. Machine Learning, 1993, pp. 330-337.
36. J. Hu and M. Wellman, "Multiagent reinforcement learning: Theoretical framework and an algorithm," in Proc. 15th Int. Conf. Machine Learning, 1998, pp. 242-250.
37. A. Agogino and K. Turner, "Reinforcement learning in large multi-agent systems," in Proc. AAMAS Workshop Coordination Large Scale Multi-Agent Syst., Utrecht, The Netherlands, 2005.
38. G. A. Carpenter and S. Grossberg, "A massively parallel architecture for a self-organizing neural pattern recognition machine,:Comput. Vis. Graph. Image Process., vol. 37, no. 1, pp. 54-115, Jan. 1987.
39. G. A. Carpenter and S. Grossberg, "ART 2: Self-organization of stable category recognition codes for analog input patterns," Appl. Opt. vol. 26, no. 23, pp. 4919-4930, Dec. 1987.
40. B. Moore, "ART 1 and pattern clustering," in Proc. Connectionist Models Summer School, 1988, pp. 174-185.
41. G. A. Carpenter, S. Grossberg, and D. B. Rosen, "Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system," Neural Netw., vol. 4, pp. 759-771, 1991.
42. A. Pérez-Uribe, "Structure-adaptable digital neural networks," Ph.D. dissertation, Swiss Federal Inst. Technol.-Lausanne, Lausanne, Switzerland, 2002.
43. R. Sun, T. Peterson, and E. Merrill, "Bottom-up skill learning in reactive sequential decision tasks," in Proc. 18th Cognitive Sci. Soc. Conf., 1996, pp. 684-690.
44. A. D. Anastasiadis, G. D. Magoulas, and M. N. Vrahatis, "A new learning rates adaptation strategy for the resilient propagation algorithm," in Proc. ESANN, 2004, pp. 1-6.
45. P. Stone, "Layered learning in multiagent systems," in Proc. AAAI/IAAI, 1997, p. 819.
46. J. Denzinger and M. Fuchs, "Experiments in learning prototypical situations for variants of the pursuit game," in Proc. 2nd ICMAS, Kyoto, Japan, 1996, pp. 48-55.
47. J. Denzinger and A. Schur, "On customizing evolutionary learning of agent behavior," in Proc. Can. Conf. AI, 2004, pp. 146-160.
48. T. W. Sandholm and R. H. Crites, "Multiagent reinforcement leaning in the iterated prisoner's dilemma," Biosystems, vol. 37, no. 1, pp. 147-166, 1995.
49. N. Ono and K. Fukomoto, "Multi-agent reinforcement learning: A modular approach," in Proc. 2nd Int. Conf. Multi-Agent Syst., 1996, pp. 252-258.
50. S. Arai and K. Sycara, "Effective learning approach for planning and scheduling in multi-agent domain," in Proc. 6th Int. Conf. Simulation Adaptive Behaviour (From Animals to Animats 6), 2000, pp. 507-516.