Cooperative information sharing to improve distributed learning in multi-agent systems

Journal of Artificial Intelligence Research

Volumn 24, Issue , 2005, Pages 407-463

  Dutta, Partha S ^a   Jennings, Nicholas R ^a   Moreau, Luc ^a  

^a UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

BENCHMARKING; HEURISTIC METHODS; INFORMATION ANALYSIS; LEARNING ALGORITHMS; MULTI AGENT SYSTEMS; ROUTERS;

BENCHMARK STRATEGIES; DISTRIBUTED LEARNING; INFORMATION-SHARING PROTOCOLS; STATE INFORMATION;

LEARNING SYSTEMS;

EID: 31144432283     PISSN: 10769757     EISSN: 10769757     Source Type: Journal    
DOI: 10.1613/jair.1735     Document Type: Article

References (65)

1. Allsopp, D. N., Beautement, P., Bradshaw, J. M., Durfee, E. H., Kirton, M., Knoblock, C. A., Suri, N., Tate, A., & Thompson, C. W. (2002). Coalition agents experiment: Multiagent cooperation in international coalitions. IEEE Intelligent Systems, 17(3), 26-35.
2. Baxter, J., & Bartlett, P. L. (1999). Direct gradient-based reinforcement learning: I. Gradient estimation algorithms. Tech. rep., Research School of Information Science and Engineering, Australian National University.
3. Becker, R., Zilberstein, S., Lesser, V., & Goldman, C. V. (2003). Transition-independent decentralized markov decision processes. In Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2003), pp. 41-48, Melbourne. ACM Press, New York.
4. Bernstein, D., Givan, R., Immerman, N., & Zilberstein, S. (2002). The complexity of decentralized control of markov decision processes. Mathematics of Operations Research, 27(4), 819-840.
5. Boutilier, C. (1999). Sequential optimality and coordination in multiagent systems. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence (IJCAI-99), pp. 478-485, Stockholm.
6. Boyan, J. A., & Littman, M. L. (1993). Packet routing in dynamically changing networks: A reinforcement learning approach. In Cowan, J. D., Tesauro, G., & Alspector, J. (Eds.), Advances in Neural Information Processing Systems, Vol. 6, pp. 671-678.
7. Carley, K. M., & Gasser, L. (1999). Computational organisation theory. In Weiss, G. (Ed.), Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence, pp. 299-330. The MIT Press, Cambridge, MA.
8. Caro, G. D., & Dorigo, M. (1998a). AntNet: Distributed stigmergetic control for communications networks. Journal of Artificial Intelligence Research, 9, 317-365.
9. Caro, G. D., & Dorigo, M. (1998b). Two ant colony algorithms for best-effort routing in datagram networks. In Proceedings of the Tenth IASTED International Conference on Parallel and Distributed Computing and Systems, pp. 541-546. IASTED/ACTA Press.
10. Chaib-draa, B. (1995). Industrial applications of distributed AI. Communications of the ACM, 38(11), 49-53.
11. Cockburn, D., & Jennings, N. R. (1996). Archon: A distributed artificial intelligence system for industrial applications. In O'Hare, G. M. P., & Jennings, N. R. (Eds.), Foundations of Distributed Artificial Intelligence, pp. 319-344. Wiley.
12. Cohen, P. R., & Levesque, H. J. (1991). Teamwork. Nous, 55(4), 487-512. Special Issue on Cognitive Science and Artificial Intelligence.
13. Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2001). Introduction to algorithms (2nd edition)., chap. 24: Single Source Shortest Paths. MIT Press.
14. Decker, K. (1995a). TAEMS: A framework for environment centered analysis and design of coordination mechanisms. In O'Hare, G., &: Jennings, N. (Eds.), Foundations of Distributed Artificial Intelligence, chap. 16. Wiley Inter-Science.
15. Decker, K. S. (1995b). Environment centered analysis and design of coordination mechanisms. Ph.D. thesis, University of Massachusetts, Amherst, Massachusetts.
16. Decker, K. S., &: Lesser, V. R. (1995). Designing a family of coordination algorithms. In Proceedings of the First International Conference on Multi-agent Systems, pp. 73-80, San Fransisco.
17. Denkena, B., Zwich, M., & Woelk, P. (2004). Holonic and Multi-Agent Systems for Manufacturing, Vol. 2744/2004 of Lecture Notes in Computer Science, chap. Multiagent-based process planning and scheduling in context of supply chains, pp. 100-109. Springer-Verlag, Heidelberg.
18. Durfee, E. H., & Lesser, V. (1991). Partial global planning: A coordination framework for distributed hypothesis formation. IEEE Transactions on Systems, Man, and Cybernetics, 21(5), 1167-1183.
19. Dutta, P. S., Dasmahapatra, S., Gunn, S. R., Jennings, N. R., & Moreau, L. (2004). Cooperative information sharing to improve distributed learning. In AAMAS-04 workshop on Learning and Evolution in Agent-Based Systems, pp. 18-23.
20. Dutta, P. S., Jennings, N. R., &: Moreau, L. (2005). Sharing information for q-learning-based network bandwidth estimation and network failure detection (poster). In Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems (to appear).
21. Dutta, P. S., Moreau, L., & Jennings, N. R. (2003). Finding interaction partners using cognition-based decision strategies. In Working notes of the IJCAI-2003 workshop on Cognitive Modeling of Agents and Multi-Agent Interactions, pp. 46-55.
22. Dutta, P. S., & Sen, S. (2003). Forming stable partnerships. Cognitive Systems Research, 4(3), 211-221.
23. Ernst, D., Glavic, M., & Wehenkel, L. (2004). Power systems stability control : Reinforcement learning framework. IEEE Transactions on Power Systems, 19(1), 427-435.
24. Feinberg, E., & Schwartz, A. (2001). Handbook of Markov Decision Processes: Models and Applications. Kluwer Academic Publishers.
25. Grosz, B., & Kraus, S. (1996). Collaborative plans for complex group action. Artificial Intelligence, 86(2), 269-357.
26. Hedrick, C. (1988). Routing Information Protocol (RFC 1058). Network Working Group, IETF. http://www.ietf.org/rfc/rfc1058.txt.
27. Jain, M., & Dovrolis, C. (2002). Pathload: A measurement tool for end-to-end available bandwidth. In Proceedings of the Passive and Active Measurements (PAM) Workshop, pp. 14-25.
28. Jennings, N. R. (1993). Commitments and conventions: The foundation of coordination in multi-agent systems. The Knowledge Engineering Review, 8(3), 223-250.
29. Jennings, N. R. (1995). Controlling cooperative problem solving in industrial multi-agent systems using joint intentions. Artificial Intelligence, 75(2), 195-240.
30. Jennings, N. R., & Bussmann, S. (2003). Agent-based control systems. IEEE Control Systems Magazine, 23(3), 61-74.
31. Jennings, N. R., Norman, T. J., & Faratin, P. (1998). Adept: An agent-based approach to business process management. ACM SIGMOD, 27(4), 32-39.
32. Krag, T., &: Buettrich, S. (2004). Wireless Mesh Networking. O'Reilly Wireless Devcenter. http://www.oreillynet.com/pub/a/wireless/2004/01/22/wirelessmesh.html.
33. Lai, K., & Baker, M. (1999). Measuring bandwidth. In Proceedings of the IEEE INFOCOM (1), pp. 235-245.
34. Lesser, V. R., &: Erman, L. D. (1988). Distributed interpretation: A model and experiment. In Bond, A. H., & Gasser, L. (Eds.), Readings in Distributed Artificial Intelligence, pp. 120-139. Morgan Kaufmann, San Mateo, CA.
35. Mahadevan, S., Marchalleck, N., Das, T., & Gosavi, A. (1997). Self-improving factory simulation using continuous-time average reward reinforcement learning. In Proceedings of the Fourteenth International Machine Learning Conference, pp. 202-210. Morgan Kaufmann.
36. Mailler, R., Lesser, V., & Horling, B. (2003). Cooperative negotiation for soft real-time distributed resource allocation. In Proceedings of the Second International Joint Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2003), pp. 576-583, Melbourne. ACM Press.
37. Maley, J. (1988). Managing the flow of intelligent parts. Robotics and Computer-Integrated Manufacturing, 4(3/4), 525-530.
38. March, J. G., & Simon, H. A. (1958). Organizations. John Wiley & Sons.
39. McCallum, R. A. (1996). Hidden state and reinforcement learning with instance-based state identification. IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics 26), 464-473.
40. Minar, N., Kramer, K. H., & Maes, P. (1999). Cooperating mobile agents for dynamic network routing. In Proceedings of the Software Agents for Future Communications Systems. Springer-Verlag. ISBN 3-540-65578-6.
41. Mitchell, T. M. (1997). Machine Learning, chap. 13: Reinforcement Learning. McGraw-Hill.
42. Parunak, H. V. D., Baker, A. D., & Clark, S. J. (2001). The aaria agent architecture: From manufacturing requirements to agent-based system design. Integrated Computer-Aided Engineering, 8(1), 45-58.
43. Peshkin, L., & Savova, V. (2002). Reinforcement learning for adaptive routing. In Proceedings of the International Joint Conference on Neural Networks (IJCNN).
44. Ribeiro, V., Coates, M., Riedi, R., Sarvotham, S., Hendricks, B., & Baraniuk, R. (2000). Multifractal cross-traffic estimation. In ITC Conference on IP Traffic, Modeling and Management.
45. Rich, C., & Sidner, C. L. (1997). COLLAGEN: When agents collaborate with people. In Proceedings of the First International Conference on Autonomous Agents (Agents '97), pp. 284-291.
46. Russel, S. J., & Norvig, P. (2002). Artificial Intelligence: A Modern Approach (2nd edition)., chap. 17: Making Complex Decisions. Prentice Hall.
47. Shen, J., Lesser, V., & Carver, N. (2003). Minimizing communication cost in a distributed Bayesian network using a decentralized MDP. In Proceedings of the Second International Joint Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2003), pp. 678-685. ACM Press.
48. Singh, S. P., Jaakkola, T., & Jordan, M. I. (1995). Reinforcement learning with soft state aggregation. In Tesauro, G., Touretzky, D., & Leen, T. (Eds.), Advances in Neural Information Processing Systems, Vol. 7, pp. 361-368. The MIT Press.
49. Stone, P. (2000). TPOT-RL applied to network routing. In Proceedings of ICML 2000, pp. 935-942.
50. Stone, P., & Veloso, M. (1999). Team partitioned, opaque transition reinforcement learning. In Proceedings of the Third Annual Conference on Autonomous Agents, pp. 206-212.
51. Sutton, R., McAllester, D., Singh, S., & Mansour, Y. (2000). Policy-gradient methods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems, 12, 1057-1063.
52. Sutton, R. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3, 9-44.
53. Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning). MIT Press.
54. Takahashi, E., & Tanaka, Y. (2003). Auction-based effective bandwidth allocation mechanism. Telecommunications Systems, 24(2), 323-338.
55. Tambe, M. (1997). Towards flexible teamwork. Journal of Artificial Intelligence Research, 7, 83-124.
56. th edition)., chap. 5: The Network Layer. Prentice Hall PTR.
57. Tong, H. (2002). Reinforcement learning for call admission control and routing under quality of service constraints in multimedia networks. Machine Learning, 49(2), 111-139.
58. Viswanathan, R., & Varshney, P. K. (1997). Distributed detection with multiple sensors: Part I - fundamentals. In Proceedings of the IEEE, Vol. 85-1, pp. 54-63.
59. Walsh, W. E., & Wellman, M. P. (2003). Decentralized supply chain formation: A market protocol and competitive equilibrium analysis. Journal of Artificial Intelligence Research, 19, 513-567.
60. Watkins, C. J. C. H. (1989). Learning from delayed rewards. Ph.D. thesis, Psychology Department, University of Cambridge.
61. Watkins, C. J. C. H., & Dayan, P. (1992). Technical note: Q-learning. Machine Learning, 8, 279-292.
62. Widrow, B., & Hoff, M. E. (1960). Adaptive switching circuits. In WESCON Convention Record Part IV, pp. 96-104.
63. Williams, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3), 229-256.
64. Xiang, Y. (1996). A probabilistic framework for cooperative multi-agent distributed interpretation and optimization of communication. Artificial Intelligence, 87(1-2), 295-342.
65. Xuan, P., Lesser, V., & Zilberstein, S. (2001). Communication decisions in multi-agent cooperation: model and experiments. In Proceedings of the Fifth International Conference on Autonomous Agents (Agents-01), pp. 616-623, Montreal.