Hierarchical multi-agent reinforcement learning

Autonomous Agents and Multi-Agent Systems

Volumn 13, Issue 2, 2006, Pages 197-229

  Ghavamzadeh, Mohammad ^a   Mahadevan, Sridhar ^b   Makar, Rajbala ^c  

^a UNIVERSITY OF ALBERTA   (Canada)

^b Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

^c AGILENT TECHNOLOGIES   (United States)

Author keywords

Communication; Cooperative multi agent systems; Coordination; Hierarchical reinforcement learning

Indexed keywords

EID: 33846942607     PISSN: 13872532     EISSN: 15737454     Source Type: Journal    
DOI: 10.1007/s10458-006-7035-4     Document Type: Article

References (45)

1. Askin, R., & Standridge, C. (1993). Modeling and analysis of manufacturing systems. John Wiley and Sons.
2. Balch, T., & Arkin, R. (1998). Behavior-based formation control for multi-robot Teams. IEEE Transactions on Robotics and Automation, 14, 1-15.
3. Barto, A., & Mahadevan, S. (2003) Recent advances in hierarchical reinforcement learning, Discrete Event Systems Special Issue on Reinforcement Learning, 13, 41-77.
4. Bernstein, D., Zilberstein, S., & Immerman, N. (2000). The complexity of decentralized control of markov decision processes. In Proceedings of the sixteenth international conference on uncertainty in artificial intelligence (pp. 32-37).
5. Boutilier, C. (1999). Sequential optimality & coordination in multi-agent systems. In Proceedings of the sixteenth international joint conference on artificial intelligence (pp. 478-485).
6. Bowling, M., & Veloso, M. (2002). Multiagent learning using a variable learning rate. Artificial Intelligence, 136, 215-250.
7. Bui, H., Venkatesh, S., & West, G. (2002). Policy recognition in the abstract hidden markov model. Journal of Artificial Intelligence Research, 17, 451-499.
8. Crites, R., & Barto, A. (1998). Elevator group control using multiple reinforcement learning agents. Machine Learning, 33, 235-262.
9. Dietterich, T. (2000). Hierarchical reinforcement learning with the MAXQ value function decomposition. Journal of Artificial Intelligence Research, 13, 227-303.
10. Filar, J., & Vrieze, K. (1997). Competitive Markov decision processes. Springer Verlag.
11. Ghavamzadeh, M., & Mahadevan, S. (2003). Hierarchical policy gradient algorithms. In Proceedings of the twentieth international conference on machine learning (pp. 226-233).
12. Ghavamzadeh, M., & Mahadevan, S. (2004). Learning to communicate and act using hierarchical reinforcement learning. In Proceedings of the third international joint conference on autonomous agents and multiagent systems (pp. 1114-1121).
13. Guestrin, C., Lagoudakis, M., & Parr, R. (2002). Coordinated reinforcement learning. In Proceedings of the nineteenth international conference on machine learning (pp. 227-234).
14. Howard, R. (1971). Dynamic probabilistic systems: Semi-Markov and decision processes. John Wiley and Sons.
15. Hu, J., & Wellman, M. (1998). Multiagent reinforcement learning: Theoretical framework and an algorithm. In Proceedings of the fifteenth international conference on machine learning (pp. 242-250).
16. Kearns, M., Littman, M., & Singh, S. (2001). Graphical models for game theory. In Proceedings of the seventeenth international conference on uncertainty in artificial intelligence (pp. 253-260).
17. Klein, C., & Kim, J. (1996). AGV dispatching. International Journal of Production Research, 34, 95-110.
18. Koller, D., & Milch, B. Multiagent influence diagrams for representing and solving games. In Proceedings of the seventeenth international joint conference on artificial intelligence (pp. 1027-1034).
19. La Mura, P. (2000). Game Networks. In Proceedings of the sixteenth international conference on uncertainty in artificial intelligence.
20. Lee, J. (1996). Composite dispatching rules for multiple-vehicle agv systems. Simulation, 66, 121-130.
21. Littman, M. (1994). Markov games as a framework for multi-agent reinforcement learning. In Proceedings of the eleventh international conference on machine learning (pp. 157-163).
22. Littman, M. (2001). Friend-or-Foe Q-learning in general-sum games. In Proceedings of the eighteenth international conference on machine learning (pp. 322-328).
23. Littman, M., Kearns, M., & Singh, S. (2001). An efficient exact algorithm for singly connected graphical games. In Proceedings of neural information processing systems (pp. 817-824).
24. Makar, R., Mahadevan, S., & Ghavamzadeh, M. (2001). Hierarchical multi-agent reinforcement learning. In Proceedings of the fifth international conference on autonomous agents (pp. 246-253).
25. Mataric, M. (1997). Reinforcement learning in the multi-robot domain (1997). Autonomous Robots, 4, 73-83.
26. Ortiz, L., & Kearns, M. (2002). Nash propagation for loopy graphical games. In Proceedings of neural information processing systems.
27. Owen, G. (1995). Game theory. Academic Press.
28. Parr, R. (1998). Hierarchical control and learning for Markov decision processes, PhD thesis, University of California, Berkeley.
29. Peshkin, L., Kim, K., Meuleau, N., & Kaelbling, L. (2000). Learning to cooperate via policy search. In Proceedings of the sixteenth international conference on uncertainty in artificial intelligence (pp. 489-496).
30. Puterman, M. (1994). Markov decision processes. Wiley Interscience.
31. Pynadath, D., & Tambe, M. (2002). The communicative multiagent team decision problem: Analyzing teamwork theories and models. Journal of Artificial Intelligence Research, 16, 389-426.
32. Rohanimanesh, K., & Mahadevan, S. Learning to take concurrent actions. In Proceedings of the sixteenth annual conference on neural information processing systems.
33. Saria, S., & Mahadevan, M. (2004). Probabilistic plan recognition in multiagent systems. In Proceedings of the fourteenth international conference on automated planning and scheduling (pp. 12-22).
34. Schneider, J., Wong, W., Moore, A., & Riedmiller, M. Distributed value functions. In Proceedings of the sixteenth international conference on machine learning (pp. 371-378).
35. Singh, S., Kearns, M., & Mansour, Y. (2000). Nash convergence of gradient dynamics in general-sum games. In Proceedings of the sixteenth international conference on uncertainty in artificial intelligence (pp. 541-548).
36. Stone, P., & Veloso, M. (1999). Team-partitioned, opaque-transition reinforcement learning. In Proceedings of the third international conference on autonomous agents (pp. 206-212).
37. Sugawara, T., & Lesser, V. Learning to improve coordinated actions in cooperative distributed problem-solving environments. Machine Learning, 33, 129-154.
38. Sutton, R., Precup, D., & Singh, S. (1999). Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112, 181-211.
39. Tadepalli, P., & Ok, D. Scaling up average reward reinforcement learning by approximating the domain models and the value function. In Proceedings of the thirteenth international conference on machine, learning (pp. 471-479).
40. Tan, M. (1993). Multi-agent reinforcement learning: Independent vs. cooperative agents. In Proceedings of the tenth international conference on machine learning (pp. 330-337).
41. Vickrey, D., & Koller, D. (2002). Multiagent algorithms for solving graphical games. In Proceedings of the national conference, on artificial intelligence, (pp. 345-351).
42. Watkins, C. (1989). Learning from delayed rewards, PhD thesis, Kings College, Cambridge, England.
43. Weiss, G. (1999). Multi-agent systems: A modern approach to distributed artificial intelligence. MIT Press.
44. Xuan, P., Lesser, V., & Zilberstein, S. (2001). Communication decisions in multi-agent cooperation: Model and experiments. In Proceedings of the fifth international conference on autonomous agents (pp. 616-623).
45. Xuan, P., & Lesser, V. (2002). Multiagent policies: From centralized ones to decentralized ones. In Proceedings of the first international joint conference on autonomous agents and multiagent systems (pp. 1098-1105).