A comprehensive survey of multiagent reinforcement learning

IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews

Volumn 38, Issue 2, 2008, Pages 156-172

  Buşoniu, Lucian ^a   Babuška, Robert ^a   De Schutter, Bart ^a,b  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^b DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

Distributed control; Game theory; Multiagent systems; Reinforcement learning

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; GAME THEORY; REINFORCEMENT LEARNING;

LEARNING DYNAMICS; MULTIAGENT REINFORCEMENT LEARNING; REPRESENTATIVE SELECTION;

MULTI AGENT SYSTEMS;

EID: 40949147745     PISSN: 10946977     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCC.2007.913919     Document Type: Review

References (132)

1. G. Weiss, Ed., Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence. Cambridge, MA: MIT Press, 1999.
2. N. Vlassis. (2003, Sep.) A concise introduction to multiagent systems and distributed AI, Fac. Sci. Univ. Amsterdam, Amsterdam, The Netherlands, Tech. Rep. [Online]. Available: http://www.science.uva.nl/ ~vlassis/cimasdai/cimasdai.pdf
3. H. V. D. Parunak, "Industrial and practical applications of DAI," in Multi-Agent Systems: A Modern Approach to Distributed Artificial Intelligence, G. Weiss, Ed. Cambridge, MA: MIT Press, 1999, ch. 9, pp. 377-412.
4. P. Stone and M. Veloso, "Multiagent systems: A survey from the machine learning perspective," Auton. Robots, vol. 8, no. 3, pp. 345-383, 2000.
5. R. H. Crites and A. G. Barto, "Elevator group control using multiple reinforcement learning agents," Mach. Learn., vol. 33, no. 2-3, pp. 235-262, 1998.
6. S. Sen and G. Weiss, "Learning in multiagent systems," in Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence, G. Weiss, Ed. Cambridge, MA: MIT Press, 1999, ch. 6, pp. 259-298.
7. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction Cambridge, MA: MIT Press, 1998.
8. L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement learning: A survey," J. Artif. Intell. Res., vol. 4, pp. 237-285, 1996.
9. V. Cherkassky and F. Mulier, Learning from Data. New York: Wiley, 1998.
10. T. J. Sejnowski and G. E. Hinton, Eds., Unsupervised Learning: Foundations of Neural Computation. Cambridge, MA: MIT Press, 1999.
11. T. Bassar and G. J. Olsder, Dynamic Noncooperative Game Theory, 2nd ed. SIAM Series in Classics in Applied Mathematics. London, U.K.: Academic, 1999.
12. D. Fudenberg and D. K. Levine, The Theory of Learning in Games. Cambridge, MA: MIT Press, 1998.
13. M. Bowling and M. Veloso, "Multiagent learning using a variable learning rate," Artif. Intell., vol. 136, no. 2, pp. 215-250, 2002.
14. Y. Shoham, R. Powers, and T. Grenager. (2003, May). "Multi-agent reinforcement learning: A critical survey," Comput. Sci. Dept., Stanford Univ., Stanford, CA, Tech. Rep. [Online]. Available: http:// multiagent.stanford.edu/papers/MALearning_ACriticalSurvey_2003_0516.pdf
15. T. Bäck, Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms. London, U.K.: Oxford Univ. Press, 1996.
16. K. D. Jong, Evolutionary Computation: A Unified Approach. Cambridge, MA: MIT Press, 2005.
17. L. Panait and S. Luke, "Cooperative multi-agent learning: The state of the art," Auton. Agents Multi-Agent Syst., vol. 11, no. 3, pp. 387-434, Nov. 2005.
18. M. A. Potter and K. A. D. Jong, "A cooperative coevolutionary approach to function optimization," in Proc. 3rd Conf. Parallel Probl. Solving Nat. (PPSN-III), Jerusalem, Israel, Oct. 9-14, 1994, pp. 249-257.
19. S. G. Ficici and J. B. Pollack, "A game-theoretic approach to the simple coevolutionary algorithm," in Proc. 6th Int. Conf. Parallel Probl. Solving Nat. (PPSN-VI), Paris, France, Sep. 18-20, 2000, pp. 467-476.
20. L. Panait, R. P. Wiegand, and S. Luke, "Improving coevolutionary search for optimal multiagent behaviors," in Proc. l8th Int. Joint Conf. Artif. Intell. (IJCAI-03), Acapulco, Mexico, Aug. 9-15, pp. 653-660.
21. T. Haynes, R. Wainwright, S. Sen, and D. Schoenefeld, "Strongly typed genetic programming in evolving cooperation strategies," in Proc. 6th Int. Conf. Genet. Algorithms (ICGA-95), Pittsburgh, PA, Jul. 15-19, pp. 271-278.
22. R. Salustowicz, M. Wiering, and J. Schmidhuber, "Learning team strategies: Soccer case studies," Mach. Learn., vol. 33, no. 2-3, pp. 263-282, 1998.
23. T. Miconi, "When evolving populations is better than coevolving individuals: The blind mice problem," in Proc. 18th Int. Joint Conf. Artif. Intell. (IJCAI-03), Acapulco, Mexico, Aug. 9-15, pp. 647-652.
24. V. Könönen, "Gradient based method for symmetric and asymmetric multiagent reinforcement learning," in Proc. 4th Int. Conf. Intell. Data Eng. Autom. Learn. (IDEAL-03), Hong Kong, China, Mar. 21-23, pp. 68-75.
25. F. Ho and M. Kamel, "Learning coordination strategies for cooperative multiagent systems," Mach. Learn., vol. 33, no. 2-3, pp. 155-177, 1998.
26. J. Schmidhuber, "A general method for incremental self-improvement and multi-agent learning," in Evolutionary Computation: Theory and Applications, X. Yao, Ed. Singapore: World Scientific, 1999, ch. 3, pp. 81-123.
27. J. M. Smith, Evolution and the Theory of Games. Cambridge, U.K.: Cambridge Univ. Press, 1982.
28. K. Tuyls and A. Nowé, "Evolutionary game theory and multi-agent re-inforcement learning," Knowl. Eng. Rev., vol. 20, no. 1, pp. 63-90, 2005.
29. D.P. Bertsekas, Dynamic Programming and Optimal Control, vol. 2, 2nd ed. Belmont, MA: Athena Scientific, 2001.
30. D. P. Bertsekas, Dynamic Programming and Optimal Control, vol. 1, 3rd ed. Belmont, MA: Athena Scientific, 2005.
31. M. L. Puterman, Markov Decision Processes - Discrete Stochastic Dynamic Programming. New York: Wiley, 1994.
32. C. J. C. H. Watkins and P. Dayan, "Q-learning," Mach. Learn., vol. 8, pp. 279-292, 1992.
33. J. Peng and R. J. Williams, "Incremental multi-step Q-learning," Mach. Learn., vol. 22, no. 1-3, pp. 283-290, 1996.
34. R. S. Sutton, "Learning to predict by the methods of temporal differences," Mach. Learn., vol. 3, pp. 9-44, 1988.
35. A. G. Barto, R. S. Sutton, and C. W. Anderson, "Neuronlike adaptive elements that can solve difficult learning control problems," IEEE Trans. Syst., Man, Cybern., vol. SMC-5, no. 5, pp. 843-846, Sep./Oct. 1983.
36. R. S. Sutton, "Integrated architectures for learning, planning, and reacting based on approximating dynamic programming," in Proc. 7th Int. Conf. Mach. Learn. (ICML-90), Austin, TX, Jun. 21-23, pp. 216-224.
37. A. W. Moore and C. G. Atkeson, "Prioritized sweeping: Reinforcement learning with less data and less time," Mach. Learn., vol. 13, pp. 103-130, 1993.
38. M. L. Littman, "Value-function reinforcement learning in Markov games," J. Cogn. Syst. Res., vol. 2, no. 1, pp. 55-66, 2001.
39. M. L. Littman, "Markov games as a framework for multi-agent reinforcement learning," in Proc. 11th Int. Conf. Mach. Learn. (ICML-94), New Brunswick, NJ, Jul. 10-13, pp. 157-163.
40. J. Hu and M. P. Wellman, "Multiagent reinforcement learning: Theoretical framework and an algorithm," in Proc. 15th Int. Conf. Mach. Learn. (ICML-98), Madison, WI, Jul. 24-27, pp. 242-250.
41. M. Lauer and M. Riedmiller, "An algorithm for distributed reinforcement learning in cooperative multi-agent systems," in Proc. 17th Int. Conf. Mach. Learn. (ICML-00), Stanford Univ., Stanford, CA, Jun. 29-Jul. 2, pp. 535-542.
42. A. Greenwald and K. Hall, "Correlated-Q learning," in Proc. 20th Int. Conf. Mach. Learn. (ICML-03), Washington, DC, Aug. 21-24, pp. 242-249.
43. T. Jaakkola, M. I. Jordan, and S. P. Singh, "On the convergence of stochastic iterative dynamic programming algorithms," Neural Comput. vol. 6, no. 6, pp. 1185-1201, 1994.
44. J. N. Tsitsiklis, "Asynchronous stochastic approximation and Q-learning," Mach. Learn., vol. 16, no. 1, pp. 185-202, 1994.
45. C. Guestrin, M. G. Lagoudakis, and R. Parr, "Coordinated reinforcement learning," in Proc. 19th Int. Conf. Mach. Learn. (ICML-02), Sydney, Australia, Jul. 8-12, pp. 227-234.
46. J. R. Kok, M. T. J. Spaan, and N. Vlassis, "Non-communicative multi-robot coordination in dynamic environment," Robot. Auton. Syst., vol. 50, no. 2-3, pp. 99-114, 2005.
47. J. R. Kok and N. Vlassis, "Sparse cooperative Q-1earning," in Proc. 21st Int. Conf. Mach. Learn. (ICML-04), Banff, AB, Canada, Jul. 4-8, pp. 481-488.
48. J. R. Kok and N. Vlassis, "Using the max-plus algorithm for multiagent decision making in coordination graphs," in Robot Soccer World Cup IX (RoboCup 2005). Lecture Notes in Computer Science, vol. 4020, Osaka, Japan, Jul. 13-19, 2005.
49. R. Fitch, B. Hengst, D. Sue, G. Calbert, and J. B. Scholz, "Structural abstraction experiments in reinforcement learning," in Proc. 18th Aust. Joint Conf. Artif. Intell. (AI-05 , Lecture Notes in Computer Science, vol. 3809, Sydney, Australia, Dec. 5-9, pp. 164-175.
50. L. Buşoniu, B. De Schutter, and R. Babuška, "Multiagent reinforcement learning with adaptive state focus," in Proc. 17th Belgian - Dutch Conf. Artif. Intell. (BNAIC-05), Brussels, Belgium, Oct. 17-18, pp. 35-42.
51. M. Tan, "Multi-agent reinforcement learning: Independent vs. cooperative agents," in Proc. 10th Int. Conf. Mach. Learn. (ICML-93) Amherst, OH, Jun. 27-29, pp. 330-337.
52. J. Clouse, "Learning from an automated training agent," presented at the Workshop Agents that Learn from Other Agents, 12th Int. Conf. Mach. Learn. (ICML-95), Tahoe City, CA, Jul. 9-12.
53. B. Price and C. Boutilier, "Accelerating reinforcement learning through implicit imitation," J. Artif. Intell. Res., vol. 19, pp. 569-629, 2003.
54. J. Hu and M. P. Wellman, "Nash Q-learning for general-sum stochastic games," J. Mach. Learn. Res., vol. 4, pp. 1039-1069, 2003.
55. R. Powers and Y. Shoham, "New criteria and a new algorithm for learning in multi-agent systems," in Proc. Adv. Neural Inf. Process. Syst. (NIPS-04), Vancouver, BC, Canada, Dec. 13-18, vol. 17, pp. 1089-1096.
56. M. Bowling and M. Veloso, "Rational and convergent learning in stochastic games," in Proc. 17th Int. Conf. Artif. Intell. (IJCAI-01), San Francisco, CA, Aug. 4-10, 2001, pp. 1021-1026.
57. M. Bowling, "Convergence and no-regret in multiagent learning," in Proc. Adv. Neural Inf. Process. Syst. (NIPS-04 , Vancouver, BC, Canada, Dec. 13-18, vol. 17, pp. 209-216.
58. G. Chalkiadakis. (2003, Mar.). Multiagent reinforcement learning: Stochastic games with multiple learning players, Dept. of Comput. Sci., Univ. Toronto. Toronto, ON, Canada, Tech. Rep. [Online]. Available: http://www.cs.toronto.edu/~gehalk/DepthReport/DepthReport.ps
59. M. Bowling and M. Veloso. (2000, Oct.). "An analysis of stochastic game theory for multiagent reinforcement learning," Dept. Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. [Online]. Available: http://www.cs.ualberta.ca/bowling/papers/00tr.pdf
60. V. Conitzer and T. Sandholm, "AWESOME: A general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents," in Proc. 20th Int. Conf. Mach. Learn. (ICML-03), Washington, DC, Aug. 21-24, pp. 83-90.
61. M. Bowling, "Multiagent learning in the presence of agents with limitations," Ph.D. dissertation, Dept. Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, May 2003.
62. C. Claus and C. Boutilier, "The dynamics of reinforcement learning in cooperative multiagent systems," in Proc. 15th Nat. Conf. Artif. Intell. 10th Conf. Innov. Appl. Artif. Intell. (AAAI/IAAI-98), Madison, WI, Jul. 26-30, pp. 746-752.
63. S. Kapetanakis and D. Kudenko, "Reinforcement learning of coordination in cooperative multi-agent systems," in Proc. 18th Nat. Conf. Artif. Intell. 14th Conf. Innov. Appl. Artif. Intell. (AAAI/IAAI-02), Menlo Park, CA, Jul. 28-Aug. 1, pp. 326-331.
64. X. Wang and T. Sandholm, "Reinforcement learning to play an optimal Nash equilibrium in team Markov games," in Proc. Adv. Neural Inf. Process. Syst. (NIPS-02), Vancouver, BC, Canada, Dec. 9-14, vol. 15, pp. 1571-1578.
65. G. W. Brown, "Iterative solutions of games by fictitious play," in Activitiy Analysis of Production and Allocation, T. C. Koopmans, Ed. New York: Wiley, 1951, ch. XXIV, pp. 374-376.
66. S. Singh, M. Kearns, and Y. Mansour, "Nash convergence of gradient dynamics in general-sum games," in Proc. 16th Conf. Uncertainty Artif. Intell. (UAI-00), San Francisco, CA, Jun. 30-Jul. 3, pp. 541-548.
67. M. Zinkevich, "Online convex programming and generalized infinitesimal gradient ascent," in Proc. 20th Int. Conf. Mach. Learn. (ICML-03) Washington, DC, Aug. 21 24, pp. 928-936.
68. G. Tesauro, "Extending Q-Iearning to general adaptive multi-agent systems," in Proc. Adv. Neural Inf. Process. Syst. (NIPS-03), Vancouver, BC, Canada, Dec. 8-13, vol. 16.
69. S. Sen, M. Sekaran, and J. Hale, "Learning to coordinate without sharing information," in Proc. 12th Nat. Conf. Artif. Intell. (AAAI-94), Seattle, WA, Jul. 31-Aug. 4, pp. 426-431.
70. M. J. Matarić, "Reinforcement learning in the multi-robot domain," Auton. Robots, vol. 4, no. 1, pp. 73-83, 1997.
71. R. H. Crites and A. G. Barto, "Improving elevator performance using reinforcement learning," in Proc. Adv. Neural Inf. Process. Syst. (NIPS-95), Denver, CO, Nov. 27-30, 1996, vol. 8, pp. 1017-1023.
72. V. Könönen, "Asymmetric multiagent reinforcement learning," in Proc. IEEE/WIC Int. Conf. Intell. Agent Technol. (IAT-03), Halifax, NS, Canada, Oct. 13-17, pp. 336-342.
73. M. Weinberg and J. S. Rosenschein, "Best-response multiagent learning in non-stationary environments," in Proc. 3rd Int. Joint Conf. Auton. Agents Multiagent Syst. (AAMAS-04), New York, NY, Aug. 19-23, pp. 506-513.
74. B. Banerjee and J. Peng, "Adaptive policy gradient in multiagent learning," in Proc. 2nd Int. Joint Conf. Auton. Agents Multiagent Syst. (AAMAS-03), Melbourne, Australia, Jul. 14-18, pp. 686-692.
75. N. Suematsu and A. Hayashi, "A multiagent reinforcement learning algorithm using extended optimal response," in Proc. 1st Int. Joint Conf. Auton. Agents Multiagent Syst. (AAMAS-02), Bologna, Italy, Jul. 15-19, pp. 370-377.
76. D. Carmel and S. Markovitch, "Opponent modeling in multi-agent systems," in Adaptation and Learning in Multi-Agent Systems, G. Weiss and S. Sen, Eds. New York: Springer-Verlag, 1996, ch. 3, pp. 40-52.
77. W. T. Uther and M. Veloso. (1997, Apr.). Adversarial reinforcement learning, School Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. [Onlilne]. Available: http://www.cs.cmu.edu/afs/cs/user/will/ www/papers/Uther97a.ps
78. D. V. Pynadath and M. Tambe, "The communicative multiagent team decision problem: Analyzing teamwork theories and models," J. Artif. Intell. Res., vol. 16, pp. 389-423, 2002.
79. M. T. J. Spaan, N. Vlassis, and F. C. A. Groen, "High level coordination of agents based on multiagent Markov decision processes with roles," in Proc. Workshop Coop. Robot., 2002 IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS-02), Lausanne, Switzerland, Oct. 1, pp. 66-73.
80. C. Boutilier, "Planning, learning and coordination in multiagent decision processes," in Proc. 6th Conf. Theor. Aspects Rationality Knowl. (TARK-96), De Zeeuwse Stromen, The Netherlands, Mar. 17-20, pp. 195-210.
81. F. Fischer, M. Rovatsos, and G. Weiss, "Hierarchical reinforcement learning in communication-mediated multiagent coordination," in Proc. 3rd Int. Joint Conf. Auton. Agents Multiagent Syst. (AAMAS-04 , New York, Aug. 19-23, pp. 1334-1335.
82. M. V. Nagendra Prasad, V. R. Lesser, and S. E. Lander, "Learning organizational roles for negotiated search in a multiagent system," Int. J. Hum. Comput. Stud., vol. 48, no. 1, pp. 51-67, 1998.
83. J. R. Kok, P. J. 't Hoen, B. Bakker, and N. Vlassis, "Utile coordination: Learning interdependencies among cooperative agents," in Proc. IEEE Symp. Comput. Intell. Games (CIG-05), Colchester, U.K., Apr. 4-6, pp. 29-36.
84. S. Nash and A. Sofer, Linear and Nonlinear Programming. New York: McGraw-Hill, 1996.
85. M. J. Matarić, "Reward functions for accelerated learning," in Proc. 11th Int. Conf. Mach. Learn. (ICML-94), New Brunswick, NJ, Jul. 10-13, pp. 181-189.
86. M. J. Matarić, "Learning in multi-robot systems," in Adaptation and Learning in Multi-Agent Systems, G. Weiss and S. Sen, Eds. New York: Springer-Verlag, 1996, ch. 10, pp. 152-163.
87. K. Tuyls, P. J. 't Hoen, and B. Vanschoenwinkel, "An evolutionary dynamical analysis of multi-agent learning in iterated games," Auton. Agents Multi-Agent Syst., vol. 12, no. 1, pp. 115-153, 2006.
88. M. Bowling, "Convergence problems of general-sum multiagent reinforcement learning," in Proc. 17th Int. Conf. Mach. Learn. (ICML-00), Stanford Univ., Stanford, CA, Jun. 29-Jul. 2, pp. 89-94.
89. M. L. Littman and P. Stone, "Implicit negotiation in repeated games," in Proc. 8th Int. Workshop Agent Theories Arch. Lang. (ATAL-2001), Seattle, WA, Aug. 21-24, pp. 96-105.
90. V. Stephan, K. Debes, H.-M. Gross, F. Wintrich, and H. Wintrich, "A reinforcement learning based neural multi-agent-system for control of a combustion process," in Proc. IEEE-INNS-ENNS Int. Joint Conf. Neural Netw. (IJCNN-00), Como, Italy, Jul. 24-27, pp. 6217-6222.
91. M. Wiering, "Multi-agent reinforcement learning for traffic light control," in Proc. 17th Int. Conf. Mach. Learn. (ICML-00), Stanford Univ., Stanford, CA, Jun. 29-Jul. 2, pp. 1151-1158.
92. B. Bakker, M. Steingrover, R. Schouten, E. Nijhuis, and L. Kester, "Cooperative multi-agent reinforcement learning of traffic lights," presented at the Workshop Coop. Multi-Agent Learn., 16th Eur. Conf. Mach. Learn. (ECML-05), Porto, Portugal, Oct. 3.
93. M. A. Riedmiller, A. W. Moore, and J. G. Schneider, "Reinforcement learning for cooperating and communicating reactive agents in electrical power grids," in Balancing Reactivity and Social Deliberation in Multi-Agent Systems, M. Hannebauer, J. Wendler, and E. Pagello, Eds. New York: Springer, 2000, pp. 137-149.
94. C. F. Touzet, "Robot awareness in cooperative mobile robot learning," Auton. Robots, vol. 8, no. 1, pp. 87-97, 2000.
95. F. Fernández and L. E. Parker, "Learning in large cooperative multirobot systems," Int. J. Robot. Autom., vol. 16, no. 4, pp. 217-226, 2001.
96. Y. Ishiwaka, T. Sato, and Y. Kakazu, "An approach to the pursuit problem on a heterogeneous multiagent system using reinforcement learning," Robot. Auton. Syst., vol. 43, no. 4, pp. 245-256, 2003.
97. P. Stone and M. Veloso, "Team-partitioned, opaque-transition reinforcement learning," in Proc. 3rd Int. Conf. Auton. Agents (Agents-99), Seattle, WA, May 1-5, pp. 206-212.
98. M. Wiering, R. Salustowicz, and J. Schmidhuber, "Reinforcement learning soccer teams with incomplete world models," Auton. Robots, vol. 7, no. 1, pp. 77-88, 1999.
99. K. Tuyls, S. Maes, and B. Manderick, "Q-Learning in simulated robotic soccer - large state spaces and incomplete information," in Proc. 2002 Int. Conf. Mach. Learn. Appl. (ICMLA-02), Las Vegas, NV, Jun. 24-27, pp. 226-232.
100. A. Merke and M. A. Riedmiller, "Karlsruhe brainstormers - A reinforcement learning approach to robotic soccer," in Robot Soccer World Cup V (RoboCup 2001). Lecture Notes in Computer Science, vol. 2377, Washington, DC, Aug. 2-10, pp. 435-440.
101. M. P. Wellman, A. R. Greenwald, P. Stone, and P. R. Wurman, "The 2001 trading agent competition," Electron. Markets, vol. 13, no. 1, pp. 4-12, 2003.
102. W.-T. Hsu and V.-W. Soo, "Market performance of adaptive trading agents in synchronous double auctions," in Proc. 4th Pacific Rim Int. Workshop Multi-Agents. Intell. Agents: Specification Model. Appl. (PRIMA-01). Lecture Notes in Computer Science Series, vol. 2132, Taipei, Taiwan, R.O.C., Jul. 28-29, pp. 108-121.
103. J. W. Lee and J. Oo, "A multi-agent Q-Learning framework for optimizing stock trading systems," in Proc. 13th Int. Conf. Database Expert Syst. Appl. (DEXA-02). Lecture Notes in Computer Science, vol. 2453, Aixen-Provence, France, Sep. 2-6, pp. 153-162.
104. J. Oo, J. W. Lee, and B.-T. Zhang, "Stock trading system using reinforcement learning with cooperative agents," in Proc. 19th Int. Conf. Mach. Learn. (ICML-02), Sydney, Australia, Jul. 8-12, pp. 451-458.
105. G. Tesauro and J. O. Kephart, "Pricing in agent economies using multiagent Q-Learning," Auton. Agents Multi-Agent Syst., vol. 5, no. 3, pp. 289-304, 2002.
106. C. Raju, Y. Narahari, and K. Ravikumar, "Reinforcement learning applications in dynamic pricing of retail markets," in Proc. 2003 IEEE Int. Conf. E-Commerce (CEC-03), Newport Beach, CA, Jun. 24-27, pp. 339-346.
107. A. Schaerf, Y. Shoham, and M. Tennenholtz, "Adaptive load balancing: A study in multi-agent learning," J. Artif. Intell. Res., vol. 2, pp. 475-500, 1995.
108. J. A. Boyan and M. L. Littman, "Packet routing in dynamically changing networks: A reinforcement learning approach," in Proc. Adv. Neural Inf. Process. Syst. (NIPS-93), Denver, CO, Nov. 29-Dec. 2, vol. 6, pp. 671-678.
109. S. P. M. Choi and D.-Y. Yeung, "Predictive Q-routing: A memory-based reinforcement learning approach to adaptive traffic control," in Proc. Adv. Neural Inf. Process. Syst. (NIPS-95), Denver, CO, Nov. 27-30, vol. 8, pp. 945-951.
110. P. Tillotson, Q. Wu, and P. Hughes, "Multi-agent learning for routing control within an Internet environment," Eng. Appl. Artif. Intell. vol. 17, no. 2, pp. 179-185, 2004.
111. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
112. G. Gordon, "Stable function approximation in dynamic programming," in Proc. 12th Int. Conf. Mach. Learn. (ICML-95 , Tahoe City, CA, Jul. 9-12, pp. 261-268.
113. J. N. Tsitsiklis and B. Van Roy, "Feature-based methods for large scale dynamic programming," Mach. Learn., vol. 22, no. 1-3, pp. 59-94, 1996.
114. R. Munos and A. Moore, "Variable-resolution discretization in optimal control," Mach. Learn., vol. 49, no. 2-3, pp. 291-323, 2002.
115. p-norm for approximate value iteration," SIAM J. Control Optim., vol. 46, no. 2. pp. 546-561, 2007.
116. C. Szepesvári and R. Munos, "Finite time bounds for sampling based fitted value iteration," in Proc. 22nd Int. Conf. Mach. Learn. (ICML-05), Bonn, Germany, Aug. 7-11, pp. 880-887.
117. J. N. Tsitsiklis and B. Van Roy, "An analysis of temporal difference learning with function approximation," IEEE Trans. Autom. Control vol. 42, no. 5, pp. 674-690, May 1997.
118. D. Ormoneit and S. Sen, "Kernel-based reinforcement learning," Mach. Learn., vol. 49, no. 2-3, pp. 161-178, 2002.
119. C. Szepesvári and W. D. Smart, "Interpolation-based Q-learning," in Proc. 21st Int. Conf. Mach. Learn. (ICML-04 , Banff, AB, Canada, Jul. 4-8.
120. D. Ernst, P. Geurts, and L. Wehenkel, "Tree-based batch mode reinforcement learning," J. Mach. Learn. Res., vol. 6, pp. 503-556, 2005.
121. M. G. Lagoudakis and R. Parr, "Least-squares policy iteration," J. Mach. Learn. Res., vol. 4, pp. 1107-1149, 2003.
122. S. Dzěroski, L. D. Raedt, and K. Driessens, "Relational reinforcement learning," Mach. Learn., vol. 43, no. 1-2, pp. 7-52, 2001.
123. O. Abul, F. Polar, and R. Alhajj, "Multiagent reinforcement learning using function approximation," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 4, no. 4, pp. 485-497, Nov. 2000.
124. L. Buşoniu, B. De Schutter, and R. Babuška, "Decentralized reinforcement learning control of a robotic manipulator," in Proc. 9th Int. Conf. Control Autom. Robot. Vis. (ICARCV-06), Singapore, Dec. 5-8, pp. 1347-1352.
125. H. Tamakoshi and S. Ishii, "Multiagent reinforcement learning applied to a chase problem in a continuous world," Artif. Life Robot., vol. 5, no. 4, pp. 202-206, 2001.
126. B. Price and C. Boutilier, "Implicit imitation in multiagent reinforcement learning," in Proc. 16th Int. Conf. Mach. Learn. (ICML-99), Bled, Slovenia, Jun. 27-30, pp. 325-334.
127. O. Buffet, A. Dutech, and F. Charpillet, "Shaping multi-agent systems with gradient reinforcement learning," Auton. Agents Multi-Agent Syst., vol. 15, no. 2, pp. 197-220, 2007.
128. M. Ghavamzadeh, S. Mahadevan, and R. Makar, "Hierarchical multiagent reinforcement learning," Auton. Agents Multi-Agent Syst., vol. 13, no. 2, pp. 197-229, 2006.
129. W. S. Lovejoy, "Computationally feasible bounds for partially observed Markov decision processes," Oper. Res., vol. 39, no. 1, pp. 162-175, 1991.
130. S. Ishii, H. Fujita, M. Mitsmake, T. Yamazaki, J. Matsuda, and Y. Matsuno, "A reinforcement learning scheme for a partially-observable multi-agent game," Mach. Learn., vol. 59, no. 1-2, pp. 31-54, 2005.
131. E. A. Hansen, D. S. Bernstein, and S. Zilberstein, "Dynamic programming for partially observable stochastic games," in Proc. 19th Natl. Conf. Artif. Intell. (AAAI-04), San Jose, CA, Jul. 25-29, pp. 709-715.
132. J. M. Vidal, "Learning in multiagent systems: An introduction from a game-theoretic perspective," in Adaptive Agents. Lecture Notes in Artificial Intelligence, vol. 2636, E. Alonso, Ed. New York: Springer-Verlag, 2003, pp. 202-215.