An investigation into mathematical programming for finite horizon decentralized POMDPs

Journal of Artificial Intelligence Research

Volumn 37, Issue , 2010, Pages 329-396

  Aras, Raghav ^a   Dutech, Alain ^b  

^a SUPELEC   (France)

^b LORIA   (France)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATE SOLUTION; CLASSICAL TESTS; COMPLEX TASK; DECENTRALIZED PLANNING; DECISION-THEORETIC; EXPERIMENTAL VALIDATIONS; FINITE HORIZONS; MIXED-INTEGER LINEAR PROGRAMMING; MODELING TOOL; NOVEL ALGORITHM; OPTIMAL SOLUTIONS; PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; UNCERTAIN ENVIRONMENTS;

ALGORITHMS; GAME THEORY; INTEGER PROGRAMMING; LINEARIZATION; MARKOV PROCESSES; OPTIMAL SYSTEMS; OPTIMIZATION;

DYNAMIC PROGRAMMING;

EID: 77952736651     PISSN: None     EISSN: 10769757     Source Type: Journal    
DOI: 10.1613/jair.2915     Document Type: Article

References (57)

1. Amato, C., Bernstein, D. S., & Zilberstein, S. (2007a). Optimizing memory-bounded controllers for decentralized POMDPs. In Proc. of the Twenty-Third Conf. on Uncertainty in Artificial Intelligence (UAI-07).
2. Amato, C., Bernstein, D. S., & Zilberstein, S. (2007b). Solving POMDPs using quadratically constrained linear programs. In Proc. of the Twentieth Int. Joint Conf. on Artificial Intelligence (IJCAI'07).
3. Amato, C., Carlin, A., & Zilberstein, S. (2007c). Bounded dynamic programming for decentralized POMDPs. In Proc. of the Workshop on Multi-Agent Sequential Decision Making in Uncertain Domains (MSDM) in AAMAS'07.
4. Amato, C., Dibangoye, J., & Zilberstein, S. (2009). Incremental policy generation for finitehorizon DEC-POMDPs. In Proc. of the Nineteenth Int. Conf. on Automated Planning and Scheduling (ICAPS-09).
5. Anderson, B., & Moore, J. (1980). Time-varying feedback laws for decentralized control. Nineteenth IEEE Conference on Decision and Control including the Symposium on Adaptive Processes, 19(1), 519-524.
6. Becker, R., Zilberstein, S., Lesser, V., & Goldman, C. (2004). Solving transition independent decentralized Markov decision processes. Journal of Artificial Intelligence Research, 22, 423-455.
7. Bellman, R. (1957). Dynamic programming. Princeton University Press, Princeton, New- Jersey.
8. Bernstein, D., Givan, R., Immerman, N., & Zilberstein, S. (2002). The complexity of decentralized control of Markov decision processes. Mathematics of Operations Research, 27(4), 819-840.
9. Bernstein, D. S., Hansen, E. A., & Zilberstein, S. (2005). Bounded policy iteration for decentralized POMDPs. In Proc. of the Nineteenth Int. Joint Conf. on Artificial Intelligence (IJCAI), pp. 1287-1292.
10. Boularias, A., & Chaib-draa, B. (2008). Exact dynamic programming for decentralized pomdps with lossless policy compression. In Proc. of the Int. Conf. on Automated Planning and Scheduling (ICAPS'08).
11. Boutilier, C. (1996). Planning, learning and coordination in multiagent decision processes. In Proceedings of the 6th Conference on Theoretical Aspects of Rationality and Knowledge (TARK '96), De Zeeuwse Stromen, Nederlands.
12. Buffet, O., Dutech, A., & Charpillet, F. (2007). Shaping multi-agent systems with gradient reinforcement learning. Autonomous Agent and Multi-Agent System Journal (AAMASJ), 15(2), 197-220.
13. Cassandra, A., Kaelbling, L., & Littman, M. (1994). Acting optimally in partially observable stochastic domains. In Proc. of the 12th Nat. Conf. on Artificial Intelligence (AAAI).
14. Chadès, I., Scherrer, B., & Charpillet, F. (2002). A heuristic approach for solving decentralized-POMDP: assessment on the pursuit problem. In Proc. of the 2002 ACM Symposium on Applied Computing, pp. 57-62.
15. Cornuéjols, G. (2008). Valid inequalities for mixed integer linear programs. Mathematical Programming B, 112, 3-44.
16. Dantzig, G. B. (1960). On the significance of solving linear programming problems with some integer variables. Econometrica, 28(1), 30-44.
17. d'Epenoux, F. (1963). A probabilistic production and inventory problem. Management Science, 10(1), 98-108.
18. Diwekar, U. (2008). Introduction to Applied Optimization (2 edition). Springer.
19. Drenick, R. (1992). Multilinear programming: Duality theories. Journal of Optimization Theory and Applications, 72(3), 459-486.
20. Fletcher, R. (1987). Practical Methods of Optimization. John Wiley & Sons, New York.
21. Ghavamzadeh, M., & Mahadevan, S. (2004). Learning to communicate and act in cooperative multiagent systems using hierarchical reinforcement learning. In Proc. of the 3rd Int. Joint Conf. on Autonomous Agents and Multi-Agent Systems (AAMAS'04).
22. Govindan, S., & Wilson, R. (2001). A global newton method to compute Nash equilibria. Journal of Economic Theory, 110, 65-86.
23. Hansen, E., Bernstein, D., & Zilberstein, S. (2004). Dynamic programming for partially observable stochastic games. In Proc. of the Nineteenth National Conference on Artificial Intelligence (AAAI-04).
24. Horst, R., & Tuy, H. (2003). Global Optimization: Deterministic Approaches (3rd edition). Springer.
25. James, M., & Singh, S. (2004). Learning and discovery of predictive state representations in dynamical systems with reset. In Proc. of the Twenty-first Int. Conf. of Machine Learning (ICML'04).
26. Kaelbling, L., Littman, M., & Cassandra, A. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101, 99-134.
27. Koller, D., Megiddo, N., & von Stengel, B. (1994). Fast algorithms for finding randomized strategies in game trees. In Proceedings of the 26th ACM Symposium on Theory of Computing (STOC '94), pp. 750-759.
28. Koller, D., & Megiddo, N. (1996). Finding mixed strategies with small supports in extensive form games. International Journal of Game Theory, 25(1), 73-92.
29. Lemke, C. (1965). Bimatrix Equilibrium Points and Mathematical Programming. Management Science, 11(7), 681-689.
30. Luenberger, D. (1984). Linear and Nonlinear Programming. Addison-Wesley Publishing Company, Reading, Massachussetts.
31. McCracken, P., & Bowling, M. H. (2005). Online discovery and learning of predictive state representations. In Advances in Neural Information Processing Systems 18 (NIPS'05).
32. Nair, R., Tambe, M., Yokoo, M., Pynadath, D., & Marsella, S. (2003). Taming decentralized POMDPs: towards efficient policy computation for multiagent setting. In Proc. of Int. Joint Conference on Artificial Intelligence, IJCAI'03.
33. Oliehoek, F., Spaan, M., & Vlassis, N. (2008). Optimal and approximate Q-value functions for decentralized POMDPs. Journal of Artificial Intelligence Research (JAIR), 32, 289-353.
34. Oliehoek, F., Whiteson, S., & Spaan, M. (2009). Lossless clustering of histories in decentralized POMDPs. In Proc. of The International Joint Conference on Autonomous Agents and Multi Agent Systems, pp. 577-584.
35. Osborne, M. J., & Rubinstein, A. (1994). A Course in Game Theory. The MIT Press, Cambridge, Mass.
36. Papadimitriou, C. H., & Steiglitz, K. (1982). Combinatorial Optimization: Algorithms and Complexity. Dover Publications.
37. Papadimitriou, C. H., & Tsitsiklis, J. (1987). The Complexity Of Markov Decision Processes. Mathematics of Operations Research, 12 (3), 441-450.
38. Parsons, S., & Wooldridge, M. (2002). Game theory and decision theory in multi-agent systems. Autonomous Agents and Multi-Agent Systems (JAAMAS), 5(3), 243-254.
39. Petrik, M., & Zilberstein, S. (2007). Average-reward decentralized Markov decision processes. In Proc. of the Twentieth Int. Joint Conf. on Artificial Intelligence (IJCAI 2007).
40. Petrik, M., & Zilberstein, S. (2009). A bilinear programming approach for multiagent planning. Journal of Artificial Intelligence Research (JAIR), 35, 235-274.
41. Puterman, M. (1994). Markov Decision Processes: discrete stochastic dynamic programming. John Wiley & Sons, Inc. New York, NY.
42. Pynadath, D., & Tambe, M. (2002). The Communicative Multiagent Team Decision Problem: Analyzing Teamwork Theories And Models. Journal of Artificial Intelligence Research, 16, 389-423. (Pubitemid 43057178)
43. Radner, R. (1959). The application of linear programming to team decision problems. Management Science, 5, 143-150.
44. Russell, S., & Norvig, P. (1995). Artificial Intelligence: A modern approach. Prentice Hall.
45. Sandholm, T. (1999). Multiagent systems, chap. Distributed rational decision making, pp. 201-258. The MIT Press. Ed. by G. Weiss.
46. Sandholm, T., Gilpin, A., & Conitzer, V. (2005). Mixed-integer programming methods for finding nash equilibria. In Proc. of the National Conference on Artificial Intelligence (AAAI).
47. Scherrer, B., & Charpillet, F. (2002). Cooperative co-learning: A model based approach for solving multi agent reinforcement problems. In Proc. of the IEEE Int. Conf. on Tools with Artificial Intelligence (ICTAI'02).
48. Seuken, S., & Zilberstein, S. (2007). Memory-bounded dynamic programming for DECPOMDPs. In Proc. of the Twentieth Int. Joint Conf. on Artificial Intelligence (IJCAI' 07).
49. Singh, S., Jaakkola, T., & Jordan, M. (1994). Learning without state estimation in partially observable markovian decision processes.. In Proceedings of the Eleventh International Conference on Machine Learning.
50. Singh, S., Littman, M., Jong, N., Pardoe, D., & Stone, P. (2003). Learning predictive state representations. In Proc. of the Twentieth Int. Conf. of Machine Learning (ICML'03).
51. Szer, D., & Charpillet, F. (2006). Point-based Dynamic Programming for DEC-POMDPs. In Proc. of the Twenty-First National Conf. on Artificial Intelligence (AAAI 2006).
52. Szer, D., Charpillet, F., & Zilberstein, S. (2005). MAA*: A heuristic search algorithm for solving decentralized POMDPs. In Proc. of the Twenty-First Conf. on Uncertainty in Artificial Intelligence (UAI'05), pp. 576- 583.
53. Thomas, V., Bourjot, C., & Chevrier, V. (2004). Interac-DEC-MDP : Towards the use of interactions in DEC-MDP. In Proc. of the Third Int. Joint Conf. on Autonomous Agents and Multi-Agent Systems (AAMAS'04), New York, USA, pp. 1450-1451.
54. Vanderbei, R. J. (2008). Linear Programming: Foundations and Extensions (3rd edition). Springer.
55. von Stengel, B. (2002). Handbook of Game Theory, Vol.3, chap. 45-"Computing equilibria for two-person games", pp. 1723-1759. North-Holland, Amsterdam.
56. Wu, J., & Durfee, E. H. (2006). Mixed-integer linear programming for transitionindependent decentralized MDPs. In Proc. of the fifth Int. Joint Conf. on Autonomous Agents and Multiagent Systems (AAMAS'06), pp. 1058-1060 New York, NY, USA. ACM.
57. Xuan, P., Lesser, V., & Zilberstein, S. (2000). Communication in multi-agent Markov decision processes. In Proc. of ICMAS Workshop on Game Theoretic and Decision Theoretics Agents Boston, MA.