Mean field for Markov decision processes: From discrete to continuous optimization

IEEE Transactions on Automatic Control

Volumn 57, Issue 9, 2012, Pages 2266-2280

  Gast, Nicolas ^a   Gaujal, Bruno ^b   Le Boudec, Jean Yves ^a  

^a EPFL   (Switzerland)

^b INRIA RHÔNE ALPES   (France)

Author keywords

Epidemic model; Hamilton Jacobi Bellman (HJB); Markov decision processes; mean field; optimal control

Indexed keywords

EPIDEMIC MODELS; HAMILTON-JACOBI-BELLMAN; MARKOV DECISION PROCESSES; MEAN FIELD; OPTIMAL CONTROLS;

DYNAMIC PROGRAMMING; MARKOV PROCESSES; OPTIMIZATION; ORDINARY DIFFERENTIAL EQUATIONS;

LEARNING ALGORITHMS;

EID: 84865675087     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2012.2186176     Document Type: Article

References (25)

1. M. Benaïm,"Dynamics of stochastic approximation algorithms," Séminaire de Probabilités XXXIII. Lecture Notes in Math, vol. 1709, pp. 1-68, 1999.
2. M. Benaim and J.-Y. L. Boudec,"A class of mean field interaction models for computer and communication systems," Perform. Eval., vol. 65, no. 11-12, pp. 823-838, 2008.
3. M. Benaim and J. Weibull,"Deterministic Approximation of Stochastic Evolution in Games: A Generalization," Tech. Rep., mimeo, 2003.
4. A. Benveniste, P. Priouret, and M. Métivier, Adaptive Algorithms and Stochastic Approximations. New York: Springer-Verlag, 1990.
5. V. G. BertenB,"Grid brokering for batch allocation using indexes," in Network Control and Optimization, volume 4465 of LNCS. New York: Springer, 2007.
6. W. Chen, D. Huang, A. A. Kulkarni, J. Unnikrishnan, Q. Zhu, P. Mehta, S. Meyn, and A. Wierman,"Approximate dynamic programming using fluid and diffusion approximations with applications to power management," in Proc. 48th IEEE Conf. Decision Control. CDC/CCC, 2009, pp. 3575-3580.
7. R. Cole,"Initial studies on worm propagation in manets for future army combat systems," Tech. Rep., Pentagon Reports, 2004.
8. D. P. De Farias and B. Van Roy,"The linear programming approach to approximate dynamic programming," Operat. Res., vol. 51, no. 6, pp. 850-865, 2003.
9. N. Gast and B. Gaujal,"Mean field limit of non-smooth systems and differential inclusions," ACM SIGMETRICS Perform. Eval. Rev., vol. 38, no. 2, pp. 30-32, 2010.
10. N. Gast and B. Gaujal,"A mean field approach for optimization in discrete time," Discrete Event Dynam. Syst., vol. 21, pp. 63-101, 2011.
11. Z. Gorodeisky,"Deterministic approximation of best-response dynamics for the matching pennies game," Games Econ. Behav., vol. 66, no. 1, pp. 191-201, 2009.
12. M. Huang, P. E. Caines, and R. P. Malhame,"Large population stochastic dynamic games: Closed-loop Mckean-Vlasov systems and the Nash certainty equivalence principle," Com. Inf. Syst., vol. 6, pp. 221-252, 2006.
13. M. Huang, P. E. Caines, and R. P. Malhame,"Nash certainty equivalence in large population stochastic dynamic games: Connections with the physics of interacting particle systems," in Proc. 45th IEEE Conf. Decision Control, San Diego, 2006, pp. 4921-4926. (Pubitemid 351283806)
14. M. Huang, P. E. Caines, and R. P. Malhame,"Large-population costcoupled lqg problems with nonuniform agents: individual-mass behavior and decentralized -Nash equilibria," IEEE Trans. Autom. Control, vol. 52, no. 9, pp. 1560-1571, Sep. 2007. (Pubitemid 47456068)
15. M. H. R. Khouzani, S. Sarkar, and E. Altman,"Maximum damage malware attack in mobile wireless networks," in Proc. IEEE Infocom, San Diego, CA, 2010, pp. 1-9.
16. D. Kondo, B. Javadi, P. Malecot, F. Cappello, and D. Erson, "Costbenefit analysis of cloud computing versus desktop grids," in Proc. 18th Int. Heterogeneity in Comput. Workshop, Rome, Italy, 2009.
17. T. Kurtz,"Solutions of ordinary differential equations as limits of pure jump Markov processes," J. Appl. Probab., vol. 7, pp. 49-58, 1970.
18. J.-M. Lasry and P.-L. Lions,"Mean field games," Jpn. J. Math., 2007.
19. J. Y. Le Boudec, D. McDonald, and J. Mundinger,"A generic mean field convergence result for systems of interacting objects," Proc. QEST '07, pp. 3-18, 2007.
20. C. H. Papadimitriou and J. N. Tsitsiklis,"The complexity of optimal queuing network control," Math. Oper. Res., vol. 24, no. 2, pp. 292-305, 1999.
21. M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York: Wiley, 1994.
22. S. Tanachaiwiwat and A. Helmy,"Vaccine:War of the worms in wired and wireless networks," in Proc. IEEE INFOCOM, 2006.
23. H. Tembine, J.-Y. Le Boudec, R. El-Azouzi, and E. Altman,"Mean field asymptotic of Markov decision evolutionary games and teams," Gamenets, 2009.
24. H. Tembine, P. Vilanova, and M. Debbah,"Noisy mean field stochastic games with network applications," 2010, Tech. Rep. 342-P-11867-83, Supelec.
25. J. N. Tsitsiklis and B. V. Roy,"An analysis of temporal-difference learning with function approximation," IEEE Trans. Autom. Control, vol. 42, no. 5, pp. 674-690, May 1997. (Pubitemid 127760263)