Online learning in Markov decision processes with arbitrarily changing rewards and transitions

Proceedings of the 2009 International Conference on Game Theory for Networks, GameNets '09

Volumn , Issue , 2009, Pages 314-322

  Yu, Jia An ^a   Mannor, Shie ^a,b  

^a MCGILL UNIVERSITY   (Canada)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

DECISION MAKERS; DECISION-MAKING PROBLEM; MARKOV DECISION PROCESSES; NONSTATIONARY; ONLINE LEARNING; TRANSITION PROBABILITIES;

COMPUTATIONAL COMPLEXITY; E-LEARNING; GAME THEORY; INTERNET; LEARNING ALGORITHMS; MARKOV PROCESSES; PROBABILITY; ROBUST CONTROL;

DECISION MAKING;

EID: 70349986740     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/GAMENETS.2009.5137416     Document Type: Conference Paper

References (21)

1. M. L. Puterman, Markov Decision Processes. Wiley, 1994.
2. L. Shapley, "Stochastic games," PNAS, vol. 39, no. 10, pp. 1095-1100, 1953.
3. E. Even-Dar, S. Kakade, and Y. Mansour, "Experts in a Markov decision process," in NIPS, 2004, pp. 401-408.
4. S. Mannor and N. Shimkin, "The empirical Bayes envelope and regret minimization in competitive Markov decision processes," Mathematics of Operations Research, vol. 28, no. 2, pp. 327-345, 2003.
5. N. Cesa-Bianchi and G. Lugosi, Prediction, learning, and games. Cambridge University Press, 2006.
6. A. Nilim and L. E. Ghaoui, "Robust control of Markov decision processes with uncertain transition matrices," Operations Research, vol. 53, no. 5, pp. 780-798, 2005.
7. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire, "The non-stochastic multiarmed bandit problem," SIAM J. Computing, vol. 32, no. 1, pp. 48-77, 2002.
8. R. I. Brafman and M. Tennenholtz, "R-max - a general polynomial time algorithm for near-optimal reinforcement learning," Journal of Machine Learning Research, vol. 3, pp. 213-231, 2003.
9. J. Y. Yu, S. Mannor, and N. Shimkin, "Markov decision processes with arbitrary reward processes," in Lecture Notes in Computer Science, vol. 5323, 2009, http://www.cim.mcgill.ca/~jiayuan/mdp.pdf.
10. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Athena Scientific, 1996.
11. J. Hannan, "Approximation to Bayes risk in repeated play," in Contributions to the Theory of Games. Princeton University Press, 1957, vol. 3, pp. 97-139.
12. E. Altman, "Applications of dynamic games in queues," Advances in Dynamic Games, vol. 7, pp. 309-342, 2005.
13. A. Lapidoth and P. Narayan, "Reliable communication under channel uncertainty," IEEE Trans. Inf. Theory, vol. 44, no. 6, pp. 2148-2177, 1998.
14. N. Littlestone and M. Warmuth, "The weighted majority algorithm," Information and Computation, vol. 108, no. 2, pp. 212-261, 1994.
15. H. Xu and S. Mannor, "The robustness-performance tradeoff in Markov decision processes," in NIPS, 2006, pp. 1537-1544.
16. J. Filar and K. Vrieze, Competitive Markov Decision Processes. Springer-Verlag, 1996.
17. A. Kalai and S. Vempala, "Efficient algorithms for online decision problems," Journal of Computer and System Sciences, vol. 71, no. 3, pp. 291-307, 2005.
18. D. P. Bertsekas, Dynamic Programming and Optimal Control, 2nd ed. Athena Scientific, 2001, vol. 2.
19. P. J. Schweitzer, "Perturbation theory and finite Markov chains," Journal of Applied Probability, vol. 5, pp. 401-413, 1968.
20. E. Even-Dar, S. Kakade, and Y. Mansour., "On-line Markov decision processes," preprint.
21. C. G. Atkeson, A. W. Moore, and S. Schaal, "Locally weighted learning," Artificial Intelligence Review, 1997.