Arbitrarily modulated Markov decision processes

Proceedings of the IEEE Conference on Decision and Control

Volumn , Issue , 2009, Pages 2946-2953

  Yu, Jia Yuan ^a   Mannor, Shie ^a,b  

^a MCGILL UNIVERSITY   (Canada)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

BEHAVIORAL RESEARCH; COMPUTATIONAL EFFICIENCY; DECISION MAKING; MARKOV PROCESSES; REINFORCEMENT LEARNING;

ARBITRARY CHANGE; DECISION-MAKING PROBLEM; MARKOV DECISION PROCESSES; NONSTATIONARY; PRIOR KNOWLEDGE; Q-LEARNING; TRANSITION PROBABILITIES;

LEARNING ALGORITHMS;

EID: 77950787050     PISSN: 07431546     EISSN: 25762370     Source Type: Conference Proceeding    
DOI: 10.1109/CDC.2009.5400662     Document Type: Conference Paper

References (24)

1. L. Shapley, "Stochastic games," PNAS, vol.39, no.10, pp. 1095-1100, 1953.
2. E. Even-Dar, S. Kakade, and Y. Mansour, "Experts in a Markov decision process," in NIPS, 2004, pp. 401-408.
3. S. Mannor and N. Shimkin, "The empirical Bayes envelope and regret minimization in competitive Markov decision processes," Mathematics of Operations Research, vol.28, no.2, pp. 327-345, 2003.
4. N. Cesa-Bianchi and G. Lugosi, Prediction, Learning, and Games. Cambridge University Press, 2006.
5. S. Özekici, "Markov modulated Bernoulli process," Mathematical Methods of Operations Research, vol.45, no.3, pp. 311-324, 1997.
6. A. Nilim and L. E. Ghaoui, "Robust control of Markov decision processes with uncertain transition matrices," Operations Research, vol.53, no.5, pp. 780-798, 2005.
7. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire, "The nonstochastic multiarmed bandit problem," SIAM J. Comput., vol.32, no.1, pp. 48-77, 2002.
8. R. I. Brafman and M. Tennenholtz, "R-max - a general polynomial time algorithm for near-optimal reinforcement learning," Journal of Machine Learning Research, vol.3, pp. 213-231, 2003.
9. J. Y. Yu, S. Mannor, and N. Shimkin, "Markov decision processes with arbitrary rewards," Math. Oper. Res., 2009, to appear.
10. J. Y. Yu and S. Mannor, "Online learning in Markov decision processes with arbitrarily changing rewards and transitions," in GameNets, 2009.
11. M. L. Puterman, Markov Decision Processes. Wiley, 1994.
12. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Athena Scientific, 1996.
13. D. L. Donoho and P. B. Stark, "Uncertainty principles and signal recovery," SIAM J. Appl. Math., vol.49, no.3, pp. 906-931, 1989.
14. C. D. Fuh, "Asymptotic operating characteristics of an optimal change point detection in hidden Markov models," Ann. Statist., pp. 2305-2339, 2004.
15. J. Filar and K. Vrieze, Competitive Markov Decision Processes. Springer-Verlag, 1996.
16. H. Xu and S. Mannor, "The robustness-performance tradeoff in Markov decision processes," in NIPS, 2006, pp. 1537-1544.
17. J. Hannan, "Approximation to Bayes risk in repeated play," in Contributions to the Theory of Games. Princeton University Press, 1957, vol.3, pp. 97-139.
18. N. Littlestone and M. Warmuth, "The weighted majority algorithm," Information and Computation, vol.108, no.2, pp. 212-261, 1994.
19. D. P. Bertsekas, Dynamic Programming and Optimal Control, 2nd ed. Athena Scientific, 2001, vol.2.
20. C. Watkins and P. Dayan, "Q-learning," Machine Learning, vol.8, pp. 279-292, 1992.
21. A. Kalai and S. Vempala, "Efficient algorithms for online decision problems," Journal of Computer and System Sciences, vol.71, no.3, pp. 291-307, 2005.
22. H. J. Kushner and G. G. Yin, Stochastic Approximation Algorithms and Applications. Springer-Verlag, 1997.
23. V. S. Borkar and S. P. Meyn, "The O.D.E. method for convergence of stochastic approximation and reinforcement learning," SIAM J. Control Optim., vol.38, no.2, pp. 447-469, 2000.
24. P. J. Schweitzer, "Perturbation theory and finite Markov chains," Journal of Applied Probability, vol.5, pp. 401-413, 1968.