Deterministic MDPs with adversarial rewards and bandit feedback

Uncertainty in Artificial Intelligence - Proceedings of the 28th Conference, UAI 2012

Volumn , Issue , 2012, Pages 93-101

  Arora, Raman ^a   Dekel, Ofer ^b   Tewari, Ambuj ^c  

^a TTI C   (United States)

^b MICROSOFT RESEARCH   (United States)

^c UNIVERSITY OF MICHIGAN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BANDIT FEEDBACKS; DECISION MAKERS; MARKOV DECISION PROCESSES; ON-LINE DECISION MAKINGS; STATE TRANSITION DYNAMICS; TRANSITION DYNAMICS;

ARTIFICIAL INTELLIGENCE; DECISION MAKING; MARKOV PROCESSES;

FEEDBACK;

EID: 84886067084     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. J. Abernethy, E. Hazan, and A. Rakhlin. Competing in the dark: An efficient algorithm for bandit linear optimization. In Proceedings of the 21st Annual Conference on Learning Theory, pages 263-274, 2008.
2. R. Arora, O. Dekel, and A. Tewari. Online bandit learning against an adaptive adversary: from regret to policy regret. In Proceedings of the 29th International Conference on Machine Learning, 2012.
3. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. Schapire. The nonstochastic multiarmed bandit problem. SIAM Journal on Computing, 32(1):48-77, 2002.
4. B. Awerbuch and R. D. Kleinberg. Adaptive routing with end-to-end feedback: Distributed learning and geometric approaches. In Proceedings of the 36th Annual ACM Symposium on the Theory of Computing, pages 45-53, 2004.
5. M. S. Bazaraa, J. J. Jarvis, and H. D. Sherali. Linear Programming and Network Flows, Third Edition. John Wiley and Sons, 2010.
6. D. P. Bertsekas. Dynamic Programming and Optimal Control, volume 1. Athena Scientific, Third edition, 2005.
7. C. Caratheodory. Uber den variabilitatsbereich der fourierschen konstanten von positiven harmonischen funktionen. Rendiconti del Circolo Matematico di Palermo, 32:193-217, 1911.
8. D. P. de Farias and N. Megiddo. Combining expert advice in reactive environments. Journal of the ACM, 53(5):762-799, 2006.
9. E. Even-Dar, S. M. Kakade, and Y. Mansour. Online Markov decision processes. Mathematics of Operations Research, 34(3):726-736, 2009.
10. V. F. Farias, C. C. Moallemi, B. Van Roy, and T. Weissman. Universal reinforcement learning. IEEE Transactions on Information Theory, 56(5): 2441-2454, 2010.
11. E. A. Feinberg and F. Yang. On polynomial cases of the unichain classification problem for Markov decision processes. Operations Research Letters, 36(5): 527-530, 2008.
12. A. D. Flaxman, A. Tauman Kalai, and H. B. McMahan. Online convex optimization in the bandit setting: gradient descent without a gradient. In Proceedings of the 16th annual ACM-SIAM symposium on discrete algorithms, pages 385-394, 2005.
13. A. Gyorgy, T. Linder, G. Lugosi, and G. Ottucsak. The on-line shortest path problem under partial monitoring. Journal of Machine Learning Research, 8:2369-2403, 2007.
14. O. Maillard and R. Munos. Adaptive bandits: Towards the best history-dependent strategy. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15 of JMLR W&CP, pages 570-578, 2011.
15. H. B. McMahan and A. Blum. Online geometric optimization in the bandit setting against an adaptive adversary. In Proceedings of the 17th Annual Conference on Learning Theory, pages 109-123, 2004.
16. G. Neu, A. Gyorgy, C. Szepesvari, and A. Antos. Online Markov decision processes under bandit feedback. In Advances in Neural Information Processing Systems 23, pages 1804-1812. MIT Press, 2010.
17. R. Ortner. Online regret bounds for Markov decision processes with deterministic transitions. Theoretical Computer Science, 411(29-30):2684-2695, 2010.
18. D. Ryabko and M. Hutter. On the possibility of learning in reactive environments with arbitrary dependence. Theoretical Computer Science, 405(3):274-284, 2008.
19. C. Szepesvari. Algorithms for reinforcement learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 4(1), 2010.
20. J. Y. Yu and S. Mannor. Arbitrarily modulated markov decision processes. In Proceedings of the IEEE Conference on Decision and Control, 2009.
21. J. Y. Yu, S. Mannor, and N. Shimkin. Markov decision processes with arbitrary reward processes. Mathematics of Operations Research, 34(3):737-757, 2009.