Provably efficient learning with typed parametric models

Journal of Machine Learning Research

Volumn 10, Issue , 2009, Pages 1955-1988

  Brunskill, Emma ^a   Leffler, Bethany R ^a   Li, Hong ^a   Littman, Michael L ^b   Roy, Nicholas ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b RUTGERS UNIVERSITY   (United States)

Author keywords

Provably efficient learning; Reinforcement learning

Indexed keywords

EFFICIENT LEARNING; MARKOV DECISION PROCESSES; PARAMETRIC MODELS; PROBABLY APPROXIMATELY CORRECT; PROVABLY EFFICIENT LEARNING; REAL WORLD DOMAIN; REAL-WORLD; ROBOT NAVIGATION; SAMPLE COMPLEXITY; SAMPLE COMPLEXITY BOUNDS; SMALL ROBOTS; STATE-SPACE; TRAJECTORY DATA;

EDUCATION; MARKOV PROCESSES; REINFORCEMENT; REINFORCEMENT LEARNING;

LEARNING ALGORITHMS;

EID: 70349416596     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (34)

1. Pieter Abbeel and Andrew Y. Ng. Exploration and apprenticeship learning in reinforcement learning. In Proceedings of the 22nd International Conference on Machine Learning (ICML), pages 1-8, 2005.
2. Justin Boyan and Andrew Moore. Generalization in reinforcement learning: Safely approximating the value function. In Advances in Neural Information Processing Systems (NIPS) 7, pages 369-376, 1995.
3. Ronen I. Brafman and Moshe Tennenholtz. R-MAX - a general polynomial time algorithm for near-optimal reinforcement learning. Journal of Machine Learning Research, 3:213-231, 2002.
4. Emma Brunskill, Bethany R. Leffler, Lihong Li, Michael L. Littman, and Nicholas Roy. CORL: A continuous-state offset-dynamics reinforcement learner. In Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence (UAI), pages 53-61, 2008.
5. Jeffrey B. Burl. Linear Optimal Control. Prentice Hall, 1998. ISBN 9780201808681.
6. Pablo Castro and Doina Precup. Using linear programming for Bayesian exploration in Markov decision processes. In Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI), pages 2437-2442, 2007.
7. Chee-Seng Chow and John N. Tsitsiklis. The complexity of dynamic programming. Journal of Complexity, 5(4):466-488, 1989.
8. Chee-Seng Chow and John N. Tsitsiklis. An optimal one-way multigrid algorithm for discrete-time stochastic control. IEEE Transactions on Automatic Control, 36(8):898-914, 1991. (Pubitemid 21674882)
9. Finale Doshi, Joelle Pineau, and Nicholas Roy. Reinforcement learning with limited reinforcement: Using Bayes risk for active learning in POMDPs. In Proceedings of the 25th International Conference on Machine Learning (ICML), pages 256-263, 2008.
10. Alain Dutech, Timothy Edmunds, Jelle Kok, Michail Lagoudakis, Michael L. Littman, Martin Riedmiller, Bryan Russell, Bruno Scherrer, Richard Sutton, Stephan Timmer, Nikos Vlassis, Adam White, and Shimon Whiteson. Reinforcement learning benchmarks and bake-offs II. In Advances in Neural Information Processing Systems (NIPS) 17 Workshop, 2005.
11. Jakob Eriksson, Hari Balakrishnan, and Samuel Madden. Cabernet: A WiFi-Based Vehicular Content Delivery Network. In Proceedings of the 14th Conference on Mobile Computing and Networking (MOBICOM), pages 199-210, 2008.
12. Gene H. Golub and Charles F. Van Loan. Matrix Computations. The Johns Hopkins University Press, 3rd edition, 1996. ISBN 0-801-85414-8.
13. Roger A. Horn and Charles R. Johnson. Matrix Analysis. Cambridge University Press, 1986. ISBN 0-521-38632-2.
14. Eric Horvitz, Johnson Apacible, Raman Sarin, and Lin Liao. Prediction, expectation, and surprise: Methods, designs, and study of a deployed traffic forecasting service. In Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (UAI), pages 275-283, 2005.
15. Nicholas K. Jong and P. Stone. Model-based exploration in continuous state spaces. In Proceedings of the 7th Symposium on Abstraction, Reformulation, and Approximation (SARA), pages 258-272, 2007.
16. Sham Kakade. On the sample complexity of reinforcement learning. PhD thesis, University College London, 2003.
17. Michael J. Kearns and Satinder P Singh. Near-optimal reinforcement learning in polynomial time. Machine Learning, 49(2-3):209-232, 2002.
18. Levente Kocsis and Csaba Szepesvári. Bandit based Monte-Carlo planning. In Proceedings of the 17th European Conference on Machine Learning (ECML), pages 282-293, 2006.
19. Solomon Kullback. A lower bound for discrimination in terms of variation. IEEE Transactions on Information Theory, 13(1): 126-127, January 1967.
20. Branislav Kveton and Milos Hauskrecht. Solving factored MDPs with exponential-family transition models. In Proceedings of the 16th International Conference on Automated Planning and Scheduling (ICAPS), pages 114-120, 2006.
21. M.G. Lagoudakis and R. Parr. Least-squares policy iteration. Journal of Machine Learning Research, 4:1107-1149, 2003.
22. Bethany R. Leffler, Michael L. Littman, and Timothy Edmunds. Efficient reinforcement learning with relocatable action models. In Proceedings of the 22nd Conference on Artificial Intelligence (AAAI), pages 572-577, 2007.
23. Lihong Li. A Unifying Framework for Computational Reinforcement Learning Theory. PhD thesis, Rutgers University, New Brunswick, NJ, 2009.
24. Lihong Li, Michael L. Littman, and Thomas J. Walsh. Knows what it knows: A framework for self-aware learning. In Proceedings of the 25th International Conference on Machine Learning (ICML), pages 568-575, 2008.
25. Janusz Marecki and Milind Tambe. Towards faster planning with continuous resources in stochastic domains. In Proceedings of the 23rd Conference on Artificial Intelligence (AAAI), pages 1049-1055, 2008.
26. Andrew Ng, H.Jin Kim, Michael Jordan, and Shankar Sastry. Autonomous helicopter flight via reinforcement learning. In Advances in Neural Information Processing Systems (NIPS) 16, pages 799-806, 2004.
27. Pascal Poupart, Nikos Vlassis, Jesse Hoey, and Kevin Regan. An analytic solution to discrete Bayesian reinforcement learning. In Proceedings of the 23rd International Conference on Machine Learning (ICML), pages 697-704, 2006.
28. Stephane Ross, Brahim Chaib-draa, and Joelle Pineau. Bayesian reinforcement learning in continuous POMDPs with application to robot navigation. In Proceedings of the International Conference on Robotics and Automation (ICRA), pages 2845-2851, 2008.
29. Alexander L. Střehl and Michael L. Littman. Online linear regression and its application to modelbased reinforcement learning. In Advances in Neural Information Processing Systems (NIPS) 20, pages 1417-1424, 2008.
30. Alexander L. Strehl, Lihong Li, and Michael L. Littman. Incremental model-based learners with formal learning-time guarantees. In Proceedings of the 22nd Conference on Uncertainty in Artificial Intelligence (UAI), pages 485-492, 2006.
31. Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 1998.
32. Gerald J. Tesauro. TD-Gammon, a self-teaching backgammon program, achieves master-level play. Neural Computation, 6(2):215-219, 1994.
33. John von Neumann. Some matrix-inequalities and metrization of matrix-space. Tomsk University Review, 1:286-300, 1937.
34. Christopher J.C.H. Watkins. Learning from delayed rewards. PhD thesis, King's College, University of Cambridge, United Kingdom, 1989.