Using linear programming for bayesian exploration in markov decision processes

IJCAI International Joint Conference on Artificial Intelligence

Volumn , Issue , 2007, Pages 2437-2442

  Castro, Pablo Samuel ^a   Precup, Doina ^a  

^a MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

BANDIT PROBLEMS; EXPLORATION ALGORITHMS; EXPLORATION METHODS; MARKOV DECISION PROCESSES; OPTIMIZATION PROBLEMS; ORIGINAL SYSTEMS; SPARSE SAMPLING; VALUE FUNCTIONS;

ARTIFICIAL INTELLIGENCE; LEARNING ALGORITHMS; LINEAR PROGRAMMING; OPTIMIZATION; REINFORCEMENT LEARNING;

MARKOV PROCESSES;

EID: 70349431917     PISSN: 10450823     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. R. Bellman and R. Kalaba. On adaptive control processes. In IRE Trans., volume 4, pages 1-9, 1959.
2. R. I. Brafman andM. Tennenholtz. R-MAX - a general polynomial time algorithm for near-optimal reinforcement learning. In IJCAI, pages 953-958, 2001.
3. D.P. de Farias and B. Van Roy. The linear programming approach to approximate dynamic programming. Operations Research, 51(6):850-865, 2003.
4. D.P. de Farias and B. Van Roy. On constraint sampling for the linear programming approach to approximate dynamic programming. Mathematics of Operations Research, 29(3):462-478, 2004.
5. R. Dearden, N. Friedman, and D. Andre. Model-based bayesian exploration. In Proc. 15th UAI Conference, pages 150-159, 1999.
6. M. O. Duff. Optimal Learning: Computational procedures for Bayes-adaptiveMarkov decision processes. PhD thesis, University of Massachussets, 2002.
7. M. O. Duff. Design for an optimal probe. In International Conference on Machine Learning, 2003.
8. J.C. Gittins and D. Jones. Bandit processes and dynamic allocation indices. Journal of the Royal Statistical Society, Series B, 41(2):148-177, 1979.
9. M. Hauskrecht and B. Kveton. Linear program approximations for factored continuous-state markov decision processes. In Advances in Neural Information Processing Systems 17, 2004.
10. M. Kearns and S. Singh. Nearoptimal reinforcement learning in polynomial time. In Proc. 15th ICML, pages 260-268, 1998.
11. M. Kearns, Y. Mansour, and A. Y. Ng. A sparse sampling algorithm for near-optimal planning in large markov decision processes. In IJCAI, pages 1324-1231, 1999.
12. J.J. Martin. Bayesian Decision Problems and Markov Chains. John Wiley & Sons, New York, 1967.
13. N. Meuleau and P. Bourgine. Exploration of multi-state environments: Local measures and back-propagation of uncertainty. Machine Learning, 35(2):117-154, 1999.
14. M. L. Puterman. Markov Decision Processes. JohnWily & Sons, New York, 1994.
15. P. Schweitzer and A. Seidmann. Generalized polynomial approximations in markovian decision processes. J. Math. Anal. Appl., 110:568-582, 1985.
16. A. L. Strehl and M. L. Littman. A theoretical analysis of model-based interval estimation. In Proc. 21st ICML, 2005.
17. A. L. Strehl, L. Li, and M. L. Littman. Incremental model-based learners with formal learningtime guarantees. In Proc. 21st UAI Conference, 2006.
18. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, Massachusetts, 1998.
19. S. Thrun. Efficient exploration in reinforcement learning. Technical Report CMU-CS-92-102, Carnegie Mellon University, School of Computer Science, 1992.
20. T. Wang, D. Lizotte, M. Bowling, and D. Schuurmans. Bayesian sparse sampling for on-line reward optimization. In Proceedings of Twenty-First International Conference on Machine Learning, 2005.