Model-based reinforcement learning in factored-state MDPs

Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007

Volumn , Issue , 2007, Pages 103-110

  Strehl, Alexander L ^a  

^a RUTGERS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA STRUCTURES; LEARNING ALGORITHMS; OPTIMAL CONTROL SYSTEMS; POLYNOMIAL APPROXIMATION; PROBLEM SOLVING;

FACTORED RMAX; INTERVAL ESTIMATION;

REINFORCEMENT LEARNING;

EID: 34548763246     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ADPRL.2007.368176     Document Type: Conference Paper

References (21)

1. M. J. Kearns and S. P. Singh, "Near-optimal reinforcement learning in polynomial time," Machine Learning, vol. 49, no. 2-3, pp. 209-232, 2002.
2. R. I. Brafman and M. Tennenholtz, "R-MAX - a general polynomial time algorithm for near-optimal reinforcement learning," Journal of Machine Learning Research, vol. 3, pp. 213-231, 2002.
3. S. M. Kakade, "On the sample complexity of reinforcement learning," Ph.D. dissertation, Gatsby Computational Neuroscience Unit, University College London, 2003.
4. L. P. Kaelbling, Learning in Embedded Systems. Cambridge, MA: The MIT Press, 1993.
5. M. Wiering and J. Schmidhuber, "Efficient model-based exploration," in Proceedings of the Fifth International Conference on Simulation of Adaptive Behavior (SAB'98), 1998, pp. 223-228.
6. A. L. Strehl and M. L. Littman, "A theoretical analysis of model-based interval estimation," in Proceedings of the Twenty-second International Conference on Machine Learning (ICML-05), 2005, pp. 857-864.
7. _, "An empirical evaluation of interval estimation for Markov decision processes," in The 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI-2004), 2004, pp. 128-135.
8. _, "An analysis of model-based interval estimation for Markov decision processes," Journal of Computer and System Sciences, in press.
9. D. Koller and R. Parr, "Computing factored value functions for policies in structured MDPs," in Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence. The AAAI Press/The MIT Press, 1999, pp. 1332-1339.
10. C. Boutilier, T. Dean, and S. Hanks, "Decision-theoretic planning: Structural assumptions and computational leverage," Journal of Artificial Intelligence Research, vol. 11, pp. 1-94, 1999.
11. M. J. Kearns and D. Koller, "Efficient reinforcement learning in factored MDPs," in Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI), 1999, pp. 740-747.
12. C. Guestrin, R. Patrascu, and D. Schuurmans, "Algorithm-directed exploration for model-based reinforcement learning in factored MDPs," in Proceedings of the International Conference on Machine Learning, 2002, pp. 235-242.
13. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. The MIT Press, 1998.
14. C. Boutilier, N. Friedman, M. Goldszmidt, and D. Koller, "Context-specific independence in Bayesian networks," in Proceedings of the Twelfth Annual Conference on Uncertainty in Artificial Intelligence (UAI 96), Portland, OR, 1996, pp. 115-123.
15. L. G. Valiant, "A theory of the learnable," Communications of the ACM, vol. 27, no. 11, pp. 1134-1142, November 1984.
16. A. L. Strehl, L. Li, and M. L. Littman, "Incremental model-based learners with formal learning-time guarantees," in UAI-06: Proceedings of the 22nd conference on Uncertainty in Artificial Intelligence, 2006, pp. 485-493.
17. T. Weissman, E. Ordentlich, G. Seroussi, S. Verdu, and M. J. Weinberger, "Inequalities for the L1 deviation of the empirical distribution," HewlettPackard Labs, Tech. Rep. HPL-2003-97R1, 2003.
18. M. L. Littman, "Probabilistic propositional planning: Representations and complexity," in Proceedings of the Fourteenth National Conference on Artificial Intelligence. AAAI Press/The MIT Press, 1997, pp. 748-754. [Online], Available: http://www.cs.rutgers.edu/ mlittman/papers/aaai97-planning.ps
19. M. L. Littman, J. Goldsmith, and M. Mundhenk, "The computational complexity of probabilistic planning," Journal of Artificial Intelligence Research, vol. 9, pp. 1-36, 1998.
20. E. Allender, S. Arora, M. Kearns, C. Moore, and A. Russell, "A note on the representational incompatabilty of function approximation and factored dynamics." in Advances in Neural Information Processing Systems (NIPS-03), 2002.
21. T. Degris, O. Sigaud, and P.-H. Wuillemin, "Learning the structure of factored Markov decision processes in reinforcement learning problems," in ICML-06: Proceedings of the 23rd international conference on Machine learning, 2006, pp. 257-264.