New inference strategies for solving Markov Decision Processes using reversible jump MCMC

Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI 2009

Volumn , Issue , 2009, Pages 223-231

  Hoffman, Matt ^a   Kueck, Hendrik ^a   De Freitas, Nando ^a   Doucet, Arnaud ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; MONTE CARLO METHODS;

HIGHER-DIMENSIONAL; MARKOV CHAIN MONTE CARLO METHOD; MARKOV DECISION PROCESSES; OPTIMAL POLICIES; PARAMETERIZED CONTROL; REVERSIBLE JUMP MCMC; STRONG CORRELATION;

MARKOV CHAINS;

EID: 78751705157     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. H. Attias. Planning by probabilistic inference. In UAI, 2003.
2. Y. Cheng. Mean shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(8):790-799, 1995.
3. P. Dayan and G. Hinton. Using EM for reinforcement learning. Neural Computation, 9:271-278, 1997.
4. A. Doucet and V. Tadic. On solving integral equations using Markov Chain Monte Carlo methods. Technical Report CUED-F-INFENG 444, Cambridge University Engineering Department, 2004.
5. A. Doucet, S. Godsill, and C. Robert. Marginal maximum a posteriori estimation using Markov chain Monte Carlo. Statistics and Computing, 12(1):77-84, 2002. (Pubitemid 37132839)
6. P. Green. Reversible jump Markov Chain Monte Carlo computation and Bayesian model determination. Biometrika, 82(4):711-732, 1995.
7. M. Hoffman, A. Doucet, N. de Freitas, and A. Jasra. Bayesian policy learning with trans-dimensional MCMC. In NIPS, 2007a.
8. M. Hoffman, A. Doucet, N. de Freitas, and A. Jasra. On solving general state-space sequential decision problems using inference algorithms. Technical Report TR-2007-04, University of British Columbia, Computer Science, 2007b.
9. M. Hoffman, N. de Freitas, A. Doucet, and J. Peters. An expectation maximization algorithm for continuous Markov Decision Processes with arbitrary reward. In AI-STATS, 2009.
10. J. Kober and J. Peters. Policy search for motor primitives in robotics. In NIPS, 2008. P. Müller. Simulation based optimal design. In Bayesian Statistics 6, 1998.
11. P.Müller, B. Sansó, and M. de Iorio. Optimal Bayesian design by inhomogeneous Markov chain simulation. Journal of the American Statistical Association, 99: 788-798, 2004. (Pubitemid 39332860)
12. A. Ng and M. Jordan. PEGASUS: A policy search method for large MDPs and POMDPs. In UAI, pages 406-415, 2000.
13. O. Papaspiliopoulos, G. Roberts, and M. Sköld. Noncentered parameterisations for hierarchical models and data augmentation. Bayesian Statistics, 7, 2003.
14. J. Peters and S. Schaal. Reinforcement learning for operational space control. In ICRA, 2007.
15. J. Spall. Introduction to stochastic search and optimization: estimation, simulation, and control. Wiley-Interscience, 2005.
16. M. Toussaint and A. Storkey. Probabilistic inference for solving discrete and continuous state Markov Decision Processes. In ICML, 2006.
17. M. Toussaint, S. Harmeling, and A. Storkey. Probabilistic inference for solving (PO)MDPs. Technical Report EDI-INF-RR-0934, University of Edinburgh, School of Informatics, 2006.
18. M. Toussaint, L. Charlin, and P. Poupart. Hierarchical POMDP controller optimization by likelihood maximization. In UAI, pages 562-570, 2008.
19. D. Verma and R. Rao. Planning and acting in uncertain environments using probabilistic inference. In IROS, 2006.
20. S. Vijayakumar, M. Toussaint, G. Petkos, and M. Howard. Planning and moving in dynamic environments: A statistical machine learning approach. In Sendhoff, Koerner, Sporns, Ritter, and Doya, editors, Creating Brain Like Intelligence: From Principles to Complex Intelligent Systems, LNAI-Vol. 5436. Springer-Verlag, 2009.