An expectation maximization algorithm for continuous Markov decision processes with arbitrary rewards

Journal of Machine Learning Research

Volumn 5, Issue , 2009, Pages 232-239

  Hoffman, Matt ^a   De Freitas, Nando ^a   Doucet, Arnaud ^a   Peters, Jan ^b  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^b MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ANALYTICAL TRACTABILITY; APPROXIMATION ERRORS; CLOSED FORM SOLUTIONS; EXPECTATION-MAXIMIZATION ALGORITHMS; GAUSSIANS; LINEAR QUADRATIC GAUSSIAN CONTROLLERS; MARKOV DECISION PROCESSES; MIXTURE OF GAUSSIANS; NUMERICAL OPTIMIZATIONS; OPTIMIZATION METHOD; PARAMETERIZED; POLICY OPTIMIZATION; REWARD FUNCTION;

ARTIFICIAL INTELLIGENCE; MARKOV PROCESSES; OPTIMIZATION;

SIGNAL RECONSTRUCTION;

EID: 84862277035     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (21)

1. H. Attias. Planning by probabilistic inference. In UAI, 2003.
2. J. Baxter and P. Bartlett. Infinite-horizon policy-gradient estimation. JAIR, 15:319-350, 2001.
3. D. Bertsekas. Dynamic Programming and Optimal Control. Athena Scientific, 1995.
4. R. Byrd, P. Lu, J. Nocedal, and C. Zhu. A Limited Memory Algorithm for Bound Constrained Optimization. SIAM Journal on Scientific Computing, 1995.
5. P. Dayan and G. Hinton. Using EM for reinforcement learning. Neural Computation, 9:271-278, 1997.
6. A. Doucet and V. Tadic. On solving integral equations using Markov Chain Monte Carlo methods. Technical Report CUED-F-INFENG 444, Cambridge University Engineering Department, 2004.
7. M. Hoffman, A. Doucet, N. de Freitas, and A. Jasra. Bayesian policy learning with trans-dimensional MCMC. In NIPS, 2008.
8. M. Klaas, M. Briers, N. de Freitas, A. Doucet, and S. Maskell. Fast particle smoothing: If I had a million particles. In ICML, 2006.
9. K. Lange. A quasi-Newton acceleration of the EM algorithm. Statistica Sinica, 5(1):1-18, 1995.
10. F. Lewis, D. Dawson, and C. Abdallah. Robot Manipulator Control: Theory and Practice. CRC Press, 2004.
11. J. Maciejowski. Predictive control with constraints. PrenticeHall, 2002.
12. A. Ng and M. Jordan. PEGASUS: A policy search method for large MDPs and POMDPs. In UAI, 2000.
13. J. Peters and S. Schaal. Natural Actor-Critic. Neurocomputing, 71(7-9):1180-1190, 2008.
14. J. Peters and S. Schaal. Reinforcement learning for operational space control. In ICRA, 2007.
15. M. Porta, N. Vlassis, M. Spaan, and P. Poupart. Pointbased value iteration for continuous POMDPs. JMLR, 7:2329-2367, 2006.
16. R. Smallwood and E. Sondik. The optimal control of partially observable Markov processes over a finite horizon. Operations Research, 1973.
17. E. Snelson and Z. Ghahramani. Sparse Gaussian processes using pseudo-inputs. In NIPS, 2006.
18. S. Thrun. Monte Carlo POMDPs. In NIPS, 2000.
19. M. Toussaint and A. Storkey. Probabilistic inference for solving discrete and continuous state Markov Decision Processes. In ICML, 2006.
20. M. Toussaint, S. Harmeling, and A. Storkey. Probabilistic inference for solving (PO)MDPs. Technical Report EDI-INF-RR-0934, University of Edinburgh, School of Informatics, 2006.
21. D. Verma and R. P. N. Rao. Planning and acting in uncertain environments using probabilistic inference. In IROS, 2006.