Model-free reinforcement learning as mixture learning

Proceedings of the 26th International Conference On Machine Learning, ICML 2009

Volumn , Issue , 2009, Pages 1081-1088

  Vlassis, Nikos ^a   Toussaint, Marc ^b  

^a TECHNICAL UNIVERSITY OF CRETE   (Greece)

^b TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

CAST MODELS; DESIGN AND ANALYSIS OF ALGORITHMS; EM ALGORITHMS; FINITE HORIZONS; MODEL FREE; NEW TOOLS; POLICY ITERATION ALGORITHMS; PROBABILISTIC MIXTURE MODELS; STOCHASTIC APPROXIMATIONS;

APPROXIMATION ALGORITHMS; APPROXIMATION THEORY; EDUCATION; LIGHT MEASUREMENT; REINFORCEMENT; REINFORCEMENT LEARNING; ROBOT LEARNING; STOCHASTIC SYSTEMS;

ITERATIVE METHODS;

EID: 71249143630     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Abbeel, P., Coates, A., Quigley, M., & Y., N. A. (2007). An application of reinforcement learning to aerobatic helicopter flight. In B. Schölkopf, J. Platt and T. Hoffman (Eds.), Advances in neural information processing systems 19, 1-8. Cambridge, MA: MIT Press.
2. Bertsekas, D. P., & Tsitsiklis, J. N. (1996). Neuro-dynamic programming. Athena Scientific.
3. Celeux, G., & Diebolt, J. (1985). The SEM algorithm: a probabilistic teacher algorithm derived from the EM algorithm for the mixture problem. Comp. Statis. Quaterly, 2, 73-82.
4. Cooper, G. F. (1988). A method for using belief networks as influence diagrams. Proc. 4th Workshop on Uncertainty in Artificial Intelligence (pp. 55-63). Minneapolis, Minnesota, USA.
5. Dayan, P., & Hinton, G. E. (1997). Using Expectation-Maximization for reinforcement learning. Neural Computation, 9, 271-278.
6. Dearden, R., Friedman, N., & Russell, S. (1998). Bayesian Q-learning. Proc. 15th National Conf. on Artificial Intelligence (pp. 761-768). Madison, Wisconsin, USA.
7. Delyon, B., Lavielle, M., & Moulines, E. (1999). Convergence of a stochastic approximation version of the EM algorithm. The Annals of Statistics, 27, 94-128.
8. Gordon, G. (1996). Chattering in Sarsa(λ) (Technical Report). CMU Learning Lab internal report.
9. Hansen, E. (1998). Solving POMDPs by searching in policy space. Proc. 14th Int. Conf. on Uncertainty in Artificial Intelligence (pp. 211-219). Madison, Wisconsin, USA.
10. Hoffman, M., Doucet, A., De Freitas, N., & Jasra, A. (2008). Bayesian policy learning with trans-dimensional MCMC. In J. Platt, D. Koller, Y. Singer and S. Roweis (Eds.), Advances in neural information processing systems 20, 665-672. Cambridge, MA: MIT Press.
11. Jaakkola, T., Singh, S. P., & Jordan, M. I. (1995). Reinforcement learning algorithm for partially observable Markov decision problems. In Advances in neural information processing systems 7, 345-352. MIT Press.
12. Kober, J., & Peters, J. (2009). Policy search for motor primitives in robotics. In D. Koller, D. Schuurmans, Y. Bengio and L. Bottou (Eds.), Advances in neural information processing systems 21, 849-856.
13. Littman, M. L., Cassandra, A. R., & Kaelbling, L. P. (1995). Learning policies for partially observable environments: Scaling up. Proc. 12th Int. Conf on Machine Learning (pp. 362-370).
14. Loch, J., & Singh, S. P. (1998). Using eligibility traces to find the best memoryless policy in partially observable Markov decision processes. Proc. 15th Int. Conf on Machine Learning (pp. 323-331). Madison, Wisconsin, USA.
15. Melo, F. S., Meyn, S. P., & Ribeiro, M. I. (2008). An analysis of reinforcement learning with function approximation. Proc. 25th Int. Conf. on Machine Learning (pp. 664-671). Helsinki, Finland.
16. Neal, R. M., & Hinton, G. E. (1998). A view of the EM algorithm that justifies incremental, sparse, and other variants. In M. I. Jordan (Ed.), Learning in graphical models, 355-368. Kluwer Academic Publishers.
17. Perkins, T. J., & Pendrith, M. D. (2002). On the existence of fixed points for Q-learning and Sarsa in partially observable domains. Proc. 19th Int. Conf. on Machine Learning (pp. 490-497).
18. Perkins, T. J., & Precup, D. (2003). A convergent form of approximate policy iteration. In S. T. S. Becker and K. Obermayer (Eds.), Advances in neural information processing systems 15, 1595-1602. Cambridge, MA: MIT Press.
19. Peters, J., & Schaal, S. (2008). Learning to control in operational space. International Journal of Robotics Research, 27, 197-212.
20. Poupart, P. (2005). Exploiting structure to efficiently solve large scale partially observable Markov decision processes. Doctoral dissertation, Dept. of Computer Science, University of Toronto.
21. Shani, G., Brafman, R. I., & Shimony, S. E. (2007). Forward search value iteration for POMDPs. In Int. Joint Conf. on Artificial Intelligence (pp. 2619-2624).
22. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
23. Toussaint, M., & Storkey, A. (2006). Probabilistic inference for solving discrete and continuous state Markov decision processes. Proc. 23rd Int. Conf. on Machine Learning (pp. 945-952). Pittsburgh, Pennsylvania, USA.
24. Tsitsiklis, J. N. (2002). On the convergence of optimistic policy iteration. Journal of Machine Learning Research, 3, 59-72.
25. Wei, G., & Tanner, M. (1990). A Monte Carlo implementation of the EM algorithm and the poor man's data augmentation algorithm. J. Amer. Statist. As-socation, 85, 699-704.