Embed to control: A locally linear latent dynamics model for control from raw images

Advances in Neural Information Processing Systems

Volumn 2015-January, Issue , 2015, Pages 2746-2754

  Watter, Manuel ^a   Springenberg, Jost Tobias ^a   Boedecker, Joschka ^a   Riedmiller, Martin ^b  

^a UNIVERSITY OF FREIBURG   (Germany)

^b DEEPMIND   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

DYNAMICAL SYSTEMS; LINEAR CONTROL SYSTEMS;

COMPLEX CONTROL PROBLEMS; DYNAMICS MODELING; GENERATIVE MODEL; IMAGE SEQUENCE; IMAGE TRAJECTORY; LONG-TERM PREDICTION; MODEL LEARNING; OPTIMAL CONTROLS;

INFORMATION SCIENCE;

EID: 84965129327     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. D. Jacobson and D. Mayne. Differential dynamic programming. American Elsevier, 1970.
2. E. Todorov and W. Li. A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems. In ACC. IEEE, 2005.
3. Y. Tassa, T. Erez, and W. D. Smart. Receding horizon differential dynamic programming. In Proc. of NIPS, 2008.
4. Y. Pan and E. Theodorou. Probabilistic differential dynamic programming. In Proc. of NIPS, 2014.
5. S. Levine and V. Koltun. Variational policy search via trajectory optimization. In Proc. of NIPS, 2013.
6. D. P. Kingma and M. Welling. Auto-encoding variational bayes. In Proc. of ICLR, 2014.
7. D. J. Rezende, S. Mohamed, and D. Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In Proc. of ICML, 2014.
8. M. D. Zeiler, D. Krishnan, G. W. Taylor, and R. Fergus. Deconvolutional networks. In CVPR, 2010.
9. A. Dosovitskiy, J. T. Springenberg, and T. Brox. Learning to generate chairs with convolutional neural networks. In Proc. of CVPR, 2015.
10. R. F. Stengel. Optimal Control and Estimation. Dover Publications, 1994.
11. W. Li and E. Todorov. Iterative Linear Quadratic Regulator Design for Nonlinear Biological Movement Systems. In Proc. of ICINCO, 2004.
12. M. Toussaint. Robot Trajectory Optimization using Approximate Inference. In Proc. of ICML, 2009.
13. M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. An introduction to variational methods for graphical models. In Machine Learning, 1999.
14. D. Kingma and J. Ba. Adam: A method for stochastic optimization. In Proc. of ICLR, 2015.
15. H. Wang, K. Tanaka, and M. Griffin. An approach to fuzzy control of nonlinear systems; stability and design issues. IEEE Trans. on Fuzzy Systems, 4(1), 1996.
16. R. S. Sutton and A. G. Barto. Introduction to Reinforcement Learning. MIT Press, Cambridge, MA, USA, 1st edition, 1998. ISBN 0262193981.
17. W. Böhmer, J. T. Springenberg, J. Boedecker, M. Riedmiller, and K. Obermayer. Autonomous learning of state representations for control. KI-Künstliche Intelligenz, 2015.
18. S. Lange and M. Riedmiller. Deep auto-encoder neural networks in reinforcement learning. In Proc. of IJCNN, 2010.
19. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis. Human-level control through deep reinforcement learning. Nature, 518(7540), 02 2015.
20. H. van Hoof, J. Peters, and G. Neumann. Learning of non-parametric control policies with highdimensional state features. In Proc. of AISTATS, 2015.
21. S. Levine, C. Finn, T. Darrell, and P. Abbeel. End-to-end training of deep visuomotor policies. CoRR, abs/1504.00702, 2015. URL http://arxiv.org/abs/1504.00702.
22. N. Wahlström, T. B. Schön, and M. P. Deisenroth. From pixels to torques: Policy learning with deep dynamical models. CoRR, abs/1502.02251, 2015. URL http://arxiv.org/abs/1502.02251.
23. K. Gregor, I. Danihelka, A. Graves, D. Rezende, and D. Wierstra. DRAW: A recurrent neural network for image generation. In Proc. of ICML, 2015.
24. J. Bayer and C. Osendorfer. Learning stochastic recurrent networks. In NIPS 2014 Workshop on Advances in Variational Inference, 2014.
25. G. Hinton, A. Krizhevsky, and S. Wang. Transforming auto-encoders. In Proc. of ICANN, 2011.
26. L. Dinh, D. Krueger, and Y. Bengio. Nice: Non-linear independent components estimation. CoRR, abs/1410.8516, 2015. URL http://arxiv.org/abs/1410.8516.
27. T. Cohen and M. Welling. Transformation properties of learned visual representations. In ICLR, 2015.
28. G. W. Taylor, L. Sigal, D. J. Fleet, and G. E. Hinton. Dynamical binary latent variable models for 3d human pose tracking. In Proc. of CVPR, 2010.
29. R. Memisevic. Learning to relate images. IEEE Trans. on PAMI, 35(8):1829-1846, 2013.
30. J. Langford, R. Salakhutdinov, and T. Zhang. Learning nonlinear dynamic models. In ICML, 2009.
31. M. West and J. Harrison. Bayesian Forecasting and Dynamic Models (Springer Series in Statistics). Springer-Verlag, February 1997. ISBN 0387947256.
32. T. Matsubara, V. Gómez, and H. J. Kappen. Latent Kullback Leibler control for continuous-state systems using probabilistic graphical models. UAI, 2014.