Combining the benefits of function approximation and trajectory optimization

Robotics: Science and Systems

Volumn , Issue , 2014, Pages

  Mordatch, Igor ^a   Todorov, Emanuel ^a  

^a Department of Health Systems and Population Health   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

INVERSE PROBLEMS; LEARNING SYSTEMS; NEURAL NETWORKS; TEACHING; TRAJECTORIES;

APPROXIMATION OPTIMIZATION; FUNCTIONS APPROXIMATIONS; HARD PROBLEMS; HIGH-FIDELITY SOLUTIONS; IN-CONTROL; MACHINE-LEARNING; NEURAL-NETWORKS; OPTIMIZERS; ROBOTIC CONTROLS; TRAJECTORY OPTIMIZATION;

AERODYNAMICS;

EID: 85131220210     PISSN: None     EISSN: 2330765X     Source Type: Conference Proceeding    
DOI: 10.15607/RSS.2014.X.052     Document Type: Conference Paper

References (39)

1. P. Abbeel, A. Coates, M. Quigley, and A. Ng. An application of reinforcement learning to aerobatic helicopter flight. NIPS, 2006.
2. M. Al Borno, M. de Lasa, and A. Hertzmann. Trajectory optimization for full-body movements with complex contacts. IEEE Trans Visualization and Computer Graphics, 2013.
3. Jos Bento, Nate Derbinsky, Javier Alonso-Mora, and Jonathan S. Yedidia. A message-passing algorithm for multi-agent trajectory planning. In NIPS, pages 521–529, 2013.
4. D. Bertsekas and J. Tsitsiklis. Neuro-dynamic programming. Athena Scientific, Belmont, MA, 1997.
5. S. Boyd, N. Pariks, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 2011.
6. L. Deng, G. Hinton, and B. Kingsbury. New types of deep neural network learning for speech recognition and related applications: An overview. ICASSP, 2013.
7. Hartmut Geyer and Hugh Herr. A muscle-reflex model that encodes principles of legged mechanics produces human walking dynamics and muscle activities. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 18(3), 2010.
8. G. Hoffmann, H. Huang, S. Waslander, and C. Tomlin. Quadrotor helicopter flight dynamics and control: Theory and experiment. Proc AIAA, 2007.
9. A. Ijspeert. Central pattern generators for locomotion control in animals and robots: a review. Neural Networks, 2008.
10. A. Krizhevsky, I. Sutskever, and G. Hinton. Imagenet classification with deep convolutional neural networks. NIPS, 2012.
11. M. Lagoudakis and R. Parr. Least-squares policy iteration. Journal of Machine Learning Research, 2003.
12. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proc of the IEEE, 1998.
13. S. Levine and V. Koltun. Variational policy search via trajectory optimization. NIPS, 2013.
14. S. Levine and V. Koltun. Guided policy search. ICML, 2013.
15. Sergey Levine and Vladlen Koltun. Learning complex neural network policies with trajectory optimization. In ICML ’14: Proceedings of the 31st International Conference on Machine Learning, 2014.
16. D. Mellinger, N. Michael, and V. Kumar. Trajectory generation and control for precise aggressive maneuvers with quadrotors. Internatiional Journal of Robotics Research, 2012.
17. W. Miller, R. Sutton, and P. Werbos. Neural Networks for Control. MIT Press, 1990.
18. I. Mordatch, Z. Popovic, and E Todorov. Contact-invariant optimization for hand manipulation. Symposium on Computer Animation, 2012.
19. I. Mordatch, E Todorov, and Z. Popovic. Discovery of complex behaviors through contact-invariant optimization. SIGGRAPH, 2012.
20. I. Mordatch, J. Wang, E. Todorov, and V. Koltun. Animating human lower limbs using contact-invariant optimization. SIGGRAPH Asia, 2013.
21. Hua Ouyang, Niao He, Long Tran, and Alexander G. Gray. Stochastic alternating direction method of multipliers. In ICML (1), volume 28 of JMLR Proceedings, pages 80–88. JMLR.org, 2013.
22. J. Peters and S. Schaal. Natural actor-critic. Neurocom- puting, 71:1180–1190, 2008.
23. M. Posa, C. Cantu, and R. Tedrake. A direct method for trajectory optimization of rigid bodies through contact. International Journal of Robotics Research, 2014.
24. Salah Rifai, Pascal Vincent, Xavier Muller, Xavier Glo-rot, and Yoshua Bengio. Contractive auto-encoders: Explicit invariance during feature extraction. In ICML, pages 833–840, 2011.
25. Stphane Ross, Geoffrey J. Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In AISTATS, volume 15 of JMLR Proceedings, pages 627–635. JMLR.org, 2011.
26. S. Schaal, A. Ijspeert, and A. Billard. Computational approaches to motor learning by imitation. Transactions of the Royal Society B, 2003.
27. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint, 2014.
28. J. Si, A. Barto, W. Powell, and D. Wunsch. Handbook of Learning and Approximate Dynamic Programming. Wiley-IEEE Press, 2004.
29. P. Simard, Y. LeCun, J. Denker, and B. Victorri. Transformation invariance in pattern recognition: Tangent distance and tangent propagation. Lecture Notes in Computer Science, 1998.
30. R. Stengel. Optimal Control and Estimation. Dover, New York, 1994.
31. R. Sutton, D. Mcallester, S. Singh, and Y. Mansour. Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems, 2000.
32. R.S. Sutton and A.G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge MA, 1998.
33. Y. Tassa, T. Erez, and E. Todorov. Synthesis and stabilization of complex behaviors through online trajectory optimization. IROS, 2012.
34. E. Theodorou, J. Buchli, and S. Schaal. Learning policy improvements with path integrals. AISTATS, 2010.
35. E. Todorov and W. Li. A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems. American Control Conference, pages 300–306, 2005.
36. E. Todorov, T. Erez, and Y. Tassa. MuJoCo: A physics engine for model-based control. IROS, 2012.
37. M. Toussaint. Robot trajectory optimization using approximate inference. International Conference on Machine Learning, 26:1049–1056, 2009.
38. A. Witkin and M. Kass. Spacetime constraints. SIG-GRAPH, 1988.
39. M. Zhong, M. abd Johnson, Y. Tassa, T. Erez, and E Todorov. Value function approximation and model-predictive control. IEEE ADPRL, 2013.