Feedback controller parameterizations for Reinforcement Learning

IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning

Volumn , Issue , 2011, Pages 310-317

  Roberts, John W ^a   Manchester, Ian R ^a   Tedrake, Russ ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FEEDBACK CONTROLLER; LEARNING CONTROLLERS; LEARNING CONVERGENCE; LINEAR CONTROLLERS; PARAMETERIZATIONS; POOR PERFORMANCE; REACHING TASK; STABLE SYSTEMS; YOULA PARAMETERIZATION;

ARTIFICIAL INTELLIGENCE; DYNAMIC PROGRAMMING; FEEDBACK CONTROL; LINEAR CONTROL SYSTEMS; NONLINEAR CONTROL SYSTEMS; PARAMETERIZATION; REINFORCEMENT LEARNING;

CONTROLLERS;

EID: 80052242432     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ADPRL.2011.5967370     Document Type: Conference Paper

References (30)

1. Pieter Abbeel, Adam Coates, Morgan Quigley, and Andrew Y. Ng. An application of reinforcement learning to aerobatic helicopter flight. In Proceedings of the Neural Information Processing Systems (NIPS '07), volume 19, December 2006.
2. Pieter Abbeel, Varun Ganapathi, and Andrew Y. Ng. Learning vehicular dynamics, with application to modeling helicopters. NIPS, 2006.
3. Anderson, J. M. Streitlien, K. Barrett, D.S. Triantafyllou, and M.S. Oscillating foils of high propulsive efficiency. Journal of Fluid Mechanics, 360:41-72, 1998. (Pubitemid 128488580)
4. Brian D.O. Anderson. From youla-kucera to identification, adaptive and nonlinear control. Automatica, 34(12):1485-1506, 1998. (Pubitemid 128399359)
5. Vincent D. Blondel and John N. Tsitsiklis. The boundedness of all products of a pair of matrices is undecidable. Systems & Control Letters, 41(2):135-140, 2000.
6. Stephen P. Boyd and Craig H. Barratt. Linear Controller Design: Limits of Performance. Prentice Hall, NJ, 1991.
7. E. F. Camacho and Carlos Bordons. Model Predictive Control. Springer- Verlag, 2nd edition, 2004.
8. C. A. Desoer and R. W. Liu. Global parametrization of feedback systems with nonlinear plants. Systems & Control Letters, 1(4):249-251, 1982.
9. John C. Doyle, Bruce A. Francis, and Allen R. Tannenbaum. Feedback Control Theory. Macmillan Publishing Company, New York, 1992.
10. M. Green and D.J.N. Limebeer. Linear Robust Control. Prentice Hall, 1995.
11. Joao P. Hespanha. Linear Systems Theory. Princeton Press, 2009.
12. Joao P. Hespanha and A. Stephen Morse. Switching between stabilizing controllers. Automatica, 38(11):1905 - 1917, 2002.
13. J. Kober, B. Mohler, and J. Peters. Imitation and reinforcement learning for motor primitives with perceptual coupling. In From Motor to Interaction Learning in Robots. Springer, 2009.
14. J. Zico Kolter and Andrew Y. Ng. Policy search via the signed derivative. Proceedings of Robotics: Science and Systems (RSS), 2009.
15. L. Lessard and S. Lall. Quadratic invariance is necessary and sufficient for convexity. In submitted to 2011 American Control Conference, 2011.
16. Daniel Liberzon. Switching in Systems and Control. Birkhauser, Boston, 2003.
17. Wei-Min Lu. A state-space approach to parameterization of stabilizing controllers for nonlinear systems. IEEE Transactions on Automatic Control, 40(9):1576-1588, sep. 1995.
18. A. D. B. Paice and J. B. Moore. On the youla-kucera parametrization for nonlinear systems. Systems & Control Letters, 14(2):121-129, 1990. (Pubitemid 20673792)
19. Rantzer, A., Megretski, and A. A convex parameterization of robustly stabilizing controllers. Automatic Control, IEEE Transactions on, 39(9):1802 -1808, sep. 1994.
20. John W. Roberts, Lionel Moret, Jun Zhang, and Russ Tedrake. Motor learning at intermediate reynolds number: Experiments with policy gradient on the flapping flight of a rigid wing. In From Motor to Interaction Learning in Robots. Springer, 2009.
21. John W. Roberts and Russ Tedrake. Signal-to-noise ratio analysis of policy gradient algorithms. In Advances of Neural Information Processing Systems (NIPS) 21, page 8, 2009.
22. Rotkowitz, M., Lall, and S. A characterization of convex problems in decentralized control. IEEE Transactions on Automatic Control, 51(2):274-286, feb. 2006.
23. Stefan Schaal, Auke Ijspeert, and Aude Billard. Computational approaches to motor learning by imitation. Philosophical Transactions: Biological Sciences, 358(1431):537-547, March 2003. (Pubitemid 36380533)
24. Russ Tedrake, Teresa Weirui Zhang, and H. Sebastian Seung. Learning to walk in 20 minutes. In Proceedings of the Fourteenth Yale Workshop on Adaptive and Learning Systems, Yale University, New Haven, CT, 2005.
25. Tsitsiklis, John N., Blondel, and Vincent D. The lyapunov exponent and joint spectral radius of pairs of matrices are hardwhen not impossibleto compute and to approximate. Mathematics of Control, Signals, and Systems (MCSS), 10:31-40, 1997. 10.1007/BF01219774. (Pubitemid 127553599)
26. R.J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8:229-256, 1992.
27. H. S. Witsenhausen. A counterexample in stochastic optimum control. SIAM Journal on Control, 6(1):131-147, 1968.
28. J. Wu and S. Lall. Nonlinear youla parametrization and information constraints for decentralized control. In American Control Conference (ACC), 2010, pages 5614 -5619, Baltimore, MD, Jun 2010.
29. D. Youla, H. Jabr, and J. Bongiorno Jr. Modern Wiener-Hopf design of optimal controllers-part II: The multivariable case. IEEE Transactions on Automatic Control, 21(3):319-338, 1976.
30. Zames and G. Feedback and optimal sensitivity: Model reference transformations, multiplicative seminorms, and approximate inverses. Automatic Control, IEEE Transactions on, 26(2):301-320, apr. 1981. (Pubitemid 11505907)