Robust reinforcement learning control using integral quadratic constraints for recurrent neural networks

IEEE Transactions on Neural Networks

Volumn 18, Issue 4, 2007, Pages 993-1002

  Anderson, Charles W ^a   Young, Peter Michael ^a   Buehner, Michael R ^a   Knight, James N ^a   Bush, Keith A ^a   Hittle, Douglas C ^a  

^a Colorado State University   (United States)

Author keywords

Integral quadratic constraints (IQCs); Recurrent neural networks (NNs); Reinforcement learning; Robust control

Indexed keywords

CONSTRAINT THEORY; CONTROL SYSTEM STABILITY; FEEDBACK CONTROL; LINEAR SYSTEMS; RECURRENT NEURAL NETWORKS; ROBUST CONTROL;

INTEGRAL QUADRATIC CONSTRAINT; TIME-INVARIANT SYSTEM;

REINFORCEMENT LEARNING;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; BIOMIMETICS; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; FEEDBACK SYSTEM; METHODOLOGY; PROBABILITY; REGRESSION ANALYSIS; REINFORCEMENT; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BIOMIMETICS; COMPUTER SIMULATION; DECISION SUPPORT TECHNIQUES; FEEDBACK; LEAST-SQUARES ANALYSIS; MARKOV CHAINS; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); REINFORCEMENT (PSYCHOLOGY);

EID: 34547097516     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2007.899520     Document Type: Article

References (20)

1. K. Zhou and J. C. Doyle, Essentials of Robust Control. Englewood Cliffs, NJ: Prentice-Hall, 1998.
2. A. Megretski and A. Rantzer, "System analysis via integral quadratic constraints," IEEE Trans. Autom. Control, vol. 42, no. 6, pp. 819-830, Jun. 1997.
3. C. W. Anderson, R. M. Kretchmar, P. M. Young, and D. C. Hittle, "Robust reinforcement learning using integral-quadratic constraints," in Learning and Approximate Dynamic Programming, J. Si, A. Barto, W. Powell, and D. Wunsch, Eds. New York: Wiley, 2004, pp. 337-358.
4. R. Kretchmar, P. Young, C. Anderson, D. Hittle, M. Anderson, C. Delnero, and J. Tu, "Robust reinforcement learning control with static and dynamic stability," Int. J. Robust Nonlinear Control, vol. 11, pp. 1469-1500, 2001.
5. R. M. Kretchmar, "A synthesis of reinforcement learning and robust control theory," Ph.D. dissertation, Dept. Comput. Sci., Colorado State Univ., Fort Collins, CO, 2000.
6. P. Gahihet, A. Nemirovski, A. J. Laub, and M. Chilali, LMI Control Toolbox. MathWorks Inc., 1995.
7. J. F. Sturm, "Using SEDUMI 1.02, a Matlab toolbox for optimization over symmetric cones," Optim. Methods Software vol. 11-12, pp. 625-653, 1999 Online. Available: http://sedumi.mcmaster.ca
8. K. C. Toh, R. H. Tutuncu, and M. J. Todd, "SDPT3 - A Matlab software for semidefinite-quadratic-linear programming," 2006 Online. Available: http://www.math.nus.edu.sg/~mattohkc/sdpt3.html
9. J. Morimoto and K. Doya, "Robust reinforcement learning," Neural Comput., vol. 17, pp. 335-359, 2005.
10. T. J. Perkins and A. G. Barto, "Lyapunov design for safe reinforcement learning," J. Mach. Learn. Res., vol. 3, pp. 803-832, 2002.
11. A. Megretski and A. Rantzer, "System analysis via integral quadratic constraints: Part II," Lund Inst. Technol., Lund, Sweden, 1997, Tech. Rep. ISRN LUTFD2/TFRT-7559-SE.
12. H. K. Khalil, Nonlinear Systems, 3rd ed. Englewood Cliffs, NJ: Prentice-Hall, 2002.
13. U. Jönsson, C.-Y. Kao, A. Megretski, and A. Rantzer, "A guide to IQCβ: Matlab toolbox for robust stability and performance analysis," [Online]. Available: http://www.ee.unimelb.edu.au/staff/cykao/
14. Robust Control Toolbox. MathWorks, Inc. [Online]. Available: http:// www.mathworks.com/products/robust
15. Control System Toolbox. MathWorks, Inc. [Online]. Available: http:// www.mathworks.com/products/control
16. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction Cambridge, MA: MIT Press, 1998.
17. R. J. Williams and D. Zipser, "A learning algorithm for continually running fully recurrent neural networks," Neural Comput., vol. 1, pp. 270-280, 1989.
18. H. Jaeger, "The echo state approach to analyzing and training recurrent neural networks," Fraunhofen, Germany, Tech. Rep., 2001.
19. K. Bush and C. Anderson, "Modeling reward functions for incomplete state representations via echo state networks," in Proc. Int. Joint Conf. Neural Netw., 2005, vol. 5, pp. 2995-3000.
20. N. E. Barabanov and D. V. Prokhorov, "Stability analysis of discrete-time recurrent neural networks," IEEE Trans. Neural Netw. vol. 13, no. 2, pp. 292-303, Mar. 2002.