Simultaneous policy update algorithms for learning the solution of linear continuous-time H∞ state feedback control

Information Sciences

Volumn 222, Issue , 2013, Pages 472-485

  Wu, Huai Ning ^a   Luo, Biao ^a  

^a BEIHANG UNIVERSITY   (China)

Author keywords

Algebra Riccati equation; H state feedback control; Lyapunov equation; Offline; Online; Simultaneous policy update algorithm

Indexed keywords

ACTION POLICIES; COMPARATIVE SIMULATION; FIXED POINT EQUATION; INTERNAL SYSTEMS; LINEAR CONTINUOUS-TIME; LYAPUNOV EQUATION; MODEL BASED APPROACH; MODEL FREE; OFFLINE; ONLINE; ONLINE VERSIONS; ZERO-SUM GAME;

ALGEBRA; FEEDBACK CONTROL; LYAPUNOV FUNCTIONS; REINFORCEMENT LEARNING; RICCATI EQUATIONS; STATE FEEDBACK;

ALGORITHMS;

EID: 84870062175     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2012.08.012     Document Type: Article

References (33)

1. M. Abu-Khalaf, F.L. Lewis, and J. Huang Neurodynamic programming and zero-sum games for constrained control systems IEEE Transactions on Neural Networks 19 7 2008 1243 1252
2. R.W. Beard, and T.W. Mclain Successive Galerkin approximation algorithms for nonlinear optimal and robust control International Journal of Control 71 5 1998 717 743
3. R.W. Beard, G.N. Saridis, and J. Wen Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation Automatica 33 12 1997 2159 2177
4. R.W. Beard, G.N. Saridis, and J. Wen Approximate solutions to the time-invariant Hamilton-Jacobi-Bellman equation Journal of Optimization Theory and Applications 96 3 1998 589 626
5. D.P. Bertsekas Approximate policy iteration a survey and some new methods Journal of Control Theory and Applications 9 3 2011 310 335
6. W. Cheney Analysis for Applied Mathematics 2001 Springer-Verlag New York
7. ∞ control problems Numerical Algorithms 57 3 2011 357 375
8. V. Dragan, and I. Ivanov A numerical procedure to compute the stabilising solution of game theoretic Riccati equations of stochastic control International Journal of Control 84 4 2011 783 800
9. Y.T. Feng, A. Varga, B.D.O. Anderson, and M. Lovera A new iterative algorithm to solve periodic Riccati differential equations with sign indefinite quadratic terms IEEE Transactions on Automatic Control 56 4 2011 929 934
10. Y.T. Feng, and B.D.O. Anderson An iterative algorithm to solve state-perturbed stochastic algebraic Riccati equations in LQ zero-sum games Systems & Control Letters 59 1 2010 50 56
11. M. Green, and D.J.N. Limebeer Linear Robust Control 1995 Prentice-Hall Englewood Cliffs, NJ
12. K.-S. Hwang, H.-Y. Lin, Y. -P Hsu, and H.-H. Yu Self-organizing state aggregation for architecture design of Q-learning Information Sciences 181 13 2011 2813 2822
13. K.-S. Hwang, Y.-J. Chen, W.-C. Jiang, and T.-W. Yang Induced states in a decision tree constructed by Q-learning Information Sciences 213 5 2012 39 49
14. L. Kantorovitch The method of successive approximation for functional equations Acta Mathematica 71 1 1939 63 97
15. D.L. Kleinman On an iterative technique for Riccati equation computations IEEE Transactions on Automatic Control 13 1 1968 114 115
16. A. Lanzon, Y. Feng, B.D.O. Anderson, and M. Rotkowitz Computing the positive stabilizing solution to algebraic Riccati equations with an indefinite quadratic term via a recursive method IEEE Transactions on Automatic Control 53 10 2008 2280 2291
17. F.L. Lewis, and D. Vrabie Reinforcement learning and adaptive dynamic programming for feedback control IEEE Circuits and Systems Magazine 9 3 2009 32 50
18. W.B. Powell Approximate Dynamic Programming: Solving the Curses of Dimensionality 2007 Wiley-Interscience Hoboken, NJ
19. L.B. Rall A note on the convergence of Newton's method SIAM Journal on Numerical Analysis 11 1 1974 34 36
20. G.N. Saridis, and C.G. Lee An approximation theory of optimal control for trainable manipulators IEEE Transactions on Systems, Man and Cybernetics Part B-Cybernetics 9 3 1979 152 159
21. B. Stevens, and F.L. Lewis Aircraft Control and Simulation second ed. 2003 John Wiley New Jersey
22. R.S. Sutton, and A.G. Barto Reinforcement Learning: An Introduction 1998 MIT Press Cambridge, Mass
23. R.A. Tapia The Kantorovich theorem for Newton's method The American Mathematical Monthly 78 4 1971 389 392
24. K.G. Vamvoudakis, F.L. Lewis, Online solution of nonlinear two-player zero-sum games using synchronous policy iteration, in: Proceedings of the 49th IEEE Conference on Decision and Control (CDC), 2010, pp. 3040-3047.
25. K.G. Vamvoudakis, and F.L. Lewis Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem Automatica 46 5 2010 878 888
26. K.G. Vamvoudakis, Online Learning Algorithms for Differential Dynamic Games and Optimal Control, Ph.D dissertation, Faculty of the Graduate School, University of Texas at Arlington, 2011.
27. ∞ control IEEE Transactions on Automatic Control 37 6 1992 770 784
28. N.A. Vien, H. Yu, and T.C. Chung Hessian matrix distribution for Bayesian policy gradient reinforcement learning Information Sciences 181 9 2011 1671 1685
29. D. Vrabie, and F.L. Lewis Adaptive dynamic programming for online solution of a zero-sum differential game Journal of Control Theory and Applications 9 3 2011 353 360
30. D. Vrabie, O. Pastravanu, M. Abu-Khalaf, and F.L. Lewis Adaptive optimal control for continuous-time linear systems based on policy iteration Automatica 45 2 2009 477 484
31. F. Wang, H. Zhang, and D. Liu Adaptive dynamic programming: an introduction IEEE Computational Intelligence Magazine 4 2 2009 39 47
32. X. Wang, Y. Cheng, and J. Yi A fuzzy actor-critic reinforcement learning network Information Sciences 177 18 2007 3764 3781
33. H. Zhang, Q. Wei, and D. Liu An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games Automatica 47 1 2011 207 214