Integral Q-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems

Automatica

Volumn 48, Issue 11, 2012, Pages 2850-2859

  Lee, Jae Young ^a   Park, Jin Bae ^a   Choi, Yoon Ho ^b  

^a Yonsei University   (South Korea)

^b Kyonggi University   (South Korea)

Author keywords

Adaptive control; LQR; Optimization under uncertainties; Policy iteration; Q learning

Indexed keywords

ADAPTIVE CONTROL; LQR; OPTIMIZATION UNDER UNCERTAINTY; POLICY ITERATION; Q-LEARNING;

CONTINUOUS TIME SYSTEMS; INTEGRAL EQUATIONS; ITERATIVE METHODS; LINEAR CONTROL SYSTEMS; OPTIMIZATION; REINFORCEMENT LEARNING;

LEARNING ALGORITHMS;

EID: 84867400046     PISSN: 00051098     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.automatica.2012.06.008     Document Type: Article

References (32)

1. ∞ control Automatica 43 3 2007 473 481
2. Baird, L. C. III (1994). Reinforcement learning in continuous-time: advantage updating. In Proc. of ICNN. vol. 4 (pp. 2448-2453).
3. S.N. Balakrishnan, D. Ding, and F.L. Lewis Issues on stability of ADP feedback controllers for dynamical systems IEEE Transactions on Systems, Man and Cybernetics, Part B 38 4 2008 913 917
4. D.P. Bertsekas, and J.N. Tsitsiklis Neuro-dynamic programming 1996 Athena Scientific Belmont, MA
5. D.P. Bertsekas, and J.N. Tsitsiklis Dynamic programming and suboptimal control: a survey from ADP to MPC European Journal of Control 11 2005 310 334
6. Bradtke, S. J.; & Ydstie, B. E. (1994). Adaptive linear quadratic control using policy iteration. In Proc. ACC (pp. 3475-3479).
7. Dong, W.; & Farrell, J. A. (2009). Adaptive approximately optimal control of unknown nonlinear systems based on locally weighted learning. In Proc. CDC (pp. 345-350).
8. K. Doya Reinforcement learning in continuous-time and space Neural Computation 12 2000 219 245
9. L.P. Kaelbling, and A.W. Moore Reinforcement learning: a survey Journal of Artificial Intelligence Research 4 1996 237 285
10. H.K. Khalil Nonlinear systems 2002 Prentice Hall
11. D. Kleinman On the iterative technique for Riccati equation computations IEEE Transactions on Automatic Control 13 1 1968 114 115
12. P. Kokotovic, H.H. Khalil, and J. O'Reilly Singular perturbation methods in control: analysis and design 1986 Academic Press, Inc.
13. Landelius, T. (1997). Reinforcement learning and distributed local model synthesis. Ph.D. dissertation. Sweden: Linkoping University.
14. J.M. Lee, and J.H. Lee Approximate dynamic programming strategies and their applicability for process control: a review and future directions International Journal of Control, Automation, and Systems (IJCAS) 2 3 2004 263 278
15. Lee, J. Y.; Park, J. B.; & Choi, Y. H. (2009). Model-free approximate dynamic programming for continuous-time linear systems. In Proc. CDC (pp. 5009-5014).
16. F.L. Lewis, and V. Syrmos Optimal control 2nd ed. 1995 Wiley New York
17. F.L. Lewis, and K.G. Vamvoudakis Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data IEEE Transactions on Systems, Man and Cybernetics, Part B 41 1 2010 14 25
18. F.L. Lewis, and D. Vrabie Reinforcement learning and adaptive dynamic programming for feedback control IEEE Circuits and Systems Magazine 9 3 2009 32 50
19. Mehta, P.; & Meyn, S. (2009). Q-learning and pontryagins minimum principle. In Proc. CDC (pp. 3598-3605).
20. J.J. Murray, C.J. Cox, G.G. Lendaris, and R. Saeks Adaptive dynamic programming IEEE Transactions on Systems, Man and Cybernetics, Part C 32 2 2002 140 153
21. W.B. Powell Approximate dynamic programming: solving the curses of dimensionality 2007 Wiley-Interscience
22. D.V. Prokhorov, and D.C. Wunsch II Adaptive critic designs IEEE Transactions on Neural Networks 8 5 1997 997 1007
23. J. Si, A.G. Barto, W.B. Powell, and D. Wunsch Handbook of learning and approximate dynamic programming 2004 Wiley-IEEE Press
24. R.S. Sutton, and A.G. Barto Reinforcement learning - an introduction 1998 MIT Press Cambridge, Massachusetts
25. K.G. Vamvoudakis, and F.L. Lewis Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem Automatica 46 5 2010 878 888
26. D. Vrabie, O. Pastravanu, M. Abu-Khalaf, and F.L. Lewis Adaptive optimal control for continuous-time linear systems based on policy iteration Automatica 45 2 2009 477 484
27. F.Y. Wang, H. Zhang, and D. Liu Adaptive dynamic programming: an introduction IEEE Computational Intelligence Magazine 4 3 2009 39 47
28. Watkins, C. J. C. H.; & Dayan, P. (1989). Learning from delayed rewards. Ph.D. dissertation. Cambridge Univ.; Cambridge, UK.
29. C.J.C.H. Watkins, and P. Dayan Q-learning Machine Learning 8 1992 279 292
30. P.J. Webos Approximate dynamic programming for real-time control and neural modeling D.A. White, D.A. Sofge, Handbook of intelligent control 1992 Van Nostrand Reinhold New York
31. J.C. Willems, P. Rapisarda, I. Markovsky, and B.L.M. Moor A note on persistency of excitation Systems & Control Letters 54 4 2005 325 329
32. Zhang, H.; Huang, J.; & Lewis, F. L. (2009). Algorithm and stability of ATC receding horizon control. In IEEE Symp. ADPRL (pp. 28-35).