Linear quadratic tracking control of partially-unknown continuous-time systems using reinforcement learning

IEEE Transactions on Automatic Control

Volumn 59, Issue 11, 2014, Pages 3051-3056

  Modares, Hamidreza ^a   Lewis, Frank L ^a  

^a UNIVERSITY OF TEXAS AT ARLINGTON   (United States)

Author keywords

Causal solution; integral reinforcement learning; linear quadratic tracking; policy iteration; reinforcement learning.

Indexed keywords

CAUSAL SOLUTIONS; LINEAR QUADRATIC TRACKING; POLICY ITERATION;

REINFORCEMENT LEARNING;

EID: 84908432682     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2014.2317301     Document Type: Article

References (28)

1. F. L. Lewis, D. Vrabie, and V. Syrmos, Optimal Control, 3rd ed. New York: Wiley, 2012.
2. A. Mannava, S. N. Balakrishnan, L. Tang, and R. G. Landers, "Optimal tracking control of motion systems," IEEE Trans. Control Syst. Technol., vol. 20, no. 6, pp. 1548-1556, Nov. 2012.
3. R. S. Sutton and A. G. Barto, Reinforcement Learning-An Introduction. Cambridge, MA: MIT Press, 1998.
4. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
5. W. B. Powell, Approximate Dynamic Programming: Solving the Curses of Dimensionality. New York: Wiley-Interscience, 2007.
6. P. J. Werbos, "A menu of designs for reinforcement learning over time," in Neural Networks for Control, W. T. Miller, R. S. Sutton, and P. J. Werbos, Eds. Cambridge, MA: MIT Press, 1991, pp. 67-95.
7. F. L. Lewis, D. Vrabie, and K. G. Vamvoudakis, "Reinforcement learning and feedback control," IEEE Control Syst. Mag., pp. 76-105, Dec. 2012.
8. F. Y. Wang, H. Zhang, and D. Liu, "Adaptive dynamic programming: An introduction," IEEE Computational Intell. Mag., vol. 4, no. 2, pp. 39-47, May 2009.
9. K. Doya, "Reinforcement learning in continuous-time and space," Neural Computation, vol. 12, pp. 219-245, 2000.
10. M. Abu-Khalaf, F. L. Lewis, and J. Huang, "Policy iterations on the Hamilton-Jacobi-Isaacs equation for H? state feedback control with input saturation," IEEE Trans. Autom. Control, vol. 51, no. 12, pp. 1986-1995, Dec. 2006.
11. S. J. Bradtke, B. E. Ydestie, and A. G. Barto, "Adaptive linear quadratic control using policy iteration," in Proc. Amer. Control Conf., Baltimore, MD, Jun. 1994, pp. 3475-3476.
12. T. Landelius, "Reinforcement Learning and Distributed Local Model Synthesis," Ph.D. dissertation, Linkoping Univ., Linkoping, Sweden, 1997.
13. F. L. Lewis and K. Vamvoudakis, "Reinforcement learning for partially observable dynamic processes: Adaptive dynamic programming using measured output data," IEEE Trans. Syst, Man Cybern. B, vol. 41, no. 1, pp. 14-23, Feb. 2011.
14. D. Vrabie, O. Pastravanu, M. Abu-Khalaf, and F. L. Lewis, "Adaptive optimal control for continuous-time linear systems based on policy iteration," Automatica, vol. 45, no. 2, pp. 477-484, February 2009.
15. J. Y. Lee, J. B. Park, and Y. H. Choi, "Integral Q-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems," Automatica, vol. 48, no. 11, pp. 2850-2859, Nov. 2012.
16. Y. Jiang and Z. P. Jiang, "Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics," Automatica, vol. 48, no. 10, pp. 2699-2704, October 2012.
17. H. Zhang, Q. Wei, and Y. Luo, "A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm," IEEE Trans. Syst., Man Cybern. B, vol. 38, no. 4, pp. 937-942, Aug. 2008.
18. T. Dierks and S. Jagannathan, "Optimal tracking control of affine nonlinear discrete-time systems with unknown internal dynamics," in Proc. Joint 48th IEEE Conf. Decision Control Conf. & 28th Chinese Control Conf., Shanghai, China, Dec. 16-18, 2009, pp. 6750-6755.
19. Q. Wei and D. Liu, "Optimal tracking control scheme for discrete-time nonlinear systems with approximation errors," Adv. Neural Networks-Lecture Notes Comp. Sci., vol. 7952, pp. 1-10, 2013.
20. R. Song, W. Xiao, and Q. Wei, "Optimal tracking control for a class of nonlinear time-delay systems with actuator saturation," Adv. Brain Inspired Cognitive Syst.-Lecture Notes Comp. Sci., vol. 7888, pp. 208-215, 2013.
21. Y. Huang and D. Liu, "Neural-network-based optimal tracking control scheme for a class of unknown discrete-time nonlinear systems using iterative ADP algorithm," Neurocomputing, vol. 125, pp. 46-56, 2014.
22. H. Zhang, L. Cui, X. Zhang, and Y. Luo, "Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method," IEEE Trans. Neural Networks, vol. 22, no. 12, pp. 2226-2236, Dec. 2011.
23. D. Vrabie and F. L. Lewis, "Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems," Neural Network, vol. 22, pp. 237-246, April 2009.
24. E. Barbieri and R. Alba-Flores, "On the infinite-horizon LQ tracker," Syst. Control Lett., vol. 40, no. 2, pp. 77-82, Jun. 2000.
25. E. Barbieri and R. Alba-Flores, "Real-time infinite horizon linearquadratic tracking controller for vibration quenching in flexible beams," in Proc. IEEE Conf. Syst., Man, Cybern., Taipei, Taiwan, Oct. 8-11, 2006, pp. 38-43.
26. J. Engwerda, LQ Dynamic Optimization and Differential Games. New York: Wiley, 2005.
27. D. L. Kleinman, "On an iterative technique for Riccati equation computations," IEEE Trans. Autom. Control, vol. 18, no. 1, pp. 114-115, Feb. 1968.
28. J. Y. Lee, J. B. Park, and Y. H. Choi, "Integral reinforcement learning with explorations for continuous-time nonlinear systems," in IEEE World Congress on Computational Intelligence, Brisbane, Australia, Jun. 2012, pp. 10-15.