Integral reinforcement learning with explorations for continuous-time nonlinear systems

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2012, Pages

  Lee, Jae Young ^a   Park, Jin Bae ^a   Choi, Yoon Ho ^b  

^a Yonsei University   (South Korea)

^b Kyonggi University   (South Korea)

Author keywords

[No Author keywords available]

Indexed keywords

ACTOR CRITIC; CONTINUOUS TIME; CONTINUOUS TIME NONLINEAR SYSTEMS; CONTROL INPUTS; INPUT-AFFINE; LEAST SQUARE; PARAMETERIZATIONS; PERSISTENTLY EXCITING; SIMULATION EXAMPLE; TIME VARYING SIGNAL;

CONTINUOUS TIME SYSTEMS; NEURAL NETWORKS; REINFORCEMENT LEARNING;

NONLINEAR SYSTEMS;

EID: 84865092901     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2012.6252508     Document Type: Conference Paper

References (22)

1. R. S. Sutton and A. G. Barto, Reinforcement Learning-An Introduction, MIT Press, Cambridge, Massachussetts, 1998.
2. J. Si, A. G. Barto, W. B. Powell, and D. Wunsch, Handbook of Learning and Approximate Dynamic Programming, Wiley-IEEE Press, 2004.
3. F. Y. Wang, H. Zhang, and D. Liu, "Adaptive dynamic programming: an introduction," IEEE Computational Magazine, vol. 4, no. 2, pp. 39-47, 2009.
4. F. L. Lewis and D. Vrabie, "Reinforcement learning and adaptive dynamic programming for feedback control," IEEE Circuits and Systems Magazine, vol. 9, no. 3, pp. 32-50, 2009.
5. D. V. Prokhorov and D. C. Wunsch II, "Adaptive critic designs," IEEE Trans. Neural Networks, vol. 8, no. 5, pp. 997-1007, 1997.
6. S. J. Bradtke and B. E. Ydstie, "Adaptive linear quadratic control using policy iteration," Proc. American Control Conference, Baltimore, Maryland, pp. 3475-3479, 1994.
7. A. Al-Tamimi, M. Abu-Khalaf, and F. L. Lewis, "Adaptive critic designs for discrete-time zero-sum games with application to H1 control," IEEE Trans. Syst., Man, Cybern.-Part B, vol. 37, no. 1, pp. 240-247, 2007.
8. A. Al-Tamimi, M. Abu-Khalaf, and F. L. Lewis, "Model-free Q-learning designs for discrete-time zero-sum games with application to H1 control," Automatica, vol. 43, no. 3, 473-481, 2007.
9. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof," IEEE Trans. Systems, Man, and Cybernetics-PART B: Cybernetics, vol. 38, no. 4, 2008.
10. K. Doya, "Reinforcement learning in continuous-time and space," Neural Computation 12, pp. 219-245, 2000.
11. T. Hanselmann, L. Noakes, and A. Zaknich, "Continuous-time adaptive critics," IEEE Trans. Neural Network, vol. 18, no. 3, pp. 631-647, 2007.
12. J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, "Adaptive Dynamic Programming," IEEE Trans. Systems, Mans and Cybernetics- PART B: Cybernetics, vol. 32, no. 2, pp. 140-153, 2002.
13. D. Vrabie, O. Pastravanu, M. Abu-Khalaf, and F. L. Lewis, "Adaptive optimal control for continuous-time linear systems based on policy iteration," Automatica, vol. 45, no. 2, pp. 477-484, 2009.
14. M. Abu-Khalaf and F. L. Lewis, "Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach," Automatica, vol. 41, no. 5, pp. 779-791, 2005.
15. D. Vrable and F. L. Lewis "Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems," Neural Networks, vol. 22, no. 3, pp. 237-246, 2009.
16. K. G. Vamvoudakis, F. L. Lewis "Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem," Automatica, pp. 878-888 vol. 46, no. 5, pp. 878-888, 2010.
17. S. Bhasin, M. Johnson, W. E. Dixon, "A model-free robust policy iteration algorithm for optimal control of nonlinear systems," 49th IEEE Conf. Decision and Control, Atlanta, GA, pp. 3060-3065, 2010.
18. J. Y. Lee, J. B. Park, and Y. H. Choi, "Policy-iteration-based adaptive optimal control for uncertain continuous-time linear systems with excitation signals," in Proc. Int'l Conf. on Control, Automation, and Systems (ICCAS), Ilsan, Kyonggi-Do, South Korea, pp. 646-651, Oct. 2010.
19. J. Y. Lee, J. B. Park, and Y. H. Choi, "Integral Q-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems," Automatica, accepted for publication, 2012.
20. G. N. Saridis and C. G. Lee, "An approximation theory of optimal control for trainable manipulators," IEEE Trans. Systems, Man, and Cybernetics-PART B: Cybernetics, vol. 9, no. 3, 1979.
21. J. C. Willems, P. Rapisarda, I. Markovsky, and Bart L. M. Moor, "A note on persistency of excitation," Systems & Control Letters, vol. 54, no. 4, pp. 325-329, 2005.
22. V. Nevistic and J. A. Primbs, "Constrained nonlinear optimal control: a converse HJB approach," Technical report CIT-CDS 96-021, California Institute of Technology, Pasadena, CA 91125, 1996