Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2009, Pages 3180-3187

  Vamvoudakis, Kyriakos G ^a   Lewis, Frank L ^a  

^a UNIVERSITY OF TEXAS AT ARLINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACTOR-CRITIC ALGORITHM; ACTOR-NETWORK; CLOSED-LOOP; CONTINUOUS TIME; DYNAMICAL STABILITY; INFINITE HORIZONS; ON-LINE ADAPTIVE ALGORITHMS; ON-LINE ALGORITHMS; OPTIMAL CONTROL PROBLEM; OPTIMAL CONTROL SOLUTION; OPTIMAL CONTROLLER; OPTIMAL VALUE FUNCTIONS; PERSISTENCE OF EXCITATION; POLICY ITERATION; SIMULATION EXAMPLE; TUNING ALGORITHM;

ADAPTIVE ALGORITHMS; LEARNING ALGORITHMS; NEURAL NETWORKS; NONLINEAR SYSTEMS; ONLINE SYSTEMS; OPTIMAL CONTROL SYSTEMS; OPTIMIZATION; SYSTEM STABILITY; TUNING;

CONTINUOUS TIME SYSTEMS;

EID: 70449382072     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2009.5178586     Document Type: Conference Paper

References (21)

1. M. Abu-Khalaf, F. L. Lewis, "Nearly Optimal Control Laws for Nonlinear Systems with Saturating Actuators Using a Neural Network HJB Approach", Automatica, vol. 41, no. 5, pp. 779-791, 2005.
2. L. C. Baird III, "Reinforcement Learning in Continuous Time: Advantage Updating", Proc. Of ICNN, Orlando FL, vol. 4, pp. 2448- 2453, 1994.
3. R. Beard, G. Saridis, J. Wen, "Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation", Automatica, vol. 33, no. 12, pp. 2159-2177, 1997.
4. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, MA, 1996.
5. J. W. Curtis, R. W. Beard, "Successive Collocation: An Approximation to Optimal Nonlinear Control", IEEE Proc. ACC01, vol. 5, pp. 3481-3485, 2001.
6. K. Doya, "Reinforcement Learning In Continuous Time and Space", Neural Computation, 12 (1), pp. 219-245, 2000.
7. T. Hanselmann, L. Noakes, and A. Zaknich, "Continuous-Time Adaptive Critics", IEEE Transactions on Neural Networks, 18 (3), pp. 631-647, 2007.
8. R. A. Howard, Dynamic Programming and Markov Processes, MIT Press, Cambridge, Massachusetts, 1960.
9. D. Kleinman, "On an Iterative Technique for Riccati Equation Computations", IEEE Trans, on Automatic Control, vol. 13, pp. 114- 115, February, 1968.
10. F. L. Lewis, S. Jagannathan, A. Yesildirek, Neural Network Control of Robot Manipulators and Nonlinear Systems, Taylor & Francis 1999.
11. F. L. Lewis, K. Liu, and A. Yesildirek, "Neural Net Controller with Guaranteed Tracking Performance", IEEE Transactions on Neural Networks, vol. 6, no. 3, pp. 703-715, 1995.
12. F. L. Lewis, V. L. Syrmos, Optimal Control, John Wiley, 1995.
13. J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, "Adaptive Dynamic Programming", IEEE Trans, on Systems, Man and Cybernetics, vol. 32, no. 2, pp 140-153, 2002.
14. D. Prokhorov, D. Wunsch, "Adaptive critic designs, " IEEE Trans, on Neural Networks, vol. 8, no 5, pp. 997-1007, 1997.
15. J. Si, A. Barto, W. Powel, D. Wunch, Handbook of Learning and Approximate Dynamic Programming, John Wiley, New Jersey, 2004.
16. R. S. Sutton, A. G. Barto, Reinforcement Learning- An Introduction, MIT Press, Cambridge, Massachusetts, 1998.
17. D. Vrabie, F. Lewis, "Adaptive Optimal Control Algorithm for Continuous-Time Nonlinear Systems Based on Policy Iteration", IEEE Proc. CDC08, pp. 73-79, 2008
18. D. Vrabie, O. Pastravanu, F. Lewis, M. Abu-Khalaf, "Adaptive Optimal Control for Continuous-Time Linear Systems Based on Policy Iteration", Automatica (to appear)
19. P. J. Werbos, Beyond Regression: New Tools for Prediction and Analysis in the Behavior Sciences, Ph. D. Thesis, 1974.
20. P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling, " Handbook of Intelligent Control, ed. D. A. White and D. A. Sofge, New York: Van Nostrand Reinhold, 1992.
21. P. Werbos, "Neural networks for control and system identification", IEEE Proc. CDC89, vol. 1, pp. 260-265, 1989.