Adaptive optimal control for a class of continuous-time affine nonlinear systems with unknown internal dynamics

Neural Computing and Applications

Volumn 23, Issue 7-8, 2013, Pages 1843-1850

  Liu, Derong ^a   Yang, Xiong ^a   Li, Hongliang ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

Adaptive dynamic programming; Adaptive optimal control; Neural network; Nonlinear system; Online control; Policy iteration; Reinforcement learning

Indexed keywords

ADAPTIVE DYNAMIC PROGRAMMING; ADAPTIVE OPTIMAL CONTROL; AFFINE NONLINEAR SYSTEMS; CONTINUOUS TIME NONLINEAR SYSTEMS; HAMILTON JACOBI BELLMAN EQUATION; ON-LINE CONTROLS; OPTIMAL CONTROL PROBLEM; POLICY ITERATION;

APPROXIMATION ALGORITHMS; CONTINUOUS TIME SYSTEMS; CONTROL; DYNAMIC PROGRAMMING; ITERATIVE METHODS; NEURAL NETWORKS; NONLINEAR SYSTEMS; REINFORCEMENT LEARNING;

OPTIMAL CONTROL SYSTEMS;

EID: 84887472008     PISSN: 09410643     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00521-012-1249-y     Document Type: Article

References (24)

1. Pontryagin LS (1959) Optimal control processes. Uspehi Mat Nauk (in Russian) 14: 3-20.
2. Bellman RE (1957) Dynamic programming. Princeton University Press, New Jersey.
3. Lewis FL, Syrmos VL (1995) Optimal control. John Wiley, New York.
4. Kailath T (1973) Some new algorithms for recursive estimation in constant linear systems. IEEE Trans Inf Theory 19(6): 750-760.
5. Laub AJ (1979) A Schur method for solving algebraic Riccati equations. IEEE Trans Autom Control 24(6): 913-921.
6. Moris K, Navasca C (2006) Iterative solution of algebraic Riccati equations for damped systems. In: Proceedings of 45th IEEE conference on decision and control, San Diego, CA, pp 2436-2440.
7. Saridis GN, Lee CS (1979) An approximation theory of optimal control for trainable manipulators. IEEE Trans Syst Man Cybern 9(3): 152-159.
8. Beard R, Saridis G, Wen J (1997) Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation. Automatica 33(12): 2159-2177.
9. Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5): 779-791.
10. Murray JJ, Cox CJ, Lendaris GG, Saeks R (2002) Adaptive dynamic programming. IEEE Transa Syst Man Cybern C Appl Rev 32(2): 140-153.
11. Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. In: White DA, Sofge DA (eds) Handbook of intelligent control: neural, fuzzy, and adaptive approaches. van Nostrand Reinhold, New York, pp 493-525.
12. Wang FY, Zhang H, Liu D (2009) Adaptive dynamic programming: an introduction. IEEE Comput Intell Mag 4(2): 39-47.
13. Lewis FL, Vrabie D (2009) Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits Syst Mag 9(3): 32-50.
14. Al-Tamimi A, Lewis FL, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern B Cybern 38(4): 943-949.
15. Liu D, Wang D, Zhao D (2011) Neural-network-based optimal control for a class of nonlinear discrete-time systems with control constraints using the iterative GDHP algorithm. In: Proceedings of international joint conference on neural networks, San Jose, CA, pp 53-60.
16. Howard RA (1960) Dynamic programming and Markov processes. MIT Press, Cambridge.
17. Vrabie D, Lewis FL (2009) Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Netw 22(3): 237-246.
18. Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5): 878-888.
19. Lee JY, Park JB, Choi YH (2010) A novel generalized value iteration scheme for uncertain continuous-time linear systems. In: Proceedings of the 49th IEEE conference on decision and control, Atlanta, GA, pp 4637-4642.
20. Guo L, Cheng DZ, Feng DX (2005) Introduction to control theory: from basic concepts to research frontiers. Science Press (in Chinese), Beijing.
21. Rudin W (1976) Principles of mathematical analysis, 3rd edn. McGraw-Hill, New York.
22. Hornik K, Stinchcombe M, White H (1990) Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks. Neural Netw 3(5): 551-560.
23. Finlayson BA (1972) The method of weighted residuals and variational principles. Academic Press, New York.
24. Lewis FL, Vamvoudakis KG (2011) Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data. IEEE Trans Syst Man Cybern B Cybern 41(1): 14-25.