Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics

IEEE Transactions on Neural Networks and Learning Systems

Volumn 24, Issue 1, 2013, Pages 145-157

  Heydari, Ali ^a   Balakrishnan, Sivasubramanya N ^a  

^a Architectural and Environmental Engineering   (United States)

Author keywords

Approximate dynamic programming; Finite horizon optimal control; Fixed final time optimal control; Input constraint; Neural networks

Indexed keywords

APPROXIMATE DYNAMIC PROGRAMMING; CONTROL-AFFINE SYSTEMS; HAMILTON JACOBI BELLMAN EQUATION; INPUT-CONSTRAINT; NON-LINEAR OPTIMAL CONTROL; OPTIMAL CONTROLS; OPTIMAL FEEDBACK CONTROL; SINGLE NETWORK ADAPTIVE CRITICS;

ATTITUDE CONTROL; CONTROL; CONTROLLERS; NEURAL NETWORKS; OPTIMAL SYSTEMS; OPTIMIZATION; REINFORCEMENT LEARNING; SPACECRAFT;

DISCRETE TIME CONTROL SYSTEMS;

EID: 84880065287     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2012.2227339     Document Type: Article

References (33)

1. P. J. Werbos, "Neural networks for control and system identification," in Proc. IEEE Conf. Decision Control, vol. 1. Dec. 1989, pp. 260-265.
2. K. S. Narendra and K. Parthasarathy, "Identification and control of dynamical systems using neural networks," IEEE Trans. Neural Netw., vol. 1, no. 1, pp. 4-27, Mar. 1990. (Pubitemid 20689507)
3. P. J. Werbos, "Backpropagation through time: What it does and how to do it," Proc. IEEE, vol. 78, no. 10, pp. 1550-1560, Oct. 1990.
4. P. J. Werbos, "Approximate dynamic programming for real time control and neural modeling," in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, D. A. White and D. A. Sofge, Eds. New York: Van Nostrand Reinhold, 1992, pp. 493-525.
5. D. P. Bertsekas and J. N. Tsitsiklis, "Neuro-dynamic programming: An overview," in Proc. IEEE Conf. Decision Control, Dec. 1995, pp. 560-564.
6. F. L. Lewis, S. Jagannathan, and A. Yesildire, Neural Network Control of Robot Manipulators and Nonlinear Systems. London, U.K.: Taylor & Francis, 1999, pp. 173-411.
7. D. V. Prokhorov and D. C. Wunsch, "Adaptive critic designs," IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 997-1007, Sep. 1997.
8. X. Liu and S. N. Balakrishnan, "Convergence analysis of adaptive critic based optimal control," in Proc. Amer. Control Conf., 2000, pp. 1929-1933.
9. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 943-949, Aug. 2008.
10. T. Dierks and S. Jagannathan, "Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using time-based policy update," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1118-1129, Jul. 2012.
11. S. N. Balakrishnan and V. Biega, "Adaptive-critic-based neural networks for aircraft optimal control," J. Guid. Control Dyn., vol. 19, no. 4, pp. 893-898, Jul.-Aug. 1996. (Pubitemid 126539437)
12. S. Ferrari and R. F. Stengel, "Online adaptive critic flight control," J. Guid. Control Dyn., vol. 27, no. 5, pp. 777-786, Sep.-Oct. 2004.
13. G. K. Venayagamoorthy, R. G. Harley, and D. C. Wunsch, "Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol of a turbogenerator," IEEE Trans. Neural Netw., vol. 13, no. 3, pp. 764-773, May 2002.
14. R. Padhi, N. Unnikrishnan, X. Wang, and S. N. Balakrishnan, "A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems," Neural Netw., vol. 19, no. 10, pp. 1648-1660, Dec. 2006. (Pubitemid 44793175)
15. J. Ding and S. N. Balakrishnan, "Approximate dynamic programming solutions with a single network adaptive critic for a class of nonlinear systems," J. Control Theory Appl., vol. 9, no. 3, pp. 370-380, 2011.
16. D. Han and S. N. Balakrishnan, "State-constrained agile missile control with adaptive-critic-based neural networks," IEEE Trans. Control Syst. Technol., vol. 10, no. 4, pp. 481-489, Jul. 2002. (Pubitemid 34798672)
17. M. Fairbank, E. Alonso, and D. Prokhorov, "Simple and fast calculation of the second-order gradients for globalized dual heuristic dynamic programming in neural networks," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 10, pp. 1671-1676, Oct. 2012.
18. D. Vrabie, O. Pastravanu, F. Lewis, and M. Abu-Khalaf, "Adaptive optimal control for continuous-time linear systems based on policy iteration," Automatica, vol. 45, no. 2, pp. 477-484, Feb. 2009.
19. K. Vamvoudakis and F. Lewis, "Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem," Automatica, vol. 46, no. 5, pp. 878-888, May 2010.
20. T. Dierks and S. Jagannathan, "Optimal control of affine nonlinear continuous-time systems," in Proc. Amer. Control Conf., 2010, pp. 1568-1573.
21. T. Çimen and S. P. Banks, "Global optimal feedback control for general nonlinear systems with nonquadratic performance criteria," Syst. Control Lett., vol. 53, no. 5, pp. 327-346, Dec. 2004.
22. V. Costanza and P. S. Rivadeneira, "Finite-horizon dynamic optimization of nonlinear systems in real time," Automatica, vol. 44, no. 9, pp. 2427-2434, Sep. 2008.
23. T. Cheng, F. L. Lewis, and M. Abu-Khalaf, "Fixed-final- timeconstrained optimal control of nonlinear systems using neural network HJB approach," IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1725-1737, Nov. 2007. (Pubitemid 350148423)
24. D. M. Adhyaru, I. N. Kar, and M. Gopal, "Fixed final time optimal control approach for bounded robust controller design using Hamilton-Jacobi- Bellman solution," IET Control Theory Appl., vol. 3, no. 9, pp. 1183-1195, Sep. 2009.
25. F.-Y. Wang, N. Jin, D. Liu, and Q. Wei, "Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound," IEEE Trans. Neural Netw., vol. 22, no. 1, pp. 24-36, Jan. 2011.
26. Q. Wei and D. Liu, "An iterative ε-optimal control scheme for a class of discrete-time nonlinear systems with unfixed initial state," Neural Netw., vol. 32, pp. 236-244, Aug. 2012.
27. D. Wang, D. Liu, and Q. Wei, "Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach," Neurocomputing, vol. 78, no. 1, pp. 14-22, Feb. 2012.
28. R. Song and H. Zhang, "The finite-horizon optimal control for a class of time-delay affine nonlinear system," Neural Comput. Appl., pp. 1-7, Jul. 2011.
29. S. E. Lyshevski, "Optimal control of nonlinear continuous-time systems: Design of bounded controllers via generalized nonquadratic functionals," in Proc. Amer. Control Conf., 1998, pp. 205-209.
30. D. E. Kirk, Optimal Control Theory: An Introduction. New York: Dover, 2004, pp. 54-75.
31. J. R. Wertz, Spacecraft Attitude Determination and Control. Amsterdam, The Netherlands: Kluwer, 1978, pp. 521-570.
32. P. H. Zipfel, "Modeling and simulation of aerospace vehicle dynamics," in Proc. Reston, 2000, pp. 119-121.
33. R. Beard, "Improving the closed-loop performance of nonlinear systems," Ph.D. thesis, Dept. Electr. Eng., Rensselaer Polytechnic Inst., Troy, NY, 1995.