Finite-approximation-error-based optimal control approach for discrete-time nonlinear systems

IEEE Transactions on Cybernetics

Volumn 43, Issue 2, 2013, Pages 779-789

  Liu, Derong ^a   Wei, Qinglai ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

Adaptive dynamic programming (ADP); Approximate dynamic programming; Finite approximation errors; Neural networks; Optimal control

Indexed keywords

ADAPTIVE DYNAMIC PROGRAMMING; APPROXIMATE DYNAMIC PROGRAMMING; APPROXIMATION ERRORS; CONVERGENCE CONDITIONS; DISCRETE-TIME NONLINEAR SYSTEMS; OPTIMAL CONTROL POLICY; OPTIMAL CONTROL PROBLEM; OPTIMAL CONTROLS;

APPROXIMATION ALGORITHMS; CONTROL; CONTROL THEORY; DYNAMIC PROGRAMMING; ERRORS; NEURAL NETWORKS; NONLINEAR SYSTEMS; OPTIMAL CONTROL SYSTEMS;

ITERATIVE METHODS;

EID: 84881555023     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2012.2216523     Document Type: Article

References (37)

1. M. Abu-Khalaf and F. L. Lewis, "Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach," Automatica, vol. 41, no. 5, pp. 779-791, May 2005.
2. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 943-949, Aug. 2008.
3. S. N. Balakrishnan and V. Biega, "Adaptive-critic-based neural networks for aircraft optimal control," J. Guid., Control, Dyn., vol. 19, no. 4, pp. 893-898, Jul./Aug. 1996.
4. R. Beard, "Improving the closed-loop performance of nonlinear systems," Ph.D. dissertation, Rensselaer Polytechnic Inst., Troy, NY, 1995.
5. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Cambridge, MA: Athena Scientific, 1996.
6. T. Cheng, F. L. Lewis, and M. Abu-Khalaf, "A neural network solution for fixed-final time optimal control of nonlinear systems," Automatica, vol. 43, no. 3, pp. 482-490, Mar. 2007.
7. R. Enns and J. Si, "Helicopter trimming and tracking control using direct neural dynamic programming," IEEE Trans. Neural Netw., vol. 14, no. 4, pp. 929-939, Aug. 2003.
8. S. Ferrari, J. E. Steck, and R. Chandramohan, "Adaptive feedback control by constrained approximate dynamic programming," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 982-987, Aug. 2008.
9. M. T. Hagan, H. B. Demuth, and M. H. Beale, Neural Network Design. Boston, MA: PWS-Kent, 1996.
10. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. Upper Saddle River, NJ: Prentice-Hall, 1999.
11. N. Jin, D. Liu, T. Huang, and Z. Pang, "Discrete-time adaptive dynamic programming using wavelet basis function neural networks," in Proc. IEEE Symp. Approx. Dyn. Programm. Reinforcement Learn., Honolulu, HI, USA, Apr. 2007, pp. 135-142.
12. R. V. Kulkarni and G. K. Venayagamoorthy, "Bio-inspired algorithms for autonomous deployment and localization of sensor nodes," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 40, no. 6, pp. 663-675, Nov. 2010.
13. G. G. Lendaris, "Higher level application of ADP: A next phase for the control field?" IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 901-912, Aug. 2008.
14. F. L. Lewis and V. G. Kyriakos, "Reinforcement learning for partially observable dynamic processes: Adaptive dynamic programming using measured output data," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 1, pp. 14-25, Jan. 2011.
15. F. L. Lewis and D. Vrabie, "Reinforcement learning and adaptive dynamic programming for feedback control," IEEE Circuits Syst. Mag., vol. 9, no. 3, pp. 32-50, Third Quarter, 2009.
16. B. Lincoln and A. Rantzer, "Relaxing dynamic programming," IEEE Trans. Autom. Control, vol. 51, no. 8, pp. 1249-1260, Aug. 2006.
17. D. Liu, H. Javaherian, O. Kovalenko, and T. Huang, "Adaptive critic learning techniques for engine torque and air-fuel ratio control," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 988-993, Aug. 2008.
18. D. Liu, Y. Zhang, and H. Zhang, "A self-learning call admission control scheme for CDMA cellular networks," IEEE Trans. Neural Netw., vol. 16, no. 5, pp. 1219-1228, Sep. 2005.
19. J. Mao and C. G. Cassandras, "Optimal control of multi-stage discrete event systems with real-time constraints," IEEE Trans. Autom. Control, vol. 54, no. 1, pp. 108-123, Jan. 2009.
20. J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, "Adaptive dynamic programming," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 32, no. 2, pp. 140-153, May 2002.
21. E. M. Navarro-Lopez, "Local feedback passivation of nonlinear discrete-time systems through the speed-gradient algorithm," Automatica, vol. 43, no. 7, pp. 1302-1306, Jul. 2007.
22. D. V. Prokhorov and D. C. Wunsch, "Adaptive critic designs," IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 997-1007, Sep. 1997.
23. J. Seiffertt, S. Sanyal, and D. C. Wunsch, "Hamilton-Jacobi-Bellman equations and approximate dynamic programming on time scales," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 918-923, Aug. 2008.
24. J. Si and Y.-T. Wang, "On-line learning control by association and reinforcement," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 264-276, Mar. 2001.
25. H. Sira-Ramirez, "Non-linear discrete variable structure systems in quasisliding mode," Int. J. Control, vol. 54, no. 5, pp. 1171-1187, May 1991.
26. K. G. Vamvoudakis and F. L. Lewis, "Multi-player non-zero-sum games: Online adaptive learning solution of coupled Hamilton-Jacobi equations," Automatica, vol. 47, no. 8, pp. 1556-1569, Aug. 2011.
27. F. Wang, N. Jin, D. Liu, and Q. Wei, "Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ∈-error bound," IEEE Trans. Neural Netw., vol. 22, no. 1, pp. 24-36, Jan. 2011.
28. F. Wang, H. Zhang, and D. Liu, "Adaptive dynamic programming: An introduction," IEEE Comput. Intell. Mag., vol. 4, no. 2, pp. 39-47, May 2009.
29. C. Watkins, "Learning from delayed rewards," Ph.D. dissertation, Cambridge Univ., Cambridge, U.K., 1989.
30. Q. Wei and D. Liu, "Adaptive dynamic programming with stable value iteration algorithm for discrete-time nonlinear systems," in Proc. Int. Joint Conf. Neural Netw., Brisbane, Australia, Jun. 2012, pp. 1-6.
31. Q. Wei and D. Liu, "A novel optimal control scheme for discrete-time nonlinear systems using iterative adaptive dynamic programming," IEEE Trans. Autom. Sci. Eng., submitted for publication.
32. Q. Wei, H. Zhang, and J. Dai, "Model-free multiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index functions," Neurocomputing, vol. 72, no. 7-9, pp. 1839-1848, Mar. 2009.
33. P. J. Werbos, "A menu of designs for reinforcement learning over time," in Neural Networks for Control, W. T. Miller, R. S. Sutton, and P. J. Werbos, Eds. Cambridge, MA: MIT Press, 1991, pp. 67-95.
34. P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling," in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, D. A. White and D. A. Sofge, Eds. NewYork: Van Nostrand, 1992, ch. 13.
35. Q. Yang, J. B. Vance, and S. Jagannathan, "Control of nonaffine nonlinear discrete-time systems using reinforcement-learning-based linearly parameterized neural networks," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 994-1001, Aug. 2008.
36. H. Zhang, Q. Wei, and Y. Luo, "A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 937-942, Aug. 2008.
37. H. Zhang, Q. Wei, and D. Liu, "An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games," Automatica, vol. 47, no. 1, pp. 207-214, Jan. 2011.