A novel iterative -adaptive dynamic programming for discrete-time nonlinear systems

IEEE Transactions on Automation Science and Engineering

Volumn 11, Issue 4, 2014, Pages 1176-1190

  Wei, Qinglai ^a   Liu, Derong ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

Adaptive critic designs; Adaptive dynamic programming; Approximate dynamic programming; Neural networks; Neuro dynamic programming; Nonlinear systems; Optima

Indexed keywords

NONLINEAR SYSTEMS;

ADAPTIVE CRITIC DESIGNS; ADAPTIVE DYNAMIC PROGRAMMING; APPROXIMATE DYNAMIC PROGRAMMING; NEURO DYNAMIC PROGRAMMING; OPTIMA;

NEURAL NETWORKS;

EID: 84908658175     PISSN: 15455955     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASE.2013.2280974     Document Type: Article

References (41)

1. Q. Kang, M. Zhou, J. An, and Q. Wu, "Swarm intelligence approaches to optimal power flow problem with distributed generator failures in power networks," IEEE Trans. Autom. Sci. Eng., vol. 10, no. 2, pp. 343-353, Apr. 2013.
2. O. Wigstrom, B. Lennartson, A. Vergnano, and C. Breitholtz, "Highlevel scheduling of energy optimal trajectories," IEEE Trans. Autom. Sci. Eng., vol. 10, no. 1, pp. 57-64, Jan. 2013.
3. U. Wikborg and T.-E. Lee, "Noncyclic scheduling for timed discrete- event systems with application to single-armed cluster tools using pareto-optimal optimization," IEEE Trans. Autom. Sci. Eng., vol. 10, no. 3, pp. 699-710, Jul. 2013.
4. Z. Xu, Q. S. Jia, X. Guan, and J. Shen, "Smart management of multiple energy systems in automotive painting shop," IEEE Trans. Autom. Sci. Eng., vol. 10, no. 3, pp. 603-614, Jul. 2013.
5. R. E. Bellman, Dynamic Programming. Princeton, NJ, USA: Princeton Univ. Press, 1957.
6. C. Chu, F. Chu, M. Zhou, H. Chen, and Q. Shen, "A polynomial dynamic programming algorithm for crude oil transportation planning," IEEE Trans. Autom. Sci. Eng., vol. 9, no. 1, pp. 42-55, Jan. 2012.
7. W. S. Lin and J.W. Sheu, "Optimization of train regulation and energy usage of metro lines using an adaptive-optimal-control algorithm," IEEE Trans. Autom. Sci. Eng., vol. 8, no. 4, pp. 121-131, Oct. 2011.
8. P. J. Werbos, "Advanced forecasting methods for global crisis warning and models of intelligence," General Syst. Yearbook, vol. 22, pp. 25-38, 1977.
9. P. J. Werbos, "A menu of designs for reinforcement learning over time," in Neural Networks for Control, W. T. Miller, R. S. Sutton, and P. J. Werbos, Eds. Cambridge, MA, USA: MIT Press, 1991, pp. 67-95.
10. A. G. Banerjee, A. Pomerance, W. Losert, and S. K. Gupta, "Developing a stochastic dynamic programming framework for optical tweezer-based automated particle transport operations," IEEE Trans. Autom. Sci. Eng., vol. 7, no. 2, pp. 218-227, Apr. 2010.
11. A. Heydari and S. N. Balakrishnan, "Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics," IEEE Trans. Neural Netw. Learning Syst., vol. 24, no. 1, pp. 145-157, Jan. 2013.
12. D. Liu, Y. Zhang, andH. Zhang, "Aself-learning call admission control scheme for CDMA cellular networks," IEEE Trans. Neural Netw., vol. 16, no. 5, pp. 855-864, Sep. 2011.
13. P. B. Luh, C. T. Wilkie, S. C. Chang, K. L. Marsh, and N. Olderman, "Modeling and optimization of building emergency evacuation considering blocking effects on crowd movement," IEEE Trans. Autom. Sci. Eng., vol. 9, no. 4, pp. 687-700, Sep. 2012.
14. D. Wang, D. Liu, Q. Wei, D. Zhao, and N. Jin, "Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming," Autom., vol. 48, no. 8, pp. 1825-1832, Aug. 2012.
15. P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling," in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches,D. A. White andD. A. Sofge, Eds. New York, NY, USA: Van Nostrand Reinhold, 1992, ch. 13.
16. D. V. Prokhorov and D. C. Wunsch, "Adaptive critic designs," IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 997-1007, Sep. 1997.
17. J. Ni, M. Liu, L. Ren, and S. X. Yang, "A Multiagent -learningbased optimal allocation approach for urban water resource management system," IEEE Trans. Autom. Sci. Eng., 2013, to be published.
18. J. Liang, G. K. Venayagamoorthy, and R. G. Harley, "Wide-area measurement based dynamic stochastic optimal power flow control for smart grids with high variability and uncertainty," IEEE Trans. Smart Grid, vol. 3, no. 1, pp. 59-69, Mar. 2012.
19. J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, "Adaptive dynamic programming," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 32, no. 2, pp. 140-153, May 2002.
20. S. K. Pradhan and B. Subudhi, "Real-time adaptive control of a flexible manipulator using reinforcement learning," IEEE Trans. Autom. Sci. Eng., vol. 9, no. 2, pp. 237-249, Apr. 2012.
21. F. Wang, H. Zhang, and D. Liu, "Adaptive dynamic programming: An introduction," IEEE Comput. Intell.Mag., vol. 4, no. 2, pp. 39-47, Mar. 2009.
22. Q. Wei, H. Zhang, and J. Dai, "Model-free multiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index functions," Neurocomput., vol. 72, no. 7-9, pp. 1839-1848, Mar. 2009.
23. Q. Wei and D. Liu, "An iterative -optimal control scheme for a class of discrete-time nonlinear systems with unfixed initial state," Neural Netw., vol. 32, pp. 236-244, 2012.
24. H. Zhang, Y. Luo, and D. Liu, "The RBF neural network-based nearoptimal control for a class of discrete-time affine nonlinear systems with control constraint," IEEE Trans. Neural Netw., vol. 20, no. 9, pp. 1490-1503, Sep. 2009.
25. H. Zhang, Q. Wei, and D. Liu, "An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games," Autom., vol. 47, no. 1, pp. 207-214, Jan. 2011.
26. H. Zhang, Q. Wei, and Y. Luo, "A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 937-942, Jul. 2008.
27. F. L. Lewis and D. Vrabie, "Reinforcement learning and adaptive dynamic programming for feedback control," IEEE Circuits Syst. Mag., vol. 9, no. 3, pp. 32-50, May 2009.
28. M. Abu-Khalaf and F. L. Lewis, "Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach," Autom., vol. 41, no. 5, pp. 779-791, May 2005.
29. F. Wang, N. Jin, D. Liu, and Q. Wei, "Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with -error bound," IEEE Trans. Neural Netw., vol. 22, no. 1, pp. 24-36, Jan. 2011.
30. R. Beard, "Improving the closed-loop performance of nonlinear systems," Ph.D. dissertation, Rensselaer Polytechnic Inst., Troy, NY, USA, 1995.
31. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 943-949, Aug. 2008.
32. D. Liu, D. Wang, D. Zhao, Q. Wei, and N. Jin, "Neural-network-based optimal control for a class of unknown discrete-time nonlinear systems using globalized dual heuristic programming," IEEE Trans. Autom. Sci. Eng., vol. 9, no. 3, pp. 628-634, Jul. 2012.
33. L. B. Koppel, Introduction to Control Theory With Applications to Process Contol. Upper Saddle River, NJ, USA: Prentice-Hall Inc., 1968.
34. X. Liao, L. Wang, and P. Yu, Stability of Dynamical Systems. Amsterdam, The Netherlands: Elsevier Press, 2007.
35. M. Abu-Khalaf, F. L. Lewis, and J. Huang, "Neurodynamic programming and zero-sum games for constrained control systems," IEEE Trans. Neural Netw., vol. 19, no. 7, pp. 1243-1252, Apr. 2008.
36. J. Si and Y. T. Wang, "On-line learning control by association and reinforcement," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 264-276, Mar. 2001.
37. Q. Yang and S. Jagannathan, "Reinforcement learning controller design for affine nonlinear discrete-time systems using online approximators," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 2, pp. 377-390, Apr. 2012.
38. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. Upper Saddle River, NJ, USA: Prentice-Hall PTR, 1999.
39. M. T. Hagan, H. B. Demuth, and M. H. Beale, Neural Network Design. Boston, MA, USA: PWS Publishing Company, 1996.
40. E. M. Navarro-Lopez, "Local feedback passivation of nonlinear discrete- time systems through the speed-gradient algorithm," Autom., vol. 43, no. 7, pp. 1302-1306, Jul. 2007.
41. H. Sira-Ramirez, "Non-linear discrete variable structure systems in quasi-sliding mode," Int. J. Control, vol. 54, no. 5, pp. 1171-1187, 1991.