Adaptive dynamic programming for optimal control of unknown nonlinear discrete-time systems

IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning

Volumn , Issue , 2011, Pages 242-249

  Liu, Derong ^a   Wang, Ding ^a   Zhao, Dongbin ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

Adaptive critic designs; adaptive dynamic programming; approximate dynamic programming; globalized dual heuristic programming; intelligent control; neural dynamic programming; neural networks; optimal control

Indexed keywords

ADAPTIVE CRITIC DESIGNS; ADAPTIVE DYNAMIC PROGRAMMING; APPROXIMATE DYNAMIC PROGRAMMING; DUAL HEURISTIC PROGRAMMING; NEURAL DYNAMIC PROGRAMMING; OPTIMAL CONTROLS;

ADAPTIVE CONTROL SYSTEMS; ALGORITHMS; CELLULAR RADIO SYSTEMS; CONVERGENCE OF NUMERICAL METHODS; COST FUNCTIONS; DIGITAL CONTROL SYSTEMS; DISCRETE TIME CONTROL SYSTEMS; HEURISTIC PROGRAMMING; INTELLIGENT CONTROL; NEURAL NETWORKS; OPTIMIZATION; REINFORCEMENT LEARNING;

DYNAMIC PROGRAMMING;

EID: 80052212355     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ADPRL.2011.5967357     Document Type: Conference Paper

References (36)

1. R. E. Bellman, Dynamic Programming. Princeton, NJ: Princeton University Press, 1957.
2. J. Sarangapani, Neural Network Control of Nonlinear Discrete-time Systems. Boca Raton, FL: CRC Press, 2006.
3. W. Yu, Recent Advances in Intelligent Control Systems. London: Springer-Verlag, 2009.
4. P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling," in Handbook of Intelligent Control, D. A. White and D. A. Sofge, Eds. New York: Van Nostrand Reinhold, 1992, ch. 13.
5. P. J. Werbos, "ADP: The key direction for future research in intelligent control and understanding brain intelligence," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 898-900, Aug. 2008.
6. P. J. Werbos, "Intelligence in the brain: a theory of how it works and how to build it," Neural Networks, vol. 22, no. 3, pp. 200-212, Apr. 2009.
7. c. Watkins and P. Dayan, "Q-learning," Machine Learning, vol. 8, pp. 279-292, 1992.
8. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
9. D. V. Prokhorov and D. C. Wunsch, "Adaptive critic designs," IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 997-1007, Sept. 1997. (Pubitemid 127763331)
10. J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, "Adaptive dynamic programming," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 32, no. 2, pp. 140-153, May 2002. (Pubitemid 35289398)
11. J. Si and Y. T. Wang, "On-line learning control by association and reinforcement," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 264-276, Mar. 2001. (Pubitemid 32371483)
12. J. Si, A. G. Barto, W. B. Powell, and D. C. Wunsch, Eds., Handbook of Learning and Approximate Dynamic Programming. New York: IEEE Press/Wiley, 2004.
13. F. Y. Wang, H. Zhang, and D. Liu, "Adaptive dynamic programming: an introduction," IEEE CompUlationallntelligence Magazine, vol. 4, no. 2, pp. 39-47, May 2009.
14. F. L. Lewis and D. Vrabie, "Reinforcement learning and adaptive dynamic programming for feedback control," IEEE Circuits and Systems Magazine, vol. 9, no. 3, pp. 32-50, July 2009.
15. D. Liu, X. Xiong, and Y. Zhang, "Action-dependent adaptive critic designs," in Proc. International Joint Conference on Neural Networks, Washington, DC, July 2001, vol. 2, pp. 990-995. (Pubitemid 32805299)
16. D. Liu and N. Jin, "E:-adaptive dynamic programming for discrete-time systems," in Proc. International Joint Conference on Neural Networks, Hong Kong, June 2008, pp. 1417-1424.
17. G. K. Venayagamoorthy, D. C. Wunsch, and R. G. Harley, "Adaptive critic based neurocontroller for turbogenerators with global dual heuristic programming," in Proc. IEEE PES Winter Meet., Singapore, Jan. 2000, vol. 1, pp. 291-294.
18. G. K. Venayagamoorthy, R. G. Harley, and D. C. Wunsch, "Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol of a turbogenerator," IEEE Trans. Neural Netw., vol. 13, no. 3, pp. 764-773, May 2002. (Pubitemid 34669664)
19. J. W. Park, R. G. Harley, and G. K. Venayagamoorthy, " Adaptivecritic- based optimal neurocontrol for synchronous generators in a power system using MLP/RBF neural networks," IEEE Trans. Ind. Appl., vol. 39, no. 5, pp. 1529-1540, Sept./Oct. 2003.
20. G. G. Yen and P. G. DeLima, "Improving the performance of globalized dual heuristic programming for fault tolerant control through an online learning supervisor," IEEE Trans. Automation Science and Engineering, vol. 2, no. 2, pp. 121-131, Apr. 2005.
21. M. Abu-Khalaf and F. L. Lewis. "Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach," Automatica, vol. 41, no. 5,779-791, May 2005. (Pubitemid 40352391)
22. T. Cheng, F. L. Lewis, and M. Abu-Khalaf, "A neural network solution for fixed-final time optimal control of nonlinear systems," Automatica, vol. 43, no. 3, pp. 482-490, Mar. 2007. (Pubitemid 46209051)
23. S. N. Balakrishnan and Y. Biega, "Adaptive-critic based neural networks for aircraft optimal control," Journal of Guidance, Control, and Dynamics, vol. 19, no. 4, pp. 893-898, July-Aug. 1996. (Pubitemid 126539437)
24. R. Padhi, S. N. Balakrishnan, and T. Randolph, "Adaptive-critic based optimal neuro control synthesis for distributed parameter systems," Automatica, vol. 37, no. 8, pp. 1223-1234, Aug. 2001. (Pubitemid 32610253)
25. D. Han and S. N. Balakrishnan, "State-constrained agile missile control with adaptive critic-based neural networks," IEEE Trans. Control Systems Technology, vol. 10, no. 4, pp. 481-489, July 2002. (Pubitemid 34798672)
26. R. Padhi, N. Unnikrishnan, X. Wang, and S. N. Balakrishnan, "A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems," Neural Networks, vol. 19, no. 10, pp. 1648-1660, Dec. 2006. (Pubitemid 44793175)
27. S. N. Balakrishnan, J. Ding, and F. L. Lewis, "Issues on stability of ADP feedback controllers for dynamic systems," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 913-917, Aug. 2008.
28. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 943-949, Aug. 2008.
29. H. Zhang, Q. Wei, and Y. Luo, "A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 937-942, Aug. 2008.
30. H. Zhang, Y. Luo, and D. Liu, "Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints," IEEE Trans. Neural Netw., vol. 20, no. 9, pp. 1490-1503, Sept. 2009.
31. D. Vrabie and F. Lewis, "Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems" Neural Networks, vol. 22, no. 3, pp. 237-246, Apr. 2009.
32. T. Dierks, B. T. Thumati, and J. Sarangapani, "Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence," Neural Networks, vol. 22, no. 5-6, pp. 851-860, July-Aug. 2009.
33. K. G. Vamvoudakis and F. L. Lewis, "Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem," Automatica, vol. 46, no. 5, pp. 878-888, May 2010.
34. ∞ control design for unknown linear discrete-time systems via Q-learning with LMI," Automatica, vol. 46, no. 8, pp. 1320-1326, Aug. 2010.
35. T. Dierks and J. Sarangapani, "Optimal tracking control of affine non liner discrete-time systems with unknown internal dynamics", in Joint 48th IEEE Conference on Decision and Control and 28th Chinese Control Conference, Shanghai, P. R. China, Dec. 2009, pp. 6750-6755.
36. J. Sarangapani and P. He, "Neural-network-based state feedback control of a nonlinear discrete-time system in nonstrict feedback form," IEEE Trans. Neural Netw., vol. 19, no. 12, pp. 2073-2087, Dec. 2008.