Reinforcement learning control based on multi-goal representation using hierarchical heuristic dynamic programming

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2012, Pages

  Ni, Zhen ^a   He, Haibo ^a   Zhao, Dongbin ^b   Prokhorov, Danil V ^c  

^a University of Rhode Island   (United States)

^b INSTITUTE OF AUTOMATION   (China)

^c America   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE CRITIC DESIGNS; COMPARATIVE STUDIES; CONTROL SYSTEM PERFORMANCE; CRITIC NETWORK; HEURISTIC DYNAMIC PROGRAMMING; HIERARCHICAL STRUCTURES; REINFORCEMENT LEARNING CONTROL;

NEURAL NETWORKS;

REINFORCEMENT LEARNING;

EID: 84865079504     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2012.6252524     Document Type: Conference Paper

References (25)

1. P. J. Werbos, "Using ADP to understand and replicate brain intelligence: the next level design," in IEEE Int. Symposium on Approximate Dynamic Programming and Reinforcement Learning, pp. 209-216, 2007.
2. P. J. Werbos, "Intelligence in the brain: A theory of how it works and how to build it," Neural Networks, pp. 200-212, 2009.
3. J. Si, A. G. Barto, W. B. Powell, and D. C. Wunsch, eds., Handbook of Learning and Approximate Dynamic Programming. IEEE Press, 2004.
4. W. B. Powell, Approximate Dynamic Programming: Solving the Curses of Dimensionality. Wiley-Interscience, 2007.
5. F. Y. Wang, N. Jin, D. Liu, and Q. Wei, "Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with "-error bound," IEEE Transactions on Neural Networks, vol. 22, no. 1, pp. 24-36, 2011.
6. J. Fu, H. He, and X. Zhou, "Adaptive learning and control for mimo system based on adaptive dynamic programming," IEEE Transactions on Neural Networks, vol. 22, no. 7, pp. 1133-1148, 2011.
7. W. Qiao, G. Venayagamoorthy, and R. Harley, "DHP-based wide-area coordinating control of a power system with a large wind farm and multiple FACTS devices," in Proc. IEEE Int. Conf. Neural Netw., pp. 2093-2098, 2007.
8. S. Ray, G. K. Venayagamoorthy, B. Chaudhuri, and R. Majumder, "Comparison of adaptive critics and classical approaches based wide area controllers for a power system," IEEE Trans. on Syst. Man, Cybern., Part B, vol. 38, no. 4, pp. 1002-1007, 2008.
9. H. He, Z. Ni, and J. Fu, "A three-network architecture for on-line learning and optimization based on adaptive dynamic programming," Neurocomputing, vol. 78, no. 1, pp. 3-13, 2012.
10. F. Y. Wang, H. Zhang, and D. Liu, "Adaptive dynamic programming: An introduction," IEEE Comput. Intel. Mag., vol. 4, no. 2, pp. 39-47, 2009.
11. F. L. Lewis, and D. Vrabie., "Reinforcement learning and adaptive dynamic programming for feedback control," IEEE Circuits Sys. Mag., vol. 9, no. 3, pp. 32-50, 2009.
12. D. V. Prokhorov and D. C. Wunsch, "Adaptive critic designs," IEEE Trans. on Neural Netw., vol. 8, no. 5, pp. 997-1007, 1997.
13. P. H. Eaton, and D. V. Prokhorov, and D. C. Wunsch II., "Neurocontroller alternatives for fuzzy ball-and-beam systems with nonuniform nonlinear friction," IEEE Trans. Neural Netw., vol. 11, no. 2, pp. 423-435, 2000.
14. J. Si and Y. T. Wang, "On-line learning control by association and reinforcement," IEEE Trans. on Neural Netw., vol. 12, no. 2, pp. 264-276, 2001.
15. J. Si, L. Yang, and D. Liu, Handbook of Learning and Approximate Dynamic Programming, ch. Direct Neural Dynamic Programming, pp. 125-151. IEEE Press, 2004.
16. L. Yang, J. Si, K. S. Tsakalis, and A. A. Rodriguez, "Direct heuristic dynamic programming for nonlinear tracking conrol with filtered tracking error," IEEE Transactions on Systems Man and Cybernetics Part B-Cybernetics, vol. 39, no. 6, pp. 1617-1622, 2009.
17. H. G. Zhang, Q. L. Wei, and Y. H. Luo, "A novel infinite-time optimaltracking control scheme for a class of discrete-time nonlinear systems via the greedy hdp iteration algorithm," IEEE Transactions on System, Man and Cybernetics, Part B, vol. 38, no. 4, pp. 937-942, 2008.
18. H. G. Zhang, Y. H. Luo, and D. Liu, "Neural-network-based nearoptimal control for a class of discrete-time affine nonlinear systems with control constraints," IEEE Transactions on Neural Networks, vol. 20, no. 9, pp. 1490-1503, 2009.
19. H. G. Zhang, Q. L. Wei, and D. Liu, "An iterative approximate dynamic programming method to solve for a class of nonlinear zero-sum differential games," Automatica, vol. 47, no. 1, pp. 207-214, 2011.
20. G. K. Venayagamoorthy, R. G. Harley, and D. C. Wunsch, "Dual heuristic programming excitation neurocontrol for generators in a multimachine power system," IEEE Trans. on Industry Applications, vol. 39, no. 2, pp. 382-394, 2003.
21. G. K. Venayagamoorthy and R. G. Harley, Handbook of Learning and Approximate Dynamic Programming, ch. Application of Approximate Dynamic Programming in Power System Control, pp. 479-515. IEEE Press, 2004.
22. D. V. Prokhorov, R. A. Santiago, and D. C. Wunsch, "Adaptive critic designs: A case study for neurocontrol," Neural Networks Letter, vol. 8, no. 9, pp. 1367-1372, 1995.
23. D. V. Prokhorov, "Adaptive critic designs and their applications," Ph.D. dissertation, Texas Tech. Univ., Lubbock, 1997.
24. H. He and B. Liu, "A hierarchical learning architecture with multiplegoal representations based on adaptive dynamic programming," in IEEE International Conference on Networking, Sensing and Control (ICNSC'2010), 2010.
25. T. L. Chien, C. C. Chen, Y. C. Huang, and W.-J. Lin, "Stability and almost disturbance decoupling analysis of nonlinear system subject to feedback linearization and feedforward neural network controller," IEEE Trans. Neural Netw., vol. 19, no. 7, pp. 1220-1230, 2008.