A three-network architecture for on-line learning and optimization based on adaptive dynamic programming

Neurocomputing

Volumn 78, Issue 1, 2012, Pages 3-13

  He, Haibo ^a   Ni, Zhen ^a   Fu, Jian ^b  

^a University of Rhode Island   (United States)

^b WUHAN UNIVERSITY OF TECHNOLOGY   (China)

Author keywords

Actor critic design; Adaptive dynamic programming; Goal representation; Multi state optimization; Online learning and control; Reinforcement learning; Three network architecture

Indexed keywords

ACTION NETWORK; ACTOR CRITIC; ADAPTIVE DYNAMIC PROGRAMMING; CONTROL PERFORMANCE; CRITIC NETWORK; DESIGN FRAMEWORKS; DETAILED DESIGN; EFFECTIVE LEARNING; GOAL REPRESENTATION; INVERTED PENDULUM; MULTI STATE; ONLINE LEARNING; REFERENCE NETWORK; REINFORCEMENT SIGNAL;

BENCHMARKING; DESIGN; DYNAMIC PROGRAMMING; E-LEARNING; LEARNING ALGORITHMS; OPTIMIZATION; REINFORCEMENT; REINFORCEMENT LEARNING;

NETWORK ARCHITECTURE;

ADAPTATION; ADAPTIVE DYNAMIC PROGRAMMING; ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; AUTOMATION; LEARNING; MACHINE LEARNING; ONLINE SYSTEM; PRIORITY JOURNAL; PROCESS DESIGN; PROCESS OPTIMIZATION; SIMULATION;

EID: 82655173881     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2011.05.031     Document Type: Article

References (31)

1. Werbos P.J. Intelligence in the brain: a theory of how it works and how to build it. Neural Netw. 2009, 200-212.
2. He H. Self-Adaptive Systems for Machine Intelligence 2011, Wiley.
3. Werbos P.J. Using ADP to understand and replicate brain intelligence: the next level design. IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning 2007, 209-216.
4. Si J., Barto A.G., Powell W.B., Wunsch D.C. Handbook of Learning and Approximate Dynamic Programming 2004, IEEE Press.
5. Prokhorov D.V., Wunsch D.C. Adaptive critic designs. IEEE Trans. Neural Netw. 1997, 8(5):997-1007.
6. White D.A., Sofge D.A. Handbook of Intelligent Control 1992, Van Nostrand, New York.
7. Powell W.B. Approximate Dynamic Programming: Solving the Curses of Dimensionality 2007, Wiley-Interscience.
8. Liu D., Jin N. Adaptive dynamic programming for discrete-time systems with infinite horizon and epsilon-error bound in the performance cost. Proceedings of the IEEE International Conference on Neural Networks 2009.
9. Wang F.Y., Zhang H., Liu D. Adaptive dynamic programming: an introduction. IEEE Comput. Intel. Mag. 2009, 4(2):39-47.
10. Vamvoudakis K., Lewis F.L. Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem. Proceedings of the IEEE International Conference on Neural Networks 2009.
11. Balakrishnan S.N., Ding J., Lewis F.L. Issues on stability of ADP feedback controllers for dynamical systems. IEEE Trans. Syst. Man Cybern., Part B 2008, 38(4):913-917. Special Issue on ADP/RL invited survey paper.
12. Al-Tamimi A., Abu-Khalaf M., Lewis F.L. Adaptive critic designs for discrete-time zero-sum games with application to h-infinity control. IEEE Trans. Syst. Man Cybern. Part B 2007, 37(1):240-247.
13. Venayagamoorthy G.K., Harley R.G. Handbook of learning and approximate dynamic programming. Application of Approximate Dynamic Programming in Power System Control 2004, 479-515. IEEE Press.
14. Ray S., Venayagamoorthy G.K., Chaudhuri B., Majumder R. Comparison of adaptive critics and classical approaches based wide area controllers for a power system. IEEE Trans. Syst. Man Cybern. Part B 2008, 38(4):1002-1007.
15. Zhang H.G., Luo Y.H., Liu D. Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans. Neural Netw. 2009, 20(9):1490-1503.
16. Wang F.Y., Jin N., Liu D., Wei Q. Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound. IEEE Trans. Neural Netw. 2011, 22(1):24-36.
17. Liu D., Zhang Y., Zhang H.G. A self-learning call admission control scheme for CDMA cellular networks. IEEE Trans. Neural Netw. 2005, 16(5):1219-1228.
18. He H., Fu J., Zhou X. Adaptive learning and control for MIMO system based on adaptive dynamic programming. IEEE Trans. Neural Netw. 2011, 22(7):1133-1148.
19. Bellman R.E. Dynamic Programming 1957, Princeton University Press, Princeton, NJ.
20. Werbos P.J. Backpropagation through time: what it does and how to do it. Proc/ IEEE 1990, vol. 78:1550-1560.
21. Werbos P.J. Backpropagation: basics and new developments. The Handbook of Brain Theory and Neural Networks 1995, 134-139. MIT Press, Cambridge, MA.
22. Ferrari S., Stengel R.F. Model-based adaptive critic designs. Handbook of Learning and Approximate Dynamic Programming 2004, IEEE Press.
23. Werbos P.J. Neuralcontrol and supervised learning. Handbook of Intelligent Control 1992, Van Nostrand, New York.
24. Si J., Wang Y.T. On-line learning control by association and reinforcement. IEEE Trans. Neural Netw. 2001, 12(2):264-276.
25. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction 1998, MIT Press, Cambridge, MA.
26. Werbos P.J. Applications of advances in nonlinear sensitivity analysis. System Modeling and Optimization 1981.
27. P.J. Werbos, Stable adaptive control using new critic designs," [online], available: 2008. http://arxiv.orgasadap-org/9810001.
28. Enns R., Si J. Helicopter flight control using direct neural dynamic programming. Handbook of Learning and Approximate Dynamic Programming 2004, 535-559. IEEE Press.
29. Si J., Liu D. Direct neural dynamic programming. Handbook of Learning and Approximate Dynamic Programming 2004, 125-151. IEEE Press.
30. Eltohamy K.D., Kuo C.-Y. Nonlinear optimal control of a triple link inverted pendulum with single control input. Int. J. Contr. 1998, 69(2):239-256.
31. Ni Z., He H., Prokhorov D.V., Fu J. An online actor-critic learning approach with Levenberg-Marquardt algorithm. Proceedings of the International Joint Conference on Neural Networks (IJCNN'11) 2011.