Approximate dynamic programming for self-learning control

Zidonghua Xuebao/Acta Automatica Sinica

Volumn 31, Issue 1, 2005, Pages 13-18

  Liu, Derong ^a  

^a 1120 Science and Engineering Offices   (United States)

Author keywords

Approximate dynamic programming; Learning control; Neural networks

Indexed keywords

COMPUTATIONAL COMPLEXITY; DYNAMIC PROGRAMMING; LEARNING SYSTEMS; NEURAL NETWORKS; OPTIMIZATION;

APPROXIMATE DYNAMIC PROGRAMMING; LEARNING CONTROL;

NONLINEAR SYSTEMS;

EID: 15044349684     PISSN: 02544156     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (36)

1. Bellman R E. Dynamic Programming, Princeton, NJ: Princeton University Press, 1957
2. Dreyfus S E, Law A M. The Art and Theory of Dynamic Programming, New York, NY: Academic Press, 1977
3. Lewis F L, Syrmos V L. Optimal Control, New York, NY: John Wiley, 1995
4. Balakrishnan S N, Biega V. Adaptive-critic-based neural networks for aircraft optimal control, Journal of Guidance, Control, Dynamics, 1996, 19(7-8): 893-898
5. Prokhorov D V, Wunsch D C. Adaptive critic designs, IEEE Transactions on Neural Networks, 1997, 8(9): 997-1007
6. Si J, Wang Y-T. On-line learning control by association and reinforcement, IEEE Transactions on Neural Networks, 2001, 12(3): 264-276
7. Werbos P J. Building and understanding adaptive systems: A statistical/numerical approach to factory automation and brain research, IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-17, 1987, 7-20
8. Werbos P J. A menu of designs for reinforcement learning over time, In: Neural Networks for Control (Chapter 3), Edited by W. T. Miller, R. S. Sutton, and P. J. Werbos, Cambridge, MA: The MIT Press, 1990
9. Werbos P J. Approximate dynamic programming for real-time control and neural modeling, In: Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches (Chapter 13), Edited by D. A. White and D. A. Sofge, New York, NY: Van Nostrand Reinhold, 1992
10. Werbos P J. Advanced forecasting methods for global crisis warning and models of intelligence, General Systems Yearbook, 1977, 22: 25-38
11. Bertsekas D P, Tsitsiklis J N. Neuro-Dynamic Programming, Belmont, MA: Athena Scientific, 1996
12. Cox C, Stepniewski S, Jorgensen C, Saeks R, Lewis C. On the design of a neural network autolander, International Journal of Robust Nonlinear Control, 1999, 9: 1071-1096
13. Dalton J, Balakrishnan S N. A neighboring optimal adaptive critic for missile guidance, Mathematical and Computer Modeling, 1996, 23(1): 175-188
14. Javaherian H, Liu D, Zhang Y, Kovalenko O. Adaptive critic learning techniques for automotive engine control, In: Proceedings of the American Control Conference, Boston, MA, 2004, 6: 4066-4071
15. Kulkarni N V, KrishnaKumar K. Intelligent engine control using an adaptive critic, IEEE Transactions on Control Systems Technology, 2003, 11(3): 164-173
16. Lendaris G G, Paintz C. Training strategies for critic and action neural networks in dual heuristic progrmming method, In: Proceedings of the 1997 IEEE International Conference on Neural Networks, Houston, TX, 1997, 6: 712-717
17. Liu D, Patino. A self-learning ship steering controller based on adaptive critic designs, In: Proceedings of the IFAC Triennial World Congress, Beijing, China, 1999, J: 367-372
18. Liu D, Xiong X, Zhang Y. Action-dependent adaptive critic designs, In: Proceedings of the INNS-IEEE International Joint Conference on Neural Networks, Washington, DC, 2001, 7: 990-995
19. Murray J J, Cox C J, Lendaris G G, Saeks R. Adaptive dynamic programming, IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews, 2002, 32(5): 140-153
20. Murray J J, Cox C J, Saeks R E. The adaptive dynamic programming theorem, In: Stability and Control of Dynamical Systems with Applications, D. Liu and P. J. Antsaklins, Editors, Boston, MA: Birkhauser, 2003, 379-394
21. Prokhorov D, Santiago R A, Wunsch C C. Adaptive critic designs: A case study for neurocontrol, Neural Networks, 1995, 8: 1367-1372
22. Saeks R E, Cox C J, Mathia K, Maren A J. Asymptotic dynamic programming: In: Preliminary concepts and results, Proceedings of the 1997 International Conference on Neural Networks, Houston, TX, 1997, 6: 2273-2278
23. Santiago R A, Werbos P J. New progress towards truly brain-like intelligent control, Proceedings of the World Congress on Neural Networks, San Diego, CA, 1994, 1(6): 27-33
24. Venayagamoorthy G K, Harley R G, Wunsch D G. Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol or a turbogenerator, IEEE Transactions on Neural Networks, 2002, 13(5): 764-773
25. Werbos P J. Consistency of HDP applied to a simple reinforcement learning problem, Neural Networks, 1990, 3: 179-189
26. Yang L, Enns R, Wang Y-T, Si J. Direct neural dynamic programming, In: Stability and Control of Dynamical Systems with Applications (Chapter 10), Edited by D. Liu and P. J. Antsaklis, Boston, MA: Birkhauser, 2003
27. Zhang Y, Liu D. Call admission control for CDMA cellular networks using adaptive critic designs, In: Proceedings of the 18th IEEE International Symposium on Intelligent Control, Houston, TX, 2003, 9: 511-516 (Invited paper)
28. Sutton R S, Barto A G. Reinforcement Learning: An Introduction, Cambridge, MA: The MIT Press, 1998
29. Haykin S. Neural Networks: A Comprehensive Foundation, Upper Saddle River, NJ: Prentice Hall, 1999
30. Barto A G. Reinforcement learning and adaptive critic methods, In: Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches (Chapter 12), Edited by D. A. White and D. A. Sofge, New York, NY: Van Nostrand Reinhold, 1992
31. Sutton R S. Learning to predict by the methods of temporal differences, Machine Learning, 1988, 3: 9-44
32. Watkins C J C H, Dayan P, Q-learning. Machine Learning, 1992, 8: 279-292
33. Anderson C W, Miller W T III. Challenging control problems, In: Neural Networks for Control (W. T. Miller III, R. S. Sutton, and P. J. Werbos, Eds.). Appendix A. The MIT Press, Cambridgem, MA, 1990
34. Seong C-Y, Widrow B. Neural dynamic optimization for control systems-Part I: Background, IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, 2001, 31(8): 482-489
35. Seong C-Y, Widrow B. Neural dynamic optimization for control systems-Part II: Theory, IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, 2001, 31(8): 490-501
36. Seong C-Y, Widrow B. Neural dynamic optimization for control systems-Part III: Applications, IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, 2001, 31(8): 502-513