Performance evaluation of direct heuristic dynamic programming using control-theoretic measures

Journal of Intelligent and Robotic Systems: Theory and Applications

Volumn 55, Issue 2-3, 2009, Pages 177-201

  Yang, Lei ^a   Si, Jennie ^a   Tsakalis, Konstantinos S ^a   Rodriguez, Armando A ^a  

^a Arizona State University   (United States)

Author keywords

Approximate dynamic programming (ADP); Direct heuristic dynamic programming (direct HDP); Linear quadratic regulator (LQR); On line learning control; Sensitivity and complementary sensitivity

Indexed keywords

APPROXIMATE DYNAMIC PROGRAMMING (ADP); DIRECT HEURISTIC DYNAMIC PROGRAMMING (DIRECT HDP); LINEAR QUADRATIC REGULATOR (LQR); ON-LINE LEARNING CONTROL; SENSITIVITY AND COMPLEMENTARY SENSITIVITY;

CONVERGENCE OF NUMERICAL METHODS; DYNAMIC PROGRAMMING; DYNAMICAL SYSTEMS; EDUCATION; ERROR STATISTICS; HEURISTIC METHODS; HEURISTIC PROGRAMMING; LEARNING ALGORITHMS; REINFORCEMENT; REINFORCEMENT LEARNING; SENSITIVITY ANALYSIS; SYSTEMS ENGINEERING;

CONTROLLERS;

EID: 67349172656     PISSN: 09210296     EISSN: 15730409     Source Type: Journal    
DOI: 10.1007/s10846-008-9307-5     Document Type: Article

References (21)

1. Si, J., Barto, A.G., Powell, W.B., Wunsch, D. (eds.): Handbook of Learning and Approximate Dynamic Programming. Wiley-IEEE, New York (2004)
2. Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957)
3. Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-Dynamic Programming. Athena Scientific, Belmont (1996)
4. Puterman, M.L.: Markov Decision Processes-Discrete Stochastic Dynamic Programming. Wiley, New York (1994)
5. R.S. Sutton 1988 Learning to predict by the methods of temporal difference Mach. Learn. 3 9 44
6. A.G. Barto R.S. Sutton C.W. Anderson 1983 Neuron like adaptive elements that can solve difficult learning control problems IEEE Trans. Syst. Man, Cybern. 13 834 847
7. R.S. Sutton 1996 Reinforcement learning with replacing eligibility traces Mach. Learn. 22 1 123 158 (Pubitemid 126724365)
8. G. Tesauro 1994 TD-Gammon, a self-teaching backgammon program achieves master-level play Neural Comput. 6 215 219
9. P.J. Werbos 1977 Advanced forecasting methods for global crisis warning and models of intelligence Gen. Syst. Yearb. 22 25 38
10. Werbos, P.J.: A menu of design for reinforcement learning over time. In: Miller, W.T., III, Sutton, R.S., Werbos, P.J. (eds.) Neural Networks for Control, ch. 3, pp. 67-95. MIT, Cambridge (1990)
11. Werbos, P.J.: Neuro-control and supervised learning: an overview and valuation. In: White, D., Sofge, D. (eds.) Handbook of Intelligent Control, pp. 65-89. Van Nostrand, New York (1992)
12. Werbos, P.J.: Approximate dynamic programming for real-time control and neural modeling. In: White, D., Sofge, D. (eds.) Handbook of Intelligent Control, pp. 493-525. Van Nostrand, New York (1992)
13. D.V. Prokhorov R.A. Santiago D.C. Wunsch 1995 Adaptive critic designs: a case study for neurocontrol Neural Netw. 8 9 1367 1372 (Pubitemid 26072896)
14. D.V. Prokhorov D.C. Wunsch 1997 Adaptive critic designs IEEE Trans. Neural Netw. 8 5 997 1007 (Pubitemid 127763331)
15. J. Si Y. Wang 2001 Online learning control by association and reinforcement IEEE Trans. Neural Netw. 12 2 264 276 (Pubitemid 32371483)
16. R. Enns J. Si 2002 Apache helicopter stabilization using neural dynamic programming AIAA J. Guid. Control Dyn. 25 1 19 25 (Pubitemid 34109509)
17. R. Enns J. Si 2003 Helicopter trimming and tracking control using direct neural dynamic programming IEEE Trans. Neural Netw. 14 4 929 939
18. R. Enns J. Si 2003 Helicopter flight-control reconfiguration for main rotor actuator failures AIAA J. Guid. Control Dyn. 26 4 572 584
19. Zhou, K., Doyle, J.C., Glover, K.: Robust and Optimal Control. Prentice Hall, Englewood Cliffs (1996)
20. K.D. Eltohamy C.Y. Kuo 1998 Nonlinear optimal control of a triple link inverted pendulum with single control input Int. J. Control 69 2 239 256
21. Santamaria, J.C., Sutton, R.S., Ram, A.: Experiments with reinforcement learning in problems with continuous state and action spaces. COINS Technical Report 96-88, University of Massachussetts, Amherst (1996)