Biologically inspired scheme for continuous-time approximate dynamic programming

Transactions of the Institute of Measurement & Control

Volumn 30, Issue 3-4, 2008, Pages 207-223

  Vrabie, Draguna ^a   Lewis, Frank ^a   Abu Khalaf, Murad ^b  

^a UNIVERSITY OF TEXAS AT ARLINGTON   (United States)

^b MATHWORKS INC   (United States)

Author keywords

Actor Critic structures; approximate dynamic programming; linear quadratic regulation; policy iteration

Indexed keywords

BIOMIMETICS; CONTINUOUS TIME SYSTEMS; CONTROL SYSTEM ANALYSIS; CONTROLLERS; DYNAMIC PROGRAMMING; ITERATIVE METHODS; LEARNING SYSTEMS; LINEAR SYSTEMS; MAN MACHINE SYSTEMS; NONLINEAR SYSTEMS; REINFORCEMENT LEARNING; RICCATI EQUATIONS; ROBUST CONTROL; STATE FEEDBACK; STATE SPACE METHODS; SYSTEM THEORY;

ALGEBRAIC RICCATI EQUATIONS; APPROXIMATE DYNAMIC PROGRAMMING; BIOLOGICALLY INSPIRED; LINEAR QUADRATIC REGULATIONS; MODELLING AND ANALYSIS; OPTIMAL STATE FEEDBACK; POLICY ITERATION; REINFORCEMENT LEARNING TECHNIQUES;

ADAPTIVE CONTROL SYSTEMS;

EID: 47949083966     PISSN: 01423312     EISSN: None     Source Type: Journal    
DOI: 10.1177/0142331207088188     Document Type: Article

References (29)

1. Al-Tamimi, A., Abu-Khalaf, M. and Lewis, F.L. 2007a: Adaptive critic designs for discrete-time zero-sum games with application to H-Infinity control. IEEE Transactions on Systems, Man, and Cybernetics-Part B 37, 240-47.
2. Al-Tamimi, A., Abu-Khalaf, M. and Lewis, F.L. 2007b: Model-free Q-learning designs for discrete-time zero-sum games with application to H-Infinity control. Automatica 43, 473-82.
3. Baird, L. 1994: Reinforcement learning in continuous-time: advantage updating. Proceedings of the International Conference on Neural Networks, Orlando, FL, June.
4. Barto, G., Sutton, R.S. and Anderson, C.W. 1983: Neuronlike elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics SMC-, 13, 835-46.
5. Bertsekas, D. and Tsitsiklis, J. 1996: Neuro-dynamic programming. Athena Scientific.
6. Bertsekas, D.P. 2005: Dynamic programming and suboptimal control: a survey from ADP to MPC. Proceeding of CDC'05.
7. Bradtke, S.J., Ydestie, B.E. and Barto, A.G. 1994: Adaptive linear quadratic control using policy iteration. Proceedings of the American Control Conference, Baltmore, MD, June, 3475-76.
8. Brewer, J.W. 1978: Kronecker products and matrix calculus in system theory. IEEE Transactions on Circuits and System CAS-, 25, 772-81.
9. Doya, K. 1996: Temporal difference learning in continuous-time and space. Advances in Neural Information Processing Systems 8, 1073-79.
10. Doya, K. 2000: Reinforcement learning in continuous-time and space. Neural Computation 12, 219-45.
11. Ferrari, S. and Stengel, R. 2002: An adaptive critic global controller. Proceedings of the American Control Conference, Anchorage, AK, 2665-70.
12. Galambos, R., Makeig, S. and Talmachoff, J. 1981: A 40-Hz auditory potential recorded from the human scalp. Proceedings of the National Academy of Sciences, 78, 2643-47.
13. Howard, R.A. 1960: Dynamic programming and Markov processes. MIT Press.
14. Kleinman, D. 1968: On an iterative technique for Riccati equation computations. IEEE Transactions on Automatic Control 13, 114-15.
15. Landelius, T. 1997: Reinforcement learning and distributed local model synthesis. PhD dissertation, Linköping University.
16. Levine, D.S. and Elsberry, W.R., editors. 1997: Optimality in biological and artificial networks? Lawrence Erlbaum Associates.
17. Levine, D.S., Brown, V.R. and Shirey, V.T., editors. 2000: Oscillations in neural systems. Lawrence Erlbaum Associates.
18. Liu, X. and Balakrishnan, S.N. 2001: Adaptive critic based neuro-observer. Proceedings of the American Control Conference, Arlington, VA, 1616-21.
19. Malsburg, C. von der and Schneider, W. 1986: A neural cocktail party processor. Biological Cybernetics 54, 29-40.
20. Murray, J., Cox, C., Lendaris, G. and Saeks, R. 2002: Adaptive dynamic programming. IEEE Transaction on Systems, Man, and Cybernetics 32, 140-53.
21. Prokhorov, D. and Wunsch, D. 1997: Adaptive critic designs. IEEE Transactions on Neural Networks 8, 997-1007.
22. Singer, W. and Gray, C.M. 1995: Visual feature integration and the temporal correlation hypothesis. Annual Review of Neuroscience 18, 555-86.
23. Soltesz, I. 2000: The interplay of intrinsic and synaptic membrane currents in delta, theta and 40-Hz oscillations. In Levine, D.S., Brown, V.R. & Shirey, V.T., editors. Oscillations in neural systems. Lawrence Erlbaum Associates.
24. Sutton, R.S. 1988: Learning to predict by the method of temporal differences. Machine Learning 3, 9-44.
25. Sutton, R.S. and Barto, A.G. 1998: Reinforcement learning: an introduction. MIT Press.
26. Vrabie, D., Abu-Khalaf, M., Lewis, F.L. and Wang, Youyi. 2007: Continuous-time ADP for linear systems with partially unknown dynamics. Proceedings of the IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, 247-53.
27. Watkins, C. 1989: Learning from delayed rewards. Ph.D. thesis, Cambridge University.
28. Werbos, P.J. 1992: Approximate dynamic programming for real-time control and neural modeling. In White, D.A. and Sofge, D.A., editors. Handbook of intelligent control. Van Nostrand.
29. Werbos, P.J. 1997: Optimization: a foundation for understanding consciousness. In Levine, D.S. and Elsberry, W.R., editors, Optimality in biological and artificial networks? Lawrence Erlbaum Associates.