Approximating Optimal Control with Value Gradient Learning

Reinforcement Learning and Approximate Dynamic Programming for Feedback Control

Volumn , Issue , 2013, Pages 142-161

  Fairbank, Michael ^a   Prokhorov, Danil ^b   Alonso, Eduardo ^a  

^a CITY UNIVERSITY   (United Kingdom)

^b Toyota Motor North America Inc   (United States)

Author keywords

ADP, DHP and "bootstrapping" parameter; Approximating optimal control with VGL; Critic learning, DHP GDHP into VGL( ); VGL and BPTT algorithms; VGL with a greedy policy, VGL( ) algorithm

Indexed keywords

EID: 84886350301     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118453988.ch7     Document Type: Chapter

References (19)

1. F.-Y.Wang, H. Zhang, and D. Liu. Adaptive dynamic programming: an introduction. IEEE Computational Intelligence Magazine, 4(2):39-47, 2009.
2. P.J. Werbos. Neural networks, system identification, and control in the chemical process industries. In White and Sofge, editors. Handbook of Intelligent Control. Van Nostrant Reinhold, New York, 1992, pp. 283-356.
3. R.E. Bellman. Dynamic Programming. Princeton University Press, Princeton, NJ, 1957.
4. P.J.Werbos. Approximating dynamic programming for real-time control and neural modeling. In White and Sofge, editors. Handbook of Intelligent Control. Van Nostrant Reinhold,New York, 1992, pp. 493-525.
5. D. Prokhorov and D. Wunsch. Adaptive critic designs. IEEE Transactions on Neural Networks, 8(5):997-1007, 1997.
6. R.S. Sutton. Learning to predict by the methods of temporal differences. Machine Learning, 3:9-44, 1988.
7. C.J.C.H. Watkins. Learning from Delayed Rewards. PhD thesis, Cambridge University, 1989.
8. P.J. Werbos. Backpropagation through time: What it does and how to do it. Proceedings of the IEEE, 78(10):1550-1560, 1990.
9. P.J. Werbos. Stable adaptive control using new critic designs. eprint arXiv:adaporg/ 9810001, 1998.
10. J.N. Tsitsiklis and B. Van Roy. An analysis of temporal-difference learning with function approximation. Technical Report LIDS-P-2322, 1996.
11. M. Fairbank and E. Alonso. The divergence of reinforcement learning algorithms with value-iteration and function approximation. eprint arXiv:1107.4606, 2011.
12. S. Ferrari and R.F. Stengel. Model-based adaptive critic designs. In J. Si, et al., editors. Handbook of Learning and Approximate Dynamic Programming.Wiley-IEEE Press, New York, 2004, pp. 65-96.
13. M. Fairbank and E. Alonso. The local optimality of reinforcement learning by value gradients and its relationship to policy gradient learning. eprint arXiv:1101.0428, 2011.
14. M. Fairbank. Reinforcement learning by value gradients. eprint arXiv:0803.3539, 2008.
15. K. Doya. Reinforcement learning in continuous time and space. Neural Computation, 12(1):219-245, 2000.
16. Ali Heydari and S.N. Balakrishnan. Finite-horizon input-constrained nonlinear optimal control using single network adaptive critics. American Control Conference ACC, 2011, pp. 3047-3052.
17. S.E. Fahlman. Faster-learning variations on back-propagation: an empirical study. In Proceedings of the 1988 Connectionist Summer School, pp. 38-51, San Mateo, CA, 1988. Morgan Kaufmann.
18. C.M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1995.
19. M. Riedmiller and H. Braun.Adirect adaptive method for faster backpropagation learning: The RPROP algorithm. In Proceedings of the IEEE International Conference on Neural Networks, pp. 586-591, San Francisco, CA, 1993.