Adaptive critic design with echo state network

Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics

Volumn , Issue , 2010, Pages 1010-1015

  Koprinkova Hristova, Petia ^a   Oubbati, Mohamed ^b   Palm, Guenther ^b  

^a BULGARIAN ACADEMY OF SCIENCES   (Bulgaria)

^b UNIVERSITY OF ULM   (Germany)

Author keywords

Adaptive critic designs; Echo state networks; Mobile robot; Reinforcement learning

Indexed keywords

ADAPTIVE CRITIC; ADAPTIVE CRITIC DESIGNS; ECHO STATE NETWORKS; MOBILE ROBOT CONTROL; NOVEL NEURAL NETWORK; ON-LINE APPLICATIONS; ONLINE TRAINING; OPTIMIZATION APPROACH; TRAINING ALGORITHMS;

CELLULAR RADIO SYSTEMS; CYBERNETICS; MACHINE DESIGN; MOBILE ROBOTS; NEURAL NETWORKS; REINFORCEMENT LEARNING; TIME SERIES ANALYSIS; WIRELESS NETWORKS;

NETWORK ARCHITECTURE;

EID: 78751554137     PISSN: 1062922X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICSMC.2010.5641744     Document Type: Conference Paper

References (18)

1. A.G. Barto, R.S. Sutton, C.W. Anderson, Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems, IEEE Trans. on Systems, Man and Cybernetics, vol. 13, No 5, 1983, pp. 834-846.
2. R.E. Bellman, Dynamic Programming, Princeton, NJ: Princeton Univ. Press, 1957.
3. D.P. Bertsekas, J.N. Tsitsiklis, Neuro-Dymanic Programming, Athena Scientific, Belmont, MA, 1996.
4. H. Jaeger, Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the "echo state network" approach, GMD Report 159, German National Research Center for Information Technology, 2002 p.48.
5. H. Jaeger, Adaptive nonlinear system identification with echo state networks, In: Advances in Neural Information Processing Systems 15 (NIPS 2002), MIT Press, Cambridge, MA, 2003, pp.593-600.
6. M. Lukosevicius, H. Jaeger, Reservoir computing approaches to recurrent neural network training, Computer Science Review, vol.3, 2009, pp.127-149.
7. A. Pacut, Neural techniques in control, In: Neural Networks for Instrumentation, Measurement and Related Industrial Applications, Edited by S. Ablameyko, L. Goras, M. Gori and V. Piuri. NATO Science Series vol. 185, IOS Press, Amsterdam, 2003, pp.78-118.
8. D.V. Prokhorov, Adaptive critic designs and their applications, Ph.D. dissertation, Department of Electrical Engineering, Texas Tech. Univ., 1997.
9. D. Prokhorov, Toward effective combination of off-line and on-line training in ADP framework, Proc. of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL 2007), pp.268-271.
10. D. Prokhorov, Training recurrent neurocontrollers for real-time applications, IEEE Trans. on Neural Networks, vol.18, N04, 2007, pp.1003-1015.
11. D. Prokhorov, Echo state networks: appeal and challenges, Proc. of Int. Joint Conf. on Neural Networks (IJCNN), Montreal, Canada, August 2005, pp.1463-1466.
12. B. Schrauwen, M. Wandermann, D. Verstraeten, J.J. Steil, Improving reservoirs using intrinsic plasticity, Neurocomputing, vol. 71, 2008, pp.1l59-1171.
13. J. Si, Y.-T. Wang, On-line learning control by association and reinforcement, IEEE Trans. on Neural Networks, vol.12, No2, 2001, pp.264-276.
14. R.S. Sutton, Learning to predict by methods of temporal differences, Machine Learning, vol.3, 1988, pp.9-44.
15. C. Watkins, P. Dayan, Q-learning, Machine Learning, vol. 8, 1992, pp.279-292.
16. P.J. Werbos, Backpropagation Through Time: What It Does and How to Do It, Proceedings of the IEEE, vol. 78, No 10, 1990, pp.1550-1560.
17. B. Widrow et al., Punish/Reward: Learning with a Critic in Adaptive Threshold Systems, IEEE Trans. on SMC, vol. 3, No 5, 1973, pp.455-465.
18. http://www.reservoir-computing.org/software : Simple and very simple Matlab toolbox for Echo State Networks by H. Jaeger and group members.