Efficient experience reuse in non-Markovian environments

Proceedings of the SICE Annual Conference

Volumn , Issue , 2008, Pages 3327-3332

  Dung, Le Tien ^a   Komeda, Takashi ^a   Takagi, Motoki ^a  

^a SHIBAURA INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

Recurrent neural networks; Reinforcement learning

Indexed keywords

EDUCATION; INFORMATION TECHNOLOGY; INSTRUMENTS; INTERNET; LEARNING SYSTEMS; NEURAL NETWORKS; REINFORCEMENT; REINFORCEMENT LEARNING;

CRITICAL ISSUES; EXPERIENCE REUSES; LEARNING TIMES; MARKOVIAN ENVIRONMENTS; ON-LINE LEARNINGS; Q VALUES; ROBOT PROBLEMS; VISION-BASED;

RECURRENT NEURAL NETWORKS;

EID: 56749173285     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SICE.2008.4655239     Document Type: Conference Paper

References (29)

1. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, 1998.
2. L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement Learning: A Survey"; Journal of Artificial Intelligence Research, 1996.
3. BSamuel W. Hasinoff, "Reinforcement Learning for Problems with Hidden State", Tech. Report, University of Toronto, 2002.
4. Jürgen H. Schmidhuber, "Reinforcement Learning in Markovian and Non-Markovian Environments", Advances in Neural Information Processing Systems, 3, pp. 500-506, 1991.
5. Christopher J. C. H. Watkins, "Learning from Delayed Rewards", PhD thesis, King's College, Cambridge, UK, 1989.
6. Masayuki Ohta, Yoichiro Kumada, Itsuki Noda, "Using Suitable Action Selection Rule in Reinforcement Learning", IEEE International Conference on Systems, Man, and Cybernetics, pp. 4358-4363, 2003.
7. M. Littman, "Memoryless Policies: Theoretical Limitations and Practical Results", From Animal to Animats 3: Proceedings of the 3rd International Conference on Simulation and Adaptive Behavior, 1994.
8. Daan Wierstra, Alexander Foerster, Jan Peters, and Juergen Schmidhuber, "Solving Deep Memory POMDPs with Recurrent Policy Gradients", Artificial Neural Networks: ICANN 2007, pp. 697-706, Springer, Germany, 2007.
9. L.-J. Lin, "Reinforcement Learning for Robots Using Neural Networks", PhD. Thesis, 1993.
10. B. Bakker, "Reinforcement learning with Long Short Term Memory", NIPS 14, 2002.
11. B. Bakker, V. Zhumatiy, G. Gruener, and J. Schmidhuber, "A Robot that Reinforcement-Learns to Identify and Memorize Important Previous Observations", Proceedings of the 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003.
12. Ho, F. and Kamel, M., '"Reinforcement Learning using a Recurrent Neural Network", IEEE World Congress on Computational Intelligence, Proceedings of the ICNN, Vol I, pp. 437-440, 1994.
13. F. Gomez, J. Schmidhuber, and R. Miikkulainen, "Efficient Non-Linear Control through Neuroevolution", Proceedings of the European Conference on Machine Learning ECML-06, Berlin, 2006.
14. Ahmet Onat, Hajime Kita, and Yoshikazu Nishikawa, "Recurrent Neural Networks for Reinforcement Learning: Architecture, Learning Algorithms and Internal Representation", Proceedings of the IJCNN98, pp. 2010-2015, 1998.
15. Whitehead S.D., Lin L.J., "Reinforcement Learning in Non-Markov Environments", Artificial Intelligence, 1994.
16. Charles W. Anderson, Peter Michael Young, Michael R. Buehner, James N. Knight, Keith A. Bush, and Douglas C. Hittle, "Robust Reinforcement Learning Control using Integral Quadratic Constraints for Recurrent Neural Networks", IEEE Transactions on Neural Networks: Special Issue on Neural Networks for Feedback Control Systems, 18(4), pp. 993-1002, 2007.
17. Daniel Schneega, Steffen Udluft, and Thomas Martinetz, "Improving Optimality of Neural Rewards Regression for Data-efficient Batch Near-optimal Policy Identification", Proceedings of the International Conference on Artificial Neural Networks, Germany, 2007.
18. Kenji Doya, "Reinforcement Learning in Continuous Time and Space", Neural Computation, 12, pp. 243-269, 2000.
19. Shivaram Kalyanakrishnan and Peter Stone, "Batch Reinforcement Learning in a Complex Domain", Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, 2007.
20. D. Ernst, P. Geurts, and L. Wehenkel, "Tree-Based Batch Mode Reinforcement Learning", Journal of Machine Learning Research, 6, pp. 503-556, 2005.
21. Marc Carreras, Junku Yuh, Joan Batlle, and Pere Ridao, "Application of SONQL for Real-Time Learning of Robot Behaviors", Robotics and Autonomous Systems, 55(8), pp. 628-642, 2007.
22. L.-J. Lin, "Self-Improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching", Machine Learning, 8, pp. 293-321, 1992.
23. B. Bakker, V. Zhumatiy, G. Gruener, and J. Schmidhuber, "Quasi-Online Reinforcement Learning for Robots", Proceedings of the International Conference on Robotics and Automation, USA, 2006.
24. S. Hochreiter and J. Schmidhuber; "Long Short-Term Memory", Neural Computation, 9, pp. 1735-1780, 1997.
25. F. A. Gers, J. Schmidhuber, and F. Cummins, "Learning to Forget: Continual Prediction with LSTM", Neural Computation, vol. 12, pp. 2451-2471, 2000.
26. F. Gers, N. Schraudolph, J. Schmidhuber, "Learning Precise Timing with LSTM Recurrent Networks", Journal of Machine Learning Research, 3, pp. 115-143, 2002.
27. J. Schmidhuber, D. Wierstra, M. Gagliolo, F. Gomez, "Training Recurrent Networks by Evolino", Neural Computation, 19(3), pp 757-779, 2007.
28. A. Y. Ng, D. Harada, and S. Russell, "Policy invariance under reward transformations: theory and application to reward shaping", Proceedings of the 16th International Conf. on Machine Learning, pp. 278-287, 1999.
29. Le Tien Dung, Takashi Komeda, and Motoki Takagi, "Reinforcement Learning for POMDP Using State Classification", International Journal of Applied Artificial Intelligence, 2008.