Autonomous reinforcement learning on raw visual input data in a real world application

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2012, Pages

  Lange, Sascha ^a   Riedmiller, Martin ^a   Voigtländer, Arne ^b  

^a UNIVERSITY OF FREIBURG   (Germany)

^b Shoogee GmbH and Co KG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

CONTROL POLICY; HIGH-DIMENSIONAL; HUMAN PLAYERS; INPUT DATAS; LEARNING ARCHITECTURES; PROOF OF CONCEPT; REAL-WORLD APPLICATION; VISUAL CONTROL;

NEURAL NETWORKS; REINFORCEMENT LEARNING; SEMANTICS; VISION;

INPUT OUTPUT PROGRAMS;

EID: 84865083902     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2012.6252823     Document Type: Conference Paper

References (38)

1. M. Riedmiller, T. Gabel, R. Hafner, and S. Lange, "Reinforcement Learning for Robot Soccer," Autonomous Robots, vol. 27, no. 1, pp. 55-74, 2009.
2. T. Gabel and M. Riedmiller, "Adaptive Reactive Job-Shop Scheduling with Reinforcement Learning Agents," International Journal of Information Technology and Intelligent Computing, vol. 24, no. 4, 2008.
3. T. Kietzmann and M. Riedmiller, "The Neuro Slot Car Racer: Reinforcement Learning in a Real World Setting," in Proc. of the 8th Int. Con! on Machine Learning and Applications, 2009.
4. D. Ernst, P. Geurts, and L. Wehenkel, "Tree-Based Batch Mode Reinforcement Learning," Journal of Machine Learning Research, vol. 6, no. 1, pp. 503-556, 2006.
5. M. Riedmiller, M. Montemerlo, and H. Dahlkamp, "Learning to Drive in 20 Minutes," in Proc. of the FBlT 2007, Jeju, Korea, 2007.
6. R. Hafner and M. Riedmiller, "Reinforcement learning in feedback control," Machine Learning, vol. 27, no. 1, pp. 55-74, 2011, 10.1007/sI0994-011-5235-x. [Online]. Available: http://dx.doLorg/IO.1007/s10994- 011-5235-x
7. G. Hinton and R. Salakhutdinov, "Reducing the Dimensionality of Data with Neural Networks," Science, vol. 313, no. 5786, pp. 504-507, 2006.
8. Y. Bengio, P. Lamblin, D. P opovici, H. Larochelle, and Q. Montreal, "Greedy Layer-Wise Training of Deep Networks," in Advances in Neural Information Processing Systems 19, 2007, pp. 153-160.
9. P. Vincent, H. Larochelle, Y. Bengio, and P. Manzagol, "Extracting and composing robust features with denoising autoencoders," in Proceedings of the 25th international conference on Machine learning. ACM,2008, pp. 1096-1103.
10. S. Chopra, R. Hadsell, and Y. LeCun, "Learning a similarity metric discriminatively, with application to face verification," in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, vol. 1, june 2005, pp. 539 - 546 vol. 1.
11. H. Mobahi, R. Collobert, and J. Weston, "Deep learning from temporal coherence in video," in Proceedings of the 26th Annual International Conference on Machine Learning, ser. ICML '09. New York, NY, USA: ACM, 2009, pp. 737-744.
12. D. Scherer, A. Miiller, and S. Behnke, "Evaluation of pooling operations in convolutional architectures for object recognition," in Proceedings of the 20th international conference on Artificial neural networks: Part III, ser. ICANN'IO. Berlin,H eidelberg: Springer-Verlag,2 010,p p. 92-101.
13. R. Collobert and J. Weston, "A unified architecture for natural language processing: deep neural networks with multitask learning," in Proceedings of the 25th international conference on Machine learning, ser. ICML '08. New York, NY, USA: ACM, 2008, pp. 160-167.
14. H. Larochelle, D. Erhan, A. Courville, 1. Bergstra, and Y. Bengio, "An empirical evaluation of deep architectures on problems with many factors of variation," in Proc. of the 24th International Conference on Machine Learning, 2007, pp. 473-480.
15. D. Erhan, P. Manzagol, Y. Bengio, S. Bengio, and P. Vincent, "The difficulty of training deep architectures and the effect of unsupervised pre-training," in Proc. of The Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS?09), 2009, pp. 153-160.
16. R. R. Salakhutdinov and G. E. Hinton, "Learning a nonlinear embedding by preserving class neighbourhood structure," in AI and Statistics, 2007.
17. M. Ranzato, F. 1. Huang, Y.-L. Boureau, and Y. LeCun, "Unsupervised learning of invariant feature hierarchies with applications to object recognition," in Proc. of CVPR '07.,2007.
18. D. C. Ciresan, U. Meier, L. M. Gambardella, and J. Schmidhuber, "Deep big simple neural nets excel on handwritten digit recognition," Neural Computation, vol. 22, no. 12, pp. 3207-3220, 2010.
19. R. Hadsell,A. Erkan,P. Sermanet,M. Scoffier,U. Muller,a nd Y. LeCun, "Deep belief net learning in a long-range vision system for autonomous off-road driving," in IEEElRSJ International Conference on Intelligent Robots and Systems, 2008. IROS 2008, 2008, pp. 628-633.
20. D. Ciresan, U. Meier, 1. Masci, and 1. Schmidhuber, "A committee of neural networks for traffic sign classification," in Neural Networks (IJCNN), The 2011 International Joint Conference on. IEEE, 2011, pp. 1918-1921.
21. H. Lee,P. Pham,Y. Largman,a nd A. Ng," Unsupervised feature learning for audio classification using convolutional deep belief networks;' in Advances in Neural Information Processing Systems 22, Y. Bengio, D. Schuurmans, 1. Lafferty, C. K. I. Williams, and A. Culotta, Eds., 2009, pp. 1096-1104.
22. G. Gordon, "Stable Function Approximation in Dynamic P rogramming," in Proc. of the 12th Int. Con! on Machine Learning (ICML), 1995, pp. 261-268.
23. D. Ernst, R. Maree, and L. Wehenkel, "Reinforcement learning with raw pixels as input states," in Int. Workshop on Intelligent Computing in Pattern Analysis/Synthesis (IW ICPAS), 2006, pp. 446-454.
24. P. Geurts, D. Ernst, and L. Wehenkel, "Extremely randomized trees," Machine Learning, vol. 63, no. 1, pp. 3-42,2006.
25. S. Jodogne and 1. Piater, "Closed-loop learning of visual control policies," Journal of Artificial Intelligence Research, vol. 28, pp. 349-391, 2007.
26. -, "Interactive learning of mappings from visual percepts to actions," in Proc. of the 22nd international conference on Machine learning, 2005, pp. 393-400.
27. S. lodogne, C. Briquet, and 1. P iater, "Approximate P olicy Iteration for Closed-Loop Learning of Visual Tasks," in Proc. of the European Conference on Machine Learning, 2006.
28. S. Lange and M. Riedmiller, "Deep auto-encoder neural networks in reinforcement learning," in Proc. of the International Joint Conference on Neural Networks (IJCNN 2010), Barcelona, Spain, 2010.
29. S. Lange, "Tiefes Reinforcement Lemen auf Basis visueller Wahrnehmungen," Dissertation, Universitat Osnabriick, 2010.
30. S. Lange and M. Riedmiller, "Deep learning of visual control policies," in European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), 2010.
31. S. Lange,T. Gabel,a nd M. Riedmiller," Batch Reinforcement Learning," in Reinforcement Learning: State of the Art, M. Wiering and M. van Otterlo, Eds. Springer, in press, 2011, in press.
32. K. Fukushima, "Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position," Biological Cybernetics, vol. 36, no. 4, pp. 193-202, 1980.
33. Y. LeCun,L. Bottou,Y. Bengio,a nd P. Haffner," Gradient-based learning applied to document recognition," Proc. of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
34. M. Riedmiller and H. Braun, "A Direct Adaptive Method for Faster Backpropagation Learning: The RPROP Algorithm," in Proc. of the Int. Con/. on Neural Networks, 1993, pp. 586-591.
35. D. Ormoneit and S. Sen, "Kernel-based reinforcement learning," Machine Learning, vol. 49, no. 2, pp. 161-178,2002.
36. T. Kohonen, Self-Organizing Maps, zweite edition ed., ser. Springer Series in Information Sciences. Springer, Heidelberg, 1997, vol. 30.
37. B. Hammer and T. Villmann, "Effizient Klassifizieren und Clustern: Lernparadigmen von Vektorquantisierern," Kiinstliche Intelligenz, vol. 6, no. 3, pp. 5-11, 2006.
38. B. Fritzke, "A growing neural gas network learns topologies," Advances in Neural Information Processing Systems 7, 1995.