Robust exploration/exploitation trade-offs in safety-critical applications

IFAC Proceedings Volumes (IFAC-PapersOnline)

Volumn 8, Issue PART 1, 2012, Pages 660-665

  Tokic, Michel ^a,d   Ertle, Philipp ^b,d   Palm, Günther ^a   Söffker, Dirk ^b   Voos, Holger ^c,d  

^a UNIVERSITY OF ULM   (Germany)

^b UNIVERSITY OF DUISBURG ESSEN   (Germany)

^c UNIVERSITY OF LUXEMBOURG   (Luxembourg)

^d UNIVERSITY OF APPLIED SCIENCES RAVENSBURG WEINGARTEN   (Germany)

Author keywords

Autonomous systems; Learning; Safety; Temporal difference learning

Indexed keywords

ACCIDENT PREVENTION; ECONOMIC AND SOCIAL EFFECTS; FAULT DETECTION; PLANT MANAGEMENT; SAFETY ENGINEERING;

AUTONOMOUS SYSTEMS; DYNAMIC ENVIRONMENTS; EXPLORATION POLICY; EXPLORATION/EXPLOITATION; EXPLORATION/EXPLOITATION DILEMMAS; LEARNING; SAFETY CRITICAL APPLICATIONS; TEMPORAL DIFFERENCE LEARNING;

REINFORCEMENT LEARNING;

EID: 84867036077     PISSN: 14746670     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3182/20120829-3-MX-2028.00160     Document Type: Conference Paper

References (18)

1. Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B., and Dolan, R.J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441(7095), 876-879.
2. Dörner, D. (2000). Die Logik des Mißlingens strategisches Denken in komplexen Situationen. Rowohlt, Reinbek bei Hamburg.
3. Ertle, P., Voos, H., and Söffker, D. (2010). On risk for-malization of on-line risk assessment for safe decision making in robotics. In 7th IARP Workshop on Technical Challenges for Dependable Robots in Human Environments, 15-22.
4. Geibel, P. (2001). Reinforcement learning with bounded risk. In Proceedings of the 18th International Conference on Machine Learning, ICML'01, 162-169. Morgan Kaufmann Publishers Inc.
5. George, A.P. and Powell, W.B. (2006). Adaptive stepsizes for recursive estimation with applications in approximate dynamic programming. Machine Learning, 65(1), 167-198.
6. Hans, A., Schneegaß, D., Schäfer, A.M., and Udluft, S. (2008). Safe exploration for reinforcement learning. In Proceedings of the 16th European Symposium on Artificial Neural Networks ESANN'08, 143-148.
7. Heger, M. (1994). Consideration of risk in reinforcement learning. In Proceedings of the 11th International Conference on Machine Learning, 105-111. Morgan Kaufmann Publishers, Inc., San Francisco, CA, USA.
8. Lussier, B., Chatila, R., Ingrand, F., Killijian, M.O., and Powell, D. (2004). On fault tolerance and robustness in autonomous systems. In 3rd IARP - IEEE/RAS - EURON Joint Workshop on Technical Challenges for Dependable Robots in Human Environments, 7-9.
9. Mihatsch, O. and Neuneier, R. (2002). Risk-sensitive reinforcement learning. Machine Learning, 49(2), 267-290.
10. Ng, A.Y. and Kim, H.J. (2004). Stable adaptive control with online learning. In Advances in Neural Information Processing Systems, 17, 13-18.
11. Perkins, T.J. and Barto, A.G. (2003). Lyapunov design for safe reinforcement learning. Journal of Machine Learning Research, 3, 803-832.
12. Rummery, G.A. and Niranjan, M. (1994). On-line Q-learning using connectionist systems. Technical Report CUED/F-INFENG/TR 166, Cambridge University.
13. Smithers, T. (1997). Autonomy in robots and other agents. Brain and Cognition, 34, 88-106.
14. Sutton, R.S. and Barto, A.G. (1998). Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA.
15. Tokic, M. (2010). Adaptive ε-greedy exploration in reinforcement learning based on value differences. In KI 2010: Advances in Artificial Intelligence, 203-210. Springer Berlin / Heidelberg.
16. Tokic, M. and Palm, G. (2011). Value-difference based exploration: Adaptive exploration between epsilon-greedy and softmax. In KI 2011: Advances in Artificial Intelligence, 335-346. Springer Berlin / Heidelberg.
17. Watkins, C. (1989). Learning from Delayed Rewards. Ph.D. thesis, University of Cambridge, England.
18. Watkins, C. and Dayan, P. (1992). Technical note: Q-learning. Machine Learning, 8(3), 279-292.