Risk-sensitive reinforcement learning applied to control under constraints

Journal of Artificial Intelligence Research

Volumn 24, Issue , 2005, Pages 81-108

  Geibel, Peter ^a   Wysotzki, Fritz ^b  

^a UNIVERSITY OF OSNABRÜCK   (Germany)

^b TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

CONSTRAINT THEORY; ERROR ANALYSIS; FUNCTIONS; HEURISTIC METHODS; MARKOV PROCESSES; REINFORCEMENT; RISK MANAGEMENT;

MARKOV DECISION PROCESSES (MDP); REINFORCEMENT LEARNING; STOCHASTIC INFLOWS; VALUE FUNCTION;

LEARNING SYSTEMS;

EID: 31144477417     PISSN: 10769757     EISSN: 10769757     Source Type: Journal    
DOI: 10.1613/jair.1666     Document Type: Article

References (39)

1. Altman, E. (1999). Constrained Markov Decision Processes. Chapman and Hall/CRC.
2. Baird, L. (1995). Residual algorithms: reinforcement learning with function approximation. In Proc. 12th International Conference on Machine Learning, pp. 30-37. Morgan Kaufmann.
3. Bawas, V. S. (1975). Optimal rules for ordering uncertain prospects. Journal of Finance, 2(1), 1975.
4. Bertsekas, D. P. (1995). Dynamic Programming and Optimal Control. Athena Scientific, Belmont, Massachusetts. Volumes 1 and 2.
5. Bertsekas, D. P., & Tsitsiklis, J. N. (1996). Neuro-Dynamic Programming. Athena Scientific, Belmont, MA.
6. Bishop, C. M. (1995). Neural Networks for Pattern Recognition. Oxford University Press, Oxford.
7. Blythe, J. (1999). Decision-theoretic planning. AI Magazine, 20(2), 37-54.
8. Borkar, V. (2002). Q-learning for risk-sensitive control. Mathematics of Operations Research, 27(2), 294-311.
9. Coraluppi, S., & Marcus, S. (1999). Risk-sensitive and minimax control of discrete-time, finite-state Markov decision processes. Automatica, 35, 301-309.
10. Crites, R. H., & Barto, A. G. (1998). Elevator group control using multiple reinforcement learning agents. Machine Learning, 55(2/3), 235-262.
11. Dolgov, D., & Durfee, E. (2004). Approximating optimal policies for agents with limited execution resources. In Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, pp. 1107-1112. AAAI Press.
12. Feinberg, E., & Shwartz, A. (1994). Markov decision models with weighted discounted criteria. Math. of Operations Research, 19, 152-168.
13. Feinberg, E., & Shwartz, A. (1996). Constrained discounted dynamic programming. Math. of Operations Research, 21, 922-945.
14. Feinberg, E., &: Shwartz, A. (1999). Constrained dynamic programming with two discount factors: Applications and an algorithm. IEEE Transactions on Automatic Control, 44, 628-630.
15. Fishburn, P. C. (1977). Mean-risk analysis with risk associated with below-target returns. American Economics Review, 67(2), 116-126.
16. Freund, R. (1956). The introduction of risk into a programming model. Econometrica, 21, 253-263.
17. Fulkerson, M. S., Littman, M. L., & Keim, G. A. (1998). Speeding safely: Multi-criteria optimization in probabilistic planning. In Proceedings of the Fourteenth National Conference on Artificial Intelligence, p. 831. AAAI Press/MIT Press.
18. Gabor, Z., Kalmar, Z., & Szepesvari, C. (1998). Multi-criteria reinforcement learning. In Proc. 15th International Conf. on Machine Learning, pp. 197-205. Morgan Kaufmann, San Francisco, CA.
19. Geibel, P. (2001). Reinforcement learning with bounded risk. In Brodley, E., & Danyluk, A. P. (Eds.), Machine Learning - Proceedings of the Eighteenth International Conference (ICML01), pp. 162-169. Morgan Kaufmann Publishers.
20. Heger, M. (1994). Consideration of risk in reinforcement learning. In Proc. 11th International Conference on Machine Learning, pp. 105-111. Morgan Kaufmann.
21. Kall, P., & Wallace, S. W. (1994). Stochastic Programming. Wiley, New York.
22. Koenig, S., & Simmons, R. G. (1994). Risk-sensitive planning with probabilistic decision graphs. In Doyle, J., Sandewall, E., & Torasso, P. (Eds.), KR'94: Principles of Knowledge Representation and Reasoning, pp. 363-373, San Francisco, California. Morgan Kaufmann.
23. Kushmerick, N., Hanks, S., & Weld, D. S. (1994). An algorithm for probabilistic least-commitment planning.. In AAAI, pp. 1073-1078.
24. Li, P., Wendt, M., Arellano-Garcia, & Wozny, G. (2002). Optimal operation of distillation processes under uncertain inflows accumulated in a feed tank. AIChe Journal, 48, 1198-1211.
25. Liu, Y., Goodwin, R., & Koenig, S. (2003a). Risk-averse auction agents. In Rosenschein, J., Sandholm, T., & Wooldridge, M. Yokoo, M. (Eds.), Proceedings of the Second International Joint Conference on Autonomous Agents and MultiAgent Systems (AAMAS-03), pp. 353-360. ACM Press.
26. Liu, Y., Goodwin, R., & Koenig, S. (2003b). Risk-averse auction agents.. In AAMAS, pp. 353-360.
27. Markowitz, H. M. (1952). Portfolio selection. The Journal of Finance, 7(1), 77-91.
28. Markowitz, H. M. (1959). Portfolio Selection. John Wiley and Sons, New York.
29. Mihatsch, O., & Neuneier, R. (2002). Risk-sensitive reinforcement learning. Machine Learning, 49(2-3), 267-290.
30. Neuneier, R., & Mihatsch, O. (1999). Risk-sensitive reinforcement learning. In Michael S. Kearns, Sara A. Solla, D. A. C. (Ed.), Advances in Neural Information Processing Systems, Vol. 11. MIT Press.
31. Ross, S. M. (2000). Introduction to Probability Models. Academic Press, New York.
32. Roy, A. D. (1952). Safety first and the holding of assets. Econometrica, 20(3), 431-449.
33. Smart, W. D., & Kaelbling, L. P. (2002). Effective reinforcement learning for mobile robots. In Proceedings of the 2002 IEEE International Conference on Robotics and Automation (ICRA 2002).
34. Stephan, V., Debes, K., Gross, H.-M., Wintrich, F., & Wintrich, H. (2001). A new control scheme for combustion processes using reinforcement learning based on neural networks. International Journal of Computational Intelligence and Applications, 1(2), 121-136.
35. Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning - An Introduction. MIT Press.
36. Tsitsiklis, J. N. (1994). Asynchronous stochastic approximation and Q-learning. Machine Learning, 16(3), 185-202.
37. Watkins, C. J. C. H. (1989). Learning from Delayed Rewards. Ph.D. thesis, King's College, Oxford.
38. Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine Learning, 5(3/4). Special Issue on Reinforcement Learning.
39. Wendt, M., Li, P., & Wozny, G. (2002). Non-linear chance constrained process optimization under uncertainty. Ind. Eng. Chem. Res., 21, 3621-3629.