Controlled Markov chains with risk-sensitive criteria: Average cost, optimality equations, and optimal solutions

Mathematical Methods of Operations Research

Volumn 49, Issue 2, 1999, Pages 299-324

  Cavazos Cadena, Rolando ^a   Fernández Gaucherand, Emmanuel ^b  

^a UNIVERSIDAD AUTÓNOMA AGRARIA ANTONIO NARRO   (Mexico)

^b University of Arizona   (United States)

Author keywords

Bounded solutions to the risk sensitive optimality equation; Constant average cost; Constant risk sensitivity; Controlled Markov chains; Exponential utility function; Simultaneous Doeblin condition

Indexed keywords

EID: 0001099755     PISSN: 14322994     EISSN: None     Source Type: Journal    
DOI: 10.1007/pl00020919     Document Type: Article

References (23)

1. Arapostathis A, Borkar VS, Fernández-Gaucherand E, Gosh MK, Marcus SI (1993) Discrete-time controlled Markov processes with average cost criteria: a survey. SIAM, Journal on Control and Optimization 31:282-334
2. Borkar VK (1984) On minimum cost per unit of time control of Markov chains. SIAM Journal on Control and Optimization 21:965-984
3. Cavazos-Cadena R (1989) Necessary conditions for the optimality equation in average reward Markov decision processes. Journal of Applied Mathematics and Optimization 19:599-613
4. Cavazos-Cadena R (1988) Necessary and sufficient conditions for a bounded solution to the optimality equation in average reward Markov decision chains. Systems & Control Letters 10:71-78
5. Fernández-Gaucherand E, Marcus SI (1997) Risk-sensitive optimal control of hidden Markov models: Structural results. IEEE Transactions on Automatic Control 42:1418-1422
6. Fleming WH, Hernández-Hernández D (1997a) Risk sensitive control of finite state machines on an infinite horizon. SIAM, Journal on Control and Optimization 35:1970-1810
7. Fleming WH, Hernández-Hernández D (1997b) Risk sensitive control of finite state machines on an infinite horizon II, Preprint
8. Hernández-Hernández D, Marcus SI (1996) Risk sensitive control of Markov processes in countable state space. Systems & Control Letters 29:147-155
9. Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes. Springer, New York
10. Hinderer K (1970) Foundations of non-stationary dynamic programming with discrete time parameter. Lecture Notes on Operations Research and Mathematical Systems, No. 33, Springer, New York
11. Hordjik A (1974) Dynamic programming and potential theory. Mathematical Centre Tract No. 51, Matematisch Centrum, Amsterdam
12. Howard RA, Matheson JE (1972) Risk-sensitive Markov decision processes. Management Sciences 18:356-369
13. Jacobson DH (1973) Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games. IEEE Transactions on Automatic Control 18:124-131
14. Marcus SI, Fernández-Gaucherand E, Hernández-Hernández D, Coraluppi S, Fard P (1996) Risk sensitive Markov Decision processes. In Byrnes CI, Datta BN, Gilliam DS, Martin CF (eds.) Systems & control in the twenty-first century. Series: Progress in Systems and Control, Birkhäuser, pp. 263-279
15. James MR, Baras JS, Elliot RJ (1994) Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems. IEEE Transactions on Automatic Control 39:780-792
16. Loève M (1980) Probability theory I. Springer, New York
17. Puterman M (1994) Markov decison processes. Wiley, New York
18. Ross SM Applied probability models with optimization applications. Holden-Day, San Francisco
19. Runolfsson T (1994) The equivalence between infinite horizon control of stochastic systems with exponential-of-integral performance index and stochastic differential games. IEEE Transactions on Automatic Control 39:1551-1563
20. Thomas LC (1980) Connectedness conditions for denumerable state Markov decision processes. In: Hartley R, Thomas LC, White DJ (eds.) Recent advances in Markov decision processes, Academic press, New York
21. Whittle P (1990) Risk-sensitive optimal control. Wiley, New York
22. Brau A, Fernández-Gaucherand E (1997) Controlled Markov chains with risk-sensitive exponential average cost criterion. Proc. 36th IEEE Conference on Decision and Control. San Diego, CA, 2260-2264
23. Fernández-Gaucherand E, Marcus SI (1995) Non-standard optimality criteria for stochastic control problems. Proc. 34th IEEE Conference on Decision and Control, New Orleans, LA, 585-589