Risk-sensitive optimal control of hidden Markov models: Structural results

IEEE Transactions on Automatic Control

Volumn 42, Issue 10, 1997, Pages 1418-1422

  Fernández Gaucher, Emmanuel ^a   Marcus, Steven I ^b  

^a University of Arizona   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

Author keywords

Hidden Markov models; Information states; Optimal stochastic control; Risk sensitive optimality criterion

Indexed keywords

CONTROL SYSTEM ANALYSIS; MARKOV PROCESSES; MATHEMATICAL MODELS; STABILITY CRITERIA; STOCHASTIC CONTROL SYSTEMS;

HIDDEN MARKOV MODELS; INFORMATION STATES; RISK SENSITIVE OPTIMAL CONTROL;

OPTIMAL CONTROL SYSTEMS;

EID: 0031257250     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/9.633830     Document Type: Article

References (20)

1. A. Arapostathis, V. Borkar, E. Fernández-Gaucherand, M. K. Ghosh, and S. I. Marcus, "Discrete-time controlled Markov processes with average cost criterion: A survey," SIAM J. Contr. Optimiz., vol. 31, pp. 282-344, 1993.
2. D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models Englewood Cliffs, NJ: Prentice-Hall, 1987.
3. J. S. Baras and M. R. James, "Robust and risk-sensitive output feedback control for finite state machines and hidden Markov models," preprint, Aug. 1994.
4. M. Bouakiz and M. J. Sobel, "Inventory control with an exponential utility criterion," Operations Res, vol. 40, pp. 603-608, 1992.
5. A. Bensoussan and J. H. Van Schuppen, "Optimal control of partially observable stochastic systems with an exponential-of-integral performance index," SIAM J. Contr. Optimiz., vol. 23, pp. 599-613, 1985.
6. K.-J. Chung and M. J. Sobel, "Discounted MDP's: Distribution functions and exponential utility maximization," SIAM J. Contr. Optimiz., vol. 25, pp. 49-62, 1987.
7. E. Fernández-Gaucherand, A. Arapostathis, and S. I. Marcus, "On the average cost optimality equation and the structure of optimal policies for partially observable Markov decision processes," Ann. Operations Res, vol. 29, pp. 439-470, 1991.
8. _, " Analysis of an adaptive control scheme for a partially observed controlled Markov chain," IEEE Trans. Automat. Contr., vol. 38, pp. 987-993, 1993.
9. E. Fernández-Gaucherand and S. I. Marcus, "Risk-sensitive optimal control of hidden Markov models: A case study," in Proc. 33rd IEEE Conf. Decision Contr., Orlando, FL, 1994, pp. 1657-1662.
10. _, "Risk-sensitive optimal control of hidden Markov models Structural results," Inst. Syst. Res Rep., Univ. Maryland, College Park, Mar. 1996.
11. A. Gheorghe, "Partially observable Markov processes with a risk-sensitive decision maker," Rev. Roum. Math. Pures et Appl., vol. 22, pp. 461-482, 1977.
12. R. A. Howard and J. E. Matheson, "Risk-sensitive Markov decision processes," Management Sci., vol. 18, pp. 356-370, 1972.
13. M. R. James, J. S. Baras, and R. J. Elliott, "Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems," IEEE Trans. Automat. Contr., vol. 39, pp. 780-792, 1994.
14. P. R. Kumar and P. Varaiya, Stochastic Systems Estimation, Identification and Adaptive Control. Englewood Cliffs, NJ: Prentice-Hall, 1986.
15. W. S. Lovejoy, "On the convexity of policy regions in partially observed systems," Operations Res, vol. 35, pp. 619-621, 1987.
16. A. N. Shiryaev, "On Markov sufficient statistics in nonadditive Bayes problems of sequential analysis," Theory Probab. Appl., vol. 9, pp. 604-618, 1964.
17. R. D. Smallwood and E. J. Sondik, "The optimal control of partially observable Markov processes over a finite horizon," Operations Res., vol. 21, pp. 1071-1088, 1973.
18. C. C. White, "A Markov quality control process subject to partial observation," Management Sci., vol. 23, pp. 843-852, 1977.
19. P. Whittle, Risk-Sensitive Optimal Control. London: Wiley, 1990.
20. _, Optimal Control: Basics and Beyond. London: Wiley, 1996.