Partially observable markov decision processes with reward information: Basic ideas and models

IEEE Transactions on Automatic Control

Volumn 52, Issue 4, 2007, Pages 677-681

  Cao, Xi Ren ^a   Guo, Xianping ^b  

^a HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

^b SUN YAT SEN UNIVERSITY   (China)

Author keywords

Partially observable Markov decision process (POMDP); Reward information policy

Indexed keywords

DECISION THEORY; MARKOV PROCESSES; MATHEMATICAL MODELS; PROBABILITY DISTRIBUTIONS; STATE ESTIMATION;

PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; REWARD-INFORMATION POLICY;

CONTROL THEORY;

EID: 34247229563     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2007.894520     Document Type: Article

References (15)

1. V. Anantharam, P. Varaiya, and J. Walrand, "Asymtotically efficient allocation rules for the mutiarmed bandit problem with multiple plays - Part II: Markovian reward," IEEE Trans. Autom. Control, vol. AC-32, no. 11, pp. 977-982, Nov. 1987.
2. A. Arapostathis, V. S. Borkar, E. Fernandez-Gaucherand, M. K. Ghosh, and S. I. Markus, "Discrete-time controlled Markov processes with average cost criterion: A survey," SIAM J. Control Optim., vol. 31, pp. 282-344, 1993.
3. V. S. Borkar, "Average cost dynamic programming equations for controlled Markov chains with partial observations," SIAM J. Control Optim., vol. 39, pp. 673-681, 2001.
4. X.-R. Cao and X. P. Guo, "A unified approach to Markov decision problems and performance sensitivity analysis with discounted and average criteria: Multichain cases," Automatica, vol. 40, pp. 1749-1759, 2004.
5. X. P. Guo and X.-R Cao, "Optimal control of ergodic continuous-time Markov chains with average sample-path rewards," SIAM J. Control Optim., vol. 44, pp. 29-48, 2005.
6. O. Hernández-Lerma and J. B. Lasserre, Further Topics on Discrete-Time Markov Control Processes. New York: Springer-Verlag, 1999.
7. O. Hernández-Lerma and R. Romera, "Limiting discounted-cost control of partially observable stochastic systems," SIAM J. Control Optim., vol. 40, pp. 348-369, 2001.
8. S. R. Kulkarni and G. Lugosi, "Finite-time lower bounds for the two-armed bandit problem," IEEE Trans. Autom. Control, vol. 45, no. 4, pp. 711-714, Apr. 2000.
9. W. S. Lovejoy, "A survey of algorithmic results for partially observable Markov decision processes," Ann. Oper. Res., vol. 35, pp. 47-66, 1991.
10. M. Ohnish, H. Kawai, and H. Mine, "An optimal inspection and replacement policy under incomplete state information," Eur. J. Oper. Res., vol. 27, pp. 117-128, 1986.
11. M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York: Wiley, 1994.
12. C. Song, X. H. Guan, and Y. C. Ho, "Constrained ordinal optimization - A feasibility based approach," Discrete Event Dyna. Syst.: Theory Appl., vol. 16, pp. 279-299, 2006.
13. C. C. White, "Bonds on optimal cost for a replacement problem with partial observation," Naval Res. Logist. Quart., vol. 26, pp. 415-422, 1979.
14. S. Yoshikazu and Y. Tsuneo, "Discrete-time Markovian decision processes with incomplete state observation," Ann. Math. Statist., vol. 41, pp.78-86, 1970.
15. Q. C. Zhao, Y. C. Ho, and Q. S. Jia, "Vector ordinal optimization," J. Optim. Theory Appl., vol. 125, pp. 259-274, 2005.