On near optimality of the set of finite-state controllers for average cost POMDP

Mathematics of Operations Research

Volumn 33, Issue 1, 2008, Pages 1-11

  Yu, Huizhen ^a   Bertsekas, Dimitri P ^b  

^a UNIVERSITY OF HELSINKI   (Finland)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Average cost criterion; Finite state and control models; Optimality conditions; Partially observable markov decision processes

Indexed keywords

CONTROLLERS; COST FUNCTIONS; OPTIMIZATION; STOCHASTIC CONTROL SYSTEMS;

AVERAGE COST CRITERION; AVERAGE COSTS; BOUNDED SOLUTIONS; CONSTANT AVERAGE COSTS; CONTROL SPACES; FINITE-STATE AND CONTROL MODELS; FINITE-STATE CONTROLLERS; INITIAL STATE; NEAR OPTIMALITY; OPTIMALITY CONDITIONS; PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES;

COSTS;

EID: 61349089285     PISSN: 0364765X     EISSN: 15265471     Source Type: Journal    
DOI: 10.1287/moor.1070.0279     Document Type: Article

References (19)

1. Aberdeen, D., J. Baxter. 2001. Internal-state policy-gradient algorithms for infinite-horizon POMDPs. Technical report, RSISE, Australian National University, Canberra, Australia.
2. Baxter, J., P. L. Bartlett. 2001. Infinite-horizon policy-gradient estimation. J. Artificial Intelligence Res. 15 319-350.
3. Bertsekas, D. P., S. Shreve. 1978. Stochastic Optimal Control: The Discrete Time Case. Academic Press, New York.
4. Bierth, K. J. 1987. An expected average reward criterion. Stochastic Process. Appl. 26 123-140.
5. Dynkin, E. B., A. A. Yushkevich. 1979. Controlled Markov Processes. Springer-Verlag, New York.
6. Feinberg, E. A. 1980. An ε-optimal control of a finite Markov chain with, an average reward criterion. Theory Probab. Appl. 25 70-81.
7. Feinberg, E. A. 1982. Controlled Markov processes with arbitrary numerical criteria. Theory Probab. Appl. 27 486-503.
8. Feinberg, E. A. 1982. Nonrandomized Markov and semi-Markov strategies in dynamic programming. Theory Probab. Appl. 27 116-126.
9. Fernández-Gaucherand, E., A. Arapostathis, S. I. Marcus. 1991. On the average cost optimality equation and the structure of optimal policies for partially observable Markov decision processes. Ann. Oper: Res. 29 439-470.
10. Hsu, S.-P., D.-M. Chuang, A. Arapostathis. 2006. On the existence of stationary optimal policies for partially observed MDPs under the long-run average cost criterion. Systems Control Lett. 55 165-173.
11. Jaakkola, T. S., S. P. Singh, M. I. Jordan. 1995. Reinforcement learning algorithm for partially observable Markov decision problems. Proc. Neural Inform. Processing Systems Conf., Denver, CO. MIT Press, Cambridge, MA.
12. Lauritzen, S. L. 1996. Graphical Models. Oxford University Press, Oxford, UK.
13. Meuleau, N., L. Peshkin, K.-E. Kim, L. P. Kaelbling. 1999. Learning finite-state controllers for partially observable environment. Proc. 15th Conf. Uncertainty in Artificial Intelligence, Stockholm, Sweden. Morgan Kaufmann, San Francisco.
14. Platzman, L. K. 1980. Optimal infinite-horizon undiscounted control of finite probabilistic systems. SIAM J. Control Optim. 18(4) 362-380.
15. Puterman, M. L. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley and Sons, Inc., New York.
16. Ross, S. M. 1968. Arbitrary state Markovian decision processes. Ann. Math. Statist. 39(6) 2118-2122.
17. Runggaldier, W. J., L. Stettner. 1994. Approximations of Discrete Time Partially Observable Control Problems, Applied Mathematics Monographs, Vol. 6. Giardini Editori e Stampatori, Pisa, Italy.
18. Yu, H. 2005. A function approximation approach to estimation of policy gradient for POMDP with structured polices. Proc. 21st Conf. Uncertainty in Artificial Intelligence, Edinburgh, UK, AUAI Press.
19. Yu, H. 2006. Approximate solution methods for partially observable Markov and semi-Markov decision processes. Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, MA.