Percentile optimization in uncertain Markov decision processes with application to efficient exploration

ACM International Conference Proceeding Series

Volumn 227, Issue , 2007, Pages 225-232

  Delage, Erick ^a   Mannor, Shie ^b  

^a Stanford University   (United States)

^b MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

DECISION MAKING; MARKOV PROCESSES; MATHEMATICAL MODELS; OPTIMIZATION; PARAMETER ESTIMATION; UNCERTAIN SYSTEMS;

EXPLORATION STRATEGY; PARAMETER UNCERTAINTY; PERCENTILE OPTIMIZATION;

DYNAMICAL SYSTEMS;

EID: 34547985785     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1273496.1273525     Document Type: Conference Paper

References (19)

1. Ben-Tal, A., & Nemirovski, A. (1998). Robust convex optimization. Mathematics of Operations Research, 23, 769-805.
2. Brafman, R., & Tennenholtz, M. (2003). R-max - a general polynomial time algorithm for near-optimal reinforcement learning. J. of Machine Learning Research., 3, 213-231.
3. Calafiore, G., & El Ghaoui, L. (2006). On distributionally robust chance-constrained linear programs. Optimization Theory and Applications, 130, 1-22.
4. Charnes, A., & Cooper, W. (1959). Chance constrained programming. Management Science, 6, 73-79.
5. Dearden, R., Friedman, N., & Andre, D. (1999). Model-based Bayesian exploration. Proc. of Uncertainty in AI (pp. 150-159).
6. Filar, J., Krass, D., & Ross, K. (1995). Percentile performance criteria for limiting average Markov control problems. IEEE Trans, on Automatic Control, 40, 2-10.
7. Gelman, A., Carlin, J., Stern, H., & Rubin, D. (2003). Bayesian data analysis, second edition. Chapman & Hall/CRC.
8. Givan, R., Leach, S., & Dean, T. (2000). Boundedparameter Markov decision processes. Artificial Intelligence, 122, 71-109.
9. Howard, R. (1966). Information value theory. IEEE Trans. on Systems Science and Cybernetics, SSC-2, 22-26.
10. Iyengar, G. (2005). Robust dynamic programming. Mathematics of Operations Research, 30, 257-280.
11. Kearns, M., & Singh, S. (1998). Near-optimal reinforcement learning in polynomial time. Proc. ICML (pp. 260-268).
12. Lobo, M., Vandenberghe, L., Boyd, S., & Lebret, H. (1998). Applications of second order cone programming. Linear Algebra and its App., 284, 193-228.
13. Mannor, S., Simester, D., Sun, P., & Tsitsiklis, J. (2007). Bias and variance in value function estimation. Management Science, 53, 308-322.
14. Nemirovski, A., & Shapiro, A. (2006). Convex approximations of chance constrained programs. SIAM Journal on Optimization, 17, 969-996.
15. Nilim, A., & El Ghaoui, L. Robust Markov decision processes with uncertain transition matrices. Operations Research, 53, 780-798.
16. Prékopa, A. (1995). Stochastic programming. Kluwer Academic Publishers.
17. Putterman, M. (1994). Markov decision processes: Discrete stochastic dynamic programming. Wiley.
18. Silver, E. (1963). Markovian decision processes with uncertain transition probabilities or rewards (Technical Report 1). Operations Research Center, MIT.
19. Støehl, A., & Littman, M. (2005). A theoretical analysis of model-based interval estimation. Proc. ICML (pp. 857-864).