Bias and variance in value function estimation

Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004

Volumn , Issue , 2004, Pages 568-575

  Mannor, Shie ^a   Simester, Duncan ^b   Sun, Peng ^c   Tsitsiklis, John N ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^c DUKE UNIVERSITY   (United States)

Author keywords

Bayesian Estimation; Bias; Markov Processes; Reinforcement Learning; Variance

Indexed keywords

BAYESIAN ESTIMATION; BIAS; REINFORCEMENT LEARNING; VARIANCE;

DATA REDUCTION; DECISION MAKING; FUNCTIONS; MARKOV PROCESSES; MATHEMATICAL MODELS; OPTIMIZATION; PARAMETER ESTIMATION;

LEARNING SYSTEMS;

EID: 14344261137     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (11)

1. Bitran, G. R., & Mondschein, S. V. (1996). Mailing decisions in the catalog sales industry. Management Science, 42, 1364-1381.
2. Bult, J., & Wansbeek, T. (1995). Optimal selection for direct mail. Marketing Science, 14, 378-394.
3. Dearden, R., Friedman, N., & Russell, S. J. (1998). Bayesian Q-learning. Proceedings of the 15th National Conference on Artificial Intelligence (pp. 761-768). AAAI Press / The MIT Press.
4. Filar, J. A., Kallenberg, L., & Lee, H. (1989). Variance-penalized Markov decision processes. Mathematics of operations Research, 14, 147-161.
5. Gelman, A., Carlin, J., Stern, H., & Rubin, D. B. (1995). Bayesian data analysis. Chapman and Hall.
6. Gönül, F., & Shi, M. (1998). Optimal mailing of catalogs: A new methodology using estimable structural dynamic programming models. Management Science, 44, 1249-1262.
7. Greensmith, E., Bartlett, P. L., & Baxter, J. (2002). Variance reduction techniques for gradient estimates in reinforcement learning. Advances in Neural Information Processing Systems 14.
8. Kearns, M., & Singh, S. (2002). Near-optimal reinforcement learning in polynomial time. Machine Learning, 49, 209-232.
9. Puterman, M. (1994). Markov decision processes. Wiley-Interscience.
10. Sobel, M. J. (1982). The variance of discounted Markov decision process. Journal of Applied Probability, 19, 794-802.
11. Strens, M. (2000). A Bayesian framework for reinforcement learning. In Proceedings of the 17th International Conference on Machine Learning (pp. 943-950).