Bias and variance approximation in value function estimates

Management Science

Volumn 53, Issue 2, 2007, Pages 308-322

  Mannor, Shie ^a   Simester, Duncan ^b   Sun, Peng ^c   Tsitsiklis, John N ^d  

^a MCGILL UNIVERSITY   (Canada)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^c DUKE UNIVERSITY   (United States)

^d MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Bias; Confidence interval; Value function; Variance

Indexed keywords

MAIL-ORDER; MARKOV DECISION PROCESS; VALUE FUNCTION ESTIMATES;

DATABASE SYSTEMS; DECISION MAKING; FUNCTION EVALUATION; MARKOV PROCESSES;

APPROXIMATION THEORY;

EID: 33847336943     PISSN: 00251909     EISSN: 15265501     Source Type: Journal    
DOI: 10.1287/mnsc.1060.0614     Document Type: Article

References (24)

1. Barberis, N. 2000. Investing for the long-run when returns are predictable. J. Finance 55 225-264.
2. Baukal-Gursoy, M., K. Ross. 1992. Variability sensitive Markov decision processes. Math. Oper. Res. 17(3) 558-571.
3. Bertsekas, D. P. 2000. Dynamic Programming and Optimal Control, 2nd ed., Vol. 1. Athena Scientific, Belmont, MA.
4. Bertsekas, D. P., J. N. Tsitsiklis. 1996. Neuro-Dynamic Programming. Athena Scientific, Belmont, MA.
5. Bitran, G. R., S. V. Mondschein. 1996. Mailing decisions in the catalog sales industry. Management Sci. 42(9) 1364-1381.
6. Bult, J., T. Wansbeek. 1995. Optimal selection for direct mail. Marketing Sci. 14(4) 378-394.
7. Campbell, J. Y., L. M. Viceira. 2002. Strategic Asset Allocation: Portfolio Choice for Long-Term Investors. Oxford University Press, Oxford, UK.
8. Clark, C. 1961. The greatest of a finite set of random variables. Oper. Res. 9 145-162.
9. Dixit, A. K., R. S. Pindyck. 1994. Investment under Uncertainty. Princeton University Press, Princeton, NJ.
10. Filar, J. A., L. Kallenberg, H. Lee. 1989. Variance-penalized Markov decision processes. Math. Oper. Res. 14 147-161.
11. Godfrey, G., W. Powell. 2002. An adaptive dynamic programming algorithm for dynamic fleet management I: Single period travel time. Transportation Sci. 36(1) 21-39.
12. Gönül, F., M. Shi. 1998. Optimal mailing of catalogs: A new methodology using estimable structural dynamic programming models. Management Sci. 44(9) 1249-1262.
13. Hendel, I., A. Nevo. 2002. Measuring the implications of sales and consumer stockpiling behavior. Econometrica. Forthcoming.
14. Keane, M., K. Wolpin. 1994. The solution and estimation of discrete choice dynamic programming models by simulation and interpolation: Monte Carlo evidence. Rev. Econom. Statist. 76(4) 648-672.
15. Leadbetter, M. R., G. Lindgren, H. Rootzén. 1983. Extremes and Related Properties of Random Sequences and Processes. Springer-Verlag, New York.
16. Luenberger, D. G. 1997. Investment Science. Oxford University Press, New York.
17. McGill, J., G. van Ryzin. 1999. Revenue management: Research overview and prospects. Transportation Sci 33 233-256.
18. Rust, J. 1987. Optimal replacement of CMC bus engines: An empirical model of Harold Zurcher. Econometrica 55(5) 999-1033.
19. Serfling, R. J. 1980. Approximation Theorems of Mathematical Statistics. Wiley, New York.
20. Simester, D., P. Sun, J. N. Tsitsiklis. 2006. Dynamic catalog mailing policies. Management Sci. 52(5) 683-696.
21. Sobel, M. J. 1982. The variance of a discounted Markov decision process. J. Appl. Probab. 19 794-802.
22. Sutton, R., A. Barto. 1998. Reinforcement Learning. MIT Press, Cambridge, MA.
23. Xia, Y. 2001. Learning about predictability: The effects of parameter uncertainty on dynamic asset allocation. J. Finance 56 205-246.
24. Zipkin, P. 2000. Foundations of Inventory Management. McGraw-Hill/Irwin, Boston, MA.