Optimal learning of transition probabilities in the two-agent newsvendor problem

Proceedings - Winter Simulation Conference

Volumn , Issue , 2010, Pages 1088-1098

  Ryzhov, Ilya O ^a   Valdez Vivas, Martin R ^b   Powell, Warren B ^a  

^a Princeton University   (United States)

^b Stanford University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COST STRUCTURE; DEMAND INFORMATION; ECONOMIC COSTS; GRADIENT ALGORITHM; LOOK-AHEAD; MARKOV DECISION PROCESSES; NEWSVENDOR PROBLEM; SIMULATION STUDIES; TRANSITION PROBABILITIES;

MARKOV PROCESSES; MATHEMATICAL MODELS; OPTIMIZATION;

PROBABILITY;

EID: 79951586758     PISSN: 08917736     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/WSC.2010.5679081     Document Type: Conference Paper

References (41)

1. Agrawal, N., and S. Smith. 1998. Estimating negative binomial demand for retail inventory management with unobservable lost sales. Naval Research Logistics 43 (6): 839-861. (Pubitemid 126579383)
2. Arrow, K., T. Harris, and T. Marschak. 1951. Optimal inventory policy. Econometrica 19 (3): 250-272.
3. Azoury, K. 1985. Bayes solution to dynamic inventory models under unknown demand distribution. Management Science 31 (9): 1150-1160. (Pubitemid 15567964)
4. Bechhofer, R., T. Santner, and D. Goldsman. 1995. Design and analysis of experiments for statistical selection, screening and multiple comparisons. New York: John Wiley and Sons.
5. Bellman, R., and R. Kalaba. 1959. On adaptive control processes. IRE Transactions on Automatic Control 4 (2): 1-9.
6. Bensoussan, A., M. Cakanyildirim, and S. Sethi. 2007. A multiperiod newsvendor problem with partially observed demand. Mathematics of Operations Research 32 (2): 322-344. (Pubitemid 351159487)
7. Berk, E., Ü. Gürler, and R. Levine. 2007. Bayesian demand updating in the lost sales newsvendor problem: A two-moment approximation. European Journal of Operational Research 182 (1): 256-281. (Pubitemid 46561159)
8. Berry, D. A., and B. Fristedt. 1985. Bandit problems. London: Chapman and Hall.
9. Cachon, G. 2003. Supply chain coordination with contracts. In Handbooks of Operations Research and Management Science, vol. 11: Supply Chain Management, ed. S. Graves and A. de Kok, 229-340. North-Holland Publishing, Amsterdam.
10. Chen, F. 2003. Information sharing and supply chain coordination. In Handbooks of Operations Research and Management Science, vol. 11: Supply Chain Management, ed. S. Graves and A. de Kok, 341-421. North-Holland Publishing, Amsterdam.
11. Chick, S., J. Branke, and C. Schmidt. 2010. Sequential Sampling to Myopically Maximize the Expected Value of Information. INFORMS J. on Computing 22 (1): 71-80.
12. Clark, A., and H. Scarf. 1960. Optimal policies for a multi-echelon inventory problem. Management Science 6 (4): 475-490.
13. Cozzolino, J., R. Gonzalez-Zubieta, and R. Miller. 1965. Markov decision processes with uncertain transition probabilities. Technical Report 11, Operations Research Center, MIT.
14. Dearden, R., N. Friedman, and D. Andre. 1999. Model-based Bayesian Exploration. In Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, 150-159.
15. Dearden, R., N. Friedman, and S. Russell. 1998. Bayesian Q-learning. In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence, 761-768.
16. DeGroot, M. H. 1970. Optimal Statistical Decisions. Hoboken, NJ: John Wiley and Sons.
17. Ding, X., M. Puterman, and A. Bisi. 2002. The censored newsvendor and the optimal acquisition of information. Operations Research 50 (3): 517-527.
18. Duff, M. 1995. Q-learning for bandit problems. Technical Report 95-26, Department of Computer Science, U. Mass Amherst.
19. Duff, M., and A. Barto. 1996. Local bandit approximation for optimal learning problems. Advances in Neural Information Processing Systems 9:1019-1025.
20. Frazier, P. I., W. B. Powell, and S. Dayanik. 2008. A knowledge gradient policy for sequential information collection. SIAM Journal on Control and Optimization 47 (5): 2410-2439.
21. Frazier, P. I., W. B. Powell, and S. Dayanik. 2009. The knowledge-gradient policy for correlated normal rewards. INFORMS J. on Computing 21 (4): 599-613.
22. Gelman, A., J. Carlin, H. Stern, and D. Rubin. 2004. Bayesian data analysis. CRC Press.
23. Gittins, J. 1989. Multi-armed bandit allocation indices. New York: John Wiley and Sons.
24. Gittins, J. C., and D. M. Jones. 1974. A dynamic allocation index for the sequential design of experiments. In Progress in Statistics, ed. J. Gani, 241-266.
25. Gupta, S., and K. Miescke. 1996. Bayesian look ahead one-stage sampling allocations for selection of the best population. Journal of statistical planning and inference 54 (2): 229-244. (Pubitemid 126161097)
26. Khouja, M. 1999. The single-period (newsvendor) problem: literature review and suggestions for future research. OMEGA-International Journal of Management Science 27 (5): 537-553.
27. Kim, S., and B. Nelson. 2006. Selecting the best system. In Handbooks of Operations Research and Management Science, vol. 13: Simulation, ed. S. Henderson and B. Nelson, 501-534. North-Holland Publishing, Amsterdam.
28. Lariviere, M. A., and E. Porteus. 1999. Stalking Information: Bayesian Inventory Management with Unobserved Lost Sales. Management Science 45 (3): 346-363.
29. Mannor, S., D. Simester, P. Sun, and J. Tsitsiklis. 2007. Bias and variance approximation in value function estimates. Management Science 53 (2): 308-322. (Pubitemid 46326182)
30. Martin, J. 1967. Bayesian Decision Problems and Markov Chains. John Wiley and Sons.
31. Nahmias, S., and S. Smith. 1994. Optimizing inventory levels in a two-echelon retailer system with partial lost sales. Management Science 40 (5): 582-596.
32. Petruzzi, N., and M. Dada. 1999. Pricing and the newsvendor problem: A review with extensions. Operations Research 47 (2): 183-194.
33. Puterman, M. L. 1994. Markov decision processes. New York: John Wiley & Sons.
34. Ross, S., J. Pineau, S. Paquet, and S. Brahim. 2008. Online planning algorithms for POMDPs. Journal of Artificial Intelligence 32:663-704.
35. Ryzhov, I. O., and W. B. Powell. 2009. The knowledge gradient algorithm for online subset selection. In Proceedings of the 2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, Nashville, TN, 137-144.
36. Ryzhov, I. O., and W. B. Powell. 2010. Information collection on a graph. Operations Research (to appear).
37. Ryzhov, I. O., W. B. Powell, and P. I. Frazier. 2009. The knowledge gradient algorithm for a general class of online learning problems. Submitted for publication.
38. Satia, J., and R. Lave. 1973. Markov decision processes with imprecise transition probabilities. Operations Research 21 (3): 755-763.
39. Scarf, H. 1959. Bayes solutions of the statistical inventory problem. The Annals of Mathematical Statistics 30 (2): 490-508.
40. Silver, E. 1963. Markovian decision processes with uncertain transition probabilities or rewards. Technical Report 1, Operations Research Center, MIT.
41. Strens, M. 2000. A Bayesian framework for reinforcement learning. In Proceedings of the 17th International Conference on Machine Learning, 943-950.