Optimal Learning

Optimal Learning

Volumn , Issue , 2012, Pages 1-384

  Powell, Warren B ^a   Ryzhov, Ilya O ^b  

^a Princeton University   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

MATHEMATICAL PROGRAMMING; MATLAB; TABLE LOOKUP;

ADAPTIVE LEARNING; CONTINUOUS MEASUREMENTS; OFF-LINE PROBLEMS; PROBABILITY AND STATISTICS; RANKING AND SELECTION; SIMULATION OPTIMIZATION; SPECIFIC LEARNING; WEBSITE FEATURES;

APPLICATION PROGRAMS;

EID: 84871543700     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118309858     Document Type: Book

References (196)

1. Abernethy, J., Hazan, E. & Rakhlin, A. (2008), Competing in the dark: An efficient algorithm for bandit linear optimization, in'Proceedings of the 21st Annual Conference on Learning Theory', pp. 263-274.
2. Aghion, P., Bolton, P., Harris, C. & Jullien, B. (1991), 'Optimal learning by experimentation', The Review of Economic Studies 58, 621-654.
3. Agrawal, R. (1995), 'Sample mean based index policies with O (log n) regret for the multi-armed bandit problem', Advance s in Applied Probability 27(4), 1054-1078.
4. Alrefaei, M. H. & Andradottir, S. (1999), 'A simulated annealing algorithm with constant temperature for discrete stochastic optimization', Management Science 45,748-764.
5. Andradottir, S. & Prudius, A. A. (2010), 'Adaptive random search for continuous simulation optimization', Naval Research Logistics 57(6), 583-604.
6. Ankenman, B., Nelson, B. L. & Staum, J. (2010), 'Stochastic kriging for simulation metamodeling', Operations Research 58(2), 371-382.
7. Araman, V. & Caldentey, R. (2009), 'Dynamic pricing for non-perishable products with demand learning', Operations Research 57(5), 1169-1188.
8. Audibert, J. Y., Bubeck, S. & Munos, R. (2010), Best arm identification in multi-armed bandits, in'Proceedings of the 23rd Annual Conference on Learning Theory (COLT)', pp. 1-14.
9. Auer, P., Cesa-Bianchi, N. & Fischer, P. (2002), 'Finite-time analysis of the multi-armed bandit problem', Machine Learning 47(2), 235-256.
10. Auer, P., Jaksch, T. & Ortner, R. (2008), Near-optimal regret bounds for reinforcement learning, in D. Koller, Y. Bengio, D. Schuurmans, L. Bottou & A. Culotta, eds,'Advances in Neural Information Processing Systems', Vol. 21, pp. 89-96.
11. Aviv, Y. & Pazgal, A. (2005), 'A partially observed Markov decision process for dynamic pricing', Management Science 51(9), 1400-1416.
12. Avriel, M. & Wilde, D. (1966), 'Optimality proof for the symmetric Fibonacci search technique', Fibonacci Quarterly 4, 265-269.
13. Azadivar, F. & Tompkins, G. (1999), 'Simulation optimization with qualitative variables and structural model changes: A genetic algorithm approach', European Journal of Operational Research 113, 169-182.
14. Azadivar, F. & Wang, J. (2000), 'Facility Layout Optimization Using Simulation and Genetic Algorithms', International Journal of Production Research 38( 17), 43694383.
15. Banks, J., Nelson, B. L. & J. S. Carson, 1.1. (1996), Discrete-Event System Simulation, Prentice-Hall, Inc., Englewood Cliffs, N.J.
16. Barr, D. R. & Rizvi, M. H. (1966), 'An introduction to ranking and selection procedures', J. Amer. Statist. Assoc. 61(315), 640-646.
17. Bartlett, P., Dani, V., Hayes, T, Kakade, S., Rakhlin, A. & Tewari, A. (2008), High-probability regret bounds for bandit online linear optimization, in'Proceedings of the 21st Annual Conference on Learning Theory', pp. 335-342.
18. Bechhofer, R. E. (1954), 'A Single-Sample Multiple Decision Procedure for Ranking Means of Normal Populations with known Variances', The Annals of Mathematical Statistics 25, 16-39.
19. Bechhofer, R. E., Kiefer, J. & Sobel, M. (1968), Sequential Identification and Ranking Procedures, University of Chicago Press, Chicago.
20. Bechhofer, R., Santner, T. & Goldsman, D. (1995), Design and Analysis of Experiments for Statistical Selection, Screening and Multiple Comparisons, J.Wiley & Sons, New York.
21. Bellman, R. & Kalaba, R. (1959), 'On adaptive control processes', IRE Trans. 4,1-9.
22. Ben-Tal, A., Ghaoui, L. E. & Nemirovski, A. (2009), Robust Optimization, Princeton University Press, Princeton NJ.
23. Benveniste, A., Metivier, M. & Priouret, P. (1990), Adaptive Algorithms and Stochastic Approximations, Springer-Verlag, New York.
24. Bergemann, D. & Välimäki, J. (1996), 'Learning and strategic pricing', Econometrica 64(5), 1125-1149.
25. Berry, D. A. & Fristedt, B. (1985), Bandit Problems, Chapman and Hall, London.
26. Bertsekas, D. P. & Tsitsiklis, J. N. (1996), Neuro-dynamic programming, Athena Scientific, Belmont, MA.
27. Bertsimas, D. J. & Nino-Mora, J. (2000), 'Restless bandits, linear programming relaxations, and a primal-dual index heuristic', Operations Research 48(1), 8090.
28. Better, M., Glover, F. W., Kochenberger, G. & Wang, H. (2008), 'Simulation optimization: applications in risk management', International Journal of Information Technology and Decision Making 7(4), 571-587.
29. Beyer, H. & Sendhoff, B. (2007), 'Robust optimization: A comprehensive survey', Computer Methods in Applied Mechanics and Engineering 196(33-34), 3190-3218.
30. Birchler, U. & Butler, M. (2007), Information Economics, Routledge, London.
31. Boyen, X. & Koller, D. (1998), Tractable inference for complex stochastic processes, in'Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence', pp. 33-42.
32. Brafman, R. I. & Tennenholtz, M. (2003), 'R-max - a general polynomial time algorithm for near-optimal reinforcement learning', Journal of Machine Learning Research 3,213-231.
33. Branke, J., Chick, S. E. & Schmidt, C. (2005), New developments in ranking and selection: an empirical comparison of the three main approaches, in M. Kuhl, N. Steiger, F. Argstrong & J. Joines, eds,'Proc. 2005 Winter Simulation Conference', IEEE, Inc., Piscataway, NJ, pp. 708-717.
34. Branke, J., Chick, S. E. & Schmidt, C. (2007), 'Selecting a Selection Procedure', Management Science 53, 1916-1932.
35. Brezzi, M. & Lai, T. L. (2002), 'Optimal learning and experimentation in bandit problems', Journal of Economic Dynamics and Control 27(1), 87-108.
36. Broder, J. & Rusmevichientong, P. (2010a), Dynamic pricing under a general parametric choice model, Technical report, Cornell University.
37. Broder, J. & Rusmevichientong, P. (2010b), Dynamic pricing under a logit choice model, Technical report, Cornell University.
38. Bruss, F. (1984), 'A unified approach to a class of best choice problems with an unknown number of options', The Annals of Probability 12(3), 882-889.
39. Bubeck, S. (2010), Bandits Games and Clustering Foundations, PhD thesis, Universite Lille.
40. Bubeck, S., Munos, R., Stoltz, G. & Szepesvari, C. (2011), 'X-Armed Bandits', Journal of Machine Learning Research 12, 1655-1695.
41. Bull, A. D. (2011), 'Convergence rates of efficient global optimization algorithms', Submitted for publication.
42. Can, B., Beham, A. & Heavey, C. (2008), A comparative study of genetic algorithm components in simulation-based optimization, in S. Mason, R. Hill, L. Mönch, O. Rose, T. Jefferson & J. Fowler, eds,'Proceedings of the 2008 Winter Simulation Conference', pp. 1829-1837.
43. Carvalho, A. & Puterman, M. (2003), Dynamic pricing and reinforcement learning, in'Proceedings of the 2003 International Joint Conference on Neural Networks', Vol. 4, pp. 2916-2921.
44. Carvalho, A. & Puterman, M. (2005), 'Learning and pricing in an Internet environment with binomial demands', Journal of Revenue and Pricing Management 3(4), 320-336.
45. Cayley, A. (1875), 'Mathematical questions with their solutions, No. 4528', Educational Times.
46. Chade, H. & Schlee, E. E. (2002), 'Another Look at the Radner-Stiglitz Nonconcavity in the Value of Information', Journal of Economic Theory 107(2), 421-452.
47. Chang, H. S., Fu, M. C, Hu, J. & Marcus, S. I. (2007), Simulation-Based Algorithms for Markov Decision Processes, Springer, New York.
48. Chaudhry, S. & Luo, W. (2005), 'Application of genetic algorithms in production and operations management: a review', International Journal of Production Research 43(19), 4083-4101.
49. Chen, C.-H. & Lee, L. H. (2010), Stochastic Simulation Optimization: An Optimal Computing Budget Allocation, World Scientific.
50. Chen, C.-H., Fu, M. C. & Shi, L. (2008), Simulation and optimization, in P. Gray, Z.-L. Chen & S. Raghavan, eds,'2008 TutORials in Operations Research', pp. 247-260.
51. Chen, C.-H., He, D. & Fu, M. C. (2006), 'Efficient Dynamic Simulation Allocation in Ordinal Optimization', IEEE Transactions Automatic Control 51, 2005-2009.
52. Chen, C.-H., Lin, J., Yücesan, E. & Chick, S. E. (2000), 'Simulation budget allocation for further enhancing the efficiency of ordinal optimization', Discrete Event Dynamic Systems 10(3), 251-270.
53. Chhabra, M. & Das, S. (2011), Learning the Demand Curve in Posted-Price Digital Goods Auctions, in'Proceedings of the 10th International Conference on Autonomous Agents and Multi-Agent Systems', pp. 63-70.
54. Chhabra, M. & Das, S. (2012), Dynamic pricing with non-conjugate Pareto priors, Technical report, Rensselaer Polytechnic Institute.
55. Chick, S. E. (2003), Expected opportunity cost guarantees and indifference zone selection procedures, in S. E. Chick, P. J. Sánchez, D. Ferrin & D. J. Morrice, eds,'Proceedings of the 2003 Winter Simulation Conference', pp. 465-473.
56. Chick, S. E. (2006), Subjective Probability and Bayesian Methodology, in S. Hen-derson & B. Nelson, eds,'Handbooks of Operations Research and Management Science, vol. 13: Simulation', North-Holland Publishing, Amsterdam, pp. 225-258.
57. Chick, S. E. & Gans, N. (2009), 'Economic analysis of simulation selection problems', Management Science 55(3), 421-37.
58. Chick, S. E. & Inoue, K. (2001), 'New two-stage and sequential procedures for selecting the best simulated system', Operations Research 49(5), 732-743.
59. Chick, S. E. & Wu, Y. (2005), 'Selection procedures with frequentist expected opportunity cost bounds', Operations Research 53(5), 867-878.
60. Chick, S. E., Branke, J. & Schmidt, C. (2010), 'Sequential Sampling to Myopically Maximize the Expected Value of Information', INFORMS Journal on Computing 22(1), 71-80.
61. Chick, S. E., He, D. H. & Chen, C.-H. (2007), 'Opportunity cost and OCBA selection procedures in ordinal optimization for a fixed number of alternative systems', IEEE Transactions on Systems Man and Cybernetics C37, 951-961.
62. Chvátal, V. (1983), Linear programming, WH Freeman.
63. Cinlar, E. (2011), Probability and Stochastics, Springer.
64. Cohen, J. D., McClure, S. M. & Yu, A. J. (2007), 'Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration.', Philosophical transactions of the Royal Society of London B362(1481), 933-42.
65. Cohn, D., Atlas, L. & Ladner, R. (1994), 'Improving generalization with active learning', Machine Learning 5(2201), 221.
66. Cohn, D., Ghahramani, Z. & Jordan, M. (1996), 'Active learning with statistical models', Journal of Artificial Intelligence Research 4, 129-145.
67. Cozzolino, J., Gonzalez-Zubieta, R. & Miller, R. (1965), Markov decision processes with uncertain transition probabilities, Technical Report 11, Operations Research Center, MIT.
68. Crooks, G. (2009), Logistic approximation to the logistic-normal integral, Technical Report 002v2, Lawrence Berkeley National Laboratory.
69. Davis, L. & Mitchell, M. (1991), Handbook of genetic algorithms, Van Nostrand Reinhold.
70. Dayanik, S., Powell, W. B. & Yamazaki, K. (2008), 'Index policies for discounted bandit problems with availability constraints', Adv. in Appl. Probab 40, 377-100.
71. Dearden, R., Friedman, N. & Andre, D. (1999), Model-based Bayesian Exploration, in'Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence', pp. 150-159.
72. Dearden, R., Friedman, N. & Russell, S. (1998), Bayesian Q-learning, in'Proceedings of the 15th National Conference on Artificial Intelligence', pp. 761-768.
73. DeGroot, M. H. (1970), Optimal Statistical Decisions, John Wiley and Sons.
74. Delara, M. & Gilotte, L. (2007), 'A tight sufficient condition for Radner-Stiglitz non-concavity in the value of information', Journal of Economic Theory 137(1), 696-708.
75. Duff, M. (2003), Design for an optimal probe, in'Proceedings of the 20th Interna-tional Conference on Machine Learning', pp. 131-138.
76. Duff, M. & Barto, A. (1996), Local bandit approximation for optimal learning problems, in M. Mozer, M. Jordan & T. Pesche, eds,'Advances in Neural Information Processing Systems', Vol. 9, Cambridge, MA: MIT Press, pp. 1019-1025.
77. Dupacova, J. (1995), 'Multistage Stochastic Programs - The State of the Art and Selected Bibliography', Kybernetica 31, 151-174.
78. Farias, V. & Van Roy, B. (2010), 'Dynamic pricing with a prior on market response', Operations Research 58(1), 16-29.
79. Forrester, A. I. J., Sobester, A. & Keane, A. J. (2008), Engineering design via surrogate modelling: a practical guide, John Wiley and Sons.
80. Frazier, P. I. & Powell, W. B. (2010), 'Paradoxes in Learning and the Marginal Value of Information', Decision Analysis 7(4), 378-403.
81. Frazier, P. I. & Powell, W. B. (2011), 'Consistency of Sequential Bayesian Sampling Policies', SI AM Journal on Control and Optimization 49(2), 712-731.
82. Frazier, P. I., Powell, W. B. & Dayanik, S. (2008), 'A Knowledge Gradient Policy for Sequential Information Collection', SI AM Journal on Control and Optimization 47(5), 2410-2439.
83. Frazier, P. I., Powell, W. B. & Dayanik, S. (2009), 'The Knowledge-Gradient Policy for Correlated Normal Beliefs', INFORMS Journal on Computing 21(4), 599-613.
84. Fu, M. C. (2002), 'Optimization for simulation: Theory vs. practice', INFORMS Journal on Computing 14(3), 192-215.
85. Fu, M. C, Glover, F. & April, J. (2005), Simulation optimization: a review, new developments, and applications, in M. E. Kuhl, N. M. Steiger, F. B. Armstrong & J. A. Joines, eds,'Proceedings of the 2005 Winter Simulation Conference', pp. 83-95.
86. Fu, M. C, Hu, J. Q., Chen, C.-H. & Xiong, X. (2004), Optimal computing budget allocation under correlated sampling, in R. G. Ingalls, M. D. Rossetti, J. S. Smith & B. A. Peters, eds,'Proceedings of the 2004 Winter Simulation Conference', pp. 595-603.
87. Gallego, G. & Van Ryzin, G. (1994), 'Optimal dynamic pricing of inventories with stochastic demand over finite horizons', Management Science 40(8), 999-1020.
88. Gelman, A., Carlin, J. B., Stern, H. S. & Rubin, D. B. (2004), 'Bayesian Data Analysis, 2nd ed', Chapman & Hall, New York p. 63.
89. Geman, S., Bienenstock, E. & Doursat, R. (1992), 'Neural networks and the bias/variance dilemma', Neural computation 4(1), 1-58.
90. Ghaffari-Hadigheh, A. & Terlaky, T. (2006), 'Sensitivity analysis in linear optimization: Invariant support set intervals', European Journal of Operational Research 169(3), 1158-1175.
91. Ginebra, J. & Clayton, M. K. (1995), 'Response Surface Bandits', Journal of the Royal Statistical Society B57, 771-784.
92. Gittins, J. C. (1979), 'Bandit processes and dynamic allocation indices', Journal of the Royal Statistical Society B41(2), 148-177.
93. Gittins, J. C. (1989), 'Multi-armed Bandit Allocation Indices', Wiley and Sons: New York.
94. Gittins, J. C. & Jones, D. M. (1974), A dynamic allocation index for the sequential design of experiments, in J. Gani, ed.,'Progress in statistics', North Holland, Amsterdam, pp. 241-266.
95. Gittins, J. C, Glazebrook, K. D. & Weber, R. R. (2011), Multi-Armed Bandit Allocation Indices, John Wiley & Sons, New York.
96. Glasserman, P. (2004), Monte Carlo Methods in Financial Engineering, Springer-Verlag, New York.
97. Glazebrook, K. D. (1982), 'On the evaluation of suboptimal strategies for families of alternative bandit processes', Journal of Applied Probability 19(3), 716-722.
98. Glazebrook, K. D. & Minty, R. (2009), 'A Generalized Gittins Index for a Class of Multiarmed Bandits with General Resource Requirements', Mathematics of Operations Research.
99. Goldberg, D. E. & Deb, K. (1991), A comparative analysis of selection schemes used in genetic algorithms, in G. Rawlings, ed.,'Foundations of genetic algorithms', Morgan Kaufmann Publishers, San Mateo, CA, pp. 69-93.
100. Gosavi, A. (2003), Simulation-Based Optimization, Kluwer Academic Publishers, Norwell, MA.
101. Gramacy, R. B. & Lee, H. K. H. (2011), 'Optimization under unknown constraints', Arxiv preprint arXiv:1004.4027.
102. Gupta, S. S. & Miescke, K. J. (1994), 'Bayesian look ahead one stage sampling allocations for selecting the largest normal mean', Statistical Papers 35,169-177.
103. Gupta, S. S. & Miescke, K. J. (1996), 'Bayesian look ahead one-stage sampling allocations for selection of the best population', Journal of Statistical Planning and Inference 54, 229-244.
104. Harrison, J., Keskin, N. & Zeevi, A. (2010), Bayesian dynamic pricing policies: Learning and earning under a binary prior distribution, Technical report, Working paper, Columbia and Stanford University.
105. Hastie, T., Tibshirani, R. & Friedman, J. (2009), The elements of statistical learning: data mining, inference and prediction, Springer, New York.
106. Hastie, T., Tibshirani, R., Friedman, J. & Franklin, J. (2005), The elements of statistical learning: data mining, inference and prediction, Vol. 27, Springer, New York.
107. Hedlund, H. E. & Mollaghasemi, M. (2001), A Genetic Algorithm and an Indifference-Zone Ranking and Selection Framework for Simulation Optimization, in B. A. Peters, J. S. Smith, D. J. Medeiros & M. W. Rohrer, eds,'Proceedings of the 2001 Winter Simulation Conference', pp. 417-421.
108. Hong, L. J. & Nelson, B. L. (2006), 'Discrete Optimization via Simulation Using COMPASS', Operations Research 54(1), 115-129.
109. Hong, L. J. & Nelson, B. L. (2009), A Brief Introduction To Optimization Via Simulation, in M. Rosetti, R. Hill, B. Johansson, A. Dunkin & R. Ingalls, eds,'Proceedings of the 2009 Winter Simulation Conference', pp. 75-85.
110. Horrace, W., Marchand, J. & Smeeding, T. (2008), 'Ranking inequality: Applications of multivariate subset selection', Journal of Economic Inequality 6(1), 5-32.
111. Horstein, M. (1963), 'Sequential transmission using noiseless feedback', IEEE Trans-actions on Information Theory 9(3), 136-143.
112. Howard, R. A. (1966), 'Information value theory', IEEE Transactions on systems science and cybernetics 2(1), 22-26.
113. Hu, J. & Hu, P. (2010), An Approximate Annealing Search Algorithm to Global Optimization and its Connection to Stochastic Approximation, in B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan & E. Yücesan, eds,'Proceedings of the 2010 Winter Simulation Conference', pp. 1223-1234.
114. Hu, J. & Hu, P. (2011), 'Annealing Adaptive Search, Cross-Entropy, and Stochastic Approximation in Global Optimization', Naval Research Logistics (to appear).
115. Hu, J. & Wang, C. (2011), Discrete Optimization Via Approximate Annealing Adaptive Search With Stochastic Averaging, in S. Jain, R. R. Creasey, J. Himmelspach, K. P. White & M. C. Fu, eds,'Proceedings of the 2011 Winter Simulation Conference'.
116. Hu, J., Fu, M. & Marcus, S. (2007), 'A model reference adaptive search method for global optimization', Operations Research 55(3), 549-568.
117. Huang, D., Allen, T. T., Notz, W. I. & Zeng, N. (2006), 'Global Optimization of Stochastic Black-Box Systems via Sequential Kriging Meta-Models', Journal of Global Optimization 34(3), 441-466.
118. Inoue, K., Chick, S. E. & Chen, C.-H. (1999), 'An empirical evaluation of several methods to select the best system', ACM Transactions on Modeling and Computer Simulation 9, 381-4-07.
119. Jaakkola, T. & Jordan, M. (2000), 'Bayesian parameter estimation via variational methods', Statistics and Computing 10(1), 25-37.
120. Jedynak, B., Frazier, P. I. & Sznitman, R. (2011), 'Questions with noise: Bayes optimal policies for entropy loss', Journal of Applied Probability (to appear).
121. Jones, D., Schonlau, M. & Welch, W. (1998), 'Efficient global optimization of expensive black-box functions', Journal of Global Optimization 13(4), 455-492.
122. Kaelbling, L. P. (1993), Learning in embedded systems, MIT Press, Cambridge, MA.
123. Kalyanam, K. (1996), 'Pricing decisions under demand uncertainty: A Bayesian mixture model approach', Marketing Science 15(3), 207-221.
124. Kaminski, P., Bryson, A. & Schmidt, S. (1971), 'Discrete square root filtering: A survey of current techniques', IEEE Transactions on Automatic Control 16(6), 727-736.
125. Katehakis, M. & Veinott, A. (1987), 'The Multi-Armed Bandit Problem: Decomposition and Computation', Mathematics of Operations Research 12(2), 262-268.
126. Katz, R. & Ionescu, F. (1977), 'Application of the Free-Wilson Technique to Structurally Related Series of Homologues. Quantitative S tract ure-Activity Relations hip Studies of Narcotic Analgetics', 20(11), 1413-1419.
127. Kearns, M. & Singh, S. (2002), 'Near-optimal reinforcement learning in polynomial time.', Machine Learning 49, 209-232.
128. Kim, S.-H. & Nelson, B. L. (2001), 'A fully sequential procedure for indifference-zone selection in simulation', ACM Trans. Model. Comput. Simul. 11, 251-273.
129. Kim, S.-H. & Nelson, B. L. (2006), 'On the asymptotic validity of fully sequential selection procedures for steady-state simulation', Operations Research 54, 475-488.
130. Kleinberg, R. (2004), Nearly Tight Bounds for the Continuum-Armed Bandit Problem, in L. Saul, Y. Weiss & L. Bottou, eds,'Advances in Neural Information Processing Systems', MIT Press, Cambridge, MA, pp. 697-704.
131. Kleinberg, R., Niculescu-Mizil, A. & Sharma, Y. (2010), 'Regret bounds for sleeping experts and bandits', Machine Learning 80(2), 245-272.
132. Kulkarni, V. (1986), 'Shortest paths in networks with exponentially distributed arc lengths', Networks 16, 255-274.
133. Kushner, H. J. & Yin, G. G. (2003), Stochastic Approximation and Recursive Algorithms and Applications, Springer.
134. Lai, T. L. (1987), 'Adaptive treatment allocation and the multi-armed bandit problem', The Annals of Statistics 15(3), 1091-1114.
135. Lai, T. L. & Robbins, H. (1985), 'Asymptotically Efficient Adaptive Allocation Rules', Advances in Applied Mathematics 6, 4-22.
136. Mandelbaum, A. (1987), 'Continuous multi-armed bandits and multiparameter processes', The Annals of Probability 15(4), 1527-1556.
137. Martin, J. (1967), Bayesian Decision Problems and Markov Chains, John Wiley and Sons.
138. Matejcik, F. & Nelson, B. (1995), 'Two-stage multiple comparisons with the best for computer simulation', Operations Research 43(4), 633-640.
139. McKinsey & Company (2007), 'Reducing U.S. Greenhouse Gas Emissions: How Much at What Cost?', U.S. Greenhouse Gas Abatement Mapping Initiative, Executive Report.
140. Miller, A. (2002), Subset selection in regression, CRC Press.
141. Minka, T. (2001), A family of algorithms for approximate Bayesian inference, PhD thesis, Massachusetts Institute of Technology.
142. Montgomery, D. C. (2008), Design and analysis of experiments (7th ed.), John Wiley and Sons.
143. Negoescu, D. M., Frazier, P. I. & Powell, W. B. (2011), 'The Knowledge-Gradient Algorithm for Sequencing Experiments in Drug Discovery', INFORMS Journal on Computing 23(3), 346-363.
144. Nelson, B. L. & Matejcik, F. J. (1995), 'Using common random numbers for indifference-zone selection and multiple comparisons in simulation', Management Science 41(12), 1935-1945.
145. Nino-Mora, J. (2010), 'Computing a Classic Index for Finite-Horizon Bandits', INFORMS Journal on Computing 23(2), 254-267.
146. Pasupathy, R. & Kim, S. (2011), 'The stochastic root-finding problem: Overview, solutions, and open questions', ACM Transactions on Modeling and Computer Simulation 21(3), 19:1-19:23.
147. Paulson, E. (1964), 'A Sequential Procedure for Selecting the Population with the Largest Mean from k Normal Populations', The Annals of Mathematical Statistics 35, 174-180.
148. Peer, S. & Sharma, D. (2007), 'Finding the shortest path in stochastic networks', Computers and Mathematics with Applications 53, 729-740.
149. Powell, W. B. (2007), Approximate Dynamic Programming: Solving the curses of dimensionality, John Wiley & Sons, Hoboken, NJ.
150. Powell, W. B. (2011), Approximate Dynamic Programming: Solving the curses of dimensionality, 2nd. edn, John Wiley & Sons, Hoboken, NJ.
151. Puterman, M. L. (1994), Markov Decision Processes, 1st edn, John Wiley and Sons, Hoboken.
152. Radner, R. & Stiglitz, J. (1984), A Nonconcavity in the Value of Information, in M. Boyer & R. Kihlstrom, eds,'Bayesian models in economic theory', Vol. 5, North-Holland, Amsterdam, chapter 3, pp. 33-52.
153. Rasmussen, C. E. & Williams, C. K. I. (2006), 'Gaussian Processes for Machine Learning', The MIT Press.
154. Rinott, Y. (1978), 'On two-stage selection procedures and related probability inequalities', Communications in Statistics A7 pp. 799-811.
155. Robbins, H. & Monro, S. (1951), 'A stochastic approximation method', The Annals of Mathematical Statistics 22(3), 400-407.
156. Roberts, C. P. & Casella, G. (2004), Monte Carlo Statistical Methods, Springer-Verlag, New York.
157. Romeijn, H. & Smith, R. (1994), 'Simulated annealing and adaptive search in global optimization', Probability in the Engineering and Informational Sciences 8(4), 571-590.
158. Rothschild, M. (1974), 'A two-armed bandit theory of market pricing', Journal of Economic Theory 9(2), 185-202.
159. Rubinstein, R. Y. & Kroese, D. R (2008), Simulation and the Monte Carlo Method, Wiley-Interscience, New York.
160. Rusmevichientong, R & Tsitsiklis, J. N. (2010), 'Linearly Parameterized Bandits', Mathematics of Operations Research 35(2), 395-411.
161. Ruszczynski, A. & Shapiro, A. (2003), Handbooks in Operations Research and Man-agement Science: Stochastic Programming, Vol. 10, Elsevier, Amsterdam.
162. Ryzhov, I. O. & Powell, W. B. (2009a), A Monte-Carlo Knowledge Gradient Method For Learning Abatement Potential Of Emissions Reduction Technologies, in M. D. Rossetti, R. R. Hill, B. Johansson, A. Dunkin & R. G. Ingalls, eds,'Proceedings of the 2009 Winter Simulation Conference'.
163. Ryzhov, I. O. & Powell, W. B. (2009o), The knowledge gradient algorithm for online subset selection, in'Proceedings of the 2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning', pp. 137-144.
164. Ryzhov, I. O. & Powell, W. B. (201 la),'Information collection for linear programs with unknown objective coefficients', Submitted for publication.
165. Ryzhov, I. O. & Powell, W. B. (2011b),'Information Collection on a Graph', Operations Research 59(1), 188-201.
166. Ryzhov, I. O. & Powell, W. B. (2011c), The value of information in multi-armed bandits with exponentially distributed rewards, in'Proceedings of the 2011 International Conference on Computational Science', pp. 1363-1372.
167. Ryzhov, I. O., Powell, W. B. & Frazier, P. I. (2011), 'The knowledge gradient algorithm for a general class of online learning problems', Operations Research (to appear).
168. Ryzhov, I. O., Valdez-Vivas, M. R. & Powell, W B. (2010), Optimal Learning of Transition Probabilities in the Two-Agent Newsvendor Problem, in B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan & E. Yucesan, eds,'Proceedings of the 2010 Winter Simulation Conference', pp. 1088-1098.
169. Sacks, J., Welch, W, Mitchell, T. J. & Wynn, H. P. (1989), 'Design and analysis of computer experiments', Statistical Science 4(4), 409-423.
170. Satia, J. & Lave, R. (1973), 'Markov decision processes with imprecise transition probabilities', Operations Research 21(3), 755-763.
171. Scott, W. R., Frazier, P. I. & Powell, W. B. (2011), 'The Correlated Knowledge Gradient for Simulation Optimization of Continuous Parameters using Gaussian Process Regression', SI AM Journal on Optimization 21(3), 996-1026.
172. Scott, W. R., Powell, W. B. & Simão, H. P. (2010), Calibrating simulation models using the knowledge gradient with continuous parameters, in B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan & E. Yiicesan, eds,'Proceedings of the 2010 Winter Simulation Conference', pp. 1099-1109.
173. Settles, B. (2009), Active learning literature survey, Computer Sciences Technical Report 1648, University of Wisconsin-Madison.
174. Si, J., Barto, A. G., Powell, W. B. & Wunsch, D. (2005), Learning and Approximate Dynamic Programming.
175. Silver, E. (1963), Markovian decision processes with uncertain transition probabilities or rewards, Technical Report 1, Operations Research Center, MIT.
176. Singh, S. P., Jaakkola, T., Szepesvari, C. & Littman, M. (2000), 'Convergence results for single-step on-policy reinforcement-learning algorithms', Machine Learning 38(3), 287-308.
177. Snyder, T. & Steele, J. (1995), Probabilistic networks and network algorithms, in M. Ball, T. Magnanti & C. Monma, eds,'Handbooks of Operations Research and Management Science, vol. 7: Networks', North-Holland Publishing, Amsterdam, pp. 401-424.
178. Spall, J. C. (2003), Introduction to Stochastic Search and Optimization: Estimation, Simulation and Control, John Wiley & Sons, Hoboken, NJ.
179. Srinivas, N., Krause, A., Kakade, S. M. & Seeger, M. (2010), Gaussian process optimization in the bandit setting: No regret and experimental design, in'Proceedings of the 27th International Conference on Machine Learning', pp. 1015-1022.
180. Steele, J. M. (2000), Stochastic Calculus and Financial Applications, Springer, New York.
181. Stein, M. (1999), Interpolation of Spatial Data: Some theory for kriging, Springer Verlag.
182. Streeter, M. & Smith, S. (2006), A simple distribution-free approach to the max k-armed bandit problem, in'Principles and Practice of Constraint Programming', Vol. 4204 of Lecture Notes in Computer Science, pp. 560-574.
183. Sutton, R. S. & Barto, A. G. (1998), Reinforcement Learning, Vol. 35, MIT Press, Cambridge, MA.
184. Sutton, R. S., Maei, H. R., Precup, D., Bhatnagar, S., Silver, D., Szepesvari, C. & Wiewiora, E. (2009), Fast gradient-descent methods for temporal-difference learning with linear function approximation, in'Proceedings of the 26th International Conference on Machine Learning', pp. 993-1000.
185. Szepesvári, C. (2010), Algorithms for reinforcement learning, Vol. 4 of Synthesis Lectures on Artificial Intelligence and Machine Learning, Morgan & Claypool Publishers.
186. Takimoto, E. & Warmuth, M. K. (2003), 'Path kernels and multiplicative updates', Journal of Machine Learning Research 4, 773-818.
187. Tesauro, G. (1992), 'Practical Issues in Temporal Difference Learning', Machine Learning 8(3-4), 257-277.
188. Vanderbei, R. J. (1980), 'The optimal choice of a subset of a population', Mathematics of Operations Research 5(4), 481-186.
189. Vanderbei, R. J. (2008), Linear programming: foundations and extensions (3rd ed.), Springer.
190. Vermorel, J. & Mohri, M. (2005), 'Multi-armed bandit algorithms and empirical evaluation', Proceedings of the 16th European Conference on Machine Learning pp. 437-448.
191. Wald, A. & Wolfowitz, J. (1948), 'Optimum Character of the Sequential Probability Ratio Test', The Annals of Mathematical Statistics 19, 326-339.
192. Weber, R. R. & Weiss, G. (1990), 'On an index policy for restless bandits', J. Appl. Prob. 27(3), 637-648.
193. Weibull, J. W., Mattsson, L.-G. & Voorneveld, M. (2007), 'Better May be Worse: Some Monotonicity Results and Paradoxes in Discrete Choice Under Uncertainty', Theory and Decision 63(2), 121-151.
194. Wetherill, G. B. & Glazebrook, K. D. (1986), Sequential methods in statistics, Chap-man & Hall.
195. Whittle, P. (1980), 'Multi-armed bandits and the Gittins index', Journal of the Royal Statistical Society B42(2), 143-149.
196. Whittle, P. (1988), 'Restless bandits: Activity Allocation in a Changing World', J. Appl. Prob. 25(1988), 287-298.