A survey on the bandit problem with switching costs

Economist

Volumn 152, Issue 4, 2004, Pages 513-541

  Jun, Tackseung ^a  

^a Kyung Hee University   (South Korea)

Author keywords

Decomposability; Multi armed bandits; Switching costs

Indexed keywords

DECOMPOSITION; METHODOLOGY;

EID: 10944236938     PISSN: 0013063X     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10645-004-2477-z     Document Type: Article

References (115)

1. P.S. Adler A Mandeldaum V. Nguyen R. Schwerer 1995 'From Project to Process Management: An Empirically-Developed Framework for Analyzing Product Development Time' Management Science. 41 458-484
2. P. Aghion P. Bolton C. Harris 1991 'Optimal Learning by Experiment' Review of Economic Studies. 58 621-654
3. R. Agrawal M. Hegde D. Teneketzis 1988 'Asymptotically Efficient Allocations Rules for Multi-armed Bandit Problem with Switching Cost' IEEE Transactions on Automatic Control AC-32 968 982
4. R. Agrawal M. Hegde D. Teneketzis 1990 'Multi-armed Bandit Problems with Multiple Plays and Switching Cost' Stochastics and Stochastic Reports. 29 437-459
5. Asawa, M. and D. Teneketzis (1996), 'Multi-Armed Bandits with Switching Penalties,' IEEE Transactions on Automatic Control, 41, pp. 328-348.
6. Azoulay-Schwartz, R., S. Kraus, and J. Wilkenfeld (2003), 'Exploitation vs. Exploration: Choosing a Supplier in an Environment of Incomplete Information,' mimeo.
7. Banks, J.S. (2003), 'Generalized Bandit Problems', mimeo.
8. J.S. Banks R.K. Sundaram 1992 'Denumerable-Armed Bandits' Econometrica 60 1071-1096
9. J.S. Banks R.K. Sundaram 1994 'Switching Costs and the Gittins index' Econometrica. 62 687-694
10. J.M. Barron D.A. Black M.A. Loewenstein 1993 'Gender Difference in Training, Capital, and Wages' Journal of Human Resources. 28 343-364
11. A. Basu A. Bose J.K. Ghosh 1990 An Expository Review of Sequential Design and Allocation Rules Technical Report 90-08, Department of Statis- tics, Purdue University
12. R. Bellman 1956 'A Problem in the Sequential Design of Experiments' Sankhya. 16 221-229
13. Bellman, R. (1957), Dynamic Programming, New Jersey, Princeton University Press.
14. L. Benkherouf 1990 'Optimal Stopping in Oil Exploration with Small and Large Oilfields' Probability in Engineering and Information Sciences. 28 529-543
15. L. Benkherouf J.A. Bather 1988 'Oil Exploration: Sequential Decisions in the Face of Uncertainty' Journal of Applied Probability 28 529-543
16. Benkherouf, L., K.D. Glazebrook, and R.W. Owen (1992), 'Gittins Indices and Oil Exploration,' Journal of Royal Statistical Society Serial B, 54, pp. 229-241.
17. Bergemann, D. and J. Välimäki, (1999), 'Entry and Innovation in Vertically Differentiated Markets', mimeo.
18. D. Bergemann J. Välimäki 2001 'Stationary Multi Choice Bandit Problems' Journal of Economic Dynamics and Control. 25 1585-1594
19. S. Berninghaus V. Seifert G. Hans 1987 'International Migration under Incomplete Information' Schweizerische Zeitschrift fur Volkswirtschaft und Statistik. 123 199-218
20. D. Bertsimas J. Niño-Mora 1996 'Conservation Laws, Extended Polymatroids and Multi-armed Bandit Problems' Mathematics of Operations Research. 21 257-306
21. D. Bertsimas J. Niño-Mora 2000 'Restless Bandits, Linear Programming Relaxations, and A Primal-Dual Index Heuristic' Operations Research 48 80-90
22. D.A. Berry 1972 'A Bernoulli Two-Armed Bandit' Annals of Mathematical Statistics. 43 871-897
23. Berry, D.A. and B. Fristedt (1985), Bandit Problems: Sequential Allocation of Experiments, London, Chapman and Hall.
24. Black, D.A. and M.A. Loewenstein (1991), 'Self-enforcing Labor Contracts with Costly Mobility,' Review of Labor Economics, 12, pp. 63-83
25. Brenner, T. and N.J. Vriend, (2003), 'On the Behavior of Proposers in Ultimatum Games', mimeo.
26. Brent, R.P. (1973), Algorithms for Minimization Without Derivatives, New Jersey, Prentice-Hall.
27. Brezzi, M. and T.L. Lai (2002), 'Optimal Learning and Experimentation in Ban- dit Problems,' Journal of Economic Dynamics and Control, 27, pp. 87-108.
28. Cvitanić, J., L. Martellini, and F. Zapatero, (2002), 'Optimal Active Management Fees', in: E. Yücesan, C.H. Chen, J.L. Snowdon and J.M. Charnes, (eds.), Proceedings of the 2002 Winter Simulation Conference
29. R. Cowan. 1991 'Tortoises and Hares: Choice Among Technologies of Unknown Merit' Economic Journal. 407 801-814
30. Cox, D.R. and W.L. Smith (1961), Queues, Monographs on Statistics and Applied Probability 2, New York, Chapman&Hall.
31. C. Derman 1962 'On Sequential Decision and Markov Chains' Management Science. 9 16 24
32. Duenyas, I. and M.P. Van Oyen (1996), 'Heuristic Scheduling of Parallel Heterogeneous Queues with Set-Ups,' Management Science, 42, pp. 814-829.
33. Dusonchet, F. and M.O. Hongler, (2000), 'Continuous time Restless Bandit an Dynamic Scheduling for Make-to-Stock Production', mimeo.
34. F. Dusonchet M.O. Hongler 2003 'Optimal Hysteresis for a Class of Deterministic Deteriorating Two-Armed Bandit Problem with Switching Costs' Automatica. 39 1947-1955
35. Easley, D. and N. Kiefer (1988), 'Controlling a Stochastic Process with Unknown Parameters,' Econometrica, 56, pp. 1045-1064.
36. Ehsan, N. and M. Liu, (2004), 'On the Optimality of an Index Policy for Bandwidth Allocation with Delayed State Observation and Differentiated Services', IEEE INFOCOM Conference
37. N. El Karoui I. Karatzas 1997 'Synchronization and Optimality for Multiarmed Bandit Problems in Continuous Time' Computational and Applied Mathematics. 16 117-151
38. M. Eswaran 1994 'Licensees as Entry Barriers' Canadian Journal of Economics. 27 673-688
39. E. Frostig G. Weiss 1999 'Four Proofs of Gittins' Multiarmed Bandit Theorem' Applied Probability Trust, 1-20
40. J. Gale K. Binmore L. Samuelson 1995 'Learning to be imperfect: The Ultimatum Games' Games and Economic Behavior 8 56-90
41. J.C. Gittins D.M. Jones 1974 'A Dynamic Allocation Index for the Sequential Design of Experiments'. in: European Meeting of Statisticians, J Gani, K. Sarkadi and I. Vince, (eds.), Progress in Statistics. Amsterdam, North-Holland 241-266
42. J.C. Gittins 1979 'Bandit Processes and Dynamic Allocation Indices' Journal of Royal Statistical Society Serial B. 14 148-177
43. Gittins, J.C. (1989), Multi-armed Bandit Allocation Indices, New York, Wiley.
44. J.M. Harrison J.A. Van Mieghem 1999 'Multi-resource Investment Strategies: Operational Hedging under Demand Uncertainty' European Journal of Operational Research. 113 17-29
45. H. He R.S. Pindyck 1992 'Investment in Flexible Production Capacity' Journal of Economic Dynamics and Control. 16 575-599
46. A.T. Ishikida P. Varaiya 1994 'Multi-armed Bandit Problem Revisited' Journal of Optimization Theory and Applications. 83 113-154
47. Johnson, W.R. (1978), 'A Theory of Job Shopping,' Quarterly Journal of Economics, 92, pp. 261-278.
48. Jonsson M. and J. Vecer, (2004), 'Insider Trading in Convergent Markets', mimeo.
49. B. Jovanovic 1979 'Job Matching and the Theory of Turnover' Journal of Political Economy. 87 972-990
50. Jovanovic, B. (1984), 'Matching, Turnover, and Unemployment,' Journal of Political Economy, 92, pp. 108-122.
51. Jun, T. (2001), Essays on Decision Theory: Effects of Changes in Environment on Decision, Ph.D. thesis, Columbia University, New York.
52. Jung, A. (2003), 'Are Product Innovation and Flexible Technology Complements?', mimeo.
53. F. Karaesmen S.M. Gupta 1997 'Control of Arrivals in a Finite Buffered Queue with Setup Costs' Journal of the Operational Research Society. 48 1113-1122
54. Karatzas, I. (1984), 'Gittins Indices in the Dynamic Allocation Problem for Diffusion Processes,'' Annals of Probability, 12, pp. 173-192.
55. Kast, R., A. Lapeid and S. Pardo, (2003), 'Virtual Underlying Security', mimeo.
56. Kavadias, S.K. and C.H. Loch, (2000), Dynamic Resource Allocation Policy in Multiproject Environments, INSEAD Working Papers, 2000/10/TM.
57. G. Keller A. Oldale 2003 'Branching Bandits: A Sequential Search Process with Correlated Pay-offs' Journal of Economic Theory. 113 302-315
58. Keller, G. and S. Rady (1999), 'Optimal Experimentation in a Changing Environment,' Review of Economic Studies, 66, pp. 475-507.
59. Kennan, J. and J.R. Walker, (2003), The Ef́lect of Expected Income on Individual Migration Decisions, NBER Working Papers 9585.
60. Klimenko, M.M. (2003), 'Industrial Targeting, Experimentation and Long-run Specialization', Journal of Development Economics, forthcoming.
61. Kolonko, M. and H. Benzing (1985), 'The Sequential Design of Bernoulli Experiments Including Switching Costs,' Operations Research, 2, pp. 412-426.
62. Koole G. (1997), 'Assigning a Single Server to Inhomogeneous Queues with Switching Costs', mimeo.
63. D. Krähmer 2003 'Entry and Experimentation in Oligopolistic Markets for Experience Goods'. International Journal of Industrial Organization. 21 1201-1213
64. P. Kuhn 1993 'Demographic Groups and Personnel Policy' Labour Economics. 1 49-70
65. N. Kulatilaka 1988 'Valuing the Flexibility of Flexible Manufacturing Systems' IEEE Transactions on Engineering Management. 35 250-257
66. T.L. Lai 1987 'Adaptive Treatment Allocation and theMulti-Armed Bandit Problem' Annals of Statistics. 15 1091-1114
67. T.L. Lai H. Robbins 1985 'Asymptotically Efficient Adaptive Allocation Rules' Advances in Applied Mathematics 6 4-42
68. A.H. Land A.G. Doig 1960 'An Automatic Method for Solving Discrete Programming Problems'Econometrica 28 497-520
69. S.A. Lippman J.W. Mamer 1993 'Preemptive Innovation' Journal of Economic Theory. 61 104-119
70. C.H. Loch S.K. Kavadias 2002 'Dynamic Portfolio Selection of NPD Programs Using Marginal Returns' Management Science: 48 1227-1241
71. C. Lott D. Tenekezis 2000 'On the Optimality of An Index Rule in Mul- tichannel Allocation for Single-Hop Mobile Networks with Multiple Service Classes' Probability in the Engineering and Informational Sciences. 14 259-197
72. G.M. MacDonald 1980 'Person-Speci.c Information in the Labor Market' Journal of Political Economy. 92 1086-1120
73. A. Manne 1960 'Linear Programming and Sequential Decisions' Management Science. 6 259-267
74. B.P. McCall J.J. McCall 1987 'A Sequential Study of Migration and Job Search' Journal of Labor Economics. 5 452-476
75. R. McDonald D. Siegel 1986 'The Value of Waiting to Invest' Quarterly Journal of Economics. 101 707-728
76. A. McLennan 1984 'Price Dispersion and Incomplete Learning in the Long Run' Journal of Economic Dynamics and Control. 7 331-347
77. R.A. Miller 1984 'Job Matching and Occupational Choice' Journal of Political Economy. 92 1086-1120
78. D.T. Mortensen 1988 'Wages, Separations, and Job Tenure: On-the-Job Speci.c Training or Matching?' Journal of Labor Economics. 6 445-471
79. R. Murnane F. Levy J. Willett 1995 The Growing Importance of Cognitive Skills in Wage Determination NBER Working Papers 50-76
80. Niño-Mora, J. (2001), 'Restless Bandit, Partial Conservation Laws and Indexibility,' Advances in Applied Probability, 33, pp. 77-98.
81. Niño-Mora, J. (2004a). Restless Bandit Marginal Productivity Indices I: Single Project Case and Optimal Control of a Make-To-Stock M/G/1 Queue, Universidad Carlos III, Departamento de Estadìstica y Econometría, WS040801.
82. Niño-Mora, J. (2004b). Restless BanditMarginal Productivity Indices II: Rest-less Bandit Marginal Productivity Indices II: Multi-project Case and Scheduling a Multiclass Make To-Order/-Stock M/G/1 Queue, Universidad Carlos III, Departamento de Estadìstica y Econometría, WS040902.
83. O'Flaherty, B. (1987). Some Results on Two-Armed Bandits When Both Projects Vary, Columbia Department of Economics Working Paper No. 359.
84. D.G. Pandelis D. Teneketzis 1995 'On the Optimality of the Gittins Index Rule in Multi-armed Bandits with Multiple Plays' Proceedings of the Conference on Decision & Control WP13 5 30 1408-1414
85. Press, W.H., B.P. Flannery, S.A. Teukolsky and W.T. Vetterling (1992), 'Secant Method, False Position Method, and Ridders' Method,' in: The Art of Scientific Computing, 2nd ed., England, Cambridge University Press. pp. 347-352.
86. Puterman, M.L. (1994), Makov Decision Processes: Discrete Stochastic Dynamic Programming, New York, Wiley.
87. M.I. Reiman L.M. Wein 1998 'Dynamic Scheduling of a Two-Class Queue with Setups' Operations Research. 46 532-547
88. Ross, S. (1983), Introduction to Stochastic Dynamic Programming, New York Academic Press.
89. M. Rothschild 1974 'A Two-armed Bandit Theory of Market Pricing' Journal of Economic Theory. 9 185-202
90. A. Rustichini A. Wolinsky 1995 'Learning about Variable Demand in the Long Run' Journal of Economic Dynamics and Control 19 1283-1292
91. C. Santos M. Magazine 1985 'Batching in Single Operation Manufacturing System' Operations Research Letters. 4 99-103
92. Schlag, K.H. (2003), 'How to Minimize Maximum Regret under Repeated Decision-Making', mimeo.
93. Schlag, K.H. (2003), 'How to Minimize Maximum Regret under Repeated Decision-Making,'mimeo.
94. Smith L. (1995), 'Optimal Job Search in a Changing World', mimeo.
95. Smith. L. and P. Sorensen (2001), 'Informational Herding and Optimal Experimentation, University of Michigan,' mimeo.
96. Smith, W.E. (1956), 'Various Optimizers for Single-State Production,' Naval Research Logistics Quarterly, 3, pp. 59-66.
97. Soberman, D.A. (1999), 'Joint Research and Development: The Lure of Dominance', INSEAD Working Paper, 99/18/MKT
98. Subramanian, A. (2002), 'Managerial Flexibility, Agency Costs and Optimal Capital Structure', mimeo.
99. Thaler, R.H. (1988), 'The Ultimatum Game,' Journal of Economic Perspective, 2, pp. 195-206.
100. Tuluca S. and P. Stalinski, (2004), 'The Manufacturing Flexibility to Switch Products: Valuation and Optimal Strategy' mimeo.
101. Van Oyen M.P. and J. Pichitlamken, (1999), 'Properties of Optimal Weighted Flowtime Policies with a Makespan Constraint and Set-up Times', mimeo.
102. M.P. Van Oyen D.G. Pandelis D. Teneketzis 1992 'Optimality of Index Policies for Stochastic Scheduling with Switching Penalties' Journal of Applied Probability. 29 957-966
103. Van Oyen, M.P., D.G. Pandelis and D. Teneketzis (1992), 'Optimality of Index Policies for Stochastic Scheduling with Switching Penalties,' Journal of Applied Probability, 29, pp. 957-966.
104. Van Oyen, M.P. and J. Pichitlamken (1999), 'Properties of Optimal Weighted Flowtime Policies with a Makespan Constraint and Set-up Times,' Department of Industrial Engineering and Management Science, mimeo.
105. W.K. Viscusi 1980 'A Theory of Job Shopping: A Bayesian Perspective' Quarterly Journal of Economics. 94 609-614
106. M. Waldman 1984 'Job Assignments, Signaling and Efficiency' Rand Journal of Economics. 15 255-267
107. R.R. Weber 1992 'On the Gittins index for Multiarmed Bandits' Annals of Probability. 2 1024-1033
108. R.R. Weber G. Weiss 1990 'On an Index Policy for Restless Bandits' Journal of Applied Probability. 27 637-648
109. G. Weiss 1995 'On Almost Optimal Priority Rules for Preemptive Schedul- ing of Stochastic Jobs on Parallel Machines' Advances in Applied Probability. 27 827-845
110. Weitzman, M.L. (1979), 'Optimal Search for the Best Alternative,' Econometrica, 47, pp. 641-654.
111. Wilk S. and P. Sackett, (1995), 'A Longitudinal Analysis of Ability-Job Complexity Fit and Job Change'. mimeo.
112. S. Wilk L. Desmaris P. Sackett 1995 'Gravitation to Jobs Commensurate with Ability: Longitudinal and Cross-Sectional Tests' Journal of Applied Psychology. 80 79-85
113. P. Whittle 1980 'Multi-armed Bandits and the Gittins Index' Journal of Royal Statistical Society Serial B. 42 143-149
114. Whittle, P. (1982), Optimization Over Time: Dynamic Programming and Stochastic Control, Vol. 1, New York, Wiley.
115. P. Whittle 1988 'Restless Bandits: Activity Allocation in a Changing World', Journal of Applied Probability. 25 287-298