Restless bandit marginal productivity indices, diminishing returns, and optimal control of make-to-order/make-to-stock M/G/1 queues

Mathematics of Operations Research

Volumn 31, Issue 1, 2006, Pages 50-84

  Niño Mora, José ^a  

^a UNIVERSIDAD CARLOS III DE MADRID   (Spain)

Author keywords

Bias; Control by price; Convex optimization; Diminishing returns; Dynamic resource allocation; Efficient frontier; Index policies; Indexability; Marginal productivity; Mixed criteria; Restless bandits; Semi Markov decision processes; Stochastic scheduling

Indexed keywords

FINITE DIFFERENCE METHOD; INDUSTRIAL ECONOMICS; MARKOV PROCESSES; OPTIMAL CONTROL SYSTEMS; OPTIMIZATION; PROBLEM SOLVING; QUEUEING THEORY; RANDOM PROCESSES; RESOURCE ALLOCATION;

CONTROL BY PRICE; CONVEX OPTIMIZATIONS; DIMINISHING RETURNS; DYNAMIC RESOURCE ALLOCATION; INDEX POLICIES; INDEXABILITY; RESTLESS BANDITS; SEMI-MARKOV DECISION PROCESSES; STOCHASTIC SCHEDULING;

OPERATIONS RESEARCH;

EID: 33646720393     PISSN: 0364765X     EISSN: 15265471     Source Type: Journal    
DOI: 10.1287/moor.1050.0165     Document Type: Article

References (51)

1. Ansell, P. S., K. D. Glazebrook, J. Niño-Mora, M. O'Keeffe. 2003. Whittle's index policy for a multi-class queueing system with convex holding costs. Math. Methods Oper. Res. 57 21-39.
2. Bell, C. E. 1971. Characterization and computation of optimal policies for operating an M/G/1 queue with removable server. Oper. Res. 19 208-218.
3. Bertsekas, D. P. 1987. Dynamic Programming: Deterministic and Stochastic Models. Prentice-Hall, Englewood Cliffs, NJ.
4. Bertsimas, D., J. Niño-Mora. 1996. Conservation laws, extended polymatroids and multiarmed bandit problems; A polyhedral approach to indexable systems. Math. Oper. Res. 21 257-306.
5. Bertsimas, D., J. Niño-Mora. 2000. Restless bandits, linear programming relaxations, and a primal-dual index heuristic. Oper. Res. 48 80-90.
6. Blackwell, D. 1962. Discrete dynamic programming. Ann. Math. Statist. 33 719-726.
7. Buzacott, J. A., J. G. Shanthikumar, 1993. Stochastic Models of Manufacturing Systems. Prentice-Hall, Englewood Cliffs, NJ.
8. Coffman, E. G., Jr., I. Mitrani. 1980. A characterization of waiting time performance realizable by single-server queues. Oper. Res. 28 810-821.
9. Cox, D. R., W. L. Smith. 1961. Queues. Methuen, London, UK.
10. Dacre, M., K. Glazebrook, J. Niño-Mora. 1999. The achievable region approach to the optimal control of stochastic systems. J. Roy. Statist. Soc., Ser: B, Statist. Methodology 61 747-791.
11. D'Epénoux, F. 1960. Sur un problème de production et de stockage dans l'aléatoire. RAIRO Rech. Opér. 14 3-16
12. [English translation. 1963. A probabilistic production and inventory problem. Management Sci. 10 98-108].
13. De Véricourt, F., F. Karaesmen, Y. Dallery. 2000. Dynamic scheduling in a make-to-stock system: A partial characterization of optimal policies. Oper. Res. 48 811-819.
14. Dusonchet, F., M.-O. Hongler. 2003. Continuous-time restless bandit and dynamic scheduling for make-to-stock production. IEEE Trans. Robotics Automation 19 977-990.
15. Edmonds, J. 1971. Matroids and the greedy algorithm. Math. Programming 1 127-136.
16. Federgruen, A., H. Groenevelt. 1988. Characterization and optimization of achievable performance in general queueing systems. Oper. Res. 36 733-741.
17. Feinberg, E. A., A. Shwartz. 2002. Mixed criteria. E. A. Feinberg, A. Shwartz, eds. Handbook of Markov Decision Processes: Methods and Applications. Kluwer, Boston, MA, 209-230.
18. Gittins, J. C. 1979. Bandit processes and dynamic allocation indices. J. Roy. Statist. Soc., Ser: B 41 148-177.
19. Gittins, J. C., D. M. Jones. 1974. A dynamic allocation index for the sequential design of experiments. Progress in Statistics, European Meeting of Statisticians, Budapest, 1972. North-Holland, Amsterdam, The Netherlands, 241-266;
20. Colloquia Math. Soc. János Bolyai, Vol. 9.
21. Glazebrook, K. D., J. Niño-Mora. 2001. Parallel scheduling of multiclass M/M/m queues: Approximate and heavy-traffic optimization of achievable performance. Oper. Res. 49 609-623.
22. Ha, A. Y. 1997. Optimal dynamic scheduling policy for a make-to-stock production system. Oper. Res. 45 42-53.
23. Kantorovich, L. V. 1959. Ekonomicheskii Raschet Nailuchshego Ispolzovania Resursov. Academy of Sciences USSR
24. [English translation. 1965. The Best Use of Economic Resources. Harvard University Press. Cambridge, MA].
25. Kleinrock, L. 1965. A conservation law for a wide class of queueing disciplines. Naval Res. Logist. Quart. 12 181-192.
26. Kleinrock, L. 1975. Queueing Systems. Vol. I: Theory. Wiley, New York.
27. Klimov, G. P. 1974. Time-sharing service systems. I. Theory Probab. Appl. 19 532-551.
28. Koopmans, T. C. 1957. Three Essays on the State of Economic Science, Chapter on Allocation of resources and the price system. McGraw-Hill, New York.
29. Lewis, M. E., M. L. Puterman. 2002. Bias optimality. E. Feinberg, A. Schwartz, eds. Handbook of Markov Decision Processes. Kluwer, Boston, MA, 89-111.
30. Manne, A. S. 1960. Linear programming and sequential decisions. Management Sci. 6 259-267.
31. Marshall, K. T., R. W. Wolff. 1971. Customer average and time average queue lengths and waiting times. J. Appl. Probab. 8 535-542.
32. Morse, P. M. 1958. Queues, Inventories and Maintenance: The Analysis of Operational Systems with Variable Demand and Supply. Wiley, New York.
33. Niño-Mora, J. 2000. Restless bandit index heuristics for multiclass queueing network control. Presentation Conf. Stochastic Networks, University of Wisconsin-Madison, Madison, WI, June 19-24.
34. Niño-Mora, J. 2001. Restless bandits: PCL-indexability conditions and queueing control applications. Presentation, 27th Conf. Stochastic Processes and Their Applications, Cambridge, UK, July 9-13.
35. Niño-Mora, J. 2001. Restless bandits, partial conservation laws and indexability. Adv. Appl. Probab. 33 76-98.
36. Niño-Mora, J. 2001. Stochastic scheduling. C. A. Floudas, P. M. Pardalos, eds. Encyclopedia of Optimization, Vol. 5. Kluwer, Dordrecht, The Netherlands, 367-372.
37. Niño-Mora, J. 2002. Dynamic allocation indices for restless projects and queueing admission control: A polyhedral approach. Math. Programming 93 361-413.
38. Niño-Mora, J. 2003. Restless bandit marginal productivity indices, diminishing returns and scheduling a multiclass make-to-order/-stock queue. R. Srikant, V. V. Veeravalli, eds. Proc. 41st Annual Allerton Conf. Comm., Control Comput., Monticello, IL, 100-109.
39. Niño-Mora, J. 2004. Restless bandit dynamic allocation indices II: Multi-project case and dynamic scheduling of a multiclass make-to-order/-stock M/G/1 queue. Working Paper 04-09, Department of Statistics, Universidad Carlos III de Madrid, Spain, February.
40. Papadimitriou, C. H., J. N. Tsitsiklis. 1999. The complexity of optimal queuing network control. Math. Oper. Res. 24 293-305.
41. Peña-Pérez, A., P. Zipkin. 1997. Dynamic scheduling rules for a multiproduct make-to-stock queue. Oper. Res. 45 919-930.
42. Puterman, M. L. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York.
43. Ross, S. M. 1970. Average cost semi-Markov decision processes. J. Appl. Probab. 7 649-656.
44. Shanthikumar, J. G., D. D. Yao. 1992. Multiclass queueing systems: Polymatroidal structure and optimal scheduling control. Oper. Res. 40 S293-S299.
45. Tsoucas, P. 1991. The region of achievable performance in a model of Klimov. Tech. Report RC16543, IBM T. J. Watson Research Center, Yorktown Heights, New York.
46. Veatch, M. H., L. M. Wein. 1996. Scheduling a multiclass make-to-stock queue: Index policies and hedging points. Oper. Res. 44 634-647.
47. Wein, L. M. 1992. Dynamic scheduling of a multiclass make-to-stock queue. Oper. Res. 40 724-735.
48. Weiss, G. 1988. Branching bandit processes. Probab. Engrg. Inform. Sci. 2 269-278.
49. Weitzman, M. L. 2000. An "economics proof" of the supporting hyperplane theorem. Econom. Lett. 68 1-6.
50. Whittle, P. 1988. Restless bandits: Activity allocation in a changing world. J. Gani, ed. A Celebration of Applied Probability, J. Appl. Probab. Special. Vol. 25A. Applied Probability Trust, Sheffield, UK. 287-298.
51. Whittle, P. 1996. Optimal Control: Basics and Beyond. Wiley, Chichester, UK.