Discrete-review policies for scheduling stochastic networks: Trajectory tracking and fluid-scale asymptotic optimality

Annals of Applied Probability

Volumn 10, Issue 3, 2000, Pages 897-929

  Maglaras, Constantinos ^a  

^a The Sanford C Bernstein Co Center for Leadership and Ethics   (United States)

Author keywords

Asymptotic optimality; Discrete review policies; Fluid models; Queueing networks; Trajectory tracking

Indexed keywords

EID: 0034346720     PISSN: 10505164     EISSN: None     Source Type: Journal    
DOI: 10.1214/aoap/1019487513     Document Type: Article

References (50)

1. ATHANS, M. and FALB, P. (1966). Optimal Control. McGraw-Hill, New York.
2. ATKINS, D. and CHEN, H. (1995). Performance evaluation of scheduling control of queueing networks: fluid model heuristics. Queueing Sys. 21 391-413.
3. AVRAM, F., BERTSIMAS, D. and RICARD, M. (1995). Fluid models of sequencing problems in open queueing networks; an optimal control approach. In Stochastic Networks, Proceedings of the IMA(F. Kelly and R. Williams, eds.) 71 199-234. Springer, New York.
4. BAMBOS, N. and WALRAND, J. (1993). Scheduling and stability aspects of a general class of parallel processing systems. Adv. in Appl. Probab. 25 176-202.
5. BERTSEKAS, D. P. (1995). Dynamic Programming and Optimal Control 1. Athena Scientific, Belmont, MA.
6. BERTSIMAS, D., GAMARNIK D. and SETHURAMAN, J. (1999). Asymptotically optimal solutions for job shop scheduling problems. Technical report, Operations Research Center, MIT.
7. BERTSIMAS, D., PASCHALIDIS, I. C. and TSITSIKLIS, J. N. (1995). Branching bandits and Klimov's problem: achievable region and side constraints. IEEE Trans. Automat. Control 40 2063-2075.
8. BERTSIMAS, D. and VAN RYZIN, G. (1991). A stochastic and dynamic vehicle routing problem in the Euclidean plane. Oper. Res. 39 601-615.
9. BRAMSON, M. (1998). Stability of two families of queueing networks and a discussion of fluid limits. Queueing Sys. 28 7-31.
10. BRYSON, A. E. and Ho, Y. C. (1975). Applied Optimal Control. Hemisphere Publishing, New York.
11. CHEN, H. (1995). Fluid approximations and stability of multiclass queueing networks: work-conserving policies. Ann. Appl. Probab. 5 637-655.
12. CHEN, H. and MANDELBAUM, A. (1991). Discrete flow networks: bottleneck analysis and fluid approximations. Math. Oper. Res. 16 408-446.
13. CHEN, H. and YAO, D. (1993). Dynamic scheduling of a multiclass fluid network. Oper. Res. 41 1104-1115.
14. CHEN, R.-R. and MEYN, S. P. (1999). Value iteration and optimization of multiclass queueing networks. Queueing Sys. 32 65-97.
15. DAI, J. G. (1995a). On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models. Ann. Appl. Probab. 5 49-77.
16. DAI, J. G. (1995b). Stability of open multiclass queueing networks via fluid models. In Stochastic Networks Proceedings of the IMA (F. Kelly and R. Williams, eds.) Springer, New York. 71 71-90.
17. DAI, J. G. and MEYN, S. (1995). Stability and convergence of moments for multiclass queueing networks via fluid limit models. IEEE Trans. Autmat. Control 40 1889-1904.
18. DAI, J. G. and WEISS, G. (1996). Stability and instability of fluid models for certain re-entrant lines. Math. Oper. Res. 21 115-134.
19. DAI, J. G. and WEISS, G. (1999). A fluid heuristic for job shop scheduling. Technical report, Dept. Statistics, Univ. Haifa.
20. DAVIS, M. H. A. (1984). Piecewise-deterministic Markov processes: a general class of nondiffusion stochastic models. J. Roy Statist. Soc. Ser. B 46 353-388.
21. ENG, D., HUMPHREY, J. and MEYN, S. P. (1996). Fluid network models: linear programs for control and performance bounds. In 13th World Congress of International Federation of Automatic Control 2 19-24 North-Holland, Amsterdam.
22. GANS, N. and VAN RYZIN, G. (1997). Optimal control of a multiclass, flexible queueing system. Oper. Res. 45 677-693.
23. HARRISON, J. M. (1988). Brownian models of queueing networks with heterogeneous customer populations. In Stochastic Differential Systems, Stochastic Control Theory and Applications Proceedings of the IMA (W. Fleming and P. L. Lions, eds.) 10 147-186. Springer, New York.
24. HARRISON, J. M. (1996). The BIGSTEP approach to flow management in stochastic processing networks. In Stochastic Networks: Theory and Applications (F. Kelly, S. Zachary and I. Ziedins, eds.) 57-90. Oxford Univ. Press.
25. HARRISON, J. M. (1998). Heavy traffic analysis of a system with parallel servers: Asymptotic optimality of discrete-review policies. Ann. Appl. Probab. 8 1-24.
26. HARRISON, J. M. and WEIN, L. M. (1989). Scheduling network of queues: heavy traffic analysis of a simple open network. Queueing Sys. 5 265-280.
27. HARRISON, J. M. and WEIN, L. M. (1990). Scheduling networks of queues: heavy traffic analysis of a two-station closed network. Oper. Res. 38 1052-1064.
28. KELLY, F. and LAWS, N. (1993). Dynamic routing in open queueing networks: Brownian models, cut constraints and resource pooling. Queueing Sys. 13 47-86.
29. KUMAR, P. R. and SEIDMAN, T. I. (1990). Dynamic instabilities and stabilization methods in distributed real-time scheduling of manufacturing systems. IEEE Trans. Autmat. Control 35 289-298.
30. KUSHNER, H. J. and MARTINS, L. F. (1996). Heavy traffic analysis of a controlled multiclass queueing network via weak convergence methods. SIAM J. Control Optim. 34 1781-1797.
31. LU, S. H. and KUMAR, P. R. (1991). Distributed scheduling based on due dates and buffer priorities. IEEE Trans. Autmat. Control 36 1406-1416.
32. LUO, X. and BERTSIMAS, D. (1996). A new algorithm for state constrained separated linear programs. Technical report, MIT.
33. MAGLARAS, C. (1997). A methodology for dynamic control policy design for stochastic processing networks via fluid models. In Proceedings IEEE Conference on Decision and Control 1208-1214. IEEE, New York.
34. MAGLARAS, C. (1999). Dynamic scheduling in multiclass queueing networks: stability under discrete-review policies. Queueing Sys. 31 171-206.
35. MAGLARAS, C. (2000). Tracking Policies for control of stochastic networks: Brownian models, heavy traffic and asymptotic optimality. Technical report, Columbia University.
36. MEYN, S. P. (1997a). The policy improvement algorithm for Markov decision processes with general state space. IEEE Trans. Autmat. Control 42 1663-1680.
37. MEYN, S. P. (1997b). Stability and optimization of queueing networks and their fluid models. In Mathematics of Stochastic Manufacturing Systems. Lectures in Applied Math. 175-200. Amer. Math. Soc. Providence, RI.
38. NEWELL, G. F. (1971). Applications of Queueing Theory. Chapman and Hall, London.
39. PUHALSKII, A. and REIMAN, M. (1998). A critically loaded multirate link with trunk reservation. Queueing Sys. 28 157-190.
40. PULLAN, M. C. (1993). An algorithm for a class of continuous linear programs. SIAM J. Control Optim. 31 1558-1577.
41. PULLAN, M. C. (1995). Form of optimal solutions for separated continuous linear programs. SIAM J. Control Optim. 33 1952-1977.
42. PULLAN, M. C. (1996). A duality theory for separated continuous linear programs. SIAM J. Control Optim. 34 931-965.
43. ROYDEN, L. H. (1963). Real Analysis. Macmillan, New York.
44. RYBKO, A. N. and STOLYAR, A. L. (1992). Ergodicity of stochastic processes describing the operations of open queueing networks. Problems Inform. Transmission 28 199-220.
45. STOLYAR, A. L. (1995). On the stability of multiclass queueing networks: a relaxed sufficient condition via limiting fluid processes. Markov Processes Related Fields 1 491-512.
46. TASSIULAS, L. and PAPAVASSILIOU, S. (1995). Optimal anticipative scheduling with asynchronous transmission opprtunities. IEEE Trans. Autmat. Control 40 2052-2062.
47. WEISS, G. (1996). Optimal draining of fluid re-entrant lines: some solved examples. In Stochastic Networks: Theory and Applications (F. Kelly, S. Zachary and I. Ziedins, eds.) 19-34. Oxford Univ. Press.
48. WEISS, G. (1997). Algorithm for minimum wait draining of two station fluid re-entrant line. Technical report, Dept. Statistics, Univ. Haifa.
49. WEISS, G. (1999). Scheduling and control of manufacturing systems via optimal control of multiclass fluid networks. In Proceedings of 37th Allerton Conference on Communications, Control and Computing.
50. WILLIAMS, R. J. (1996). On the approximation of queueing networks in heavy traffic. In Stochastic Networks: Theory and Applications (F. Kelly, S. Zachary, and I. Ziedins, eds.) 35-56. Oxford Univ. Press.