Asymptotic optimality of tracking policies in stochastic networks

Annals of Applied Probability

Volumn 10, Issue 4, 2000, Pages 1065-1083

  Bäuerle, Nicole ^a  

^a UNIVERSITY OF ULM   (Germany)

Author keywords

Fluid model; Markov decision process; Stochastic network; Stochastic orderings; Weak convergence

Indexed keywords

EID: 0034310135     PISSN: 10505164     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (38)

1. ALANYALI, M. and HAJEK, B. (1998). Analysis of simple algorithms for dynamic load balancing. Math. Oper. Res. 22 840-871.
2. ATKINS, D. and CHEN, H. (1995). Performance evaluation of scheduling control of queueing networks: fluid model heuristics. Queueing Sys. 21 391-413.
3. AVRAM F. (1997) Optimal control of fluid limits of queueing networks and stochasticity corrections. In Mathematics of Stochastic Manufacturing Systems. Lectures in Applied Mathematics (G. G. Yin and Q. Zhang, eds.) 33 1-36. Amer. Math. Soc., Providence, RI.
4. AVRAM F., BERTSIMAS, D. and RICARD, M. (1995). Fluid models of sequencing problems in open queueing networks: an optimal control approach. In Stochastic Networks (F. P. Kelly and R. J. Williams, eds.) 199-234. Springer, New York.
5. BÄUERLE, N. (1998). The advantage of small machines in a stochastic fluid production process. Math. Methods Oper. Res. 47 83-97.
6. BÄUERLE, N. and RIEDER, U. (2000). Optimal control of single-server fluid networks. Queueing Systems 35 185-200.
7. BERTSEKAS, D. P. and SHREVE, S. E. (1978). Stochastic Optimal Control: The Discrete Time Case. Academic Press, New York.
8. BRAMSON, M. (1996). Convergence to equilibria for fluid models of FIFO queueing networks. Queueing Systems 22 5-45.
9. CHEN, H. (1995). Fluid approximations and stability of multiclass queueing networks: work-conserving disciplines. Ann. Appl. Probab. 5 637-665.
10. CHEN, R. R. and MEYN, S. (1998). Value iteration and optimization of multiclass queueing networks. Queueing Systems 32 65-97.
11. DAI, J. G. (1995). On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models. Ann. Appl. Probab. 5 49-77.
12. DAI, J G (1998). Stability of fluid and stochastic processing networks. Lecture note, Univ. Aarhus.
13. DAVIS M. H. A. (1993). Markov Models and Optimization. Chapman and Hall, London.
14. ETHIER, S. N. and KURTZ, T. G. (1986). Markov Processes. Wiley, New York.
15. HARRISON, J. M. (1996). The BIGSTEP approach to flow management in stochastic processing networks. In Stochastic Networks: Stochastic Control Theory and Applications. Royal Statistical Society Lecture Notes Series (F. Kelly, S. Zachary and I. Ziedins, eds.) 57-90. Clarendon, Oxford.
16. KITAEV, M. and RYKOV, V. (1995). Controlled Queueing Systems. CRC Press, Boca Raton, FL.
17. KUMAR, S. and KUMAR, P. R. (1996). Fluctuation smoothing policies are stable for stochastic re-entrant lines. J. Discrete Event Dynam. Sys. Theory Appl. 6 361-370.
18. KUSHNER, H. J. (1990). Numerical methods for stochastic control problems in continuous time. SIAM J. Control Optim. 5 999-1048.
19. KUSHNER, H. J. and DUPUIS, P. G. (1992). Numerical Methods for Stochastic Control Problems in Continuous Time. Springer, New York.
20. LUO, X. and BERTSIMAS, D. (1998). A new algorithm for state-constrained separated continuous linear programs. SIAM J. Control Optim. 37 177-210.
21. MAGLARAS, C. (1998). Discrete-review policies for scheduling stochastic networks: fluid-scale asymptotic optimality. Preprint.
22. MAGLARAS, C. (1999a). Dynamic scheduling in multiclass queueing networks: stability under discrete-review policies. Queueing Systems 31 171-206.
23. MAGLARAS, C. (1999b). Discrete-review policies for scheduling stochastic networks: trajectory tracking and fluid-scale asymptotic optimality. Preprint.
24. MEYN, S. P. (1997a). The policy iteration algorithm for average reward Markov decision processes with general state space. IEEE Trans. Automat. Control 42 1663-1679.
25. MEYN, S. P. (1997b). Stability and optimization of queueing networks and their fluid models. In Mathematics of Stochastic Manufacturing Systems. Lectures in Applied Mathematics (G. G. Yin and Q. Zhang, eds.) 33 175-199. Amer. Math. Soc., Providence, RI.
26. POLLARD, D. (1984). Convergence of Stochastic Processes. Springer, New York.
27. PULLAN, M. C. (1995). Forms of optimal solutions for separated continuous linear programs. SIAM J. Control Optim. 33 1952-1977.
28. RIEDER, U. (1975). Bayesian dynamic programming. Adv. in Appl. Probab. 7 330-348.
29. RYBKO, A. N. and STOLYAR, A. L. (1992). Ergodicity of stochastic processes describing the operation of open queueing networks. Problems Inform. Transmission 28 199-220.
30. SEIERSTAD, A. and SYDSÆTER, K. (1987). Optimal Control Theory with Economic Applications. North-Holland, Amsterdam.
31. SENNOTT, L. I. (1998). Stochastic Dynamic Programming and the Control of Queues. Wiley, New York.
32. SHAKED, M. and SHANTHIKUMAR, J. G. (1994). Stochastic Orders and Their Applications. Academic, New York.
33. STIDHAM, S. and WEBER, R. (1993). A survey of Markov decision models for control of networks of queues. Queueing Systems 13 291-314.
34. TIJMS, H. C. (1986). Stochastic Modelling and Analysis: A Computational Approach. Wiley, Chichester.
35. WEISS, G. (1995). On optimal draining of re-entrant fluid lines. In Stochastic Networks (F. P. Kelly and R. J. Williams, eds.) 91-103. Springer, New York.
36. WEISS, G. (1996). Optimal draining of fluid re-entrant lines: some solved examples. In Stochastic Networks: Stochastic Control Theory and Applications. Royal Statistical Society Lecture Notes Series (F. Kelly, S. Zachary, and I. Ziedins, eds.) 19-34. Clarendon, Oxford.
37. WEISS, G. (1997). Algorithm for minimum wait draining of two-station fluid re-entrant line. Technical report, Dept. Statistics, Univ. Haifa.
38. WILLIAMS, R. J. (1998). Some recent developments for queueing networks. Probability towards 2000. Lecture Notes in Statist. 340-356. Springer, New York.