Performance evaluation and policy selection in multiclass networks

Discrete Event Dynamic Systems: Theory and Applications

Volumn 13, Issue 1-2, 2003, Pages 149-189

  Henderson, Shane G ^a   Meyn, Sean P ^b   Tadić, Vladislav B ^c  

^a School of Operations Research and Industrial Engineering   (United States)

^b United States of America   (United States)

^c UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; DYNAMIC PROGRAMMING; FUNCTION EVALUATION; MATHEMATICAL MODELS; QUEUEING NETWORKS; QUEUEING THEORY; STOCHASTIC PROGRAMMING;

MULTICLASS QUEUEING NETWORKS; POLICY SELECTION; POLICY SYNTHESIS; STOCHASTIC APPROXIMATION;

SYSTEM THEORY;

EID: 0037289503     PISSN: 09246703     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1022197004856     Document Type: Article

References (62)

1. Asmussen, S. 1992. Queueing simulation in heavy traffic. Math. Operation Res. 17: 84-111.
2. Atlason, J., Epelman, M. A., and Henderson, S. G. 2002. Combining simulation and cutting plane methods in service systems. In P. Reinig, editor, Proceedings of the 2002 National Science Foundation Design, Service and Manufacture Grantees and Research Conference.
3. Bäuerle, N. 2000. Asymptotic optimality of tracking-policies in stochastic networks. Annals of Applied Probability 10: 1065-1083.
4. Bell, S. L., and Williams, R. J. 1999. Dynamic scheduling of a system with two parallel servers: Asymptotic optimality of a continuous review threshold policy in heavy traffic. In Proceedings of the 38th Conference on Decision and Control. Phoenix, Arizona. pp. 1743-1748.
5. Benveniste, A., Métivier, M., and Priouret, P. 1990. Adaptive Algorithms and Stochastic Approximations. Berlin: Springer-Verlag. Translated from the French by Stephen S. Wilson.
6. Bertsimas, D., Paschalidis, I., and Tsitsiklis, J. N. 1994. Optimization of multiclass queueing networks: polyhedral and nonlinear characterizations of achievable performance. Ann. Appl. Probab. 4: 43-75.
7. Borkar, V. S., and Meyn, S. P. 2000. The O.D.E. method for convergence of stochastic approximation and reinforcement learning. SIAM J. Control Optim. 38(2): 447-469 (electronic).
8. Bramson, M., 1998. State space collapse with application to heavy traffic limits for multiclass queueing networks. Queueing Systems 30: 89-148.
9. Bramson, M., and Dai, J. G. 2001. Heavy traffic limits for some queueing networks. Ann. Appl. Probab. 11(1): 49-90.
10. Chen, H., and Mandelbaum, A. 1991. Stochastic discrete flow networks: diffusion approximations and bottlenecks. Ann. Probab. 19(4): 1463-1519.
11. Chen, H., and Yao, D. D. 1993. Dynamic scheduling of a multiclass fluid network. Operations Research 41(6): 1104-1115.
12. Chen, M., Pandit, C., and Meyn, S. P. 2001. In search of sensitivity in network optimization. Submitted for publication, and presented at the 11th INFORMS Applied Probability Society Conference.
13. Chen, R.-R., and Meyn, S. P. 1999. Value iteration and optimization of multiclass queueing networks. Queueing Systems Theory Appl. 32(1-3): 65-97.
14. Courtois, P. J. 1977. Decomposability. New York: Academic Press.
15. Dai, J. G., and Wang, Y. 1993. Nonexistence of Brownian models of certain multiclass queueing networks. Queueing Systems: Theory and Applications 13 (May): 41-46.
16. Dembo, A., and Zeitouni, O. 1998. Large deviations techniques and applications. Second edition. New York: Springer-Verlag.
17. Dubrawski, R. 2000. Myopic and far-sighted strategies for control of demand-driven networks. Master's thesis, Department of Electrical Engineering, UIUC, Urbana, Illinois, USA.
18. Dupuis, P., and Kushner, H. 2001. Numerical Methods for Stochastic Control Problems in Continuous Time. Applications of Mathematics, Vol. 24, New York: Springer Verlag.
19. Feinberg, E. A., and Shwartz, A. 2001. Handbook of Markov Decision Processes. Boston: Kluwer Academic Publishers.
20. Glynn, P, W., and Whitt, W. 1989. Indirect estimation via l = λw. Operations Research 37(1): 82-103.
21. Harrison, J. M., and Van Mieghem, J. A. 1997. Dynamic control of Brownian networks: state space collapse and equivalent workload formulations. Ann. Appl. Probab. 7(3): 747-771.
22. Harrison, J. M., and Wein, L. M. 1990. Scheduling networks of queues: Heavy traffic analysis of a two-station closed network. Operations Research 38(6): 1052-1064.
23. Harrison, J. M. The BIGSTEP approach to flow management in stochastic processing networks. In F. P. Kelly, S. Zachary, and I. Ziedins, editors. Stochastic Networks Theory and Applications. Oxford, UK: Clarendon Press, 1996, pp. 57-89.
24. Henderson, S. G., and Meyn, S. P. 1997. Efficient simulation of multiclass queueing networks. In S. Andradottir, K. Healy, D. H. Withers, and B. L. Nelson, editors, Proceedings of the 1997 Winter Simulation Conference. Piscataway, NJ: IEEE, pp. 216-223.
25. Henderson, S. G., and Glynn, P. W. 2001. Approximating martingales for variance reduction in general discrete-event simulation. Working paper.
26. Henderson, S. G., and Glynn, P. W. 2002. Approximating martingales for variance reduction in Markov process simulation. Mathematics of Operations Research 27: 253-271.
27. Henderson, S. G., and Meyn, S. P. 1999. Variance reduction for simulation in multiclass queueing networks. Submitted for publication.
28. Henderson, S. G., and Meyn, S. P. 2002. Identifying effective polices for multiclass networks. In P. Reining, editor, Proceedings of the 2002 National Science Foundation Design, Service and Manufacture Grantees and Research Conference.
29. Henderson, S. G. 1997. Variance Reduction Via an Approximating Markov Process. Ph.D. thesis, Department of Operations Research, Stanford University, Stanford University, Stanford, California, USA.
30. Hordijk, A., and Spieksma, F. 1992. On ergodicity and recurrence properties of a Markov chain with an application to an open Jackson network. Adv. in Appl. Probab. 24(2): 343-376.
31. Kartashov, N. V. 1984. Criteria for uniform ergodicity and strong stability of Markov chains with a common phase space. Teor. Veroyatnost. i Mat. Statist. 151: 65-81.
32. Kelley, J. E., Jr. The cutting-plane method for solving convex programs. Journal of the Society for Industrial and Applied Mathematics 8(4): 703-712.
33. Kontoyiannis, I., and Meyn, S. P. 2001. Spectral theory and limit theorems for geometrically ergodic Markov processes. Ann. Appl. Probab. 2002. Presented at the 2001 INFORMS Applied Probability Conference, New York, July.
34. Kumar, P. R., and Meyn, S. P. 1996. Duality and linear programs for stability and performance analysis queueing networks and scheduling policies. IEEE Transactions on Automatic Control 41(1): 4-17.
35. Kumar, S. and Kumar, P. R. 1994. Performance bounds for queueing networks and scheduling policies. IEEE Trans. Automat. Control AC-39 (August): 1600-1611.
36. Kumar, S., and Muthuraman, M. 2000. A numerical method for solving singular Brownian control problems. In Proceedings of the 39th Conference on Decision and Control.
37. Kushner, H. J. 2001. Heavy traffic analysis of controlled queueing and communication networks. Stochastic Modelling and Applied Probability. New York: Springer-Verlag.
38. Kushner, H. J., and Yin, G. G. 1997. Stochastic Approximation Algorithms and Applications. New York: Springer-Verlag.
39. Law, A. M., and Kelton, W. D. 2000. Simulation Modeling and Analysis. New York: McGraw-Hill: 3rd edition.
40. Ljung, L., Pflug, G., and Walk, H. 1992. Stochastic approximation and optimization of random systems. Basel: Birkhäuser Verlag.
41. Luo, X., and Bertsimas, D. 1998. A new algorithm for state-constrained separated continuous linear programs. SIAM J. Control Optim. 37: 177-210.
42. Maglaras, C. 1997. Design of dynamic control policies for stochastic processing networks via fluid models. In Proceedings of the 38th Conference on Decision and Control, pp. 1208-1213.
43. Maglaras, C. 1999. Dynamic scheduling in multiclass queueing networks: Stability under discrete-review policies. Queueing Systems 31: 171-206.
44. Maglaras, C. 2000. Discrete-review policies for scheduling stochastic networks: Trajectory tracking and fluid-scale asymptotic optimality. Ann. Appl. Probab. 10: 897-929.
45. Meyn, S. P. 2001. Columbia University lecture series on stochastic networks. Center for Applied Probability, Columbia University, New York, NY, July. http://black.csl.uiuc.edu/meyn/pages/ColumbiaLectures.pdf.
46. Meyn, S. P. 1996. The policy iteration algorithm for average reward Markov decision processes with general state space. IEEE Trans. Automat. Control 42(12): 1663-1680, 1997. Also presented at the 35th IEEE Conference on Decision and Control, Kobe, Japan, December.
47. Meyn, S. P. 1997. Stability and optimization of queueing networks and their fluid models. In Mathematics of Stochastic Manufacturing Systems (Williamsburg, VA, 1996). American Mathematical Society, Providence, RI, pp. 175-199.
48. Meyn, S. P. 2001. Sequencing and routing in multiclass queueing networks. Part I: Feedback regulation. SIAM J. Control Optim. 40(3): 741-776.
49. Meyn, S. P. 2002. Sequencing and routing in multiclass queueing networks. Part II: Workload relaxations. SIAM J. Control Optim. To appear.
50. Meyn, S. P., and Down, D. G. 1994. Stability of generalized Jackson networks. Ann. Appl. Probab. 4: 124-148.
51. Meyn, S. P. and Tweedie, R. L. 1993. Markov Chains and Stochastic Stability. London: Springer-Verlag. Available on-line at http://black.csl.uiuc.edu/meyn.
52. Morrison, J. R., and Kumar, P. R. 1999. New linear program performance bounds for queueing networks. Journal of Optimization Theory and Applications 100(3): 575-597.
53. Nevel'son, M. B., and Has'minskiǐ, R. Z. 1973. Stochastic Approximation and Recursive Estimation. American Mathematical Society, Providence, RI. Translated from the Russian by the Israel Program for Scientific Translations, Translations of Mathematical Monographs. Vol. 47.
54. Perkins, J. 1993. Control of Push and Pull Manufacturing Systems. Ph.D. thesis, University of Illinois, Urbana, IL, September. Technical report no. UILU-ENG-93-2237 (DC-155).
55. Reiman, M. I. 1984. Open queueing networks in heavy traffic. Mathematics of Operations Research 9: 441-458.
56. Reiman, M. I. 1984. Some diffusion approximations with state space collapse. In Modelling and performance evaluation methodology (Paris, 1983). Berlin: Springer, pp. 209-240.
57. Ross, S. M. 1996. Stochastic Processes. New York: Wiley, 2nd edition.
58. Schwerer, E. 2001. A linear programming approach to the steady-state analysis of reflected Brownian motion. Stochastic Models 17: 341-368.
59. Tadic, V., and Meyn, S. P. 2002. A stochastic approximation algorithm for variance reduction in simulation and reinforcement learning. In preparation.
60. Weiss, G. 2001. A simplex based algorithm to solve separated continuous linear programs. Technical Report, Department of Statistics The University of Haifa Mount Carmel 31905, Israel.
61. Whitt, W. 1989. Planning queueing simulations. Management Science 35: 1341-1366.
62. Williams, R. J. 1998. Diffusion approximation for open-multiclass queueing networks: sufficient conditions involving state space collapse. Queueing Systems 30(1-2): 27-88.