Workload models for stochastic networks: Value functions and performance evaluation

IEEE Transactions on Automatic Control

Volumn 50, Issue 8, 2005, Pages 1106-1122

  Meyn, Sean P ^a  

^a University of Illinois at Urbana Champaign   (United States)

Author keywords

Computer network performance; Networks; Optimal control; Routing; Scheduling; Simulation; Singular optimal control; Stochastic systems

Indexed keywords

BOUNDARY CONDITIONS; BROWNIAN MOVEMENT; COMPUTER NETWORKS; COMPUTER SIMULATION; DATA COMMUNICATION SYSTEMS; FUNCTION EVALUATION; LINEAR PROGRAMMING; MATHEMATICAL MODELS; OPTIMAL CONTROL SYSTEMS; ROUTERS; SCHEDULING;

APPROXIMATE VALUE FUNCTION; COMPUTER NETWORK PERFORMANCE; CONTROLLED BROWNIAN MOTION (CBM); DYNAMIC PROGRAMMING EQUATIONS; EXPONENTIAL ERGODICITY; FLUID VALUE-FUNCTION; ROUTING; STOCHASTIC NETWORKS; VARIANCE-REDUCTION TECHNIQUE;

STOCHASTIC CONTROL SYSTEMS;

EID: 23944498849     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2005.852564     Document Type: Article

References (61)

1. S. Kumar and P. R. Kumar. "Performance bounds for queueing networks and scheduling policies" IEEE Trans. Autom. Control, vol. 39, no. 8, pp. 1600-1611, Aug. 1994.
2. D. Bertsimas, I. Paschalidis, and J. N. Tsitsiklis, "Optimization of multiclass queueing networks: Polyhedral and nonlinear characterizations of achievable performance" Ann. Appl. Probab., vol. 4, pp. 43-75, 1994.
3. P. R. Kumar and S. P. Meyn, "Duality and linear programs for stability and performance analysis queueing networks and scheduling policies" IEEE Trans. Autom. Control, vol. 41, no. 1, pp. 4-17, Jan. 1996.
4. E. Schwerer, "A linear programming approach to the steady-state analysis of Markov processes" Ph.D. dissertation. Stanford Univ., Stanford, CA, 1997.
5. J. R. Morrison and P. R. Kumar. "New linear program performance bounds for queueing networks" J. Optim. Theory Appl., vol. 100, no. 3, pp. 575-597, 1999.
6. S. P. Meyn, "The policy iteration algorithm for average reward Markov decision processes with general state space" IEEE Trans. Autom. Control. vol. 42, no. 12, pp. 1663-1680, Dec. 1997.
7. R.-R. Chen and S. P. Meyn, "Value iteration and optimization of multiclass queueing networks" Queueing Syst.: Theory Appl., vol. 32, no. 1-3, pp. 65-97, 1999.
8. M. H. Veatch, "Approximate dynamic programming for networks: Fluid models and constraint reduction" submitted for publication, 2005.
9. S. G. Henderson and P. W. Glynn, "Approximating martingales for variance reduction in Markov process simulation" Math. Oper. Res., vol. 27, no. 2, pp. 253-271, 2002.
10. S. G. Henderson, S. P. Meyn, and V. B. Tadić "Performance evaluation and policy selection in multiclass networks" Discrete Event Dyn. Syst.: Theory Appl., vol. 13, no. 1-2. pp. 149-189, 2003.
11. S. P. Meyn, "Large deviation asymptotics and control variates for simulating large functions" Ann. Appl. Probab., 2005, to be published.
12. J. M. Harrison and R. J. Williams. "Brownian models of open queueing networks with homogeneous customer populations" Stochastics, vol. 22, no. 2, pp. 77-115, 1987.
13. J. M. Harrison and J. A. Van Mieghem, "Dynamic control of Brownian networks: State space collapse and equivalent workload formulations" Ann. Appl. Probab., vol. 7, no. 3, pp. 747-771, 1997.
14. J. M. Harrison. "Brownian models of open processing networks: Canonical representations of workload" Ann. Appl. Probab., vol. 10, pp. 75-103, 2000.
15. M. Bramson and R. J. Williams, "Two workload properties for Brownian networks" Queueing Syst.: Theory Appl., vol. 45, no. 3, pp. 191-221, 2003.
16. S. P. Meyn, "Sequencing and routing in multiclass queueing networks. Part II: Workload relaxations" SIAM J. Control Optim., vol. 42, no. 1, pp. 178-217, 2003.
17. M. Chen, C. Pandit, and S. P. Meyn, "In search of sensitivity in network optimization" Queueing Syst.: Theory Appl., vol. 44, no. 4, pp. 313-363, 2003.
18. H. Chen and D. D. Yao, Fundamentals of Queueing Networks: Performance, Asymptotics, and Optimization, ser. Stochastic Modeling and Applied Probability. New York: Springer-Verlag, 2001.
19. S. P. Meyn, "Dynamic safety-stocks for asymptotic optimality in stochastic networks" Queueing Syst., Aug. 2005, to be published.
20. M. I. Reiman, "Open queueing networks in heavy traffic" Math. Oper. Res., vol. 9, pp. 441-458, 1984.
21. H. Chen and A. Mandelbaum, "Stochastic discrete flow networks: Diffusion approximations and bottlenecks" Ann. Probab., vol. 19, no. 4, pp. 1463-1519, 1991.
22. R. J. Williams, "Diffusion approximations for open multiclass queueing networks: Sufficient conditions involving state space collapse" Queueing Syst.: Theory Appl., vol. 30, no. 1-2, pp. 27-88, 1998.
23. P. Dupuis and R. J. Williams, "Lyapunov functions for semimartingale reflecting Brownian motions" Ann. Appl. Probab., vol. 22, no. 2, pp. 680-702, 1994.
24. R. Atar, A. Budhiraja, and P. Dupuis, "On positive recurrence of constrained diffusion processes" Ann. Probab., vol. 29, pp. 979-1000, 2001.
25. S. P. Meyn and R. L. Tweedie, "Stability of markovian processes III: Foster-Lyapunov criteria for continuous time processes" Adv. Appl. Probab., vol. 25, pp. 518-548, 1993.
26. S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability. London, U.K.: Springer-Verlag, 1993.
27. J. G. Dai and S. P. Meyn, "Stability and convergence of moments for multiclass queueing networks via fluid limit models" IEEE Trans. Autom. Control, vol. 40, no. 11, pp. 1889-1904, Nov. 1995.
28. M. Chen, R. Dubrawski, and S. P. Meyn, "Management of demand-driven production systems" IEEE Trans. Autom. Control. vol. 49, no. 5, pp. 686-698, May 2004.
29. M. H. Veatch, "Fluid analysis of arrival routing" IEEE Trans. Autom. Control, vol. 46, no. 8, pp. 1254-1257, Aug. 2001.
30. J. G. Dai and G. Weiss, "A fluid heuristic for minimizing makespan in job shops" Oper. Res., vol. 50, no. 4, pp. 692-707, 2002.
31. N. Laws, "Dynamic routing in queueing networks" Ph.D. dissertation, Cambridge Univ., Cambridge, U.K., 1990.
32. S. P. Meyn, "Sequencing and routing in multiclass queueing networks. Part I: Feedback regulation" SIAM J. Control Optim., vol. 40, no. 3, pp. 741-776, 2001.
33. L. M. Wein, "Dynamic scheduling of a multiclass make-to-stock queue" Oper. Res., vol. 40, no. 4, pp. 724-735, 1992.
34. I.-K. Cho and S. Meyn, "Optimization and the price of anarchy in a dynamic newsboy model" submitted for publication, 2005.
35. S. Asmussen, "Queueing simulation in heavy traffic" Math. Oper. Res., vol. 17, pp. 84-111, 1992.
36. W. Whitt, "Planning queueing simulations" Manage. Sci., vol. 35, pp. 1341-1366, 1994.
37. P. Glynn and R. Szechtman, "Some new perspectives on the method of control variates" in Monte Carlo and Quasi-Monte Carlo Methods 2000: Proceedings of a Conference Held at Hong Kong Baptist University, Hong Kong SAR, China, K. Fang, F. Hickernell, and H. Niederreiter, Eds. Berlin, Germany: Springer-Verlag, 2002, pp. 27-49.
38. S. G. Henderson and B. L. Nelson, Eds., Handbooks in Operations Research and Management Science, XII. ser. Simulation. Cambridge, MA: Elsevier, 2005.
39. M. I. Reiman and R. J. Williams, "A boundary property of semimartingale reflecting Brownian motions" Probab. Theory Related Fields, vol. 77, no. 1, pp. 87-97. 1988. (correction in 80 (1989), no. 4, 633).
40. L. M. Taylor and R. J. Williams, "Existence and uniqueness of semimartingale reflecting Brownian motions in an orthant" Probab. Theory Related Fields, vol. 96, no. 3. pp. 283-317, 1993.
41. P. Dupuis and K. Ramanan. "Convex duality and the Skorokhod problem. I, II" Probab. Theory Related Fields, vol. 115, no. 2, pp. 153-195, 1999. 197-236.
42. P. Dupuis and I. Hitoshi, "On Lipschitz continuity of the solution mapping to the Skorokhod problem, with applications" Stoch. Stoch. Rep., vol. 35, no. 1, pp. 31-62, 1991.
43. S. N. Ethier and T. G. Kurtz, Markov Processes: Characterization and Convergence. New York: Wiley, 1986.
44. N. V. Krylov, Controlled Diffusion Processes, ser. Applications of Mathematics. New York: Springer-Verlag, 1980, vol. 14. translated from the Russian by A. B. Aries.
45. N. V. Krylov, "On a proof of Itô's formula" Trudy Mat. Inst. Steklov., vol. 202, pp. 170-174, 1993.
46. R. Chitashvili and M. Mania, "Generalized Itô formula and derivation of Bellman's equation" in Stochastic Processes and Related Topics (Siegmundsberg, 1994), ser. Stochastics Monograph. Yverdon, France: Gordon and Breach, 1996, vol. 10, pp. 1-21.
47. H. Föllmer and P. Protter, "On Itâ's formula for multidimensional Brownian motion" Probab. Theory Related Fields. vol. 116, no. 1, pp. 1-20, 2000.
48. P. W. Glynn and S. P. Meyn, "A Liapounov bound for solutions of the Poisson equation" Ann. Probab., vol. 24, no. 2, pp. 916-931, 1996.
49. G. Weiss, "A simplex based algorithm to solve separated continuous linear programs" Dept. Statist., Univ. Haifa-Mount Carmel, Haifa, Israel, Tech. Rep. 31 905, 2001.
50. D. Q. Mayne, "Control of constrained dynamic systems" Eur. J. Control, vol. 7, pp. 87-99, 2001.
51. F. Borrelli, Constrained Optimal Control of Linear and Hybrid Systems, ser. Lecture Notes in Control and Information Sciences. Berlin, Germany: Springer-Verlag, 2003, vol. 290.
52. H. Thorisson, Coupling, Stationarity, and Regeneration, ser. Probability and its Applications. New York: Springer-Verlag, 2000.
53. L. C. G. Rogers and D. Williams, Diffusions, Markov Processes, and Martingales, ser. Cambridge Mathematical Library. Cambridge, U.K.: Cambridge Univ. Press, 2000, vol. 2.
54. D. Down, S. P. Meyn, and R. L. Tweedie, "Exponential and uniform ergodicity of Markov processes" Ann. Probab., vol. 23, no. 4, pp. 1671-1691, 1995.
55. I. Kontoyiannis and S. P. Meyn, "Spectral theory and limit theorems for geometrically ergodic Markov processes" Ann. Appl. Probab., vol. 13, pp. 304-362, 2003.
56. S. P. Meyn and R. L. Tweedie, "Generalized resolvents and Harris recurrence of Markov processes" Contemp. Math., vol. 149, pp. 227-250, 1993.
57. J. M. Harrison and M. I. Reiman, "On the distribution of multidimensional reflected Brownian motion" SIAM J. Appl. Math., vol. 41, no. 2, pp. 345-361, 1981.
58. J. M. Harrison and R. J. Williams, "Brownian models of feedforward queueing networks: Quasireversibility and product form solutions" Ann. Appl. Probab., vol. 2, no. 2, pp. 263-293, 1992.
59. J. G. Dai and J. M. Harrison, "Reflected Brownian motion in an orthant: Numerical methods for steady-state analysis" Ann. Appl. Prob., vol. 2, pp. 65-86, 1992.
60. V. S. Borkar and S. P. Meyn, "The O.D.E. method for convergence of stochastic approximation and reinforcement learning" SIAM J. Control Optim., vol. 38, no. 2, pp. 447-469, 2000.
61. presented at the INFORMS Applied Probability Conf., New York, Jul. 2001.