Control techniques for complex networks

Control Techniques for Complex Networks

Volumn , Issue , 2007, Pages 1-562

  Meyn, Sean ^a  

^a University of Illinois at Urbana Champaign   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84925067999     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/CBO9780511804410     Document Type: Book

References (509)

1. D. Adelman. A price-directed approach to stochastic inventory/routing. Oper. Res., 52(4): 499–514, 2004.
2. R. Agrawal, R. L. Cruz, C. Okino, and R. Rajan. Performance bounds for flow control protocols. IEEE/ACM Trans. Netw., 7:310–323, 1999.
3. R. Ahlswede. Multi-way communication channels. In Second International Symposium on Information Theory (Tsahkadsor, 1971), pages 23–52. Akadémiai Kiadó, Budapest, 1973.
4. M. S. Akturk and F. Erhun. An overview of design and operational issues of KANBAN systems. Int. J. Prod. Res, 37(17):3859–3881, 1999.
5. E. Altman. Constrained Markov Decision Processes. Stochastic Modeling. Chapman & Hall/CRC, Boca Raton, FL, 1999.
6. E. Altman. Applications of Markov decision processes in communication networks: a survey. In E. Feinberg and A. Shwartz, editors, Markov Decision Processes: Models, Methods, Directions, and Open Problems, pages 489–536. Kluwer, Holland, 2001.
7. E. Altman, B. Gaujal, and A. Hordijk. Multimodularity, convexity, and optimization properties. Math. Oper. Res, 25(2):324–347, 2000.
8. E. Altman, B. Gaujal, and A. Hordijk. Discrete-Event Control of Stochastic Networks: Multimodularity and Regularity. Lecture Notes in Mathematics. Springer-Verlag, New York, 2004.
9. E. Altman, T. Jiménez, and G. Koole. On the comparison of queueing systems with their fluid limits. Probab. Eng. Inf. Sci., 15(2):165–178, 2001.
10. V. Anantharam. The stability region of the finite-user, slotted ALOHA system. IEEE Trans. Inf. Theory, 37:535–540, 1991.
11. E. J. Anderson. A Continuous Model for Job-Shop Scheduling. PhD thesis, University of Cambridge, Cambridge, UK, 1978.
12. E. J. Anderson. A new continuous model for job-shop scheduling. Int. J. Syst. Sci., 12:1469–1475, 1981.
13. E. J. Anderson and P. Nash. Linear Programming in Infinite-Dimensional Spaces. Wiley- Interscience Series in Discrete Mathematics and Optimization. John Wiley Chichester, UK, 1987. Theory and applications, A Wiley–Interscience Publication.
14. L.-E. Andersson, G. Z. Chang, and T. Elfving. Criteria for copositive matrices using simplices and barycentric coordinates. In Proceedings of the Workshop “Nonnegative Matrices, Applications and Generalizations” and the Eighth Haifa Matrix Theory Conference (Haifa, 1993), volume 220, pages 9–30, 1995.
15. APX homepage. http://www.apx.nl/home.html, 2004. Formally Amsterdam Power Exchange.
16. A. Arapostathis, V. S. Borkar, E. Fernandez-Gaucherand, M. K. Ghosh, and S. I. Marcus. Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J. Control Optim., 31:282–344, 1993.
17. K. J. Arrow, T. Harris, and J. Marschak. Optimal inventory policy. Econometrica, 19:250–272, 1951.
18. K. J. Arrow, S. Karlin, and H. E. Scarf. Studies in the Mathematical Theory of Inventory and Production. Stanford Mathematical Studies in the Social Sciences, I. Stanford University Press, Stanford, CA, 1958.
19. S. Asmussen. Applied Probability and Queues. John Wiley, New York, 1987.
20. S. Asmussen. Queueing simulation in heavy traffic. Math. Oper. Res., 17:84–111, 1992.
21. B. Ata and S. Kumar. Heavy traffic analysis of open processing networks with complete resource pooling: asymptotic optimality of discrete review policies complete resource pooling: asymptotic optimality of discrete review policies. Ann. Appl. Probab., 15(1A):331–391, 2005.
22. R. Atar and A. Budhiraja. Singular control with state constraints on unbounded domain. Ann. Probab., 34(5):1864–1909, 2006.
23. R. Atar, A. Budhiraja, and P. Dupuis. On positive recurrence of constrained diffusion processes. Ann. Probab, 29(2):979–1000, 2001.
24. D. Atkins and H. Chen. Performance evaluation of scheduling control of queueing networks: fluid model heuristics. Queueing Syst. Theory Appl., 21(3–4):391–413, 1995.
25. F. Avram, D. Bertsimas, and M. Ricard. Fluid models of sequencing problems in open queueing networks: an optimal control approach. Technical Report, Massachussetts Institute of Technology, MA, 1995.
26. F. Avram, D. Bertsimas, and M. Ricard. An optimal control approach to optimization of multi-class queueing networks. In F. Kelly and R. Williams, editors, Volume 71 of IMA Volumes in Mathematics and its Applications, New York, 1995. Springer-Verlag. Proceedings of Workshop on Queueing Networks of the Mathematical Institute, Minneapolis, 1994.
27. Y. Azar, A. Z. Broder, A. R. Karlin, and E. Upfal. Balanced allocations. SIAMJ. Comput., 29(1):180–200 (electronic), 1999.
28. K. Azuma. Weighted sums of certain dependent random variables. Tôhoku Math. J., 19:357–367, 1967.
29. F. Baccelli and P. Brémaud. Elements of Queueing Theory: Palm Martingale Calculus and Stochastic Recurrences. Springer, Berlin, 2003.
30. F. Baccelli and S. Foss. Ergodicity of Jackson-type queueing networks. Queueing Syst. Theory Appl, 17:5–72, 1994.
31. T. Başar and G. J. Olsder. Dynamic Noncooperative Game Theory, 2nd edition. Academic Press, London, 1995.
32. F. Baskett, K. M. Chandy, R. R. Muntz, and F. G. Palacios. Open, closed, and mixed networks of queues with different classes of customers. J. ACM, 22:248–260, 1975.
33. N. Bäuerle. Asymptotic optimality of tracking policies in stochastic networks. Ann. Appl. Probab., 10(4):1065–1083, 2000.
34. P. H. Baxendale. Renewal theory and computable convergence rates for geometrically ergodic Markov chains. Adv. Appl. Probab., 15(1B):700–738, 2005.
35. S. L. Bell and R. J. Williams. Dynamic scheduling of a system with two parallel servers: Asymptotic optimality of a continuous review threshold policy in heavy traffic. In Proceedings of the 38th Conference on Decision and Control, pages 1743–1748, Phoenix, Arizona, 1999.
36. S. L. Bell and R. J. Williams. Dynamic scheduling of a system with two parallel servers in heavy traffic with complete resource pooling: asymptotic optimality of a continuous review threshold policy. Ann. Appl. Probab., 11:608–649, 2001.
37. R. Bellman. Dynamic Programming. Princeton University Press, Princeton, NJ, 1957.
38. R. Bellman. On a routing problem. Q. Appl. Math, 16:87–90, 1958.
39. R. Bellman, I. Glicksberg, and O. Gross. On the optimal inventory equation. Manage. Sci, 2:83–104, 1955.
40. G. Bennett. Probability inequalities for the sum of independent random variables. J. Am. Stat. Assoc., 57: 33–45, 1962.
41. A. Benveniste, M. Métivier, and P. Priouret. Adaptive Algorithms and Stochastic Approximations. Applications of Mathematics (New York), volume 22. Springer-Verlag, Berlin, 1990. Translated from the French by Stephen S. Wilson.
42. D. S. Bernstein, R. Givan, N. Immerman, and S. Zilberstein. The complexity of decentralized control of Markov decision processes. Math. Oper. Res, 27(4):819–840, 2002.
43. D. Bertsekas and R. Gallager. Data Networks. Prentice Hall, Englewood Cliffs, NJ, 1992.
44. D. P. Bertsekas. A new value iteration method for the average cost dynamic programming problem. SIAM J. Control Optim., 36(2): 742–759 (electronic), 1998.
45. D. P. Bertsekas. Dynamic Programming and Optimal Control, 3rd edition. Athena Scientific, Cambridge, MA, 2007.
46. D. P. Bertsekas, V. Borkar, and A. Nedic. Improved temporal difference methods with linear function approximation. In J. Si, A. Barto, W. Powell, and D. Wunsch, editors, Handbook of Learning and Approximate Dynamic Programming, pages 690–705. Wiley-IEEE Press, Piscataway, NJ, 2004.
47. D. P. Bertsekas and J. N. Tsitsiklis. Parallel and Distributed Computation: Numerical Methods. Prentice Hall, Upper Saddle River, NJ, USA, 1989.
48. D. P. Bertsekas and J. N. Tsitsiklis. An analysis of stochastic shortest path problems. Math. Oper. Res., 16(3):580–595, 1991.
49. D. P. Bertsekas and J. N. Tsitsiklis. Neuro-Dynamic Programming. Athena Scientific, Cambridge, MA, 1996.
50. D. Bertsimas, D. Gamarnik, and J. Sethuraman. From fluid relaxations to practical algorithms for high-multiplicity job-shop scheduling: the holding cost objective. Oper. Res., 51(5):798–813, 2003.
51. D. Bertsimas, D. Gamarnik, and J. N. Tsitsiklis. Stability conditions for multiclass fluid queueing networks. IEEE Trans. Autom. Control, 41(11):1618–1631, 1996.
52. D. Bertsimas, D. Gamarnik, and J. N. Tsitsiklis. Correction: “Stability conditions for multiclass fluid queueing networks” IEEE Trans. Autom. Control, 41(11): 1618–1631, 1996; MR1419686 (97f:90028)]. IEEE Trans. Autom. Control, 42(1):128, 1997.
53. D. Bertsimas and J. Niño-Mora. Restless bandits, linear programming relaxations, and a primal-dual index heuristic. Oper. Res, 48(1):80–90, 2000.
54. D. Bertsimas, I. Paschalidis, and J. N. Tsitsiklis. Optimization of multiclass queueing networks: polyhedral and nonlinear characterizations of achievable performance. Ann. Appl. Probab., 4:43–75, 1994.
55. D. Bertsimas and I. Ch. Paschalidis. Probabilistic service level guarantees in make-to-stock manufacturing systems. Oper. Res., 49(1):119–133, 2001.
56. D. Bertsimas and J. Sethuraman. From fluid relaxations to practical algorithms for job shop scheduling: the makespan objective. Math. Program., 92(1A):61–102, 2002.
57. P. Bhagwat, P. P. Bhattacharya, A. Krishna, and S. K. Tripathi. Enhancing throughput over wireless LANs using channel state dependent packet scheduling. INFOCOM 3:1133–1140, 1996.
58. S. Bhardwaj, R. J. Williams, and A. S. Acampora. On the performance of a two-user mimo downlink system in heavy traffic. IEEE Trans. Inf. Theory, 53(5), 2007.
59. N. P. Bhatia and G. P. Szegö. Stability Theory of Dynamical Systems. Springer-Verlag, New York, 1970.
60. A. G. Bhatt and V. S. Borkar. Existence of optimal Markov solutions for ergodic control of Markov processes. Sankhyā, 67(1):1–18, 2005.
61. P. Billingsley. Probability and Measure. John Wiley, New York, 1995.
62. F. Black and M. Scholes. The pricing of options and corporate liabilities. J. Pol. Econ., 81(3):637–654, 1973.
63. G. Bolch, S. Greiner, H. de Meer, and K. S. Trivedi. Queueing Networks and Markov Chains: Modeling and Performance Evaluation with Computer Science Applications 2nd edition. Wiley-Interscience, New York, 2006.
64. A. M. Bonvik, C. Couch, and S. B. Gershwin. A comparison of production-line control mechanisms. Int. J. Prod. Res., 35(3):789–804, 1997.
65. V. S. Borkar. Controlled Markov chains and stochastic networks. SIAM J. Control Optim., 21(4):652–666, 1983.
66. V. S. Borkar. On minimum cost per unit time control of Markov chains. SIAM J. Control Optim., 22(6):965–978, 1984.
67. V. S. Borkar. Control of Markov chains with long-run average cost criterion: the dynamic programming equations. SIAM J. Control Optim., 27(3):642–657, 1989.
68. V. S. Borkar. Topics in Controlled Markov Chains. Pitman Research Notes in Mathematics Series 240, Longman Scientific & Technical, UK, 1991.
69. V. S. Borkar. Ergodic control of Markov chains with constraints the general case. SIAM J. Control Optim., 32:176–186, 1994.
70. V. S. Borkar. Convex analytic methods in Markov decision processes. In Handbook of Markov Decision Processes. International Series in Operations Research and Management Science, volume 40, pages 347–375. Kluwer Academic Publisher, Boston, MA, 2002.
71. V. S. Borkar and S. P. Meyn. The O.D.E. method for convergence of stochastic approximation and reinforcement learning. SIAM J. Control Optim., 38(2): 447–469, 2000. Also presented at the IEEE CDC, December 1998.
72. A. N. Borodin and P. Salminen. Handbook of Brownian Motion – Facts and Formulae. Probability and its Applications 1st edition. Birkhäuser Verlag, Basel, 1996. Second edition published 2002.
73. A. A. Borovkov. Limit theorems for queueing networks. Theory Probab. Appl., 31:413–427, 1986.
74. A. A. Borovkov, O. J. Boxma, and Z. Palmowski. On the integral of the workload process of the single server queue. J. Appl. Probab., 40(1):200–225, 2003.
75. S. C. Borst and P. A. Whiting. Dynamic rate control algorithms for HDR throughput optimization. IEEE INFOCOM, 2:976–985, 2001.
76. D. D. Botvich and A. A. Zamyatin. Ergodicity of conservative communication networks. Technical Report, Institut National de Recherche en Informatique et en Automatique, Rocquencourt, 1993.
77. J. R. Bradley and P. W. Glynn. Managing capacity and inventory jointly in manufacturing systems. Manage. Sci., 48(2):273–288, 2002.
78. S. J. Bradtke and A. G. Barto. Linear least-squares algorithms for temporal difference learning. Mach. Learn., 22(1–3):33–57, 1996.
79. M. Bramson. Correction: Instability of FIFO queueing networks. Ann. Appl. Probab, 4(3):952, 1994.
80. M. Bramson. Instability of FIFO queueing networks. Ann. Appl. Probab., 4(2):414–431, 1994.
81. M. Bramson. State space collapse with application to heavy traffic limits for multiclass queueing networks. Queueing Syst. Theory Appl., 30:89–148, 1998.
82. M. Bramson. A stable queueing network with unstable fluid model. Ann. Appl. Probab., 9(3):818–853, 1999.
83. M. Bramson and R. J. Williams. On dynamic scheduling of stochastic networks in heavy traffic and some new results for the workload process. In Proceedings of the 39th Conference on Decision and Control, Sydney, Australia, page 516, 2000.
84. M. Bramson and R. J. Williams. Two workload properties for Brownian networks. Queueing Syst. Theory Appl., 45(3):191–221, 2003.
85. A. Budhiraja and C. Lee. Stationary distribution convergence for generalized Jackson networks in heavy traffic. Submitted for publication, 2007.
86. C. Buyukkoc, P. Varaiya, and J. Walrand. The c-μ rule revisited. Adv. Appl. Probab., 17(1):237–238, 1985.
87. California's Electricity, UIC Nuclear Issues Briefing Paper No. 61. Uranium Information Centre. http://www.uic.com.au/nip61.htm, May 2004.
88. C. G. Cassandras and S. Lafortune. Introduction to discrete event systems. The Kluwer International Series on Discrete Event Dynamic Systems, 11. Kluwer Academic Publishers, Boston, MA, 1999.
89. R. Cavazos-Cadena. Value iteration in a class of communicating Markov decision chains with the average cost criterion. SIAM J. Control Optim., 34(6):1848–1873, 1996.
90. R. Cavazos-Cadena and E. Ferndndez-Gaucherand. Value iteration in a class of controlled Markov chains with average criterion: unbounded costs case. In Proceedings of the 34th Conference on Decision and Control, New Orleans, LA, page TP05 3:40, 1995.
91. CBS. More Enron tapes, more gloating. http://www.cbsnews.com/stories/2004/06/08/eveningnews/~main621856.shtml, June 8, 2004.
92. C.-S. Chang, X. Chao, M. Pinedo, and R. Weber. On the optimality of LEPT and c – μ rules for machines in parallel. J. Appl. Probab., 29(3):667–681, 1992.
93. C. -S. Chang, J. A. Thomas, and S.-H. Kiang. On the stability of open networks: a unified approach by stochastic dominance. Queueing Syst. Theory Appl., 15:239–260, 1994.
94. H. Chen and A. Mandelbaum. Discrete flow networks: bottleneck analysis and fluid approximations. Math. Oper. Res., 16(2):408–446, 1991.
95. H. Chen and D. D. Yao. Dynamic scheduling of a multiclass fluid network. Oper. Res., 41(6):1104–1115, November–December 1993.
96. H. Chen and D. D. Yao. Fundamentals of Queueing Networks: Performance, Asymptotics, and Optimization. Stochastic Modelling and Applied Probability. Springer–Verlag, New York, 2001.
97. H. Chen and H. Zhang. Stability of multiclass queueing networks under FIFO service discipline. Math. Oper. Res., 22(3):691–725, 1997.
98. H. Chen and H. Zhang. Stability of multiclass queueing networks under priority service disciplines. Oper. Res., 48(1):26–37, 2000.
99. H. -F. Chen. Stochastic Approximation and Its Applications. Nonconvex Optimization and its Applications, volume 64. Kluwer Academic Publishers, Dordrecht, 2002.
100. M. Chen. Modelling and Control of Complex Stochastic Networks, with Applications to Manufacturing Systems and Electric Power Transmission Networks. PhD thesis, University of Illinois at Urbana Champaign, University of Illinois, Urbana, IL, 2005.
101. M. Chen, I.-K. Cho, and S. P. Meyn. Reliability by design in a distributed power transmission network. Automatica, 42: 1267–1281, August 2006. Invited.
102. M. Chen, R. Dubrawski, and S. P. Meyn. Management of demand-driven production systems. IEEE Trans. Autom. Control, 49(2):686–698, May 2004.
103. M. Chen, C. Pandit, and S. P. Meyn. In search of sensitivity in network optimization. Queueing Syst. Theory Appl., 44(4): 313–363, 2003.
104. R. C. Chen and G. L. Blankenship. Dynamic programming equations for discounted constrained stochastic control. IEEE Trans. Autom. Control, 49(5):699–709, 2004.
105. R.-R. Chen and S. P. Meyn. Value iteration and optimization of multiclass queueing networks. Queueing Syst. Theory Appl., 32(1–3):65–97, 1999.
106. R. Chitashvili and M. Mania. Generalized Itô formula and derivation of Bellman's equation. In Stochastic Processes and Related Topics (Siegmundsberg, 1994). Stochastics Monographs, volume 10, pages 1–21. Gordon and Breach, Yverdon, 1996.
107. I.-K. Cho and S. P. Meyn. The dynamics of the ancillary service prices in power networks. In Proceedings of the 42nd IEEE Conference on Decision and Control, volume 3, Maui, H1, pages 2094– 2099, December 9–12, 2003.
108. A. J. Chorin. Conditional expectations and renormalization. J. Multiscale Model. Simul., 1(1):105–118, 2003.
109. K. L. Chung. A Course in Probability Theory, 2nd edition. Academic Press, New York, 1974.
110. A. J. Clark and H. E. Scarf. Optimal policies for a multi-echelon inventory problem. Manage. Sci., 6:465–490, 1960.
111. E. G. Coffman Jr., D. S. Johnson, P. W. Shor, and R. R. Weber. Markov chains, computer proofs, and average-case analysis of best fit bin packing. In STOC’93: Proceedings of the Twenty-Fifth Annual ACM Symposium on Theory of Computing, pages 412–421, New York, 1993. ACM Press.
112. E. G. Coffman Jr. and I. Mitrani. A characterization of waiting times realizable by single server queues. Oper. Res., 28:810–821, 1980.
113. E. G. Coffman Jr. and M. I. Reiman. Diffusion approximations for storage processes in computer systems. In SIGMETRICS’83: Proceedings of the 1983 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, pages 93–117, New York, 1983. ACM Press.
114. J. W. Cohen. The Single Server Queue, 2nd edition. North-Holland, Amsterdam, 1982.
115. T. H. Cormen, C. E. Leiserson, and R. L. Rivest. Introduction to Algorithms. MIT Press, MA, 1990.
116. C. Courcoubetis and R. Weber. Stability of on-line bin packing with random arrivals and long-run average constraints. Probab. Eng. Inf. Sci., 4:447–460, 1990.
117. T. M. Cover. Comments on broadcast channels. IEEE Trans. Inf. Theory, 44(6): 2524– 2530, 1998. Information theory: 1948–1998.
118. T. M. Cover and J. A. Thomas. Elements of information theory. John Wiley, New York, 1991. A Wiley-Interscience Publication.
119. D. R. Cox and W. L. Smith. Queues. Methuen's Monographs on Statistical Subjects. Methuen, London, 1961.
120. J. Cox, S. Ross, and M. Rubinstein. Option pricing: a simplified approach. J. Financ. Econ., 7:229–263, 1979.
121. R. L. Cruz. A calculus for network delay. I. Network elements in isolation. IEEE Trans. Inf. Theory, 37(1):114–131, 1991.
122. J. Csirik, D. S. Johnson, C. Kenyon, J. B. Orlin, P. W. Shor, and R. R. Weber. On the sum-of-squares algorithm for bin packing. J. Assoc. Comput. Mach., 53(1):1–65, 2006.
123. M. Dacre, K. Glazebrook, and J. Niño-Mora. The achievable region approach to the optimal control of stochastic systems. J. R. Stat. Soc. Ser. B, 61(4):747–791, 1999.
124. J. G. Dai. On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models. Ann. Appl. Probab., 5(1):49–77, 1995.
125. J. G. Dai. A fluid-limit model criterion for instability of multiclass queueing networks. Ann. Appl. Probab., 6:751–757, 1996.
126. J. G. Dai and J. M. Harrison. Reflected Brownian motion in an orthant: numerical methods for steady-state analysis. Ann. Appl. Probab., 2:65–86, 1992.
127. J. G. Dai, J. J. Hasenbein, and J. H. Vande Vate. Stability and instability of a two-station queueing network. Ann. Appl. Probab., 14(1):326–377, 2004.
128. J. G. Dai and W. Lin. Maximum pressure policies in stochastic processing networks. Oper. Res, 53(2):197–218, 2005.
129. J. G. Dai and S. P. Meyn. Stability and convergence of moments for multiclass queueing networks via fluid limit models. IEEE Trans. Aut. Control, 40:1889–1904, November 1995.
130. J. G. Dai and J. H. Vande Vate. The stability of two-station multi-type fluid networks. Oper. Res, 48:721–744, 2000.
131. J. G. Dai and Y. Wang. Nonexistence of Brownian models of certain multiclass queueing networks. Queueing Syst. Theory Appl., 13:41–46, May 1993.
132. J. G. Dai and G. Weiss. Stability and instability of fluid models for reentrant lines. Math. Oper. Res, 21(1):115–134, 1996.
133. J. G. Dai and G. Weiss. A fluid heuristic for minimizing makespan in job shops. Oper. Res, 50(4):692–707, 2002.
134. Y. Dallery and S. B. Gershwin. Manufacturing flow line systems: a review of models and analytical results. Queueing Syst. Theory Appl., 12(1–2):3–94, 1992.
135. G. B. Dantzig. Linear programming under uncertainty. Manage. Sci., 7(3): 197–206, April–July 1995.
136. G. B. Danzig, J. Folkman, and N. Shapiro. On the continuity of the minimum set of a continuous function. J. Math. Anal. Appl., 17:519–548, 1967.
137. D. P. de Farias and B. Van Roy. The linear programming approach to approximate dynamic programming. Oper. Res., 51(6):850–865, 2003.
138. Y. De Serres. Simultaneous optimization of flow control and scheduling in a single server queue with two job classes. Oper. Res. Lett., 10(2):103–112, 1991.
139. R. Dekker. Counterexamples for compact action Markov decision chains with average reward criteria. Comm. Stat. Stoch. Models, 3(3):357–368, 1987.
140. A. Dembo and O. Zeitouni. Large Deviations Techniques And Applications, 2nd edition. Springer-Verlag, New York, 1998.
141. E. Denardo. Contraction mappings underlying the theory of dynamic programming. SIAM Rev, 9:165–177, 1967.
142. E. V. Denardo and B. L. Fox. Multichain Markov renewal programs. SIAMJ. Appl. Math, 16:468–487, 1968.
143. J. L. Doob. Stochastic Processes. John Wiley, New York, 1953.
144. B. T. Doshi. Optimal control of the service rate in an M/G/1 queueing system. Adv. Appl. Probab, 10:682–701, 1978.
145. R. Douc, G. Fort, E. Moulines, and P. Soulier. Practical drift conditions for subgeometric rates of convergence. Ann. Appl. Probab., 14(3):1353–1377, 2004.
146. D. Down and S. P. Meyn. Stability of acyclic multiclass queueing networks. IEEE Trans. Autom. Control, 40:916–919, 1995.
147. D. Down and S. P. Meyn. Piecewise linear test functions for stability and instability of queueing networks. Queueing Syst. Theory Appl., 27(3–4):205–226, 1997.
148. D. Down, S. P. Meyn, and R. L. Tweedie. Exponential and uniform ergodicity of Markov processes. Ann. Probab., 23(4):1671–1691, 1995.
149. L. E. Dubins. On extreme points of convex sets. J. Math. Anal. Appl., 5:237–244, 1962.
150. L. E. Dubins and Leonard J. Savage. How to Gamble If You Must: Inequalities for Stochastic Processes. McGraw-Hill, New York, 1965.
151. R. Dubrawski. Myopic and Far-Sighted Strategies for Control of Demand-Driven Networks. Master's thesis, Department of Electrical Engineering, University of Illinois at Urbana Champaign, Urbana, Illinois, 2000.
152. I. Duenyas. A simple release policy for networks of queues with controllable inputs. Oper. Res, 42(6):1162–1171, 1994.
153. N. G. Duffield and Neil O’Connell. Large deviations and overflow probabilities for the general single-server queue, with applications. Math. Proc. Camb. Phil. Soc., 118(2):363–374, 1995.
154. W. T. M. Dunsmuir, S. P. Meyn, and G. Roberts. Obituary: Richard Lewis Tweedie. J. Appl. Probab, 39(2):441–454, 2002.
155. P. Dupuis and R. S. Ellis. A Weak Convergence Approach to the Theory of Large Deviations. Wiley Series in Probability and Statistics: Probability and Statistics. John Wiley, New York, 1997. A Wiley-Interscience Publication.
156. P. Dupuis and I. Hitoshi. On Lipschitz continuity of the solution mapping to the Skorokhod problem, with applications. Stochastics, 35(1):31–62, 1991.
157. P. Dupuis and I. Hitoshi. SDEs with oblique reflection on nonsmooth domains. Ann. Probab, 21(1):554–580, 1993.
158. P. Dupuis and K. Ramanan. Convex duality and the Skorokhod problem. I, II. Probab. Theory Relat. Fields, 115(2): 153–195, 197–236, 1999.
159. P. Dupuis and K. Ramanan. An explicit formula for the solution of certain optimal control problems on domains with corners. Theory Probab. Math. Stat. 63: 33–49, 2001; Teor. Jmovirn. Mat. Stat. 63:32–48, 2000.
160. P. Dupuis and R. J. Williams. Lyapunov functions for semimartingale reflecting Brownian motions. Ann. Appl. Probab., 22(2):680–702, 1994.
161. A. Dvoretzky, J. Kiefer, and J. Wolfowitz. The inventory problem. I. Case of known distributions of demand. Econometrica, 20:187–222, 1952.
162. E. B. Dynkin and A. A. Yushkevich. Controlled Markov Processes. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], volume 235. Springer-Verlag, Berlin, 1979. Translated from the Russian original by J. M. Danskin and C. Holland.
163. D. Eng, J. Humphrey, and S. P. Meyn. Fluid network models: linear programs for control and performance bounds. In J. Cruz, J. Gertler, and M. Peshkin, editors, Proceedings of the 13th IFAC World Congress, volume B, pages 19–24, San Francisco, California, 1996.
164. A. Ephrimedes, P. Varaiya, and J. Walrand. A simple dynamic routing problem. IEEE Trans. Autom. Control, 25:690–693, 1980.
165. A. Eryilmaz and R. Srikant. Joint congestion control, routing, and MAC for stability and fairness in wireless networks. IEEE J. Sel. Areas Commun., 24(8):1514–1524, 2006.
166. A. Eryilmaz, R. Srikant, and J. R. Perkins. Stable scheduling policies for fading wireless channels. IEEE/ACM Trans. Netw., 13(2):411–424, 2005.
167. S. N. Ethier and T. G. Kurtz. Markov Processes: Characterization and Convergence. John Wiley, New York, 1986.
168. P. Fairley. The unruly power grid. IEEE Spectr., 41(8):22–27, August 2004.
169. G. Fayolle. On random walks arising in queueing systems: ergodicity and transience via quadratic forms as Lyapounov functions – part I. Queueing Syst., 5:167–183, 1989.
170. G. Fayolle, V. A. Malyshev, and M. V. Menʹshikov. Topics in the Constructive Theory of Countable Markov Chains. Cambridge University Press, Cambridge, 1995.
171. G. Fayolle, V. A. Malyshev, M. V. Menshikov, and A. F. Sidorenko. Lyapunov functions for jackson networks. Math. Oper. Res., 18(4):916–927, 1993.
172. A. Federgruen and H. Groenevelt. Characterization and control of achievable performance in general queueing systems. Oper. Res., 36:733–741, 1988.
173. A. Federgruen, A. Hordijk, and H. C. Tijms. Denumerable state semi-Markov decision processes with unbounded costs, average cost criterion unbounded costs, average cost criterion. Stoch. Proc. Appl., 9(2):223–235, 1979.
174. A. Federgruen, P. J. Schweitzer, and H. C. Tijms. Denumerable undiscounted semi-Markov decision processes with unbounded rewards. Math. Oper. Res., 8(2):298–313, 1983.
175. A. Federgruen and P. Zipkin. Computing optimal (s, S) policies in inventory models with continuous demands. Adv. Appl. Probab., 17(2):424–442, 1985.
176. A. Federgruen and P. Zipkin. An inventory model with limited production capacity and uncertain demands. I. The average-cost criterion. Math. Oper. Res., 11(2):193–207, 1986.
177. A. Federgruen and P. Zipkin. An inventory model with limited production capacity and uncertain demands. II. The discounted-cost criterion. Math. Oper. Res., 11(2):208–215, 1986.
178. E. A. Feinberg and A. Shwartz, editors. Handbook of Markov Decision Processes: Methods and Applications. International Series in Operations Research & Management Science, 40. Kluwer Academic Publishers, Boston, MA, 2002.
179. Final report on the August 14, 2003 blackout in the United States and Canada: causes and recommendations. https://reports.energy.gov/, April 2004.
180. L. K. Fleischer. Faster algorithms for the quickest transshipment problem. SIAM J. Control Optim., 12(1): 18–35 (electronic), 2001.
181. L. K. Fleischer. Universally maximum flow with piecewise-constant capacities. Networks, 38(3):115–125, 2001.
182. S. Floyd and V. Jacobson. Random early detection gateways for congestion avoidance. IEEE/ACM Trans. Netw, 1(4):397–413, 1993.
183. R. D. Foley and D. R. McDonald. Join the shortest queue: stability and exact asymptotics. Ann. Appl. Probab., 11(3):569–607, 2001.
184. R. D. Foley and D. R. McDonald. Large deviations of a modified Jackson network: Stability and rough asymptotics. Ann. Appl. Probab., 15(1B):519–541, 2005.
185. H. Föllmer and P. Protter. On Itô's formula for multidimensional Brownian motion. Probab. Theory Relat. Fields, 116(1):1–20, 2000.
186. N.-T. Fong and X. Y. Zhou. Hierarchical feedback controls in two-machine flow shops under uncertainty. In Proceedings of the 35th Conference on Decision and Control, pages 1743–1748, Kobe, Japan, 1996.
187. H. Ford. My Life and Work. Kessinger Publishing, Montana, 1922. Reprint January 2003.
188. L. R. Ford Jr. Network flow theory. Paper P-923, The RAND Corporation, Santa Moncia, CA, August 1956.
189. G. Fort, S. Meyn, E. Moulines, and P. Priouret. ODE methods for Markov chain stability with applications to MCMC. In Proceedings of VALUETOOLS’06, Pisa Italy, 2006.
190. G. Foschini and J. Salz. A basic dynamic routing problem and diffusion. IEEE Trans. Commun, 26:320–327, 1978.
191. F. G. Foster. On the stochastic matrices associated with certain queuing processes. Ann. Math. Stat, 24:355–360, 1953.
192. A. Gajrat, A. Hordijk, and A. Ridder. Large deviations analysis of the fluid approximation for a controllable tandem queue. Ann. Appl. Probab., 13:1423–1448, 2003.
193. R. G. Gallager. An inequality on the capacity region of multiaccess multipath channels. In U. Maurer, R. E. Blahut, D. J. Costello Jr. and T. Mittelholzer, editors, Communications and Cryptography: Two Sides of One Tapestry. The International Series in Engineering and Computer Science, pages 129–139. Kluwer, Norwell, MA, 1994.
194. D. Gamarnik. On deciding stability of constrained homogeneous random walks and queueing systems. Math. Oper. Res., 27(2):272–293, 2002.
195. D. Gamarnik and J. J. Hasenbein. Instability in stochastic and fluid queueing networks. Ann. Appl. Probab., 15(3):1652–1690, 2005.
196. D. Gamarnik and S. P. Meyn. On exponential ergodicity in multiclass queueing networks. Asymptotic Analysis of Stochastic Systems, invited session at the INFORMS Annual Meeting, November 13–16, 2005.
197. D. Gamarnik and A. Zeevi. Validity of heavy traffic steady-state approximations in generalized jackson networks. Adv. Appl. Probab., 16(1):56–90, 2006.
198. A. Ganesh, N. O’Connell, and D. Wischik. Big Queues. Lecture Notes in Mathematics, volume 1838. Springer-Verlag, Berlin, 2004.
199. N. Gans, G. Koole, and A. Mandelbaum. Commissioned paper: telephone call centers: tutorial, review, and research prospects. Manuf. Serv. Oper.Manage., 5(2):79–141, 2003.
200. E. Gelenbe and I. Mitrani. Analysis and Synthesis of Computer Systems. Academic Press, New York-London, 1980.
201. L. Georgiadis, W. Szpankowski, and L. Tassiulas. A scheduling policy with maximal stability region for ring networks with spatial reuse. Queueing Syst. Theory Appl., 19(1– 2):131–148, 1995.
202. S. B. Gershwin. A hierarchical framework for discrete event scheduling in manufacturing systems. In Proceedings of IIASA Conference on Discrete Event System: Models and Applications (Sopron. Hungary, 1987), pages 197–216, 1988.
203. S. B. Gershwin. Manufacturing Systems Engineering. Prentice Hall, Englewood Cliffs, NJ, 1993.
204. S. B. Gershwin. Production and subcontracting strategies for manufacturers with limited capacity and volatile demand. IIE Trans. Des. Manuf., 32(2): 891–906, 2000. Special Issue on decentralized control of manufacturing systems.
205. R. J. Gibbens and F. P. Kelly. Dynamic routing in fully connected networks. IMA J. Math. Cont. Inf., 7:77–111, 1990.
206. R. J. Gibbens and F. P. Kelly. Measurement-based connection admission control, 1997. In Fifteenth International Teletraffic Congress, Amsterdam, June 1997.
207. J. C. Gittins. Bandit processes and dynamic allocation indices. J. R. Stat. Soc. Ser. B, 41:148–177, 1979.
208. P. Glasserman and D. D. Yao. Monotone Structure in Discrete–Event Systems. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. John Wiley, New York, 1994. A Wiley-Interscience Publication.
209. P. Glynn and R. Szechtman. Some new perspectives on the method of control variates. In K. T. Fang, F. J. Hickernell, and H. Niederreiter, editors, Monte Carlo and Quasi-Monte Carlo Methods 2000: Proceedings of a Conference held at Hong Kong Baptist University, Hong Kong SAR, China, pages 27–49, Berlin, 2002. Springer-Verlag.
210. P. W. Glynn. Some asymptotic formulas for markov chains with applications to simulation. J. Stat. Comput. Simul., 19:97–112, 1984.
211. P. W. Glynn. Diffusion approximations. In Handbooks in Operations Research and Management Science, 2: Stochastic Models, pages 145–198. North–Holland, Amsterdam, 1990.
212. P. W. Glynn and D. L. Iglehart. A joint Central Limit Theorem for the sample mean and the regenerative variance estimator. Ann. Oper. Res., 8:41–55, 1987.
213. P. W. Glynn and D. L. Iglehart. Simulation methods for queues: an overview. Queueing Syst. Theory Appl, 3(3):221–255, 1988.
214. P. W. Glynn and S. P. Meyn. A Liapounov bound for solutions of the Poisson equation. Ann. Probab, 24(2):916–931, 1996.
215. P. W. Glynn and D. Ormoneit. Hoeffding's inequality for uniformly ergodic Markov chains. Stat. Probab. Lett., 56:143–146, 2002.
216. P. W. Glynn and W. Whitt. The asymptotic efficiency of simulation estimators. Oper. Res, 40(3):505–520, 1992.
217. P. W. Glynn and W. Whitt. Logarithmic asymptotics for steady-state tail probabilities in a single-server queue. Stud. Appl. Probab., 17:107–128, 1993.
218. C. C. Goodwin, M. M. Seron, and J. A. De Doná. Constrained Control and Estimation: An Optimisation Approach. Communications and Control Engineering. Springer-Verlag, London, 2005.
219. G. C. Goodwin and K. S. Sin. Adaptive Filtering Prediction and Control. Prentice Hall, Englewood Cliffs, NJ, 1984.
220. T. C. Green and S. Stidham. Sample-path conservation laws, with applications to scheduling queues and fluid systems. Queueing Syst. Theory Appl., 36:175–199, 2000.
221. F. Guillemin and D. Pinchon. On the area swept under the occupation process of an M/M/1 queue in a busy period. Queueing Syst. Theory Appl, 29(2–4):383–398, 1998.
222. B. Hajek. Optimal control of two interacting service stations. IEEE Trans. Autom. Control, AC-29:491–499, 1984.
223. B. Hajek. Extremal splittings of point processes. Math. Oper. Res., 10(4):543–556, 1985.
224. B. Hajek and R. G. Ogier. Optimal dynamic routing in communication networks with continuous traffic. Networks, 14(3):457–487, 1984.
225. P. Hall and C. C. Heyde. Martingale Limit Theory and Its Application. Probability and Mathematical Statistics. Academic Press. [Harcourt Brace Jovanovich Publishers], New York, 1980.
226. S. V. Hanly and D. N. C. Tse. Multiaccess fading channels. II. Delay-limited capacities. IEEE Trans. Inf. Theory, 44(7):2816–2831, 1998.
227. R. Hariharan, M. S. Moustafa, and S. Stidham Jr. Scheduling in a multi-class series of queues with deterministic service times. Queueing Syst. Theory Appl., 24:83–89, 1997.
228. J. M. Harrison. Discrete dynamic programming with unbounded rewards. Ann. Math. Statist, 43:636–644, 1972.
229. J. M. Harrison. A limit theorem for priority queues in heavy traffic. J. Appl. Probab, 10:907–912, 1973.
230. J. M. Harrison. Dynamic scheduling of a multiclass queue: discount optimality. Oper. Res, 23(2): 370–382, March–April 1975.
231. J. M. Harrison. Brownian Motion and Stochastic Flow Systems. John Wiley, New York, 1985.
232. J. M. Harrison. Brownian models of queueing networks with heterogeneous customer populations. In Stochastic Differential Systems, Stochastic Control Theory and Applications (Minneapolis, Minn., 1986), pages 147–186. Springer, New York, 1988.
233. J. M. Harrison. Brownian Motion and Stochastic Flow Systems. Robert E. Krieger, Malabar, FL, 1990. Reprint of the 1985 original.
234. J. M. Harrison. In F. P. Kelly, S. Zachary, and I. Ziedins, editors, The BIGSTEP approach to flow management in stochastic processing Networks, pages 57–89. In Stochastic Networks Theory and Applications. Clarendon Press, Oxford, 1996.
235. J. M. Harrison. Heavy traffic analysis of a system with parallel servers: asymptotic optimality of discrete-review policies. Ann. Appl. Probab., 8(3):822–848, 1998.
236. J. M. Harrison. Brownian models of open processing networks: canonical representations of workload. Ann. Appl. Probab., 10:75–103, 2000.
237. J. M. Harrison. Stochastic networks and activity analysis. In Y. Suhov, editor, Analytic Methods in Applied Probability. In Memory of Fridrih Karpelevich. American Mathematical Society, Providence, RI, 2002.
238. J. M. Harrison. Correction: “Brownian models of open processing networks: canonical representation of workload” [Ann. Appl. Probab. 10(1): 75–103, 2000; MR1765204 (2001g:60230)]. Ann. Appl. Probab., 13(1):390–393, 2003.
239. J. M. Harrison and M. J. López. Heavy traffic resource pooling in parallel-server systems. Queueing Syst. Theory Appl., 33:339–368, 1999.
240. J. M. Harrison and M. I. Reiman. On the distribution of multidimensional reflected Brownian motion. SIAM J. Appl. Math., 41(2):345–361, 1981.
241. J. M. Harrison and J. A. Van Mieghem. Dynamic control of Brownian networks: state space collapse and equivalent workload formulations. Ann. Appl. Probab., 7(3):747–771, 1997.
242. J. M. Harrison and L. M. Wein. Scheduling networks of queues: heavy traffic analysis of a simple open network. Queueing Syst. Theory Appl., 5(4):265–279, 1989.
243. J. M. Harrison and L. M. Wein. Scheduling networks of queues: heavy traffic analysis of a two-station closed network. Oper. Res., 38(6):1052–1064, 1990.
244. J. M. Harrison and R. J. Williams. Brownian models of open queueing networks with homogeneous customer populations. Stochastics, 22(2):77–115, 1987.
245. J. M. Harrison and R. J. Williams. Multidimensional reflected Brownian motions having exponential stationary distributions. Ann. Probab., 15(1):115–137, 1987.
246. J. M. Harrison and R. J. Williams. On the quasireversibility of a multiclass Brownian service station. Ann. Probab., 18(3):1249–1268, 1990.
247. J. M. Harrison and R. J. Williams. Brownian models of feedforward queueing networks: quasireversibility and product form solutions. Ann. Appl. Probab., 2(2):263–293, 1992.
248. M. Hauskrecht and B. Kveton. Linear program approximations to factored continuous- state Markov decision processes. In Advances in Neural Information Processing Systems, volume 16. MIT Press, Cambridge, MA, 895–902, 2004.
249. P. Heidelberger. Variance reduction techniques for simulating markov chains. In WSC’77: Proceedings of the 9th Conference on Winter Simulation, pages 160–164. Winter Simulation Conference, Gaitherburg, MD, 1977.
250. P. Heidelberger. Variance Reduction Techniques for the Simulation of Markov Processes. PhD thesis, Stanford University, Palo Alto, CA, 1977.
251. S. G. Henderson. Variance Reduction Via an Approximating Markov Process. PhD thesis, Stanford University, Stanford, CA, 1997.
252. S. G. Henderson. Mathematics for simulation. In S. G. Henderson and B. L. Nelson, editors, Handbook of Simulation. Handbooks in Operations Research and Management Science. Elsevier, 2006.
253. S. G. Henderson and P. W. Glynn. Approximating martingales for variance reduction in Markov process simulation. Math. Oper. Res., 27(2):253–271, 2002.
254. S. G. Henderson and S. P. Meyn. Efficient simulation of multiclass queueing networks. In S. Andradottir, K. Healy, D. H. Withers, and B. L. Nelson, editors, Proceedings of the 1997 Winter Simulation Conference, pages 216–223, Piscataway, NJ, 1997. IEEE.
255. S. G. Henderson and S. P. Meyn. Variance reduction for simulation in multiclass queueing networks. IIE Trans. Oper. Eng., to appear.
256. S. G. Henderson, S. P. Meyn, and V. B. Tadić. Performance evaluation and policy selection in multiclass networks. Discrete Event Dyn. Syst. Theory Appl, 13(1–2):149–189, 2003. Special issue on learning, optimization and decision making. Invited.
257. S. G. Henderson and B. L. Nelson, editors. Handbook of Simulation. Handbooks in Operations Research and Management Science, XII. Elsevier, Cambridge, MA, 2005.
258. O. Hernández-Lerma and J. B. Lasserre. Discrete–Time Markov Control Processes. Basic Optimality Criteria. Applications of Mathematics (New York), volume 30. SpringerVerlag, New York, 1996.
259. O. Hernández-Lerma and J. B. Lasserre. Further Topics on Discrete–Time Markov Control Processes. Stochastic Modeling and Applied Probability. Springer–Verlag, Berlin, Germany, 1999.
260. O. Hernández-Lerma and J. B. Lasserre. The linear programming approach. In Handbook of Markov Decision Processes. International Series in Operations Research and Management Science, volume 40. Kluwer Academic Publisher, Boston, MA, pp. 377–407, 2002.
261. O. Hernández-Lerma and J. B. Lasserre. Markov Chains and Invariant Probabilities. Progress in Mathematics, volume 211. Birkhauser Verlag, Basel, 2003.
262. D. P. Heyman and M. J. Sobel. Stochastic Models in Operations Research, 1: Stochastic Processes and Operating Characteristics. Dover Publications, Mineola, NY, 2004. Reprint of the 1982 original.
263. D. P. Heyman and M. J. Sobel. Stochastic Models in Operations Research, 2: Stochastic Optimization. Dover Publications, Mineola, NY, 2004. Reprint of the 1984 original.
264. D. P. Heyman and M. J. Sobel, editors. In Handbooks in Operations Research and Management Science, 2: Stochastic Models. North–Holland, Amsterdam, 1990.
265. F. S. Hillier and G. J. Lieberman. Introduction to Operations Research, 7th edition. McGraw–Hill, New York, 2002.
266. W. Hoeffding. Probability inequalities for sums of bounded random variables. J. Am. Stat. Assoc, 58:13–30, 1963.
267. W. J. Hopp and M. L. Spearman. Factory Physics: Foundations of Manufacturing Management. Irwin/McGraw–Hill, New York, 2001.
268. A. Hordijk. Dynamic Programming and Markov Potential Theory, Second Edition. Mathematical Centre Tracts 51, Mathematisch Centrum, Amsterdam, 1977.
269. A. Hordijk, D. L. Iglehart, and R. Schassberger. Discrete time methods for simulating continuous time Markov chains. Adv. Appl. Probab., 8:772–788, 1979.
270. A. Hordijk and L. C. M. Kallenberg. Linear programming and Markov decision processes. In Second Symposium on Operations Research, Teil 1, Aachen 1977 (RheinischWestfälische Tech. Hochsch. Aachen, Aachen, 1977), Teil 1, pages 400–406, Hain, 1978
271. A. Hordijk and N. Popov. Large deviations bounds for face–homogeneous random walks in the quarter–plane. Probab. Eng. Inf. Sci., 17(3), 2003.
272. A. Hordijk and F. M. Spieksma. On ergodicity and recurrence properties of a Markov chain with an application. Adv. Appl. Probab., 24:343–376, 1992.
273. R. A. Howard. Dynamic Programming and Markov Processes. John Wiley/MIT Press, New York, 1960.
274. C. Huitema. Routing in the Internet. Prentice Hall, Englewood Cliffs, NJ, 1999.
275. D. L. Iglehart and G. S. Shedler. Regenerative Simulation of Response Times in Networks of Queues. Lecture Notes in Control and Information Sciences, volume 26. SpringerVerlag, New York, 1980.
276. D. L. Iglehart and W. Whitt. Multiple channel queues in heavy traffic. I. Adv. Appl. Probab, 2:150–177, 1970.
277. D. L. Iglehart and W. Whitt. Multiple channel queues in heavy traffic. II. Sequences, networks, and batches. Adv. Appl. Probab., 2:355–369, 1970.
278. M. J. Ingenoso. Stability Analysis for Certain Queueing Systems and Multi–Access Communication Channels. PhD thesis, University of Wisconsin, Madison, WI, 2004.
279. ITRS public home page. Provides a roadmap of the needs and challenges facing the semiconductor industry. http://public.itrs.net/.
280. J. R. Jackson. Jobshop–like queueing systems. Manage. Sci., 10:131–142, 1963.
281. P. A. Jacobson and E. D. Lazowska. Analyzing queueing networks with simultaneous resource possession. Commun. ACM, 25(2):142–151, 1982.
282. G. L. Jones. On the Markov chain central limit theorem. Probab. Surv., 1:299–320 (electronic), 2004.
283. I. Karatzas and S. Shreve. Brownian Motion and Stochastic Calculus, 2nd edition. Springer–Verlag, New York, 1988.
284. M. J. Karol, M. G. Hluchyj, and S. P. Morgan. Input versus output queueing on a spacedivision packet switch. IEEE Trans. Commun., 35:1347–1356, 1987.
285. N. V. Kartashov. Criteria for uniform ergodicity and strong stability of Markov chains with a common phase space. Theory Probab. Appl., 30:71–89, 1985.
286. N. V. Kartashov. Inequalities in theorems of ergodicity and stability for Markov chains with a common phase space. Theory Probab. Appl., 30:247–259, 1985.
287. H. Kaspi and A. Mandelbaum. Regenerative closed queueing networks. Stoch. Stoch. Rep., 39(4):239–258, 1992.
288. M. Kearns and S. Singh. Bias–variance error bounds for temporal difference updates. In Proceedings of the 13th Annual Conference on Computational Learning Theory, Stanford University, pages 142–147, 2000.
289. J. B. Keller and H. P. McKean, editors. Stochastic Differential Equations. American Mathematical Society, Providence, RI, 1973.
290. F. P. Kelly. Reversibility and Stochastic Networks. Wiley Series in Probability and Mathematical Statistics. John Wiley, Chichester, UK, 1979.
291. F. P. Kelly. Dynamic routing in stochastic networks. In F. Kelly and R. Williams, editors, IMA Volumes in Mathematics and Its Applications, volume 71, pages 169–186, SpringerVerlag, New York, 1995.
292. F. P. Kelly and C. N. Laws. Dynamic routing in open queueing networks: Brownian models, cut constraints and resource pooling. Queueing Syst. Theory Appl., 13:47–86, 1993.
293. D. G. Kendall. Some problems in the theory of queues. J. R. Stat. Soc. Ser. B, 13:151–185, 1951.
294. D. G. Kendall. Stochastic processes occurring in the theory of queues and their analysis by means of the imbedded Markov chain. Ann. Math. Stat., 24:338–354, 1953.
295. I. G. Kevrekidis, C. W. Gear, and G. Hummer. Equation–free: the computer–assisted analysis of complex, multiscale systems. Ann. Inst. Chem. Eng. J, 50:1346–1354, 2004.
296. R. Z. Khas’minskii. Stochastic Stability of Differential Equations. Sijthoff & Noordhoff, Netherlands, 1980.
297. J. Kiefer and J. Wolfowitz. Stochastic estimation of the maximum of a regression function. Ann. Math. Stat., 23:462–466, 1952.
298. E. Kim and M. P. Van Oyen. Beyong the c–p rule: dynamic scheduling of a two–class loss queue. Math. Methods Oper. Res, 48(1):17–36, 1998.
299. S. Kim and S. G. Henderson. Adaptive control variates. In R. Ingalls, M. Rossetti, J. Smith, and B. Peters, editors, Proceedings of the 2004 Winter Simulation Conference, pages 621–629, Piscataway, NJ, 2004. IEEE.
300. T. Kimura. A bibliography of research on heavy traffic limit theorems for queues. Econ.J. Hokkaido Univ., 22:167–179, 1993.
301. J. F. C. Kingman. The single server queue in heavy traffic. Proc. Camb. Phil. Soc., 57:902–904, 1961.
302. J. F. C. Kingman. On queues in heavy traffic. J. R. Stat. Soc. Ser. B, 24:383–392, 1962.
303. L. Kleinrock. Queueing Systems Vol. 1: Theory. John Wiley, New York, 1975.
304. L. Kleinrock. Queueing Systems Vol. 2: Computer Applications. John Wiley, New York, 1976.
305. G. P. Klimov. Time–sharing queueing systems. I. Teor. Veroyatn. Primen., 19:558–576, 1974.
306. G. P. Klimov. Procesy obstugi masowej (Polish) [Queueing Theory]. Wydawnictwa Naukowo–Techniczne (WNT), Warsaw, 1979. Translated from the Russian by E. Fidelis and T. Rolski.
307. P. V. Kokotovic, J. O’Reilly, and J. K. Khalil. Singular Perturbation Methods in Control: Analysis and Design. Academic Press, Orlando, FL, 1986.
308. V. R. Konda and J. N. Tsitsiklis. On actor–critic algorithms. SIAM J. Control Optim., 42(4):1143–1166 (electronic), 2003.
309. I. Kontoyiannis, L. A. Lastras-Montaño, and S. P. Meyn. Relative entropy and exponential deviation bounds for general Markov chains. In Proceedings of the IEEE International Symposium on Information Theory, Adelaide, Australia, 4–9 September, 2005.
310. I. Kontoyiannis and S. P. Meyn. Spectral theory and limit theorems for geometrically ergodic Markov processes. Ann. Appl. Probab., 13:304–362, 2003. Presented at the INFORMS Applied Probability Conference, NYC, July 2001.
311. I. Kontoyiannis and S. P. Meyn. Large deviations asymptotics and the spectral theory of multiplicatively regular Markov processes. Electron. J. Probab., 10(3):61–123 (electronic), 2005.
312. I. Kontoyiannis and S. P. Meyn. Computable exponential bounds for screened estimation and simulation. To appear Ann. Appl. Prob., 2008.
313. L. Kruk, J. Lehoczky, K. Ramanan, and S. Shreve. An explicit formula for the skorohod map on [0, a]. Submitted for publication, March 2006.
314. N. V. Krylov. Controlled Diffusion Processes. Applications of Mathematics, volume 14. Springer–Verlag, New York, 1980. Translated from the Russian by A. B. Aries.
315. N. V. Krylov. On a proof of Ito's formula. Trudy Mat. Inst. Steklov., 202:170–174, 1993.
316. P. R. Kumar and S. P. Meyn. Stability of queueing networks and scheduling policies. IEEE Trans. Autom. Control, 40(2):251–260, February 1995.
317. P. R. Kumar and S. P. Meyn. Duality and linear programs for stability and performance analysis queueing networks and scheduling policies. IEEE Trans. Autom. Control, 41(1):4–17, 1996.
318. P. R. Kumar and T. I. Seidman. Dynamic instabilities and stabilization methods in distributed real–time scheduling of manufacturing systems. IEEE Trans. Autom. Control, AC–35(3):289–298, March 1990.
319. S. Kumar and P. R. Kumar. Performance bounds for queueing networks and scheduling policies. IEEE Trans. Autom. Control, AC–39:1600–1611, August 1994.
320. S. Kumar and P. R. Kumar. Closed queueing networks in heavy traffic: fluid limits and efficiency. In P. Glasserman, K. Sigman, and D. Yao, editors, Stochastic Networks: Stability and Rare Events. Lecture Notes in Statistics, volume 117, pages 41–64. Springer–Verlag, New York, 1996.
321. S. Kumar and P. R. Kumar. Fluctuation smoothing policies are stable for stochastic reentrant lines. Discrete Event Dyn. Syst. Theory Appl., 6(4):361–370, October 1996.
322. H. J. Kushner. Numerical methods for stochastic control problems in continuous time. SIAM J. Control Optim., 28(5):999–1048, 1990.
323. H. J. Kushner. Heavy Traffic Analysis of Controlled Queueing and Communication Networks. Stochastic Modelling and Applied Probability. Springer–Verlag, New York, 2001.
324. H. J. Kushner and Y. N. Chen. Optimal control of assignment of jobs to processors under heavy traffic. Stochastics, 68(3–4):177–228, 2000.
325. H. J. Kushner and P. G. Dupuis. Numerical Methods for Stochastic Control Problems in Continuous Time. Springer–Verlag, London, 1992.
326. H. J. Kushner and L. F. Martins. Numerical methods for stochastic singular control problems. SIAM J. Control Optim., 29(6):1443–1475, 1991.
327. H. J. Kushner and K. M. Ramchandran. Optimal and approximately optimal control policies for queues in heavy traffic. SIAM J. Control Optim., 27:1293–1318, 1989.
328. H. J. Kushner and G. G. Yin. Stochastic Approximation Algorithms and Applications. Applications of Mathematics (New York), volume 35. Springer–Verlag, New York, 1997.
329. H. Kwakernaak and R. Sivan. Linear Optimal Control Systems. Wiley–Interscience, New York, 1972.
330. S. S. Lavenberg and P. D. Welch. A perspective on the use of control variables to increase the efficiency of Monte Carlo simulations. Manage. Sci., 27:322–335, 1981.
331. A. M. Law and W. D. Kelton. Simulation Modeling and Analysis, 3rd edition. McGraw– Hill, New York, 2000.
332. C. N. Laws and G. M. Louth. Dynamic scheduling of a four–station queueing network. Probab. Eng. Inf. Sci., 4:131–156, 1990.
333. N. Laws. Dynamic Routing in Queueing Networks. PhD thesis, Cambridge University, Cambridge, UK, 1990.
334. D. Leith and V. Subramanian. Draining time based scheduling algorithm. IEEE Conf. Dec. and Control. Submitted for publication, 2007.
335. H. Liao. Multiple–Access Channels. PhD thesis, Unversity of Hawaii, Manoa, Hawaii, 1972.
336. G. Liberopoulos and Y. Dallery. A unified framework for pull control mechanisms in multi–stage manufacturing systems. Ann. Oper. Res, 93(1):325–355, 2000.
337. W. Lin and P. R. Kumar. Optimal control of a queueing system with two heterogeneous servers. IEEE Trans. Autom. Control, AC–29:696–703, August 1984.
338. S. Lippman. Applying a new device in the optimization of exponential queueing systems. Oper. Res, 23:687–710, 1975.
339. Steven A. Lippman. On dynamic programming with unbounded rewards. Manage. Sci., 21(11):1225–1233, 1974/75.
340. W. W. Loh. On the Method of Control Variates. PhD thesis, Department of Operations Research, Stanford University, Stanford, CA, 1994.
341. S. H. Lu and P. R. Kumar. Distributed scheduling based on due dates and buffer priorities. IEEE Trans. Autom. Control, 36(12):1406–1416, December 1991.
342. D. G. Luenberger. Linear and Nonlinear Programming, 2nd edition. Kluwer Academic Publishers, Norwell, MA, 2003.
343. R. B. Lund, S. P. Meyn, and R. L. Tweedie. Computable exponential convergence rates for stochastically ordered Markov processes. Ann. Appl. Probab., 6(1):218–237, 1996.
344. X. Luo and D. Bertsimas. A new algorithm for state–constrained separated continuous linear programs. SIAM J. Control Optim, 37:177–210, 1998.
345. C. Maglaras. Dynamic scheduling in multiclass queueing networks: stability under discrete–review policies. Queueing Syst. Theory Appl., 31:171–206, 1999.
346. C. Maglaras. Discrete–review policies for scheduling stochastic networks: trajectory tracking and fluid–scale asymptotic optimality. Ann. Appl. Probab., 10(3):897–929, 2000.
347. V. A. Malyšev and M. V. Men'šikov. Ergodicity, continuity and analyticity of countable Markov chains. Trudy Moskov. Mat. Obshch., 39:3–48, 235, 1979; Trans. Mosc. Math. Soc., 1:1–48, 1981.
348. A. Mandelbaum and A. L. Stolyar. Scheduling flexible servers with convex delay costs: heavy–traffic optimality of the generalized cμ–rule. Oper. Res., 52(6):836–855, 2004.
349. A. S. Manne. Linear programming and sequential decisions. Manage. Sci., 6(3):259–267, 1960.
350. S. Mannor, I. Menache, and N. Shimkin. Basis function adaptation in temporal difference reinforcement learning. Ann. Oper. Res., 134(2):215–238, 2005.
351. D. Q. Mayne, J. B. Rawlings, C. V. Rao, and P. O. M. Scokaert. Constrained model predictive control: stability and optimality. Automatica, 36(6):789–814, June 2000.
352. R. D. McBride. Progress made in solving the multicommodity flow problem. SIAM J. Control Optim., 8(4):947–955, 1998.
353. H. P. McKean. Stochastic Integrals. AMS Chelsea Publishing, Providence, RI, 2005. Reprint of the 1969 edition, with errata.
354. M. Medard, S. P. Meyn, J. Huang, and A. J. Goldsmith. Capacity of time–slotted aloha systems. Trans. Wirel. Commun., 3(2):486–499, 2004/03. In Proceedings of ISIT, page 407, 2000.
355. S. P. Meyn. Transience of multiclass queueing networks via fluid limit models. Ann. Appl. Probab, 5:946–957, 1995.
356. S. P. Meyn. The policy iteration algorithm for average reward Markov decision processes with general state space. IEEE Trans. Autom. Control, 42(12):1663–1680, 1997.
357. S. P. Meyn. Stability and optimization of queueing networks and their fluid models. In Mathematics of Stochastic Manufacturing Systems (Williamsburg, VA, 1996), pages 175–199. American Mathematical Society, Providence, RI, 1997.
358. S. P. Meyn. Algorithms for optimization and stabilization of controlled Markov chains. Sādhanā, 24(4–5):339–367, 1999. Special invited issue: Chance as necessity.
359. S. P. Meyn. Sequencing and routing in multiclass queueing networks. Part I: Feedback regulation. SIAM J. Control Optim., 40(3):741–776, 2001.
360. S. P. Meyn. Stability, performance evaluation, and optimization. In E. Feinberg and a.Shwartz, editors, Markov Decision Processes: Models, Methods, Directions, and Open Problems, pages 43–82. Kluwer, Holland, 2001.
361. S. P. Meyn. Sequencing and routing in multiclass queueing networks. Part II: Workload relaxations. SIAM J. Control Optim., 42(1):178–217, 2003.
362. S. P. Meyn. Dynamic safety–stocks for asymptotic optimality in stochastic networks. Queueing Syst. Theory Appl., 50:255–297, 2005.
363. S. P. Meyn. Workload models for stochastic networks: value functions and performance evaluation. IEEE Trans. Autom. Control, 50(8):1106–1122, August 2005.
364. S. P. Meyn. Large deviation asymptotics and control variates for simulating large functions. Ann. Appl. Probab., 16(1):310–339, 2006.
365. S. P. Meyn. Stability and asymptotic optimality of generalized MaxWeight policies. Submitted for publication, 2006.
366. S. P. Meyn and D. G. Down. Stability of generalized Jackson networks. Ann. Appl. Probab., 4:124–148, 1994.
367. S. P. Meyn and R. L. Tweedie. Generalized resolvents and Harris recurrence of Markov processes. Contemp. Math., 149:227–250, 1993.
368. S. P. Meyn and R. L. Tweedie. Markov Chains and Stochastic Stability, 2nd edition. Springer–Verlag, London, 1993. http://black.csl.uiuc.edu/~meyn/pages/book.html.
369. S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: Foster–Lyapunov criteria for continuous time processes. Adv. Appl. Probab., 25:518–548, 1993.
370. S. P. Meyn and R. L. Tweedie. Computable bounds for convergence rates of Markov chains. Ann. Appl. Probab, 4:981–1011, 1994.
371. S. P. Meyn and R. L. Tweedie. State–dependent criteria for convergence of Markov chains. Ann. Appl. Probab., 4:149–168, 1994.
372. B. Mitchell. Optimal service–rate selection in an M/G/Î queue. SIAM J. Appl. Math., 24(1):19–35, 1973.
373. M. Mitzenmacher. The Power of Two Choices in Randomized Load Balancing. PhD thesis, University of California, Berkeley, CA, 1996.
374. M. Mitzenmacher. The power of two choices in randomized load balancing. IEEE Trans. Parallel Distrib. Syst., 12(10):1094–1104, 2001.
375. H. Mori. Transport, collective motion, and brownian motion. Prog. Theor. Phys., 33:423–455, 1965.
376. J. R. Morrison and P. R. Kumar. New linear program performance bounds for queueing networks. J. Optim. Theory Appl., 100(3):575–597, 1999.
377. A. Muharremoglu and J. N. Tsitsiklis. A single–unit decomposition approach to multiechelon inventory systems. Under revision for Operations Research. Preprint available at http://web.mit.edu/jnt/www/publ.html, 2003.
378. K. Murota. Note on multimodularity and L–convexity. Math. Oper. Res., 30(3):658–661, 2005.
379. K. G. Murty and S. N. Kabadi. Some np–complete problems in quadratic and nonlinear programming. Math. Program, 39(2):117–129, 1987.
380. P. Nash. Optimal Allocation of Resources between Research Projects. PhD thesis, Cambridge University, Cambridge, England, 1973.
381. A. Nedic and D. P. Bertsekas. Least squares policy evaluation algorithms with linear function approximation. Discrete Event Dyn. Syst. Theory Appl., 13(1–2):79–110, 2003.
382. B. L. Nelson. Control–variate remedies. Oper. Res., 38(4):974–992, 1990.
383. M. F. Neuts. Matrix–Geometric Solutions in Stochastic Models: An Algorithmic Approach. Dover Publications, New York, 1994. Corrected reprint of the 1981 original.
384. M. B. Nevel'son and R. Z. Has'minskiĭ. Stochastic Approximation and Recursive Estimation. American Mathematical Society, Providence, RI, 1973. Translated from the Russian by the Israel Program for Scientific Translations, Translations of Mathematical Monographs, volume 47.
385. G. F. Newell. Applications of Queueing Theory, 2nd edition. Monographs on Statistics and Applied Probability. Chapman & Hall, London, 1982.
386. V. Nguyen. Fluid and diffusion approximations of a two–station mixed queueing network. Math. Oper. Res, 20(2):321–354, 1995.
387. J. Niño-Mora. Restless bandit marginal productivity indices, diminishing returns, and optimal control of make–to–order/make–to–stock M/G/l queues. Math. Oper. Res., 31(1): 50–84, 2006.
388. E. Nummelin. General Irreducible Markov Chains and Nonnegative Operators. Cambridge University Press, Cambridge, 1984.
389. J. Ou and L. M. Wein. Performance bounds for scheduling queueing networks. Ann. Appl. Probab, 2:460–480, 1992.
390. C. H. Papadimitriou and J. N. Tsitsiklis. Intractable problems in control theory. SIAM J. Control Optim, 24(4):639–654, 1986.
391. C. H. Papadimitriou and J. N. Tsitsiklis. The complexity of Markov decision processes. Math. Oper. Res, 12(3):441–450, 1987.
392. C. H. Papadimitriou and J. N. Tsitsiklis. The complexity of optimal queueing network control. Math. Oper. Res, 24(2):293–305, 1999.
393. I. Ch. Paschalidis, D. Bertsimas and J. N. Tsitsiklis. Scheduling of multiclass queueing networks: bounds on achievable performance. In Workshop on Hierarchical Control for Real–Time Scheduling of Manufacturing Systems, Lincoln, New Hampshire, October 1618, 1992.
394. A. Peña-Perez and P. Zipkin. Dynamic scheduling rules for a multiproduct make–to–stock. Oper. Res., 45:919–930, 1997.
395. J. Perkins. Control of Push and Pull Manufacturing Systems. PhD thesis, University of Illinois, Urbana, IL, September 1993. Technical report no. UILU–ENG–93–2237 (DC– 155).
396. J. R. Perkins and P. R. Kumar. Stable distributed real–time scheduling of flexible manufacturing/assembly/disassembly systems. IEEE Trans. Autom. Control, AC–34:139–148, 1989.
397. J. R. Perkins and P. R. Kumar. Optimal control of pull manufacturing systems. IEEE Trans. Autom. Control, AC–40:2040–2051, 1995.
398. J. R. Perkins and R. Srikant. Failure–prone production systems with uncertain demand. IEEE Trans. Autom. Control, 46:441–449, 2001.
399. M. C. Pullan. An algorithm for a class of continuous linear programs. SIAM J. Control Optim, 31:1558–1577, 1993.
400. M. C. Pullan. Existence and duality theory for separated continuous linear programs. Math. Model. Syst, 3(3):219–245, 1995.
401. M. C. Pullan. Forms of optimal solutions for separated continuous linear programs. SIAM J. Control Optim., 33:1952–1977, 1995.
402. M. L. Puterman. Markov Decision Processes. John Wiley, New York, 1994.
403. M. L. Puterman and S. L. Brumelle. The analytic theory of policy iteration. In Dynamic Programming and Its Applications (Proc. Conf, Univ. British Columbia, Vancouver, BC, 1977), pages 91–113. Academic Press, New York, 1978.
404. M. L. Puterman and S. L. Brumelle. On the convergence of policy iteration in stationary dynamic programming. Math. Oper. Res., 4(1):60–69, 1979.
405. K. Ramanan. Reflected diffusions defined via the extended Skorokhod map. Electron. J. Probab., 11:934–992 (electronic), 2006.
406. R. R. Rao and A. Ephremides. On the stability of interacting queues in a multiple–access system. IEEE Trans. Inf. Theory, 34(5, part 1):918–930, 1988.
407. M. I. Reiman. Open queueing networks in heavy traffic. Math. Oper. Res, 9:441–458, 1984.
408. R. K. Ritt and L. I. Sennott. Optimal stationary policies in general state space Markov decision chains with finite action sets. Math. Oper. Res., 17(4):901–909, 1992.
409. H. Robbins and S. Monro. A stochastic approximation method. Ann. Math. Stat., 22:400–407, 1951.
410. J. Robinson. Fabtime – cycle time management for wafer fabs. http://www.FabTime.com.
411. J. K. Robinson, J. W. Fowler, and E. Neacy. Capacity loss factors in semiconductor manufacturing (working paper). http://www.fabtime.com/bibliogr.shtml, 1996.
412. S. Robinson. The price of anarchy. SIAM Newsl., 37(5):1–4, 2004.
413. L. C. G. Rogers and D. Williams. Diffusions, Markov Processes, and Martingales, volume 2. Cambridge Mathematical Library. Cambridge University Press, Cambridge, 2000. Ito calculus, Reprint of the second (1994) edition.
414. Z. Rosberg, P. P. Varaiya, and J. C. Walrand. Optimal control of service in tandem queues. IEEE Trans. Autom. Control, 27:600–610, 1982.
415. J. S. Rosenthal. Correction: “Minorization conditions and convergence rates for Markov chain Monte Carlo”. J. Am. Stat. Assoc., 90(431):1136, 1995.
416. J. S. Rosenthal. Minorization conditions and convergence rates for Markov chain Monte Carlo. J. Am. Stat. Assoc., 90(430):558–566, 1995.
417. S. M. Ross. Introduction to Stochastic Dynamic Programming. Academic Press, New York, 1984.
418. S. M. Ross. Applied Probability Models with Optimization Applications. Dover Publications, New York, 1992. Reprint of the 1970 original.
419. R. Y. Rubinstein and B. Melamed. Modern Simulation and Modeling. John Wiley, Chichester, UK, 1998.
420. A. N. Rybko and A. L. Stolyar. On the ergodicity of random processes that describe the functioning of open queueing networks. Probl. Pereda. Inf, 28(3):3–26, 1992.
421. C. H. Sauer and E. A. MacNair. Simultaneous resource possession in queueing models of computers. SIGMETRICS Perform. Eval. Rev, 7(1–2):41–52, 1978.
422. H. E. Scarf. The optimality of (S, s) policies in the dynamic inventory problem. In Mathematical Methods in the Social Sciences, 1959, pages 196–202. Stanford University Press, Stanford, CA, 1960.
423. H. E. Scarf. A survey of analytic techniques in inventory theory. In Multistage Inventory Models and Techniques, pages 185–225. Stanford University Press, Stanford, CA, 1963.
424. M. Schreckenberg and S. D. Sharma, editors. Pedestrian and Evacuation Dynamics. Springer, Berlin, 2002.
425. P. J. Schweitzer. Aggregation methods for large Markov chains. In Mathematical Computer Performance and Reliability (Pisa, 1983), pages 275–286. North–Holland, Amsterdam, 1984.
426. E. Schwerer. A Linear Programming Approach to the Steady–state Analysis of Markov Processes. PhD thesis, Stanford University, Stanford, CA, 1997.
427. E. Schwerer. A linear programming approach to the steady–state analysis of reflected Brownian motion. Stoch. Models, 17(3):341–368, 2001.
428. T. I. Seidman. First come first serve can be unstable. IEEE Trans. Autom. Control, 39(10):2166–2170, October 1994.
429. T. I. Seidman and L. E. Holloway. Stability of pull production control methods for systems with significant setups. IEEE Trans. Autom. Control, 47(10):1637–1647, 2002.
430. L. I. Sennott. Stochastic Dynamic Programming and the Control of Queueing Systems. Wiley Series in Probability and Statistics: Applied Probability and Statistics. John Wiley, New York, 1999. A Wiley–Interscience Publication.
431. L. I. Sennott. Average cost optimal stationary policies in infinite state Markov decision processes with unbounded cost. Oper. Res., 37:626–633, 1989.
432. L. I. Sennott. The convergence of value iteration in average cost Markov decision chains. Oper. Res. Lett., 19:11–16, 1996.
433. S. P. Sethi and G. L. Thompson. Optimal Control Theory: Applications to Management Science and Economics. Kluwer Academic Publishers, Boston, 2000.
434. S. P. Sethi and Q. Zhang. Hierarchical Decision Making in Stochastic Manufacturing Systems. Birkhauser Verlag, Basel, Switzerland, 1994.
435. D. Shah and D. Wischik. Optimal scheduling algorithm for input–queued switch. IEEE INFOCOM, 2006. 25th IEEE Intl. Conf. on Computer Comm., pp. 1–11.
436. M. Shaked and J. G. Shanthikumar. Stochastic Orders and Their Applications. Probability and Mathematical Statistics. Academic Press, Boston, MA, 1994.
437. S. Shakkottai and R. Srikant. Scheduling real–time traffic with deadlines over a wireless channel. In Proceedings of the 2nd ACM International Workshop on Wireless Mobile Multimedia, pages 35–42, Seattle, WA, 1999.
438. S. Shakkottai, R. Srikant, and A. L. Stolyar. Pathwise optimality of the exponential scheduling rule for wireless channels. Adv. Appl. Probab., 36(4):1021–1045, 2004.
439. S. Shamai and A. Wyner. Information theoretic considerations for symmetric, cellular, multiple–access fading channels – part I. IEEE Trans. Inf. Theory, 43(6):1877–1894, 1997.
440. J. G. Shanthikumar and U. Sumita. Convex ordering of sojourn times in single–server queues: extremal properties of fifo and lifo service disciplines. J. Appl. Probab., 24:737–748, 1987.
441. J. G. Shanthikumar and D. D. Yao. Stochastic monotonicity in general queueing networks. J. Appl. Probab., 26:413–417, 1989.
442. J. G. Shanthikumar and D. D. Yao. Multiclass queueing systems: polymatroid structure and optimal scheduling control. Oper. Res., 40:S293–S299, 1992.
443. X. Shen, H. Chen, J. G. Dai, and W. Dai. The finite element method for computing the stationary distribution of an SRBM in a hypercube with applications to finite buffer queueing networks. Queueing Syst. Theory Appl., 42(1):33–62, 2002.
444. S. Shreve. Lectures on stochastic calculus and finance. http://www2.cs.cmu.edu/~chal/shreve.html, 2003.
445. S. E. Shreve. Stochastic Calculus for Finance II: Continuous–Time Models. Springer Finance. Springer–Verlag, New York, 2004.
446. A. Shwartz and A. Weiss. Large Deviations for Performance Analysis: Queues, Communications, and Computing. Stochastic Modeling Series. Chapman & Hall, London, 1995. With an appendix by Robert J. Vanderbei.
447. K. Sigman. Queues as harris–recurrent Markov chains. Queueing Syst., 3:179–198, 1988.
448. K. Sigman. The stability of open queueing networks. Stoch. Proc. Appl., 35:11–25, 1990.
449. K. Sigman. A note on a sample–path rate conservation law and its relationship with h = Xg. Adv. Appl. Probab., 23:662–665, 1991.
450. R. Simon, R. Alonso-Zldivar, and T. Reiterman. Enron memos prompt calls for a wider investigation electricity: regulators order all energy trading companies to preserve documents on tactics. Los Angeles Times, July 2002.
451. A. V. Skorokhod. Stochastic equations for diffusions in a bounded region. Theory Probab. Appl, 6:264–274, 1961.
452. J. M. Smith and D. Towsley. The use of queuing networks in the evaluation of egress from buildings. Environ. Plann. B Plann. Des., 8(2):125–139, 1981.
453. W. E. Smith. Various optimizers for single–stage production. Nav. Res. Log. Q, 3:59–66, 1956.
454. H. M. Soner and S. E. Shreve. Regularity of the value function for a two–dimensional singular stochastic control problem. SIAM J. Control Optim, 27(4):876–907, 1989.
455. E. D. Sontag. Mathematical Control Theory: Deterministic Finite–Dimensional Systems. Texts in Applied Mathematics, volume 6, 2nd edition. Springer–Verlag, New York, 1998.
456. M. L. Spearman, W. J. Hopp, and D. L. Woodruff. A hierarchical control architecture for CONWIP production systems. J. Manuf. Oper. Manage., 3:147–171, 1989.
457. M. L. Spearman, D. L. Woodruff, and W. J. Hopp. CONWIP: a pull alternative to KANBAN. Int. J. Prod. Res, 28:879–894, 1990.
458. F. M. Spieksma. Geometrically Ergodic Markov Chains and the Optimal Control of Queues. PhD thesis, University of Leiden, Leiden, the Netherlands, 1991.
459. R. Srikant. The Mathematics of Internet Congestion Control. Systems & Control: Foundations & Applications. Birkhäuser Boston, Boston, MA, 2004.
460. S. Stidham Jr. and R. Weber. A survey of Markov decision models for control of networks of queues. Queueing Syst. Theory Appl., 13(1–3):291–314, 1993.
461. A. L. Stolyar. On the stability of multiclass queueing networks: a relaxed sufficient condition via limiting fluid processes. Markov Process. Relat. Fields, 1(4):491–512, 1995.
462. A. L. Stolyar. Maxweight scheduling in a generalized switch: state space collapse and workload minimization in heavy traffic. Adv. Appl. Probab., 14(1):1–53, 2004.
463. S. H. Strogatz. Exploring complex networks. Nature, 410:268–276, 2001.
464. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, MA, 1998, http://www.cs.ualberta.ca/~sutton/book/the-book.html.
465. V. Tadic and S. P. Meyn. Adaptive Monte–Carlo algorithms using control–variates. In Proceedings of the American Control Conference, Denver, CO, June 2003.
466. H. Takagi. Queueing Analysis: A Foundation of Performance Evaluation: Finite Systems, volume 2. North–Holland, Amsterdam, 1993.
467. L. Tassiulas. Adaptive back–pressure congestion control based on local information. IEEE Trans. Autom. Control, 40(2):236–250, 1995.
468. L. Tassiulas and A. Ephremides. Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks. IEEE Trans. Autom. Control, 37(12):1936–1948, 1992.
469. L. M. Taylor and R. J. Williams. Existence and uniqueness of semimartingale reflecting Brownian motions in an orthant. Prob.Theory Relat. Fields, 96(3):283–317, 1993.
470. D. N. C. Tse and S. V. Hanly. Multiaccess fading channels. I. Polymatroid structure, optimal resource allocation and throughput capacities. IEEE Trans. Inf. Theory, 44(7):2796–2815, 1998.
471. P. Tseng. Solving H–horizon, stationary Markov decision problems in time proportional to log(H). Oper. Res. Lett., 9(5):287–297, 1990.
472. J. N. Tsitsiklis. Periodic review inventory systems with continuous demand and discrete order sizes. Manage. Sci., 30(10):1250–1254, 1984.
473. J. N. Tsitsiklis. A short proof of the Gittins index theorem. Ann. Appl. Probab, 4(1):194–199, 1994.
474. J. N. Tsitsiklis and B. Van Roy. An analysis of temporal–difference learning with function approximation. IEEE Trans. Autom. Control, 42(5):674–690, 1997.
475. P. Tsoucas and J. Walrand. Monotonicity of throughput in non–Markovian networks. J.Appl. Probab., 26:134–141, 1989.
476. P. Tuominen and R. L. Tweedie. Subgeometric rates of convergence of f–ergodic Markov chains. Adv. Appl. Probab., 26:775–798, 1994.
477. J. A. Van Mieghem. Dynamic scheduling with convex delay costs: the generalized c–p rule. Ann. Appl. Probab., 5(3):809–833, 1995.
478. J. A. E. E. van Nunen and J. Wessels. A note on dynamic programming with unbounded rewards. Management Sci., 24(5):576–580, 1977/78.
479. B. Van Roy. Neuro–dynamic programming: overview and recent trends. In E. Feinberg and A. Shwartz, editors, Markov Decision Processes: Models, Methods, Directions, and Open Problems, pages 43–82. Kluwer, Holland, 2001.
480. J. S. Vandergraft. A fluid flow model of networks of queues. Manage. Sci., 29:1198–1208, 1983.
481. M. H. Veatch. Using fluid solutions in dynamic scheduling. In S. B. Gershwin, Y. Dallery, C. T. Papadopoulos, and J. M. Smith, editors, Analysis and Modeling of Manufacturing Systems. Operations Research and Management Science, pages 399–426. Kluwer– Academic International, New York, 2002.
482. M. H. Veatch. Approximate dynamic programming for networks: fluid models and constraint reduction. Submitted for publication, 2004.
483. A. F. Veinott Jr. Discrete dynamic programming with sensitive discount optimality criteria. Ann. Math. Stat., 40(5):1635–1660, 1969.
484. P. Viswanath, D. Tse, and R. Laroia. Opportunistic beam–forming using dumb antennas. IEEE Trans. Inf. Theory, 48(6):1277–1294, 2002.
485. J. Walrand and P. Varaiya. High–Performance Communication Networks. The Morgan Kaufmann Series in Networking. Morgan Kaufman, San Francisco, CA, 2000.
486. R. Weber. On the optimal assignment of customers to parallel servers. J. Appl. Probab., 15:406–413, 1978.
487. R. Weber and S. Stidham. Optimal control of service rates in networks of queues. Adv.Appl. Probab., 19:202–218, 1987.
488. L. M. Wein. Dynamic scheduling of a multiclass make–to–stock queue. Oper. Res., 40(4):724–735, 1992.
489. L. M. Wein and P. B. Chevalier. A broader view of the job–shop scheduling problem. Manage. Sci., 38(7):1018–1033, 1992.
490. L. M. Wein and M. H. Veatch. Scheduling a make–to–stock queue: index policies and hedging points. Oper. Res., 44:634–647, 1996.
491. G. Weiss. Approximation results in parallel machines stochastic scheduling. Ann. Oper. Res., 26(1–4):195–242, 1990.
492. G. Weiss. On the optimal draining of re–entrant fluid lines. Technical Report, Georgia Institute of Technology and Technion, Atlanta, Georgia, 1994.
493. J. Wessels. Markov programming by successive approximations with respect to weighted supremum norms. J. Math. Anal. Appl., 58(2):326–335, 1977.
494. D. J. White. Dynamic programming, Markov chains, and the method of successive approximations. J. Math. Anal. Appl., 6:373–376, 1963.
495. W. Whitt. Some useful functions for functional limit theorems. Math. Oper. Res., 5(1):67–85, 1980.
496. W. Whitt. Simulation run length planning. In WSC’89: Proceedings of the 21st Conference on Winter Simulation, pages 106–112, New York, 1989. ACM Press.
497. W. Whitt. An overview of Brownian and non–Brownian FCLTs for the single–server queue. Queueing Syst. Theory Appl., 36(1–3):39–70, 2000.
498. W. Whitt. Stochastic–Process Limits. Springer Series in Operations Research. SpringerVerlag, New York, 2002.
499. P. Whittle. Multi–armed bandits and the Gittins index. J. R. Stat. Soc. Ser. B, 42:143–149, 1980.
500. P. Whittle. Optimization over Time: Dynamic Programming and Stochastic Control. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics, volume 1. John Wiley, Chichester, 1982.
501. P. Whittle. Optimization over Time: Dynamic Programming and Stochastic Control. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics, volume 2. John Wiley, Chichester, 1983. Dynamic programming and stochastic control.
502. P. Whittle. Networks: Optimisation and Evolution. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge, 2007.
503. R. J. Williams. Diffusion approximations for open multiclass queueing networks: sufficient conditions involving state space collapse. Queueing Syst. Theory Appl., 30(1–2):27– 88, 1998.
504. W. Willinger, V. Paxson, R. H. Riedi, and M. S. Taqqu. Long–range dependence and data network traffic. In Theory and Applications of Long–Range Dependence, pages 373–407. Birkhäuser Boston, Boston, MA, 2003.
505. W. Winston. Optimality of the shortest line discipline. J. Appl. Probab., 14(1):181–189, 1977.
506. P. Yang. Pathwise Solutions for a Class of Linear Stochastic Systems. PhD thesis, Department of Operations Research, Stanford University, Stanford, CA, 1988.
507. G. G. Yin and Q. Zhang. Discrete–Time Markov Chains: Two–Time–Scale Methods and Applications: Stochastic Modelling and Applied Probability. Applications of Mathematics, volume 55. Springer–Verlag, New York, 2005.
508. Y. Zou, I. G. Kevrekidis, and D. Armbruster. Multiscale analysis of re–entrant production lines: an equation–free approach. Phys. A Stat. Mech. Appl., 363:1–13, 2006.
509. R. Zwanzig. Nonequilibrium Statistical Mechanics. Oxford University Press, Oxford, England, 2001.