Constrained Markov Decision Processes

Constrained Markov Decision Processes

Volumn , Issue , 2021, Pages 1-257

  Altman, Eitan ^a  

^a INRIA   (France)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85126423720     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1201/9781315140223     Document Type: Book

References (173)

1. E. Altman (1993), 'Asymptotic properties of constrained Markov decision processes', ZOR-Methods and Models in Operations Research, 37, Issue 2, pp. 151-170.
2. E. Altman (1994), 'Denumerable constrained Markov decision processes and finite approximations', Math, of Operations Research, 19, pp. 169-191.
3. E. Altman (1996), 'Constrained Markov decision processes with total cost criteria: occupation measures and primal LP', ZOR-Mathematical Methods in Operations Research, 43, Issue 1, pp. 45-72.
4. E. Altman (1998), 'Constrained Markov decision processes with total cost criteria: Lagrange approach and dual LP, ZOR-Mathematical Methods in Operations Research, 48, pp. 387-417, 1998.
5. E. Altman and V. A. Gaitsgory (1993), 'Stability and singular perturbations in constrained Markov decision problems', IEEE Transactions on Automatic Control, 38, pp. 971-975.
6. E. Altman and V. A. Gaitsgory (1995), 'A hybrid (differential-stochastic) zero-sum game with fast stochastic part', Annals of the International Society of Dynamic Games, 3, pp. 47-59.
7. E. Altman, A. Hordijk and L. C. M. Kallenberg (1996), 'On the value in constrained control of Markov chains', ZOR-Methods and Models in Operations Research, 44, Issue 3, pp. 387-400.
8. E. Altman, A. Hordijk and F. M. Spieksma (1997), 'Contraction conditions for average and α-discount optimality in countable state Markov games with unbounded rewards', MOR, 22 No. 3, pp. 588-618.
9. E. Altman and A. Shwartz (1988), 'Markov optimization problems: state-action frequencies revisited', 27th IEEE Conference on Decision and Control, Austin, Texas, December 1988 (invited paper).
10. E. Altman and A. Shwartz (1989), 'Optimal priority assignment: a time sharing approach', IEEE Transactions on Automatic Control AC-34, pp. 1089-1102.
11. E. Altman and A. Shwartz (1991a), tMarkov decision problems and state-action frequencies', SIAM J, Control and Optimization, 29, pp. 786-809.
12. E. Altman and A. Shwartz (1991b), 'Adaptive control of constrained Markov chains', IEEE Transactions on Automatic Control, 36, pp. 454-462.
13. E. Altman and A. Shwartz (1991c), 'Sensitivity of constrained Markov Decision Problems', Annals of Operations Research, 32, pp. 1-22.
14. E. Altman and A. Shwartz (1991d), 'Adaptive control of constrained Markov chains: criteria and policies', Annals of Operations Research 28, special issue on 'Markov Decision Processes', Eds. O. HemAndez-Lerma and J. B. Lasserre, pp. 101-134.
15. E. Altman and A. Shwartz (1993), 'Time-sharing policies for controlled Markov chains', Operations Research, 41, pp. 1116-1124.
16. E. Altman and A. Shwartz (1995), 'Constrained Markov games: Nash equilibria', submitted to Annals of Dynamic Games.
17. E. Altman and F. Spieksma (1995), 'The Linear Program approach in Markov decision problems revisited', ZOR-Methods and Models in Operations Research, 42, Issue 2, pp. 169-188.
18. E. Altman and O. Zeitouni (1994), 'Rate of convergence of empirical measures and costs in controlled Markov chains and transient optimality', Math, of Operations Research, 19, pp. 955-974.
19. J. Anderson and P. Nash (1987), Linear Programming in Infinite-Dimensional Spaces, Wiley, England.
20. A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus (1993), 'Discrete-time controlled Markov processes with average cost criterion: a survey', SIAM J. Control and Optimization, 31, pp. 282-344.
21. J. P. Aubin (1993), Optima and Equilibria, An Introduction to Nonlinear Analysis, Springer-Verlag, Berlin, Heidelberg, New York, London, Paris, Tokyo, Hong Kong, Barcelona, Budapest.
22. R. J. Aumann (1964), 'Mixed and behavior strategies in infinite extensive games', Advances in Game Theory, Ann. Math. Study, 52, pp. 627-650.
23. J. Bather (1973), 'Optimal decision procedures for finite Markov chains. Part II: Communicating systems', Advances in Applied Probability, 5, pp. 521-540.
24. M. Bayal-Gursoy and K. W. Ross (1992), 'Variability sensitive Markov decision processes', Math, of Operations Research, 17, pp. 558-571.
25. P. Bernhard (1992), 'Information and strategies in dynamic games', SIAM J. Cont. and Opt., 30, pp. 212-228.
26. F. J. Beutler and K. W. Ross (1985), 'Optimal policies for controlled Markov chains with a constraint', J. Mathematical Analysis and Applications, 112, 236-252.
27. F. J. Beutler and K. W. Ross (1986), 'Time-average optimal constrained Semi-Markov decision processes', Advances of Applied Probability, 18, pp. 341-359.
28. P. Billingsley (1968), Convergence of Probability Measures, J. Wiley, New York.
29. J. R. Birge and R. J. Wets (1986), 'Designing approximating schemes for stochastic optimization problems', Math. Programm. Study, 27, pp. 54-102.
30. V. S. Borkar (1983), 'On minimum cost per unit time control of Markov chains', SIAM J. Control Optim., 22, pp. 965-978.
31. V. S. Borkar (1988), 'A convex analytic approach to Markov decision processes', Prob. Th. Rel. Fields, 78, pp. 583-602.
32. V. S. Borkar (1990), Topics in Controlled Markov Chains, Longman Scientific & Technical.
33. V. S. Borkar (1993), 'Controlled diffusions with constraints, IΓ, Journal of Math. Analysis and Appli., 176, No. 2, pp. 310-321.
34. V. S. Borkar (1994), 'Ergodic control of Markov Chains with constraints-the general case', SIAM J. Control and Optimization, 32, pp. 176-186.
35. V. S. Borkar and M. M. Ghosh (1990), 'Controlled diffusions with constraints', Mathematical Analysis and Applications, 152, No. 1, pp. 88-108.
36. A. D. Bovopoulos and A. A. Lazar (1991), 'The effect of delayed feedback information on network performance', Annals of Operations Res. 36, pp. 581-588.
37. E. B. N. Bui (1989), Contrôle de Vallocation dynamique de trame dans un multiplexeur integrant voix et données, TELECOM, Département Réseaux, Paris 89 E 005, June.
38. R. Cavazos-Cadena (1986), 'Finite-state approximations for denumerable state discounted Markov decision processes', J. Applied Mathematics and Optimization, 14, pp. 27-47.
39. R. Cavazos-Cadena (1989), tWeak conditions for the existence of optimal stationary policies in average cost Markov decision chains with unbounded cost', Kybemetika, 25, 145-156.
40. R. Cavazos-Cadena (1992), 'Existence of optimal stationary policies in average Markov decision processes with a recurrent state', Appl. Math. Optim., 26, pp. 171-194.
41. R. Cavazos-Cadena and O. HernAndez-Lerma (1992), 'Equivalence of Lyapunov stability criteria in a class of Markov decision processes', Appl. Math. Optim., 26, pp. 113-137.
42. R. Cavazos-Cadena and L. I. Sennott (1992), 'Comparing recent assumptions for the existence of average optimal stationary policies', Operations Research Letters, 11, pp. 33-37.
43. K. L. Chung (1967), Markov chains with stationary transition probabilities, 2nd edition, Springer-Verlag, New York.
44. D. J. Daley and D. Vere-Jones (1988), An Introduction to the Theory of Point Processes, Springer-Verlag, New York.
45. G. B. Dantzig, J. Folkman and N. Shapiro (1967), 'On the continuity of the minimum set of a continuous function', J. Math. Anal, and Applications, 17, pp. 519-548.
46. G. T. De Ghellinck (1960), 'Les problèmes de décisions séquentielles', Cahiers du Centre de Recherche Opérationelle, 2, pp. 161-179.
47. R. Dekker and A. Hordijk (1988), 'Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards', Mathematics of Operations Research, 13, pp. 395-421.
48. R. Dekker, A. Hordijk and F. M. Spieksma (1994), 'On the relation between recurrence and ergodicity properties in denumerable Markov decision chains', Math. Operat. Res., 19, pp. 539-559.
49. E. V. Denardo (1970), 'On linear programming in a Markov decision problem', Management Science, 16, pp. 281-288.
50. E. V. Denardo and B. L. Fox (1968), 'Multichain Markov renewal programs', SIAM J. of Applied Math., 16, pp. 468-487.
51. F. D'Epenoux (1960), 'Sur un problème de production et de stockage dans l'aléatoire', Revue Française de Recherche Opérationelle, 14, pp. 3-16.
52. F. D'Epenoux (1963), 'A probabilistic production and inventory problem', Management Science, 10, 98-108.
53. C. Derman (1970), Finite State Markovian Decision Processes, Academic Press, New York and London.
54. C. Derman and M. Klein (1965), 'Some remarks on finite horizon Markovian decision models', Operations Research, 13, pp. 272-278.
55. C. Derman and R. E. Strauch (1966), 'On memoryless rules for controlling sequential control processes', Ann. Math. Stat., 37, pp. 276-278.
56. C. Derman and A. F. Veinott, Jr. (1972), 'Constrained Markov decision chains', Management Science, 19, pp. 389-390.
57. J. L. Doob (1994), Measure Theory, Springer-Verlag, New York, Berlin, Heidelberg, London, Paris, Tokyo, Hong Kong, Barcelona, Budapest.
58. N. Dunford and J. T. Schwartz (1988), Linear operators, part I, John Wiley & Sons, New York, Chichester, Brisbane, Toronto, Singapore.
59. E. Dynkin and A. Yushkevich (1979), Controlled Markov Processes, Springer-Verlag, Berlin.
60. A. Federgruen (1979), 'Geometric convergence of value-iteration in multichain Markov decision problems', Adv. Appl. Prob., 11, pp. 188-217.
61. E. A. Feinberg (1982), 'Non-randomized Markov and semi-Markov strategies in dynamic programming', Theor. Probab. and its Applications, 27, pp. 116-126.
62. E. A. Feinberg (1986), 'Sufficient classes of strategies in discrete dynamic programming I: decomposition of randomized strategies and embedded models', SIAM Theory Probab. Appl., 31, pp. 658-668.
63. E. A. Feinberg (1986), 'Sufficient classes of strategies in discrete dynamic programming', Theory Probability Appl., 31, pp. 658-668.
64. E. A. Feinberg (1991), 'Non-randomized strategies in stochastic decision processes', Annals of Operations Research, 29, pp. 315-332.
65. E. A. Feinberg (1995), 'Constrained semi-Markov decision processes with average rewards', ZOR-Methods and Models in Operations Research, 39, pp. 257-288.
66. E. A. Feinberg and D. J. Kim (1996), 'Bicriterion optimization of an M/G/l queue with a removable server', Probab. in the Eng. and Inf. Sciences, 10, pp. 57-73.
67. E. A. Feinberg and M. I. Reiman (1994), 'Optimality of randomized trunk reservation', Probability in the Engineering and Informational Sciences, 8, pp. 463-489.
68. E. A. Feinberg and I. Sonin (1983), 'Stationary and Markov policies in countable state dynamic programming', Lecture Notes in Mathematics, 1021, pp. 111-129.
69. E. A. Feinberg and I. Sonin (1993), 'The existence of an equivalent stationary strategy in the case of discount factor equal one', unpublished draft.
70. E. A. Feinberg and I. Sonin (1995), 'Notes on equivalent stationary policies in Markov decision processes with total rewards', ZOR-Methods and Models in Operations Research, 44, pp. 205-221.
71. E. A. Feinberg E. and A. Shwartz (1995), 'Constrained Markov decision models with weighted discounted rewards', Math, of Operations Research, 20, pp. 302-320.
72. E. A. Feinberg E. and A. Shwartz (1996), 'Constrained discounted dynamic programming', Math, of Operations Research, 21, pp. 922-945.
73. A. V. Fiacco (1974), 'Convergence properties of local solutions of convex optimization problems', J. Optim. Theory Appl., 13, pp. 1-12.
74. J. A. Filar, L. C. M Kallenberg and H. M. Lee (1989), 'Variance-penalized Markov decision processes', Math, of Operations Research, 14, pp. 147-161.
75. J. A. Filar and H. M. Lee (1985), 'Gain/variability tradeoffs in undiscounted Markov decision processes', Proceedings of 24th Conference on Decision and Control IEEE, pp. 1106-1112.
76. L. Fisher (1968), 'On recurrent denumerable decision processes', Ann. Math. Stat., 39, pp. 424-434.
77. L. Fisher and S. M. Ross (1968), 'An example in denumerable decision processes', Ann. Math. Stat., 39, pp. 674-675.
78. V. A. Gaitsgory and A. A. Pervozvanskii (1986), 'Perturbation theory for mathematical programming problems', JOTA, pp. 389-410.
79. K. Golabi, R. B. Kulkarni and G. B. Way (1982), 'A statewide Pavement Management System', Interfaces, 12, pp. 5-21.
80. M. Haviv (1995), 'On constrained Markov decision processes', OR Letters, 19, Issue 1, pp. 25-28.
81. W. R. Heilmann (1977), 'Generalized linear programming in Markovian decision problems', Bonner Math. Schriften, 98, pp. 33-39.
82. W. R. Heilmann (1978), 'Solving stochastic dynamic programming prob-lems by linear programming-an annotated bibliography', Z. Oper. Res., 22, pp. 43-53.
83. O. Hernández-Lerma (1986), 'Finite state approximations for denumerable multidimensional-state discounted Markov decision processes', J. Mathematical Analysis and Applications, 113, pp. 382-389.
84. O. Hernández-Lerma (1989), Adaptive Control of Markov Processes, Springer-Verlag, New York, Berlin, Heidelberg, London, Paris, Tokyo.
85. O. Hernández-Lerma and D. Hernández-Hernández (1994), 'Discounted cost Markov decision processes on Borel spaces: the linear programming formulation', J. of Math. Anal, and Appl., 183, pp. 335-351.
86. O. Hernández-Lerma and J. B. Lasserre (1994), 'Linear programming and average optimality on Borel spaces-unbounded costs', SIAM J. Control and Optimization, 32, pp. 480-500.
87. O. Hernández-Lerma and J. B. Lasserre (1995), Discrete-Time Markov Control Processes, Basic Optimality Criteria, Springer-Verlag, New York, Berlin, Heidelberg.
88. K. Hinderer (1970), Foundation of Non-Stationary Dynamic Programming with Discrete Time Parameter, Vol. 33, Lecture Notes in Operations Research and Mathematical Systems, Springer-Verlag, Berlin.
89. A. Hordijk (1977), Dynamic Programming and Markov Potential Theory, second edition, Mathematical Centre Tracts 51, Mathematisch Centrum, Amsterdam.
90. A. Hordijk and L. C. M. Kallenberg (1979), 'Linear programming and Markov decision chains', Management Science, 25, pp. 352-362.
91. A. Hordijk and L. C. M. Kallenberg (1984), 'Constrained undiscounted stochastic dynamic programming', Mathematics of Operations Research, 9, pp. 276-289.
92. A. Hordijk and J. B. Lasserre (1994), 'Linear programming formulation of MDPs in countable state space: the multichain case', ZOR-Methods and Models in Operations Research, 40, pp. 91-108.
93. A. Hordijk and F. Spieksma (1989), 'Constrained admission control to a queuing system', Advances of Applied Probability, 21, pp. 409-431.
94. R. Horn and C. R. Johnson (1985), Matrix Analysis, Cambridge Univ. Press.
95. M. T. Hsiao and A. A. Lazar (1991), 'Optimal decentralized flow control of Markovian queueing networks with multiple controllers', Performance Evaluation, 13, pp. 181-204.
96. Y. Huang and L. C. M. Kallenberg (1994), 'On finding optimal policies for Markov decision chains: A unifying framework for mean-variance tradeoffs', Math, of Operations Research, 19, pp. 434-448.
97. D. Kadelka (1983), 'On randomized policies and mixtures of deterministic policies in dynamic programming', Methods of Operations Research, 46, pp. 67-75.
98. L. C. M. Kallenberg (1983), Linear Programming and Finite Markovian Control Problems, Mathematical Centre Tracts 148, Amsterdam.
99. L. C. M. Kallenberg (1994), 'Survey of linear programming for standard and nonstandard Markovian control problems, Part I: Theory', ZOR-Methods and Models in Operations Research, 40, pp. 1-42.
100. P. Kanniappan and S. M. A. Sastry (1974), 'Uniform convergence of convex optimization problems', J. Math. Anal. Appli., 96, pp. 1-12.
101. H. Kawai (1987), 'A variance minimization problem for a Markov decision process', European Journal of Operations Research, 31, pp. 140-145.
102. J. G. Kemeney, J. L. Snell and A. W. Knapp (1976), Denumerable Markov Chains, Springer-Verlag.
103. L. Kleinrock (1976), Queuing systems, Volume I. John Wiley, New York.
104. P. Kolesar (1970), 'A Markovian model for hospital admission and scheduling', Management Science, 16, pp. 384-396.
105. G. M. Koole (1988), Stochastische Dynamische Programmering met Bi-jvoorwaarden (translation: Stochastic dynamic programming with additional constraints) Master thesis, Leiden University, The Netherlands.
106. Y. A. Korilis and A. Lazar (1995a), 'On the existence of equilibria in noncooperative optimal flow control', J. of the Association for Computing Machinery, 42, No. 3, pp. 584-613.
107. Y. A. Korilis and A. Lazar (1995b), 'Why is flow control hard: optimality, fairness, partial and delayed information', preprint.
108. D. Krass (1989), Contributions to the Theory and Applications of Markov Decision Processes, Ph. D. thesis, Department of Mathematical Sciences, Johns Hopkins Univ., Baltimore, MD.
109. M. Krein and D. Milman (1940), 'On extreme points of regularly convex sets', Studia Math., 9, pp. 133-138.
110. N. Krylov (1985), 'Once more about the connection between elliptic operators and Ito's stochastic equations', Statistics and Control of Stochastic Processes, Steklov Seminar 1984 (Krylov N. et al., Eds.), Optimization Software, New York, 69-101.
111. H. W. Kuhn (1953), 'Extensive games and the problem of information', Ann. Math. Stud., 28, pp. 193-216.
112. H. Kushner and J. Kleinman (1971), 'Mathematical programming and the control of Markov chains', Internat. J. Control, 13, pp. 801-820.
113. J. B. Lasserre (1994), 'Average optimal stationary policies and linear programming in ∞untable state Markov decision processes', J. Math. Anal. Appl., 183, pp. 233-249.
114. A. Lazar (1983), 'Optimal flow control of a class of queuing networks in equilibrium', IEEE Transactions on Automatic Control, 28, pp. 1001-1007.
115. M. B. Lignota and J. Morgan (1992), 'Convergences of marginal functions with dependent constraints', Optimization, 23, pp. 189-213.
116. R. Lucchetti and R. J. B Wets (1993), 'Convergence of minima of integral functionals, with applications to optimal control and stochastic optimization', Statistics and Decisions, 11, pp. 69-84.
117. D.-J. Ma and A. M. Makowski (1988), 'A class of steering policies under a recurrence condition', 27th IEEE Conference on Decision and Control, Austin, TX, December, pp. 1192-1197.
118. D.-J. Ma and A. M. Makowski (1992), 'A class of two-dimensional stochastic approximations and steering policies for Markov decision processes', 31st IEEE Conference on Decision and Control, Tucson, Arizona, pp. 3344-3349.
119. D.-J. Ma, A. M. Makowski and A. Shwartz (1990), 'Stochastic approximations for finite state Markov chains', Stochastic Processes and Their Applications, 35, pp. 27-45.
120. B. Maglaris and M. Schwartz (1982), 'Optimal fixed frame multiplexing in integrated line-and packet-switched Commrmication networks', IEEE Transactions on Information Theory, IT-28, pp. 263-273.
121. A. M. Makowski and A. Shwartz (1987), 'Recurrence properties of a system of competing queues with applications', Research report EE Pub. No. 627, Technion, Haifa, Israel.
122. A. M. Makowski and A. Shwartz (1992), 'Stochastic approximations and adaptive control of a discrete-time single server network with random routing', SIAM J. Control and Optimization, 30, pp. 1476-1506.
123. A. S. Manne (1960), 'Linear programming and sequential decisions', Management Science, 6, pp. 259-267.
124. S. Meyn and R. Tweedie (1994), Markov Chains and Stochastic Stability, Springer-Verlag, New York.
125. P. Nain and K. W. Ross (1986), 'Optimal priority assignment with hard constraint', Transactions on Automatic Control, 31, pp. 883-888.
126. A. S. Nowak (1985), 'Existence of equilibrium stationary strategies in discounted noncooperative stochastic games with uncountable state space', JOTA, 45, pp. 592-602.
127. A. A. Pervozvanskii and V. A. Gaitsgory (1988), Theory of Suboptimal Decision: Decomposition and Aggregation, Kluwer Academic Publishers, Dordrecht.
128. A. B. Piunovskiy (1993), 'Control of random sequences in problems with constraints', Theory Probab. Appl., 38, No. 4, translated from Russian.
129. A. B. Piunovskiy (1994), 'Control of jump-like processes in constrained problems', Avtomatika i Telemekhanika, 4, pp. 75-89. Translated into English in Automation and Remote Control, 55, No. 4, 1994.
130. A. B. Piunovskiy (1995), 'Multicriteria control problems for stochastic jump processes', Proceedings of 3rd European Control Conference, Rome, Italy, September, pp. 492-495.
131. A. B. Piunovskiy (1996), 'A multicriteria model of optimal control of a stochastic linear system', Automation and Remote Control, 57, No. 6, Part 1, pp. 831-842.
132. A. B. Piunovskiy (1997a), Optimal Control of Random Sequences in Problems with Constraints, Mathematics and its Applications, Kluwer Academic Publishers, Dordrecht, Boston, London.
133. A. B. Piunovskiy (1997b), 'Optimal control of stochastic sequences in sequences with constraints', Stochastic Analysis and Applications, No. 2.
134. M. Puterman (1994), Markov Decision Processes, John Wiley & Sons, New York.
135. T. E. S. Raghavan and J. A. Filar (1991), 'Algorithms for stochastic pames-a survey', ZOR-Methods and Models in Operations Research, 35, pp. 437-472.
136. D. Revuz (1975), Markov Chains, North-Holland, Amsterdam, The Netherlands.
137. R. T. Rockafellar (1989), Conjugate Duality and Optimization, Society for Industrial and Applied Mathematics, 2nd printing, Philadelphia.
138. K. W. Ross (1989), 'Randomized and past-dependent policies for Markov decision processes with multiple constraints', Operations Research, 37, pp. 474-477.
139. K. W. Ross and B. Chen (1988), 'Optimal scheduling of interactive and non-interactive traffic in telecommunication systems', IEEE Transactions on Automatic Control, 33, pp. 261-267.
140. K. Ross and R. Varadarajan (1989), 'Markov decision processes with sample path constraints: the communicating case', Operations Research, 37, pp. 780-790.
141. K. Ross and R. Varadarajan (1991), 'Multichain Markov decision processes with a sample path constraint: a decomposition approach', Math, of Operations Research, 16, pp. 195-207.
142. H. L. Royden (1988), Real Analysis, 3rd edition, Macmillan Publishing Company, New York.
143. M. Schäl (1975), 'Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal', Z. Wahrscheinlichkeitstheorie und verw. Geb., 32, pp. 179-196.
144. M. Schäl (1987), 'Estimation and control in discounted dynamic programming', Stochastics, 20, pp. 51-71.
145. I. E. Schochetman (1990), 'Pointwise versions of the maximum theorem with applications to optimization', Appl. Math. Lett., 3, pp. 89-92.
146. L E. Schochetman and R. L. Smith (1991), 'Convergence of selections with applications in optimization', J. Math. Anal. Appl., 155, pp. 278-242.
147. L. I. Sennott (1989), 'Average cost optimal stationary policies in average cost Markov decision processes', Operations Research, 37, pp. 626-633.
148. L. I. Sennott (1991), 'Constrained discounted Markov decision chains', Probability in the Engineering and Informational Sciences, 5, pp. 463-475.
149. L. I. Sennott (1993), 'Constrained average cost Markov decision chains', Probability in the Engineering and Informational Sciences, 7, pp. 69-83.
150. L. I. Sennott (1997), 'On computing average optimal policies with appli-cation to routing to parallel queues', Mathematical Methods of Operations Research, 45, pp. 45-62.
151. L. S. Shapley (1953), 'Stochastic games', Proceedings Nat. Acad, of Science USA, 39, pp. 1095-1100.
152. N. Shimkin (1994), 'Stochastic games with average cost constraints', Annals of the International Society of Dynamic Games, Vol. 1: Advances in Dynamic Games and Applications, Eds. T. Basar and A. Haurie, Birkhauser, Boston.
153. M. J. Sobel (1985), 'Maximal mean/standard deviation ratio in undiscounted MDP', OR Letters, 4, pp. 157-159.
154. M. J. Sobel (1994), 'Mean-variance tradeoffs in an undiscounted MDP, Operations Research, 42, pp. 175-188.
155. F. M. Spieksma (1990), Geometrically Ergodic Markov Chains and the Optimal Control of Queues, Ph. D. thesis, University of Leiden, The Netherlands.
156. R. Sznadjer and J. A. Filar (1992), 'Some comments on a theorem of Hardy and Littlewood', J. Optim. Theory Appl., 75, pp. 210-218.
157. L. C. Thomas and D. Stengos (1985), 'Finite state approximation algorithms for average cost denumerable state Markov decision processes', OR Spectrum, 7, pp. 27-37.
158. M. Tidball and E. Altman (1996a), 'Approximations in dynamic zerosum games, I', SIAM J. Control and Optimization, 34, No. 1, pp. 311-328.
159. M. Tidball, O. Pourtallier and E. Altman (1996b), 'Continuity of optimal values and solutions of convex optimization, and constrained control of Markov chains', submitted to SIAM J. Control and Optimization.
160. M. Tidball, O. Pourtallier and E. Altman (1997), 'Approximations in dynamic zero-sum games, II', SIAM J. Control and Optimization, 35, pp. 2101-2117.
161. F. Vakil and A. A. Lazar (1987), 'Flow control protocols for integrated networks with partially observed voice traffic', IEEE Transactions on Automatic Control, AC-32, pp. 2-14.
162. J. Van Der Wal (1981a), Stochastic Dynamic Programming, Mathematical Centre Tract 139, Mathematisch Centrum, Amsterdam.
163. J. Van Der Wal (1981b), 'On stationary strategies', Eindhoven Univ, of Technology, Dept. of Math., Memorandum-COSOR 81-14, 1981.
164. J. Wessels (1977), 'Markov Games with unbounded rewards', Dynamische Optimierung, M. Schäl (Editor) Bonner Mathematische Schriften, Nr. 98, Bonn.
165. D. J. White (1980), 'Finite state approximations for denumerable state infinite horizon discounted Markov decision Processes', J. Mathematical Analysis and Applications, 74, pp. 292-295.
166. D. J. White (1982), 'Finite state approximations for denumerable state infinite horizon discounted Markov decision processes with unbounded rewards', J. Mathematical Analysis and Applications, 86, pp. 292-306.
167. D. J. White (1987), 'Utility, probabilistic constraints, mean variance of discounted rewards in Markov decision processes', OR Spectrum, 9, pp. 13-22.
168. D. J. White (1994), 'A mathematical programming approach to a problem in variance penalized Markov decision processes', OR Spectrum, 15, pp. 225-230.
169. W. Whitt (1978), 'Approximations of dynamic programs, Γ, Mathematics of Operations Research, 3, No. 3, pp. 231-243.
170. W. Whitt (1980), 'Representation and approximation of noncooperative sequential games', SIAM J. Control and Opt., 18, No. 1, pp. 33-43.
171. C. V. Winden and R. Dekker (1994), 'Markov Decision Models for Building Maintenance: A Feasibility Study', Report 9473/A, ERASMUS University Rotterdam, The Netherlands.
172. D. Williams (1992), Probability and Martingales, Cambridge University Press, Cambridge.
173. A. A. Yushkevich (1973), 'On a class of strategies in general Markov decision models', Theory Probab. Appl., 18, pp. 777-779.