ε-equilibria for stochastic games with uncountable state space and unbounded costs

SIAM Journal on Control and Optimization

Volumn 40, Issue 6, 2002, Pages 1821-1839

  Nowak, Andrzej S ^a   Altman, Eitan ^b  

^a UNIVERSITY OF ZIELONA GÓRA   (Poland)

^b INRIA   (France)

Author keywords

Approximate equilibria; General state space; Long run average payoff criterion; Nonzero sum stochastic games

Indexed keywords

APPROXIMATION THEORY; BOUNDARY CONDITIONS; CONVERGENCE OF NUMERICAL METHODS; GEOMETRY; MARKOV PROCESSES; PROBABILITY; STATE SPACE METHODS; SYSTEM STABILITY;

APPROXIMATE EQUILIBRIA; STOCHASTIC GAMES;

GAME THEORY;

EID: 0036930365     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012900378371     Document Type: Article

References (48)

1. E. Altman, Constrained Markov Decision Processes, Chapman and Hall, London, 1998.
2. E. Altman, Non zero-sum stochastic games in admission, service and routing control in queueing systems, QUESTA, 23 (1996), pp. 259-279.
3. E. Altman and A. Hordijk, Zero-sum Markov games and worst-case optimal control of queueing systems, QUESTA, 21, (1995), pp. 415-447.
4. E. Altman, A. Hordijk, and F. M. Spieksma, Contraction conditions for average and α-discount optimality in countable state Markov games with unbounded rewards, Math. Oper. Res., 22 (1997), pp. 588-618.
5. R. Amir, Continuous stochastic games of capital accumulation with convex transitions. Games Econom. Behav., 15 (1996), pp. 111-131.
6. V. S. Borkar and M. K. Ghosh, Denumerable state stochastic games with limiting average payoff, J. Optim. Theory Appl., 76 (1993), pp. 539-560.
7. L. O. Curtat, Markov equilibria of stochastic games with complementarities, Games Econom. Behav., 17, (1996), pp. 177-199.
8. R. Dekker and A. Hordijk, Recurrence conditions for average and Blackwell optimality in denumerable state Markov decision chains, Math. Oper. Res., 17 (1992), pp. 271-290.
9. R. Dekker, A. Hordijk, and F. M. Spieksma, On the relation between recurrence and ergodicity properties in denumerable Markov decision chains, Math. Oper. Res., 19 (1994), pp. 539-559.
10. D. Duffie, J. Geanakoplos, A. Mas-Colell, and A. McLennan, Stationary Markov equilibria, Econometrica, 62 (1994), pp. 745-782.
11. A. Federgruen, On N-person stochastic games with denumerable state space, Adv. in Appl. Probab., 10 (1978), pp. 452-471.
12. J. A. Filar and K. Vrieze, Competitive Markov Decision Processes, Springer-Verlag, New York, 1997.
13. C. Harris, P. Reny, and A. Robson, The existence of subgame-perfect equilibrium in continuous games with almost perfect information: A case for public randomization, Econometrica, 63 (1995), pp. 507-544.
14. O. Hernández-Lerma and J. B. Lasserre, Further Topics in Discrete-Time Markov Control Processes, Springer-Verlag, New York, 1999.
15. O. Hernández-Lerma and J. B. Lasserre, Zero-sum stochastic games in Borel spaces: Average payoff criteria, SIAM J. Control Optim., 39 (2001), pp. 1520-1539.
16. A. Jaśkiewicz, On strong 1-optimal policies in Markov control processes with Borel state spaces, Bull. Polish Acad. Sci. 48 (2000), pp. 439-450.
17. A. Jaśkiewicz, Zero-sum semi-Markov games, SIAM J. Control Optim., to appear.
18. A. Jaśkiewicz and A. S. Nowak, On the optimality equation for zero-sum ergodic stochastic games, Math. Methods Oper. Res., 54 (2001), pp. 291-301.
19. H.-U. Küenle, Equilibrium strategies in stochastic games with additive cost and transition structure, Int. Game Theory Rev., 1 (1999), pp. 131-147.
20. H.-U. Küenle, Stochastic games with complete information and average cost criterion, Ann. Internat. Soc. Dynam. Games, 5 (2000) pp. 325-338.
21. A. Maitra and W. D. Sudderth, Borel stochastic games with limsup payoff, Ann. Probab., 21 (1993), pp. 861-885.
22. A. Maitra and W. D. Sudderth, Discrete Gambling and Stochastic Games, Springer-Verlag, New York, 1996.
23. J.-F. Mertens and A. Neyman, Stochastic games, Internat. J. Game Theory, 10 (1981), pp. 53-66.
24. J.-F. Mertens and T. Parthasarathy, Equilibria for discounted stochastic games, Research paper 8750, CORE, Université Catholique de Louvain, Louvain-la-Neuver, Belgium, 1987.
25. S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability, Springer-Verlag, New York, 1993.
26. S. P. Meyn and R. L. Tweedie, Computable bounds for geometric convergence rates of Markov chains, Ann. Appl. Probab., 4 (1994), pp. 981-1011.
27. S. P. Meyn and R. L. Tweedie, State-dependent criteria for convergence of Markov chains, Ann. Appl. Probab., 4 (1994), pp. 149-168.
28. J. Neveu, Mathematical Foundations of the Calculus of Probability, Holden-Day, San Francisco, 1965.
29. A. S. Nowak, Existence of equilibrium stationary strategies in discounted noncooperative stochastic games with uncountable state space, J. Optim. Theory Appl., 45 (1985), pp. 591-602.
30. A. S. Nowak, Stationary equilibria for nonzero-sum average payoff ergodic stochastic games with general state space, Ann. Internat. Soc. Dynam. Games, 1 (1994), pp. 231-246.
31. A. S. Nowak, Zero-sum average payoff stochastic games with general state space, Games Econom. Behav., 7 (1994), pp. 221-232.
32. A. S. Nowak, A generalization of Ueno's inequality for n-step transition probabilities, Applicationes Mathematicae, 25 (1998), pp. 295-299.
33. A. S. Nowak, Sensitive equilibria for ergodic stochastic games with countable state spaces, Math. Methods Oper. Res., 50 (1999), pp. 65-76.
34. A. S. Nowak, Optimal strategies in a class of zero-sum ergodic stochastic games, Math. Methods Oper. Res., 50 (1999), pp. 399-420.
35. A. S. Nowak and A. Jaśkiewicz, Remarks on Sensitive Equilibria in Stochastic Games with Additive Reward and Transition Structure, Technical report, Institute of Mathematics, University of Zielona Gó,ra, Poland, 2002.
36. A. S. Nowak and T. E. S. Raghavan, Existence of stationary correlated equilibria with symmetric information for discounted stochastic games, Math. Oper. Res., 17 (1992). pp. 519-526.
37. T. Parthasarathy and S. Sinha, Existence of stationary equilibrium strategies in non-zero-sum discounted stochastic games with uncountable state space and state independent transitions, Internat. J. Game Theory, 18 (1989), pp. 189-194.
38. O. Passchier, The Theory of Markov Games and Queueing Control, Ph.D. thesis, Department of Mathematics and Computer Science, Leiden University, Leiden. The Netherlands, 1998.
39. U. Rieder, Equilibrium plans for nonzero-sum Markov games, in Game Theory and Related Topics, O. Moeschlin andD. Pallaschke, eds., North-Holland, Amsterdam, 1979, pp. 91-102.
40. L. I. Sennott, Nonzero-sum stochastic games with unbounded costs: discounted and average cost cases, Z. Oper. Res., 40 (1994), pp. 145-162.
41. F. M. Spieksma, Geometrically Ergodic Markov Chains and the Optimal Control of Queues, Ph.D. thesis, Department of Mathematics and Computer Science, Leiden University, Leiden, The Netherlands, 1990.
42. L. Stettner, On nearly self-optimizing strategies for a discrete-time uniformly ergodic adaptive model, Appl. Math. Optim., 27, (1993), pp. 161-177.
43. J. Van der Wal, Stochastic Dynamic Programming, Mathematical Center Tracts 139, Mathematisch Centrum, Amsterdam, 1981.
44. N. Vieille, Equilibrium in 2-player stochastic games 1: A reduction, Israel J. Math., 119 (2000), pp. 55-91.
45. N. Vieille, Equilibrium in 2-player stochastic games 2: The case of recursive games, Israel J. Math., 119, (2000), pp. 93-126.
46. T. Ueno, Some limit theorems for temporally discrete Markov processes, J. Fac. Sci. Univ. Tokyo, 7 (1957), pp. 449-462.
47. J. Wagra, Optimal Control of Differential and Functional Equations, Academic Press, New York, 1977.
48. W. Whitt, Representation and approximation of noncooperative sequential games, SIAM J. Control Optim., 18 (1980), pp. 33-48.