Characterization and sufficient conditions for normed ergodicity of Markov chains

Advances in Applied Probability

Volumn 36, Issue 1, 2004, Pages 227-242

  Borovkov, A A ^a   Hordijk, A ^b  

^a SOBOLEV INSTITUTE OF MATHEMATICS   (Russian Federation)

^b LEIDEN UNIVERSITY   (Netherlands)

Author keywords

Lyapunov function; Markov chain; Normed ergodicity

Indexed keywords

LYAPUNOV FUNCTIONS; NORMED ERGODICITY;

CONVERGENCE OF NUMERICAL METHODS; LYAPUNOV METHODS; MATHEMATICAL OPERATORS; PROBABILITY; QUEUEING THEORY; RANDOM PROCESSES; THEOREM PROVING;

MARKOV PROCESSES;

EID: 0345164745     PISSN: 00018678     EISSN: None     Source Type: Journal    
DOI: 10.1239/aap/1077134471     Document Type: Article

References (19)

1. BOROVKOV, A. A. (1998). Ergodicity and Stability of Stochastic Processes. John Wiley, Chichester.
2. BOROVKOV, A. A. AND HORDIJK, A. (2000). On normed ergodicity of Markov chains. Tech. Rep. MI 2000-40, Leiden University.
3. BOROVKOV, A. A. AND HORDIJK, A. (2004). On linear Lyapunov functions and normed ergodicity for stochastic networks. In preparation.
4. DEKKER, R. AND HORDIJK, A. (1988). Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. Math. Operat. Res. 13, 395-421.
5. DEKKER, R., HORDIJK, A. AND SPIEKSMA, F. M. (1994). On the relation between recurrence and ergodicity properties in denumerable Markov decision chains. Math. Operat. Res. 19, 539-559.
6. FAYOLLE, G., MALYSHEV, V. A. AND MENSHIKOV, M. V. (1995). Topics in the Constructive Theory of Countable Markov Chains. Cambridge University Press.
7. FEDERGRUEN, A., HORDIJK, A. AND TIJMS, H. C. (1978). Recurrence conditions in denumerable state Markov decision processes. In Dynamic Programming and Its Applications, ed. M. L. Puterman, Academic Press, New York, pp. 3-22.
8. FEDERGRUEN, A., HORDIJK, A. AND TIJMS, H. C. (1978). A note on simultaneous recurrence conditions on a set of denumerable stochastic matrices. J. Appl. Prob. 15, 842-847.
9. HEIDERGOTT, B. AND HORDIJK, A. (2003). Taylor series expansions for stationary Markov chains. Adv. Appl. Prob. 35, 1046-1070.
10. HEIDERGOTT, B., HORDIJK, A. AND WEISSHAUPT, H. (2002). Measure-valued differentiation for stationary Markov chains. EURANDOM report 2002-027.
11. HORDIJK, A. (1974). Dynamic Programming and Markov Potential Theory (Math. Centre Tract 51). CWI, Amsterdam.
12. HORDIJK, A. AND DEKKER, R. (1983). Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. Report No. 83-36, Institute of Applied Mathematics and Computing Science, Leiden University.
13. HORDIJK, A. AND SPIEKSMA, F. M. (1992). On ergodicity and recurrence properties of a Markov chain with an application to an open Jackson network. Adv. Appl. Prob. 24, 343-376.
14. HORDIJK, A. AND YUSHKEVICH, A. A. (1999). Blackwell optimality in the class of all policies in Markov decision chains with a Borel state space and unbounded rewards. Math. Meth. Operat. Res. 50, 421-448.
15. HORDIJK, A., SPIEKSMA, F. M. AND TWEEDIE, R. L. (1995). Uniform stability for general state space Markov decision processes. Tech. Rep., Leiden University.
16. KARTOSCHOV, N. (1985). Inequalities in theorems of ergodicity and stability for Markov chains with common phase space. Theory Prob. Appl. 30, 247-259.
17. MEYN, S. P. AND TWEEDIE, R. L. (1993). Markov chains and stochastic stability. Springer, London.
18. NEVEU, J. (1965). Mathematical Foundations of the Calculus of Probability. Holden Day, San Francisco.
19. SPIEKSMA, F. M. AND TWEEDIE, R. L. (1994). Strenghtening ergodicity to geometric ergodicity for Markov chains. Stoch. Models 10, 45-75.