The deviation matrix of a continuous-time Markov chain

Probability in the Engineering and Informational Sciences

Volumn 16, Issue 3, 2002, Pages 351-366

  Coolen Schrijner, Pauline ^a   Van Doorn, Erik A ^b  

^a DURHAM UNIVERSITY   (United Kingdom)

^b UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

CONTINUOUS TIME SYSTEMS;

BIRTH-DEATH PROCESS; CONTINUOUS TIME MARKOV CHAIN; ERGODICS; NEW APPLICATIONS; STATIONARITY; STOCHASTICALLY INCREASING; TRANSITION PROBABILITY MATRIX;

MARKOV CHAINS;

EID: 0035993475     PISSN: 02699648     EISSN: None     Source Type: Journal    
DOI: 10.1017/S0269964802163066     Document Type: Article

References (25)

1. Anderson, W.J. (1991). Continuous-time Markov chains. New York: Springer-Verlag.
2. Coolen-Schrijner, P. & van Doorn, E.A. (2001). On the convergence to stationarity of birth-death processes. Journal of Applied Probability 38: 696-706.
3. Dekker, R. & Hordijk, A. (1988). Average sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. Mathematics of Operations Research 13:395-420.
4. Glynn, P. (1984). Some asymptotic formulas for Markov chains with applications to simulation. Journal of Statistical Computing and Simulation 19: 97-112.
5. Heyman, D.P. & O'Leary, D.P. (1995), What is fundamental for Markov chains: First passage times, fundamental matrices, and group generalized inverses. In W.J. Stewart (ed.), Computations with Markov chains: Proceedings 2nd international workshop on the numerical solution of Markov chains. Boston: Kluwer, pp. 151-161.
6. Heyman, D.P. & O'Leary, D.P. (1998). Overcoming instability in computing the fundamental matrix of a Markov chain. SIAM Journal on Matrix Analysis and Applications 19: 534-540.
7. Hordijk, A. & Spieksma, F.M. (1994). A new formula for the deviation matrix. In F.P. Kelly (ed.), Probability, statistics and optimisation - A tribute to Peter Whittle. Chichester: John Wiley & Sons, pp. 497-507.
8. Hunter, J.J. (1982). Generalized inverses and their application to applied probability problems. Linear Algebra and Its Applications 45: 157-198.
9. Iosifescu, M. (1980), Finite Markov processes and their applications. Chichester: John Wiley & Sons.
10. Keilson, J. (1979). Markov chain models: Rarity and exponentiality. New York: Springer-Verlag.
11. Kemeny, J.G. & Snell, J.L. (1960). Finite Markov chains. New York: Van Nostrand.
12. Kemeny, J.G. & Snell, J.L. (1961). Finite continuous time Markov chains. Theory of Probability and Its Applications 6:101-105.
13. Kemeny, J.G., Snell, J.L., & Knapp, A.W. (1966). Denumerable Markov chains. New York: Van Nostrand.
14. Kijlma, M. (1997), Markov processes for stochastic modeling. London: Chapman & Hall.
15. Knopp, K. (1964). Theorie und Anwendung der Unendlichen Reihen, 5th ed. Berlin: Springer-Verlag.
16. Koole, G.M. (1998). The deviation matrix of the M/M/1/∞ and M/M/1/N queue, with applications to controlled queueing models, Proceedings of the 37th IEEE CDC, Tampa, FL, pp. 56-59.
17. Koole, G.M. & Spieksma, P.M. (2001). On deviation matrices for birth-death processes. Probability in the Engineering and Informational Sciences 15: 239-258.
18. Lamond, B.F. & Puterman, M.L. (1989). Generalized Inverses in discrete time Markov decision processes. SIAM Journal on Matrix Analysis and Applications 10: 118-134.
19. Meyer, C.D. (1975). The role of the group generalized inverse in the theory of finite Markov chains. SIAM Review 17: 443-464.
20. Puterman, M.L. (1994). Markov decision processes. New York: John Wiley & Sons.
21. Stadje, W. & Parthasarathy, P.R. (1999). On the convergence to stationarity of the many-server Poisson queue. Journal of Applied Probability 36: 546-557.
22. Syski, R. (1977). Perturbation models. Stochastic Processes and Their Applications 5: 93-130.
23. Syski, R. (1978). Ergodic potential. Stochastic Processes and Their Applications 7: 311-336.
24. Whitt, W. (1992), Asymptotic formulas for Markov processes with applications to simulation. Operations Research 40: 279-291.
25. Yao, D.D. (1985). First-passage-time moments of Markov processes. Journal of Applied Probability 22: 939-945.