Computing moments of first passage times to a subset of states in Markov chains

SIAM Journal on Matrix Analysis and Applications

Volumn 27, Issue 2, 2006, Pages 396-412

  Dayar, Tuǧrul ^a   Akar, Nail ^a  

^a BILKENT UNIVERSITY   (Turkey)

Author keywords

First passage times; Grassmann Taksar Heyman algorithm; Markov chains; Mean; Moments; Unsafe states; Variance

Indexed keywords

ALGORITHMS; LINEAR EQUATIONS; LINEAR SYSTEMS; METHOD OF MOMENTS; PROBLEM SOLVING;

FIRST PASSAGE TIMES; GRASSMANN-TAKSAR-HEYMAN ALGORITHM; MEAN; MOMENTS; UNSAFE STATES; VARIANCE;

MARKOV PROCESSES;

EID: 33646230759     PISSN: 08954798     EISSN: 10957162     Source Type: Journal    
DOI: 10.1137/S0895479804442462     Document Type: Article

References (23)

1. K. E. AVRACHENKOV AND M. HAVIV, The first Laurent series coefficients for singularly perturbed stochastic matrices, Linear Algebra Appl., 386 (2004), pp. 243-259.
2. A. BERMAN AND R. J. PLEMMONS, Nonnegative Matrices in the Mathematical Sciences, SIAM, Philadelphia, 1994.
3. P.-J. COURTOIS, Decomposability: Queueing and Computer System Applications, Academic Press, New York, 1977.
4. T. DAYAR AND W. J. STEWART, On the effects of using the Grassmann-Taksar-Heyman method in iterative aggregation-disaggregation, SIAM J. Sci. Comput., 17 (1996), pp. 287-303.
5. R. L. GRAHAM, D. E. KNUTH, AND O. PATASHNIK, Concrete Mathematics: A Foundation for Computer Science, Addison-Wesley, Reading, MA, 1989.
6. W. K. GRASSMANN, M. I. TAKSAR, AND D. P. HEYMAN, Regenerative analysis and steady state distributions for Markov chains, Oper. Res., 33 (1985), pp. 1107-1116.
7. P. G. HARRISON AND W. J. KNOTTENBELT, Passage time distribution in large Markov chains, Performance Evaluation Review, 30 (2002), pp. 77-85.
8. W. J. HARROD AND R. J. PLEMMONS, Comparison of some direct methods for computing the stationary distributions of Markov chains, SIAM J. Sci. Statist. Comput., 5 (1984), pp. 453-469.
9. R. HASSIN AND M. HAVIV, Mean passage times and nearly uncoupled Markov chains, SIAM J. Discrete Math., 5 (1992), pp. 386-397.
10. D. P. HEYMAN, Accurate computation of the fundamental matrix of a Markov chain, SIAM J. Matrix Anal. Appl., 16 (1995), pp. 954-963.
11. D. P. HEYMAN AND D. P. O'LEARY, What is fundamental for Markov chains: First passage times, fundamental matrices, and group generalized inverses, in Computations with Markov Chains, W. J. Stewart, ed., Kluwer, Boston, MA, 1995, pp. 151-161.
12. D. P. HEYMAN AND D. P. O'LEARY, Overcoming instability in computing the fundamental matrix for a Markov chain, SIAM J. Matrix Anal. Appl., 19 (1998), pp. 534-540.
13. C. JORDAN, Calculus of Finite Differences, 3rd ed., Chelsea, New York, 1979.
14. J. G. KEMENY AND J. L. SNELL, Finite Markov Chains, Springer-Verlag, New York, 1983.
15. G. LATOUCHE AND V. RAMASWAMI, Introduction to Matrix Analytic Methods in Stochastic Modeling, SIAM, Philadelphia, 1999.
16. I. MAREK, Iterative aggregation/disaggregation methods for computing some characteristics of Markov chains. II. Fast convergence, Appl. Numer. Math., 45 (2003), pp. 11-28.
17. C. D. MEYER, JR., The role of the group generalized inverse in the theory of finite Markov chains, SIAM Rev., 17 (1975), pp. 443-464.
18. C. D. MEYER, Stochastic complementation, uncoupling Markov chains, and the theory of nearly reducible systems, SIAM Rev., 31 (1989), pp. 240-272.
19. C. A. O'CINNEIDE, Entrywise perturbation theory and error analysis for Markov chains, Numer. Math., 65 (1993), pp. 109-120.
20. C. C. PAIGE, P. H. STYAN, AND P. G. WACHTER, Computation of the stationary distribution of a Markov chain, J. Stat. Comput. Simul., 4 (1975), pp. 1156-1179.
21. Software for computing moments of first passage times to a subset of states in Markov chains, http://www.cs.bilkent.edu.tr/~tugrul/software.html.
22. E. DE SOUZA E SILVA AND H. R. GAIL, Calculating cumulative operational time distributions of repairable computer systems, IEEE Trans. Comput., C-35 (1986), pp. 322-332.
23. W. J. STEWART, Introduction to the Numerical Solution of Markov Chains, Princeton University Press, Princeton, NJ, 1994.