Northwest Corner and Banded Matrix Approximations to a Markov Chain

Naval Research Logistics

Volumn 46, Issue 2, 1999, Pages 187-197

  Zhao, Yiqiang Q ^a   Braun, W John ^a   Li, Wei ^a  

^a UNIVERSITY OF WINNIPEG   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; MARKOV PROCESSES; MATRIX ALGEBRA; PROBABILITY;

NORTHWEST CORNER;

APPROXIMATION THEORY;

EID: 0033101608     PISSN: 0894069X     EISSN: None     Source Type: Journal    
DOI: 10.1002/(SICI)1520-6750(199903)46:2<187::AID-NAV4>3.0.CO;2-V     Document Type: Article

References (29)

1. S. Asmussen, Applied probability and queues, Wiley, Chichester, 1987.
2. H.R. Gail, S.L. Hantler, and B.A. Taylor, Spectral analysis of M/G/1 and GI/M/1 type Markov chains, Adv Appl Prob 28 (1996), 114-165.
3. H.R. Gail, S.L. Hantler, and B.A. Taylor, M/G/1 and G/M/1 type Markov chains with multiple boundary levels, Adv Appl Prob 29 (1997), 733-758.
4. D. Gibson and E. Seneta, Augmented truncations of infinite stochastic matrices, J Appl Prob 24 (1987), 600-608.
5. W.K. Grassmann, M.I. Taksar, and D.P. Heyman, Regenerative analysis and steady state distributions for Markov chains, Operations Res 33 (1985), 1107-1116.
6. W.K. Grassmann and D.P. Heyman, Equilibrium distribution of block-structured Markov chains with repeating rows, J Appl Prob 27 (1990), 557-576.
7. W.K. Grassmann and D.P. Heyman, Computation of steady-state probabilities for infinite-state Markov chains with repeating rows, ORSA J Comput 5 (1993), 292-303.
8. J.J. Hunter, Mathematical techniques of applied probability, Vol. I., Academic Press, New York, 1983.
9. D.P. Heyman, Approximating the stationary distribution of an infinite stochastic matrix, J Appl Prob 28 (1991), 96-103.
10. E.P.C. Kao, Using state reduction for computing steady state probabilities of queues of GI/PH/1 type, ORSA J Comp 3 (1991), 231-240.
11. E.P.C. Kao, Using state reduction for computing steady state vectors in Markov chains of M/G/1 type, Queueing Syst 10 (1992), 89-104.
12. S.K. Karlin and H.M. Taylor, A second course in stochastic processes, Academic Press, New York, 1981.
13. J.G. Kemeny, J.L. Snell, and A.W. Knapp, Denumerable Markov chains, 2nd ed, Springer-Verlag, New York, 1976.
14. G. Latouche, "Algorithms for infinite Markov chains with repeating columns," Linear algebra, Markov chains and queueing models, C.D. Meyer and R.J. Plemmons (Editors), Springer-Verlag, New York 1993, pp. 231-265.
15. G.R. Murthy, M. Kim, and E.J. Coyle, Equilibrium analysis of skip free Markov chains: nonlinear matrix equations, Stoch Models 7 (1991), 547-571.
16. M.F. Neuts, Matrix-geometric solutions in stochastic models: An algorithmic approach, The Johns Hopkins University Press, Baltimore, 1981.
17. M.F. Neuts, Structured stochastic matrices of M/G/1 type and their applications, Marcel Decker, New York, 1989.
18. M.L. Puterman, Markov decision processes - discrete stochastic dynamic programming, Wiley, New York, 1994.
19. S.M. Ross, Stochastic processes, Wiley, New York, 1983.
20. U.G. Rothblum and P. Whittle, Growth optimality for branching Markov decision chains, Math Operations Res 7 (1982), 582-601.
21. E. Seneta, Finite approximation to infinite non-negative matrices, Proc Camb Phil Soc 63 (1967), 983-992.
22. E. Seneta, Finite approximation to infinite non-negative matrices, II, Proc Camb Phil Soc 64 (1968), 465-470.
23. E. Seneta, Non-negative matrices and Markov chains, 2nd ed, Springer-Verlag, New York, 1981.
24. L.I. Sennott, The computation of average optimal policies in denumerable state Markov decision chains, Adv Appl Prob 29 (1997), 114-137.
25. T.J. Sheskin, A Markov chain partitioning algorithm for computing steady state probabilities, Operations Res 33 (1985), 229-235.
26. R.L. Tweedie, Criteria for classifying general Markov chains, Adv Appl Prob 8 (1971), 737-771.
27. D. Wolf, Approximation of the invariant probability measure of an infinite stochastic matrix, Adv Appl Prob 12 (1980), 710-726.
28. Y.Q. Zhao and D. Liu, The censored Markov chain and the best augmentation, J Appl Prob 33 (1996), 623-629.
29. Y.Q. Zhao, W. Li, and W.J. Braun, Infinite block-structured transition matrices and their properties, Adv Appl Prob 30 (1998), 365-384.