The state reduction and related algorithms and their applications to the study of Markov chains, graph theory, and the optimal stopping problem

Advances in Mathematics

Volumn 145, Issue 2, 1999, Pages 159-188

  Sonin, Isaac ^a  

^a UNIVERSITY OF NORTH CAROLINA AT CHARLOTTE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0033542959     PISSN: 00018708     EISSN: None     Source Type: Journal    
DOI: 10.1006/aima.1998.1813     Document Type: Article

References (25)

1. Sheskin T. J. A Markov partitioning algorithm for computing steady-state probabilities. Oper. Res. 33:1985;228-535.
2. Grassmann W. K., Taksar M. I., Heyman D. P. Regenerative analysis and steady-state distributions for Markov chains. Oper. Res. 33:1985;1107-1116.
3. Kemeny J. G., Snell J. L. Finite Markov Chains. 1960;Van Nostrand-Reinhold, Princeton.
4. Kemeny J. G., Snell J. L., Knapp A. V. Denumerable Markov Chains. 1976;Springer-Verlag, New York.
5. Isaacson D. L., Madsen R. W. Markov Chains: Theory and Applications. 1976;Wiley, New York.
6. Meyer C. D., Plemmons R. J. Linear Algebra, Markov Chains, and Queueing models. IMA Vol. Math. Appl. 48:1993;Springer-Verlag, New York.
7. Heyman D. P. Accurate computation of the fundamental matrix of a Markov chain. SIAM J. Matrix Anal. Appl. 16:1995;954-963.
8. Kohlas J. Numerical computation of mean passage times and absorption probabilities in Markov and semi-markov models. Z. Operat. Res. 30:1986;A197-A207.
9. Kumar S., Grassmann W. K., Billinton R. A stable algorithm to calculate steady-state probability and frequency of a Markov system. IEEE Trans. Reliability. R36:1987;58-62.
10. Heyman D. P. Further comparisons of direct methods for computing stationary distributions of Markov chains. SIAM J. Algebraic Discrete Methods. 8:1987;226-232.
11. Heyman D. P., Reeves A. Numerical solution of linear equations arising in Markov chain models. ORSA J. Comput. 1:1989;52-60.
12. Lal R., Bhat U. N. Reduced systems algorithms for Markov chains. Management Sci. 34:1988;1202-1220.
13. Grassmann W. K., Heyman D. P. Equilibrium distribution of block-structured Markov chains with repeating rows. J. Appl. Probab. 27:1990;557-576.
14. Heyman D. P. A direct algorithm for computing the stationary distribution of a p-cyclic Markov chain. Linear Algebra, Markov Chains, and Queueing Models. IMA Vol. Math. Appl. 48:1993;Springer-Verlag, New York. p. 205-209.
15. Freidlin M. I., Wentzell A. D. Perturbations of Stochastic Dynamic Systems. 1984;Springer-Verlag, Berlin/New York.
16. O'Cinneide C. A. Entrywise perturbation theory and error analysis for Markov chains. Numer. Math. 65:1993;109-120.
17. C. Alexopoulos, A. El-Tannir, and, R. Serfozo, Partition-reversible Markov processes, preprint, 1997.
18. Sheskin T. J. Computing the fundamental matrix for a reducible Markov chain. Math. Mag. 68:1995;393-398.
19. I. M. Sonin, Two simple theorems in the problems of optimal stopping, in, Proceedings, INFORMS Appl. Probab. Conf, Atlanta, GA, 1995.
20. I. M. Sonin, Two simple theorems in the problems of optimal stopping and the Elimination Algorithm, preprint, 1997.
21. Chow Y. S., Robbins H., Sigmund D. Great Expectations: The Theory of Optimal Stopping. 1971;Houghton Mifflin, New York.
22. Shiryayev A. N. Optimal Stopping Rules. 1978;Springer-Verlag, New York.
23. Bogart K. Introductory Combinatorics. 1990;Harcourt Brace Jovanovich, New York.
24. Diaconis P., Strook D. Geometric bounds for eigenvalues of Markov chains. Ann. of Appl. Probab. 1:1991;36-61.
25. Chen Wai-Kai. The Circuits and Filters. Handbook. 1995;CRC Press, Boca Raton.