Control of singularly perturbed Markov chains: A numerical study

ANZIAM Journal

Volumn 45, Issue 1, 2003, Pages 49-74

  Yang, H ^a   Yin, G ^b   Yin, K ^c   Zhang, Q ^d  

^a University of Minnesota   (United States)

^b WAYNE STATE UNIVERSITY   (United States)

^c UNIVERSITY OF MINNESOTA   (United States)

^d UNIVERSITY OF GEORGIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0142039886     PISSN: 14461811     EISSN: None     Source Type: Journal    
DOI: 10.1017/S1446181100013158     Document Type: Article

References (28)

1. M. Abbad, J. A. Filar and T. R. Bielecki, "Algorithms for singularly perturbed limiting average Markov control problems", IEEE Trans. Automat. Control AC-37 (1992) 1421-1425.
2. D. Bertsekas, Dynamic programming: deterministic and stochastic models (Prentice-Hall, Englewood Cliffs, New Jersey, 1987).
3. G. Blankenship, "Singularly perturbed difference equations in optimal control problems", IEEE Trans. Automat. Control T-AC 26 (1981) 911-917.
4. P. J. Courtois, Decomposability: queuing and computer system applications (Academic Press, New York, 1977).
5. M. H. A. Davis, Markov models and optimization (Chapman and Hall, London, 1993).
6. F. Delebecque and J. Quadrat, "Optimal control for Markov chains admitting strong and weak interactions", Automatica 17 (1981) 281-296.
7. S. N. Ethier and T. G. Kurtz, Markov processes: characterization and convergence (J. Wiley, New York, 1986).
8. W. H. Fleming and R. W. Rishel, Deterministic and stochastic optimal control (Springer, New York, 1975).
9. F. C. Hoppensteadt and W. L. Miranker, "Multitime methods for systems of difference equations", Studies Appl. Math. 56 (1977) 273-289.
10. R. Z. Khasminskii, G. Yin and Q. Zhang, "Asymptotic expansions of singularly perturbed systems involving rapidly fluctuating Markov chains", SIAM J. Appl. Math. 56 (1996) 277-293.
11. R. Z. Khasminskii, G. Yin and Q. Zhang, "Constructing asymptotic series for probability distribution of Markov chains with weak and strong interactions", Quart. Appl. Math. 55 (1997) 177-200.
12. H. J. Kushner, Approximation and weak convergence methods for random processes, with applications to stochastic systems theory (MIT Press, Cambridge, MA, 1984).
13. H. J. Kushner and G. Yin, Stochastic approximation algorithms and applications (Springer, New York, 1997).
14. R. H. Liu, Q. Zhang and G. Yin, "Nearly optimal control of singularly perturbed Markov decision processes in discrete time", Appl. Math. Optim. 44 (2001) 105-129.
15. ∞-control of Markovian jump linear systems and solutions to associated piecewise-deterministic differential games", in New trends in dynamic games and applications (ed. G. J. Olsder), (Birkhäuser, Boston, 1995) 61-94.
16. A. A. Pervozvanskii and V. G. Gaitsgori, Theory of suboptimal decisions: decomposition and aggregation (Kluwer, Dordrecht, 1988).
17. R. G. Phillips and P. V. Kokotovic, "A singular perturbation approach to modelling and control of Markov chains", IEEE Trans. Automat. Control 26 (1981) 1087-1094.
18. S. Ross, Introduction to stochastic dynamic programming (Academic Press, New York, 1983).
19. S. P. Sethi and Q. Zhang, Hierarchical decision making in stochastic manufacturing systems (Birkhäuser, Boston, 1994).
20. H. A. Simon and A. Ando, "Aggregation of variables in dynamic systems", Econometrica 29 (1961) 111-138.
21. W. A. Thompson, Jr., Point process models with applications to safety and reliability (Chapman and Hall, New York, 1988).
22. D. N. C. Tse, R. G. Gallager and J. N. Tsitsiklis, "Statistical multiplexing of multiple time-scale Markov streams", IEEE J. Selected Areas Comm. 13 (1995) 1028-1038.
23. D. J. White, Markov decision processes (Wiley, New York, 1992).
24. G. Yin and Q. Zhang, Continuous-time Markov chains and applications: a singular perturbation approach (Springer, New York, 1998).
25. G. Yin and Q. Zhang, "Singularly perturbed discrete-time Markov chains", SIAM J. Appl. Math. 61 (2000) 834-854.
26. G. Yin, Q. Zhang and G. Badowski, "Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states", Ann. Appl. Probab. 10 (2000) 549-572.
27. G. Yin, Q. Zhang and G. Badowski, "Discrete-time singularly perturbed Markov chains: aggregation, occupation measures, and switching diffusion limit", to appear in Adv. Appl. Probab.
28. Q. Zhang and G. Yin, "On nearly optimal controls of hybrid LQG problems", IEEE Trans. Automat. Control 44 (1999) 2271-2282.