Perturbation analysis for denumerable Markov chains with application to queueing models

Advances in Applied Probability

Volumn 36, Issue 3, 2004, Pages 839-853

  Altman, Eitan ^a   Avrachenkov, Konstantin E ^a   Núnez Queija, Rudesindo ^b  

^a INRIA   (France)

^b EINDHOVEN UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

Denumerable Markov chain; Geometric ergodicity; Perturbation analysis; Quasi birth and death process; Queueing model

Indexed keywords

LYAPUNOV METHODS; MATHEMATICAL MODELS; PERTURBATION TECHNIQUES; PROBABILITY; QUEUEING THEORY; STATE SPACE METHODS; THEOREM PROVING;

DENUMERABLE MARKOV CHAIN; GEOMETRIC ERGODICITY; PERTURBATIN ANALYSIS; QUASI-BIRTH-AND-DEATH PROCESS; QUEUEING MODEL;

MARKOV PROCESSES;

EID: 7444231143     PISSN: 00018678     EISSN: None     Source Type: Journal    
DOI: 10.1239/aap/1093962237     Document Type: Article

References (30)

1. Altman, E., Artiges, D. and Traore, K. (1999). On the integration of best-effort and guaranteed performance services. Europ. Trans. Telecommun. 10, 125-134.
2. Anisimov, V. V. (2001). Asymptotic analysis of some type optimal maintenance policies in multicomponent systems with Markov switches. In Proc. 10th Internat. Symp. Appl. Stoch. Models Data Anal. (Compiegne, June 2001), pp. 112-117. Available at http://www.hds.utc.fr /asmda2001/.
3. Avrachenkov, K. E. (1999). Analytic perturbation theory and its applications. Doctoral Thesis, University of South Australia. Available at http://www-sop.inria.fr/mistral/personnel/K.Avrachenkov/moi.html
4. Avrachenkov, K. E., Filar, J. A. and Haviv, M. (2002). Singular perturbations of Markov chains and decision processes. In Handbook of Markov Decision Processes: Methods and Applications, eds E. A. Feinberg and A. Shwartz, Kluwer, Boston, MA, pp. 113-150.
5. Bielecki, T. R. and Stettner, L. (1998). Ergodic control of singularly perturbed Markov process in discrete lime with general state and compact action spaces. Appl. Math. Optimization 38, 261-281.
6. Cao, X.-R. and Chen, H.-F. (1997). Perturbation realization, potentials, and sensitivity analysis of Markov processes. IEEE Trans. Automatic Control 42, 1382-1393.
7. Chang, C.-S. and Nelson, R. (1993). Perturbation analysis of the M/M/1 queue in a Markovian environment via the matrix-geometric method. Commun. Statist. Stoch. Models 9, 233-246.
8. Courtois, P. J. (1977). Decomposability: Queueing and Computer System Applications. Academic Press, New York.
9. Dekker, R., Hopduk, A. and Spieksma, F. M. (1994). On the relation between recurrence and ergodicity properties in denumerable Markov decision chains. Math. Operat. Res. 19, 539-559.
10. Delebecque, F. (1983). A reduction process for perturbed Markov chains. SIAM J. Appl. Math. 43, 325-350.
11. Gelenbe, E. and Rosenberg, C. (1990). Queues with slowly varying arrival and service processes. Management Sci. 36, 928-937.
12. Glasserman, P. (1991). Gradient Estimation via Perturbation Analysis. Kluwer, Boston, MA.
13. Heidergott, B. and Cao, X.-R. (2002). A note on the relation between weak derivatives and perturbation realization. IEEE Trans. Automatic Control 47, 1112-1115.
14. Heidergott, B., Horduk, A. and Weisshaupt, H. (2002). Measure-valued differentiation for stationary Markov chains. Res. Rep. 2002-27, EURANDOM. Available at http://staff.feweb.vu.rd/bheidergott/.
15. Kolmogorov, A. N. and Fomin, S. V. (1975). Introductory Real Analysis. Dover, New York.
16. Kontoyiannis, I. and Meyn, S. P. (2003). Spectral theory and limit theory for geometrically ergodic Markov processes. Ann. Appl. Prob. 13, 304-362.
17. Koole, G. (1998). The deviation matrix of the M/M/1/∞ and MIMI1 /N queue with applications to controlled queueing models. In Proc. IEEE CDC'98 (Tampa, FL), pp. 56-59.
18. Korolyuk V.S.and Turbin, A.F. (1993). Mathematical Foundations of The State Lumping of Large Systems. Kluwer, Dordrecht.
19. Latouche, G. and Ramaswami, V. (1999). Introduction to Matrix Analytic Methods in Stochastic Modeling. SIAM, Philadelphia, PA.
20. Latouche, G. and Schweitzer, P. G. (1995).A Markov modulated, nearly completely decomposable M/M/1 queue. In Computations with Markov chains, ed. W. J. Stewart, Kluwer, Boston, MA.
21. Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.
22. Meyn, S. P. and Tweedie, R. L. (1994). Computable bounds for geomelric convergence rates of Markov chains. Ann. Appl. Prob. 4, 981-1012.
23. Neuts, M. F. (1981). Matrix-Geometric Solutions in Stochastic Models: An Algorithmic Approach. Johns Hopkins University Press, Baltimore, MD.
24. Núnez-queija, R. (1999). Processor-sharing models for integrated-services networks. Doctoral Thesis, Eindhoven University of Technology. Available at http://www.cwi.nl/~sindo/.
25. Pervozvanski, A. A. and Gaitsgori, V. G. (1988). Theory of Suboptimal Decisions. Kluwer, Dordrecht. Russian original: Decomposition, Aggregation and Approximate Optimization, Nauka, Moscow, 1979.
26. Pflug, G. (1992). Gradient estimates for the performance of Markov chains and discrete event processes. Ann. Operat. Res. 39, 173-194.
27. Spieksma, F. M. (1990). Geometrically ergodic Markov chains and the optimal control of queues. Doctoral Thesis, Leiden University.
28. Spieksma, F. M. and Tweedie, R. L. (1994). Strengthening ergodicity to geometric ergodicity for Markov chains. Commun. Statist. Stoch. Models 10, 45-74.
29. Yin, G. and Zhang, H. (2002). Countable-state-space Markov chains with two time scales and applications to queueing systems. Adv. Appl. Prob. 34, 662-688.
30. Yin, G. and Zhang, Q. (1998). Continuous-Time Markov Chains and Applications. A Singular Perturbation Approach (Appl. Math. 37). Springer, New York.