An algorithmic approach for sensitivity analysis of perturbed quasi-birth-and-death processes

Queueing Systems

Volumn 48, Issue 3-4, 2004, Pages 365-397

  Li, Quan Lin ^a   Liu, Liming ^b  

^a TSINGHUA UNIVERSITY   (China)

^b HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

Author keywords

Markovian arrival process (MAP); perturbation analysis; perturbed Markov chain; phase type (PH) distribution; QBD process; RG factorization; sensitivity analysis; stochastic integral functional

Indexed keywords

CONTINUOUS TIME SYSTEMS; FACTORIZATION; INTEGRAL EQUATIONS; MARKOV CHAINS; STOCHASTIC SYSTEMS; TRANSIENT ANALYSIS;

MARKOVIAN ARRIVAL PROCESS; PERTURBATION ANALYSIS; PHASE TYPE DISTRIBUTIONS; QBD PROCESS; STOCHASTIC INTEGRAL;

SENSITIVITY ANALYSIS;

EID: 8144221250     PISSN: 02570130     EISSN: 15729443     Source Type: Journal    
DOI: 10.1023/B:QUES.0000046582.57514.35     Document Type: Article

References (58)

1. E. Altman, K.E. Avrachenkov and R. Núñez-Queija, Perturbation analysis for denumerable Markov chains with application to queueing models, Technical Report. No. 4713, INRIA, Sophia Antipolis (2003).
2. G. Bao, C.G. Cassandras and M.A. Zazanis, First- and second-derivatives estimators for cyclic closed-queueing networks, IEEE Trans. Automat. Control 41 (1996) 1106-1124.
3. T.R. Bielecki and L. Stettner, Ergodic control of singularly perturbed Markov process in discrete time with general state and compact action spaces, Appl. Math. Optim. 38 (1998) 261-281.
4. L.W. Bright and P.G. Taylor, Calculating the equilibrium distribution in level-dependent quasi-birth-and-death processes, Stochastic Models 11 (1995) 497-526.
5. L.W. Bright and P.G. Taylor, Equilibrium distribution in level-dependent quasi-birth-and-death processes, in: Matrix Analytic Methods in Stochastic Models, eds. A.S. Alfa and S. Chakravarthy (Marcel Dekker, New York, 1996) pp. 359-375.
6. X.R. Cao, Realization Probabilities: The Dynamics of Queuing Systems (Springer, London, 1994).
7. X.R. Cao, The Maclaurin series for performance functions of Markov chains, Adv. in Appl. Probab. 30 (1998) 676-692.
8. X.R. Cao, The relations among potentials, perturbation analysis, and Markov decision processes, Discrete Event Dyn. Systems 8 (1998) 71-87.
9. X.R. Cao, From perturbation analysis to Markov decision processes and reinforcement learning, Discrete Event Dyn. Systems 13 (2003) 9-39.
10. X.R. Cao, Semi-Markov decision problems and performance sensitivity analysis, IEEE Trans. Automat. Control 48 (2003) 758-769.
11. X.R. Cao and H.F. Chen, Perturbation realization, potentials, and sensitivity analysis of Markov processes, IEEE Trans. Automat. Control 42 (1997) 1382-1393.
12. X.R. Cao and Y.W. Wan, Algorithms for sensitivity analysis of Markov systems through potentials and perturbation realization, IEEE Trans. Control Systems Technol. 6 (1998) 482-494.
13. X.R. Cao, X.M. Yuan and L. Qiu, A single sample path-based performance sensitivity formula for Markov chains, IEEE Trans. Automat. Control 41 (1996) 1814-1817.
14. C.G. Cassandras, Discrete-Event Systems: Modeling and Performance Analysis (Aksen Associates, Homewood, IL, 1993).
15. C.S. Chang and F. Nelson, Perturbation analysis of the M/M/1 queue in a Markovian environment via the matrix-geometric method, Stochastic Models 9 (1993) 233-246.
16. L.Y. Dai, Sensitivity analysis of stationary performance measures for general Markov chains, Special Issue on Discrete-Event Dynamic Systems of Math. Computer Modeling 23 (12) (1996) 143-160.
17. D.J. Daley, The total waiting time in a busy period of a stable single-server queue, I, J. Appl. Probab. 6 (1969) 550-564.
18. D.J. Daley and D.R.J. Jacobs, The total waiting time in a busy period of a stable single-server queue, II, J. Appl. Probab. 6 (1969) 565-572.
19. E. de Souza e Silva and H.R. Gail, Transient solutions for Markov chains, in: Computational Probability, ed. W.K. Grassmann (Kluwer Academic, Dordrecht, 2000) pp. 43-79.
20. D. Freedman, Markov Chains (Springer, New York, 1983).
21. P. Glasserman, Gradient Estimation via Perturbation Analysis (Kluwer, Boston, MA, 1991).
22. P. Glasserman, Infinitesimal perturbation analysis of a birth and death process, Oper. Res. Lett. 40 (1992) 292-308.
23. P. Glasserman, Derivative estimates from simulation of continuous-time Markov chains, Oper. Res. 40 (1992) 292-308.
24. W.B. Gong, J. Pan and C.G. Cassandras, Obtaining derivatives of stationary queue length probabilities through the matrix-geometric solution method, Appl. Math. Lett. 5 (1992) 69-73.
25. W.K. Grassmann, Means and variances in Markov reward systems, in: Linear Algebra, Markov Chains, and Queueing Models, eds. C.D. Meyer and R.J. Plemmons (Springer, New York, 1993) pp. 193-204.
26. W.K. Grassmann, Optimizing steady state Markov chains by state reduction, European J. Oper. Res. 89 (1996) 277-284.
27. W.K. Grassmann and X. Chen, The use of derivatives for optimizing steady state queues, J. Oper. Res. Soc. 46 (1995) 104-115.
28. W.K. Grassmann, X. Chen and B.R.K Kashyap, Optimal service rates for the state-dependent M/G/1 queues in steady state, Oper. Res. Lett. 29 (2001) 57-63.
29. W.K. Grassmann and D.P. Heyman, Equilibrium distribution of block-structured Markov chains with repeating rows, J. Appl. Probab. 27 (1990) 557-576.
30. W.K. Grassmann and D.P. Heyman, Computation of steady-state probabilities for infinite-state Markov chains with repeating rows, ORSA J. Comput. (1993) 282-303.
31. Q.M. He, Differentiability of matrices R and G in the matrix-analytic method, Stochastic Models 11 (1995) 123-132.
32. Q.M. He and M.F. Neuts, On episodic queues, SIAM J. Matrix Anal. Appl. 18 (1997) 223-248.
33. D.P. Heyman, A decomposition theorem for infinite stochastic matrices, J. Appl. Probab. 32 (1995) 893-901.
34. Y.C. Ho and X.R. Cao, Perturbation Analysis of Discrete-Event Dynamic Systems (Kluwer, Boston, MA, 1991).
35. J.G. Kemeny, J.L. Snell and A.W. Knapp, Denumerable Markov Chains (Springer, New York, 1976).
36. V.S. Korolyuk and A.F. Turbin, Mathematical Foundations of the State Lumping of Large System (Kluwer Academic, Dordrecht, 1993).
37. V.G. Kulkarni, V.F. Nicola and K.S. Trivedi, The completion time of a job on multimode systems, Adv. in Appl. Probab. 19 (1987) 932-954.
38. G. Latouche, Perturbation analysis of phase-type queue with weakly correlated arrivals, Adv. in Appl. Probab. 20 (1988) 896-912.
39. G. Latouche and V. Ramaswami, A logarithmic reduction algorithm for quasi-birth-death processes, J. Appl. Probab. 30 (1993) 650-674.
40. G. Latouche and V. Ramaswami, Introduction to Matrix Analytic Methods in Stochastic Models (ASA-SIAM, 1999).
41. Q.L. Li and Y.Q. Zhao, A constructive method for finding β-invariant measures for transition matrices of M/G/1 type, in: Matrix Analytic Methods Theory and Applications, eds. G. Latouche and P.G. Taylor (World Scientific, Singapore, 2002) pp. 237-263.
42. Q.L. Li and Y.Q. Zhao, The RG-factorization in block-structured Markov renewal process with applications, Technical Report, No. 355, Laboratory for Research in Statistics and Probability, Carleton University and University of Ottawa (2002).
43. Q.L. Li and Y.Q. Zhao, β-invariant measures for transition matrices of GI/M/1 type, Stochastic Models 19 (2003) 201-233.
44. J. Meyer, On evaluating the performability of degradable computing systems, IEEE Trans. Comput. 29 (1980) 720-731.
45. E. Naddor, Inventory Systems (Wiley, New York, 1966).
46. V. Naoumov, Matrix-multiplicative approach to quasi-birth-and-death processes, in: Matrix Analytic Methods in Stochastic Models, eds. A.S. Alfa and S. Chakravarthy (Marcel Dekker, New York, 1996) pp. 87-106.
47. M.F. Neuts, Matrix-Geometric Solutions in Stochastic Models - An Algorithmic Approach (Johns Hopkins Univ. Press, Baltimore, MD, 1981).
48. M.F. Neuts, Structured Stochastic Matrices of M/G/1 Type and Their Applications (Marcel Dekker, New York, 1989).
49. M.F. Neuts, Matrix-analytic methods in stochastic models, in: Advances in Queueing: Theory, Methods and Open Problems, ed. J. Dhashalow (CRC Press, Boca Raton, FL, 1995) pp. 265-292.
50. P.S. Puri, On the homogeneous birth-and-death process and its integral, Biometrika 53 (1966) 61-71.
51. P.S. Puri, Some further results on the homogeneous birth and death process and its integral, Proc. Cambridge Philos. Soc. 64 (1968) 141-154.
52. M.L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming (Wiley, New York, 1994).
53. V. Ramaswami, A tutorial overview of matrix analytic method: with some extensions & new results, in: Matrix Analytic Methods in Stochastic Models, eds. A.S. Alfa and S. Chakravarthy (Marcel Dekker, New York, 1996) pp. 261-296.
54. V. Ramaswami and P.G. Taylor, Some properties of the rate operators in level dependent quasi-birth-and-death processes with a countable number of phases, Stochastic Models 12 (1996) 143-164.
55. U. Sumita and Y. Masuda, An alternative approach to the analysis of finite semi-Markov and related processes, Stochastic Models 3 (1987) 67-87.
56. R. Suri and M.A. Zazanis, Perturbation analysis gives strongly consistent sensitivity estimates for the M/G/1 queue, Managm. Sci. 34 (1988) 39-64.
57. M.I. Taksar and W.K. Grassmann, Probabilistic approach to computational algorithms for finding stationary distributions of Markov chains, J. Comput. Appl. Math. 36 (1991) 131-136.
58. Y.Q. Zhao, Censoring technique in studying block-structured Markov chains, in: Advances in Algorithmic Methods for Stochastic Models, eds. G. Latouche and P.G. Taylor (Notable Publications, New York, 2000) pp. 417-433.