Applications of borovkovs renovation theory to non-stationary stochastic recursive sequences and their control

Advances in Applied Probability

Volumn 29, Issue 2, 1997, Pages 388-413

  Altman, Eitan ^a   Hordijk, Arie ^b  

^a INRIA   (France)

^b LEIDEN UNIVERSITY   (Netherlands)

Author keywords

Coupling; Renovating events

Indexed keywords

EID: 0002260714     PISSN: 00018678     EISSN: None     Source Type: Journal    
DOI: 10.1017/S0001867800028056     Document Type: Article

References (40)

1. AKHMAROV, I. AND LEONT'EVA, N. P. (1976) Conditions for convergence to limit processes and the strong law of large numbers for queueing systems. Tear. Veroyatnost Primenen 21, 559-571. (In Russian.)
2. ASMUSSEN, S. AND FOSS, S. G. (1993) Renovation, regeneration, and coupling in multiple-server queues in continuous time. Frontiers Pure Appl. Prob. 1, 1-6.
3. BACCELLI, F. AND BRÉMAUD, P. (1994) Elements of Queueing Theory. Springer, Berlin.
4. BACCELLI, F. AND FOSS, S. (1994) Stability of Jackson-type queueing networks. QUESTA 17, 5-72.
5. BACCELLI, F. AND FOSS, S. (1995) On the saturation rule for the stability of queues. J. Appl. Prob. 32, 494-507.
6. BERTSEKAS, D. P. (1987) Dynamic Programming. Deterministic and Stochastic Models. Prentice Hall, Englewood Cliffs, NJ.
7. BOROVKOV, A. A. (1984) Asymptotic Methods in Queueing Theory. Wiley, New York.
8. BOROVKOV, A. A. (1996) Ergodicity and stability of stochastic processes. To appear. (In Russian.)
9. BOROVKOV, A. A. (1996) Remarks on boundedness of stochastic sequences and on estimates of their moments. Preprint.
10. BOROVKOV, A. A. AND FOSS, S. G. (1992) Stochastically recursive sequences and their generalizations. Siberian Adv. Math. 2, 16-81.
11. BOROVKOV, A. A. AND FOSS, S. G. (1994) Two ergodicity criteria for stochastically recursive sequences. ACTA Appl. Math. 34, 125-134.
12. BRANDT, A., FRANKEN, P. AND LISEK, B. (1992) Stationary Stochastic Models. Akademie, Berlin.
13. CORNFELD, I. P., FOMIN, S. V. AND SINAI, YA. G. (1982) Ergodic Theory. Springer, New York.
14. DEKKER, R. AND HORDIJK, A. (1988) Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. Math. Operat. Res. 13, 395-421.
15. DEKKER, R., HORDIJK, A. AND SPIEKSMA, F. M. (1994) On the relation between recurrence and ergodicity properties in denumerable Markov decision chains. Math. Operat. Res. 19, 539-559.
16. DERMAN, C. (1970) Finite State Markovian Decision Processes. Academic Press, New York.
17. DYNKIN, E. B. AND YUSHKEVICH, A. A. (1979) Controlled Markov Processes. Springer, Berlin.
18. FEDERGRUEN, A., HORDIJK, A. AND TIJMS, H. C. (1978) A note on simultaneous recurrence conditions on a set of denumerable stochastic matrices. J. Appl. Prob. 15, 842-847.
19. FEDERGRUEN, A. AND TIJMS, H. C. (1978) The optimality equation in average cost denumerable state semi-Markov decision problems, recurrency conditions and algorithms. J. Appl. Prob. 15, 356-373.
20. Foss, S. G. (1984) The method of renovating events and its applications in queueing theory. Semi Markov Models, Theory and Applications (Proc. 1st Symp. on Semi-Markov Processes). Brussels. (Plenum 86).
21. FOSS, S. G. Private communication.
22. Foss, S. G. AND KALASHNIKOV, V. V. (1991) Regeneration and renovation in queues. Queueing Systems 8, 211-223.
23. HAJEK, B. (1982) Hitting-time and occupation-time bounds implied by drift analysis with applications. Adv. Appl. Prob. 14, 502-525.
24. HAJEK, B. (1985) Extremal splitting of point processes. Math. Operat. Res. 10, 543-556.
25. HORDIJK, A. (1976) Regenerative Markov decision models. Math. Prog. Study 6, 49-72.
26. HORDIJK, A. (1977) Dynamic Programming and Markov Potential Theory. (Mathematical Centre Tracts 51.) 2nd edn. Mathematisch Centrum, Amsterdam.
27. HORDIJK, A. AND HOLEWIJN, P. J. (1975) On the convergence of moments in stationary Markov chains. Stoch. Proc. Appl. 3, 55-64.
28. HORDIJK, A., KOOLE, G. M. AND LOEVE, J. A. (1994) Analysis of a customer assignment model with no state information. Prob. Eng. Inf. Sci. 2, 419-429.
29. HORDIJK, A. AND LOEVE, J. A. (1994) Undiscounted Markov decision chains with partial information; an algorithm for computing a locally optimal periodic policy. Math. Meth. Operat. Res. 40, 163-181.
30. HORDIJK, A. AND LOEVE, J. A. (1996) Markov decision chains with partial information: optimality results. Technical report. University of Leiden.
31. KENDALL, D. G. (1960) Geometric ergodicity in the theory of queues. In Mathematical Methods in the Social Sciences. ed. K. J. Arrow, S. Karlin and P. Suppes. Stanford University Press, Stanford, CA. pp. 176-195.
32. KULKARNI, V. G. AND SERING, Y. (1992) Optimal implementable policies: average cost and minimax criteria. Technical report UNC/OR TR-90/9. University of North Carolina.
33. KULKARNI, V. G. AND SERING, Y. (1995) Optimal implementable policies: discounted cost case. In Computations with Markov Chains. ed. W. Stewart. Kluwer, Dordrecht. pp. 283-307.
34. LIPPMAN, S. A. AND STIDHAM, S. (1977) Individual versus social optimization in exponential congestion systems. Operat. Res. 25, 233-247.
35. LINDVALL, T. (1992) Lecture on the Coupling Method. Wiley, New York.
36. LOYNES, R. (1962) The stability of a queue with non-independent inter-arrival and service times. Proc. Camb. Phil. Soc. 58, 497-520.
37. MA, D. J. AND MAKOWSKI, A. M. (1987) Optimality results for a simple flow control problem. Proc. 26th IEEE Conf. on Decision and Control. Los Angeles, CA.
38. MEYN, S. AND TWEEDIE, R. L. (1993) Markov Chains and Stochastic Stability. Springer, Berlin.
39. SPIEKSMA, F. M. (1990) Geometrically ergodic Markov chains and the optimal control of queues. PhD thesis. University of Leiden.
40. TWEEDIE, R. L. (1983) The existence of moments for stationary Markov chains. J. Appl. Prob. 20, 191-196.