Computing average optimal constrained policies in stochastic dynamic programming

Probability in the Engineering and Informational Sciences

Volumn 15, Issue 1, 2001, Pages 103-133

  Sennott, Linn I ^a  

^a ILLINOIS STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ITERATIVE METHODS; LAGRANGE MULTIPLIERS; QUALITY OF SERVICE; STOCHASTIC SYSTEMS;

CONSTRAINED PROBLEM; FINITE STATE; LAGRANGE MULTIPLIER APPROACH; OPTIMAL POLICIES; SERVICE COSTS; STOCHASTIC DYNAMIC PROGRAM; STOCHASTIC DYNAMIC PROGRAMMING; VALUE ITERATION;

DYNAMIC PROGRAMMING;

EID: 0035625735     PISSN: 02699648     EISSN: None     Source Type: Journal    
DOI: 10.1017/S0269964801151089     Document Type: Article

References (23)

1. Altman, E. (1999).Constrained Markov decision process. Boca Raton, FL: Chapman & Hall/CRC Press.
2. Altman, E. & Shwartz, A. (1989). Optimal priority assignment: A time sharing approach. IEEE Transactions on Automatic Control AC-34: 1098-1102.
3. Altman, E. & Shwartz, A. (1993). Time-sharing policies for controlled Markov chains. Operations Research 41: 1116-1124.
4. Beutler, F.J. & Ross, K.W. (1985). Optimal policies for controlled Markov chains with a constraint. Journal of Mathematical Analysis and Applications 112: 236-252.
5. Denardo, E.V. (1982). Dynamic programming, models and applications. Englewood Cliffs, NJ: Prentice-Hall.
6. Derman, C. (1970). Finite state Markovian decision processes. New York: Academic.
7. Haviv, M. (1996). On constrained Markov decision processes. Operations Research Letters 19: 25-28.
8. Hordijk, A. & Kallenberg, L.C. (1984). Constrained undiscounted stochastic dynamic programming. Mathematics of Operations Research 9: 276-289.
9. Kallenberg, L.C. (1980). Linear programming and finite Markovian control problems. Unpublished Ph.D. dissertation, Leiden University, The Netherlands.
10. Koole, G.M. (1988). Stochastische Dynamische Programmering met Bijvoorwaarden. Unpublished Master's thesis, Leiden University, The Netherlands.
11. Ma, D.-J. & Makowski, A.M. (1988). A class of steering policies under a recurrence condition. Proceedings 27th Conference on Decision and Control. Austin, TX.
12. Ma, D.-J., Makowski, A.M., & Shwartz, A. (1990). Stochastic approximation for finite state Markov chains. Stochastic Processes and Their Applications 35:27-45.
13. Makowski, A.M. & Shwartz, A. (1988). Implementation issues for Markov decision processes. In W. Fleming & P.L. Lions (eds.), Stochastic differential systems, stochastic control theory and applications. New York: Springer-Verlag, pp. 323-337.
14. Makowski, A.M. & Shwartz, A. (1993). On constrained optimization of the Klimov network and related Markov decision processes. IEEE Transactions on Automatic Control AC-38: 354-359.
15. Piunovskiy, A.B. (1997). Optimal control of random sequences in problems with constraints. Dordrecht: Kluwer.
16. Puterman, M.L. (1994). Markov decision processes: Discrete dynamic programming. New York: Wiley.
17. Ross, K.W. (1985). Constrained Markov decision processes with queueing applications. Unpublished Ph.D. dissertation, University of Michigan, Ann Arbor.
18. Ross, K.W. (1989). Randomized and past-dependent policies for Markov decision processes with multiple constraints. Operations Research 37: 474-477.
19. Sennott, L.I. (1993). Constrained average cost Markov decision chains. Probability in the Engineering and Informational Sciences 7: 69-83.
20. Sennott, L.I. (1999). Stochastic dynamic programming and the control of queueing systems. New York: Wiley.
21. Shwartz, A. & Makowski, A.M. (1986). An optimal adaptive scheme for two competing queues with constraints. In A. Bensoussan & J.L. Lions (eds.), Analysis and optimization of systems. New York: Springer-Verlag, pp. 515-532.
22. Spieksma, F.M. (1990). Geometrically ergodic Markov chains and the optimal control of queues. Unpublished Ph.D. dissertation, Leiden University, The Netherlands.
23. Webster, R. (1994). Convexity. Oxford: Oxford University Press.