Continuous time discounted jump Markov decision processes: A discrete-event approach

Mathematics of Operations Research

Volumn 29, Issue 3, 2004, Pages 492-524

  Feinberg, Eugene A ^a  

^a STONY BROOK UNIVERSITY   (United States)

Author keywords

Continuous time jump Markov decision process; Semi Markov decision process

Indexed keywords

CONSTRAINT THEORY; DECISION THEORY; DIFFERENTIAL EQUATIONS; MARKOV PROCESSES; MATHEMATICAL MODELS; OPTIMIZATION; PROBABILITY DISTRIBUTIONS;

CONTINUOUS TIME JUMP MARKOV DECISION PROCESS; SEMI MARKOV DECISION PROCESS;

OPERATIONS RESEARCH;

EID: 4043163559     PISSN: 0364765X     EISSN: None     Source Type: Journal    
DOI: 10.1287/moor.1040.0089     Document Type: Article

References (39)

1. Altman, E. 1999. Constrained Markov Decision Processes. Chapman & Hall/CRC, Boca Raton, FL.
2. Bertsekas, D. P. 1995. Dynamic Programming and Optimal Control, Volume II. Athena Scientific, Belmont, MA.
3. Bertsekas, D. P., S. E. Shreve. 1978. Stochastic Optimal Control: The Discrete-Time Case. Academic Press, New York. (Republished by Athena Scientific, 1997.)
4. Borkar, V. S. 2002. Convex analytic methods in Markov decision processes. E. A. Feinberg, and A. Shwartz, eds. Handbook of Markov Decision Processes: Methods and Applications. Kluwer, Boston, MA, 347-375.
5. Cassandras, C. G. 1993. Discrete Event Systems. IRWIN, Boston, MA.
6. Derman, C., R. E. Strauch. 1966. A note on memoryless rules for controlling sequential control processes. Ann. Math. Statist. 37 276-278.
7. Dynkin, E. B., A. A. Yushkevich. 1979. Controlled Markov Processes. Springer, Berlin.
8. Feinberg, E. A. 1994. A generalization of "expectation equals reciprocal of intensity" to non-stationary exponential distributions. J. Appl. Probab. 31 262-267.
9. Feinberg, E.A. 2002. Constrained discounted semi-Markov decision processes. Z. How, J. A. Filar, and A. Chen, eds. Markov Processes and Controlled Markov Chains. Kluwer, Dordrecht, The Netherlands, 233-244.
10. Feinberg, E. A. 2002a. Constrained finite continuous-time Markov decision processes with average rewards. Proc. 2002 IEEE Conference on Decisions and Control, Las Vegas, December 2002. IEEE Control Systems Society, Piscataway, NJ, 3805-3810.
11. Feinberg, E. A., A. B. Piunovskiy. 2002. Nonatomic total reward Markov decision processes with multiple criteria. J. Math. Anal. Appl 273 93-111.
12. Feinberg, E. A., A. Shwartz. 1995. Constrained Markov decision models with weighted discounted rewards. Math. Oper. Res. 20 302-320.
13. Feinberg, E. A., A. Shwartz. 1996. Constrained discounted dynamic programming. Math. Oper. Res. 21 922-945.
14. Gonzáles-Hernández, J., O. Hernández-Lerma. 1999. Envelopes of sets of measures, tightness, and Markov control processes. Appl. Math. Optim. 40 377-392.
15. Hernández-Lerma, O., J. Gonzáles-Hernández. 2000. Constrained Markov control processes in Borel spaces. Math. Meth. Oper. Res. 52 271-285.
16. Heyman, D. P., M. J. Sobel. 1984. Stochastic Models in Operations Research, Volume II: Stochastic Optimization. McGraw-Hill, New York.
17. Hordijk, A., F. A. van der Duyn Schouten. 1979. Discretization procedures for continuous time decision processes. Trans. Eighth Prague Conf. on Inform. Theory, Statistical Decision Functions, Random Processes, 1978, Volume C. Academia, Prague, Czechoslovakia, 143-154,
18. Jacod, J. 1975. Multivariate point processes: Predictable projections, Radon-Nikodym derivatives, representation of martingales. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 31 235-253.
19. Kakumanu, P. 1971. Continuously discounted Markov decision models with countable state and action space. Ann. Math. Statist. 42 919-926.
20. Kakumanu, P. 1977. Relation between continuous and discrete time Markovian decision problems. Naval Res. Logist. Quart. 24 431-439.
21. Kallianpur, G. 1980. Stochastic Filtering Theory. Springer, New York.
22. Kitaev, Yu. M. 1985. Semi-Markov and jump Markov controlled models: Average cost criterion. SIAM Theory Probab. Appl. 30 272-288.
23. Kitaev, Yu. M., V. V. Rykov. 1995. Controlled Queueing Systems. CRC Press, New York.
24. Lippman, S. A. 1975. Applying a new device in the optimization of exponential service systems. Oper. Res. 23 687-710.
25. Miller, B. L. 1968. Finite state continuous time Markov decision processes with a finite planning horizon. SIAM J. Control 6 266-280.
26. Miller, B. L. 1968a. Finite state continuous time Markov decision processes with an infinite planning horizon. J. Math. Anal. Appl. 22 552-569.
27. Neveu, J. 1965. Mathematical Foundations of the Calculus of Probability. Holden-Day, San Francisco, CA.
28. Parthasarathy, K. R. 1967. Probability Measures on Metric Spaces. Academic Press, New York.
29. Pinedo, M. 1995. Scheduling: Theory, Algorithms, and Systems. Prentice Hall, Englewood Cliffs, NJ.
30. Piunovskiy, A. B. 1997. Optimal Control of Random Sequences in Problems with Constraints. Kluwer, Boston, MA.
31. Piunovskiy, A. B. 1997a. A controlled jump discounted model with constraints. SIAM Theory Probab. Appl. 42 51-72.
32. Puterman, M. L. 1994. Markov Decision Processes. John Wiley, New York.
33. Ross, S. M. 1983. Introduction to Stochastic Dynamic Programming. Academic Press, New York.
34. Sennott, L. I. 1999. Stochastic Dynamic Programming and the Control of Queueing Systems. Wiley, New York.
35. Serfozo, R. F. 1979. An equivalence between continuous and discrete time Markov decision processes. Oper. Res. 27 616-620.
36. Yushkevich, A. A. 1977. Controlled Markov models with countable state space and continuous time. SIAM Theory Probab. Appl. 22 215-235.
37. Yushkevich, A. A. 1980. On reducing a jump controllable Markov model to a model with discrete time. SIAM Theory Probab. Appl. 25 58-69.
38. Yushkevich, A. A. 1980a. Controlled jump Markov models. SIAM Theory Probab. Appl. 25 244-266.
39. Yushkevich, A. A., E. A. Feinberg. 1979. On homogeneous Markov models with continuous time and finite or countable state space. SIAM Theory