Dynamic programming equations for discounted constrained stochastic control

IEEE Transactions on Automatic Control

Volumn 49, Issue 5, 2004, Pages 699-709

  Chen, Richard C ^a   Blankenship, Gilmer L ^b  

^a NAVAL RESEARCH LABORATORY   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

Author keywords

Constrained Markov decision processes; Dynamic programming; Pure policies; Stochastic control

Indexed keywords

CONSTRAINT THEORY; CONTROL SYSTEM SYNTHESIS; DYNAMIC PROGRAMMING; MARKOV PROCESSES; MATHEMATICAL MODELS; NUMERICAL ANALYSIS;

CONSTRAINED MARKOV DECISION PROCESSES; CONSTRAINED STOCHASTIC CONTROL; PURE POLICIES;

STOCHASTIC CONTROL SYSTEMS;

EID: 2942566408     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2004.826725     Document Type: Article

References (19)

1. R. Bellman and S. Dreyfus, Applied Dynamic Programming. Princeton, NJ: Princeton Univ. Press, 1962.
2. D. P. Bertsekas, Dynamic Programming and Optimal Control (Volume One). Belmont, MA: Athena Scientific, 1995.
3. R. C. Chen and W. D. Blair, "Optimal measurement scheduling for track accuracy control for cued target acquisition," in Proc. Signal Data Processing Small Targets 1996, O. E. Oliver E. Drummond, Ed., 1996, pp. 406-417.
4. R. C. Chen, "Dynamic programming equations for constrained stochastic control," Ph. D. dissertation, Univ. Maryland, College Park, MD, 2000.
5. R. C. Chen and G. L. Blankenship, "Dynamic programming equations for constrained stochastic control," in Proc. 2002 Amer. Control Conf., vol. 3, Anchorage, AK, May, 8-10 2002, pp. 2014-2022.
6. S. Coraluppi and S. I. Marcus, "Mixed risk-neutral/minimax control of discrete-time, finite-state Markov decision processes," IEEE Trans. Automat. Contr., vol. 45, pp. 528-532, Mar. 2000.
7. S. Coraluppi, "Optimal control of Markov decision processes for performance and robustness," Ph.D. dissertation, Univ. Maryland, College Park, MD, 1997.
8. C. Derman and M. Klein, "Some remarks on finite horizon Markovian decision models," Oper. Res., vol. 13, pp. 272-278, 1965.
9. E. A. Feinberg and A. Schwartz, "Constrained dynamic programming with two discount factors: applications and an algorithm," IEEE Trans. Automat. Contr., vol. 44, pp. 628-631, Mar. 1999.
10. ____, "Constrained discounted dynamic programming," Math. Oper. Res., vol. 21, no. 4, pp. 922-945, Nov. 1996.
11. ____, "Constrained Markov decision models with weighted discounted rewards," Math. Oper. Res., vol. 20, no. 2, pp. 302-320, May 1995.
12. E. B. Frid, "On optimal strategies in control problems with constraints," Theory Prob. Appl., vol. 17, pp. 188-192, 1972.
13. O. Hernandez-Lerma, Adaptive Markov Control Processes. New York: Springer-Verlag, 1989.
14. D. H. Jacobson, "Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games," IEEE Trans. Automat. Contr., vol. AC-18, pp. 124-131, Jan. 1973.
15. W. M. McEneaney, "Risk-sensitive control of nonlinear systems," SIAG/CST Newsletter, SIAM Activity Group Control System Theory, vol. 4, no. 1, Mar. 1996.
16. ____, "Risk-sensitive control of nonlinear systems," SIAG/CST Newsletter, SIAM Activity Group Control System Theory, vol. 4, no. 2, Mar. 1996.
17. A. B. Piunovskiy and X. Mao, "Constrained Markovian decision processes: the dynamic programming approach," Oper. Res. Lett., vol. 27, pp. 119-126, 2000.
18. D. J. White, "Mean, variance, and probabilistic criteria in finite Markov decision processes: a review," J. Optim. Theory Applicat., vol. 56, no. 1, pp. 1-29, 1988.
19. P. Whittle, "Risk-sensitive linear/quadratic/gaussian control," Adv. Appl. Probab., vol. 13, pp. 764-777, 1981.