Networked markov decision processes with delays

IEEE Transactions on Automatic Control

Volumn 57, Issue 4, 2012, Pages 1013-1018

  Adlakha, Sachin ^a   Lall, Sanjay ^b   Goldsmith, Andrea ^b  

^a CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^b Stanford University   (United States)

Author keywords

Delayed systems; Markov decision processes; networked systems

Indexed keywords

CONTROL ACTIONS; DELAYED SYSTEMS; EXPLICIT BOUNDS; HISTORY OF MEASUREMENTS; INFORMATION STATE; MARKOV DECISION PROCESSES; NETWORKED SYSTEMS; OPTIMAL CONTROLS; PARTIALLY OBSERVED MARKOV DECISION PROCESS; STATE INFORMATION; UNDERLYING GRAPHS;

LEARNING ALGORITHMS; NETWORKED CONTROL SYSTEMS;

MARKOV PROCESSES;

EID: 84859733887     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2011.2168111     Document Type: Article

References (19)

1. J. Kuri and A. Kumar, "Optimal control of arrivals to queues with delayed queue length information," IEEE Trans. Autom. Control, vol. 40, no. 8, pp. 1444-1450, Aug. 1995.
2. E. Altman and S. Stidham, "Optimality of monotonic policies for twoaction Markovian decision processes, with applications to control of queues with delayed information," Queueing Syst., vol. 21, no. 3, pp. 267-291, 1995.
3. E. Altman and P. Nain, "Closed-loop control with delayed information," Perform. Eval. Rev., vol. 20, pp. 193-204, 1992.
4. M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York: Wiley, 1994.
5. K. J. Astrom, "Optimal control of Markov processes with incomplete state estimation," J. Math. Anal. Appl., vol. 10, pp. 174-205, 1965.
6. P. R. Kumar and P. Varaiya, Stochastic Systems: Estimation, Identification and Adaptive Control. Englewood Cliffs, NJ: Prentice-Hall, 1986.
7. R. D. Smallwood and E. J. Sondik, "The optimal control of partially observable Markov processes over a finite horizon," Oper. Res., vol. 21, no. 5, pp. 1071-1088, 1973.
8. J. L. Bander and C. C. White, III, "Markov decision processes with noise-corrupted and delayed state observations," J. Oper. Res. Soc., vol. 50, pp. 660-668, 1999.
9. E. Altman, T. Basar, and R. Srikant, "Congestion control as a stochastic control problem with action delays," Automatica, vol. 12, pp. 1937-1950, 1999.
10. K. V. Katsikopoulos and S. E. Engelbrecht, "Markov decision processes with delays and asynchronous cost collection," IEEE Trans. Autom. Control, vol. 48, no. 4, pp. 568-574, Apr. 2003.
11. M. Basin, J. Rodriguez-Gonzalez, and R. Martinez-Zuniga, "Optimal control for linear systems with time delay in control input based on the duality principle," in Proc. Amer. Control Conf., 2003, pp. 2144-2148.
12. P. Varaiya and J.Walrand, "On delayed sharing patterns," IEEE Trans. Autom. Control, vol. AC-23, no. 3, pp. 443-445, Jun. 1978.
13. P. G. Voulgaris, "Optimal control of systems with delayed observation sharing patterns via input-output methods," in Proc. IEEE Conf. Decision Control, 2000, pp. 2311-2316.
14. S. Adlakha, R. Madan, S. Lall, and A. Goldsmith, "Optimal control of distributed Markov decision processes with network delays," in Proc. IEEE Conf. Decision Control, 2007, pp. 3308-3314.
15. S. Adlakha, S. Lall, and A. Goldsmith, "Information state for Markov decision processes with network delays," in Proc. IEEE Conf. Decision Control, 2008, pp. 3840-3847.
16. G. E. Monahan, "A survey of partially observable Markov decision processes: Theory, models, and algorithms," Manag. Sci., vol. 28, no. 1, pp. 1-16, 1982.
17. D. Bertsekas and S. Shreve, Stochastic Optimal Control: The Discrete Time Case. New York: Academic Press, 1978.
18. D. Bertsekas, Dynamic Programming and Optimal Control. Belmont, MA: Athena Scientific, 1995, vol. 1.
19. S. Adlakha, S. Lall, and A. Goldsmith, "A Bayesian network approach to control of networked Markov decision processes," in Proc. Allerton Conf. Commun., Control, Comp., 2008, pp. 446-451.