Lebesgue-sampling-based optimal control problems with time aggregation

IEEE Transactions on Automatic Control

Volumn 56, Issue 5, 2011, Pages 1097-1109

  Xu, Yan Kai ^a,b   Cao, Xi Ren ^c,d  

^a SCHLUMBERGER   (China)

^b TSINGHUA UNIVERSITY   (China)

^c SHANGHAI JIAO TONG UNIVERSITY   (China)

^d HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

Author keywords

Aggregation; Markov decision processes (MDPs); performance potentials; reinforcement learning

Indexed keywords

AGGREGATION; ANALYTICAL SOLUTIONS; LEARNING-BASED METHODS; LEBESGUE SAMPLING; MARKOV DECISION PROCESSES; MARKOV DECISION PROCESSES (MDPS); OPTIMAL CONTROL PROBLEM; OPTIMAL POLICIES; PERFORMANCE POTENTIALS; TIME AGGREGATION;

CONTROL; DECISION THEORY; MARKOV PROCESSES;

OPTIMIZATION;

EID: 79955884542     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2010.2073610     Document Type: Article

References (27)

1. H. Abou-Kandil, O. D. Smet, G. Freiling, and G. Jank, "Flow control in a failure-prone multi-machine manfacturing system," in Proc. INRIA/ IEEE Symp. Emerg. Technol.Factory Autom., 1995, vol. 2, pp. 575-583.
2. J. Abounadi, D. Bertsekas, and V. S. Borkar, "Learning algorithms for Markov decision processes with average cost," SIAM J. Control Optim., vol. 40, no. 3, pp. 681-698, 2001.
3. K. E. Arzen, "A simple event-based PID controller," in Proc. IFAC World Cong., Beijing, China, 1999, vol. 18, pp. 423-428.
4. K. J. Astrom and B. M. Bernhardsson, "Comparison of Riemann and Lebesgue sampling for first order stochastic systems," in Proc. 41th IEEE Conf. Decision Control, Las Vegas, NV, USA, December 2002.
5. K. J. Astrom, K. Johan, and B.Wittenmark, Computer-Controlled Systems, 3rd ed. Englewood Cliffs, NJ: Prentice-Hall, 1997.
6. B. Bernhardsson, "Event triggered sampling," in Research Problem Formulations in the DICOSMOS Project, M. Torngren and M. Sanfridson, Eds. Lund, Sweden: Lund Inst. Technol. Press, 1998.
7. D. P. Bertsekas and T. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
8. M. S. Branicky, V. S. Borkar, and S. Mitter, "A unified framework for hybrid control," IEEE Trans. Autom. Control, vol. 43, no. 1, pp. 31-45, Jan. 1998.
9. X. R. Cao, Stochastic Learning and Optimization-A Sensitivity-Based Approach. New York: Springer, 2007.
10. X. R. Cao, Z. Ren, S. Bhatnagar,M. Fu, and S. Marcus, "A time aggregation approach to Markov decision processes," Automatica, vol. 38, pp. 929-943, 2002. (Pubitemid 34249748)
11. S. DeWeerth, L. Nielsen, C. Mead, and K. J. Astrom, "A neuron-based pulse servo for motion control," in Proc. IEEE Int. Conf. Robot. Autom., Cincinnati, OH, 1990, vol. 3, pp. 1698-1703.
12. J. P. Forestier and P. Varaiya, "Multilayer control of large Markov chains," IEEE Trans. Autom. Control, vol. AC-23, no. 2, pp. 298-305, Apr. 1978. (Pubitemid 8595812)
13. S. Karlin and H. M. Taylor, A Second Course in Stochastic Processes. San Diego, CA: Academic, 1981.
14. R. McCann, A. K. Gunda, and S. D. Damugatla, "Improved operation of networked control systems using Lebesgue sampling," in Proc. Ind. Appl. Conf., 2004, vol. 2, pp. 1211-1216. (Pubitemid 40016654)
15. M. Miskowicz, "The event-triggered sampling optimization criterion for distributed networked monitoring and control systems," in Proc. IEEE Int. Conf. Ind. Technol.,Maribor, Slovenia, 2003, pp. 1083-1088. (Pubitemid 40761363)
16. M. Miskowicz and S. Kuta, "Application-driven flow control in distributed monitoring and control systems," in Proc. IEEE Int. Conf. Ind. Technol., Maribor, Slovenia, 2003, pp. 421-425. (Pubitemid 40761477)
17. C. De Persis, "N-bit stabilization of n-dimensional nonlinear systems in feedforward form," IEEE Trans. Autom. Control, vol. 30, no. 3, pp. 299-311, Mar. 2005. (Pubitemid 40448582)
18. N. Persson and F. Gustafsson, "Event based sampling with application to vibration analysis in pneumatic tires," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Salt Lake City,UT, 2001, pp. 3885-3888. (Pubitemid 32838469)
19. M. L. Puterman, Markov Decision Processes. New York: Wiley, 1994.
20. M. Rabi and J. S. Baras, "Sampling of diffusion processes for real-time estimation," in Proc. IEEE Conf. Decision Control, Atlantis, Bahamas, Dec. 2004, vol. 4, pp. 4163-4168. (Pubitemid 40287059)
21. S. Sastry, Nonlinear Systems: Analysis, Stability, and Control. New York: Springer-Verlag, 1999.
22. J. Si, A. G. Barto, W. B. Powell, and D. Wunsch, Handbook of Learning and Approximate Dynamic Programming. Hoboken, NJ: Wiley-IEEE Press, 2004.
23. H. Sira-Ramirez, "A geometric approach to pulse-width modulated control in nonlinear dynamical systems," IEEE Trans. Autom. Control, vol. 34, no. 2, pp. 184-187, Feb. 1989.
24. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
25. A.Wald, "Fitting of straight lines if both variables are subject to error," Ann. Math. Stat., vol. 11, pp. 284-300, 1940.
26. P. J.Werbos, "Approximate dynamic programming for real-time control and neural modeling," in Handbook of Intelligent Control, D. A.White and D. A. Sofge, Eds. NewYork:Van Nostrand Reinhold, 1992.
27. K. J. Zhang, Y. K. Xu, X. Chen, and X. R. Cao, "Policy iteration based feedback control," Automatica, vol. 44, no. 4, pp. 1055-1061, 2008.