Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses

Automatica

Volumn 48, Issue 6, 2012, Pages 1017-1030

  Xu, Hao ^a   Jagannathan, S ^a   Lewis, F L ^b  

^a University of Missouri Rolla   (United States)

^b University of Texas at Arlington   (United States)

Author keywords

Adaptive estimator; Networked control system (NCS); Optimal control; Q function

Indexed keywords

ADAPTIVE ESTIMATOR; ASYMPTOTICALLY STABLE; CERTAINTY EQUIVALENCE; CONTROL SIGNAL; INFINITE HORIZONS; LINEAR TIME-INVARIANT; LYAPUNOV THEORIES; NETWORKED CONTROL SYSTEM (NCS); ONLINE TUNING; OPTIMAL CONTROL SCHEME; OPTIMAL CONTROLS; Q-FUNCTIONS; Q-LEARNING; RANDOM DELAY; STOCHASTIC OPTIMAL CONTROL; SUBOPTIMAL CONTROL; SYSTEM MATRICES; TIME VARYING; UNKNOWN PARAMETERS;

NETWORKED CONTROL SYSTEMS; PACKET LOSS; RICCATI EQUATIONS; STOCHASTIC CONTROL SYSTEMS;

LINEAR CONTROL SYSTEMS;

EID: 84862815087     PISSN: 00051098     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.automatica.2012.03.007     Document Type: Article

References (39)

1. A. Al-Tamimi, F.L. Lewis, and M. Abu-Khalaf Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control Automatica 43 2007 473 481 (Pubitemid 46209050)
2. P. Antsaklis, and J. Baillieul Special issue on networked control systems IEEE Transactions on Automatic Control 49 2004 1421 1423
3. K.J. strom Introduction to stochastic control theory 1970 Academic Press New York
4. Azimi-Sadjadi, B. (2003) Stability of networked control systems in the presence of packet losses. In Proceedings of conference on decision and control (pp. 676-681).
5. A.G. Barto, R.S. Sutton, and C.W. Anderson Neuron like elements that can solve difficult learning control problems IEEE Transactions on Systems, Man and Cybernetics 13 1983 835 846
6. D.P. Bertsekas, and S.E. Shreve Stochastic optimal control: the discrete time case 1978 Academic Press New York
7. Branicky, M.S., Phillips, S.M., & Zhang, W. (2000) Stability of networked control systems: explicit analysis of delay. In Proceedings of american control conference, (pp. 2352-2357). (Pubitemid 32028075)
8. L. Busoniu, R. Babuska, and B.D. Schutter Reinforcement learning and dynamic programming using function approximators 2010 CRC Press New York
9. D. Carnevale, A.R. Teel, and D. Nesic A Lyapnov proof of improved maximum allowable transfer interval for networked control systems IEEE Transactions on Automatic Control 52 2007 892 897 (Pubitemid 46796076)
10. M.B.G Cloosterman, N. van de Wouw, W.P.M.H. Heemels, and H. Nijmeijer Stability of networked control systems with uncertain time-varying delays IEEE Transactions on Automatic Control 54 2009 1575 1580
11. Dierks, T., & Jagannathan, S. (2009) Optimal control of affine nonlinear discrete-time systems with unknown internal dynamics. In Proceedings of conference on decision and control, (pp. 6750-6755).
12. Dierks, T., Thumati, T.B., & Jagannathan, S. (2009) Adaptive dynamics programming-based optimal control of unknown affine nonlinear discrete-time systems In Proc. Intern. Joint Conf. Neur. Netwo. Vol. 711-716.
13. G.F. Franklin, J.D. Powell, and A. Emani-Naeini Feedback control of dynamic systems 3rd ed 1994 Addison-Wesley Reading, Massachusetts
14. A. Goldsmith Wireless communications 2003 Cambridge University Press Cambridge, UK
15. M. Green, and J.B. Moore Persistency of excitation in linear systems Systems and Control Letters 7 1986 351 360
16. L. Guo Estimating time-varying parameters by the Kalman filter based algorithm: stability and convergence IEEE Transactions on Automatic Control 35 1990 141 147 (Pubitemid 20666486)
17. Y. Halevi, and A. Ray Integrated communication and control systems: part I - analysis Journal of Dynamic Systems Measurement and Control 110 1988 367 373
18. W.P.M.H. Heemels, A.R. Teel, N. van de Wouw, and D. Nesic Networked control systems with communication constraints: tradeoff between transmission intervals, delays and performance IEEE Transactions on Automatic Control 55 2010 1781 1796
19. Hesphanha, J.P., Naghshtabrizi, P., & Xu, Y. (2007) A survey of recent results in networked control system. In Proceedings of IEEE. 95. (pp. 138-162).
20. S.S. Hu, and Q.X. Zhu Stochastic optimal control and analysis of stability of networked control systems with long delay Automatica 39 2003 1877 1884
21. S. Jagannathan Neural network control of nonlinear discrete-time systems 2006 CRC Press
22. F.L. Lewis, and V.L. Syrmos Optimal control 2nd ed 1995 Wiley New York
23. F. Lian, J. Moyne, and D. Tilbury Modeling and optimal controller design of networked control systems with multiple delays International Journal of Control 76 2003 591 606
24. L.W. Liou, and A. Ray A stochastic regulator for integrated communication and control systems: part I - formulation of control law ASME Journal of Dynamic Systems Measurement and Control 4 1991 604 611
25. P.S. Maybeck Stochastic models, estimation and control 1982 Academic press New York
26. R.H. Middleton, and G.C. Goodwin Adaptive control of time-varying linear systems IEEE Transactions on Automatic Control 33 1988 150 155
27. J. Nilsson, B. Bernhardsson, and B. Wittenmark Stochastic analysis and control of real-time systems with random time delays Automatica 34 1998 57 64 (Pubitemid 128387949)
28. A. Papoulis Probability, random variables, and stochastic processes (3rd ed.) 1991 McGraw-Hill New York
29. S. Sastry, and M. Bodson Adaptive control: stability, convergence, and robustness 1989 Prentice Hall Englewood Cliffs, NJ
30. Schenato, L., Sinopoli, B., Franceschetti, M., Poolla, K., & Sastry, S. (2007). Foundations of control and estimation over lossy networks. In Proceedings of IEEE. 95. (pp. 163-187).
31. R.F. Stengel Stochastic optimal control, theory and application 1986 John Wiley and Sons New York
32. Walsh, G.C., Ye, H., & Bushnell, L. (1999) Stability analysis of networked control systems. In Proceedings of american control conference (pp. 2876-2880). (Pubitemid 32079591)
33. Watkins, C. (1989). Learning from delayed rewards, Ph.D. Thesis, Cambridge, England: Cambridge University.
34. P.J. Werbos A menu of designs for reinforcement learning over time 1991 MIT Press MA
35. P.J. Werbos Approximate dynamic programming for real-time control and neural modeling Handbook of intelligent control 1992 Van Nostrand Reinhold New York
36. W.M. Wonham On a matrix Riccati equation of stochastic control SIAM Journal of Control 6 1968 681 697
37. J. Wu, and T. Chen Design of networked control systems with packet dropouts IEEE Transactions on Automatic Control 52 2007 1314 1319 (Pubitemid 47098945)
38. W. Zhang, M.S. Branicky, and S. Phillips Stability of networked control systems IEEE Control Systems Magazine 21 2001 84 99 (Pubitemid 32869524)
39. H. Zhang, Y. Luo, and D. Liu Neural network based near optimal control for a class of discrete-time affine nonlinear system with control constraints IEEE Transactions on Neural Network 20 2009 1490 1503