Stochastic optimal controller design for uncertain nonlinear networked control system via neuro dynamic programming

IEEE Transactions on Neural Networks and Learning Systems

Volumn 24, Issue 3, 2013, Pages 471-484

  Xu, Hao ^a   Jagannathan, Sarangapani ^a  

^a Missouri University of Science and Technology   (United States)

Author keywords

Neuro dynamic programming; Nonlinear networked control system; Stochastic optimal control

Indexed keywords

AFFINE NONLINEAR SYSTEMS; NEURO DYNAMIC PROGRAMMING; NONLINEAR NETWORKED CONTROL SYSTEMS; NONLINEAR NETWORKED CONTROL SYSTEMS (NNCS); ON-LINE NEURAL NETWORKS; STOCHASTIC OPTIMAL CONTROL; STOCHASTIC OPTIMAL CONTROLLERS; UNIFORMLY ULTIMATELY BOUNDED;

DYNAMIC PROGRAMMING; NETWORKED CONTROL SYSTEMS; OPTIMIZATION; REAL TIME CONTROL; STOCHASTIC CONTROL SYSTEMS; STOCHASTIC SYSTEMS; UNCERTAINTY ANALYSIS;

CONTROLLERS;

ARTIFICIAL NEURAL NETWORK; NONLINEAR SYSTEM; STATISTICS;

NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; STOCHASTIC PROCESSES;

EID: 84884958993     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2012.2234133     Document Type: Article

References (31)

1. J. Nilsson, "Real-time control systems with delays," Ph.D. dissertation, Dept. Automatic Control, Lund Inst. Technology, Lund, Sweden, 1998.
2. G. C. Walsh, O. Beldiman, and L. Bushnell, "Error encoding algorithm for networked control system," Automatica, vol. 38, no. 2, pp. 261-267, 2002. (Pubitemid 33143495)
3. I. G. Polushin, P. X. Liu, and C. H. Lung, "On the model-based approach to nonlinear networked control systems," Automatica, vol. 44, no. 9, pp. 2409-2414, 2008.
4. L. L. Feng, J. Moyne, and D. Tilbury, "Optimal control design and evaluation for a class of networked control systems with distributed constant delays," in Proc. Amer. Control Conf., 2002, pp. 3009-3014.
5. M. Tabbara, "A linear quadratic gaussian framework for optimal networked control system design," in Proc. Amer. Control Conf., Jun. 2008, pp. 3804-3809.
6. A. K. Dehghani, "Optimal networked control system design: A dual-rate approach," in Proc. Can. Conf. ECE, May 2005, pp. 790-793.
7. H. Xu, S. Jagannathan, and F. L. Lewis, "Stochastic optimal control of unknown linear networked control system in presence of random delays and packet losses," Automatica, vol. 48, no. 6, pp. 1017-1030, Jun. 2012.
8. D. P. Bertsekas and J. Tsitsiklis, Neuro-Dynamics Programming. Belmont, MA: Athena Scientic, 1996.
9. P. J. Werbos, "A menu of designs for reinforcement learning over time," Journal Neural Network Control, MA: MIT Press, 1991.
10. D. V. Prokhorov and D. C. Wunsch II, "Adaptive critic designs," IEEE Trans. Neural Network, vol. 8, no. 5, pp. 997-1007, Sep. 1997.
11. J. Dankert, L. Yang, and J. Si, "A performance gradient perspective on approximate dynamic programming and its application to partially observable markov decision process," in Proc. Int. Symp. Intell. Control, 2006, pp. 458-463.
12. R. Enns, and J. Si, "Helicopter trimming and tracking control using direct neural dynamic programming," IEEE Trans. Neural Network, vol. 14, no. 4, pp. 929-939, Jul. 2003.
13. M. Fairbank, E. Alonso, and D. Prokhorov, "Simple and fast calculation of the second-order gradients for globalized dual heuristic dynamic programming in neural networks," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 10, pp. 1671-1676, Oct. 2012.
14. Q. Yang, Z. Yang, and Y. Sun, "Universal neural network control of MIMO uncertain nonlinear system," IEEE Trans. Neur. Netw. Learn. Syst., vol. 23, no. 7, pp. 1163-1169, Jul. 2012.
15. H. Zhang, Y. Luo, and D. Liu, "Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints," IEEE Trans. Neural Network, vol. 20, no. 9, pp. 1490-1503, Sep. 2009.
16. H. Zhang, Q. Wei, and Y. Luo, "A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm," IEEE Trans. Syst., Man, Cybern. B, vol. 38, no. 4, pp. 937-942, Aug. 2008.
17. F.L. Lewis and D. Vrabie, "Reinforcement learning and adaptive dynamic programming for feedback control," IEEE Circuits Syst. Mag., vol. 9, no. 3, pp. 32-50, Jul.-Sep. 2009.
18. C. Zheng and S. Jagannathan, "Generalized Hamilton-Jacobi-Bellman formulation-based neural network control of affine nonlinear discrete-time systems," IEEE Trans. Neural Netw., vol. 19, no. 1, pp. 90-106, Jan. 2008. (Pubitemid 351279228)
19. T. Dierks, and S. Jagannathan, "Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using time-based policy update," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1118-1129, Jul. 2012.
20. A. Al-Tamimi and F. L. Lewis, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof," IEEE Trans. Syst., Man., Cybern. B, vol. 38, no. 4, pp. 943-949, Aug. 2008.
21. M. G. Safonov and T. C. Tsao, "The unfalsified control concept and learning," IEEE Trans. Automat. Control, vol. 42, no. 6, pp. 843-847, Jun. 1997.
22. L. W. Liou and A. Ray, "A stochastic regulator for integrated communication and control systems: Part I - formulation of control law," ASME J. Dynamic Syst., Meas. Control, vol. 113, no. 4, pp. 604-611, 1991.
23. H. Shousong and Z. Qixin, "Stochastic optimal control and analysis of stability of networked control systems with long delay," Automatica, vol. 39, no. 11, pp. 1877-1884, Nov. 2003.
24. X. Liu and A. Goldsmith, "Wireless medium access control in networked control systems," in Proc. IEEE Amer. Contr Conf., 2004, pp. 688-694.
25. W. Stallings, Wireless Communications and Networks, 1st ed. Englewood Cliffs, NJ: Prentice-Hall, 2002.
26. ∞ performance bound: A Riccati equation approach," IEEE Trans. Automat. Control, vol. 34, no. 3, pp. 293-305, Mar. 1989.
27. S. Jagannathan, Neural network control of nonlinear discrete-time systems, Boca Raton, FL: CRC Press, 2006.
28. H. F. Chen and L. Guo, Identification and Stochastic Adaptive Control. Cambridge, MA: MIT Press, 1991.
29. P. R. Kumar, "A survey of some results in stochastic adaptive control," SIAM J. Control Optimatizat., vol. 23, no. 3, pp. 329-380, 1985.
30. B. A. Finlayson, The Method of Weighted Residuals and Variational Principles. New York: Academic, 1972.
31. D. P. Bertsekas, Dynamic Programming and Optimal Control, 3rd ed, Belmont, MA: Athena Scientific, 2007.