Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems

IEEE Transactions on Automation Science and Engineering

Volumn 11, Issue 2, 2014, Pages 627-632

  Wang, Ding ^a   Liu, Derong ^a   Li, Hongliang ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

Adaptive dynamic programming; neural networks; optimal control; policy iteration; robust control; uncertain nonlinear systems

Indexed keywords

ALGORITHMS; CONTINUOUS TIME SYSTEMS; CONTROL; ITERATIVE METHODS; NEURAL NETWORKS; NONLINEAR SYSTEMS; OPTIMAL CONTROL SYSTEMS;

ADAPTIVE DYNAMIC PROGRAMMING; CONTINUOUS TIME NONLINEAR SYSTEMS; HAMILTON-JACOBI-BELLMAN EQUATIONS; OPTIMAL CONTROLS; POLICY ITERATION; POLICY ITERATION ALGORITHMS; UNCERTAIN NONLINEAR SYSTEMS; UNIFORM ULTIMATE BOUNDEDNESS;

ROBUST CONTROL;

EID: 84898803345     PISSN: 15455955     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASE.2013.2296206     Document Type: Article

References (28)

1. B. R. Barmish and Z. Shi, "Robust stability of a class of polynomialswith coefficients depending multilinearly on perturbations," IEEETrans. Automat. Control, vol. 35, no. 9, pp. 1040-1043, Sep. 1990.
2. C. Kravaris and S. Palanki, "A Lyapunov approach for robust nonlinearstate feedback synthesis," IEEE Trans. Automat. Control, vol. 33, no.12, pp. 1188-1191, Dec. 1988.
3. F. Lin, R. D. Brand, and J. Sun, "Robust control of nonlinear systems:Compensating for uncertainty," Int. J. Control, vol. 56, no. 6,pp. 1453-1459, 1992.
4. P. J. Werbos, "Approximate dynamic programming for real-time controland neural modeling," in Handbook of Intelligent Control: Neural,Fuzzy, Adaptive Approaches, D. A.White andD. A. Sofge, Eds. NewYork, NY, USA: Van Nostrand Reinhold, 1992, ch. 13.
5. P. J. Werbos, "Intelligence in the brain: A theory of how it works andhow to build it," Neural Netw., vol. 22, no. 3, pp. 200-212, Apr. 2009.
6. Handbook of Learning and Approximate Dynamic Programming, J. Si,A. G. Barto,W. B. Powell, and D. C. Wunsch, Eds. New York, NY,USA: Wiley, 2004.
7. D. V. Prokhorov and D. C. Wunsch, "Adaptive critic designs," IEEETrans. Neural Netw., vol. 8, no. 5, pp. 997-1007, Sep. 1997.
8. Q. Kang, M. Zhou, J. An, and Q.Wu, "Swarm intelligence approachesto optimal power flow problem with distributed generator failures inpower networks," IEEE Trans. Autom. Sci. Eng., vol. 10, no. 2, pp.343-353, Apr. 2013.
9. H. He, Z. Ni, and J. Fu, "A three-network architecture for on-linelearning and optimization based on adaptive dynamic programming, "Neurocomputing, vol. 78, no. 1, pp. 3-13, Feb. 2012.
10. J. Fu, H. He, and X. Zhou, "Adaptive learning and control for MIMOsystem based on adaptive dynamic programming," IEEE Trans. NeuralNetw., vol. 22, no. 7, pp. 1133-1148, Jul. 2011.
11. H. N. Wu and B. Luo, "Neural network based online simultaneouspolicy update algorithm for solving the HJI equation in nonlinearcontrol," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 12, pp.1884-1895, Dec. 2012.
12. H. Zhang, L. Cui, and Y. Luo, "Near-optimal control for nonzero-sumdifferential games of continuous-time nonlinear systems using singlenetworkADP," IEEE Trans. Cybern., vol. 43, no. 1, pp. 206-216, Feb.2013.
13. F. L. Lewis and D. Vrabie, "Reinforcement learning and adaptive dynamicprogramming for feedback control," IEEE Circuits Syst. Mag.,vol. 9, no. 3, pp. 32-50, Jul. 2009.
14. W. S. Lin and J.W. Sheu, "Optimization of train regulation and energyusage of metro lines using an adaptive-optimal-control algorithm,"IEEE Trans. Autom. Sci. Eng., vol. 8, no. 4, pp. 855-864, Oct. 2011.
15. T. Dierks and S. Jagannathan, "Online optimal control of affine nonlineardiscrete-time systems with unknown internal dynamics by usingtime-based policy update," IEEE Trans. Neural Netw. Learn. Syst., vol.23, no. 7, pp. 1118-1129, Jul. 2012.
16. D. Wang, D. Liu, and Q. Wei, "Finite-horizon neuro-optimal trackingcontrol for a class of discrete-time nonlinear systems using adaptivedynamic programming approach," Neurocomputing, vol. 78, no. 1, pp.14-22, Feb. 2012.
17. D. Liu, D.Wang, D. Zhao, Q.Wei, and N. Jin, "Neural-network- basedoptimal control for a class of unknown discrete-time nonlinear systemsusing globalized dual heuristic programming," IEEE Trans. Autom. Sci.Eng., vol. 9, no. 3, pp. 628-634, Jul. 2012.
18. D. Wang, D. Liu, Q. Wei, D. Zhao, and N. Jin, "Optimal control ofunknown nonaffine nonlinear discrete-time systems based on adaptivedynamic programming," Automatica, vol. 48, no. 8, pp. 1825-1832,Aug. 2012.
19. D. Wang and D. Liu, "Neuro-optimal control for a class of unknownnonlinear dynamic systems using SN-DHP technique," Neurocomputing,vol. 121, pp. 218-225, Dec. 2013.
20. S. K. Pradhan and B. Subudhi, "Real-time adaptive control of a flexiblemanipulator using reinforcement learning," IEEE Trans. Autom. Sci.Eng., vol. 9, no. 2, pp. 237-249, Apr. 2012.
21. Q. Wei and D. Liu, "A novel iterative-adaptive dynamic programmingfor discrete-time nonlinear systems," IEEE Trans. Autom. Sci.Eng., 10.1109/TASE.2013.2280974.
22. M. Abu-Khalaf and F. L. Lewis, "Nearly optimal control laws for nonlinearsystems with saturating actuators using a neural network HJBapproach," Automatica, vol. 41, no. 5, pp. 779-791, May 2005.
23. K. G. Vamvoudakis and F. L. Lewis, "Online actor-critic algorithm tosolve the continuous-time infinite horizon optimal control problem,"Automatica, vol. 46, no. 5, pp. 878-888, May 2010.
24. S. Bhasin, M. Johnson, and W. E. Dixon, "A model-free robust policyiteration algorithm for optimal control of nonlinear systems," in Proc.49th IEEE Conf. Decision Control, Atlanta, GA, USA, Dec. 2010, pp.3060-3065.
25. H. Modares, F. L. Lewis, and M.-B. Naghibi-Sistani, "Adaptive optimalcontrol of unknown constrained-input systems using policy iterationand neural networks," IEEE Trans. Neural Netw. Learn. Syst.,vol. 24, no. 10, pp. 1513-1525, Oct. 2013.
26. D. M. Adhyaru, I. N. Kar, and M. Gopal, "Bounded robust control ofnonlinear systems using neural network-based HJB solution," NeuralComput. Appl., vol. 20, no. 1, pp. 91-103, 2011.
27. R. S. Sutton and A. G. Barto, Reinforcement Learning-An Introduction.Cambridge, MA, USA: MIT Press, 1998.
28. V. Nevistic and J. A. Primbs, "Constrained nonlinear optimal control:A converse HJB approach," California Inst. Techn., Pasadena, CA,USA, Tech. Memo. No. CIT-CDS 96-021, Dec. 1996.