Robust adaptive dynamic programming and feedback stabilization of nonlinear systems

IEEE Transactions on Neural Networks and Learning Systems

Volumn 25, Issue 5, 2014, Pages 882-893

  Jiang, Yu ^a   Jiang, Zhong Ping ^a  

^a POLYTECHNIC UNIVERSITY   (United States)

Author keywords

Adaptive dynamic programming (ADP); nonlinear uncertain systems; robust optimal control.

Indexed keywords

BACKSTEPPING; CONTROL; JET ENGINES; NONLINEAR SYSTEMS;

ADAPTIVE DYNAMIC PROGRAMMING; BACKSTEPPING TECHNIQUE; FEEDBACK STABILIZATION; NONLINEAR CONTROL THEORY; NONLINEAR SMALL-GAIN; NONLINEAR UNCERTAIN SYSTEMS; ROBUST OPTIMAL CONTROL; UNCERTAIN NONLINEAR SYSTEMS;

DYNAMIC PROGRAMMING;

EID: 84899471403     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2013.2294968     Document Type: Article

References (55)

1. M. Abu-Khalaf and F. L. Lewis, "Neurodynamic programming and zerosum games for constrained control systems," IEEE Trans. Neural Netw., vol. 19, no. 7, pp. 1243-1252, Jul. 2008.
2. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control," Automatica, vol. 43, no. 3, pp. 473-481, Mar. 2007. (Pubitemid 46209050)
3. L. C. Baird, "Reinforcement learning in continuous time: Advantage updating," in Proc. IEEE Int. Conf. Neural Netw., World Congr. Comput. Intell., vol. 4. Jul. 1994, pp. 2448-2453.
4. A. G. Barto, R. S. Sutton, and C. W. Anderson, "Neuronlike adaptive elements that can solve difficult learning control problems," IEEE Trans. Syst., Man, Cybern., vol. 13, no. 5, pp. 835-846, Oct. 1983.
5. R. Beard, G. Saridis, and J. Wen, "Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation," Automatica, vol. 33, no. 12, pp. 2159-2177, Dec. 1997.
6. R. E. Bellman, Dynamic Programming. Princeton, NJ, USA: Princeton Univ. Press, 1957.
7. D. P. Bersekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA, USA: Athena Scientific, 1996.
8. K. Doya, "Reinforcement learning in continuous time and space," Neural Comput., vol. 12, no. 1, pp. 219-245, Jan. 2000.
9. R. Howard, Dynamic Programming and Markov Processes. Cambridge, MA, USA: MIT Press, 1960.
10. A. Isidori, Nonlinear Control Systems, vol. 2. New York, NY, USA: Springer-Verlag, 1999.
11. Y. Jiang and Z. P. Jiang, "Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics," Automatica, vol. 48, no. 10, pp. 2699-2704, Oct. 2012.
12. Y. Jiang and Z. P. Jiang, "Robust adaptive dynamic programming with an application to power systems," IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 7, pp. 1150-1156, Jul. 2013.
13. Y. Jiang and Z. P. Jiang, "Robust approximate dynamic programming and global stabilization with nonlinear dynamic uncertainties," in Proc. 50th IEEE CDC-ECC, Orlando, FL, USA, Dec. 2011, pp. 115-120.
14. Y. Jiang and Z. P. Jiang, "Robust adaptive dynamic programming," in Reinforcement Learning and Approximate Dynamic Programming for Feedback Control, F. L. Lewis and D. Liu, Eds. New York, NY, USA: Wiley, 2012.
15. Z. P. Jiang and Y. Jiang, "Robust adaptive dynamic programming for linear and nonlinear systems: An overview," Eur. J. Control, vol. 19, no. 5, pp. 417-425, Sep. 2013.
16. Z. P. Jiang, I. Mareels, and Y. Wang, "A Lyapunov formulation of the nonlinear small gain theorem for interconnected ISS systems," Automatica, vol. 32, no. 8, pp. 1211-1215, Aug. 1996. (Pubitemid 126363671)
17. Z. P. Jiang and I. M. Y. Mareels, "A small-gain control method for nonlinear cascaded systems with dynamic uncertainties," IEEE Trans. Autom. Control, vol. 42, no. 3, pp. 292-308, Mar. 1997. (Pubitemid 127760593)
18. Z. P. Jiang, A. R. Teel, and L. Praly, "Small-gain theorem for ISS systems and applications," Math. Control, Signals, Syst., vol. 7, no. 2, pp. 95-120, 1994.
19. I. Karafyllis and Z. P. Jiang, Stability and Stabilization of Nonlinear Systems. New York, NY, USA: Springer-Verlag, 2011.
20. H. K. Khalil, Nonlinear Systems, 3rd ed. Englewood Cliffs, NJ, USA: Prentice-Hall, 2002.
21. M. Krstic, I. Kanellakopoulos, and P. V. Kokotovic, Nonlinear and Adaptive Control Design. New York, NY, USA: Wiley, 1995.
22. P. Kundur, N. J. Balu, and M. G. Lauby, Power System Stability and Control. New York, NY, USA: McGraw-Hill, 1994.
23. F. L. Lewis, D. Vrabie, and V. L. Syrmos, Optimal Control, 3rd ed. New York, NY, USA: Wiley, 2012.
24. F. L. Lewis and D. Vrabie, "Reinforcement learning and adaptive dynamic programming for feedback control," IEEE Circuits Syst. Mag., vol. 9, no. 3, pp. 32-50, Apr./Jun. 2009.
25. F. L. Lewis and K. G. Vamvoudakis, "Reinforcement learning for partially observable dynamic processes: Adaptive dynamic programming using measured output data," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 1, pp. 14-23, Feb. 2011.
26. F. L. Lewis and D. Liu, Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. New York, NY, USA: Wiley, 2012.
27. R. Marino and P. Tomei, Nonlinear Control Design: Geometric, Adaptive, Robust. New York, NY, USA: Wiley, 1995.
28. J. M. Mendel and R. W. McLaren, "Reinforcement learning control and pattern recognition systems," in Adaptive, Learning and Pattern Recognition Systems: Theory and Applications, J. M. Mendel and K. S. Fu, Eds. New York, NY, USA: Academic, 1970, pp. 287-318.
29. M. Minsky, "Steps toward artificial intelligence," Proc. IRE, vol. 49, no. 1, pp. 8-30, Jan. 1961.
30. F. K. Moore and E. M. Greitzer, "A theory of post-stall transients in axial compression systems-Part I: Development of equations," J. Eng. Gas Turbines Power, vol. 108, no. 1, pp. 68-76, Jan. 1986. (Pubitemid 16475797)
31. J. J. Murray, C. J. Cox, and G. G. Lendaris, "Adaptive dynamic programming," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 32, no. 2, pp. 140-153, May 2002. (Pubitemid 35289398)
32. M. J. D. Powell, Approximation Theory and Methods. Cambridge, U.K.: Cambridge Univ. Press, 1981.
33. L. Praly and Y. Wang, "Stabilization in spite of matched unmodeled dynamics and an equivalent definition of input-to-state stability," Math. Control, Signals, Syst., vol. 9, no. 1, pp. 1-33, 1996. (Pubitemid 126597701)
34. G. N. Saridis and C.-S. G. Lee, "An approximation theory of optimal control for trainable manipulators," IEEE Trans. Syst., Man, Cybern., vol. 9, no. 3, pp. 152-159, Mar. 1979.
35. E. D. Sontag, "Smooth stabilization implies coprime factorization," IEEE Trans. Autom. Control, vol. 34, no. 4, pp. 435-443, Apr. 1989.
36. E. D. Sontag, "Further facts about input to state stabilization," IEEE Trans. Autom. Control, vol. 35, no. 4, pp. 473-476, Apr. 1990.
37. E. D. Sontag and Y. Wang, "On characterizations of the input-to-state stability property," Syst. Control Lett., vol. 24, no. 5, pp. 351-359, Apr. 1995.
38. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA, USA: MIT Press, 1998.
39. R. S. Sutton, "Learning to predict by the method of temporal difference," Mach. Learn., vol. 3, no. 1, pp. 9-44, Aug. 1988.
40. A. Teel and L. Praly, "Tools for semiglobal stabilization by partial state and output feedback," SIAM J. Control Optim., vol. 33, no. 5, pp. 1443-1488, Sep. 1995.
41. J. Tsinias, "Partial-state global stabilization for general triangular systems," Syst. Control Lett., vol. 24, no. 2, pp. 139-145, Jan. 1995.
42. K. G. Vamvoudakis and F. L. Lewis, "Multi-player non zero sum games: Online adaptive learning solution of coupled Hamilton-Jacobi equations," Automatica, vol. 47, no. 8, pp. 1556-1569, Aug. 2011.
43. D. Vrabie and F. Lewis, "Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems," Neural Netw., vol. 22, no. 3, pp. 237-246, Apr. 2009.
44. D. Vrabie, O. Pastravanu, M. Abu-Khalaf, and F. L. Lewis, "Adaptive optimal control for continuous-time linear systems based on policy iteration," Automatica, vol. 45, no. 2, pp. 477-484, Feb. 2009.
45. M. D. Waltz and K. S. Fu, "A heuristic approach to reinforcement learning control systems," IEEE Trans. Autom. Control, vol. 10, no. 4, pp. 390-398, Oct. 1965.
46. F. Y. Wang, H. Zhang, and D. Liu, "Adaptive dynamic programming: An introduction," IEEE Comput. Intell. Mag., vol. 4, no. 2, pp. 39-47, May 2009.
47. C. Watkins, "Learning from delayed rewards," Ph.D. thesis, King's College, Cambridge Univ., Cambridge, U.K., May 1989.
48. P. J. Werbos, The Elements of Intelligence. Namur, Belgium: Cybernetica, 1968, no. 3.
49. P. J. Werbos, "Beyond regression: New tools for prediction and analysis in the behavioral sciences," Ph.D. thesis, Committee Appl. Math., Harvard Univ., Cambridge, MA, USA, 1974.
50. P. J. Werbos, "Neural networks for control and system identification," in Proc. 28th IEEE Conf. Decision Control, vol. 1. Dec. 1989, pp. 260-265.
51. P. J. Werbos, "A menu of designs for reinforcement learning over time," in Neural Networks for Control, W. T. Miller, R. S. Sutton, and P. J. Werbos, Eds. Cambridge, MA, USA: MIT Press, 1991, pp. 67-95.
52. P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling," in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, D. A. White and D. A. Sofge, Eds. New York, NY, USA: Van Nostrand, 1992.
53. P. J. Werbos, "Intelligence in the brain: A theory of how it works and how to build it," Neural Netw., vol. 22, no. 3, pp. 200-212, Apr. 2009.
54. H. Zhang, Q. Wei, and D. Liu, "An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games," Automatica, vol. 47, no. 1, pp. 207-214, Jan. 2011.
55. Y. Zhang, P. Y. Peng, and Z. P. Jiang, "Stable neural controller design for unknown nonlinear systems using backstepping," IEEE Trans. Neural Netw., vol. 11, no. 6, pp. 1347-1360, Nov. 2000.