Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems

IEEE Transactions on Automatic Control

Volumn 61, Issue 12, 2016, Pages 4164-4169

  Gao, Weinan ^a   Jiang, Zhong Ping ^a  

^a POLYTECHNIC UNIVERSITY   (United States)

Author keywords

Adaptive control; approximate adaptive dynamic programming (ADP); optimal control; output regulation

Indexed keywords

ADAPTIVE CONTROL SYSTEMS; CLOSED LOOP SYSTEMS; CONTINUOUS TIME SYSTEMS; CONTROLLERS; DISTURBANCE REJECTION; LINEAR SYSTEMS; REINFORCEMENT LEARNING;

ADAPTIVE CONTROL; ADAPTIVE DYNAMIC PROGRAMMING; APPROXIMATE/ADAPTIVE DYNAMIC PROGRAMMING (ADP); CONTINUOUS-TIME LINEAR SYSTEMS; DISTRIBUTED GENERATION SYSTEM; OPTIMAL CONTROLS; OUTPUT REGULATION; OUTPUT REGULATION PROBLEM;

DYNAMIC PROGRAMMING;

EID: 85005983375     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2016.2548662     Document Type: Article

References (46)

1. K. H. Ahmed, A.M. Massoud, S. J. Finney, and B.W. Williams, "A modified stationary reference frame-based predictive current control with zero steady-state error for LCL coupled inverter-based distributed generation systems," IEEE Trans. Ind. Electron., vol. 58, no. 4, pp. 1359-1370, 2011.
2. T.-B. Airimitoaie, A. Castellanos-Silva, and I. Landau, "Indirect adaptive regulation strategy for the attenuation of time varying narrow-band disturbances applied to a benchmark problem," Eur. J. Control, vol. 19, no. 4, pp. 313-325, 2013.
3. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control," Automatica, vol. 43, no. 3, pp. 473-481, 2007.
4. R. Bellman, Dynamic Programming. Princeton, NJ, USA: Princeton University Press, 1957.
5. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA, USA: Athena Scientific, 1996.
6. S. Bhasin, N. Sharma, P. Parte, and W. E. Dixon, "Asymptotic tracking by a reinforcement learning-based adaptive critic controller," J. Control Theroy Appl., vol. 9, no. 3, pp. 400-409, 2011.
7. T. Bian, Y. Jiang, and Z. P. Jiang, "Adaptive dynamic programming and optimal control of nonlinear nonaffine systems," Automatica, vol. 50, no. 10, pp. 2624-2632, 2014.
8. C. Bonivento, L. Marconi, and R. Zanasi, "Output regulation of nonlinear systems by sliding mode," Automatica, vol. 37, no. 4, pp. 535-542, 2001.
9. S. Boyd and L. Vandenberghe, Convex Optimization. New York, NY, USA: Cambridge University Press, 2004.
10. S. J. Bradtke, B. E. Ydstie, and A. G. Barto, "Adaptive linear quadratic control using policy iteration," in Pro. Amer. Control Conf., Baltimore, MD, USA, 1994, pp. 3475-3479.
11. Z. Ding, "Output regulation of uncertain nonlinear systems with nonlinear exosystems," IEEE Trans. Autom. Control, vol. 51, no. 3, pp. 498-503, Mar. 2006.
12. S. Ferrari and R. F. Stengel, "An adaptive critic global controller," in Proc. Amer. Control Conf., Anchorage, AK, USA, 2002, pp. 2665-2670.
13. B. Francis, "The linear multivariable regulator problem," SIAM J. Control Optimiz., vol. 15, no. 3, pp. 486-505, 1977.
14. W. Gao and Z. P. Jiang, "Global optimal output regulation of partially linear systems via robust adaptive dynamic programming," in Proc. 1st Conf. Modelling Identification Control Nonlinear Syst., Saint Petersburg, Russia, 2015, vol. 48, no. 11, pp. 742-747.
15. J. Huang, Nonlinear Output Regulation: Theory and Applications. Philadelphia, PA, USA: SIAM, 2004.
16. Y. Huang and D. Liu, "Neural-network-based optimal tracking control scheme for a class of unknown discrete-time nonlinear systems using iterative ADP algorithm," Neurocomputing, vol. 125, pp. 46-56, 2014.
17. P. A. Ioannou and B. Fidan, Adaptive Control Tutorial. Philadelphia, PA, USA: SIAM, 2006.
18. A. Isidori, L. Marconi, and A. Serrani, Robust Autonomous Guidance: An Internal Model Approach. London, U.K.: Springer-Verlag, 2003.
19. Y. Jiang and Z. P. Jiang, "Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics," Automatica, vol. 48, no. 10, pp. 2699-2704, 2012.
20. Y. Jiang and Z. P. Jiang, "Adaptive dynamic programming as a theory of sensorimotor control," Biol. Cybern., vol. 108, no. 4, pp. 459-473, 2014.
21. Y. Jiang and Z. P. Jiang, "Robust adaptive dynamic programming and feedback stabilization of nonlinear systems," IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 5, pp. 882-893, 2014.
22. Y. Jiang and Z. P. Jiang, "Global adaptive dynamic programming for continuous-time nonlinear systems," IEEE Trans. Autom. Control, vol. 60, no. 11, pp. 2917-2929, 2015.
23. Y. Jiang and Z. P. Jiang, "A robust adaptive dynamic programming principle for sensorimotor control with signal-dependent noise," J. Syst. Sci. Complex., vol. 28, no. 2, pp. 261-288, 2015.
24. Z. P. Jiang and Y. Jiang, "Robust adaptive dynamic programming for linear and nonlinear systems: An overview," Eur. J. Control, vol. 19, no. 5, pp. 417-425, 2013.
25. R. Kamalapurkar, H. Dinh, S. Bhasin, and W. E. Dixon, "Approximate optimal trajectory tracking for continuous-time nonlinear systems," Automatica, vol. 51, pp. 40-48, 2015.
26. R. Kamalapurkar, P. Walters, and W. E. Dixon, "Model-based reinforcement learning for approximate optimal regulation," Automatica, vol. 64, pp. 94-104, 2016.
27. D. Kleinman, "On an iterative technique for Riccati equation computations," IEEE Trans. Autom. Control, vol. AC-13, no. 1, pp. 114-115, 1968.
28. A. J. Krener, "The construction of optimal linear and nonlinear regulators," inSystems, Models and Feedback: Theory and Applications, A. Isidori, and T. J. Tarn, Eds. Boston, MA, USA: Birkhauser, 1992, vol. 12, pp. 301-322.
29. M. Krstic, I. Kanellakopoulos, and P. V. Kokotovic, Nonlinear and Adaptive Control Design. New York, NY, USA: Wiley, 1995.
30. F. L. Lewis and D. Liu, Eds., Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. Hoboken, NJ, USA: Wiley, 2013.
31. F. L. Lewis and D. Vrabie, "Reinforcement learning and adaptive dynamic programming for feedback control," IEEE Circuits Syst. Mag., vol. 9, no. 3, pp. 32-50, 2009.
32. F. L. Lewis, D. Vrabie, and V. L. Syrmos, Optimal Control. Hoboken, NJ, USA:Wiley, 2012.
33. F. L. Lewis, D. Vrabie, and K. G. Vamvoudakis, "Reinforcement learning and feedback control: Using natural decision methods to design optimal adaptive controllers," IEEE Control Syst. Mag., vol. 32, no. 6, pp. 76-105, 2012.
34. T. Liu, Z. P. Jiang, and D. J. Hill, Nonlinear Control of Dynamic Networks. Boca Raton, FL, USA: CRC Press, 2014.
35. X. Liu and S. N. Balakrishnan, "Convergence analysis of adaptive critic based optimal control," in Proc. American Control Conf., Chicago, IL, USA, 2000, pp. 1929-1933.
36. P. Mehta and S. Meyn, "Q-learning and Pontryagin's minimum principle," in Proc. Joint 48th IEEE Conf. Decision Control 28th Chinese Control Conf., 2009, pp. 3598-3605.
37. H. Modares and F. L. Lewis, "Linear quadratic tracking control of partially-unknown continuous-time systems using reinforcement learning," IEEE Trans. Autom. Control, vol. 59, no. 11, pp. 3051-3056, Nov. 2014.
38. W. B. Powell, Approximate Dynamic Programming: Solving the Curse of Dimensionality. New York, NY, USA: Wiley, 2007.
39. C. Qin, H. Zhang, and Y. Luo, "Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programming," Int. J. Control, vol. 87, no. 5, pp. 1000-1009, 2014.
40. A. Saberi, A. Stoorvogel, P. Sannuti, and G. Shi, "On optimal output regulation for linear systems," Int. J. Control, vol. 76, no. 4, pp. 319-333, 2003.
41. E. D. Sontag, "Adaptation and regulation with signal detection implies internal model," Syst. Control Lett., vol. 50, no. 2, pp. 119-126, 2003.
42. R. S. Sutton and A. G. Barto, Introduction to Reinforcement Learning. Cambridge, MA, USA: MIT Press, 1998.
43. H. Trentelman, A. Stoorvogel, and M. Hautus, Control Theory for Linear Systems. London, U.K.: Springer-Verlag, 2002.
44. P. J. Werbos, "Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences," Ph.D. dissertation, Harvard Univ., Cambridge, PA, 1974.
45. H. Zhang, L. Cui, X. Zhang, and Y. Luo, "Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method," IEEE Trans. Neur. Netw., vol. 22, no. 12, pp. 2226-2236, Dec. 2011.
46. H. Zhang, Q. Wei, and Y. Luo, "A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 937-942, 2008.