Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using time-based policy update

IEEE Transactions on Neural Networks and Learning Systems

Volumn 23, Issue 7, 2012, Pages 1118-1129

  Dierks, Travis ^a   Jagannathan, Sarangapani ^b  

^a DRS Sustainment Systems Inc   (United States)

^b Architectural and Environmental Engineering   (United States)

Author keywords

Hamilton Jacobi Bellman; online approximators; online nonlinear optimal control; time based policy update

Indexed keywords

ADAPTIVE DYNAMIC PROGRAMMING; APPROXIMATORS; HAMILTON JACOBI BELLMAN EQUATION; HAMILTON-JACOBI-BELLMAN; NONLINEAR DISCRETE-TIME SYSTEMS; ONLINE NONLINEAR OPTIMAL CONTROLS; TIME-BASED POLICY UPDATE; UNIFORMLY ULTIMATELY BOUNDED;

CONTROL; COST FUNCTIONS; DIGITAL CONTROL SYSTEMS; DISCRETE TIME CONTROL SYSTEMS; DYNAMIC PROGRAMMING; HARDWARE;

OPTIMAL CONTROL SYSTEMS;

EID: 84875270081     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2012.2196708     Document Type: Article

References (30)

1. S. Jagannathan, Neural Network Control of Nonlinear Discrete-Time Systems. Boca Raton, FL: CRC Press, 2006.
2. A. U. Levin and K. S. Narendra, "Control of nonlinear dynamical systems using neural networks: Controllability and stabilization," IEEE Trans. Neural Netw., vol. 4, no. 2, pp. 192-206, Mar. 1993.
3. F. L. Lewis and V. L. Syrmos, Optimal Control, 2nd ed. Hoboken, NJ: Wiley, 1995.
4. J. Shamma and J. Cloutier, "Existence of SDRE stabilizing feedback," IEEE Trans. Autom. Control, vol. 48, no. 3, pp. 513-517, Mar. 2003.
5. S. Keerthi and E. Gilbert, "An existence theorem for discrete-time infinite-horizon optimal control problems," IEEE Trans. Autom. Control, vol. 30, no. 9, pp. 907-909, Sep. 1985.
6. T. Parisini and R. Zoppoli, "Neural approximations for infinite-horizon optimal control of nonlinear stochastic systems," IEEE Trans. Neural Netw., vol. 9, no. 6, pp. 1388-1408, Nov. 1998.
7. D. P. Bertsekas, Dynamic Programming and Optimal Control, 2nd ed. Belmonth, MA: Athena Scientific, 2000.
8. Z. Chen and S. Jagannathan, "Generalized Hamilton-Jacobi-Bellman formulation based neural network control of affine nonlinear discretetime systems," IEEE Trans. Neural Netw., vol. 10, no. 1, pp. 90-106, Jan. 2008.
9. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof," IEEE Trans. Syst., Man, Cybern., B, vol. 38, no. 4, pp. 943-949, Aug. 2008.
10. H. Zhang, Y. Luo, and D. Liu, "Neural network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints," IEEE Trans. Neural Netw., vol. 20, no. 9, pp. 1490-1503, Sep. 2009.
11. X. Xu, D. Hu, and X. Lu, "Kernel-based least squares policy iteration for reinforcement learning," IEEE Trans. Neural Netw., vol. 18, no. 4, pp. 973-992, Jul. 2007.
12. Y. Tassa and T. Erez, "Least squares solutions of the HJB equation with neural network value-function approximators," IEEE Trans. Neural Netw., vol. 18, no. 4, pp. 1031-1041, Jul. 2007.
13. T. Cheng, F. L. Lewis, and M. Abu-Khalaf, "Fixed-final-time- constrained optimal control of nonlinear systems using neural network HJB approach," IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1725-1736, Nov. 2007.
14. Y. Becerikli, Y. Oysal, and A. F. Konar, "Trajectory priming with dynamic fuzzy networks in nonlinear optimal control," IEEE Trans. Neural Netw., vol. 15, no. 2, pp. 383-394, Mar. 2004.
15. J. Si, A. G. Barto,W. B. Powell, and D. Wunsch, Handbook of Learning and Approximate Dynamics Programming. New York: Wiley, 2004.
16. T. Hanselmann, L. Noakes, and A. Zaknich, "Continuous-time adaptive critics," IEEE Trans. Neural Netw., vol. 18, no. 3, pp. 631-647, May 2007.
17. D. V. Prokhorov and D. Wunsch, "Adaptive critic designs," IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 997-1007, Sep. 1997.
18. J. Si and Y. T. Wang, "Online learning control by association and reinforcement," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 264-276, Mar. 2001.
19. B. Li and J. Si, "Approximate robust policy iteration using multilayer perceptron neural networks for discounted infinite-horizon Markov decision processes with uncertain correlate transition matrices," IEEE Trans. Neural Netw., vol. 21, no. 8, pp. 1270-1280, Aug. 2010.
20. D. Vrabie, O. Pastravanu, M. Abu-Khalaf, and F. L. Lewis, "Adaptive optimal control for continuous-time linear systems based on policy iteration," Automatica, vol. 45, no. 2, pp. 477-484, 2009.
21. K. G. Vamvoudakis and F. L. Lewis, "Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem," Automatica, vol. 46, no. 5, pp. 878-888, 2010.
22. K. Hornik, M. Stinchcombe, and H. White, "Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks," Neural Netw., vol. 3, no. 5, pp. 551-560, 1990.
23. A. S. Dutka, A. W. Ordys, and M. J. Grimble, "Optimized discretetime state dependent Riccati equation regulator," in Proc. Amer. Control Conf., vol. 4. 2005, pp. 2293-2298.
24. D. Swaroop, J. K. Hedrick, P. P. Yip, and J. C. Gerdes, "Dynamic surface control for a class of nonlinear systems," IEEE Trans. Autom. Control, vol. 45, no. 10, pp. 1893-1899, Oct. 2000.
25. J. W. Brewer, "Kronecker products and matrix calculus in system theory," IEEE Trans. Circuits Syst., vol. 25, no. 9, pp. 772-781, Sep. 1978.
26. T. Dierks and S. Jagannathan, "Optimal control of affine nonlinear discrete-time systems," in Proc. Medit. Conf. Control Autom., Jun. 2009, pp. 1390-1395.
27. G. Toussaint, T. Basar, and F. Bullo, "H∞-optimal tracking control techniques for nonlinear underactuated systems," in Proc. IEEE Decision Control Conf., Dec. 2000, pp. 2078-2083.
28. T. Dierks and S. Jagannathan, "Optimal control of affine nonlinear continuous-time systems," in Proc. IEEE Amer. Control Conf., Jun. 2010, pp. 1568-1573.
29. R. W. Beard, "Improving the closed-loop performance of nonlinear systems," Ph.D. dissertation, Electr. Eng. Dept., Rensselaer Polytechnic Inst., Troy, NY, 1995.
30. J. N. Tsitsiklis and B. Van Roy, "An analysis of temporal-difference learning with function approximation," IEEE Trans. Autom. Control, vol. 42, no. 5, pp. 674-690, May 1997.