Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints

IEEE Transactions on Neural Networks

Volumn 20, Issue 9, 2009, Pages 1490-1503

  Zhang, Huaguang ^a   Luo, Yanhong ^a   Liu, Derong ^b  

^a NORTHEASTERN UNIVERSITY   (China)

^b INSTITUTE OF AUTOMATION   (China)

Author keywords

Adaptive dynamic programming; Approximate dynamic programming; Control constraints; Convergence analysis; Near optimal control; Neural networks

Indexed keywords

ADAPTIVE DYNAMIC PROGRAMMING; APPROXIMATE DYNAMIC PROGRAMMING; CONTROL CONSTRAINTS; CONVERGENCE ANALYSIS; NEAR-OPTIMAL CONTROL;

ADAPTIVE ALGORITHMS; CONTINUOUS TIME SYSTEMS; CONVERGENCE OF NUMERICAL METHODS; DIGITAL CONTROL SYSTEMS; DISCRETE TIME CONTROL SYSTEMS; DYNAMIC PROGRAMMING; FEEDBACK; FUNCTIONAL PROGRAMMING; NEURAL NETWORKS; NONLINEAR SYSTEMS; OPTIMIZATION; SYSTEMS ENGINEERING;

ADAPTIVE CONTROL SYSTEMS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; NONLINEAR SYSTEM; TIME;

ALGORITHMS; COMPUTER SIMULATION; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; TIME FACTORS;

EID: 70349253929     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2009.2027233     Document Type: Article

References (47)

1. M. Abu-Khalaf and F. L. Lewis, "Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach," Automatica, vol. 41, no. 5, pp. 779-791, May 2005.
2. ∞ control," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 37, no. 1, pp. 240-247, Feb. 2007.
3. A. Al-Tamimi and F. L. Lewis, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof," in Proc. IEEE Int. Symp. Approx. Dyn. Programm. Reinforcement Learn., Honolulu, HI, Apr. 2007, pp. 38-43.
4. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 38, no. 4, pp. 943-949, Aug. 2008.
5. J. Bagnell, S. Kakade, A. Ng, and J. Schneider, "Policy search by dynamic programming," in Proc. 17th Annu. Conf. Neural Inf. Process. Syst., Vancouver, BC, Canada, Dec. 2003, vol. 16, pp. 831-838.
6. M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Boston, MA: Birkhauser, 1997.
7. A. G. Barto, R. S. Sutton, and C. W. Anderson, "Neuronlike adaptive elements that can solve difficult learning control problems," IEEE Trans. Syst. Man Cybern., vol. SMC-13, no. 5, pp. 835-846, Sep./Oct. 1983.
8. R. Beard, "Improving the closed-loop performance of nonlinear systems," Ph.D. dissertation, Electr. Eng. Dept., Rensselaer Polytech. Inst., Troy, NY, 1995.
9. R. E. Bellman, Dynamic Programming. Princeton, NJ: Princeton Univ. Press, 1957.
10. D. S. Bernstein, "Optimal nonlinear, but continuous, feedback control of systems with saturating actuators," Int. J. Control, vol. 62, no. 5, pp. 1209-1216, 1995.
11. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena, 1996.
12. Z. Chen and S. Jagannathan, "Generalized Hamilton-Jacobi-Bellman formulation-based neural network control of affine nonlinear discretetime systems," IEEE Trans. Neural Netw, vol. 19, no. 1, pp. 90-106, Jan. 2008.
13. T. Cheng, F. L. Lewis, and M. Abu-Khalaf, "Fixed-final-time-constrained optimal control of nonlinear systems using neural network HJB approach," IEEE Trans. Neural Netw, vol. 18, no. 6, pp. 1725-1736, Nov. 2007.
14. J. Dankert, Y. Lei, and J. Si, "A performance gradient perspective on approximate dynamic programming and its application to partially observable Markov decision processes," in Proc. Int. Symp. Intell. Control, Munich, Germany, Oct. 2006, pp. 458-463.
15. R. Enns and J. Si, "Helicopter trimming and tracking control using direct neural dynamic programming," IEEE Trans. Neural Netw., vol. 14, no. 4, pp. 929-939, Jul. 2003.
16. S. Ferrari and R. F. Stengel, "Online adaptive critic flight control," J. Guid. Control Dyn., vol. 27, no. 5, pp. 777-786, 2004.
17. T. Hanselmann, L. Noakes, and A. Zaknich, "Continuous-time adaptive critics," IEEE Trans. Neural Netw., vol. 18, no. 3, pp. 631-647, May 2007.
18. G. G. Lendaris and C. Paintz, "Training strategies for critic and action neural networks in dual heuristic programming method," in Proc. Int. Joint Conf. Neural Netw., Houston, TX, Jun. 1997, vol. 2, pp. 712-717.
19. N. Jin, D. Liu, T. Huang, and Z. Pang, "Discrete-time adaptive dynamic programming using wavelet basis function neural networks," in Proc. IEEE Int. Symp. Approx. Dyn. Programm. Reinforcement Learn., Honolulu, HI, Apr. 2007, pp. 135-142.
20. B. Li and J. Si, "Robust dynamic programming for discounted infinite-horizon Markov decision processes with uncertain stationary transition matrices," in Proc. IEEE Int. Symp. Approx. Dyn. Programm. Reinforcement Learn., Honolulu, HI, Apr. 2007, pp. 96-102.
21. X. Liu and S. N. Balakrishnan, "Convergence analysis of adaptive critic based optimal control," in Proc. Amer. Control Conf., Chicago, IL, Jun. 2000, pp. 1929-1933.
22. D. Liu, H. Javaherian, O. Kovalenko, and T. Huang, "Adaptive critic learning techniques for engine torque and air-fuel ratio control," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 38, no. 4, pp. 988-993, Aug. 2008.
23. D. Liu, X. Xiong, and Y. Zhang, "Action-dependent adaptive critic designs," in Proc. Int. Joint Conf. Neural Netw., Washington, DC, Jul. 2001, vol. 2, pp. 990-995.
24. D. Liu and H. Zhang, "A neural dynamic programming approach for learning control of failure avoidance problems," Int. J. Intell. Control Syst., vol. 10, no. 1, pp. 21-32, 2005.
25. D. Liu, Y. Zhang, and H. Zhang, "A self-learning call admission control scheme for CDMA cellular networks," IEEE Trans. Neural Netw., vol. 16, no. 5, pp. 1219-1228, Sep. 2005.
26. S. E. Lyshevski, "Optimization of dynamic systems using novel performance functionals," in Proc. 41st Conf. Decision Control, Las Vegas, NV, Dec. 2002, pp. 753-758.
27. S. E. Lyshevski, "Optimal control of nonlinear continuous-time systems: Design of bounded controllers via generalized nonquadratic functionals," in Proc. Amer. Control Conf., Philadelphia, PA, Jun. 1998, pp. 205-209.
28. S. E. Lyshevski, "Nonlinear discrete-time systems: Constrained optimization and application of nonquadratic costs," in Proc. Amer. Control Conf., Philadelphia, PA, Jun. 1998, pp. 3699-3703.
29. J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, "Adaptive dynamic programming," IEEE Trans. Syst. Man Cybern. C, Appl. Rev. vol. 32, no. 2, pp. 140-153, May 2002.
30. R. Padhi, N. Unnikrishnan, X.Wang, and S. N. Balakrishnan, "A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems," Neural Netw., vol. 19, no. 10, pp. 1648-1660, Dec. 2006.
31. J.-W. Park, R. G. Harley, and G. K. Venayagamoorthy, "Adaptive-critic-based optimal neurocontrol for synchronous generators in a power system using MLP/RBF neural networks," IEEE Trans. Ind. Appl., vol. 39, no. 5, pp. 1529-1540, Sep./Oct. 2003.
32. D. V. Prokhorov and D. C. Wunsch, "Adaptive critic designs," IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 997-1007, Sep. 1997.
33. A. Saberi, Z. Lin, and A. Teel, "Control of linear systems with saturating actuators," IEEE Trans. Autom. Control, vol. 41, no. 3, pp. 368-378, Mar. 1996.
34. G. Saridis and C. S. Lee, "An approximation theory of optimal control for trainable manipulators," IEEE Trans. Syst. Man Cybern., vol. SMC-9, no. 2, pp. 152-159, Mar. 1979.
35. G. N. Saridis and F. Y. Wang, "Suboptimal control of nonlinear stochastic systems," Control-Theory Adv. Technol., vol. 10, no. 4, pp. 847-871, 1994.
36. S. Shervais, T. T. Shannon, and G. G. Lendaris, "Intelligent supply chain management using adaptive critic learning," IEEE Trans. Syst. Man Cybern. A, Syst. Humans, vol. 33, no. 2, pp. 235-244, Mar. 2003.
37. J. Si and Y.-T. Wang, "On-line learning control by association and reinforcement," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 264-276, Mar. 2001.
38. H. Sussmann, E. D. Sontag, and Y. Yang, "A general result on the stabilization of linear systems using bounded controls," IEEE Trans. Autom. Control, vol. 39, no. 12, pp. 2411-2425, Dec. 1994.
39. F.-Y. Wang and G. N. Saridis, "Suboptimal control for nonlinear stochastic systems," in Proc. 31st IEEE Conf. Decision Control, Tucson, AZ, Dec. 1992, pp. 1856-1861.
40. C.Watkins, "Learning from delayed rewards," Ph.D. dissertation, Dept. Psychol., Cambridge University, Cambridge, U.K., 1989.
41. P. J.Werbos, "A menu of designs for reinforcement learning over time," in Neural Networks for Control, W. T. Miller, R. S. Sutton, and P. J. Werbos, Eds. Cambridge, MA: MIT Press, 1991, pp. 67-95.
42. P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling," in Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches, D. A.White and D. A. Sofge, Eds. New York: Van Nostrand, 1992, ch. 13.
43. P. J. Werbos, "Using ADP to understand and replicate brain intelligence: The next level design," in Proc. IEEE Int. Symp. Approx. Dyn. Programm. Reinforcement Learn., Honolulu, HI, Apr. 2007, pp. 209-216.
44. B. Widrow, N. Gupta, and S. Maitra, "Punish/reward: Learning with a critic in adaptive threshold systems," IEEE Trans. Syst. Man Cybern. vol. SMC-3, no. 5, pp. 455-465, Sep. 1973.
45. V. Yadav, R. Padhi, and S. N. Balakrishnan, "Robust/optimal temperature profile control of a high-speed aerospace vehicle using neural networks," IEEE Trans. Neural Netw., vol. 18, no. 4, pp. 1115-1128, Jul. 2007.
46. Q. Yang and S. Jagannathan, "Online reinforcement learning neural network controller design for nanomanipulation," in Proc. IEEE Symp. Approx. Dyn. Programm. Reinforcement Learn., Honolulu, HI, Apr. 2007, pp. 225-232.
47. H. Zhang, Q.Wei, and Y. Luo, "A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear system based on greedy HDP iteration algorithm," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 38, no. 4, pp. 937-942, Aug. 2008.