Air-Breathing Hypersonic Vehicle Tracking Control Based on Adaptive Dynamic Programming

IEEE Transactions on Neural Networks and Learning Systems

Volumn 28, Issue 3, 2017, Pages 584-598

  Mu, Chaoxu ^a,b   Ni, Zhen ^c   Sun, Changyin ^d   He, Haibo ^b  

^a TIANJIN UNIVERSITY   (China)

^b University of Rhode Island   (United States)

^c South Dakota State University   (United States)

^d SOUTHEAST UNIVERSITY   (China)

Author keywords

Action dependent heuristic dynamic programming (ADHDP); air breathing hypersonic vehicle (AHV); learning and adaptation capabilities; parameter uncertainties and disturbances; Tracking control

Indexed keywords

ALTITUDE CONTROL; CONTROLLERS; FLIGHT SIMULATORS; FUNCTIONS; HEURISTIC PROGRAMMING; HYPERSONIC AERODYNAMICS; HYPERSONIC VEHICLES; NAVIGATION; SLIDING MODE CONTROL; TRACKING (POSITION); VEHICLES;

ACTION DEPENDENT HEURISTIC DYNAMIC PROGRAMMING; ADAPTIVE DYNAMIC PROGRAMMING; ADAPTIVE LEARNING; AIR BREATHING; CONTROL ACTIONS; CONTROL APPROACH; PARAMETER UNCERTAINTY; SIMULATION STUDIES;

DYNAMIC PROGRAMMING;

EID: 84957645179     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2016.2516948     Document Type: Article

References (75)

1. B. Fidan, M. Mirmirani, P. A. Ioannou, "Flight dynamics and control of air-breathing hypersonic vehicles: Review and new directions, " in Proc. 12th AIAA Int. Space Planes Hypersonic Syst. Technol., 2003, pp. 1-38, paper AIAA 2003-7081.
2. R. T. Voland, L. D. Huebner, C. R. McClinton, "X-43A hypersonic vehicle technology development, " Acta Astron., vol. 59, nos. 1-5, pp. 181-191, Jul./Sep. 2006.
3. B. Xu and Z. Shi, "An overview on flight dynamics and control approaches for hypersonic vehicles, " Sci. China Inf. Sci., vol. 58, no. 7, pp. 1-19, Jul. 2015.
4. M. A. Bolender and D. B. Doman, "A non-linear model for the longitudinal dynamics of a hypersonic air-breathing vehicle, " in Proc. AIAA Guid., Navigat., Control Conf., 2005, paper AIAA 2005-6255.
5. J. Dickeson, A. Rodriguez, S. Sridharan, A. Korad, "Elevator sizing, placement, control-relevant tradeoffs for hypersonic vehicles, " in Proc. AIAA Guid., Navigat., Control Conf., 2010, paper AIAA 2010-8339.
6. J. D. Shaughnessy, S. Z. Pinckney, J. D. McMinn, C. I. Cruz, M. L. Kelley, "Hypersonic vehicle simulation model: Winged-cone configuration, " Nat. Aeronautics Space Admin., Hampton, VA, USA, Tech. Rep. 102610, 1990.
7. M. A. Bolender and D. B. Doman, "Nonlinear longitudinal dynamical model of an air-breathing hypersonic vehicle, " J. Spacecraft Rockets, vol. 44, no. 2, pp. 374-387, 2007.
8. C. Sun, C. Mu, Y. Yu, "Some control problems for near space hypersonic vehicles, " Automat. Sinica, vol. 39, no. 11, pp. 1901-1913, 2013.
9. O. U. Rehman, B. Fidan, I. R. Petersen, "Uncertainty modeling and robust minimax LQR control of multivariable nonlinear systems with application to hypersonic flight, " Asian J. Control, vol. 14, no. 5, pp. 1180-1193, Sep. 2012.
10. Y. Huang, C. Sun, C. Qian, L. Wang, "Linear parameter varying switching attitude control for a near space hypersonic vehicle with parametric uncertainties, " Int. J. Syst. Sci., vol. 46, no. 16, pp. 3019-3031, 2015.
11. C. Mu, C. Sun, W. Xu, "Fast sliding mode control on air-breathing hypersonic vehicles with transient response analysis, " Proc. Inst. Mech. Eng. I, J. Syst. Control Eng., vol. 230, no. 1, pp. 23-34, 2016.
12. W. A. Butt, L. Yan, K. Amezquita, "Adaptive integral dynamic surface control of a hypersonic flight vehicle, " Int. J. Syst. Sci., vol. 46, no. 10, pp. 1717-1728, 2015.
13. B. Xu and Y. Zhang, "Neural discrete back-stepping control of hypersonic flight vehicle with equivalent prediction model, " Neurocomputing, vol. 154, pp. 337-346, Apr. 2015.
14. D. Gao and Z. Sun, "Fuzzy tracking control design for hypersonic vehicles via T-S model, " Sci. China Inf. Sci., vol. 54, no. 3, pp. 521-528, Mar. 2011.
15. B. Xu, D. Wang, H. Wang, S. Zhu, "Adaptive neural control of a hypersonic vehicle in discrete time, " J. Intell. Robot. Syst., vol. 73, no. 1, pp. 219-231, Jan. 2014.
16. D. V. Prokhorov, R. A. Santiago, D. C. Wunsch, "Adaptive critic designs: A case study for neurocontrol, " Neural Netw., vol. 8, no. 9, pp. 1367-1372, 1995.
17. J. Si and Y.-T. Wang, "Online learning control by association and reinforcement, " IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 264-276, Mar. 2001.
18. Z. Ni, H. He, X. Zhong, D. V. Prokhorov, "Model-free dual heuristic dynamic programming, " IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 8, pp. 1834-1839, Aug. 2015.
19. D. Wang, D. Liu, Q. Wei, D. Zhao, N. Jin, "Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming, " Automatica, vol. 48, no. 8, pp. 1825-1832, 2012.
20. M. Fairbank, D. Prokhorov, E. Alonso, "Clipping in neurocontrol by adaptive dynamic programming, " IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 10, pp. 1909-1920, Oct. 2014.
21. D. Liu, D. Wang, F. Wang, H. Li, X. Yang, "Neural-networkbased online HJB solution for optimal robust guaranteed cost control of continuous-time uncertain nonlinear systems, " IEEE Trans. Cybern., vol. 44, no. 12, pp. 2834-2847, Dec. 2014.
22. D. Liu, X. Yang, D. Wang, Q. Wei, "Reinforcement-learning-based robust controller design for continuous-time uncertain nonlinear systems subject to input constraints, " IEEE Trans. Cybern., vol. 45, no. 7, pp. 1372-1385, Jul. 2015.
23. R. E. Bellman and S. E. Dreyfus, Applied Dynamic Programming. Santa Monica, CA, USA: Rand Corporation, 1962.
24. A. Al-Tamimi, F. L. Lewis, M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 943-949, Aug. 2008.
25. M. Fairbank, E. Alonso, D. Prokhorov, "An equivalence between adaptive dynamic programming with a critic and backpropagation through time, " IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 12, pp. 2088-2100, Dec. 2013.
26. Q. Wei, F. Wang, D. Liu, X. Yang, "Finite-approximation-errorbased discrete-time iterative adaptive dynamic programming, " IEEE Trans. Cybern., vol. 44, no. 12, pp. 2820-2833, Dec. 2014.
27. X. Yang, D. R. Liu, D. Wang, Q. Wei, "Discrete-time online learning control for a class of unknown nonaffine nonlinear systems using reinforcement learning, " Neural Netw., vol. 55, pp. 30-41, Jul. 2014.
28. R. Song, W. Xiao, H. Zhang, C. Sun, "Adaptive dynamic programming for a class of complex-valued nonlinear systems, " IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 9, pp. 1733-1739, Sep. 2014.
29. R. Song, F. Lewis, Q. Wei, H.-G. Zhang, Z.-P. Jiang, D. Levine, "Multiple actor-critic structures for continuous-time optimal control using input-output data, " IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 4, pp. 851-865, Apr. 2015.
30. X. Yang, D. Liu, H. Ma, Y. Xu, "Online approximate solution of HJI equation for unknown constrained-input nonlinear continuous-time systems, " Inf. Sci., vol. 328, pp. 435-454, Jan. 2016.
31. A. W. Wilhite, et al., "Concepts leading to the national aero-space plane program, " in Proc. 28th Aerosp. Sci. Meeting, 1990, paper AIAA 90-0294.
32. J. D. Shaughnessy and I. M. Gregory, "Trim drag reduction concepts for horizontal takeoff single-stage-to-orbit vehicles, " Nat. Aeronautics Space Admin., Hampton, VA, USA, Tech. Rep. 102687, 1991.
33. C. H. Fox, "Subsonic longitudinal and lateral-directional static aerodynamic characteristics of a slender wing-body configuration, " Nat. Aeronautics Space Admin., Langley Res. Center, Hampton, VA, USA, Tech. Rep. 1011, 1988.
34. Q. Wang and R. F. Stengel, "Robust nonlinear control of a hypersonic aircraft, " J. Guid., Control, Dyn., vol. 23, no. 4, pp. 577-585, 2000.
35. J. Yang, S. Li, C. Sun, L. Guo, "Nonlinear-disturbance-observerbased robust flight control for airbreathing hypersonic vehicles, " IEEE Trans. Aerosp. Electron. Syst., vol. 49, no. 2, pp. 1263-1275, Apr. 2013.
36. C. Mu, C. Sun, X. Yu, "Observation and control for air-breathing hypersonic aircrafts based on sliding mode method, " in Proc. 37th Annu. Conf. IEEE Ind. Electron. Soc., Nov. 2011, pp. 3965-3970.
37. C. Mu and C. Sun, "A new finite time convergence condition for supertwisting observer based on Lyapunov analysis, " Asian J. Control, vol. 17, no. 3, pp. 1050-1060, May 2015.
38. C. Mu, Q. Zong, B. Tian, W. Xu, "Continuous sliding mode controller with disturbance observer for hypersonic vehicles, " IEEE/CAA J. Automat. Sinica, vol. 2, no. 1, pp. 45-55, Jan. 2015.
39. H. Xu, M. D. Mirmirani, P. A. Ioannou, "Adaptive sliding mode control design for a hypersonic flight vehicle, " J. Guid., Control, Dyn., vol. 27, no. 5, pp. 829-838, 2004.
40. W. A. Butt, L. Yan, S. K. Amezquita, "Adaptive integral dynamic surface control of a hypersonic flight vehicle, " Int. J. Syst. Sci., vol. 46, no. 10, pp. 1717-1728, 2015.
41. V. I. Utkin, "Sliding mode control: Mathematical tools, design and applications, " in Nonlinear and Optimal Control Theory, P. Nistri and G. Stefani, Eds. Berlin, Germany: Springer, 2008, pp. 289-347.
42. B. Xu, D. Wang, F. Sun, Z. Shi, "Direct neural control of hypersonic flight vehicles with prediction model in discrete time, " Neurocomputing, vol. 115, pp. 39-48, Sep. 2013.
43. Y. Du, Q. Wu, C. Jiang, Y. Xue, "Adaptive recurrent-functional-linknetwork control for hypersonic vehicles with atmospheric disturbances, " Sci. China Inf. Sci., vol. 54, no. 3, pp. 482-497, Mar. 2011.
44. Q. Zong, F. Wang, B. Tian, R. Su, "Robust adaptive dynamic surface control design for a flexible air-breathing hypersonic vehicle with input constraints and uncertainty, " Nonlinear Dyn., vol. 78, no. 1, pp. 289-315, Oct. 2014.
45. X. Hu, H. Gao, H. R. Karimi, L. Wu, C. Hu, "Fuzzy reliable tracking control for flexible air-breathing hypersonic vehicles, " Int. J. Fuzzy Syst., vol. 13, no. 4, pp. 1-9, 2011.
46. J. Fu, H. He, X. Zhou, "Adaptive learning and control for MIMO system based on adaptive dynamic programming, " IEEE Trans. Neural Netw., vol. 22, no. 7, pp. 1133-1148, Jul. 2011.
47. Y. Tang, H. He, J. Wen, J. Liu, "Power system stability control for a wind farm based on adaptive dynamic programming, " IEEE Trans. Smart Grid, vol. 6, no. 1, pp. 166-177, Jan. 2015.
48. H. Zhang, Q. Wei, Y. Luo, "A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 937-942, Aug. 2008.
49. H. He, Self-Adaptive Systems for Machine Intelligence. New York, NY, USA: Wiley, 2011.
50. M. Fairbank, E. Alonso, D. Prokhorov, "Simple and fast calculation of the second-order gradients for globalized dual heuristic dynamic programming in neural networks, " IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 10, pp. 1671-1676, Oct. 2012.
51. F. L. Lewis and D. Liu, Eds., Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. New York, NY, USA: Wiley, 2013.
52. X. Xu, Z. Hou, C. Lian, H. He, "Online learning control using adaptive critic designs with sparse kernel machines, " IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 5, pp. 762-775, May 2013.
53. D. Wang, D. Liu, H. Li, H. Ma, "Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming, " Inf. Sci., vol. 282, pp. 167-179, Oct. 2014.
54. D. Wang, D. Liu, H. Li, "Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems, " IEEE Trans. Autom. Sci. Eng., vol. 11, no. 2, pp. 627-632, Apr. 2014.
55. D. V. Prokhorov, "Adaptive critic designs and their applications, " Ph. D. dissertation, Dept. Elect. Eng., Texas Tech Univ., Lubbock, TX, USA, 1997.
56. M. Fairbank, S. Li, X. Fu, E. Alonso, D. Wunsch, "An adaptive recurrent neural-network controller using a stabilization matrix and predictive inputs to solve a tracking problem under disturbances, " Neural Netw., vol. 49, pp. 74-86, Jan. 2014.
57. Z. Ni, H. He, J. Wen, "Adaptive learning in tracking control based on the dual critic network design, " IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 6, pp. 913-928, Jun. 2013.
58. P. He and S. Jagannathan, "Reinforcement learning neural-networkbased controller for nonlinear discrete-time systems with input constraints, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 37, no. 2, pp. 425-436, Apr. 2007.
59. Z. Ni, H. He, D. Zhao, X. Xu, D. V. Prokhorov, "GrDHP: A general utility function representation for dual heuristic dynamic programming, " IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 3, pp. 614-627, Mar. 2015.
60. H. He, Z. Ni, J. Fu, "A three-network architecture for on-line learning and optimization based on adaptive dynamic programming, " Neurocomputing, vol. 78, no. 1, pp. 3-13, 2012.
61. P. Shih, B. C. Kaul, S. Jagannathan, J. A. Drallmeier, "Reinforcement-learning-based dual-control methodology for complex nonlinear discrete-time systems with application to spark engine EGR operation, " IEEE Trans. Neural Netw., vol. 19, no. 8, pp. 1369-1388, Aug. 2008.
62. W. Liu, G. K. Venayagamoorthy, D. C. Wunsch, "A heuristicdynamic-programming-based power system stabilizer for a turbogenerator in a single-machine power system, " IEEE Trans. Ind. Appl., vol. 41, no. 5, pp. 1377-1385, Sep./Oct. 2005.
63. Y. Tang, H. He, Z. Ni, J. Wen, X. Sui, "Reactive power control of grid-connected wind farm based on adaptive dynamic programming, " Neurocomputing, vol. 125, pp. 125-133, Feb. 2014.
64. Z. Ni, H. He, J. Wen, X. Xu, "Goal representation heuristic dynamic programming on maze navigation, " IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 12, pp. 2038-2050, Dec. 2013.
65. D. V. Prokhorov and D. C. Wunsch, "Adaptive critic designs, " IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 997-1007, Sep. 1997.
66. R. Enns and J. Si, "Helicopter trimming and tracking control using direct neural dynamic programming, " IEEE Trans. Neural Netw., vol. 14, no. 4, pp. 929-939, Aug. 2003.
67. R. Enns and J. Si, "Apache helicopter stabilization using neural dynamic programming, " J. Guid., Control, Dyn., vol. 25, no. 1, pp. 19-25, 2002.
68. X. Luo, Y. Chen, J. Si, F. Liu, "Longitudinal control of hypersonic vehicles based on direct heuristic dynamic programming using ANFIS, " in Proc. Int. Joint Conf. Neural Netw. (IJCNN), Jul. 2014, pp. 3685-3692.
69. P. J. Werbos, "Consistency of HDP applied to a simple reinforcement learning problem, " Neural Netw., vol. 3, no. 2, pp. 179-189, 1990.
70. J. T. Parker, A. Serrani, S. Yurkovich, M. A. Bolender, D. B. Doman, "Control-oriented modeling of an air-breathing hypersonic vehicle, " J. Guid., Control, Dyn., vol. 30, no. 3, pp. 856-869, 2007.
71. F. Poulain, H. Piet-Lahanie, L. Serre, "Nonlinear control of a airbreathing hypersonic vehicle, " in Proc. AIAA 16th Int. Space Planes Hypersonic Syst. Technol. Conf., 2009, pp. 1-14, paper AIAA 2009-7290.
72. P. He and S. Jagannathan, "Reinforcement learning-based output feedback control of nonlinear systems with input constraints, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 35, no. 1, pp. 150-154, Feb. 2005.
73. L. Yang, J. Si, K. S. Tsakalis, A. A. Rodriguez, "Direct heuristic dynamic programming for nonlinear tracking control with filtered tracking error, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 39, no. 6, pp. 1617-1622, Dec. 2009.
74. F. Liu, J. Sun, J. Si, W. Guo, S. Mei, "A boundedness result for the direct heuristic dynamic programming, " Neural Netw., vol. 32, pp. 229-235, Aug. 2012.
75. Y. Sokolov, R. Kozma, L. D. Werbos, P. J. Werbos, "Complete stability analysis of a heuristic approximate dynamic programming control design, " Automatica, vol. 59, pp. 9-18, Sep. 2015.