Data-Driven H∞ Control for Nonlinear Distributed Parameter Systems

IEEE Transactions on Neural Networks and Learning Systems

Volumn 26, Issue 11, 2015, Pages 2949-2961

  Luo, Biao ^a   Huang, Tingwen ^b   Wu, Huai Ning ^c   Yang, Xiong ^a  

^a UNIVERSITY OF SCIENCE AND TECHNOLOGY BEIJING   (China)

^b TEXAS A AND M UNIVERSITY AT QATAR   (Qatar)

^c BEIHANG UNIVERSITY   (China)

Author keywords

Data driven; distributed parameter systems (DSPs); H control; Hamilton Jacobi Isaacs (HJI) equation; neural network (NN); off policy learning; partial differential equation (PDE)

Indexed keywords

DATA REDUCTION; EIGENVALUES AND EIGENFUNCTIONS; LEAST SQUARES APPROXIMATIONS; PERTURBATION TECHNIQUES;

DATA DRIVEN; DISTRIBUTED PARAMETER SYSTEMS; HAMILTON-JACOBI-ISAACS; NEURAL NETWORK (NN); PARTIAL DIFFERENTIAL EQUATIONS (PDE); POLICY LEARNING;

DISTRIBUTED PARAMETER CONTROL SYSTEMS;

EID: 85027939867     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2015.2461023     Document Type: Article

References (63)

1. M. Krstic and A. Smyshlyaev, Boundary Control of PDEs: A Course on Backstepping Designs, vol. 16. Philadelphia, PA, USA: SIAM, 2008.
2. W. H. Ray, Advanced Process Control. New York, NY, USA: McGraw-Hill, 1981.
3. P. D. Christofides, Nonlinear and Robust Control of PDE Systems: Methods and Applications to Transport-Reaction Processes. Boston, MA, USA: Birkhäuser, 2001.
4. H.-X. Li and C. Qi, Spatio-Temporal Modeling of Nonlinear Distributed Parameter Systems. New York, NY, USA: Springer-Verlag, 2011.
5. B. Luo and H.-N. Wu, "Approximate optimal control design for nonlinear one-dimensional parabolic PDE systems using empirical eigenfunctions and neural network," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 6, pp. 1538-1549, Dec. 2012.
6. B. Luo and H.-N. Wu, "Online policy iteration algorithm for optimal control of linear hyperbolic PDE systems," J. Process Control, vol. 22, no. 7, pp. 1161-1170, 2012.
7. 2 disturbance attenuation for highly dissipative nonlinear spatially distributed processes via HJI approach," J. Process Control, vol. 24, no. 5, pp. 550-567, 2014.
8. A. Armaou and P. D. Christofides, "Dynamic optimization of dissipative PDE systems using nonlinear order reduction," Chem. Eng. Sci., vol. 57, no. 24, pp. 5083-5114, 2002.
9. A. Armaou and P. D. Christofides, "Crystal temperature control in the Czochralski crystal growth process," AIChE J., vol. 47, no. 1, pp. 79-106, 2001.
10. J. Baker and P. D. Christofides, "Output feedback control of parabolic PDE systems with nonlinear spatial differential operators," Ind. Eng. Chem. Res., vol. 38, no. 11, pp. 4372-4380, 1999.
11. A. Kalani and P. D. Christofides, "Simulation, estimation and control of size distribution in aerosol processes with simultaneous reaction, nucleation, condensation and coagulation," Comput. Chem. Eng., vol. 26, nos. 7-8, pp. 1153-1169, 2002.
12. Y. Ou et al., "Optimal tracking control of current profile in tokamaks," IEEE Trans. Control Syst. Technol., vol. 19, no. 2, pp. 432-441, Mar. 2011.
13. A. M. Bayen, R. L. Raffard, and C. J. Tomlin, "Adjoint-based control of a new Eulerian network model of air traffic flow," IEEE Trans. Control Syst. Technol., vol. 14, no. 5, pp. 804-818, Sep. 2006.
14. R. F. Curtain and H. Zwart, An Introduction to Infinite-Dimensional Linear Systems Theory, vol. 21. New York, NY, USA: Springer-Verlag, 1995.
15. A. Smyshlyaev and M. Krstic, Adaptive Control of Parabolic PDEs. Princeton, NJ, USA: Princeton Univ. Press, 2010.
16. B. Luo, H.-N. Wu, and H.-X. Li, "Adaptive optimal control of highly dissipative nonlinear spatially distributed processes with neuro-dynamic programming," IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 4, pp. 684-696, Apr. 2015.
17. B. Luo, H.-N. Wu, and H.-X. Li, "Data-based suboptimal neuro-control design with reinforcement learning for dissipative spatially distributed processes," Ind. Eng. Chem. Res., vol. 53, no. 19, pp. 8106-8119, 2014.
18. B.-S. Chen and Y.-T. Chang, "Fuzzy state-space modeling and robust observer-based control design for nonlinear partial differential systems," IEEE Trans. Fuzzy Syst., vol. 17, no. 5, pp. 1025-1043, Oct. 2009.
19. ∞ boundary control of semilinear parabolic and hyperbolic systems," Automatica, vol. 45, no. 9, pp. 2060-2066, 2009.
20. ∞ fuzzy controllers with constraint for nonlinear hyperbolic PDE systems," Automatica, vol. 48, no. 10, pp. 2535-2543, 2012.
21. ∞-Control for Distributed Parameter Systems: A State-Space Approach. New York, NY, USA: Springer-Verlag, 1993.
22. Y.-T. Chang and B.-S. Chen, "A fuzzy approach for robust reference tracking-control design of nonlinear distributed parameter time-delayed systems and its application," IEEE Trans. Fuzzy Syst., vol. 18, no. 6, pp. 1041-1057, Dec. 2010.
23. H.-N. Wu and H.-X. Li, "Adaptive neural control design for nonlinear distributed parameter systems with persistent bounded disturbances," IEEE Trans. Neural Netw., vol. 20, no. 10, pp. 1630-1644, Oct. 2009.
24. Z. Ni, H. He, and J. Wen, "Adaptive learning in tracking control based on the dual critic network design," IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 6, pp. 913-928, Jun. 2013.
25. D. Wang, D. Liu, and H. Li, "Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems," IEEE Trans. Autom. Sci. Eng., vol. 11, no. 2, pp. 627-632, Apr. 2014.
26. F. L. Lewis and K. G. Vamvoudakis, "Reinforcement learning for partially observable dynamic processes: Adaptive dynamic programming using measured output data," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 1, pp. 14-25, Feb. 2011.
27. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control," Automatica, vol. 43, no. 3, pp. 473-481, 2007.
28. D. Vrabie and F. Lewis, "Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems," Neural Netw., vol. 22, no. 3, pp. 237-246, 2009.
29. H. Modares, F. L. Lewis, and M.-B. Naghibi-Sistani, "Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems," Automatica, vol. 50, no. 1, pp. 193-202, 2014.
30. Y. Jiang and Z.-P. Jiang, "Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics," Automatica, vol. 48, no. 10, pp. 2699-2704, 2012.
31. J. Y. Lee, J. B. Park, and Y. H. Choi, "Integral Q-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems," Automatica, vol. 48, no. 11, pp. 2850-2859, 2012.
32. H. Zhang, L. Cui, X. Zhang, and Y. Luo, "Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method," IEEE Trans. Neural Netw., vol. 22, no. 12, pp. 2226-2236, Dec. 2011.
33. H. Modares, F. L. Lewis, and M.-B. Naghibi-Sistani, "Adaptive optimal control of unknown constrained-input systems using policy iteration and neural networks," IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 10, pp. 1513-1525, Oct. 2013.
34. ∞ control design," IEEE Trans. Cybern., vol. 45, no. 1, pp. 65-76, Jan. 2015.
35. D. Liu, D. Wang, D. Zhao, Q. Wei, and N. Jin, "Neural-network-based optimal control for a class of unknown discrete-time nonlinear systems using globalized dual heuristic programming," IEEE Trans. Autom. Sci. Eng., vol. 9, no. 3, pp. 628-634, Jul. 2012.
36. D. Vrabie and F. Lewis, "Adaptive dynamic programming for online solution of a zero-sum differential game," J. Control Theory Appl., vol. 9, no. 3, pp. 353-360, 2011.
37. H. Li, D. Liu, and D. Wang, "Integral reinforcement learning for linear continuous-time zero-sum games with completely unknown dynamics," IEEE Trans. Autom. Sci. Eng., vol. 11, no. 3, pp. 706-714, Jul. 2014.
38. ∞ control," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 12, pp. 1884-1895, Dec. 2012.
39. ∞ state feedback control," Inf. Sci., vol. 222, pp. 472-485, Feb. 2013.
40. B. Luo, H.-N. Wu, T. Huang, and D. Liu, "Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design," Automatica, vol. 50, no. 12, pp. 3281-3290, 2014.
41. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. London, U.K.: MIT Press, 1998.
42. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA, USA: Athena Scientific, 1996.
43. R. Hafner and M. Riedmiller, "Reinforcement learning in feedback control," Mach. Learn., vol. 84, nos. 1-2, pp. 137-169, 2011.
44. L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement learning: A survey," J. Artif. Intell. Res., vol. 4, pp. 237-285, Jan./Jun. 1996.
45. S. Pitchaiah and A. Armaou, "Output feedback control of distributed parameter systems using adaptive proper orthogonal decomposition," Ind. Eng. Chem. Res., vol. 49, no. 21, pp. 10496-10509, 2010.
46. S. Pitchaiah and A. Armaou, "Output feedback control of dissipative PDE systems with partial sensor information based on adaptive model reduction," AIChE J., vol. 59, no. 3, pp. 747-760, 2013.
47. A. Armaou and P. D. Christofides, "Finite-dimensional control of nonlinear parabolic PDE systems with time-dependent spatial domains using empirical eigenfunctions," Appl. Math. Comput. Sci., vol. 11, no. 2, pp. 287-318, 2001.
48. P. D. Christofides, "Robust control of parabolic PDE systems," Chem. Eng. Sci., vol. 53, no. 16, pp. 2949-2965, 1998.
49. ∞ control," IEEE Trans. Autom. Control, vol. 37, no. 6, pp. 770-784, Jun. 1992.
50. ∞ state feedback control with Galerkin's method," Int. J. Robust Nonlinear Control, vol. 23, no. 9, pp. 991-1012, 2013.
51. D. Precup, R. S. Sutton, and S. Dasgupta, "Off-policy temporaldifference learning with function approximation," in Proc. 18th Int. Conf. Mach. Learn., 2001, pp. 417-424.
52. B. A. Finlayson, Ed., The Method of Weighted Residuals and Variational Principles: With Application in Fluid Mechanics, Heat and Mass Transfer, vol. 87. New York, NY, USA: Academic, 1972.
53. G. P. Lepage, "A new algorithm for adaptive multidimensional integration," J. Comput. Phys., vol. 27, no. 2, pp. 192-203, 1978.
54. K. G. Vamvoudakis and F. L. Lewis, "Online solution of nonlinear two-player zero-sum games using synchronous policy iteration," Int. J. Robust Nonlinear Control, vol. 22, no. 13, pp. 1460-1483, 2012.
55. ∞ control of constrained input systems," Int. J. Adapt. Control Signal Process., vol. 28, nos. 3-5, pp. 232-254, 2014.
56. D. P. Bertsekas, "Approximate policy iteration: A survey and some new methods," J. Control Theory Appl., vol. 9, no. 3, pp. 310-335, 2011.
57. ∞ state feedback control with input saturation," IEEE Trans. Autom. Control, vol. 51, no. 12, pp. 1989-1995, Dec. 2006.
58. ∞ control design for unknown linear discrete-time systems via Q-learning with LMI," Automatica, vol. 46, no. 8, pp. 1320-1326, 2010.
59. H. Zhang, Q. Wei, and D. Liu, "An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games," Automatica, vol. 47, no. 1, pp. 207-214, 2011.
60. D. Liu, H. Li, and D. Wang, "Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm," Neurocomputing, vol. 110, no. 13, pp. 92-100, 2013.
61. Y. Jiang and Z.-P. Jiang, "Robust adaptive dynamic programming and feedback stabilization of nonlinear systems," IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 5, pp. 882-893, May 2014.
62. C. A. Tovey, "Tutorial on computational complexity," Interfaces, vol. 32, no. 3, pp. 30-61, 2002.
63. G. Sewell, Computational Methods of Linear Algebra, 2nd ed. Hoboken, NJ, USA: Wiley, 2005.