Adaptive Optimal Control of Highly Dissipative Nonlinear Spatially Distributed Processes With Neuro-Dynamic Programming

IEEE Transactions on Neural Networks and Learning Systems

Volumn 26, Issue 4, 2015, Pages 684-696

  Luo, Biao ^a   Wu, Huai Ning ^a   Li, Han Xiong ^b  

^a BEIHANG UNIVERSITY   (China)

^b CITY UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Adaptive optimal control; empirical eigenfunction (EEF); highly dissipative partial differential equations (PDEs); neuro dynamic programming (NDP); spatially distributed processes (SDPs)

Indexed keywords

ADAPTIVE CONTROL SYSTEMS; DIFFERENTIAL EQUATIONS; EIGENVALUES AND EIGENFUNCTIONS; NONLINEAR EQUATIONS; OPTIMAL CONTROL SYSTEMS; ORDINARY DIFFERENTIAL EQUATIONS; PARTIAL DIFFERENTIAL EQUATIONS; PERTURBATION TECHNIQUES; SOCIAL NETWORKING (ONLINE); SPATIAL DISTRIBUTION;

ADAPTIVE OPTIMAL CONTROL; EMPIRICAL EIGENFUNCTIONS; NEURO-DYNAMIC PROGRAMMING; PARTIAL DIFFERENTIAL EQUATIONS (PDES); SPATIALLY DISTRIBUTED PROCESS;

DYNAMIC PROGRAMMING;

ALGORITHM; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; FEEDBACK SYSTEM; HUMAN; NONLINEAR SYSTEM; TEMPERATURE;

ALGORITHMS; COMPUTER SIMULATION; FEEDBACK; HUMANS; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; TEMPERATURE;

EID: 84925883034     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2014.2320744     Document Type: Article

References (68)

1. P. D. Christofides, Nonlinear and Robust Control of PDE Systems: Methods and Applications to Transport-Reaction Processes. Boston, MA, USA: Birkhäuser, 2001.
2. V. Yadav, R. Padhi, and S. Balakrishnan, "Robust/optimal temperature profile control of a high-speed aerospace vehicle using neural networks," IEEE Trans. Neural Netw., vol. 18, no. 4, pp. 1115-1128, Jul. 2007.
3. H.-N. Wu and H.-X. Li, "Adaptive neural control design for nonlinear distributed parameter systems with persistent bounded disturbances,"IEEE Trans. Neural Netw., vol. 20, no. 10, pp. 1630-1644, Oct. 2009.
4. A. Armaou and P. D. Christofides, "Dynamic optimization of dissipative PDE systems using nonlinear order reduction," Chem. Eng. Sci., vol. 57, no. 24, pp. 5083-5114, 2002.
5. A. Armaou and P. D. Christofides, "Crystal temperature control in the Czochralski crystal growth process," AIChE J., vol. 47, no. 1, pp. 79-106, 2001.
6. Y. Ou et al., "Optimal tracking control of current profile in tokamaks,"IEEE Trans. Control Syst. Technol., vol. 19, no. 2, pp. 432-441, Mar. 2011.
7. J.-W. Wang, H.-N. Wu, and H.-X. Li, "Distributed proportional-spatial derivative control of nonlinear parabolic systems via fuzzy PDE modeling approach," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 3, pp. 927-938, Jun. 2012.
8. M. Krstic, "On global stabilization of Burgers' equation by boundary control," Syst. Control Lett., vol. 37, no. 3, pp. 123-141, 1999.
9. P. Holmes, J. L. Lumley, and G. Berkooz, Turbulence, Coherent Structures, Dynamical Systems and Symmetry. Cambridge, U.K.: Cambridge Univ. Press, 1998.
10. M. J. Balas, "The Galerkin method and feedback control of linear distributed parameter systems," J. Math. Anal. Appl., vol. 91, no. 2, pp. 527-546, 1983.
11. W. H. Ray, Advanced Process Control. New York, NY, USA: McGraw-Hill, 1981.
12. E. S. Titi, "On approximate inertial manifolds to the Navier-Stokes equations," J. Math. Anal. Appl., vol. 149, no. 2, pp. 540-557, 1990.
13. A. Armaou and P. D. Christofides, "Robust control of parabolic PDE systems with time-dependent spatial domains," Automatica, vol. 37, no. 1, pp. 61-69, 2001.
14. A. Varshney, S. Pitchaiah, and A. Armaou, "Feedback control of dissipative PDE systems using adaptive model reduction," AIChE J., vol. 55, no. 4, pp. 906-918, 2009.
15. S. Pitchaiah and A. Armaou, "Output feedback control of distributed parameter systems using adaptive proper orthogonal decomposition,"Ind. Eng. Chem. Res., vol. 49, no. 21, pp. 10496-10509, 2010.
16. S. Pitchaiah and A. Armaou, "Output feedback control of dissipative PDE systems with partial sensor information based on adaptive model reduction," AIChE J., vol. 59, no. 3, pp. 747-760, 2013.
17. E. Bendersky and P. D. Christofides, "Optimization of transport-reaction processes using nonlinear model reduction," Chem. Eng. Sci., vol. 55, no. 19, pp. 4349-4366, 2000.
18. D. V. Prokhorov, "Optimal neurocontrollers for discretized distributed parameter systems," in Proc. Amer. Control Conf., vol. 1. Jun. 2003, pp. 549-554.
19. A. Alessandri, M. Gaggero, and R. Zoppoli, "Feedback optimal control of distributed parameter systems by using finite-dimensional approximation schemes," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 6, pp. 984-996, Jun. 2012.
20. C. Xu, Y. Ou, and E. Schuster, "Sequential linear quadratic control of bilinear parabolic PDEs based on POD model reduction," Automatica, vol. 47, no. 2, pp. 418-426, 2011.
21. B. Luo and H.-N. Wu, "Approximate optimal control design for nonlinear one-dimensional parabolic PDE systems using empirical eigenfunctions and neural network," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 6, pp. 1538-1549, Dec. 2012.
22. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, U.K.: Cambridge Univ. Press, 1998.
23. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA, USA: Athena Scientific, 1996.
24. L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement learning: A survey," J. Artif. Intell. Res., vol. 4, pp. 237-285, 1996.
25. F.-Y. Wang, H. Zhang, and D. Liu, "Adaptive dynamic programming: An introduction," IEEE Comput. Intell. Mag., vol. 4, no. 2, pp. 39-47, May 2009.
26. F. L. Lewis and D. Vrabie, "Reinforcement learning and adaptive dynamic programming for feedback control," IEEE Circuits Syst. Mag., vol. 9, no. 3, pp. 32-50, Third Quarter, 2009.
27. F. L. Lewis and D. Liu, Reinforcement Learning and Approximate Dynamic Programming for Feedback Control, vol. 17. Hoboken, NJ, USA: Wiley, 2013.
28. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time non-linear HJB solution using approximate dynamic programming: Convergence proof," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 943-949, Aug. 2008.
29. Z. Chen and S. Jagannathan, "Generalized Hamilton-Jacobi-Bellman formulation-based neural network control of affine nonlinear discretetime systems," IEEE Trans. Neural Netw., vol. 19, no. 1, pp. 90-106, Jan. 2008.
30. H. Zhang, Q. Wei, and Y. Luo, "A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 937-942, Aug. 2008.
31. H. Zhang, Y. Luo, and D. Liu, "Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints," IEEE Trans. Neural Netw., vol. 20, no. 9, pp. 1490-1503, Sep. 2009.
32. F.-Y.Wang, N. Jin, D. Liu, and Q.Wei, "Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound," IEEE Trans. Neural Netw., vol. 22, no. 1, pp. 24-36, Jan. 2011.
33. D. Liu and Q. Wei, "Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems," IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 3, pp. 621-634, Mar. 2014.
34. H. Zhang, R. Song, Q. Wei, and T. Zhang, "Optimal tracking control for a class of nonlinear discrete-time systems with time delays based on heuristic dynamic programming," IEEE Trans. Neural Netw., vol. 22, no. 12, pp. 1851-1862, Dec. 2011.
35. J. Fu, H. He, and X. Zhou, "Adaptive learning and control for MIMO system based on adaptive dynamic programming," IEEE Trans. Neural Netw., vol. 22, no. 7, pp. 1133-1148, Jul. 2011.
36. A. Heydari and S. N. Balakrishnan, "Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics," IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 1, pp. 145-157, Jan. 2013.
37. Q. Yang and S. Jagannathan, "Reinforcement learning controller design for affine nonlinear discrete-time systems using online approximators,"IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 2, pp. 377-390, Apr. 2012.
38. T. Dierks and S. Jagannathan, "Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using time-based policy update," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1118-1129, Jul. 2012.
39. D. Liu, D. Wang, D. Zhao, Q. Wei, and N. Jin, "Neural-network-based optimal control for a class of unknown discrete-time nonlinear systems using globalized dual heuristic programming," IEEE Trans. Autom. Sci. Eng., vol. 9, no. 3, pp. 628-634, Jul. 2012.
40. D. Wang, D. Liu, Q. Wei, D. Zhao, and N. Jin, "Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming," Automatica, vol. 48, no. 8, pp. 1825-1832, 2012.
41. D. Liu, D. Wang, and X. Yang, "An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs," Inform. Sci., vol. 220, pp. 331-342, Jan. 2013.
42. R. W. Beard, G. N. Saridis, and J. T. Wen, "Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation," Automatica, vol. 33, no. 12, pp. 2159-2177, 1997.
43. M. Abu-Khalaf and F. L. Lewis, "Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach," Automatica, vol. 41, no. 5, pp. 779-791, 2005.
44. K. Doya, "Reinforcement learning in continuous time and space," Neural Comput., vol. 12, no. 1, pp. 219-245, 2000.
45. J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, "Adaptive dynamic programming," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 32, no. 2, pp. 140-153, May 2002.
46. D. Vrabie, O. Pastravanu, M. Abu-Khalaf, and F. L. Lewis, "Adaptive optimal control for continuous-time linear systems based on policy iteration," Automatica, vol. 45, no. 2, pp. 477-484, 2009.
47. D. Vrabie and F. L. Lewis, "Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems," Neural Netw., vol. 22, no. 3, pp. 237-246, 2009.
48. K. G. Vamvoudakis and F. L. Lewis, "Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem,"Automatica, vol. 46, no. 5, pp. 878-888, 2010.
49. B. Luo, H.-N. Wu, T. Huang, and D. Liu. (2013). Data-based approximate policy iteration for nonlinear continuous-time optimal control design. arXiv preprint arXiv:1311.0396 [Online]. Available: http://arxiv.org/abs/1311.0396
50. ∞ control design," IEEE Trans. Cybern., 2014, doi: 10.1109/TCYB.2014.2319577.
51. D. Liu, D. Wang, and H. Li, "Decentralized stabilization for a class of continuous-time nonlinear interconnected systems using online learning optimal control approach," IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 2, pp. 418-428, Feb. 2014.
52. M. Li and P. D. Christofides, "Optimal control of diffusion-convection-reaction processes using reduced-order models," Comput. Chem. Eng., vol. 32, no. 9, pp. 2123-2135, 2008.
53. M. Li and P. D. Christofides, "An input/output approach to the optimal transition control of a class of distributed chemical reactors," Chem. Eng. Sci., vol. 62, no. 11, pp. 2979-2988, 2007.
54. J. Baker and P. D. Christofides, "Finite-dimensional approximation and control of non-linear parabolic PDE systems," Int. J. Control, vol. 73, no. 5, pp. 439-456, 2000.
55. A. Armaou and P. D. Christofides, "Finite-dimensional control of nonlinear parabolic PDE systems with time-dependent spatial domains using empirical eigenfunctions," Int. J. Appl. Math. Comput. Sci., vol. 11, no. 2, pp. 287-318, 2001.
56. H.-X. Li and C. Qi, Spatio-Temporal Modeling of Nonlinear Distributed Parameter Systems. New York, NY, USA: Springer-Verlag, 2011.
57. B. Luo, H.-N. Wu, and H.-X. Li, "Data-based suboptimal neuro-control design with reinforcement learning for dissipative spatially distributed processes," Ind. Eng. Chem. Res., 2014, doi: 10.1021/ie4031743.
58. 2 disturbance attenuation for highly dissipative nonlinear spatially distributed processes via HJI approach," J. Process Control, vol. 24, no. 5, pp. 550-567, 2014.
59. P. D. Christofides, "Robust control of parabolic PDE systems," Chem. Eng. Sci., vol. 53, no. 16, pp. 2949-2965, 1998.
60. F. L. Lewis, D. Vrabie, and V. L. Syrmos, Optimal Control. Hoboken, NJ, USA: Wiley, 2013.
61. R. Courant and D. Hilbert, Methods of Mathematical Physics, vol. 1. New York, NY, USA: Wiley, 2004.
62. S. Ge, C. C. Hang, T. H. Lee, and T. Zhang, Stable Adaptive Neural Network Control. Norwell, MA, USA: Kluwer, 2001.
63. P. D. Christofides and P. Daoutidis, "Robust control of hyperbolic PDE systems," Chem. Eng. Sci., vol. 53, no. 1, pp. 85-105, 1998.
64. H.-N. Wu and B. Luo, "Heuristic dynamic programming algorithm for optimal control design of linear continuous-time hyperbolic PDE systems," Ind. Eng. Chem. Res., vol. 51, no. 27, pp. 9310-9319, 2012.
65. I. Aksikas, A. Fuxman, J. F. Forbes, and J. J. Winkin, "LQ control design of a class of hyperbolic PDE systems: Application to fixed-bed reactor," Automatica, vol. 45, no. 6, pp. 1542-1548, 2009.
66. B. Luo and H.-N. Wu, "Online policy iteration algorithm for optimal control of linear hyperbolic PDE systems," J. Process Control, vol. 22, no. 7, pp. 1161-1170, 2012.
67. H. R. Maei, C. Szepesvári, S. Bhatnagar, D. Precup, D. Silver, and R. S. Sutton, "Convergent temporal-difference learning with arbitrary smooth function approximation," in Advances in Neural Information Processing Systems. Red Hook, NY, USA: Curran & Associates Inc., 2009, pp. 1204-1212.
68. H. K. Khalil and J. Grizzle, Nonlinear Systems, vol. 3. Upper Saddle River, NJ, USA: Prentice-Hall, 2002.