Data-based suboptimal neuro-control design with reinforcement learning for dissipative spatially distributed processes

Industrial and Engineering Chemistry Research

Volumn 53, Issue 19, 2014, Pages 8106-8119

  Luo, Biao ^a,b   Wu, Huai Ning ^a   Li, Han Xiong ^c,d  

^a BEIHANG UNIVERSITY   (China)

^b INSTITUTE OF AUTOMATION   (China)

^c CITY UNIVERSITY OF HONG KONG   (Hong Kong)

^d CENTRAL SOUTH UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

COST FUNCTIONS; DATA REDUCTION; EIGENVALUES AND EIGENFUNCTIONS; ITERATIVE METHODS; NONLINEAR EQUATIONS; ORDINARY DIFFERENTIAL EQUATIONS; PARTIAL DIFFERENTIAL EQUATIONS; PERTURBATION TECHNIQUES; REINFORCEMENT LEARNING; SPATIAL DISTRIBUTION;

EMPIRICAL EIGENFUNCTIONS; HAMILTON JACOBI BELLMAN; KARHUNEN-LOEVE DECOMPOSITION; NONLINEAR PARTIAL DIFFERENTIAL EQUATIONS; SEMI-GLOBALLY UNIFORMLY ULTIMATELY BOUNDED; SINGULAR PERTURBATION THEORY; SPATIALLY DISTRIBUTED PROCESS; SUBOPTIMAL CONTROL DESIGN;

ADAPTIVE CONTROL SYSTEMS;

EID: 84988290534     PISSN: 08885885     EISSN: 15205045     Source Type: Journal    
DOI: 10.1021/ie4031743     Document Type: Article

References (59)

1. Curtain, R. F. and Zwart, H. J. An Introduction to Infinite-dimensional Linear Systems Theory; Springer Verlag: New York, 1995.
2. Luo, B.; Wu, H.-N. Online policy iteration algorithm for optimal control of linear hyperbolic PDE systems J. Process Control 2012, 22, 1161-1170
3. Yadav, V.; Padhi, R.; Balakrishnan, S. N. Robust/optimal temperature profile control of a high-speed aerospace vehicle using neural networks IEEE Trans. Neural Networks 2007, 18, 1115-1128
4. Li, M.; Christofides, P. D. Optimal control of diffusion-convection- reaction processes using reduced-order models Comput. Chem. Eng. 2008, 32, 2123-2135
5. Luo, B.; Wu, H.-N. Approximate optimal control design for nonlinear one-dimensional parabolic PDE systems using empirical eigenfunctions and neural network IEEE Trans. Syst. Man Cybern. Part B 2012, 42, 1538-1549
6. Armaou, A.; Christofides, P. D. Dynamic optimization of dissipative PDE systems using nonlinear order reduction Chem. Eng. Sci. 2002, 57, 5083-5114
7. Dubljevic, S.; Christofides, P. D. Predictive output feedback control of parabolic partial differential equations (PDEs) Ind. Eng. Chem. Res. 2006, 45, 8421-8429
8. Aksikas, I.; Winkin, J. J.; Dochain, D. Optimal LQ-feedback regulation of a nonisothermal plug flow reactor model by spectral factorization IEEE Trans. Automat. Control 2007, 52, 1179-1193
9. Iftime, O. V.; M. A. Demetriou, M. A. Optimal control of switched distributed parameter systems with spatially scheduled actuators Automatica 2009, 45, 312-323
10. Wu, H.-N.; Luo, B. Heuristic dynamic programming algorithm for optimal control design of linear continuous-time hyperbolic PDE systems Ind. Eng. Chem. Res. 2012, 51, 9310-9319
11. Christofides, P. D. Control of nonlinear distributed process systems: Recent developments and challenges AIChE J. 2001, 47, 514-518
12. Dubljevic, S.; El-Farra, N. H.; Mhaskar, P.; Christofides, P. D. Predictive control of parabolic PDEs with state and control constraints Int. J. Robust Nonlinear Control 2006, 16, 749-772
13. Dubljevic, S.; Mhaskar, P.; El-Farra, N. H.; Christofides, P. D. Predictive control of transport-reaction processes Comput. Chem. Eng. 2005, 29, 2335-2345
14. Li, M. H.; Christofides, P. D. An input/output approach to the optimal transition control of a class of distributed chemical reactors Chem. Eng. Sci. 2007, 62, 2979-2988
15. Sutton, R. S.; Barto, A. G. Reinforcement Learning: An Introduction; MIT Press: Cambridge, MA, 1998.
16. Bertsekas, D. P.; Tsitsiklis, J. N. Neuro-Dynamic Programming; Athena Scientific: Belmont, MA, 1996.
17. Hafner, R.; Riedmiller, M. Reinforcement learning in feedback control Mach. Learn. 2011, 84, 137-169
18. Hernandez-del-Olmo, F.; Gaudioso, E.; Nevado, A. Autonomous adaptive and active tuning up of the dissolved oxygen setpoint in a wastewater treatment plant using reinforcement learning IEEE Trans. Syst. Man Cybern. Part C 2012, 42, 768-774
19. Kaelbling, L. P.; Littman, M. L.; Moore, A. W. Reinforcement learning: A survey J. Artif. Intell. Res. 1996, 4, 237-285
20. Busoniu, L.; Babuska, R.; De Schutter, B. A comprehensive survey of multiagent reinforcement learning IEEE Trans. Syst. Man Cybern., Part C 2008, 38, 156-172
21. Doya, K. Reinforcement learning in continuous time and space Neural Comput. 2000, 12, 219-245
22. Grondman, I.; Busoniu, L.; Lopes, G. A. D.; Babuska, R. A survey of actor-critic reinforcement learning: standard and natural policy gradients IEEE Trans. Syst. Man Cybern. Part C 2012, 42, 1291-1307
23. Powell, W. B. Approximate Dynamic Programming: Solving the Curses of Dimensionality; Wiley-Interscience: Hoboken, NJ, 2007.
24. Tsitsiklis, J. N.; Van Roy, B. An analysis of temporal-difference learning with function approximation IEEE Trans. Automat. Control 1997, 42, 674-690
25. Lewis, F. L.; Vrabie, D. Reinforcement learning and adaptive dynamic programming for feedback control IEEE Circ. Syst. Mag. 2009, 9, 32-50
26. He, P.; Jagannathan, S. Reinforcement learning neural-network-based controller for nonlinear discrete-time systems with input constraints IEEE Trans. Syst. Man Cybern. Part B 2007, 37, 425-436
27. Al-Tamimi, A.; Lewis, F. L.; Abu-Khalaf, M. Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof IEEE Trans. Syst. Man Cybern. Part B 2008, 38, 943-949
28. Lu, C.; Si, J.; Xie, X. R. Direct heuristic dynamic programming for damping oscillations in a large power system IEEE Trans. Syst. Man Cybern. Part B 2008, 38, 1008-1013
29. Zhang, H.; Song, R.; Wei, Q.; Zhang, T. Optimal tracking control for a class of nonlinear discrete-time systems with time delays based on heuristic dynamic programming IEEE Trans. Neural Netw. 2011, 22, 1851-1862
30. Tosukhowong, T.; Lee, J. H. Approximate dynamic programming based optimal control applied to an integrated plant with a reactor and a distillation column with recycle AIChE J. 2009, 55, 919-930
31. Wang, F.; Jin, N.; Liu, D.; Wei, Q. Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound IEEE Trans. Neural Netw. 2011, 22, 24-36
32. Lewis, F. L.; Vamvoudakis, K. G. Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data IEEE Trans. Syst. Man Cybern. Part B 2011, 41, 14-25
33. Yang, Q.; Jagannathan, S. Reinforcement learning controller design for affine nonlinear discrete-time systems using online approximators IEEE Trans. Syst. Man Cybern. Part B 2012, 42, 377-390
34. Si, J.; Wang, Y. Online learning control by association and reinforcement IEEE Trans. Neural Netw. 2001, 12, 264-276
35. Murray, J. J.; Cox, C. J.; Lendaris, G. G.; Saeks, R. Adaptive dynamic programming IEEE Trans. Syst. Man Cybern. Part C 2002, 32, 140-153
36. Vrabie, D.; Pastravanu, O.; Abu-Khalaf, M.; Lewis, F. L. Adaptive optimal control for continuous-time linear systems based on policy iteration Automatica 2009, 45, 477-484
37. Vrabie, D.; Lewis, F. L. Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems Neural Netw. 2009, 22, 237-246
38. Vamvoudakis, K. G.; Lewis, F. L. Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem Automatica 2010, 46, 878-888
39. Lee, T. T.; Wang, F. Y.; Newell, R. B. Advances in distributed parameter approach to the dynamics and control of activated sludge processes for wastewater treatment Water Res. 2006, 40, 853-869
40. Baker, J.; Christofides, P. D. Output feedback control of parabolic PDE systems with nonlinear spatial differential operators Ind. Eng. Chem. Res. 1999, 38, 4372-4380
41. Armaou, A.; Christofides, P. D. Crystal temperature control in the Czochralski crystal growth process AIChE J. 2001, 47, 79-106
42. Holmes, P.; Lumley, J. L.; Berkooz, G. Turbulence, Coherent Structures, Dynamical Systems and Symmetry; Cambridge University Press: Cambridge, UK, 1996.
43. Qi, C.-K.; Li, H.-X.; Li, S.; Zhao, X.; Gao, F. Probabilistic PCA-based spatiotemporal multimodeling for nonlinear distributed parameter processes Ind. Eng. Chem. Res. 2012, 51, 6811-6822
44. Pitchaiah, S.; Armaou, A. Output feedback control of dissipative PDE systems with partial sensor information based on adaptive model reduction AIChE J. 2013, 59, 747-760
45. Pitchaiah, S.; Armaou, A. Output feedback control of distributed parameter systems using adaptive proper orthogonal decomposition Ind. Eng. Chem. Res. 2010, 49, 10496-10509
46. Ho, C.; Tai, Y. REVIEW: MEMS and its applications for flow control ASME J. Fluid Eng. 1996, 118, 437-447
47. Bamieh, B.; Paganini, F.; Dahleh, M. A. Distributed control of spatially invariant systems IEEE Trans. Automat. Control 2002, 47, 1091-1107
48. Wolff, J.; Papathanasiou, A. G.; Rotermund, H. H.; Ertl, G.; Li, X.; Kevrekidis, I. G. Local manipulation of catalytic surface reactivity J. Catal. 2003, 216, 246-256
49. Christofides, P. D.; Daoutidis, P. Finite-dimensional control of parabolic PDE systems using approximate inertial manifolds J. Math. Anal. Appl. 1997, 216, 398-420
50. Christofides, P. D. Robust control of parabolic PDE systems Chem. Eng. Sci. 1998, 53, 2949-2965
51. Saridis, G. N.; Lee, C. G. An approximation theory of optimal control for trainable manipulators IEEE Trans. Syst. Man Cybern. Part B 1979, 9, 152-159
52. Courant, R.; Hilbert, D. Methods of Mathematical Physics; Wiley-VCH: Hoboken, NJ, 1989; Vol. 1.
53. Sastry, S.; Bodson, M. Adaptive Control: Stability, Convergence, and Robustness; Prentice-Hall, Inc.: Upper Saddle River, NJ, 1989.
54. Ioannou, P. A.; Sun, J. Robust adaptive control; PTR Prentice-Hall: Upper Saddle River, NJ, 1996.
55. Wang, C.; Hill, D. J. Learning from neural control IEEE Trans. Neural Netw. 2006, 17, 130-146
56. Bitmead, R. Persistence of excitation conditions and the convergence of adaptive schemes IEEE Trans. Inf. Theory 1984, 30, 183-191
57. Watkins, C. J. C. H.; Dayan, P. Q-learning Mach. Learn. 1992, 8, 279-292
58. Ge, S. S.; Hang, C. C.; Lee, T. H.; Zhang, T. Stable Adaptive Neural Network Control; Kluwer: Boston, MA, 2001.
59. Khalil, H. K. Nonlinear Systems, 3rd ed, Prentice-Hall: Upper Saddle River, NJ, 2002.