Online policy iteration algorithm for optimal control of linear hyperbolic PDE systems

Journal of Process Control

Volumn 22, Issue 7, 2012, Pages 1161-1170

  Luo, Biao ^a   Wu, Huai Ning ^a  

^a BEIHANG UNIVERSITY   (China)

Author keywords

Convergence; Distributed optimal control; Hyperbolic PDE systems; Policy iteration; Space dependent Riccati differential equation

Indexed keywords

CONVERGENCE; DISTRIBUTED OPTIMAL CONTROL; HYPERBOLIC PDES; POLICY ITERATION; RICCATI DIFFERENTIAL EQUATION;

ALGORITHMS; CONTROL; DISTRIBUTED PARAMETER CONTROL SYSTEMS; LEAST SQUARES APPROXIMATIONS; ONLINE SYSTEMS; PARTIAL DIFFERENTIAL EQUATIONS; REINFORCEMENT LEARNING;

OPTIMAL CONTROL SYSTEMS;

EID: 84864324494     PISSN: 09591524     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jprocont.2012.04.011     Document Type: Article

References (47)

1. W.H. Ray Advanced Process Control 1981 McGraw-Hill New York
2. P.D. Christofides Nonlinear and Robust Control of PDE Systems: Methods and Applications to Transport-Reaction Processes 2001 Birkhäuser Boston, MA
3. J.L. Lions Optimal Control of Systems Governed by Partial Differential Equations 1971 Springer Verlag New York
4. R.F. Curtain, and H.J. Zwart An Introduction to Infinite-dimensional Linear Systems Theory 1995 Springer Verlag New York
5. A. Bensoussan, G. Da Prato, M.C. Delfour, and S.K. Mitter Representation and Control of Infinite Dimensional Systems Second ed. 2007 Birkhäuser Boston
6. B. Luo, H.-N. Wu, Approximate optimal control design for nonlinear one-dimensional parabolic PDE systems using empirical eigenfunctions and neural network, IEEE Transactions on Systems, Man, and Cybernetics. Part B, in press.
7. M. Li, and P.D. Christofides Optimal control of diffusion-convection- reaction processes using reduced-order models Computers and Chemical Engineering 32 9 2008 2123 2135
8. C. Xu, Y. Ou, and E. Schuster Sequential linear quadratic control of bilinear parabolic PDEs based on POD model reduction Automatica 47 2 2011 418 426
9. V. Yadav, R. Padhi, and S.N. Balakrishnan Robust/optimal temperature profile control of a high-speed aerospace vehicle using neural networks IEEE Transactions Neural Network 18 4 2007 1115 1128 (Pubitemid 47098885)
10. R. Padhi, S.N. Balakrishnan, and T. Randolph Adaptive-critic based optimal neuro control synthesis for distributed parameter systems Automatica 37 8 2001 1223 1234 (Pubitemid 32610253)
11. M.J. Balas Finite dimensional control of distributed parameter systems by Galerkin approximation of infinite dimensional controller Journal of Mathematical Analysis and Applications 114 1 1986 17 36
12. I.Y. Smets, D. Dochain, and J.F. Van Impe Optimal temperature control of a steady-state exothermic plug-flow reactor AIChE Journal 48 2 2002 279 286 (Pubitemid 37442376)
13. I. Aksikas, J.J. Winkin, and D. Dochain Optimal LQ-feedback regulation of a nonisothermal plug flow reactor model by spectral factorization IEEE Transactions on Automatic Control 52 7 2007 1179 1193 (Pubitemid 47098931)
14. I. Aksikas, A. Fuxman, J.F. Forbes, and J.J. Winkin LQ control design of a class of hyperbolic PDE systems: application to fixed-bed reactor Automatica 45 6 2009 1542 1548
15. M. Krstic, and A. Smyshlyaev Backstepping boundary control for first-order hyperbolic PDEs and application to systems with actuator and sensor delays Systems Control Letters 57 9 2008 750 758
16. J.M. Coron, B. D'Andrea-Novel, and G. Bastin A strict Lyapunov function for boundary control of hyperbolic systems of conservation laws IEEE Transactions on Automatic Control 52 1 2007 2 11 (Pubitemid 46343993)
17. A. Diagne, G. Bastin, and J.M. Coron Lyapunov exponential stability of 1D linear hyperbolic systems of balance laws Automatica 48 1 2011 109 114
18. H. Sira-Ramirez Distributed sliding mode control in systems described by quasilinear partial differential equations Systems Control Letters 13 2 1989 177 181 (Pubitemid 20721277)
19. H. Shang, J.F. Forbes, and M. Guay Model predictive control for quasilinear hyperbolic distributed parameter systems Industrial and Engineering Chemistry Research 43 9 2004 2140 2149 (Pubitemid 38507182)
20. P.D. Christofides, and P. Daoutidis Feedback control of hyperbolic PDE systems AIChE Journal 42 11 1996 3063 3086
21. J.-W. Wang, H.-N. Wu, and H.-X. Li Distributed fuzzy control design of nonlinear hyperbolic PDE systems with application to nonisothermal plug-flow reactor IEEE Transactions on Fuzzy Systems 19 3 2011 514 526
22. H.-X. Li, and C.-K. Qi Spatio-Temporal Modeling of Nonlinear Distributed Parameter Systems: A Time/Space Separation Based Approach 2011 Springer Verlag New York
23. T. Okita, Y. Kobayashi, and S. Tanaka An identification of hyperbolic distributed parameter systems using transient response Electrical Engineering in Japan 101 3 1981 131 138
24. H. Zhang, and Y. Quan Modeling, identification, and control of a class of nonlinear systems IEEE Transactions on Fuzzy Systems 9 2 2001 349 354
25. L.P. Kaelbling, M.L. Littman, and A.W. Moore Reinforcement learning: a survey Journal of Artificial Intelligence Research 4 1996 237 285 (Pubitemid 126646155)
26. R.S. Sutton, and A.G. Barto Reinforcement Learning: An Introduction 1998 MIT Press Cambridge, Mass
27. J. Morimoto, and K. Doya Robust reinforcement learning Neural Computation 17 2 2005 335 359 (Pubitemid 40194970)
28. F.L. Lewis, and D. Vrabie Reinforcement learning and adaptive dynamic programming for feedback control IEEE Circuits and Systems Magazine 9 3 2009 32 50
29. W.B. Powell Approximate Dynamic Programming: Solving the Curses of Dimensionality 2007 Wiley-Interscience Hoboken, N.J
30. J. Si, and Y. Wang Online learning control by association and reinforcement IEEE Transactions Neural Network 12 2 2001 264 276 (Pubitemid 32371483)
31. D. Vrabie, O. Pastravanu, M. Abu-Khalaf, and F.L. Lewis Adaptive optimal control for continuous-time linear systems based on policy iteration Automatica 45 2 2009 477 484
32. D. Vrabie, and F.L. Lewis Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems Neural Networks 22 3 2009 237 246
33. K.G. Vamvoudakis, and F.L. Lewis Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem Automatica 46 5 2010 878 888
34. J.J. Murray, C.J. Cox, G.G. Lendaris, and R. Saeks Adaptive dynamic programming IEEE Transactions on Systems, Man, and Cybernetics. Part C 32 2 2002 140 153 (Pubitemid 35289398)
35. D.L. Kleinman On an iterative technique for Riccati equation computations IEEE Transactions on Automatic Control 13 1 1968 114 115
36. M. Abu-Khalaf, and F.L. Lewis Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach Automatica 41 5 2005 779 791 (Pubitemid 40352391)
37. K. Vamvoudakis, D. Vrabie, and F. Lewis Online policy iteration based algorithms to solve the continuous-time infinite horizon optimal control problem IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning Nashville, TN 2009 36 41
38. H. Zhang, Q. Wei, and Y. Luo A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm IEEE Transactions on Systems, Man, and Cybernetics. Part B 38 4 2008 937 942
39. H. Zhang, L. Cui, X. Zhang, and Y. Luo Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method IEEE Transactions Neural Network 22 2011 2226 2236 (12, part: 2)
40. I. Karafyllis, and P. Daoutidis Control of hot spots in plug flow reactors Computers and Chemical Engineering 26 7-8 2002 1087 1094 (Pubitemid 36076782)
41. J.J. Winkin, D. Dochain, and P. Ligarius Dynamical analysis of distributed parameter tubular reactors Automatica 36 3 2000 349 361 (Pubitemid 30561637)
42. H.K. Rhee, R. Aris, and N.R. Amundson First-Order Partial Differential Equations: Vols. I and II 1986 Prentice Hall New Jersey
43. J. Smoller Shock Waves and Reaction-Diffusion Equations 1983 Springer-Verlag Berlin
44. R.F. Curtain, and A.J. Pritchard Functional Analysis in Modern Applied Mathematics 1977 Academic Press London, New York
45. S. Dubljevic, P. Mhaskar, N.H. El-Farra, and P.D. Christofides Predictive control of transport-reaction processes Computers and Chemical Engineering 29 11-12 2005 2335 2345 (Pubitemid 41660481)
46. E.F. Camacho, and C. Bordons Model Predictive Control 1999 Springer-Verlag London
47. D. Ernst, M. Glavic, F. Capitanescu, and L. Wehenkel Reinforcement learning versus model predictive control: a comparison on a power system problem IEEE Transactions on Systems, Man, and Cybernetics. Part B 39 2 2009 517 529