Choice of approximator and design of penalty function for an approximate dynamic programming based control approach

Journal of Process Control

Volumn 16, Issue 2, 2006, Pages 135-156

  Lee, Jong Min ^a   Kaisare, Niket S ^a   Lee, Jay H ^a  

^a Georgia Institute of Technology   (United States)

Author keywords

Approximate dynamic programming; k nearest neighbor; Neural network

Indexed keywords

APPROXIMATION THEORY; CONTROL SYSTEMS; EXTRAPOLATION; ITERATIVE METHODS; NEURAL NETWORKS;

APPROXIMATE DYNAMIC PROGRAMMING (ADP); FUNCTION APPROXIMATORS;

DYNAMIC PROGRAMMING;

EID: 27144544987     PISSN: 09591524     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jprocont.2005.04.010     Document Type: Article

References (35)

1. R.E. Bellman Dynamic Programming 1957 Princeton University Press New Jersey
2. R.S. Sutton, and A.G. Barto Toward a modern theory of adaptive networks: expectation and prediction Psycol. Rev. 88 2 1981 135 170
3. R.S. Sutton, and A.G. Barto Reinforcement Learning: An Introduction 1998 MIT Press Cambridge, MA
4. D.P. Bertsekas, and J.N. Tsitsiklis Neuro-dynamic Programming 1996 Athena Scientific Belmont, MA
5. N.S. Kaisare, J.M. Lee, and J.H. Lee Simulation based strategy for nonlinear optimal control: application to a microbial cell reactor Int. J. Robust Nonlinear Control 13 3-4 2002 347 363
6. J.M. Lee, and J.H. Lee Simulation-based learning of cost-to-go for control of nonlinear processes Korean J. Chem. Eng. 21 2 2004 338 344
7. J.M. Lee, J.H. Lee, An approximate dynamic programming based approach to dual adaptive control, Automatica, submitted.
8. J.M. Lee, J.H. Lee, Simulation-based dual mode controller for nonlinear processes, in: Preprints of 7th International Symposium on Advanced Control of Chemical Processes, IFAC, 2003, pp. 225-230.
9. G.J. Tesauro Practical issues in temporal-difference learning Mach. Learn. 8 1992 257 277
10. S.J. Bradtke Reinforcement learning applied to linear quadratic regulation S.J. Hanson J. Cowan C.L. Giles Advances in Neural Information Processing Systems vol. 5 1993 Morgan Kaufmann 295 302
11. N.N. Schraudolph, P. Dayan, and T.J. Sejnowski Temporal difference learning of position evaluation in the game of Go J.D. Cowan G. Tesauro J. Alspector Advances in Neural Information Processing Systems vol. 6 1994 Morgan Kaufmann San Mateo, CA 817 824
12. S. Thrun, and A. Schwartz Issues in using function approximation for reinforcement learning Proceedings of the Fourth Connectionist Models Summer School 1993 Lawrence Erlbaum Hillsdale, NJ 255 263
13. P. Sabes Approximating Q-values with basis function representations Proceedings of the Fourth Connectionist Models Summer School 1993 Lawrence Erlbaum Hillsdale, NJ
14. J.A. Boyan, and A.W. Moore Generalization in reinforcement learning: safely approximating the value function G. Tesauro D. Touretzky Advances in Neural Information Processing Systems vol. 7 1995 Morgan Kaufmann
15. R.S. Sutton Generalization in reinforcement learning: successful examples using sparse coarse coding D.S. Touretzky M.C. Mozer M.E. Hasselmo Advances in Neural Information Processing Systems vol. 8 1996 MIT Press 1038 1044
16. G.J. Gordon, Stable function approximation in dynamic programming, Tech. Rep. CMU-CS-95-103, School of Computer Science, Carnegie Mellon University, 1995.
17. J.N. Tsitsiklis, and B. Van Roy An analysis of temporal-difference learning with function approximation IEEE Trans. Automat. Control 42 5 1997 674 690
18. P.J. Werbos Consistency of HDP applied to a simple reinforcement learning problem Neural Networks 3 1990 179 189
19. D. Ormoneit, and S. Sen Kernel-based reinforcement learning Mach. Learn. 49 2002 161 178
20. D. Ormoneit, and P. Glynn Kernel-based reinforcement learning in average-cost problems IEEE Trans. Automat. Control 47 10 2002 1624 1636
21. W.D. Smart, and L.P. Kaelbling Practical reinforcement learning in continuous spaces Proceedings of 17th International Conference on Machine Learning 2000 Morgan Kaufmann San Francisco, CA 903 910
22. E. Parzen On estimation of a probability density function and mode Ann. Math. Statist. 33 1962 1065 1076
23. M.L. Puterman Markovian Decision Problems 1994 Wiley New York
24. R. Munos A convergent reinforcement learning algorithm in the continuous case based on a finite difference method Proceedings of the International Joint Conference on Artificial Intelligence 1997 Morgan Kaufmann 826 831
25. R.A. Howard Dynamic Programming and Markov Processes 1960 MIT Press Cambridge, MA
26. W. Zhang, and T.G. Dietterich A reinforcement learning approach to job-shop scheduling Proceedings of the Twelfth International Conference on Machine Learning (ICML'95) 1995 Morgan Kaufmann San Francisco, CA 1114 1120
27. S. Singh, and D. Bertsekas Reinforcement learning for dynamic channel allocation in cellular telephone systems M.C. Mozer M.I. Jordan T. Petsche Advances in Neural Information Processing Systems vol. 9 1997 MIT Press 974 980
28. H. Demuth, M. Beale, Neural Network Toolbox User's Guide (MATLAB), The MathWorks, Inc., Natick, MA, 2002.
29. J.H. Lee, and N.L. Ricker Extended Kalman filter based nonlinear model predictive control Ind. Eng. Chem. Res. 33 6 1994 1530 1541
30. M. Morari, N.L. Ricker, Model Predictive Control Toolbox User's Guide (MATLAB), The MathWorks, Inc., Natick, MA, 1995.
31. S.-M. Ahn, M.-J. Park, and H.-K. Rhee Extended Kalman filter-based nonlinear model predictive control for a continuous MMA polymerization reactor Indust. Eng. Chem. Res. 38 1999 3942 3949
32. B.W. Silverman Density Estimation for Statistics and Data Analysis 1986 Chapman and Hall
33. T. Cacoullos Estimation of a multivariate density Ann. Inst. Statist. Math. (Tokyo) 18 2 1966 179 189
34. J. Nocedal, and S.J. Wright Numerical Optimization 1999 Springer-Verlag New York
35. J.M. Lee, and J.H. Lee Approximate dynamic programming based approaches for input-output data-driven control of nonlinear processes Automatica 41 7 2005 1281 1288