Approximate dynamic programming based approach to process control and scheduling

Computers and Chemical Engineering

Volumn 30, Issue 10-12, 2006, Pages 1603-1618

  Lee, Jay H ^a   Lee, Jong Min ^b  

^a Georgia Institute of Technology   (United States)

^b UNIVERSITY OF VIRGINIA   (United States)

Author keywords

Approximate dynamic programming; Function approximation; Markov decision process

Indexed keywords

APPROXIMATION THEORY; COMPUTER SIMULATION; DYNAMIC PROGRAMMING; MARKOV PROCESSES; PROCESS CONTROL; STOCHASTIC CONTROL SYSTEMS;

APPROXIMATE DYNAMIC PROGRAMMING (ADP); FUNCTION APPROXIMATION; MARKOV DECISION PROCESS; SCHEDULING PROBLEMS;

PROCESS ENGINEERING;

APPROXIMATION THEORY; COMPUTER SIMULATION; DYNAMIC PROGRAMMING; MARKOV PROCESSES; PROCESS CONTROL; PROCESS ENGINEERING; STOCHASTIC CONTROL SYSTEMS;

EID: 33747872589     PISSN: 00981354     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.compchemeng.2006.05.043     Document Type: Article

References (59)

1. Ahn S.-M., Park M.-J., and Rhee H.-K. Extended Kalman filter-based nonlinear model predictive control for a continuous MMA polymerization reactor. Industrial & Engineering Chemistry Research 38 (1999) 3942-3949
2. Anderson B.D. Adaptive systems, lack of persistency of excitation and bursting phenomena. Automatica 21 3 (1985) 247-258
3. Astrom K.J., and Helmersson A. Dual control of an integrator with unknown gain. Computers and Mathematics with Applications 12A 6 (1986) 653-662
4. Astrom K.J., and Wittenmark B. Problems of identification and control. Journal of Mathematical Analysis and Applications 34 (1971) 90-113
5. Aufderheide B., and Bequette B.W. Extension of dynamic matrix control to multiple models. Computers & Chemical Engineering 27 (2003) 1079-1096
6. Bellman R.E. Dynamic programming (1957), Princeton University Press, New Jersey
7. Bemporad A., Morari M., Dua V., and Pistikopoulos E.N. The explicit linear quadratic regulator for constrained systems. Automatica 38 (2002) 3-20
8. Bertsekas D.P. Dynamic programming and optimal control. 2nd ed. (2000), Athena Scientific, Belmont, MA
9. Bertsekas D.P., and Tsitsiklis J.N. Neuro-dynamic programming (1996), Athena Scientific, Belmont, Massachusetts
10. Birge J. Decomposition and partitioning methods for multistage stochastic linear programs. Operational Research 33 (1985) 989-1007
11. Birge J., and Louveaux F. A multicit algorithm for two-stage stochastic linear programs. European Journal of Operational Research 34 3 (1988) 384-392
12. Blazewicz J., Lenstra J.K., and Kan A. Scheduling subject to resource contraints: Classification and complexity. Discrete Applied Mathematics 5 (1983) 11-24
13. Bradtke S.J. Reinforcement learning applied to linear quadratic regulation. In: Hanson S.J., Cowan J., and Giles C.L. (Eds). Advances in neural information processing systems 5 (1993), Morgan Kaufmann 295-302
14. Cacoullos T. Estimation of a multivariate density. Annals of the Institute of Statistical Mathematics 18 2 (1966) 179-189
15. Chikkula Y., Lee J.H., and Ogunnaike B.A. Dynamically scheduled MPC of nonlinear processes using hinging hyperplane models. AIChE Journal 44 2 (1998) 2658-2674
16. Choi, J. (2004). Algorithmic framework for improving heuristics in stochastic, stage-wise optimization problems. Ph.D. dissertation. Georgia Institute of Technology.
17. Choi J., Realff M.J., and Lee J.H. Dynamic programming in a heuristically confined state space: A stochastic resource-constrained project scheduling application. Computers & Chemical Engineering 28 (2004) 1039-1058
18. Choi J., Realff M.J., and Lee J.H. An algorithmic framework for improving heuristics. Part I: A deterministic discount coupon travelling salesman problem. Computers & Chemical Engineering 28 (2004) 1285-1296
19. Choi J., Lee J.H., and Realff M.J. An algorithmic framework for improving heuristics. Part II: A new version of the stochastic travelling salesman problem. Computers & Chemical Engineering 28 (2004) 1297-1307
20. De Farias D.P., and Van Roy B. The linear programming approach to approximate dynamic programming. Operations Research 51 6 (2003) 850-865
21. Fel'dbaum A.A. Dual control theory I-IV. Automation Remote Control 21-22 (1960-1961) 874-880 1453-1464, 1-12, 109-121
22. Gulaian M., and Lane J. Titration curve estimation for adaptive pH control. Proceedings of the American Control Conference. San Diego, CA (1990) 1414-1419
23. Hedengren J.D., and Edgar T.F. Order reduction of large scale DAE models. Computers and Chemical Engineering 29 (2005) 2069-2077
24. Higle J., and Sen S. Stochastic decomposition: An algorithm for two stage stochastic linear programs with recourse. Mathematics of Operations Research 16 (1991) 650-669
25. Howard R.A. Dynamic programming and markov processes (1960), MIT Press, Cambridge, MA
26. Infanger G. Planning under uncertainty: Solving large scale stochastic linear programs (1994), Boyd and Fraser Publishing Co.
27. Kaisare N.S., Lee J.M., and Lee J.H. Simulation based strategy for nonlinear optimal control: Application to a microbial cell reactor. International Journal of Nonlinear and Robust Control 13 (2003) 347-363
28. Kall P., and Wallace S.W. Stochastic programming (1994), John Wiley & Sons, Chichester, England
29. Kondili E., Pantelides C., and Sargent R. A general algorithm for short-term scheduling of batch operations-I. Milp formulation. Computers & Chemcial Engineering 17 (1993) 211-227
30. Kothare M.V., Mettler B., Morari M., Bendotti P., and Falinower C.M. Level control in the steam generator of a nuclear power plant. IEEE Transactions on Control Systems Technology 8 1 (2000) 55-69
31. Lee J.M., and Lee J.H. Simulation-based dual mode controller for nonlinear processes. Proceedings of 7th International Symposium on Advanced Control of Chemical Processes. Hong Kong, China (2004) 225-230
32. Lee J.M., and Lee J.H. Approximate dynamic programming based approaches for input-output data-driven control of nonlinear processes. Automatica 41 7 (2005) 1281-1288
33. Lee J.H., and Ricker N.L. Extended Kalman filter based nonlinear model predictive control. Industrial & Engineering Chemistry Research 33 6 (1994) 1530-1541
34. Lee J.M., Kaisare N.S., and Lee J.H. Choice of approximator and design of penalty function for an approximate dynamic programming based control approach. Journal of Process Control 16 2 (2006) 135-156
35. Lindoff B., Holst J., and Wittenmark B. Analysis of approximations of dual control. International Journal of Adaptive Control and Signal Processing 13 (1999) 593-620
36. Luyben W.L. Process modeling, simulation, and control for chemical engineers (1990), McGraw-Hill, New York, NY
37. Maitelli A.L., and Yoneyama T. A multistage suboptimal dual controller using optimal predictors. IEEE Transactions on Automatic Control 44 5 (May 1999)
38. Marbach P., and Tsitsiklis J.N. Simulation-based optimization of Markov reward processes. IEEE Transactions on Automatic Control 46 2 (2001) 191-209
39. Mayne D.Q., Rawlings J.B., Rao C.V., and Scokaert P.O.M. Constrained model predictive control: Stability and optimality. Automatica 36 (2000) 789-814
40. Morari M., and Lee J. Model predictive control: Past, present and future. Computers & Chemcial Engineering 23 4 (1999) 667-682
41. Nocedal J., and Wright S.J. Numerical optimization (1999), Springer-Verlag, New York
42. Pannocchia, G., Rawlings, J. B., & Wright, S. J. (in press). Fast, large-scale model predictive control by partial enumeration. Automatica.
43. Peroni C.V., Kaisare N., and Lee J.H. Optimal control of a fed batch bioreactor using simulation-based dynamic programming. IEEE Transactions in Control Systems Technology 13 (2005) 786-790
44. Pratikakis, N., Realff, M. J., & Lee, J. H. (in press). Strategic capacity decisions in manufacturing using real-time adaptive dynamic programming. Naval Research Logistics.
45. Puterman M.L. Markov decision processes (1994), Wiley, New York, NY
46. Qin J., and Badgwell T. A survey of industrial model predictive control technology. Control Engineering Practice 11 (2003) 733-764
47. Rockafellar R., and Wets R. Scenarios and policy aggregation in optimization under uncertainty. Mathematics of Operational Research 16 1 (1991) 119-147
48. Schraudolph N.N., Dayan P., and Sejnowski T.J. Temporal difference learning of position evaluation in the game of Go. In: Cowan J.D., Tesauro G., and Alspector J. (Eds). Advances in neural information processing systems 6 (1994), Morgan Kaufmann, San Mateo, CA 817-824
49. Shamma J.S., and Athans M. Gain scheduling: Potential hazards and possible remedies. IEEE Control Systems Magazine 12 (1992) 101-107
50. Shinskey F.G. Distillation control: For productivity and energy conservation (1977), McGraw-Hill, New York, NY
51. Si J., Barto A.G., Powell W.B., and Wunsch D. Handbook of learning and approximate dynamic programming (2004), Wiley-IEEE Press
52. Singh S., and Bertsekas D. Reinforcement learning for dynamic channel allocation in cellular telephone systems. In: Mozer M.C., Jordan M.I., and Petsche T. (Eds). Advances in neural information processing systems 9 (1997), MIT Press 974-980
53. Smith S. Reactive scheduling systems. In: Brown D.E., and Scherer W.T. (Eds). Intelligent scheduing systems (1995), Kluwer Press
54. Subramanian D., Pekny J., and Reklaitis G. A simulation-optimization framework for addressing combinatorial and stochastic aspects of an R&D pipeline management problem. Computers & Chemical Engineering 24 (2000) 1005-1011
55. Sutton R., McAllester D., Singh S., and Mansour Y. Policy gradient methods for reinforcement learning with function approximation. In: Solla S.A., Leen T.K., and Muller K.-R. (Eds). Advances in neural information processing systems 12 (2000), MIT Press 1057-1063
56. Tesauro G.J. Practical issues in temporal-difference learning. Machine Learning 8 (1992) 257-277
57. Wenk C.J., and Bar-Shalom Y. A multiple model adaptive dual control algorithm for stochastic systems with unknown parameters. IEEE Transactions on Automatic Control AC-25 (1980) 703-710
58. Ying Y., Rao M., and Sun Y. Bilinear control strategy for paper-making process. Chemical Engineering Communications 111 (1992) 13-28
59. Zhang W., and Dietterich T.G. A reinforcement learning approach to job-shop scheduling. Proceedings of the Twelfth International Conference on Machine Learning (ICML'95). San Francisco, CA: Morgan Kaufmann (1995) 1114-1120