Learning surrogate models for simulation-based optimization

AIChE Journal

Volumn 60, Issue 6, 2014, Pages 2211-2227

  Cozad, Alison ^a,b   Sahinidis, Nikolaos V ^a,b   Miller, David C ^a  

^a NATIONAL ENERGY TECHNOLOGY LABORATORY   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Design (process simulation); Machine learning; Optimization

Indexed keywords

ALGEBRA; EXPERIMENTS; FUEL ADDITIVES; INTEGER PROGRAMMING; LEARNING SYSTEMS; OPTIMIZATION;

COMPUTATIONAL COMPARISONS; COMPUTATIONAL IMPLEMENTATIONS; DERIVATIVE-FREE OPTIMIZATION; DESIGN (PROCESS SIMULATION); INTEGER PROGRAMMING FORMULATIONS; LEAST SQUARES REGRESSION; MACHINE LEARNING TECHNIQUES; SIMULATION-BASED OPTIMIZATIONS;

COMPUTER SIMULATION;

EID: 84899948547     PISSN: 00011541     EISSN: 15475905     Source Type: Journal    
DOI: 10.1002/aic.14418     Document Type: Article

References (52)

1. Seader JD, Seider WD, Pauls AC, Hughes RR. FLOWTRAN Simulation: An Introduction. Cambridge, MA: CACHE, 1977.
2. Pantelides CC. SpeedUp-recent advances in process simulation. Comput Chem Eng. 1988;12:745-755.
3. Evans LB, Boston JF, Britt HI, Gallier PW, Gupta PK, Joseph B, Mahalec V, Ng E, Seider WD, Yagi H. ASPEN: an advanced system for process engineering. Comput Chem Eng. 1979;3:319-327.
4. Turton R, Bailie RC, Whiting WB, Shaeiwitz JA. Modeling and Simulation in Chemical Engineering. Upper Saddle River, NJ: Pearson Education, 2008.
5. Fu MC. Optimization for simulation: theory vs. practice. INFORMS J Comput. 2002;14:192-215.
6. April J, Glover F, Kelly JP, Laguna M. Practical introduction to simulation optimization. Proceedings of the 2003 Winter Simulation Conference, 2003, IEEE, New Orleans, LA, Vol. 1. 2003:71-78.
7. Fu MC, Glover FW, April J. Simulation optimization: a review, new developments, and applications. Proceedings of the Winter Simulation Conference, 2005. IEEE, Austin, TX, 2005:13.
8. Drud A, ARKI Consulting and Development A. CONOPT 3 Solver Manual. 2003. Available at http://www.gams.com/dd/docs/solvers/conopt.pdf. Last accessed January 15, 2014.
9. Wächter A, Biegler LT. On the implementation of a primal-dual interior point filter line search algorithm for large-scale nonlinear programming. Math Program. 2006;106:25-57.
10. Gill PE, Murray W, Saunders MA. User's Guide for SNOPT Version 7: Software for Large-Scale Nonlinear Programming. 2008. Available at http://www.sbsi-sol-optimize.com/manuals/SNOPT%20Manual.pdf. Last accessed January 15, 2014.
11. Tawarmalani M, Sahinidis NV. A polyhedral branch-and-cut approach to global optimization. Math Program. 2005;103:225-249.
12. Chen W, Shao Z, Qian J. Interfacing IPOPT with Aspen open solvers and CAPE-OPEN. 10th International Symposium on Process Systems Engineering: Part A. Vol. 27. Amsterdam, The Netherlands: Elsevier, 2009:201-206.
13. Caballero J, Grossmann IE. An algorithm for the use of surrogate models in modular flowsheet optimization. AIChE J. 2008;54:2633-2650.
14. Jones DR, Schonlau M, Welch WJ. Efficient global optimization of expensive black-box functions. J Glob Optim. 1998;13:455-492.
15. Rios LM, Sahinidis NV. Derivative-free optimization: a review of algorithms and comparison of software implementations. J Glob Optim. 2013;56:1247-1293.
16. Jones DR. A taxonomy of global optimization methods based on response surfaces. J Glob Optim. 2001;21:345-383.
17. Wang GG, Shan S. Review of metamodeling techniques in support of engineering design optimization. J Mech Des. 2007;129:370-380.
18. Antoulas AC, Sorensen DC, Gugercin S. A survey of model reduction methods for large-scale systems. Contemp Math. 2001;280:193-218.
19. Palmer K, Realff M. Metamodeling approach to optimization of steady-state flowsheet simulations: model generation. Chem Eng Res Des. 2002;80:760-772.
20. Huang D, Allen T, Notz W, Zeng N. Global optimization of stochastic black-box systems via sequential kriging meta-models. J Glob Optim. 2006;34:441-466.
21. Davis E, Ierapetritou M. A kriging method for the solution of nonlinear programs with black-box functions. AIChE J. 2007;53:2001-2012.
22. Henao CA, Maravelias CT. Surrogate-based superstructure optimization framework. AIChE J. 2011;57:1216-1232.
23. Gorissen D, Crombecq K, Couckuyt I, Dhaene T, Demeester P. A surrogate modeling and adaptive sampling toolbox for computer based design. J Mach Learn Res. 2010;11:2051-2055.
24. Schmidt M, Lipson H. Distilling free-form natural laws from experimental data. Science. 2009;324:81-85.
25. McKay MD, Beckman RJ, Conover WJ. A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics. 1979;21:239-245.
26. Simpson TW, Peplinski J, Koch PN, Allen JK. Metamodels for computer-based engineering design: survey and recommendations. Eng Comput. 2001;17:129-150.
27. Krige DG. A statistical approach to some mine valuations and allied problems at the witwatersrand. Masters Thesis. South Africa: University of Witwatersrand, 1951.
28. Cressie N. Spatial prediction and ordinary kriging. Math Geol. 1988;20:405-421.
29. Hassoun M. Fundamentals of Artificial Neural Networks. Cambridge, MA: MIT Press, 1995.
30. Burnham KP, Anderson DR. Model selection and multimodel inference: A practical information-theoretic approach. New York: Springer, 2002.
31. Gatu C, Yanev PI, Kontoghiorghes EJ. A graph approach to generate all possible regression submodels. Comput Stat. 2007;52:799-815.
32. Gatu C, Kontoghiorghes EJ. Parallel algorithms for computing all possible subset regression models using the QR decomposition. Parallel Comput. 2003;29:505-521.
33. Tibshirani R. Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B Methodol. 1996;58:267-288.
34. Stoica P, Selén Y. Model-order selection: a review of information criterion rules. IEEE Signal Process Mag. 2004;21:36-47.
35. Akaike H. A new look at the statistical model identification. IEEE Trans Automat Control. 1974;19:716-723.
36. McQuarrie AD, Tsai CL. Regression and Time Series Model Selection. Singapore: World Scientific Publishing Company, 1998.
37. Hurvich CM, Tsai CL. A corrected Akaike information criterion for vector autoregressive model selection. J Time Ser Anal. 1993;14:271-279.
38. Von Zur Gathen J, Sieveking M. A bound on solutions of linear integer equalities and inequalities. Proc Am Math Soc. 1978;72:155-158.
39. Miller DC, Sahinidis NV, Cozad A, Lee A, Kim H, Morinelly J, Eslick J, Yuan Z. Computational tools for accelerating carbon capture process development. The 38th International Technical Conference on Clean Coal & Fuel Systems. Coal Technology Associated, Clearwater, FL, Vol. 1. 2013.
40. Fang K-T, Lin DKJ. Uniform experimental designs and their applications in industry. Chapter 4 in Statistics in Industry, edited by Khattree R, Rao CR, Elsevier, Amsterdam, The Netherlands, 2003;22:131-170.
41. Crombecq K, Laermans E, Dhaene T. Efficient space-filling and non-collapsing sequential design strategies for simulation-based modeling. Eur J Oper Res. 2011;214:683-696.
42. Crombecq K, Tommasi L, Gorissen D, Dhaene T. A novel sequential design strategy for global surrogate modeling. Winter Simulation Conference. IEEE. Austin, TX, Vol. 1. 2009:731-742.
43. Provost F, Jensen D, Oats T. Efficient progressive sampling. Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, San Diego, CA, Vol. 1. 1999:23-32.
44. Geest J, Dhaene T, Fach N, Zutter DD. Adaptive CAD-model building algorithm for general planar microwave structures. IEEE Trans Microwave Theory Tech. (Special Issue on Multilayer Microwave Circuits). 1999;9:1801-1809.
45. Thompson S. Sampling. Hoboken, NJ: Wiley, 2002.
46. Pukelsheim F. Optimal Design of Experiments. New York: Society for Industrial and Applied Mathematics, 2006.
47. Conn AR, Scheinberg K, Vicente LN. Introduction to Derivative-Free Optimization. Philadelphia, PA: Society for Industrial and Applied Mathematics, 2009.
48. Huyer W, Neumaier A. SNOBFIT-stable noisy optimization by branch and fit. ACM Trans Math Softw. 2008;9:1-25.
49. Neumaier A. MINQ-general definite and bound constrained indefinite quadratic programming. 1998. Available at http://www.mat.univie.ac.at/neum/software/minq/. Last accessed January 15, 2014.
50. Lee A, Miller DC. A one-dimensional, three region model for a bubbling fluidised bed adsorber. Ind Eng Chem Res. 2013;52:469-484.
51. NETL Power Systems Financial Model Version 5.0. 2008. Available at http://www.netl.doe.gov/business/solicitations/ssc2008/references/PSFM.
52. Conn AR, Scheinberg K, Vicente LN. Geometry of interpolation sets in derivative-free optimization. Math Program. 2008;111:141-172.