Metamodeling in multidisciplinary design optimization: How far have we really come?

AIAA Journal

Volumn 52, Issue 4, 2014, Pages 670-690

  Viana, Felipe A C ^a   Simpson, Timothy W ^b   Balabanov, Vladimir ^c   Toropov, Vasilli ^d  

^a UNIVERSITY OF FLORIDA   (United States)

^b PENNSYLVANIA STATE UNIVERSITY   (United States)

^c BOEING COMPANY   (United States)

^d UNIVERSITY OF LEEDS   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

DESIGN AIDS;

COMMERCIAL SOFTWARE; COMPLEX SIMULATION; DESIGN AND ANALYSIS OF COMPUTER EXPERIMENTS; DESIGN SPACE EXPLORATION; FUTURE RESEARCH DIRECTIONS; META-MODELING TECHNIQUE; MULTIDISCIPLINARY DESIGN OPTIMIZATION; RESEARCH MOTIVATIONS;

RESEARCH;

EID: 84896913144     PISSN: 00011452     EISSN: None     Source Type: Journal    
DOI: 10.2514/1.J052375     Document Type: Conference Paper

References (220)

1. Sacks, J., Welch, W. J., Mitchell, T. J., and Wynn, H. P., "Design and Analysis of Computer Experiments, " Statistical Science, Vol. 4, No. 4, 1989, pp. 409-435. doi:10.1214/ss/1177012413
2. Kleijnen, J. P. C, "A Comment on Blanning's Metamodel for Sensitivity Analysis: The Regression Metamodel in Simulation, " Interfaces, Vol. 5, No. 3, 1975, pp. 21-23. doi:10.1287/inte.5.3.21
3. Schmit, L. A. Jr., and Farshi, B., "Some Approximation Concepts for Structural Synthesis, " AIAA Journal, Vol. 12, No. 5, 1974, pp. 692-699. doi:10.2514/3.49321
4. Schmit, L. A., "Structural Synthesis-Its Genesis and Development, " AIAA Journal, Vol. 19, No. 10, 1981, pp. 1249-1263. doi:10.2514/3.7859
5. Barthelemy, J.-F. M., and Haftka, R. T., "Approximation Concepts for Optimum Structural Design-A Review, " Structural Optimization, Vol. 5, No. 3, 1993, pp. 129-144. doi:10.1007/BF01743349
6. Box, G. E. P., Hunter, W. G, and Hunter, J. S., Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building, Wiley, New York, 1978.
7. Free, J. W., Parkinson, A. R., Bryce, G R., and Balling, R. J., "Approximation of Computationally Expensive and Noisy Functions for Constrained Nonlinear Optimization, " Journal of Mechanisms, Transmissions, and Automation in Design, Vol. 109, No. 4, 1987, pp. 528-532. doi:10.1115/1.3258832 (Pubitemid 18549488)
8. Giunta, A.A., Dudley, J. M., Narducci, R., Grossman, B., Haftka, R. T., Mason, W. H., and Watson, L. T., "Noisy Aerodynamic Response and Smooth Approximations in HSCT Design, " 5th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, AIAA, Washington, D.C., 1994, pp. 1117-1128.
9. Toropov V V "Multipoint Approximation Method for Structural Optimization Problems with Noisy Function Values, " Stochastic Programming: Numerical Techniques and Engineering Applications, edited by Marti, K., and Kall, P., Vol. 423, Lecture Notes in Economics and Mathematical Systems, Springer-Verlag, London, England, U.K. 1995, pp. 109-122.
10. Toropov V V "Modelling and Approximation Strategies in Optimization-Global and Mid-Range Metamodels, Response Surface Methods, Genetic Programming, and Low/High Fidelity Models, " Emerging Methods for Multidisciplinary Optimization, edited by Blachut, J., and Eschenauer, H. A., Vol. 425, CISM Courses and Lectures, Springer-Verlag, New York, 2001 pp. 205-256.
11. Fang, K.-T., Li, R., and Sudjianto, A., Design and Modeling for Computer Experiments, Taylor and Francis, Boca Raton, FL, 2006.
12. Giunta, A. A., Wojtkiewicz, S. F Jr., and Eldred, M. S., "Overview of Modern Design of Experiments Methods for Computational Simulations, " 41st AIAA Aerospace Sciences Meeting and Exhibit, AIAA Paper 2003-649, 2003.
13. Pedersen, P., "The Integrated Approach of FEM-SLP for Solving Problems of Optimal Design, " Optimization of Distributed Parameter Structures, edited by Haug, E. J., and Cea, J., Vol. 49, Proceedings of the NATO Advanced Study Institute, Series E, Sijhoff and Noordhoff, Alphen aan den Rijn, The Netherlands, 1981, pp. 739-756.
14. Fleury, C, and Braibant, V, "Structural Optimization: A New Dual Method Using Mixed Variables, " International Journal of Numerical Methods in Engineering, Vol. 23, No. 3, 1986, pp. 409-428. doi:10.1002/nme.1620230307 (Pubitemid 16517200)
15. Fleury, C, "First and Second Order Convex Approximation Strategies in Structural Optimization, " Structural Optimization, Vol. 1, No. 1, 1989, pp. 3-10. doi:10.1007/BF01743804
16. Fleury, C, "CONLIN: An Efficient Dual Optimizer Based on Convex Approximation Concepts, " Structural Optimization, Vol. 1, No. 1, 1989, pp. 81-89. doi:10.1007/BF01743804
17. Svanberg, K, "The Method of Moving Asymptotes-A New Method for Structural Optimization, " International Journal of Numerical Methods in Engineering, Vol. 24, No. 2, 1987, pp. 359-373. doi:10.1002/nme.1620240207 (Pubitemid 17525942)
18. Haftka, R. T, Nachlas, J. A., Watson, L. T, Rizzo, T, and Desai, R., "Two-Point Constraint Approximation in Structural Optimization, " Computational Methods in Applied Mechanics and Engineering, Vol. 60, No. 3, 1987, pp. 289-301. doi:10.1016/0045-7825(87)90136-8 (Pubitemid 17572347)
19. Fadel, G M., Riley, M. F., and Barthelemy, J. M., "Two Point Exponential Approximation Method for Structural Optimization, " Structural Optimization, Vol. 2, No. 2, 1990, pp. 117-124. doi:10.1007/BF01745459
20. Rasmussen, J., "Accumulated Approximation: A New Method for Structural Optimization by Iterative Improvement, " 3rd Air Force/NASA Symposium on Recent Advances in Multidisciplinary Analysis and Optimization, San Francisco, CA, 1990, pp. 253-258; also NASA TR 19940004696.
21. Toropov V V "Simulation Approach to Structural Optimization, " Structural Optimization, Vol. 1, No. 1, 1989, pp. 37-46. doi:10.1007/BF01743808
22. Toropov V V, Filatov, A. A., and Polynkin, A. A. "Multiparameter Structural Optimization Using FEM and Multipoint Explicit Approximations, " Structural Optimization, Vol. 6, No. 1, 1993, pp. 7-14. doi:10.1007/BF01743169
23. Vanderplaats, G. N, "Effective Use of Numerical Optimization in Structural Design, " Finite Elements in Analysis and Design, Vol. 6, No. 1, 1989, pp. 97-112. doi:10.1016/0168-874X(89)90038-3 (Pubitemid 20634184)
24. Schoofs, A. J. G, "Experimental Design and Structural Optimization, " Ph.D. Dissertation, Univ. of Technology, Eindhoven, The Netherlands, 1987.
25. Rikards, R., "Elaboration of Optimal Design Models for Objects from Data of Experiments, " Optimal Design with Advanced Materials, edited by Pederson, P., Elsevier, Amsterdam, 1993, pp. 113-130.
26. Box, G E. P., and Wilson, K B., "On the Experimental Attainment of Optimal Conditions, " Journal of the Royal Statistical Society, No. 13, 1951, pp. 1-38.
27. Myers, R. H., and Montgomery, D. C, Response Surface Methodology: Process and Product Optimization Using Designed Experiments, Wiley, New York, 1995.
28. Hajela, P., and Berke, L., "Neural Networks in Structural Analysis and Design: An Overview, " Computing Systems in Engineering, Vol. 3, Nos. 1-4, 1992, pp. 525-538. doi:10.1016/0956-0521(92)90138-9
29. Sobieszczanski-Sobieski, J., and Haftka, R. T, "Multidisciplinary Aerospace Design Optimization: Survey of Recent Developments, " Structural Optimization, Vol. 14, No. 1, 1997, pp. 1-23. doi:10.1007/BF01197554 (Pubitemid 127676975)
30. Cox, D. D., and John, S., "SDO: A Statistical Method for Global Optimization, " Proceedings of the ICASE/NASA Langley Workshop on Multidisciplinary Optimization, Soc. for Industrial and Applied Mechanics, Hampton, VA, 1995, pp. 315-329.
31. Dennis, J. E., and Torczon, V, "Managing Approximation Models in Optimization, " Proceedings of the ICASE/NASA Langley Workshop on Multidisciplinary Design Optimization, Soc. for Industrial and Applied Mechanics, Hampton, VA, 1995, pp. 330-347.
32. Giunta, A. A., Balabanov, V, Kaufman, M., Burgee, S., Grossman, B., Haftka, R. T, Mason, W H., and Watson, L. T, "Variable-Complexity Response Surface Design of an HSCT Configuration, " Multidisciplinary Design Optimization: State of the Art-Proceedings of the ICASE/NASA Langley Workshop on Multidisciplinary Design Optimization, Soc. for Industrial and Applied Mechanics, Hampton, VA, 1996, pp. 348-367.
33. Otto, J., Paraschivoiu, M., Yesilyurt, S., and Patera, A. T, "Bayesian-Validated Computer-Simulation Surrogates for Optimization and Design, " Proceedings of the ICASE/NASA Langley Workshop on Multidisciplinary Optimization, Soc. for Industrial and Applied Mechanics, Hampton, VA, 1995, pp. 368-392.
34. Wujek, B. A., Renaud, J. E., and Batill, S. M., "A Concurrent Engineering Approach for Multidisciplinary Design in a Distributed Computing Environment, " Multidisciplinary Design Optimization: State of the Art-Proceedings of the ICASE/NASA Langley Workshop on Multidisciplinary Design Optimization, Soc. for Industrial and Applied Mechanics, Hampton, VA, 1995, pp. 189-208.
35. Kaufman, M., Balabanov, V, Giunta, A. A., Grossman, B., Mason, W H., Burgee, S., Haftka, R. T, and Watson, L. T, "Variable-Complexity Response Surface Approximations for Wing Structural Weight in HSCT Design, " Computational Mechanics, Vol. 18, No. 2, 1996, pp. 112-126. doi: 10.1007/BF00350530 (Pubitemid 126545867)
36. Giunta, A. A., Balabanov, V, Haim, D., Grossman, B., Mason, W. H., Watson, L. T, and Haftka, R. T, "Aircraft Multidisciplinary Design Optimization Using Design of Experiments Theory and Response Surface Modeling, " Aeronautical Journal, Vol. 101, No. 1008, 1997, pp. 347-356. (Pubitemid 127547509)
37. Renaud, J. E., and Gabriele, G A., "Approximation in Nonhierarchic System Optimization, " AIAA Journal, Vol. 32, No. 1, 1994, pp. 198-205. doi:10.2514/3.11967
38. Renaud, J. E., and Gabrielle, G A., "Sequential Global Approximation in Non-Hierarchic System Decomposition and Optimization, " Advances in Design Automation-Design Automation and Design Optimization, American Soc. of Mechanical Engineers, Fairfield, NJ, 1991, pp. 191-200.
39. Wujek, B., Renaud, J. E., Batill, S. M., and Brockman, J. B., "Concurrent Subspace Optimization Using Design Variable Sharing in a Distributed Computing Environment, " Concurrent Engineering: Research and Applications, Vol. 4, No. 4, 1996, pp. 361-378. doi:10.1177/1063293X9600400405
40. Mavris, D. N, Bandte, O., and DeLaurentis, D. A., "Robust Design Simulation: A Probabilistic Approach to Multidisciplinary Design, " Journal of Aircraft, Vol. 36, No. 1, 1999, pp. 298-307. doi:10.2514/2.2437
41. Chen, W., Allen, J. K., Mavris, D., and Mistree, F., "A Concept Exploration Method for Determining Robust Top-Level Specifications, " Engineering Optimization, Vol. 26, No. 2, 1996, pp. 137-158. doi:10.1080/03052159608941114 (Pubitemid 126633535)
42. Koch, P. N, Allen, J. K., Mistree, F., and Barlow, A., "Facilitating Concept Exploration for Configuring Turbine Propulsion Systems, " Journal of Mechanical Design, Vol. 120, No. 4, 1998, pp. 702-706. doi:10.1115/1.2829334 (Pubitemid 128652211)
43. Haftka, R., Scott, E. P., and Cruz, J. R., "Optimization and Experiments: A Survey, " Applied Mechanics Review, Vol. 51, No. 7, 1998, pp. 435-448. doi:10.1115/1.3099014 (Pubitemid 128436152)
44. Simpson, T W., Peplinski, J., Koch, P. N, and Allen, J. K, "Metamodels for Computer-Based Engineering Design: Survey and Recommendations, " Engineering with Computers, Vol. 17, No. 2, 2001, pp. 129-150. doi:10.1007/PL00007198 (Pubitemid 32659225)
45. Balabanov, V, Kaufman, M., Knill, D. L., Golovidov, O., Giunta, A. A., Haftka, R. T, Grossman, B., Mason, W. H., and Watson, L. T, "Dependence of Optimal Structural Weight on Aerodynamic Shape for a High Speed Civil Transport, " 6th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, AIAA, Reston, VA, 1996, pp. 599-612.
46. Koch, P. N, Simpson, T W, Allen, J. K, and Mistree, F., "Statistical Approximations for Multidisciplinary Optimization: The Problem of Size, " Journal of Aircraft, Vol. 36, No. 1, 1999, pp. 275-286. doi:10.2514/2.2435
47. Simpson, T W, Mauery, T M., Korte, J. J., and Mistree, F., "Kriging Metamodels for Global Approximation in Simulation-Based Multidisciplinary Design Optimization, " AIAA Journal, Vol. 39, No. 12, 2001, pp. 2233-2241. doi:10.2514/2.1234 (Pubitemid 34023417)
48. Venter, G, Haftka, R. T, and Starnes, J. H. Jr., "Construction of Response Surface Approximations for Design Optimization, " AIAA Journal, Vol. 36, No. 12, 1998, pp. 2242-2249. doi:10.2514/2.333 (Pubitemid 128551582)
49. Roux, W J., Stander, N, and Haftka, R. T, "Response Surface Approximations for Structural Optimization, " 6th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, AIAA, Reston, VA, 1996, pp. 565-578.
50. Koehler, J. R., and Owen, A. B., "Computer Experiments, " Handbook of Statistics, edited by Ghosh, S., and Rao, C. R., Vol. 13, Elsevier, New York, 1996, pp. 261-308.
51. Simpson, T W, Lin, D. K J., and Chen, W., "Sampling Strategies for Computer Experiments: Design and Analysis, " International Journal of Reliability and Applications, Vol. 2, No. 3, 2001, pp. 209-240.
52. van Keulen, F., and Vervenne, K, "Gradient-Enhanced Response Surface Building, " Structural and Multidisciplinary Optimization, Vol. 27, No. 5, 2004, pp. 337-351. doi:10.1007/s00158-004-0392-1
53. Liu, W, andBatill, S., "Gradient-Enhanced Neural Network Response Surface Approximations, " 8th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Long Beach, CA, AIAA Paper 2000-4923, 2000.
54. Liu, W., and Batill, S. M., "Gradient-Enhanced Response Surface Approximations Using Kriging Models, " 9thAIAA/ISSMO Symposium and Exhibit on Multidisciplinary Analysis and Optimization, AIAA Paper 2002-5456, 2002.
55. Toropov, V, van Keulen, F., Markine, V, and de Boer, H, "Refinements in the Multi-Point Approximation Method to Reduce the Effects of Noisy Structural Responses, " 6th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, AIAA, Reston, VA, 1996, pp. 941-951.
56. Alexandrov, N, Dennis, J. E. Jr., Lewis, R. M., and Torczon, V, "A Trust Region Framework for Managing the Use of Approximation Models in Optimization, " Structural Optimization, Vol. 15, No. 1, 1998, pp. 16-23. doi:10.1007/BF01197433 (Pubitemid 128647612)
57. Rodriguez, J. F, Renaud, J. E., and Watson, L. T, "Trust Region Augmented Lagrangian Methods for Sequential Response Surface Approximation and Optimization, " Journal of Mechanical Design, Vol. 120, No. 1, 1998, pp. 58-66. doi:10.1115/1.2826677 (Pubitemid 128652140)
58. Booker, A. J., Dennis, J. E. Jr., Frank, P. D., Serafini, D. B., Torczon, V, and Trosset, M. W, "A Rigorous Framework for Optimization of Expensive Functions by Surrogates, " Structural Optimization, Vol. 17, No. 1, 1999, pp. 1-13. doi:10.1007/BF01197708
59. Koch, P. N, Evans, J. P., and Powell, D., "Interdigitation for Effective Design Space Exploration Using iSIGHT, " Structural and Multidisciplinary Optimization, Vol. 23, No. 2, 2002, pp. 111-126. doi:10.1007/s00158-002-0171-9 (Pubitemid 41388629)
60. Balabanov, V, Charpentier, C, Ghosh, D. K, Quinn, G, Vanderplaats, G, and Venter, G, "VisualDOC: A Software System for General Purpose Integration and Design Optimization, " 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, AIAA Paper 2002-5513, 2002.
61. Tzannetakis, N, and Van de Peer, J., "Design Optimization Through Parallel-Generated Surrogate Models, Optimization Methodologies and the Utility of Legacy Simulation Software, " Structural and Multidisciplinary Optimization, Vol. 23, No. 2, 2002, pp. 170-186. doi:10.1007/s00158-002-0175-5 (Pubitemid 41388633)
62. ModelCenter, Software Package, Ver. 2.01., Phoenix Integration, Inc., Blacksburg, VA, 1999.
63. Eldred, M. S., Giunta, A. A., van Bloemen Waanders, B. G, Wojtkiewicz, S. F Jr., Hart, W E., and Alleva, M. P., DAKOTA, A Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis Version 3.0 User's Manual, Sandia National Laboratories, Albuquerque, NM, 2002.
64. Jones, D. R., Schonlau, M., and Welch, W J., "Efficient Global Optimization of Expensive Black-Box Functions, " Journal of Global Optimization, Vol. 13, No. 4, 1998, pp. 455-492. doi:10.1023/A: 1008306431147 (Pubitemid 128507405)
65. Currin, C, Mitchell, T, Morris, M., and Ylvisaker, D., "Bayesian Prediction of Deterministic Functions, with Applications to the Design and Analysis of Computer Experiments, " Journal of the American Statistical Association, Vol. 86, No. 416, 1991, pp. 953-963. doi:10.1080/01621459.1991. 10475138
66. Chaloner, K, and Verdinelli, I., "Bayesian Experimental Design: A Review, " Statistical Science, Vol. 10, No. 3, 1995, pp. 273-304. doi:10.1214/ss/1177009939
67. Pacheco, J. E., Amon, C. H., and Finger, S., "Developing Bayesian Surrogates for Use in Preliminary Design, " ASME Design Engineering Technical Conferences-Design Theory and Methodology Conference, American Soc. of Mechanical Engineers, New York, 2001.
68. O'Hagan, A., "Bayesian Analysis of Computer Code Outputs: A Tutorial, " Reliability Engineering and System Safety, Vol. 91, Nos. 10-11, 2006, pp. 1290-1300. doi:10.1016/j.ress.2005.11.025
69. Kumar, N, Subramaniyan, A., and Wang, L., "Improving High-Dimensional Physics Models Through Bayesian Calibration with Uncertainty Data, " Proceedings of ASME Turbo Expo, American Soc. of Mechanical Engineers Paper GT2012-69058, Fairfield, NJ, 2012.
70. Hardy, R. L., "Multiquadratic Equations of Topography and Other Irregular Surfaces, " Journal of Geophysical Research, Vol. 76, No. 8, 1971, pp. 1905-1915. doi:10.1029/JB076i008p01905
71. Dyn, N, Levin, D., and Rippa, S., "Numerical Procedures for Surface Fitting of Scattered Data by Radial Basis Functions, " SIAM Journal on Scientific and Statistical Computing, Vol. 7, No. 2, 1986, pp. 639-659. doi:10.1137/0907043
72. Mullur, A. A., and Messac, A., "Metamodeling Using Extended Radial Basis Functions: A Comparative Approach, " Engineering with Computers, Vol. 21, No. 3, 2006, pp. 203-217. doi:10.1007/s00366-005-0005-7
73. Friedman, J. H., "Multivariate Adaptive Regression Splines, " Annals of Statistics, Vol. 19, No. 1, 1991, pp. 1-67. doi:10.1214/aos/1176347963
74. Cressie, N. A. C, Statistics for Spatial Data, Wiley, New York, 1993.
75. Gunn, S. R., "Support Vector Machines for Classification and Regression, " Image Speech and Intelligent Systems Research Group, Univ. of Southampton, Southampton, England, U.K., 1997.
76. Smola, A. J., and Schölkopf, B., "A Tutorial on Support Vector Regression, " Statistics and Computing, Vol. 14, No. 3, 2004, pp. 199-222. doi:10.1023/B:STCO.0000035301.49549.88 (Pubitemid 39063488)
77. Kim, C, Wang, S., and Choi, K. K, "Efficient Response Surface Modeling by Using Moving Least-Squares Method and Sensitivity, " AIAA Journal, Vol. 43, No. 11, 2005, pp. 2404-2411. doi:10.2514/1.12366 (Pubitemid 41769000)
78. Breitkopf, P., Naceur, H., Rassineux, A., and Villon, P., "Moving Least Squares Response Surface Approximation: Formulation and Metal Forming Applications, " Computers and Structures, Vol. 83, Nos. 17-18, 2005, pp. 1411-1428. doi:10.1016/j.compstruc.2004.07.011 (Pubitemid 40574504)
79. Coelho, R. F, Lebon, J., and Bouillard, P., "Hierarchical Stochastic Metamodels Based on Moving Least Squares and Polynomial Chaos Expansion, " Structural and Multidisciplinary Optimization, Vol. 43, No. 5, 2011, pp. 707-729. doi:10.1007/s00158-010-0608-5
80. Giunta, A. A., "Aircraft Multidisciplinary Design Optimization Using Design of Experiments Theory and Response Surface Modeling, " Ph.D. Dissertation, Dept. of Aerospace and Ocean Engineering, Virginia Polytechnic Inst. and State Univ., Blacksburg, VA, 1997.
81. Etman, L. F P., "Optimization of Multibody Systems Using Approximation Concepts, " Ph.D. Dissertation, Dept. of Mechanical Engineering, Eindhoven Univ. of Technology, Eindhoven, The Netherlands, 1997.
82. Sasena, M. J., "Flexibility and Efficiency Enhancements for Constrained Global Design Optimization with Kriging Approximations, " Ph.D. Dissertation, Dept. of Mechanical Engineering. Univ. of Michigan, Ann Arbor, MI, 2002.
83. Simpson, T W., "A Concept Exploration Method for Product Family Design, " Ph.D. Dissertation, G W. Woodruff School of Mechanical Engineering, Georgia Inst. of Technology, Atlanta, GA, 1998.
84. Wang, L., and Beeson, D., "Valuable Theoretical Lessons Learned from the Application of Metamodels to a Variety of Industrial Problems, " ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference-Design Automation Conference, American Soc. of Mechanical Engineers Paper DETC2009-87616, Fairfield, NJ, 2009.
85. Hollingsworth, P. M., and Mavris, D. N, "Gaussian Process Meta-Modeling: Comparison of Gaussian Process Training Methods, " AIAA 3rd Annual Aviation Technology, Integration and Operations (ATIO) Conference, AIAA Paper 2003-6761, 2003.
86. Toal, D. J. J., Bressloff, N. W., and Keane, A. J., "Kriging Hyperparameter Tuning Strategies, " AIAA Journal, Vol. 46, No. 5, 2008, pp. 1240-1252. doi:10.2514/1.34822
87. Yamamoto, J. K., "An Alternative Measure of the Reliability of Ordinary Kriging Estimates, " Mathematical Geology, Vol. 32, No. 4, 2000, pp. 489-509. doi:10.1023/A:1007577916868 (Pubitemid 30343626)
88. den Hertog, D., Kleijnen, J. P. C, and Siem, A. Y. D., "The Correct Kriging Variance Estimated by Bootstrapping, " Journal of the Operational Research Society, Vol. 57, No. 4, 2005, pp. 400-409. doi:10.1057/palgrave.jors. 2601997
89. Gano, S. E., Renaud, J. E., and Sanders, B., "Hybrid Variable Fidelity Optimization Using a Kriging-Based Scaling Function, " AIAA Journal, Vol. 43, No. 11, 2005, pp. 2422-2430. doi:10.2514/1.12466 (Pubitemid 41769002)
90. Gano, S. E., Renaud, J. E., Martin, J. D., and Simpson, T W, "Update Strategies for Kriging Models for Using in Variable Fidelity Optimization, " Structural and Multidisciplinary Optimization, Vol. 32, No. 4, 2006, pp. 287-298. doi:10.1007/s00158-006-0025-y (Pubitemid 44268028)
91. Schonlau, M., "Computer Experiments and Global Optimization, " Ph.D. Dissertation, Dept. of Statistics and Actuarial Science, Univ. of Waterloo, Waterloo, Ontario, Canada, 1997.
92. Jin, R., Chen, W, and Sudjianto, A., "On Sequential Sampling for Global Metamodeling in Engineering Design, " ASME Design Engineering Technical Conferences-Design Automation Conference, American Soc. of Mechanical Engineers, New York, NY, 2002.
93. Kleijnen, J. P. C, and Van Beers, W C. M., "Application-Driven Sequential Designs for Simulation Experiments: Kriging Metamodel-ling, " Journal of the Operational Research Society, Vol. 55, No. 8, 2004, pp. 876-883. doi:10.1057/palgrave.jors.2601747
94. Jin, R., Chen, W., and Simpson, T W, "Comparative Studies of Metamodeling Techniques Under Multiple Modeling Criteria, " Structural and Multidisciplinary Optimization, Vol. 23, No. 1, 2001, pp. 1-13. doi:10.1007/s00158-001-0160-4 (Pubitemid 34271934)
95. Gearhart, C, and Wang, B. P., "Bayesian Metrics for Comparing Response Surface Models for Data with Uncertainty, " Structural and Multidisciplinary Optimization, Vol. 22, No. 3, 2001, pp. 198-207. doi:10.1007/s001580100137 (Pubitemid 33046330)
96. Yang, R. J., Gu, L., Liaw, L., Gearhart, C, Tho, C. H, Liu, X., and Wang, B. P., "Approximations for Safety Optimization of Large Systems, " ASME Design Engineering Technical Conferences-Design Automation Conference, American Soc. Mechanical Engineers, New York, 2000.
97. Simpson, T W., Booker, A. J., Ghosh, D., Giunta, A. A., Koch, P. N, and Yang, R.-J., "Approximation Methods in Multidisciplinary Analysis and Optimization: A Panel Discussion, " Structural and Multidisciplinary Optimization, Vol. 27, No. 5, 2004, pp. 302-313. doi:10.1007/s00158-004-0389-9
98. Giunta, A. A., Eldred, M. S., Trucano, T G, and Wojtkiewicz, S. F Jr., "Optimization Under Uncertainty Methods for Computational Shock Physics Applications, " 43rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, AIAA Paper 2002-1642, 2002. ^
99. Koch, P. N, Yang, R.-J., and Gu, L., "Design for Six Sigma Through Robust Optimization, " Structural and Multidisciplinary Optimization, Vol. 26, Nos. 3-4, 2004, pp. 235-248. doi:10.1007/s00158-003-0337-0
100. Feil, B., Kucherenko, S., and Shah, N, "Comparison of Monte Carlo and Quasi Monte Carlo Sampling Methods in High Dimensional Model Representation, " First International Conference on Advances in System Simulation, IEEE Publ., Piscataway, NJ, 2009, pp. 12-17.
101. Zerpa, L., Queipo, N. V, Pintos, S., and Salager, J., "An Optimization Methodology of Alkaline-Surfactant-Polymer Flooding Processes Using Field Scale Numerical Simulation and Multiple Surrogates, " Journal of Petroleum Science and Engineering, Vol. 47, Nos. 3-4, 2005, pp. 197-208. doi:10.1016/j.petrol.2005.03.002 (Pubitemid 40829362)
102. Goel, T, Haftka, R. T, Shyy, W, and Queipo, N. V, "Ensemble of Surrogates, " Structural and Multidisciplinary Optimization, Vol. 33, No. 3, 2007, pp. 199-216. doi:10.1007/s00158-006-0051-9 (Pubitemid 46198778)
103. Acar, E., and Rais-Rohani, M., "Ensemble of Metamodels with Optimized Weight Factors, " Structural and Multidisciplinary Optimization, Vol. 37, No. 3, 2009, pp. 279-294. doi:10.1007/s00158-008-0230-y
104. Joseph, V R., Hung, Y, and Sudjianto, A., "Blind Kriging: A New Method for Developing Metamodels, " Journal of Mechanical Design, Vol. 130, No. 3, 2008, Paper 031102. doi:10.1115/1.2829873 (Pubitemid 351772347)
105. Viana, F A. C, Haftka, R. T, and Steffen, V Jr., "Multiple Surrogates: How Cross-Validation Errors Can Help Us to Obtain the Best Predictor, " Structural and Multidisciplinary Optimization, Vol. 39, No. 4, 2009, pp. 439-457. doi:10.1007/s00158-008-0338-0
106. Viana, F A. C, and Haftka, R. T, "Using Multiple Surrogates for Metamodeling, " 7th ASMO-UK/ISSMO International Conference on Engineering Design Optimization, International Society for Structural and Multidisciplinary Optimization, Bath, England, U.K., 2008.
107. Queipo, N. V, Haftka, R. T, Shyy, W, Goel, T, Vaidyanathan, R., and Tucker, P. K., "Surrogate-Based Analysis and Optimization, " Progress in Aerospace Sciences, Vol. 41, No. 1, 2005, pp. 1-28. doi:10.1016/j.paerosci. 2005.02.001 (Pubitemid 40537187)
108. Wang, G G, and Shan, S., "Review of Metamodeling Techniques in Support of Engineering Design Optimization, " Journal of Mechanical Design, Vol. 129, No. 4, 2007, pp. 370-380. doi:10.1115/1.2429697 (Pubitemid 46736217)
109. Forrester, A. I. J., and Keane, A. J., "Recent Advances in Surrogate-Based Optimization, " Progress in Aerospace Sciences, Vol. 45, Nos. 1-3, 2009, pp. 50-79. doi:10.1016/j.paerosci.2008.11.001
110. Venkataraman, S., and Haftka, R. T, "Structural Optimization Complexity: What Has Moore's Law Done for Us?, " Structural and Multidisciplinary Optimization, Vol. 28, No. 6, 2004, pp. 375-387. doi:10.1007/s00158-004-0415-y
111. Wang, L., Shan, S., and Wang, G. G, "Mode-Pursuing Sampling Method for Global Optimization on Expensive Black-Box Functions, " Engineering Optimization, Vol. 36, No. 4, 2004, pp. 419-438. doi:10.1080/ 03052150410001686486
112. Sharif, B., Wang, G G, and ElMekkawy, T, "Mode Pursing Sampling Method for Discrete Variable Optimization on Expensive Black-Box Functions, " Journal of Mechanical Design, Vol. 130, No. 2, 2008, Paper 021402. doi:10.1115/1.2803251 (Pubitemid 351772365)
113. Wang, D., Wang, G G, and Naterer, G, "Extended Collaboration Pursuing Method for Solving Larger MDO Problems, " AIAA Journal, Vol. 45, No. 6, 2007, pp. 1208-1221. doi:10.2514/1.21167 (Pubitemid 46997888)
114. Keane, A. J., andNair, P. B., Computational Approaches for Aerospace Design: The Pursuit of Excellence, Wiley, West Sussex, England, U.K., 2005.
115. Box, G. E. P., and Draper, N. R., Empirical Model Building and Response Surfaces, Wiley, New York, 1987.
116. Toropov V. V., and van der Giessen, E. "Parameter Identification for Nonlinear Constitutive Models: Finite Element Simulation-Optimization- Nontrivial Experiments, " Optimal Design with Advanced Materials, edited by Pedersen, P., Proceedings of IUTAM Symposium Elsevier, New York 1993 pp. 113-130.
117. Haftka, R. T., "Combining Global and Local Approximations, " AIAA Journal, Vol. 29, No. 9, 1991, pp. 1523-1525. doi:10.2514/3.10768
118. Hutchison, M. G, Unger, E. R., Mason, W H, Grossman, B., and Haftka, R. T., "Variable-Complexity Aerodynamic-Structural Design of a High-Speed Civil Transport, " Journal of Aircraft, Vol. 31, No. 1, 1994, pp. 110-116. doi:10.2514/3.46462
119. Chang, K. J., Haftka, R. T., Giles, G L., and Kao, P.-J., "Sensitivity-Based Scaling for Approximating Structural Response, " Journal of Aircraft, Vol. 30, No. 2, 1993, pp. 283-287. doi:10.2514/3.48278
120. Hutchison, M. G, Unger, E. R., Mason, W. M., Grossman, B., and Haftka, R. T., "Variable Complexity Aerodynamic Optimization of a High-Speed Civil Transport Wing, " Journal of Aircraft, Vol. 31, No. 1, 1994, pp. 110-120. doi:10.2514/3.46462
121. Alexandrov, N. M., and Lewis, R. M., "An Overview of First-Order Model Management for Engineering Optimization, " Optimization and Engineering, Vol. 2, No. 4, 2001, pp. 413-430. doi:10.1023/A:1016042505922
122. Eldred, M., Giunta, A., and Collis, S., "Second-Order Corrections for Surrogate-Based Optimization with Model Hierarchies, " 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, AIAA Paper 2004-4457, 2004.
123. Eldred, M. S., and Dunlavy, D. M., "Formulations for Surrogate-Based Optimization with Data Fit, Multifidelity, and Reduced-Order Models, " 11th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, AIAA Paper 2006-7117, 2006.
124. Knill, D. L., Giunta, A. A., Baker, C. A., Grossman, B., Mason, W. H, Haftka, R. T, and Watson, L. T, "Response Surface Models Combining Linear and Euler Aerodynamics for Supersonic Transport Design, " Journal of Aircraft, Vol. 36, No. 1, 1999, pp. 75-86. doi:10.2514/2.2415
125. Giunta, A. A., Narducci, R., Burgee, S., Grossman, B., Haftka, R. T, Mason, W H, and Watson, L. T, "Variable-Complexity Response Surface Aerodynamic Design of an HSCT Wing, " 13th AIAA Applied Aerodynamics Conference, AIAA Paper 1995-1886, 1995.
126. Mason, B. H., Haftka, R. T, Johnson, E. R., and Farley, G L., "Variable Complexity Design of Composite Fuselage Frames by Response Surface Techniques, " Thin-Walled Structures, Vol. 32, No. 4, 1998, pp. 235-261. doi:10.1016/S0263-8231(98)00016-0 (Pubitemid 128415404)
127. Balabanov, V, Haftka, R. T, Grossman, B., Mason, W. H., and Watson, L. T, "Multifidelity Response Surface Model for HSCT Wing Bending Material Weight, " 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, AIAA, Reston, VA, 1998, pp. 778-788.
128. Venkataraman, S., and Haftka, R. T, "Design of Shell Structures for Buckling Using Correction Response Surface Approximations, " 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, AIAA, Reston, VA, 1998, pp. 1131-1144.
129. Vitali, R., Haftka, R. T, and Sankar, B. V, "Correction Response Approximation for Stress Intensity Factors for Composite Stiffened Plates, " 39th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Material Conference, AIAA, Reston, VA, 1998, pp. 2917-2922.
130. Madsen, J. I., and Langthjem, M., "Multifidelity Response Surface Approximations for the Optimum Design of Diffuser Flows, " Optimization and Engineering, Vol. 2, No. 4, 2001, pp. 453-468. doi:10.1023/A:1016046606831
131. Umakant, J., Sudhakar, K., Mujumdar, P., and Rao, C, "Customized Regression Model for Improving Low Fidelity Analysis Tool, " 11th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, AIAA Paper 2006-7118, 2006.
132. Sharma, A., Sankar, B. V, and Haftka, R. T, "Multi-Fidelity Analysis of Corrugated-Core Sandwich Panels for Integrated Thermal Protection Systems, " 50th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, AIAA Paper 2009-2201, 2009.
133. Markine, V L., Meijers, P., Meijaard, J. P., and Toropov, V V, "Multilevel Optimization of Dynamic Behaviour of a Linear Mechanical System with Multipoint Approximation, " Engineering Optimization, Vol. 25, No. 4, 1996, pp. 295-307. doi:10.1080/03052159608941268 (Pubitemid 126633585)
134. Berci, M., Toropov, V V, Hewson, R. W., and Gaskell, P., "Metamodelling Based on High and Low Fidelity Models Interaction for UAV Gust Performance Optimization, " 50th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, andMaterials Conference, AIAA Paper 2009-2215, 2009.
135. Zadeh, P. M., Toropov, V V, and Wood, A. S., "Metamodel-Based Collaborative Optimization Framework, " Structural and Multidisciplinary Optimization, Vol. 38, No. 1, 2009, pp. 103-115. doi:10.1007/s00158-008-0286-8
136. Hino, R., Yoshida, F, and Toropov, V V, "Optimum Blank Design for Sheet Metal Forming Based on the Interaction of High-and Low-Fidelity FE Models, " Archive of Applied Mechanics, Vol. 75, Nos. 10-12, 2006, pp. 679-691. doi:10.1007/s00419-006-0047-3 (Pubitemid 44495632)
137. Markine, V L., "Optimization of the Dynamic Behaviour of Mechanical Systems, " Ph.D. Dissertation, Dept. of Mechanical Maritime and Materials Engineering, Delft Univ. of Technology, Delft, The Netherlands, 1999.
138. Markine, V L., and Toropov, V V, "Structural Optimization Using Approximations Based on Simplified Numerical Models, " 43rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, AIAA Paper 2002-1585, 2002.
139. Goldfeld, Y, Vervenne, K., Arbocz, J., and van Keulen, F, "Multi-Fidelity Optimization of Laminated Conical Shells for Buckling, " Structural and Multidisciplinary Optimization, Vol. 30, No. 2, 2005, pp. 128-141. doi:10.1007/s00158-004-0506-9 (Pubitemid 41009792)
140. Rodriguez, J. F, Renaud, J. E., and Watson, L. T, "Convergence of Trust Region Managed Augmented Lagrangian Methods Using Variable Fidelity Approximation Data, " Structural Optimization, Vol. 15, Nos. 3-4, 1998, pp. 141-156. doi:10.1007/BF01203525 (Pubitemid 128647627)
141. Rodríguez, J. F, Pérez, V M., Padmanabhan, D., and Renaud, J. E., "Sequential Approximate Optimization Using Variable Fidelity Response Surface Approximations, " Structural and Multidisciplinary Optimization, Vol. 22, No. 1, 2001, pp. 24-34. doi:10.1007/s001580100122 (Pubitemid 32840611)
142. Keane, A. J., "Wing Optimization Using Design of Experiment, Response Surface, and Data Fusion Methods, " Journal of Aircraft, Vol. 40, No. 4, 2003, pp. 741-750. doi:10.2514/2.3153
143. Leary, S. J., Bhaskar, A., and Keane, A. J., "A Knowledge-Based Approach to Response Surface Modelling in Multifidelity Optimization, " Journal of Global Optimization, Vol. 26, No. 3, 2003, pp. 297-319. doi:10.1023/A:1023283917997
144. Forrester, A. I. J., Sobester, A., and Keane, A. J., "Multi-Fidelity Optimization via Surrogate Modelling, " Proceedings of the Royal Society of London, Series A: Mathematical and Physical Sciences, Vol. 463, No. 2088, 2007, pp. 3251-3269. doi:10.1098/rspa.2007.1900
145. Balabanov, V, and Venter, G, "Multi-Fidelity Optimization with High-Fidelity Analysis and Low-Fidelity Gradients, " 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, AIAA Paper 2004-4459, 2004.
146. Koziel, S., Bandler, J. W, and Madsen, K, "Quality Assessment of Coarse Models and Surrogates for Space Mapping Optimization, " Optimization and Engineering, Vol. 9, No. 4, 2008, pp. 375-391. doi:10.1007/s11081-007-9032- 0
147. Koziel, S., and Bandler, J. W., "Space-Mapping Optimization with Adaptive Surrogate Model, " IEEE Transactions on Microwave Theory and Techniques, Vol. 55, No. 3, 2007, pp. 541-547. doi:10.1109/TMTT.2006.890524 (Pubitemid 46393318)
148. Bandler, J. W., Biernacki, R. M., Chen, S. H., Grobelny, R. H., and Hemmers, R. H., "Space Mapping Technique for Electromagnetic Optimization, " IEEE Transactions on Microwave Theory and Techniques, Vol. 42, No. 12, 1994, pp. 2536-2544. doi:10.1109/22.339794
149. Bakr, M. H., Bandler, J. W., Madsen, K, and Søndergaard, J., "An Introduction to the Space Mapping Technique, " Optimization and Engineering, Vol. 2, No. 4, 2001, pp. 369-384. doi:10.1023/A: 1016086220943
150. Koziel, S., Bandler, J. W., and Madsen, K., "A Space-Mapping Framework for Engineering Optimization-Theory and Implementation, " IEEE Transactions on Microwave Theory and Techniques, Vol. 54, No. 10, 2004, pp. 3721-3730. doi:10.1109/TMTT.2006.882894
151. Bakr, M.H., Bandler, J.W., Biernacki, R.M., Chen, S.H., and Madsen, K., "A Trust Region Aggressive Space Mapping Algorithm for EM Optimization, " IEEE Transactions on Microwave Theory and Techniques, Vol. 46, No. 12, 1998, pp. 2412-2425. doi:10.1109/22.739229 (Pubitemid 128738981)
152. Bandler, J. W., Cheng, Q. S., Dakroury, S. A., Mohamed, A. S., Bakr, M. H., Madsen, K., and Sondergaard, J., "SpaceMapping: The State of the Art, " IEEE Transactions on Microwave Theory and Techniques, Vol. 52, No. 1, 2004, pp. 337-361. doi:10.1109/TMTT.2003.820904
153. Leary, S. J., Bhaskar, A., and Keane, A. J., "A Constraint Mapping Approach to the Structural Optimization of an Expensive Model Using Surrogates, " Optimization and Engineering, Vol. 2, No. 4, 2001, pp. 385-398. doi:10.1023/A:1016038305014
154. Ignatovich, C. L., and Diaz, A., "Physical Surrogates in Design Optimization for Enhanced Crashworthiness, " 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, AIAA Paper 2002-5537, 2002.
155. Redhe, M., and Nilsson, L., "A Multipoint Version of Space Mapping Optimization Applied to Vehicle Crashworthiness Design, " Structural and Multidisciplinary Optimization, Vol. 31, No. 2, 2006, pp. 134-146. doi:10.1007/s00158-005-0544-y (Pubitemid 43130324)
156. Hu, W., Enying, L., Li, G. Y., and Zhong, Z. H., "Optimization of Sheet Metal Forming Processes by the Use of Space Mapping Based Metamodeling Method, " International Journal of Advanced Manufacturing Technology, Vol. 39, Nos. 7-8, 2008, pp. 642-655. doi:10.1007/s00170-007-1253-z
157. Jansson, T, Andersson, A., and Nilsson, L., "Optimization of Draw-In for an Automotive Sheet Metal Part: An Evaluation Using Surrogate Models and Response Surfaces, " Journal of Materials Processing Technology, Vol. 159, No. 3, 2005, pp. 426-434. doi:10.1016/j.jmatprotec.2004.06.011 (Pubitemid 40072612)
158. Kennedy, M. C, and O'Hagan, A., "Predicting the Output from Complex Computer Code When Fast Approximations Are Available, " Biometrika, Vol. 87, No. 1, 2000, pp. 1-13. doi:10.1093/biomet/87.1.1
159. Nava, E., Pintos, S., and Queipo, N, "A Geostatistical Perspective for the Surrogate-Based Integration of Variable Fidelity Models, " Journal of Petroleum Science and Engineering, Vol. 71, Nos. 1-2, 2010, pp. 56-66. doi:10.1016/j.petrol.2010.01.005
160. Chung, H. S., and Alonso, J., "Using Gradients to Construct Cokriging Approximation for High-Dimensional Design Optimization Models, " 40th AIAA Aerospace Sciences Meeting and Exhibit, AIAA Paper 2002-0317, 2002.
161. Chung, H. S., and Alonso, J., "Design of a Low Boom Supersonic Business Jet Using Cokriging Approximation Models, " 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, AIAA Paper 2002-5598, 2002.
162. Robinson, T, Eldred, M., Willcox, K., and Haimes, R., "Surrogate-Based Optimization Using Multifidelity Models with Variable Parameterization and Corrected Space Mapping, " AIAA Journal, Vol. 46, No. 11, 2008, pp. 2814-2822. doi:10.2514/1.36043
163. Robinson, T D., "Surrogate-Based Optimization Using Multifidelity Models with Variable Parameterization, " Ph.D. Dissertation, Dept. of Aeronautics and Astronautics, Massachusetts Inst. of Technology, Cambridge, MA, 2007.
164. Lophaven, S. N, Nielsen, H. B., and Søndergaard, J., "DACE-A MATLAB Kriging Toolbox. Informatics and Mathematical Modelling, " Technical Univ. of Denmark, 2002.
165. Mack, Y., Goel, T, Shyy, W., Haftka, R. T, and Queipo, N. V, "Multiple Surrogates for the Shape Optimization of Bluff Body-Facilitated Mixing, " 43rd AIAA Aerospace Sciences Meeting and Exhibit, AIAA Paper 2005-0333, 2005.
166. Samad, A., Kim, K, Goel, T, Haftka, R. T, and Shyy, W., "Multiple Surrogate Modeling for Axial Compressor Blade Shape Optimization, " Journal of Propulsion and Power, Vol. 24, No. 2, 2008, pp. 302-310. doi:10.2514/1.28999 (Pubitemid 351683640)
167. Sanchez, E., Pintos, S., and Queipo, N. V, "Toward an Optimal Ensemble of Kernel-Based Approximations with Engineering Applications, " Structural and Multidisciplinary Optimization, Vol. 36, No. 3, 2008, pp. 247-261. doi:10.1007/s00158-007-0159-6 "
168. Glaz, B., Goel, T, Liu, L., Friedman, P. P., and Haftka, R. T, "Multiple-Surrogate Approach to Helicopter Rotor Blade Vibration Reduction, " AIAA Journal, Vol. 47, No. 1, 2009, pp. 271-282. doi:10.2514/1.40291
169. Lin, Y, "An Efficient Robust Concept Exploration Method and Sequential Exploratory Experimental Design, " Ph.D. Dissertation, Dept. of Mechanical Engineering, Georgia Inst. of Technology, Atlanta, GA, 2004. *
170. Araújo, M. B., and Luoto, M., "The Importance of Biotic Interactions for Modelling Species Distributions Under Climate Change, " Global Ecology and Biogeography, Vol. 16, No. 6, 2007, pp. 743-753. doi:10.1111/geb.2007.16.issue-6 (Pubitemid 47602777)
171. Stoica, P., and Selen, Y, "Model-Order Selection: A Review of Information Criterion Rules, " IEEE Signal Processing Magazine, Vol. 21, No. 4, 2004, pp. 36-47. doi:10.1109/MSP2004.1311138
172. Picard, R. R., and Cook, R. D., "Cross-Validation of Regression Models, " Journal of the American Statistical Association, Vol. 79, No. 387, 1984, pp. 575-583. doi:10.1080/01621459.1984.10478083
173. Kohavi, R., "A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection, " The International Joint Conference on Artificial Intelligence, Morgan Kaufmann, San Mateo, CA, 1995, pp. 1137-1143.
174. Roecker, E. B., "Prediction Error and Its Estimation for Subset-Selected Models, " Technometrics, Vol. 33, No. 4, 1991, pp. 459-468. doi:10.1080/00401706.1991.10484873
175. Utans, J., andMoody, J., "Selecting Neural Network Architectures via the Prediction Risk: Application to Corporate Bond Rating Prediction, " Proceedings of the IEEE 1st International Conference on AI Applications on Wall Street, IEEE Publ., Piscataway, NJ, 1991, pp. 35-41.
176. Zhang, P., "Model Selection via Multifold Cross Validation, " Annals of Statistics, Vol. 21, No. 1, 1993, pp. 299-313. doi:10.1214/aos/ 1176349027
177. Yang, Y, "Regression with Multiple Candidate Models: Selecting or Mixing?, " Statistica Sinica, Vol. 13, No. 387, 2003, pp. 783-809. doi:10.1080/01621459.1984.10478083 (Pubitemid 37155854)
178. Leblanc, M., and Tibshirani, R., "Combining Estimates in Regression and Classification, " Journal of the American Statistical Association, Vol. 91, No. 436, 1996, pp. 1641-1650. doi:10.1080/01621459.1996.10476733
179. Carrero, E., Zerpa, L., Queipo, N, and Pintos, S., "Global Sensitivity Analysis of Alkali-Surfactant-Polymer Enhanced Oil Recovery Processes, " Journal of Petroleum Science and Engineering, Vol. 58, Nos. 1-2, 2006, pp. 30-42. (Pubitemid 47090144)
180. Kim, N. H., Wang, H., and Queipo, N, "Adaptive Reduction of Design Variables Using Global Sensitivity in Reliability-Based Optimization, " International Journal of Reliability and Safety, Vol. 1, Nos. 1-2, 2006, pp. 102-119. doi:10.1504/IJRS.2006.010692
181. Kammer, D. C, and Alvin, K. F, "Component Metamodel Synthesis for the Construction of Master Response Surfaces, " Computer Methods in Applied Mechanics and Engineering, Vol. 191, Nos. 41-42, 2002, pp. 4555-4568. doi:10.1016/S0045-7825(02)00353-5 (Pubitemid 35197833)
182. Pronzato, L., and Müller, W G, "Design of Computer Experiments: Space Filling and Beyond, " Statistics and Computing, Vol. 22, No. 3, 2012, pp. 681-701. doi:10.1007/s11222-011-9242-3
183. Rai, R., and Campbell, M., "Q2S2: A New Methodology for Merging Quantitative and Qualitative Information in Experimental Design, " Journal of Mechanical Design, Vol. 130, No. 3, 2008, Paper 031103. doi:10.1115/1.2829884 (Pubitemid 351772348)
184. Turner, C. J., Crawford, R. H., and Campbell, M. I., " Multidimensional Sequential Sampling for NURBs-based Metamodel Development, " Engineering with Computers, Vol. 23, No. 3, 2007, pp. 155-174. doi:10.1007/s00366-006-0051-9 (Pubitemid 47163362)
185. Gorissen, D., Dhaene, T, and de Turck, F, "Evolutionary Model Type Selection for Global Surrogate Modeling, " Journal of Machine Learning Research, Vol. 10, No. 1, 2009, pp. 2039-2078.
186. Zheng, L., and Wang, X. C, "Global Optimization Based on Weighting-Integral Expected Improvement, " Advanced Materials Research, Vol. 630, 2012, pp. 383-388. doi:10.4028/www.scientific.net/AMR.630.383
187. Benassi, R., Bect, J., and Vazquez, E., "Robust Gaussian Process-Based Global Optimization Using a Fully Bayesian Expected Improvement Criterion, " Lecture Notes in Computer Science, Vol. 6683, 2011, pp. 176-190. doi:10.1007/978-3-642-25566-3
188. Forrester, A., and Jones, D., "Global Optimization of Deceptive Functions with Sparse Sampling, " 12th AIAA/ISSMO Multidiscipli-nary Analysis and Optimization Conference, AIAA Paper 2008-5996, 2008.
189. Couckuyt, I., Forrester, A., Gorissen, D., deTurck, F, and Dhaene, T., "Blind Kriging: Implementation and Performance Analysis, " Advances in Engineering Software, Vol. 49, No. 1, 2012, pp. 1-13. doi:10.1016/j.advengsoft. 2012.03.002
190. Viana, F A. C, Haftka, R. T., and Watson, L. T., "Efficient Global Optimization Algorithm Assisted by Multiple Surrogate Techniques, " Journal of Global Optimization, Vol. 56, No. 2, 2012, pp. 669-689. doi:10.1007/s10898- 012-9892-5 "
191. Goodman, T., and Spence, R., "The Effect of System Response Time on Interactive Computer-Aided Design, " Computer Graphics, Vol. 12, No. 3, 1978, pp. 100-104. doi:10.1145/965139 (Pubitemid 9404402)
192. Simpson, T W., Barron, K., Rothrock, L., Frecker, M., Barton, R. R., and Ligetti, C, "Impact of Response Delay and Training on User Performance with Text-Based and Graphical User Interfaces for Engineering Design, " Research in Engineering Design, Vol. 18, No. 2, 2007, pp. 49-65. doi:10.1007/s00163-007-0033-y (Pubitemid 350112196)
193. Simpson, T W., Iyer, P., Barron, K., Rothrock, L., Frecker, M., Barton, R. R., and Meckesheimer, M., "Metamodel-Driven Interfaces for Engineering Design: Impact of Delay and Problem Size on User Performance, " 46th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference and 1st AIAA Multidisciplinary Design Optimization Specialist Conference, AIAA Paper 2005-2060, 2005.
194. Winer, E. H., and Bloebaum, C. L., "Visual Design Steering for Optimization Solution Improvement, " Structural Optimization, Vol. 22, No. 3, 2001, pp. 219-229. doi:10.1007/s001580100139 (Pubitemid 33046332)
195. Winer, E. H., and Bloebaum, C. L., "Development of Visual Design Steering as an Aid in Large-Scale Multidisciplinary Design Optimization. Part I: Method Development, " Structural and Multidisciplinary Optimization, Vol. 23, No. 6, 2002, pp. 412-424. doi:10.1007/s00158-002-0203-5 (Pubitemid 34882366)
196. Eddy, J., and Lewis, K., "Visualization of Multi-Dimensional Design and Optimization Data Using Cloud Visualization, " ASME 2002 Design Engineering Technical Conferences-Design Automation Conference, American Soc. of Mechanical Engineers, Montreal, Canada, 2002. "
197. Kanukolanu, D., Lewis, K. E., and Winer, E. H., "A Multidimensional Visualization Interface to Aid in Trade-Off Decisions During the Solution of Coupled Subsystems Under Uncertainty, " Journal of Computing and Information Science in Engineering, Vol. 6, No. 3, 2006, pp. 288-299. doi:10.1115/1.2218370 (Pubitemid 44449710)
198. Zhang, R., Noon, C, Oliver, J., Winer, E., Gilmore, B., and Duncan, J., "Development of a Software Framework for Conceptual Design of Complex Systems, " 3rd AIAA Multidisciplinary Design Optimization Specialists Conference, AIAA Paper 2007-1931, 2007.
199. Zhang, R., Noon, C, Oliver, J., Winer, E., Gilmore, B., and Duncan, J., "Immersive Product Configurator for Conceptual Design, " 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, AIAA Paper 2007-1931, 2007.
200. Agrawal, G, Lewis, K., Chugh, K., Huang, C.-H., Parashar, S., and Bloebaum, C. L., "Intuitive Visualization of Pareto Frontier for Multi-Objective Optimization in n-Dimensional Performance Space, " 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, AIAA Paper 2004-4434, 2004.
201. Agrawal, G, Parashar, S., and Bloebaum, C. L., "Intuitive Visualization of Hyperspace Pareto Frontier for Robustness in Multi-Attribute Decision-Making, " 11th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, AIAA Paper 2006-6962, 2006.
202. De Baets, P. W G, and Mavris, D. N, "Methodology for Parametric Structural Conceptual Design of Hypersonic Vehicles, " 5th World Aviation Congress and Exposition, AIAA Paper 01-5618, 2000. "
203. Kodiyalam, S., Yang, R.-J., and Gu, L., "High Performance Computing and Surrogate Modeling for Rapid Visualization with Multidisciplinary Optimization, " AIAA Journal, Vol. 42, No. 11, 2004, pp. 2347-2354. doi:10.2514/1.1997
204. Simpson, T W, Carlsen, D. E., Congdon, C. D., Stump, G, and Yukish, M. A., "Trade Space Exploration of a Wing Design Problem Using Visual Steering and Multi-Dimensional Data Visualization, " 4th AIAA Multidisciplinary Design Optimization Specialist Conference, AIAA Paper 2008-2139, 2008.
205. Stump, G, Lego, S., Yukish, M., Simpson, T W, and Donndelinger, J. A., "Visual Steering Commands for Trade Space Exploration: User-Guided Sampling with Example, " Journal of Computing and Information Science in Engineering, Vol. 9, No. 4, 2009, Paper 044501. doi: 10.1115/1.3243633
206. Ligetti, C, and Simpson, T W., "Metamodel-Driven Design Optimization Using Integrative Graphical Design Interfaces: Results from a Job Shop Manufacturing Simulation Experiment, " Journal of Computing and Information Science in Engineering, Vol. 5, No. 1, 2005, pp. 8-17. doi:10.1115/1.1794698 (Pubitemid 40496893)
207. Forrester, A. I. J, Sobester, A., and Keane, A. J., Engineering Design via Surrogate Modeling: A Practical Guide, Wiley, West Sussex, England, U.K., 2008.
208. Shan, S., and Wang, G G, "Survey of Modeling and Optimization Strategies to Solve High-Dimensional Design Problems with Computationally- Expensive Black-Box Functions, " Structural and Multidisciplinary Optimization, Vol. 41, No. 2, 2010, pp. 219-241. doi:10.1007/s00158-009-0420-2
209. Shan, S., and Wang, G G, "Metamodeling for High Dimensional Simulation-Based Design Problems, " Journal of Mechanical Design, Vol. 132, No. 5, 2010, Paper 051009. doi:10.1115/1.4001597
210. Shan, S., "Metamodeling Strategies for High-Dimensional Simulation-Based Design Problems, " Ph.D. Dissertation, Dept. Mechanical and Manufacturing Engineering, Univ. of Manitoba, Winnipeg, MB, Canada, 2010.
211. Wang, H., Tang, L., and Li, G Y, "Adaptive MLS-HDMR Metamodeling Techniques for High Dimensional Problems, " Expert Systems with Applications, Vol. 38, No. 11, 2011, pp. 14117-14126. doi:10.1016/j.eswa.2011. 04.220
212. Pineda, L., Fregly, B. J., Haftka, R., and Queipo, V, "Estimating Training Data Boundaries in Surrogate-Based Modeling, " Structural and Multidisciplinary Optimization, Vol. 42, No. 6, 2010, pp. 811-821. doi:10.1007/s00158-010-0541-7
213. Queipo, N, Pintos, S., and Nava, E., "Setting Targets in Surrogate-Based Optimization, " Journal of Global Optimization, Vol. 55, No. 4, 2013, pp. 857-875. doi:10.1007/s10898-011-9837-4
214. Chaudhuri, A., Haftka, R. T, and Viana, F A. C, "Efficient Global Optimization with Adaptive Target for Probability of Targeted Improvement, " 8th AIAA Multidisciplinary Design Optimization Specialist Conference, AIAA Paper 2012-1681, 2012.
215. Verde, A., Pintos, S., Queipo, N, and Haftka, R., "Assessing the Value of Another Cycle in Gaussian Process Surrogate-Based Optimization, " Structural and Multidisciplinary Optimization, Vol. 39, No. 5, 2009, pp. 459-475.
216. Chaudhuri, A., Haftka, R. T, and Watson, L. T, "How to Decide Whether to Run One More Cycle in Efficient Global Optimization, " 14th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, AIAA Paper 2012-5668, 2012.
217. Meckesheimer, M., Barton, R. R., Simpson, T W, and Booker, A., "Computationally Inexpensive Metamodel Assessment Strategies, " AIAA Journal, Vol. 40, No. 10, 2002, pp. 2053-2060. doi:10.2514/2.1538 (Pubitemid 35295818)
218. Seok, K., Hwang, D., and Cho, T W, "Prediction Intervals for Support Vector Machine Regression, " Communications in Statistics: Theory and Methods, Vol. 31, No. 10, 2002, pp. 1887-1898. doi:10.1081/STA-120014918 (Pubitemid 35336299)
219. Viana, F A. C, and Haftka, R. T, "Importing Uncertainty Estimates from One Surrogate to Another, " 50th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, AIAA Paper 2009-2237, 2009.
220. Martin, J. D., and Simpson, T W., "A Methodology to Manage System-Level Uncertainty during Conceptual Design, " ASME Journal of Mechanical Design, Vol. 128, No. 4, 2006, pp. 959-968. doi:10.1115/1.2204975 (Pubitemid 44123185)