Using cross validation to design conservative surrogates

AIAA Journal

Volumn 48, Issue 10, 2010, Pages 2286-2298

  Viana, Felipe A C ^a,b   Picheny, Victor ^a,b   Haftka, Raphael T ^a,b  

^a UNIVERSITY OF FLORIDA   (United States)

^b Department of Mechanical and Aerospace Engineering ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPOSITE LAMINATE; CROSS VALIDATION; DESIGN UNDER UNCERTAINTY; ENGINEERING APPLICATIONS; FITTING ERROR; KRIGING; MAXIMUM STRESS; META MODEL; PROBABILISTIC DESIGN OPTIMIZATION; RESPONSE SURFACE; SAFE PREDICTION; SAFETY MARGIN; STATISTICAL ANALYSIS; SURROGATE MODEL;

DESIGN; OPTIMIZATION; QUALITY ASSURANCE; STRUCTURAL ANALYSIS;

SAFETY ENGINEERING;

EID: 78649607539     PISSN: 00011452     EISSN: None     Source Type: Journal    
DOI: 10.2514/1.J050327     Document Type: Article

References (58)

1. Sacks, J., Welch, W. J., Mitchell, T. J., and Wynn, H. P., "Design and Analysis of Computer Experiments," Statistical Science, Vol. 4, No. 4, 1989, pp. 409-435. doi:10.1214/ss/1177012413
2. Toropov, V. V., "Simulation Approach to Structural Optimization," Structural and Multidisciplinary Optimization, Vol. 1, No. 1, 1989, pp. 37-46. doi:10.1007/BF01743808
3. Queipo, N. V., Haftka, R. T., Shyy,W., Goel, T., Vaidyanathan, R., and Tucker, P. K., "Surrogate-Based Analysis and Optimization," Progress in Aerospace Sciences, Vol. 41, 2005, pp. 1-28. doi:10.1016/j.paerosci.2005.02. 001
4. Keane, A. J., and Nair, P. B., Computational Approaches for Aerospace Design: The Pursuit of Excellence, Wiley, Chichester, England, U.K., 2005.
5. Forrester, A., Sobester, A., and Keane, A., Engineering Design Via Surrogate Modelling: A Practical Guide, Wiley, New York, 2008.
6. Simpson, T. W., Toropov, V. V., Balabanov, V. O., and Viana, F. A. C., "Design and Analysis of Computer Experiments in Multidisciplinary Design Optimization: A Review of How Far We Have Come-Or Not," 12th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Victoria, BC, Canada, AIAA Paper 2008-5802, Sept. 2008.
7. Starnes, J. H., Jr., and Haftka, R. T., "Preliminary Design of Composite Wings for Buckling, Stress and Displacement Constraints," Journal of Aircraft, Vol. 16, 1979, pp. 564-570. doi:10.2514/3.58565
8. Picheny, V., Kim, N-H., Haftka, R. T., and Peters, J., "Conservative Estimation of Probability of Failure," 11th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, AIAA Paper 2006- 7038, Portsmouth, VA, Sept. 2006.
9. Acar, E., Kale, A., and Haftka, R. T., "Comparing Effectiveness of Measures that Improve Aircraft Structural Safety," Journal of Aerospace Engineering, Vol. 20, No. 3, 2007, pp. 186-199. doi:10.1061/(ASCE)0893- 1321(2007)20:3(186)
10. Acar, E., Haftka, R. T., and Johnson, T. F., "Tradeoff of Uncertainty Reduction Mechanisms for Reducing Structural Weight," Journal of Mechanical Design, Vol. 129, No. 3, 2007, pp. 266-274. doi:10.1115/1.2406097
11. Apley, D. W., Liu, J., and Chen, W., "Understanding the Effects of Model Uncertainty in Robust Design with Computer Experiments," Journal of Mechanical Design, Vol. 128, No. 4, 2006, pp. 945-958. doi:10.1115/1.2204974
12. Chen, S., Xiong, Y., and Chen, W., "Multiresponse and Multistage Metamodeling Approach for Design Optimization," AIAA Journal, Vol. 47, No. 1, 2009, pp. 206-218. doi:10.2514/1.38187
13. Lee, J., Jeong, H., Choi, D. H., Volovoi, V., and Mavris, D., "An Enhancement of Constraint Feasibility in BPN Based Approximate Optimization," Computer Methods in Applied Mechanics and Engineering, Vol. 196, 2007, pp. 2147-2160. doi:10.1016/j.cma.2006.11.005
14. Lee, J., Jeong, H., and Kang, S., "Derivative and GA-Based Methods in Metamodeling of Back-Propagation Neural Networks for Constrained Approximate Optimization," Structural and Multidisciplinary Optimization, Vol. 35, No. 1, 2008, pp. 29-40. doi:10.1007/s00158-007-0110-x
15. Picheny,V., Kim, N-H., Haftka, R. T., and Queipo, N.V., "Conservative Predictions Using Surrogate Modeling," 49th AIAA/ASME/ASCE/AHS/ ASC Structures, Structural Dynamics, and Materials Conference, Schaumburg, IL, AIAA Paper 2008-1716, April 2008.
16. Audet, C., Dennis, J. E., Moore, D.W., Booker, A., and Frank, P. D., "A Surrogate-Model-Based Method for Constrained Optimization," 8th AIAA/NASA/USAF/ISSMO Symposium on Multidisciplinary Analysis and Optimization, AIAA Paper 2000-4891, Long Beach, CA, Sept. 2000.
17. Picheny, V., Ginsbourger, D., Roustant, O., Haftka, R. T., and Kim, N-H., "Adaptive Design of Experiments for Accurate Approximation of a Target Region," Journal of Mechanical Design (submitted for publication).
18. Bichon, B. J., Mahadevan, S., and Eldred, M. S., "Reliability-Based Design Optimization Using Efficient Global Reliability Analysis," 50th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, AIAA Paper 2009-2261, Palm Springs, CA, May 2009.
19. Wu, Y-T., Shin, Y., Sues, R., and Cesare M "Safety-Factor Based Approach for Probability-Based Design Optimization," 42nd AIAA/ ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, AIAA Paper 2001-1522, Seattle, WA, April 2001.
20. Elishakoff, I., "Safety Factors and Reliability: Friends or Foes?," Kluwer Academic, Norwell, MA, 2004.
21. Acar, E., Kale, A., and Haftka, R. T., "Comparing Effectiveness of Measures that Improve Aircraft Structural Safety," Journal of Aerospace Engineering, Vol. 20, No. 3, 2007, pp. 186-199. doi:10.1061/(ASCE)0893- 1321(2007)20:3(186)
22. Kim, C., and Choi, K. K., "Reliability-Based Design Optimization Using Response Surface Method with Prediction Interval Estimation," Journal of Mechanical Design, Vol. 130, No. 12, 2008, Paper 121401. doi:10.1115/1.2988476
23. Picheny, V., "Improving Accuracy and Compensating for Uncertainty in Surrogate Modeling," Ph.D., Dissertation, Univ. of Florida, Gainesville, FL, 2009.
24. Kohavi, R., "A Study of Cross-Validation and Bootstrap for Accuracy Estimation And Model Selection," Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, Morgan Kaufmann, San Francisco, 1995, pp. 1137-1143.
25. Meckesheimer, M., Barton, R. R., Simpson, T. W., and Booker, A., "Computationally Inexpensive Metamodel Assessment Strategies," AIAA Journal, Vol. 40, No. 10, 2002, pp. 2053-2060. doi:10.2514/2.1538
26. Goel, T., Haftka, R. T., Shyy, W., and Queipo, N. V., "Ensemble of surrogates," Structural and Multidisciplinary Optimization, Vol. 33, No. 3, 2007, pp. 199-216. doi:10.1007/s00158-006-0051-9
27. Goel, T., Haftka, R. T., and Shyy, W., "Comparing Error Estimation Measures for Polynomial and Kriging Approximation of Noise-Free Functions," Structural and Multidisciplinary Optimization, Vol. 38, No. 5, 2009, pp. 429-442. doi:10.1007/s00158-008-0290-z
28. Viana, F. A. C., and Haftka, R. T., "Cross Validation Can Estimate How Well Prediction Variance Correlates with Error," AIAA Journal,Vol. 47, No. 9, 2009, pp. 2266-2270. doi:10.2514/1.42162
29. Viana, F. A. C., Haftka, R. T., and Steffen, V., "Multiple Surrogates: How Cross-Validation Errors Can Help Us to Obtain the Best Predictor," Structural and Multidisciplinary Optimization, Vol. 39, No. 4, 2009, pp. 439-457. doi:10.1007/s00158-008-0338-0
30. Stein, M. L., Interpolation of Spatial Data: Some Theory for Kriging, Springer, New York, 1999.
31. Simpson, T. W., Mauery, T. M., Korte, J. J., and Mistree, F., "Kriging Models for Global Approximation in Simulation-Based Multidisciplinary Design Optimization," AIAA Journal, Vol. 39, No. 12, 2001, pp. 2233-2241. doi:10.2514/2.1234
32. Box, G. E. P., Hunter, J. S., and Hunter, W. G., Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building, Wiley, New York, 1978.
33. Myers, R. H., and Montgomery, D. C., Response Surface Methodology: Process and Product Optimization Using Designed Experiments, Wiley, New York, 1995.
34. Simpson, T. W., Lin, D. K. J., and Chen, W., "Sampling Strategies for Computer Experiments: Design and Analysis," International Journal of Reliability and Applications, Vol. 2, No. 3, 2001, pp. 209-240. doi:10.1.1.7.3757
35. Giunta, A. A., Wojtkiewicz, S. F., and Eldred, M. S., "Overview of Modern Design of Experiments Methods for Computational Simulations," 41st AIAA Aerospace Sciences Meeting and Exhibit, AIAA Paper 2003-0649, Reno, NV, 6-9 Jan. 2003.
36. Myers, R. H., Classical and Modern Regression with Applications, Duxbury Press, Belmont, CA, 2000.
37. Congdon, C. D., and Martin, J. D., "On Using Standard Residuals as a Metric of Kriging Model Quality," 48th AIAA/ASME/ASCE/AHS/ ASC Structures, Structural Dynamics, and Materials Conference, AIAA Paper 2007-1928, Honolulu, HI, April 2007.
38. Goel, T., and Stander, N., "Comparing Three Error Criteria for Selecting Radial Basis Function Network Topology," Computer Methods in Applied Mechanics and Engineering, Vol. 198, Nos. 27-29, 2009, pp. 2137-2150. doi:10.1016/j.cma.2009.02.016
39. Stone, C. J., A Course in Probability and Statistics, Duxbury Press, Belmont, CA, 1995.
40. Brown, L. D., Cai, T. T., and DasGupta, A., "Interval Estimation for a Binomial Proportion," Statistical Science, Vol. 16, No. 2, 2001, pp. 101-133.
41. Wilson, E. B., "Probable Inference, the Law of Succession, and Statistical Inference," Journal of the American Statistical Association, Vol. 22, No. 158, 1927, pp. 209-212. doi:10.2307/2276774
42. Lophaven, S. N., Nielsen, H. B., and Søndergaard, J., DACE-A MATLAB Kriging Toolbox, Informatics and Mathematical Modelling, Rept. IMM-TR-2002-12, Technical University of Denmark, Lyngby, Copenhagen, Denmark, 2002.
43. Dixon, L. C. W., Szegö, G. P., Towards Global Optimization 2, North Holland, Amsterdam, 1978.
44. McKay, M. D., Beckman, R. J., and Conover,W. J., "A Comparison of Three Methods for SelectingValues of InputVariables from a Computer Code," Technometrics, Vol. 21, 1979, pp. 239-245. doi:10.2307/1268522
45. MATLAB, Software Package, Ver. 7.6 R2008a, The MathWorks, Inc., Natick, MA, 2008.
46. Qu, X., Haftka, R. T., Venkataraman, S., and Johnson, T., "Deterministic- and Reliability-Based Optimization of Composite Laminates for Propeller Tanks," AIAA Journal, Vol. 41, No. 10, 2003, pp. 2029-2036. doi:10.2514/2.1893
47. Kwon, Y. W., and Berner, J. M., "Matrix Damage of Fibrous Composites: Effect of Thermal Residual Stresses and Layer Sequences," Computers and Structures, Vol. 64, Nos. 1-4, 1997, pp. 375-382. doi:10.1016/S0045-7949(96)00432-4
48. Melchers, R. E., Structural Reliability Analysis and Prediction, 2nd ed., Wiley, New York, 1999.
49. Smarslok, B. P., Alexander, D., Haftka, R. T., Carraro, L., and Ginsbourger, D., "Separable Monte Carlo Applied to Laminated Composite Plates Reliability," 49th AIAA/ASME/ ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Schaumburg, IL, AIAA, 2008.
50. Park, J., and Sandberg, I.W., "Universal Approximation Using Radial- Basis-Function Networks," Neural Computation, Vol. 3, No. 2, 1991, pp. 246-257. doi:10.1162/neco.1991.3.2.246
51. Cheng, B., and Titterington, D. M., "Neural Networks:AReviewfrom a Statistical Perspective," Statistical Science, Vol. 9, No. 1, 1994, pp. 2-54. doi:10.1214/ss/1177010638
52. Smola, A. J., and Scholkopf, B., "A Tutorial on Support Vector Regression," Statistics and Computing, Vol. 14, No. 3, 2004, pp. 199- 222. doi:10.1023/B:STCO.0000035301.49549.88
53. Scholkopf, B., and Smola, A. J., Learning with Kernels, MIT Press, Cambridge, MA, 2002.
54. Viana, F. A .C., and Haftka, R. T., "Using Multiple Surrogates for Metamodeling," 7th ASMO-UK/ISSMO International Conference on Engineering Design Optimization [CD-ROM], Bath, England, U.K., July 2008.
55. Samad, A., Kim, K., Goel, T., Haftka, R. T., and Shyy, W., "Multiple Surrogate Modeling for Axial Compressor Blade Shape Optimization," Journal of Propulsion and Power, Vol. 24, No. 2, 2008, pp. 302-310. doi:10.2514/1.28999
56. Cherkassky, V., and, Ma., Y., "Practical Selection of SVM Parameters and Noise Estimation for SVM Regression," Neural Networks, Vol. 17, No. 1, 2004, pp. 113-126. doi:10.1016/S0893-6080(03)00169-2
57. Gunn, S. R., "Support Vector Machines for Classification and Regression," Image Speech and Intelligent Systems Research Group, Univ. of Southampton, Southampton, England, U.K., 1997.
58. Sanchez, E., Pintos, S., and Queipo, N. V., "Toward an Optimal Ensemble of Kernel-Based Approximations with Engineering Applications," Structural and Multidisciplinary Optimization, Vol. 36, No. 3, 2008, pp. 247-261. doi:10.1007/s00158-007-0159-6