Cross validation can estimate how well prediction variance correlates with error

AIAA Journal

Volumn 47, Issue 9, 2009, Pages 2266-2270

  Viana, Felipe A C ^a,b   Haftka, Raphael T ^a,b  

^a UNIVERSITY OF FLORIDA   (United States)

^b Department of Mechanical and Aerospace Engineering ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ABSOLUTE VALUES; ACCURATE PREDICTION; ANALYTICAL FUNCTIONS; BENCH-MARK PROBLEMS; CROSS VALIDATION; KRIGING; PREDICTION VARIANCE; RESPONSE SURFACE; SQUARE ROOTS; UNCERTAINTY STRUCTURE;

SURFACE PROPERTIES;

ERRORS;

EID: 73649132299     PISSN: 00011452     EISSN: None     Source Type: Journal    
DOI: 10.2514/1.42162     Document Type: Article

References (20)

1. Sacks, J., Welch, W. J., Mitchell, T. J., and Wynn, H. P., "Design and Analysis of Computer Experiments," Statistical Science, Vol. 4, No. 4, 1989, pp. 409-435. doi:10.1214/ss/1177012413.
2. Toropov, V. V., "Simulation Approach to Structural Optimization," Structural and Multidisciplinary Optimization, Vol. 1, No. 1, 1989, pp. 37-46.
3. Queipo, N. V., Haftka, R. T., Shyy, W., Goel, T., Vaidyanathan, R., and Tucker, P. K., "Surrogate-Based Analysis and Optimization," Progress in Aerospace Sciences, Vol. 41, No. 1, 2005, pp. 1-28. doi:10.1016/j.paerosci. 2005.02.001. (Pubitemid 40537187)
4. Simpson, T. W., Toropov, V. V., Balabanov, V. O., and Viana, F. A. C., "Design and Analysis of Computer Experiments in Multidisciplinary Design Optimization: A Review of How Far We Have Come-or Not," Proceedings of the 12th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, AIAA, Reston, VA, 10- 12 Sept. 2008.
5. Stein, M. L., Interpolation of Spatial Data: Some Theory for Kriging, Springer, New York, 1999.
6. Simpson, T. W., Mauery, T. M., Korte, J. J., and Mistree, F., "Kriging Models for Global Approximation in Simulation-Based Multidisciplinary Design Optimization," AIAA Journal, Vol. 39, No. 12, 2001, pp. 2233-2241. doi:10.2514/2.1234.
7. Box, G. E. P., Hunter, W. G., and Hunter, J. S., Statistics for Experimenters, Wiley, New York, 1978.
8. Myers, R. H., and Montgomery, D. C., Response Surface Methodology: Process and Product Optimization Using Designed Experiments, 2nd ed., Wiley, Hoboken, NJ, 2002.
9. Apley, D. W., Liu, J., and Chen, W., "Understanding the Effects of Model Uncertainty in Robust Design with Computer Experiments," Journal of Mechanical Design, Vol. 128, No. 4, 2006, pp. 745- 958.
10. Jones, D., Schonlau, M., and Welch, W., "Efficient Global Optimization of Expensive Black-Box Functions," Journal of Global Optimization, Vol. 13, No. 4, 1998, pp. 455-492. doi:10.1023/A:1008306431147.
11. Jin, R., Chen, W., and Sudjianto, A., "On Sequential Sampling for Global Metamodeling in Engineering Design," ASME 2002 Design Engineering Technical Conferences and Computer and Information in Engineering Conference, DETC-DAC34092, ASME, Fairfield, NJ, 2002.
12. den Hertog, D., Kleijnen, J. P. C., and Siem, A. Y. D., "The Correct Kriging Variance Estimated by Bootstrapping," Journal of the Operational Research Society, Vol. 57, No. 4, 2006, pp. 400- 409.
13. Kohavi, R., "A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection," Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, Morgan Kaufmann Publishers, Los Altos, CA, 1995, Vol. 2, No. 12, pp. 1137- 1143.
14. Meckesheimer, M., Booker, A. J., Barton, R. R., and Simpson, T. W., "Computationally Inexpensive Metamodel Assessment Strategies," AIAA Journal, Vol. 40, No. 10, 2002, pp. 2053-2060. doi:10.2514/2.1538.
15. Viana, F. A. C., Haftka, R. T., and Steffen, V., "Multiple Surrogates: How Cross-Validation Errors Can Help Us to Obtain the Best Predictor," Structural and Multidisciplinary Optimization (to be published).
16. Congdon, C. D., and Martin, J. D., "On Using Standard Residuals as a Metric of Kriging Model Quality," AIAA 2007-1928, 23- 26 April 2007.
17. Lophaven, S. N., Nielsen, H. B., and Søndergaard, J., DACE-A MATLAB Kriging Toolbox, Informatics and Mathematical Modelling, Technical University of Denmark, TR IMM-TR-2002-12, 2002.
18. Viana, F. A. C., SURROGATES ToolBox User's Guide, http://fchegury. googlepages.com, 2008.
19. Mckay, M. D., Beckman, R. J., and Conover, W. J., "A Comparison of Three Methods for SelectingValues of InputVariables from a Computer Code," Technometrics, Vol. 21, No. 1, 1979, pp. 239-245. doi:10.2307/1268522.
20. Dixon, L. C. W., and Szegö, G. P., Towards Global Optimization 2, North-Holland, Amsterdam, The Netherlands, 1978.