The dangers of sparse sampling for the quantification of margin and uncertainty

Reliability Engineering and System Safety

Volumn 96, Issue 9, 2011, Pages 1220-1231

  Hemez, François M ^a   Atamturktur, Sezer ^b  

^a LOS ALAMOS NATIONAL LABORATORY   (United States)

^b Clemson University   (United States)

Author keywords

Gaussian process modeling; Sparse sampling; Statistical emulator

Indexed keywords

ADVERSE EFFECT; COMPUTER CODES; COMPUTER EXPERIMENT; DESIGN SPACES; ENGINEERED SYSTEMS; GAUSSIAN PROCESSES; GLOBAL SENSITIVITY ANALYSIS; HIGH DIMENSIONAL SPACES; MATHEMATICAL PROOF; METAMODELING; MODEL CALIBRATION; MODEL PARAMETERS; NUMERICAL MODELS; OPTIMIZATION SOLVERS; PHYSICS-BASED; PREDICTION ACCURACY; SAMPLING ALGORITHM; SPARSE SAMPLING; STATISTICAL EFFECTS; STATISTICAL EMULATOR; UNCERTAINTY PROPAGATION; VERIFICATION AND VALIDATION;

CALIBRATION; COMPUTATIONAL METHODS; COMPUTER SIMULATION; DESIGN; EXPERIMENTS; GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); PARAMETER ESTIMATION; RELIABILITY ANALYSIS; SENSITIVITY ANALYSIS; UNCERTAINTY ANALYSIS;

SAMPLING;

EID: 79959594169     PISSN: 09518320     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ress.2011.02.015     Document Type: Article

References (33)

1. J. Sacks, W.J. Welch, T.J. Mitchell, and H.P. Wynn Design and analysis of computer experiments Statistical Science 4 1989 409 423
2. T.J. Santner, B.J. Williams, and W.I. Notz The design and analysis of computer experiments 2003 Springer-Verlag Publishers
3. Maupin R, Hylok J, Rutherford A, Anderson M. Validation of a threaded assembly, Part I: overview. In: Proceedings of the sixth European conference on structural dynamics, Paris, France, September 57, 2005.
4. Hylok J, Rutherford A, Maupin R, Anderson M, Groethe M. Validation of a threaded assembly, Part II: experiments. In: Proceedings of the sixth European conference on structural dynamics, Paris, France, September 57, 2005 (Los Alamos technical report LA-UR-05-0931).
5. Rutherford A, Maupin R, Hylok J, Anderson M. Validation of a threaded assembly, Part III: validation. In: Proceedings of the sixth European conference on structural dynamics, Paris, France, September 57, 2005 (Los Alamos technical report LA-UR-05-2500).
6. Lam K, Allen D, Tippetts T. Engineering verification and validation assessment of Truchas for induction heating. Technical report LA-UR-07-7134, Los Alamos National Laboratory, Los Alamos, New Mexico, 2007.
7. Sigeti DE, Buescher KL, Vaughan DE, Cooley JH, Hemez FM. Simulation of integrated effects tests for the fiscal year 2008 ASC primary verification and validation milestone. Technical report LA-CP-08-1154, Los Alamos National Laboratory, Los Alamos, New Mexico, September 2008 (not available for public release).
8. Hemez FM, Atamturktur SH, Unal C. Prediction with quantified uncertainty of temperature and rate dependent material behavior. In: Proceedings of the 11th AIAA non-deterministic approaches conference, Palm Springs, California, May 47, 2009 (Los Alamos technical report LA-UR-08-6741).
9. Atamturktur HS, Lebensohn R, Higdon D, Williams B, Hemez FM, Unal C. Predicative maturity: a quantitative metric to optimize complex simulations via systematic experimental validation. Technical report LA-UR-09-7226, Los Alamos National Laboratory, Los Alamos, New Mexico, October 2007.
10. D.H. Sharp, and M.M. Wood-Schultz QMU and nuclear weapons certification: whats under the hood? Los Alamos Science 28 2003 47 53 (Los Alamos technical report LA-UR-03-5704)
11. Rutherford AC, Pepin JE, Hemez FM, Allen DW, Buechler MA, Hylok JE, et al. Defining a practical quantification of margins and uncertainties metric for engineering calculations. Technical report W-13-TR-008, Los Alamos National Laboratory, Los Alamos, New Mexico, October 2008.
12. Bement TR, Booker JM, Wood-Schultz MM, Nuclear package reliability and margin: a Los Alamos perspective. In: Proceedings of the 23rd aging, compatibility and stockpile stewardship conference, Sandia National Laboratories, Livermore, California, November 1416, 2000 (Los Alamos technical report LA-CP-00-0401, not available for public release).
13. Hemez FM, A technical note on the goodness-of-fit of statistical emulators. Technical report LA-CP-10-0062, Los Alamos National Laboratory, Los Alamos, New Mexico, January 2010 (not available for public release)
14. R.H. Myers, and D.C. Montgomery Response surface methodology: process and product optimization using designed experiments 1995 Wiley Inter-Science Publishers
15. A. Saltelli, K. Chan, and M. Scott Sensitivity analysis 2000 John Wiley & Sons Publishers
16. M.D. McKay, R.J. Beckman, and W.J. Conover A comparison of three methods for selecting values of input variables in the analysis of output from a computer code Technometrics 21 2 1979 239 245 (Pubitemid 9452235)
17. B. Tang Orthogonal array-based Latin hypercubes Journal of the American Statistical Association 88 1993 1392 1397
18. A. Saltelli, S. Tarantola, F. Campolongo, and M. Ratto Sensitivity analysis in practice: a guide to assessing scientific models Probability and statistics series 2004 John Wiley & Sons Publishers
19. H.H. Rosenbrock An automatic method for finding the greatest or least value of a function Computer Journal 3 1960 175 184
20. H.R. Lindman Analysis of variance in complex experimental designs 1975 W.H. Freeman & Co. Ltd Publishers
21. G.E. Box, J.S. Hunter, and W.G. Hunter Statistics for experimenters: design, innovation, and discovery 2nd Ed. 2005 Wiley-Interscience Publishers
22. M. Kennedy, and A. O'Hagan Predicting the output from a complex computer code when fast approximations are available Biometrika 87 2000 1 13
23. B. Williams, D. Higdon, J. Gattiker, L. Moore, M. McKay, and S. Keller-McNulty Combining experimental data and computer simulations with an application to flyer plate experiments, Bayesian Analysis 1 2006 765 792
24. D. Higdon, J. Gattiker, B. Williams, and M. Rightley Computer model calibration using high-dimensional output Journal of the American Statistical Association 103 482 June 2008 570 583
25. N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, and E. Teller Equations of state calculations by fast computing machines Journal of Chemical Physics 21 1953 1087 1091
26. W.K. Hastings Monte Carlo sampling methods using Markov chains and their applications Biometrika 57 1970 97 109
27. J. Oakley, and A. O'Hagan Probabilistic sensitivity analysis of complex models Journal of the Royal Statistical Society 66 2004 751 769 (Pubitemid 39015911)
28. Williams B, Gattiker J. Using the Gaussian process model for simulation analysis (GPM/SA) code. Technical report LA-UR-06-5431, Los Alamos National Laboratory, Los Alamos, New Mexico, December 2006.
29. Bingham D, Higdon D, Williams B. Gaussian process models for high dimensional computer experiments. Working technical paper, Simon-Frasier University, Vancouver, Canada, September 2009.
30. J.L. Loeppky, J. Sacks, and W.J. Welch Choosing the sample size of a computer experiment: a practical guide Technometrics 51 4 2009 366 376
31. M.D. Morris Factorial sampling plans for preliminary computational experiments Technometrics 33 1991
32. F. Campolongo, J. Cariboni, and A. Saltelli An effective screening design for sensitivity analysis of large models Environmental Modelling and Software vol. 22 2007 Elsevier Publishers pp. 15108
33. Hemez FM. Performance of the Morris one-at-a-time sensitivity analysis to explore large-dimensional functions. Technical report LA-UR-10-0069, Los Alamos National Laboratory, Los Alamos, New Mexico, January 2010.