Measures of agreement between computation and experiment: Validation metrics

Journal of Computational Physics

Volumn 217, Issue 1, 2006, Pages 5-36

  Oberkampf, William L ^a   Barone, Matthew F ^a  

^a SANDIA NATIONAL LABORATORIES   (United States)

Author keywords

Assessment of model accuracy; Experimental uncertainty; Fluid dynamics; Model credibility; Solid dynamics; Uncertainty quantification; Validation of computational models

Indexed keywords

COMPUTATION THEORY; COMPUTATIONAL METHODS; DECOMPOSITION; SHEAR FLOW; UNCERTAINTY ANALYSIS;

ASSESSMENT OF MODEL ACCURACY; COMPUTATIONAL MODELLING; EXPERIMENTAL UNCERTAINTY; FLUID-DYNAMICS; MODEL CREDIBILITY; MODELING ACCURACY; SOLID DYNAMIC; UNCERTAINTY QUANTIFICATIONS; VALIDATION METRIC; VALIDATION OF COMPUTATIONAL MODEL;

CURVE FITTING;

EID: 33746327131     PISSN: 00219991     EISSN: 10902716     Source Type: Journal    
DOI: 10.1016/j.jcp.2006.03.037     Document Type: Article

References (89)

1. W.L. Oberkampf, T.G. Trucano, Validation methodology in computational fluid dynamics, AIAA Paper 2000-2549.
2. Oberkampf W.L., and Trucano T.G. Verification and validation in computational fluid dynamics. Progress in Aerospace Sciences 38 (2002) 209-272
3. AIAA. Guide for the Verification and Validation of Computational Fluid Dynamics Simulations (1998), American Institute of Aeronautics and Astronautics AIAA-G-077-1998
4. Roache P.J. Need for control of numerical accuracy. Journal of Spacecraft and Rockets 27 (1990) 98-102
5. Roache P.J. Verification and Validation in Computational Science and Engineering (1998), Hermosa Publishers, Albuquerque, NM
6. Roache P.J. Verification of Codes and Calculations. AIAA Journal 36 (1998) 696-702
7. Oberkampf W.L., Trucano T.G., and Hirsch C. Verification, validation, and predictive capability in computational engineering and physics. Applied Mechanics Reviews 57 (2004) 345-384
8. Knupp P., and Salari K. Verification of Computer Codes in Computational Science and Engineering (2002), Chapman & Hall/CRC, Boca Raton, FL
9. Roache P.J. Code verification by the method of manufactured solutions. Journal of Fluids Engineering 124 (2002) 4-10
10. Pelletier D., Turgeon E., Lacasse D., and Borggaard J. Adaptivity, sensitivity, and uncertainty: towards standards of good practice in computational fluid dynamics. AIAA Journal 41 (2003) 1925-1933
11. Roy C.J., McWherter-Payne M.A., and Oberkampf W.L. Verification and validation for laminar hypersonic flowfields, Part 1: Verification. AIAA Journal 41 (2003) 1934-1943
12. Peraire J., and Patera A.T. Bounds for linear-functional outputs of coercive partial differential equations: local indicators and adaptive refinement. In: Ladeveze P., and Oden J.T. (Eds). Advances in Adaptive Computational Methods in Mechanics (1998), Elsevier
13. Ainsworth M., and Oden J.T. A Posteriori Error Estimation in Finite Element Analysis (2000), John Wiley, New York
14. Babuska I., and Strouboulis T. The Finite Element Method and its Reliability (2001), Oxford University Press, Oxford, UK
15. ASME. Council on Codes and Standards, Board of Performance Test Codes: Committee on Verification and Validation in Computational Solid Mechanics (2003), American Society of Mechanical Engineers
16. Oberkampf W.L., and Blottner F.G. Issues in computational fluid dynamics code verification and validation. AIAA Journal 36 (1998) 687-695
17. Oberkampf W.L. Design, execution, and analysis of validation experiments. In: Grasso F., Periauz J., and Deconinck H. (Eds). Verification and Validation of Computational Fluid Dynamics (2000), von Karman Institute for Fluid Dynamics, Rhode Saint Genese, Belgium
18. Aeschliman D.P., and Oberkampf W.L. Experimental methodology for computational fluid dynamics code validation. AIAA Journal 36 (1998) 733-741
19. Trucano T.G., Pilch M., and Oberkampf W.L. General Concepts for Experimental Validation of ASCI Code Applications (2002), Sandia National Laboratories SAND2002-0341
20. Roy C.J., Oberkampf W.L., and McWherter-Payne M.A. Verification and validation for laminar hypersonic flowfields, Part 2: Validation. AIAA Journal 41 (2003) 1944-1954
21. Bendat J.S., and Piersol A.G. Random Data: Analysis & Measurement Procedures (2000), Wiley, New York
22. Wirsching P., Paez T., and Ortiz K. Random Vibrations: Theory and Practice (1995), Wiley, New York
23. Devore J.L. Probability and Statistics for Engineers and Scientists (2000), Duxbury, Pacific Grove, CA
24. Montgomery D.C. Design and Analysis of Experiments (2000), John Wiley, Hoboken, NJ
25. Hills R.G., and Trucano T.G. Statistical Validation of Engineering and Scientific Models: A Maximum Likelihood Based Metric (2002), Sandia National Laboratories SAND2001-1783
26. T.L. Paez, A. Urbina, Validation of mathematical models of complex structural dynamic systems, in: Proceedings of the Ninth International Congress on Sound and Vibration, Orlando, FL, 2002.
27. Hills R.G., and Leslie I. Statistical Validation of Engineering and Scientific Models: Validation Experiments to Application (2003), Sandia National Laboratories SAND2003-0706
28. Dowding K.J., Hills R.G., Leslie I., Pilch M., Rutherford B.M., and Hobbs M.L. Case Study for Model Validation: Assessing a Model for Thermal Decomposition of Polyurethane Foam (2004), Sandia National Laboratories SAND2004-3632
29. Rutherford B.M., and Dowding K.J. An Approach to Model Validation and Model-Based Prediction-Polyurethane Foam Case Study (2003), Sandia National Laboratories SAND2003-2336
30. Chen W., Baghdasaryan L., Buranathiti T., and Cao J. Model validation via uncertainty propagation. AIAA Journal 42 (2004) 1406-1415
31. Gelman A.B., Carlin J.S., Stern H.S., and Rubin D.B. Bayesian Data Analysis (1995), Chapman & Hall, London
32. Bernardo J.M., and Smith A.F.M. Bayesian Theory (1994), John Wiley, New York
33. Leonard T., and Hsu J.S.J. Bayesian Methods: An Analysis for Statisticians and Interdisciplinary Researchers (1999), Cambridge University Press, Cambridge, UK
34. Hanson K.M. A framework for assessing uncertainties in simulation predictions. Physica D 133 (1999) 179-188
35. M.C. Anderson, T.K. Hasselman, T.G. Carne, Model correlation and updating of a nonlinear finite element model using crush test data, Paper No. 376, in: 17th International Modal Analysis Conference (IMAC) on Modal Analysis, Kissimmee, FL, 1999.
36. Kennedy M.C., and O'Hagan A. Bayesian calibration of computer models. Journal of the Royal Statistical Society Series B-Statistical Methodology 63 (2001) 425-450
37. T.K. Hasselman, G.W. Wathugala, J. Crawford, A hierarchical approach for model validation and uncertainty quantification, in: Fifth World Congress on Computational Mechanics, Vienna, Austria. Available from: , 2002.
38. DeVolder B., Glimm J., Grove J.W., Kang Y., Lee Y., Pao K., Sharp D.H., and Ye K. Uncertainty quantification for multiscale simulations. Journal of Fluids Engineering 124 (2002) 29-41
39. M.J. Bayarri et al., A framework for validation of computer models, in: Proceedings of the Workshop on Foundations for Verification and Validation in the 21st Century, John Hopkins University/Applied Physics Lab., 2002.
40. Zhang R., and Mahadevan S. Bayesian methodology for reliability model acceptance. Reliability Engineering and System Safety 80 (2003) 95-103
41. Geers T.L. An objective error measure for the comparison of calculated and measured transient response histories. The Shock and Vibration Bulletin 54 (1984) 99-107
42. D.M. Russell, Error measures for comparing transient data: Part I, development of a comprehensive error measure, in: Proceedings of the 68th Shock and Vibration Symposium, Hunt Valley, MD, 1997.
43. D.M. Russell, Error measures for comparing transient data: Part II, error measures case study, in: Proceedings of the 68th Shock and Vibration Symposium, Hunt Valley, MD, 1997.
44. Coleman H.W., and Stern F. Uncertainties and CFD code validation. Journal of Fluids Engineering 119 (1997) 795-803
45. Sprague M.A., and Geers T.L. Response of empty and fluid-filled, submerged spherical shells to plane and spherical, step-exponential acoustic waves. Shock and Vibration 6 (1999) 147-157
46. Stern F., Wilson R.V., Coleman H.W., and Paterson E.G. Comprehensive approach to verification and validation of CFD simulations-Part 1: methodology and procedures. Journal of Fluids Engineering 123 (2001) 793-802
47. Easterling R.G. Measuring the Predictive Capability of Computational Models: Principles and Methods. Issues and Illustrations (2001), Sandia National Laboratories SAND2001-0243
48. R.G. Easterling, Statistical Foundations for Model Validation: Two Papers, SAND2003-0287, Sandia National Laboratories, 2003.
49. W.L. Oberkampf, M.F. Barone, Measures of agreement between computation and experiment: validation metrics, AIAA Paper 2004-2626.
50. Coleman H.W., and Steele Jr. W.G. Experimentation and Uncertainty Analysis for Engineers (1999), John Wiley, New York
51. Taylor J.R. An Introduction to Error Analysis: The Sudy of Uncertainties in Physical Measurements (1997), University Science Books, Sausalito, CA
52. Youden W.J. Enduring values. Technometrics 14 (1972) 1-11
53. Morgan M.G., and Henrion M. Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis (1990), Cambridge University Press, Cambridge, UK
54. Winkler R.L. An Introduction to Bayesian Inference and Decision (1972), Holt, Rinehart, and Winston, New York
55. Haldar A., and Mahadevan S. Probability, Reliability, and Statistical Methods in Engineering Design (2000), John Wiley, New York
56. Hobbs M.L., Erickson K.L., and Chu T.Y. Modeling Decomposition of Unconfined Rigid Polyurethane Foam, SAND99-2758 (1999), Sandia National Laboratories
57. Gartling D.K., Hogan R.E., and Glass M.W. Coyote - A Finite Element Computer Program for Nonlinear Heat Conduction Problems, Part I - Theoretical Background (1994), Sandia National Laboratories SAND94-1173
58. Pruett C.D., Gatski T.B., Grosch C.E., and Thacker W.D. The temporally filtered Navier-Stokes equations: properties of the residual stress. Physics of Fluids 15 (2003) 2127-2140
59. DesJardin P.E., O'Hern T.J., and Tieszen S.R. Large eddy simulation of experimental measurements of the near-field of a large turbulent helium plume. Physics of Fluids 16 (2004) 1866-1883
60. Tieszen S.R., Domino S.P., and Black A.R. Validation of a simple turbulence model suitable for closure of temporally-filtered Navier-Stokes equations using a helium plume (2005), Sandia National Laboratories SAND 2005-3210
61. C.D. Moen, G.H. Evans, S.P. Domino, S.P. Burns, A multi-mechanics approach to computational heat transfer, IMECE-2002-33098, in: 2002 ASME International Mechanical Engineering Congress and Exhibition, New Orleans, LA, 2002.
62. O'Hern T.J., Weckman E.J., Gerhart A.L., Tieszen S.R., and Schefer R.W. Experimental Study of a Turbulent Buoyant Helium Plume, SAND2004-0549J (2004), Sandia National Laboratories
63. Miller R.G. Simultaneous Statistical Inference (1981), Springer-Verlag, New York
64. Draper N.R., and Smith H. Applied Regression Analysis (1998), John Wiley, New York
65. Seber G.A.F., and Wild C.J. Nonlinear Regression (2003), John Wiley, New York
66. M.F. Barone, W.L. Oberkampf, F.G. Blottner, Validation of compressibility corrections for two-equation turbulence models, AIAA Journal (in press).
67. Bogdanoff D.W. Compressibility effects in turbulent shear layers. AIAA Journal 21 (1983) 926-927
68. Lele S.K. Compressibility effects on turbulence. In: Lumley J.L., and Van Dyke M. (Eds). Annual Review of Fluid Mechanics (1994), Annual Reviews, Inc., Palo Alto, CA
69. Papamoschou D., and Roshko A. The compressible turbulent shear layer: an experimental study. Journal of Fluid Mechanics 197 (1988) 453-477
70. Chinzei N., Masuya G., Komuro T., Murakami A., and Kudou K. Spreading of two-stream supersonic turbulent mixing layers. Physics of Fluids 29 (1986) 1345-1347
71. Samimy M., and Elliott G.S. Effects of compressibility on the characteristics of free shear layers. AIAA Journal 28 (1990) 439-445
72. Elliot G.S., and Samimy M. Compressibility effects in free shear layers. Physics of Fluids A 2 (1990) 1231-1240
73. Goebel S.G., and Dutton J.C. Experimental study of compressible turbulent mixing layers. AIAA Journal 29 (1991) 538-546
74. J.C. Dutton, R.F. Burr, S.G. Goebel, N.L. Messersmith, Compressibility and mixing in turbulent free shear layers, in: 12th Symposium on Turbulence, Rolla, MO, 1990.
75. Gruber M.R., Messersmith N.L., and Dutton J.C. Three-dimensional velocity field in a compressible mixing layer. AIAA Journal 31 (1993) 2061-2067
76. Debisschop J.R., and Bonnet J.P. Mean and fluctuating velocity measurements in supersonic mixing layers. In: Rodi W., and Martelli F. (Eds). Engineering Turbulence Modeling and Experiments 2: Proceedings of the Second International Symposium on Engineering Turbulence Modeling and Measurement (1993), Elsevier, New York
77. Debisschop J.R., Chambers O., and Bonnet J.P. Velocity-field characteristics in supersonic mixing layers. Experimental Thermal and Fluid Science 9 (1994) 147-155
78. Barre S., Braud P., Chambres O., and Bonnet J.P. Influence of inlet pressure conditions on supersonic turbulent mixing layers. Experimental Thermal and Fluid Science 14 (1997) 68-74
79. Wilcox D.C. Turbulence Modeling for CFD (2002), DCW Industries
80. Nagano Y., and Hishida M. Improved form of the k-epsilon model for wall turbulent shear flows. Journal of Fluids Engineering 109 (1987) 156-160
81. Zeman O. Dilatation dissipation: the concept and application in modeling compressible mixing layers. Physics of Fluids A 2 (1990) 178-188
82. C.C. Wong, F.G. Blottner, J.L. Payne, M. Soetrisno, Implementation of a parallel algorithm for thermo-chemical nonequilibrium flow solutions, AIAA Paper 95-0152.
83. Wong C.C., Soetrisno M., Blottner F.G., Imlay S.T., and Payne J.L. PINCA: A Scalable Parallel Program for Compressible Gas Dynamics with Nonequilibrium Chemistry, SAND94-2436 (1995), Sandia National Laboratories
84. H.C. Yee, Implicit and Symmetric Shock Capturing Schemes, NASA-TM-89464, NASA, 1987.
85. S. Yoon, A. Jameson, An LU-SSOR Scheme for the Euler and Navier-Stokes Equations, AIAA Paper 87-0600.
86. Roache P.J. Perspective: a method for uniform reporting of grid refinement studies. Journal of Fluids Engineering 116 (1994) 405-413
87. Celik I., and Zhang W.M. Calculation of numerical uncertainty using Richardson extrapolation: application to some simple turbulent flow calculations. Journal of Fluids Engineering 117 (1995) 439-445
88. Paciorri R., and Sabetta F. Compressibility correction for the Spalart-Allmaras model in free-shear flows. Journal of Spacecraft and Rockets 40 (2003) 326-331
89. MathWorks, Matlab, The MathWorks, Inc.