Verification and validation benchmarks

Nuclear Engineering and Design

Volumn 238, Issue 3, 2008, Pages 716-743

  Oberkampf, William L ^a   Trucano, Timothy G ^a  

^a SANDIA NATIONAL LABORATORIES   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BENCHMARKING; CALIBRATION; COMPUTATIONAL METHODS; COMPUTER SIMULATION; RELIABILITY ANALYSIS;

COMPUTATIONAL SIMULATIONS; NUCLEAR WEAPON;

RADIOACTIVE WASTES;

EID: 39149123528     PISSN: 00295493     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.nucengdes.2007.02.032     Document Type: Article

References (100)

1. Abanto, J., Pelletier, D., Garon, A., Trepanier, J.-Y., Reggio, M., 2005. Verification of Some Commercial CFD Codes on Atypical CFD Problems. 43rd AIAA Aerospace Sciences Meeting and Exhibit, Reno, NV, AIAA Paper 2005-0682.
2. ABAQUS. ABAQUS Benchmarks Manual (2006), ABAQUS Inc.
3. Aeschliman D.P., and Oberkampf W.L. Experimental methodology for computational fluid dynamics code validation. AIAA J. 36 5 (1998) 733-741
4. AIAA. Guide for the Verification and Validation of Computational Fluid Dynamics Simulations (1998), American Institute of Aeronautics and Astronautics AIAA-G-077-1998
5. Ainsworth M., and Oden J.T. A Posteriori Error Estimation in Finite Element Analysis (2000), John Wiley, New York
6. ANS. Guidelines for the Verification and Validation of Scientific and Engineering Computer Programs for the Nuclear Industry (1987), American Nuclear Society ANSI/ANS-10.4-1987
7. ANSYS. ANSYS Verification Manual (2005), ANSYS, Inc. Release 10.0
8. ASME. Guide for Verification and Validation in Computational Solid Mechanics (2006), American Society of Mechanical Engineers, ASME V&V 10-2006
9. Axelsson O. Iterative Solution Methods (1996), Cambridge University Press, Cambridge, UK
10. Babuska I., and Strouboulis T. The Finite Element Method and its Reliability (2001), Oxford University Press, Oxford, UK
11. Beven K. Towards a coherent philosophy of modelling the environment. Proc. R. Soc. Lond. Ser. A 458 2026 (2002) 2465-2484
12. Blottner F.G. Influence of boundary approximations and conditions on finite-difference solutions. J. Comput. Phys. 48 2 (1982) 246-269
13. Botella O., and Peyret R. Computing singular solutions of the Navier-Stokes equations with the Chebyshev-Collocation Method. Int. J. Numer. Methods Fluids 36 2 (2001) 125-163
14. Box G.E.P., Hunter J.S., and Hunter W.G. Statistics for Experimenters: Design, Innovation, and Discovery (2005), John Wiley, New York
15. Carpenter M.H., and Casper J.H. Accuracy of Shock Capturing in Two Spatial Dimensions. AIAA J. 37 9 (1999) 1072-1079
16. In: Casey M., and Wintergerste T. (Eds). ERCOFTAC Special Interest Group on Quality and Trust in Industrial CFD: Best Practices Guidelines (2000), European Research Community on Flow, Turbulence, and Combustion
17. Chen W., Baghdasaryan L., Buranathiti T., and Cao J. Model validation via uncertainty propagation. AIAA J. 42 7 (2004) 1406-1415
18. Coleman H.W., and Steele Jr. W.G. Experimentation and Uncertainty Analysis for Engineers (1999), John Wiley, New York
19. Diskin, B., Thomas, J.L., 2002. Analysis of Boundary Conditions for Factorizable Discretizations of the Euler Equations. NASA/ICASE, NASA/CR-2002-211648.
20. DoD, 1996. DoD Instruction 5000.61: Modeling and Simulation (M&S) Verification, Validation, and Accreditation (VV&A). From www.dmso.mil/docslib.
21. DoD, 1996. Verification, Validation, and Accreditation (VV&A) Recommended Practices Guide. From www.dmso.mil/docslib.
22. Dowding K. Quantitative Validation of Mathematical Models. ASME International Mechanical Engineering Congress Exposition (2001), American Society of Mechanical Engineers, New York
23. Dowding K.J., Hills R.G., Leslie I., Pilch M., Rutherford B.M., et al. Case Study for Model Validation: Assessing a Model for Thermal Decomposition of Polyurethane Foam (2004), Sandia National Laboratories, SAND2004-3632
24. Eca L., and Hoekstra M. An Introduction to CFD Code Verification Including Eddy-Viscosity Models. ECCOMAS CFD 2006, Egmond aan Zee (2006), European Community on Computational Methods in Applied Sciences, The Netherlands
25. Eca L., and Hoekstra M. Verification of Turbulence Models with a Manufactured Solution. ECCOMAS CFD 2006, Egmond aan Zee (2006), European Community on Computational Methods in Applied Sciences, The Netherlands
26. Eca, L., Hoekstra, M., Hay, A., Pelletier, D., Roache, P.J., 2005. A Manufactured Solution for a Two-Dimensional Steady Wall-Bounded Incompressible Turbulent Flow. Instituto Superior Tecnico, IST Report D72-34.
27. ERCOFTAC, 2000. Portal to Fluid Dynamics Database Resources. From http://ercoftac.mech.surrey.ac.uk.
28. Ferson S., and Tucker W.T. Sensitivity in Risk Analyses with Uncertain Numbers (2006), Sandia National Laboratories, SAND2006-2801
29. Ferziger J.H., and Peric M. Computational Methods for Fluid Dynamics (1996), Springer-Verlag, New York
30. Fishman G.S. Monte Carlo: Concepts, Algorithms, and Applications (1995), Springer, New York
31. Helton J.C., and Davis F.J. Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems. Reliab. Eng. Syst. Saf. 81 1 (2003) 23-69
32. Helton J.C., Johnson J.D., Sallaberry C.J., and Storlie C.B. Survey of sampling-based methods for uncertainty and sensitivity analysis. Reliab. Eng. Syst. Saf. 91 10-11 (2006) 1175-1209
33. Hemsch M. Statistical analysis of computational fluid dynamic solutions from the Drag Prediction Workshop. J. Aircraft 41 1 (2004) 95-103
34. Hemsch M., and Morrison J.H. Statistical Analysis of CFD Solutions from 2nd Drag Prediction Workshop. 42nd AIAA Aerospace Sciences Meeting and Exhibit (2004), American Institute of Aeronautics and Astronautics, Reno, NV
35. Hills R.G. Model validation: model parameter and measurement uncertainty. J. Heat Transfer 128 4 (2006) 339-351
36. Hills R.G., and Leslie I. Statistical Validation of Engineering and Scientific Models: Validation Experiments to Application (2003), Sandia National Laboratories, SAND2003-0706
37. Hills R.G., and Trucano T.G. Statistical Validation of Engineering and Scientific Models: A Maximum Likelihood Based Metric (2002), Sandia National Laboratories, SAND2001-1783
38. Hirsch C. Numerical Computation of Internal and External Flows Fundamentals of Numerical Discretization vol. 1 (1988), John Wiley, New York
39. Hirsch C. Numerical Computation of Internal and External Flows: Computational Methods for Inviscid and Viscous Flows vol. 2 (1990), John Wiley, New York
40. IEEE, 1984. IEEE Standard Dictionary of Electrical and Electronics Terms, ANSI/IEEE Std 100-1984.
41. ISO. ISO 9000-3: Quality Management and Quality Assurance Standards-Part 3: Guidelines for the Application of ISO 9001 to the Development, Supply and Maintenance of Software (1991), International Standards Organization
42. Keller H.B. Accurate difference methods for linear ordinary differential systems subject to linear constraints. SIAM J. Numer. Anal. 6 (1969) 8-30
43. Knupp P., and Salari K. Verification of Computer Codes in Computational Science and Engineering (2002), Chapman & Hall/CRC, Boca Raton, FL
44. Kusnezov D.F. Advanced Simulation & Computing: The Next Ten Years (2004), Sandia National Laboratories, SAND2004-3740P
45. Laflin K.R., Klausmeyer S.M., Zickuhr T., Vassberg J.C., Wahls R.A., et al. Data summary from the second AIAA computational fluid dynamics drag prediction workshop. J. Aircraft 42 5 (2005) 1165-1178
46. Laney C.B. Computational Gasdynamics (1998), Cambridge University Press, Cambridge, UK
47. Levy D.W., Zickuhr T., Wahls R.A., Pirzadeh S., and Hemsch M.J. Data summary from the first AIAA computational fluid dynamics drag prediction workshop. J. Aircraft 40 5 (2003) 875-882
48. Mahadevan S., and Ramesh R. Validation of reliability computational models using Bayes networks. Reliab. Eng. Syst. Saf. 87 (2005) 223-232
49. Mehta, U.B., 1995. Guide to Credible Computational Fluid Dynamics Simulations. 26th AIAA Fluid Dynamics Conference. American Institute of Aeronautics and Astronautics, San Diego, CA, AIAA Paper 95-2225.
50. Montgomery D.C. Design and Analysis of Experiments (2000), John Wiley, Hoboken, NJ
51. Morton K.W. Numerical Solution of Convection-Diffusion Problems (1996), CRC Press, Boca Raton, FL
52. NAFEMS, 2006. NAFEMS Website. From www.NAFEMS.org.
53. NEA, 1977. Loss of Coolant Accident Standard Problems. Nuclear Energy Agency, Committee on the Safety of Nuclear Installations, Report No. 17.
54. NEA, 2004. CSNI International Standard Problem Procedures. Nuclear Energy Agency, Committee on the Safety of Nuclear Installations, Report No. 17-revision 4.
55. NPARC, 2000. CFD Verification &Validation: NPARC Alliance. From http://www.grc.nasa.gov/WWW/wind/valid/homepage.html.
56. Oberkampf, W.L., 1994. A Proposed Framework for Computational Fluid Dynamics Code Calibration/Validation. 18th AIAA Aerospace Ground Testing Conference. American Institute of Aeronautics and Astronautics, Colorado Springs, CO, AIAA Paper 94-2540.
57. Oberkampf W.L., and Aeschliman D.P. Joint computational/experimental aerodynamics research on a hypersonic vehicle: Part 1. Experimental results. AIAA J. 30 8 (1992) 2000-2009
58. Oberkampf, W.L., Aeschliman, D.P., Henfling, J.F., Larson, D.E., 1995. Surface Pressure Measurements for CFD Code Validation in Hypersonic Flow. 26th AIAA Fluid Dynamics Conference. American Institute of Aeronautics and Astronautics, San Diego, CA, AIAA Paper 95-2273.
59. Oberkampf W.L., Aeschliman D.P., Tate R.E., and Henfling J.F. Experimental Aerodynamics Research on a Hypersonic Vehicle (1993), Sandia National Laboratories, SAND92-1411
60. Oberkampf, W.L., Barone, M.F., 2004. Measures of Agreement Between Computation and Experiment: Validation Metrics. 34th AIAA Fluid Dynamics Conference. American Institute of Aeronautics and Astronautics, Portland, OR, AIAA Paper 2004-2626.
61. Oberkampf W.L., and Barone M.F. Measures of agreement between computation and experiment: validation metrics. J. Comput. Phys. 217 1 (2006) 5-36
62. Oberkampf W.L., and Blottner F.G. Issues in computational fluid dynamics code verification and validation. AIAA J. 36 5 (1998) 687-695
63. Oberkampf W.L., and Trucano T.G. Verification and validation in computational fluid dynamics. Prog. Aerospace Sci. 38 3 (2002) 209-272
64. Oberkampf W.L., Trucano T.G., and Hirsch C. Verification, validation, and predictive capability in computational engineering and physics. Appl. Mech. Rev. 57 5 (2004) 345-384
65. Oden J.T. In: Whiteman J.R. (Ed). Error Estimation and Control in Computational Fluid Dynamics. The Mathematics of Finite Elements and Applications (1993), John Wiley, New York 1-23
66. Pace, D.K., 1998. Synopsis of Fidelity Ideas and Issues. 1998 Spring Simulation Interoperatibility Workshop Papers.
67. Paez T.L., and Urbina A. Validation of Mathematical Models of Complex Structural Dynamic Systems. Proceedings of the Ninth International Congress on Sound and Vibration (2002), International Institute of Acoustics and Vibration, Orlando, FL
68. Prabhakar V., and Reddy J.N. Spectral/hp penalty least-squares finite element formulation for the steady incompressible Navier-Stokes Equations. J. Comput. Phys. 215 1 (2006) 274-297
69. QNET-CFD, 2001. Thematic Network on Quality and Trust for the Industrial Applications of CFD. From www.qnet-cfd.net.
70. Rebba R., Mahadevan S., and Huang S. Validation and error estimation of computational models. Reliab. Eng. Syst. Saf. 91 10-11 (2006) 1390-1397
71. Refsgaard J.C., and Henriksen H.J. Modelling guidelines-terminology and guiding principles. Adv. Water Resour. 27 (2004) 71-82
72. Rizzi A., and Vos J. Toward establishing credibility in computational fluid dynamics simulations. AIAA J. 36 5 (1998) 668-675
73. Roache P.J. Need for control of numerical accuracy. J. Spacecraft Rockets 27 2 (1990) 98-102
74. Roache P.J. Perspective: a method for uniform reporting of grid refinement studies. J. Fluids Eng. 116 (1994) 405-413
75. Roache, P.J., 1997. Quantification of Uncertainty in Computational Fluid Dynamics. Annual Review of Fluid Mechanics. J.L. Lumley, M. Van Dyke, Palo Alto, CA. Annu. Rev. 29, 126-160.
76. Roache P.J. Verification and Validation in Computational Science and Engineering (1998), Hermosa Publishers, Albuquerque, NM
77. Roache P.J. Verification of codes and calculations. AIAA J. 36 5 (1998) 696-702
78. Robert C.P. Monte Carlo Statistical Methods (1999), Springer-Verlag, New York
79. Roy C.J. Verification of Euler/Navier-Stokes codes using the method of manufactured solutions. Int. J. Numer. Methods Fluids 44 6 (2004) 599-620
80. Roy C.J., and Blottner F.B. Assessment of one- and two-equation turbulence models for hypersonic flows. J. Spacecraft Rockets 38 5 (2001) 699-710
81. Roy, C.J., McWherter-Payne, M.A., Oberkampf, W.L., 2000. Verification and Validation for Laminar Hypersonic Flowfields. Fluids 2000 Conference. American Institute of Aeronautics and Astronautics, Denver, CO, AIAA Paper 2000-2550.
82. Roy C.J., McWherter-Payne M.A., and Oberkampf W.L. Verification and validation for laminar hypersonic flowfields. Part 1: Verification. AIAA J. 41 10 (2003) 1934-1943
83. Roy C.J., Oberkampf W.L., and McWherter-Payne M.A. Verification and validation for laminar hypersonic flowfields. Part 2: Validation. AIAA J. 41 10 (2003) 1944-1954
84. Rumsey C.L., Rivers S.M., and Morrison J.H. Study of CFD variation on transport configurations for the second drag-prediction workshop. Comput. Fluids 34 7 (2005) 785-816
85. Rutherford B.M., and Dowding K.J. An Approach to Model Validation and Model-Based Prediction-Polyurethane Foam Case Study (2003), Sandia National Laboratories, SAND2003-2336
86. Rykiel E.J. Testing Ecological Models: the meaning of validation. Ecol. Modell. 90 3 (1996) 229-244
87. In: Saltelli A., Chan K., and Scott E.M. (Eds). Sensitivity Analysis (2000), Wiley, New York
88. Saltelli A., Tarantola S., Campolongo F., and Ratto M. Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models (2004), John Wiley, Chichester, England
89. Santner T.J., Williams B.J., and Notz W. The Design and Analysis of Computer Experiments (2003), Springer, New York
90. Soudah, J., Doebling, S., Sefcik, J., Pilch, M., Trucano, T. ASC V&V Program Strategy: Toward a Predictive Enterprise. National Nuclear Security Administration, in preparation.
91. Srivastava B.N., Werle M.J., and Davis R.T. A finite difference technique involving discontinuous derivatives. Comput. Fluids 7 1 (1979) 69-74
92. Trucano T.G., Easterling R.G., Dowding K.J., Paez T.L., Urbina A., et al. Description of the Sandia Validation Metrics Project (2001), Sandia National Laboratories, SAND2001-1339
93. Trucano T.G., Pilch M., and Oberkampf W.L. General Concepts for Experimental Validation of ASCI Code Applications (2002), Sandia National Laboratories, SAND2002-0341
94. Trucano T.G., Pilch M., and Oberkampf W.L. On the Role of Code Comparisons in Verification and Validation (2003), Sandia National Laboratories, SAND2003-2752
95. Trucano T.G., Post D.E., Pilch M., and Oberkampf W.L. Software Engineering Intersection with Verification and Validation of Higher Performance Computational Science Software: Some Observations (2005), Sandia National Laboratories, SAND2005-3662P
96. Trucano T.G., Swiler L.P., Igusa T., Oberkampf W.L., and Pilch M. Calibration, validation, and sensitivity analysis: What's What. Reliab. Eng. Syst. Saf. 91 10-11 (2006) 1331-1357
97. Turkel E. Accuracy of schemes with nonuniform meshes for compressible fluid-flows. Appl. Numer. Math. 2 6 (1986) 529-550
98. Vos J.B., Rizzi A., Darracq D., and Hirschel E.H. Navier-Stokes solvers in european aircraft design. Prog. Aerospace Sci. 38 8 (2002) 601-697
99. Walker M.A., and Oberkampf W.L. Joint computational/experimental aerodynamics research on a hypersonic vehicle: Part 2. Computational results. AIAA J. 30 8 (1992) 2010-2016
100. White F.M. Viscous Fluid Flow (1991), McGraw Hill, New York