Validation methodology in computational fluid dynamics

Fluids 2000 Conference and Exhibit

Volumn , Issue , 2000, Pages

  Oberkampf, William L ^a   Trucano, Timothy G ^a  

^a SANDIA NATIONAL LABORATORIES   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL FLUID DYNAMICS; ERRORS; OPERATIONS RESEARCH;

COMPUTATIONAL SIMULATION; COMPUTATIONAL VALIDATION; EXPERIMENTAL UNCERTAINTY; HYPERSONIC CRUISE MISSILE; PROBABILISTIC OUTPUT; QUANTITY UNCERTAINTIES; VALIDATION METHODOLOGIES; VERIFICATION-AND-VALIDATION;

UNCERTAINTY ANALYSIS;

EID: 85065572743     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.2514/6.2000-2549     Document Type: Conference Paper

References (159)

1. DoD, "DoD Directive No. 5000.59: Modeling and Simulation (M&S) Management," Department of Defense, available: www.dmso.mil/docslib/, 1994.
2. DoD, "Verification, Validation, and Accreditation (VV&A) Recommended Practices Guide," Defense Modeling and Simulation Office, Office of the Director of Defense Research and Engr., available: www.dmso.mil/docslib, 1996.
3. Cohen, M. L., Rolph, J. E., and Steffey, D. L., eds. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements, National Acadey Press, Washington, DC, 1998.
4. Chapman, D. R., Mark, H., and Pirtle, M. W., Computer vs. Wind Tunnels, in Astronautics & Aeronautics, 1975, pp. 22-30.
5. Bradley, R. G., "CFD Validation Philosophy," North Atlantic Treaty Organization, AGARD-CP-437, Fluid Dynamics Panel Symposium: Validation of Computational Fluid Dynamics, Lisbon, Portugal, 1988.
6. Marvin, J. G., "Accuracy Requirements and Benchmark Experiments for CFD Validation," AGARD, AGARD-CP-437, Fluid Dynamics Panel Symposium: Validation of Computational Fluid Dynamics, Lisbon, Portugal, 1988.
7. Neumann, R. D.,"CFD Validation-The Interaction of Experimental Capabilities and Numerical Computations," American Institute of Aeronautics and Astronautics, AIAA Paper No. 90-3030, 1990.
8. Mehta, U. B., "Some Aspects of Uncertainty in Computational Fluid Dynamics Results," Journal of Fluids Engineering, Vol. 113, No. 4, 1991, pp. 538-543.
9. Oberkampf, W. L., and Aeschliman, D. P., "Joint Computational/Experimental Aerodynamics Research on a Hypersonic Vehicle: Part 1, Experimental Results," AIAA Journal, Vol. 30, No. 8, 1992, pp. 2000-2009.
10. Dwoyer, D., "The Relation Between Computational Fluid Dynamics and Experiment," American Institute of Aeronautics and Astronautics, AIAA 17th Ground Testing Conf, Nashville, TN, 1992.
11. Bertin, J. J., Martellucci, A., Neumann, R. D., and Stetson, K. F., "Developing a Data Base for the Calibration and Validation of Hypersonic CFD Codes - Sharp Cones," American Institute of Aeronautics and Astronautics, AIAA Paper No. 93-3044, 24th AIAA Fluid Dynamics Conf, Orlando, FL, 1993.
12. Cosner, R. R., "Issues in Aerospace Application of CFD Analysis," AIAA Paper No. 94-0464, 32nd Aerospace Sciences Meeting & Exhibit, Reno, NV, 1994.
13. Marvin, J. G., "Perspective on Computational Fluid Dynamics Validation," AIAA Journal, Vol. 33, No. 10, 1995, pp. 1778-1787.
14. Oberkampf, W. L., Aeschliman, D. P., Henfling, J. F., and Larson, D. E., "Surface Pressure Measurements for CFD Code Validation in Hypersonic Flow," American Institute of Aeronautics and Astronautics, AIAA Paper No. 95-2273, 26th AIAA Fluid Dynamics Conf, San Diego, CA, 1995.
15. Aeschliman, D. P., and Oberkampf, W. L., "Experimental Methodology for Computational Fluid Dynamics Code Validation," AIAA Journal, Vol. 36, No. 5, 1998, pp. 733-741.
16. AIAA, "Guide for the Verification and Validation of Computational Fluid Dynamics Simulations," American Institute of Aeronautics and Astronautics, AIAA-G-077-1998, Reston, VA, 1998.
17. Coleman, H. W., and Stem, F., "Uncertainties and CFD Code Validation," Journal of Fluids Engineering, Vol. 119, 1997, pp. 795-803.
18. Popper, K. R., The Logic of Scientific Discovery, Basic Books, New York, 1959.
19. Popper, K. R., Conjectures and Refutations: The Growth of Scientific Knowledge, Routledge and Kegan, London, 1969.
20. Carnap, R., "Testability and Meaning," Philosophy of Science, Vol. Ill, 1963.
21. Conway, R. W., "Some Tactical Problems in Digital Simulation," Management Science, Vol. 10, No. 1, 1963, pp. 47-61.
22. Naylor, T. H., and Finger, J. M., "Verification of Computer Simulation Models," Management Science, Vol. 14, No. 2, 1967, pp. 92-101.
23. Churchman, C. W., The Systems Approach, Dell, New York, 1968.
24. Naylor, T. H., Computer Simulation Experiments with Models of Economic Systems, Wiley, New York, 1971.
25. Shannon, R. E., Systems Simulation: The Art and Science, Prentice-Hall, Inc., 1975.
26. Zeigler, B. P., Theory of Modelling and Simulation, 1st ed., John Wiley & Sons, New York, 1976.
27. Schlesinger, S., "Terminology for Model Credibility," Simulation, Vol. 32, No. 3, 1979, pp. 103-104.
28. Checkland, P. B., Systems Thinking, Systems Practice, John Wiley & Sons, New York, 1981.
29. Oren, T. I., "Concepts and Criteria to Assess Acceptability of Simulation Studies: A Frame of Reference," Communications of the ACM, Vol. No. 4, 1981, pp. 180-189.
30. Landry, M., Malouin, J.-L., and Oral, M., "Model Validation in Operations Research," European Journal of Operational Research, Vol. 14, 1983, pp. 207-220.
31. Sargent, R. G., "Simulation Model Validation," in Simulation and Model-Based Methodologies: An Integrative View, T.I. Oren, B.P. Zeigler, and M.S. Elzas Eds., Springer-Verlag, Berlin, 1984, pp. 537-555.
32. Oren, T. I., Zeigler, B. P., and Elzas, M. S., eds. Simulation and Model-Based Methodologies: An Integrative View, Springer-Verlag Berlin Heidelberg, 1984.
33. Banks, J., and Carson, J. S., II, Discrete-Event System Simulation, 1st ed., Prentice-Hall, Inc., Englewood Cliffs, New Jersey, 1984.
34. Sargent, R. G., "An Expository on Verification and Validation of Simulation Models," 1985 Winter Simulation Conference, Sacramento, CA, 1985.
35. Balci, O., and Nance, R. E., "Formulated Problem Verification as an Explicit Requirement of Model Credibility," Simulation, Vol. 45, No. 2, 1985, pp. 76-86.
36. Neelamkavil, F., Computer Simulation and Modelling, 1st ed., John Wiley & Sons, New York, 1987.
37. Bratley, P., Fox, B. L., and Schrage, L. E., A Guide to Simulation, 2nd ed., Springer-Verlag, New York, 1987.
38. Sargent, R. G., "Validation of Mathematical Models," Symposium on Validation of Geosphere Flow and Transport Models, Stockholm, Sweden, 1990.
39. Law, A. M., and Kelton, W. D., Simulation Modeling and Analysis, 2nd ed., McGraw-Hill, New York, 1991.
40. Fossett, C. A., Harrison, D., Weintrob, H., and Gass, S. I., "An Assessment Procedure for Simulation Models: A Case Study," Operations Research, Vol. 39, No. 5, 1991, pp. 710-723.
41. Hodges, J. S., and Dewar, J. A., "Is it You or Your Model Talking? A Framework for Model Validation," RAND, R-4114-AF/A/OSD, Santa Monica, 1992.
42. Davis, P. K., "Generalizing Concepts and Methods of Verification, Validation, and Accreditation (VV&A) for Military Simulations," RAND, R-4249-ACQ, Santa Monica, 1992.
43. Bailey, M. P., and Kemple, W. G., "The Scientific Method of Choosing Model Fidelity," 1992 Winter Simulation Conference Proceedings, Arlington, VA, 1992.
44. Wise, J. A., Hopkin, V. D., and Stager, P., eds. Verification and Validation of Complex Systems: Human Factors Issues, Springer-Verlag Berlin, Berlin, 1993.
45. Miser, H. J., "A Foundational Concept of Science Appropriate for Validation in Operational Research," European Journal of Operational Research, Vol. 66, 1993, pp. 204-215.
46. Gass, S. I., "Model Accreditation: A rationale and process for determining a numerical rating," European Journal of Operational Research, Vol. 66, 1993, pp. 250-258.
47. Landry, M., and Oral, M., "In Search of a Valid View of Model Validation for Operations Research," European Journal of Operational Research, Vol. 66, 1993, pp. 161-167.
48. Dery, R., Landry, M., and Banville, C., "Revisiting the Issue of Model Validation in OR: An Epistemological View," European Journal of Operational Research, Vol. 66, 1993, pp. 168-183.
49. Oral, M., and Kettani, O., "The Facets of the Modeling and Validation Process in Operations Research," European Journal of Operational Research, Vol. 66, 1993, pp. 216-234.
50. Leijnse, A., and Hassanizadeh, S. M., "Model Definition and Model Validation," Advances in Water Resources, Vol. 17, 1994, pp. 197-200.
51. Kleijnen, J. P. C., "Verification and Validation of Simulation Models," European Journal of Operational Research, Vol. 82, 1995, pp. 145-162.
52. Caughlin, D., "Verification, Validation, and Accreditation (VV&A) of Models and Simulations through Reduced Order Metamodels," 1995 Winter Simulation Conference, Arlington, VA, 1995.
53. Kleijnen, J. P. C., "Statistical Validation of Simulation Models," European Journal of Operational Research, 1995, pp. 21-34.
54. Sargent, R. G., "Verifying and Validating Simulation Models," 1996 Winter Simulation Conference, Coronado, California, 1996.
55. Pace, D. K., "Fidelity Considerations for RDE Distributed Simulation," 1, 1997 Fall Simulation Interoperability Workshop Papers, 1997.
56. Pace, D. K., "Dimensions and Attributes of Simulation Fidelity," 1, 1998 Fall Simulation Interoperability Workshop Papers, 1998.
57. Box, G. E. P., "Sampling and Bayes' Inference in Scientific Modeling and Robustness," Journal Statist. Soc. A, Vol. 143, No. A, 1980, pp. 383-430.
58. Oberkampf, W. L., Martellucci, A., and Kaestner, P. C., "SWERVE Surface Pressure Measurments at Mach Numbers 3 and 8," Sandia National Laboratories, SAND84-2149, SECRET Formerly Restricted Data, Albuquerque, NM, 1985.
59. Roache, P. J., "Need for Control of Numerical Accuracy," Journal of Spacecraft and Rockets, Vol. 27, No. 2, 1990, pp. 98-102.
60. Blottner, F. G., "Accurate Navier-Stokes Results for the Hypersonic Flow Over a Spherical Nosetip," Journal of Spacecraft and Rockets, Vol. 27, No. 2, 1990, pp. 113-122.
61. Mehta, U. B., "Computational Requirements for Hypersonic Flight Performance Estimates," Journal of Spacecraft and Rockets, Vol. 27, No. 2, pp. 103-112.
62. Martellucci, A., "The Challenging Process of Validating CFD Codes," American Institute of Aeronautics and Astronautics, AIAA-90-1402, AIAA 16th Aerodynamic Ground Testing Conference, Seattle, WA, 1990.
63. Desideri, J. A., Glowinski, R., and Periaux, J., eds. Hypersonic Flows for Reentry Problems, Vol. I: Survey Lectures and Test Cases for Analysis, Springer-Verlag, Berlin, 1991.
64. Desideri, J. A., Glowinski, R., and Periaux, J., eds. Hypersonic Flows for Reentry Problems, Vol. II: Test Cases-Experiments and Computations, Springer-Verlag, Berlin, 1991.
65. Walker, M. A., and Oberkampf, W. L., "Joint Computational/Experimental Aerodynamics Research on a Hypersonic Vehicle: Part 2, Computational Results," AIAA Journal, Vol. 30, No. 8, 1992, pp. 2010-2016.
66. Marvin, J. G., "CFD Validation Experiments for Hypersonic Flows," AIAA, AIAA-92-4024, AIAA 17th Aerospace Ground Testing Conference, Nashville, TN, 1992.
67. Lin, S. J., Barson, S. L., and Sindir, M. M., "Development of Evaluation Criteria and a Procedure for Assessing Predictive Capability and Code Performance," Advanced Earth-to-Orbit Propulsion Technology Conference, Marshall space Flight Center, Huntsville, AL, 1992.
68. Singhal, A. K., "Validation of CFD Codes and Assessment of CFD Simulations," Fourth International Symposium of Transport Phenomena and Dynamics of Rotating Machinery, Honolulu, Hawaii, 1992.
69. Dolling, D. S., "Problems in the Validation of CFD Codes Through Comparison with Experiment," North Atlantic Treaty Organization, AGARD, AGARD-CP-514, Fluid Dynamics Panel Symposium: Theoretical and Experimental Methods in Hypersonic Flows, Torino, Italy, 1992.
70. Graves, R., Jr., Software Validation: The Bridge from R&D to Industrial Application," American Institute of Aeronautics and Astronautics, AIAA-92-0587, 30th Aerospace Sciences Meeting & Exhibit, Reno, NV, 1992.
71. Abgrall, R., and Desideri, J. A., "The European Hypersonic Data Base: A New CFD Validation Tool for the Design of Space Vehicles," AIAA, AIAA-93-3045, AIAA 24th Fluid Dynamics Conference, Orlando, FL, 1993.
72. Oberkampf, W. L., "A Proposed Framework for Computational Fluid Dynamics Code Calibration/Validation," American Institute of Aeronautics and Astronautics, AIAA Paper No. 94-2540, 18th AIAA Aerospace Ground Testing Conference, Colorado Springs, CO, 1994.
73. Bussoletti, J. E., "CFD Calibration and Validation: The Challenges of Correlating Computational Model Results with Test Data," American Institute of Aeronautics and Astronautics, AIAA-94-2542, 18th AIAA Aerospace Ground Testing Conference, Colorado Springs, CO, 1994.
74. Settles, G. S., and Dodson, L. J., "Supersonic and Hypersonic Shock/Boundary-Layer Interaction Database," AIAA Journal, Vol. 32, No. 7, 1994, pp. 1377-1383.
75. AGARD, "A Selection of Experimental Test Cases for the Validation of CFD Codes," NATO Advisory Group for Aerospace Research & Development, AGARD-AR-303-Vol. I, 1994.
76. AGARD, "A Selection of Experimental Test Cases for the Validation of CFD Codes," NATO Advisory Group for Aerospace Research & Development, AGARD-AR-303-Vol. II, 1994.
77. AGARD, "Quality Assessment for Wind Tunnel Testing," NATO Advisory Group for Aerospace Research & Development (AGARD), AGARD-AR-304, 1994.
78. Mehta, U. B., "Guide to Credible Computational Fluid Dynamics Simulations," American Institute of Aeronautics and Astronautics, AIAA Paper No. 95-26th AIAA Fluid Dynamics Conference, San Diego, CA, 1995.
79. Roache, P. J., "Verification of Codes and Calculations," American Institute of Aeronautics and Astronautics, AIAA Paper No. 95-2224, 26th AIAA Fluid Dynamics Conference, San Diego, CA, 1995.
80. Aeschliman, D. P., Oberkampf, W. L., and Blottner, F. G., "A Proposed Methodology for CFD Code Verification, Calibration, and Validation," ICIASF, Paper 95-CH3482-7, 16th International Congrerss on Instrumentation for Aerospace Simulation Facilities, Dayton, OH, 1995.
81. Cosner, R. R.,"CFD Validation Requirements for Technology Transition," American Institute of Aeronautics and Astronautics, AIAA Paper No. 95-2227, 26th AIAA Fluid Dynamics Conference, San Diego, CA, 1995,
82. Oberkampf, W. L., Blottner, F. G., and Aeschliman, D. P., "Methodology for Computational Fluid Dynamics Code Verification/Validation," American Institute of Aeronautics and Astronautics, AIAA Paper No. 95-26th AIAA Fluid Dynamics Conf., San Diego, CA, 1995.
83. McDonald, W. W., "Introduction to the Submarine Damage Mechanisms Project," Naval Surface Warfare Center, IHTR 1824, Indian Head, 1995.
84. Mehta, U. B., "Guide to Credible Computer Simulations of Fluid Flows," Journal of Propulsion and Power, Vol. 12, No. 5, 1996, pp. 940-948.
85. Sindir, M. M., Barson, S. L., Chan, D. C., and Lin, W. H., "On the Development and Demonstration of a Code Validation Process for Industrial Applications," American Institute of Aeronautics and Astronautics, AIAA Paper No. 96-2032, 27th AIAA Fluid Dynamics Conf., New Orleans, LA, 1996.
86. Dolling, D. S., "Considerations in the Comparison of Experimental Data with Simulations Consistency of Math Models and Flow Physics," American Institute of Aeronautics and Astronautics, AIAA-96-2030, 27th Fluid Dynamics Conference, New Orleans, LA, 1996.
87. Cosner, R. R., "The Role of Validation in the CFD Process at McDonnell Douglas/St. Louis," American Institute of Aeronautics and Astronautics, ALAA-96-2273, 19th AIAA Advanced Measurement and Ground Testing Technology Conference, New Orleans, LA, 1996.
88. Barber, T. J., "The Role of Code Validation and Certification in the Design Environment," American Institute of Aeronautics and Astronautics, AIAA Paper No. 96-2033, 27th AIAA Fluid Dynamics Conference, New Orleans, LA, 1996.
89. Benek, J. A., Kraft, E. M., and Lauer, R. F., "Validation Issues for Engine/Airframe Integration," American Institute of Aeronautics and Astronautics, AIAA 96-2031, 27th AIAA Fluid Dynamics Conference, New Orleans, LA, 1996.
90. Rizzi, A., and Vos, J., "Towards Establishing Credibility in CFD Simulations," American Institute of Aeronautics and Astronautics, AIAA Paper No. 96-2029, 27th AIAA Fluid Dynamics Conference, New Orleans, LA, 1996.
91. Van Wie, D. M., and Rice, T., "Quantification of Data Uncertainties and Validation of CFD Results in the Development of Hypersonic Airbreathing Engines," American Institute of Aeronautics and Astronautics, AIAA 96-2028, 27th AIAA Fluid Dynamics Conference, New Orleans, LA, 1996.
92. Oberkampf, W. L., Aeschliman, D. P., Henfling, J. F., Larson, D. E., and Payne, J. L., "Surface Pressure Measurements on a Hypersonic Vehicle," American Institute of Aeronautics and Astronautics, AIAA Paper No. 96-0669, 34th Aerospace Sciences Meeting, Reno, NV, 1996.
93. Haynes, T. S., Reed, H. L., and Saric, W. S., "CFD Validation Issues in Transition Modeling," American Institute of Aeronautics and Astronautics, AIAA-96-2051, 27th AIAA Fluid Dynamics Conference, New Orleans, LA, 1996.
94. th International Space Planes and Hypersonic Systems and Technologies Conf, Norfolk, VA, 1996.
95. Porter, J. L., "A Summary/Overview of Selected Computational Fluid Dynamics (CFD) Code Validation/Calibration Activities," American Institute of Aeronautics and Astronautics, AIAA Paper No. 96-2053, 27th AIAA Fluid Dynamics Conference, New Orleans, LA, 1996.
96. Mair, H. U., "Preliminary Compilation of Underwater Explosion Benchmarks," SAVIAC, 1, 67th Shock and Vibration Symposium, 1996.
97. Beck, M. B., "Water Quality Modeling: A Review of the Analysis of Uncertainty," Water Resources Research, Vol. 23, No. 8, 1987, pp. 1393-1442.
98. Tsang, C.-F., "A Broad View of Model Validation," OECD, Paris, France, Proceedings of the Symposium on Safety Assessment of Radioactive Waste Repositories, Paris, France, 1989.
99. Ketelle, R. H., Lee, R. R., Bownds, J. M., and Rizk, T. A., "Model Validation Lessons Learned: A Case Study at Oak Ridge National Laboratory," Oak Ridge National Laboratory, CONF-89085406/DE89 015900, Oak Ridge, TN, 1989.
100. LeGore, T., "Benchmark Test Cases for Computational Fluid Dynamics," American Society of Mechanical Engineers, FED-Vol. 93, 1990 Spring Meeting of the Fluids Engineering Division, Toronto, Ontario, 1990.
101. Davis, P. A., Olague, N. E., and Goodrich, M. T., "Approaches for the Validation of Models Used for Performance Assessment of High-Level Nuclear Waste Repositories," Sandia National Laboratories, NUREG/CR-5537; SAND90-0575, Albuquerque, NM, 1991.
102. Sheng, G., Elzas, M. S., Oren, T. I., and Cronhjort, B. T., "Model Validation: A Systemic and Systematic Approach," Reliability Engineering and System Safety, Vol. 42, 1993, pp. 247-259.
103. Oreskes, N, Shrader-Frechette, K., and Belitz, K., Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences, in Science, 1994, pp. 641-646.
104. IEEE, "ANSI/IEEE Std 100-1984: IEEE Standard Dictionary of Electrical and Electronics Terms," 1984.
105. IEEE, "IEEE Standard Glossary of Software Engineering Terminology," IEEE, IEEE Std 610.12-1990, New York, 1991.
106. Knepell, P. L., and Arangno, D. C., Simulation Validation: A Confidence Assessment Methodology, 1st ed., IEEE Computer Society Press, Washington, 1993.
107. Rakitin, S. R., Software Verification and Validation, Artech House, Boston, MA, 1997.
108. ANS, "American Nuclear Society: Guidelines for the Verification and Validation of Scientific and Engineering Computer Programs for the Nuclear Industry," ANSI/ANS-10.4-1987, 1987.
109. NRC, "Regulatory Guide 1.168: Verification, Validation, Reviews, and Audits for Digital Computer Software Used in Safety Systems of Nuclear Power Plants," US Nuclear Regulatory Commission, Available: www.nrc.gov/NRC/RG/01/01-168.htmt, Washington, 1997.
110. ISO, "ISO 9000-3: Quality Management and Quality Assurance Standards - Part 3: Guidelines for the application of ISO 9001 to the development, supply and maintenance of software," International Standards Org., Geneva, Switzerland, 1991.
111. Schmauch, C. H., ISO 9000 for Software Developers, ASQC Quality Press, Milwaukee, WI, 1994.
112. Schwer, L. E., Personal Communication, 1999.
113. Sindir, M. M., and Lynch, E. D., "Overview of the State-of-Practice of Computational Fluid Dynamics in Advanced Propulsion System Design," American Institute of Aeronautics and Astronautics, AIAA Paper No. 97-2124, 28th AIAA Fluid Dynamics Conference, Snowmass, CO, 1997.
114. Saaty, T. L., The Analytic Hierarchy Process, McGraw-Hill, New York, 1980.
115. Lee, L. H., and Poolla, K., "Statistical Validation for Uncertainty Models," Springer-Verlag, Lecture Notes in Control and Information Sciences, Vol. 202, Feedback Control, Complexity, and Identification: A festschrift for Professor George Zames, Montreal, Canada, 1994.
116. Lee, L. H., and Poolla, K., "On Statistical Model Validation," Journal of Dynamic Systems, Measurement and Control, Vol. 118, 1996, pp. 226-236.
117. Draper, D., "Assessment and Propagation of Model Uncertainty," Journal of the Royal Statistical Society B, Vol. 57, No. 1, 1995, pp. 45-97.
118. Laskey, K. B., "Model Uncertainty: Theory and Practical Implications," IEEE Transactions on Systems, Man and Cybernetics-Part A: Systems and Humans, Vol. 26, No. 3, 1996, pp. 340-348.
119. McKay, M. D., "Evaluating Prediction Uncertainty," Los Alamos National Labs., NUREG/CR-6311 LA-12915-MS, Los Alamos, NM, 1995.
120. Hanson, K. M., "A Framework for Assessing Uncertainties in Simulation Predictions," Physica Vol. 133, 1999, pp. 179-188.
121. Hills, R. G., "Statistical Validation of Engineering and Scientific Models with Application to CTH," Sandia National Labs., Albuquerque, NM, 2000.
122. Wilson, G. E., and Boyack, B. E., "The Role of the PIRT in Experiments, Code Development and Code Applications Associated With Reactor Safety Assessment," Nuclear Engineering and Design, Vol. 186, 1998, pp. 23-37.
123. Oberkampf, W. L., Aeschliman, D. P., Tate, R., and Henfling, J. F., "Experimental Aerodynamics Research on a Hypersonic Vehicle," Sandia National Labs., SAND92-1411, Albuquerque, NM, 1993.
124. Oberkampf, W. L., and Blottner, F. G., "Issues in Computational Fluid Dynamics Code Verification and Validation," AIAA Journal, Vol. 36, No. 5, 1998, pp. 687-695.
125. AIAA, "Assessment of Wind Tunnel Data Uncertainty With Application to Wind Tunnel Testing," American Institute of Aeronautics and Astronautics, S-071A-1999, Reston, VA, 1999.
126. Coleman, H. W., and Steele, W. G., Jr., Experimentation and Uncertainty Analysis for Engineers, Second Edition ed., John Wiley & Sons, New York, 1999.
127. Box, G. E. P., Hunter, W. G., and Hunter, J. S., Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building, Wiley, New York, 1978.
128. Milliken, G. A., and Johnson, D. E., Analysis of Messy Data: Vol. I; Designed Experiments, Lifetime Learning Pubs., Belmont, CA, 1984.
129. Youden, W. J., "Enduring Values," Technometrics, Vol. 14, No. 1, 1972, pp. 1-11.
130. Roache, P. J., Verification and Validation in Computational Science and Engineering, Hermosa Publishers, Albuquerque, NM, 1998.
131. Oberkampf, W. L., Diegert, K. V., Alvin, K. F., and Rutherford, B. M., "Variability, Uncertainty, and Error in Computational Simulations," ASME, ASME-HTD-Vol. 357-2, AlAA/ASME Joint Thermophysics and Heat Transfer Conference, Albuquerque, NM, 1998.
132. Klir, G. J., and Folger, T. A., Fuzzy Sets, Uncertainty', and Information, 1st ed., Prentice Hall, Englewood Cliffs, NJ, 1988.
133. Walley, P., Statistical Reasoning with Imprecise Probabilities, Chapman and Hall, London, 1991.
134. Klir, G. J., and Wierman, M. J., Uncertainty-Based Information: Elements of Generalized Information Theory, Vol. 15, Physica-Verlag, Heidelberg, 1998.
135. Pal, N. R., "On Qunatification of Different Facets of Uncertainty," Fuzzy Sets and Systems, Vol. 1999, pp. 81-91.
136. Oberkampf, W. L., DeLand, S. M., Rutherford, B. M., Diegert, K. V., and Alvin, K. F., "Estimation of Total Uncertainty in Computational Simulation," Sandia National Laboratories, SAND2000-0824, Albuquerque, NM, 2000.
137. Richtmeyer, R. D., and Morton, K. W., Difference Methods for Initial-Value Problems, Interscience, New York, NY, 1967.
138. Morton, K. W., Numerical Solution of Convection-Diffusion Problems, CRC Press, Boca Raton, FL, 1996.
139. Alvin, K. F., Oberkampf, W. L., Rutherford, B. M., and Diegert, K. V., "Methodology for Characterizing Modeling and Discretization Uncertainties in Computational Simulation," Sandia National Laboratories, SAND2000-0515, Albuquerque, NM, 2000.
140. Richardson, L. F., and Gaunt, J. A., "The Deferred Approach to the Limit," Transactions of the Royal Society of London, Series A: Mathematical and Physical Sciences, Vol. 226, 1927, pp. 299-361
141. Babuska, I., and Oh, H.-S., "Pollution Problem of the p-and h-p Versions of the Finite Element Method," Communications in Applied Numerical Methods, Vol. 3, 1987, pp. 553-561.
142. Babuska, I., Strouboulis, T., Upadhyay, C. S., and Gangaraj, S. K., "A Posteriori Estimation and Adaptive Control of the Pollution Error in the Inversion of the Finite Element Method," International Journal of Numerical Methods in Engineering, Vol. 38, 1995, pp. 4207-4235.
143. Babuska, I., Ihlenburg, F., Strouboulis, T., and Gangaraj, S. K., "A Posteriori Error Estimation for Finite Element Solutions of Helmholtz' Equation - Part II: Estimation of the Pollution Error," International Journal of Numerical Methods in Engineering, Vol. 40, 1997, pp. 3883-3900.
144. Oden, J. T., Feng, Y., and Prudhomme, S., "Local and Pollution Eiror Estimation For Stokesian Flow," International Journal of Numerical Methods in Fluids, Vol. 27, 1998, pp. 33-39.
145. Zhang, X. D., Pelletier, D., Trepanier, J. Y., and Camarero, R., "Verification of Error Estimators for the Euler Equations," American Institute of Aeronautics and Astronautics, AIAA-2000-1001, 38th A1AA Aerospace Sciences Meeting, Reno, NV, 2000.
146. Yee, H. C., and Sweby, P. K., "Aspects of Numerical Uncertainties in Time Marching to Steady-State Numerical Solutions," AIAA Journal, Vol. 36, No. 5, 1998, pp. 712-724.
147. Gamerman, D., Markov Chain Monte Carlo, Chapman & Hall, London, 1997.
148. Red-Horse, J. R., Paez, T. L., Field, R. V., and Romero, V., "Nondeterministic Analysis of Mechanical Systems," Sandia National Laboratories, SAND2000-0890, Albuqueque, NM, 2000.
149. Kleijnen, J. P. C., Statistical Tools for Simulation Practitioners, 1st ed., Marcel Dekker, Inc., New York, 1987.
150. Palmer, T. N., "Predicting Uncertainty in Forecasts of Weather and Climate," Reports on Progress in Physics, Vol. 63, 2000, pp. 71-116.
151. Glimm, J., and Sharp, D. H., "Stochastic Methods for the Prediction of Complex Multiscale Phenomena," Los Alamos National Laboratory, LAUR-97-3748, 1997.
152. Earman, J., Bayes or Bust?, The MIT Press, Cambridge, MA, 1992.
153. Chen, S., Margolin, L., and Sharp, D. H., eds. "Predicatability: Quantifying Uncertainty in Models of Complex Phenomena," 133, Physic a D, 1999.
154. Glimm, J., Hou, S., Kim, H., Sharp, D. H., and Ye, K., "A Probability Model for Errors in the Numerical Solutions of a Partial Differential Equation," Los Alamos National Laboratory, LAUR-99-5352, Los Alamos, NM, 1999.
155. Chorin, A. J., Kast, A. P., and Kupferman, R., "On the Prediction of Large-Scale Dynamics Using Unresolved Computations," Lawrence Berkeley National Laboratory, LBNL-42283, Berkeley, CA, 1998.
156. Chorin, A. J., Kast, A. P., and Kupferman, R., "Optimal Prediction of Underresolved Dynamics," Proceedings of the National Academy of Sciences, Vol. 95, 1998, pp. 4094-4098.
157. Chorin, A. J., Kast, A. P., and Kupferman, R., "Unresolved Computation and Optimal Prediction," Communications in Pure and Applied Mathematics, Vol. 52, 1999, pp. 1231-1254.
158. Lehmann, E. L., Testing Statistical Hypotheses, Wiley, New York, 1986.
159. Roache, P. J., "Discussion: Uncertainties and CFD Code Validation," Journal of Fluids Engineering, Vol. 120, 1998, pp. 635-636.