Verification and validation of simulation models

Proceedings - Winter Simulation Conference

Volumn , Issue , 2009, Pages 162-176

  Sargent, Robert G ^a  

^a Syracuse University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONCEPTUAL MODEL; DATA VALIDITY; MODEL ACCREDITATION; MODEL DEVELOPMENT; MODEL VALIDATION; MODEL VALIDITY; MODEL VERIFICATION; SIMULATION MODEL; VERIFICATION AND VALIDATION;

ACCREDITATION;

EID: 77951551371     PISSN: 08917736     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/WSC.2009.5429327     Document Type: Conference Paper

References (60)

1. Anderson, H. A. and R. G. Sargent. 1974. An investigation into scheduling for an interactive computer system. IBM Journal of Research and Development 18 (2): 125-137.
2. Balci, O. 1989. How to assess the acceptability and credibility of simulation results. In Proc. 1989 Winter Simulation Conf., ed. E. A. MacNair, K. J. Musselman, and P. Heidelberger, 62-71. Piscataway, New Jersey: IEEE.
3. Balci, O. 2001. A methodology for certification of modeling and simulation applications. ACM Trans. on Modeling and Computer Simulation 11 (4), 352-377.
4. Balci, O. 2003. Validation, verification, and certification of modeling and simulation applications. In Proc. 2003 Winter Simulation Conf., ed. S. Chick, P. J. Sanchez, E. Ferrin, and D. J. Morrice, 150-158. Piscataway, New Jersey: IEEE.
5. Balci, O. 2004. Quality assessment, verification, and validation of modeling and simulation applications. In Proc. 2004 Winter Simulation Conf., ed. R. G. Ingalls, M. D. Rossetti, J. S. Smith, and H. A. Peters, 122-129. Piscataway, New Jersey: IEEE. (Pubitemid 40575487)
6. Balci, O., R. E. Nance, J. D. Arthur, and W. F. Ormsby. 2002. Expanding our horizons in verification, validation, and accreditation research and practice. In Proc. 2001 Winter Simulation Conf., ed. B. A. Peters, J. S. Smith, D. J Medeiros, and M. W. Rohrer, 653-663. Piscataway, New Jersey: IEEE.
7. Balci, O. and R. G. Sargent. 1981. A methodology for cost-risk analysis in the statistical validation of simulation models. Comm. of the ACM 24 (6): 190-197. (Pubitemid 12444324)
8. Balci, O. and R. G. Sargent. 1982a. Validation of multivariate response simulation models by using Hotelling's two-sample T2 test. Simulation 39(6): 185-192. (Pubitemid 13559952)
9. Balci, O. and R. G. Sargent. 1982b. Some examples of simulation model validation using hypothesis testing. In Proc. 1982 Winter Simulation Conf., ed. H. J. Highland, Y. W. Chao, and O. S. Madrigal, 620-629. Piscataway, New Jersey: IEEE.
10. Balci, O. and R. G. Sargent. 1983. Validation of multivariate response trace-driven simulation models. In Performance 83, ed. Agrawada and Tripathi, 309-323. North Holland.
11. Balci, O. and R. G. Sargent. 1984a. A bibliography on the credibility assessment and validation of simulation and mathematical models. Simuletter 15 (3): 15-27.
12. Balci, O. and R. G. Sargent. 1984b. Validation of simulation models via simultaneous confidence intervals. American Journal of Mathematical and Management Science 4 (3): 375-406. (Pubitemid 17573175)
13. Banks, J., J. S. Carson II, B. L. Nelson, and D. Nicol. 2005. Discrete-event system simulation. 4th ed. Englewood Cliffs, NJ: Prentice-Hall.
14. Banks, J., D. Gerstein, and S. P. Searles. 1988. Modeling processes, validation, and verification of complex simulations: a survey. In Methodology and Validation, Simulation Series, Vol. 19, No. 1, The Society for Computer Simulation, 13-18. San Diego, CA: Society for Modeling and Simulation International. (Pubitemid 18614631)
15. Carson II, J. S. 2002. Model verification and validation. In Proc. 2002 Winter Simulation Conf., ed. E. Yucesan, C.-H. Chen, J.L. Snowdon, J. M. Charnes, 52-58. Piscataway, New Jersey: IEEE.
16. DoDI 5000.61. 2003. DoD modeling and simulation verification, validation, and accreditation, May 13, 2003.
17. Fairley, R. E. 1976. Dynamic testing of simulation software. In Proc. 1976 Summer Computer Simulation Conf., 40-46. San Diego, CA: Society for Modeling and Simulation International.
18. Gass, S. I. 1983. Decision-aiding models: validation, assessment, and related issues for policy analysis. Operations Research 31 (4): 601-663.
19. Gass, S. I. 1984. Documenting a computer-based model. Interfaces 14 (3): 84-93.
20. Gass, S. I. 1993. Model accreditation: a rationale and process for determining a numerical rating. European Journal of Operational Research 66 (2): 250-258.
21. Gass, S. I. and L. Joel. 1987. Concepts of model confidence. Computers and Operations Research 8 (4): 341-346.
22. Gass, S. I. and B. W. Thompson. 1980. Guidelines for model evaluation: an abridged version of the U. S. general accounting office exposure draft. Operations Research 28 (2): 431-479.
23. Johnson, R. A. 2005. Miller and Freund's probability and statistics for engineers, 7th ed. Englewood Cliffs, NJ: Prentice-Hall.
24. Kleijnen, J. P. C. 1987. Statistical tools for simulation practitioners. New York: Marcel Dekker.
25. Kleijnen, J. P. C. 1999. Validation of models: statistical techniques and data availability. In Proc. 1999 Winter Simulation Conf., ed. P. A. Farrington, H. Black Nembhard, D. T. Sturrock, and G. W. Evans, 647-654. Piscataway, New Jersey: IEEE.
26. Kleindorfer, G. B. and R. Ganeshan. 1993. The philosophy of science and validation in simulation. In Proc. 1993 Winter Simulation Conf., ed. G. W. Evans, M. Mollaghasemi, E. C. Russell, and W. E. Biles, 50-57. Piscataway, New Jersey: IEEE.
27. Knepell, P. L. and D. C. Arangno. 1993. Simulation validation: a confidence assessment methodology. IEEE Computer Society Press.
28. Landry, M, J. L. Malouin, and M. Oral. 1983. Model validation in operations research. European Journal of Operational Research 14: 207-220. (Pubitemid 14471442)
29. Law, A. M. 2007. Simulation modeling and analysis. 4th ed. New York: McGraw-Hill.
30. Law, A. M. 2008. How to build valid and credible simulation models. In Proc. 2008 Winter Simulation Conf., ed. S. J. Mason, R. Hill, L. Moench, and O. Rose, 39-47. Piscataway, New Jersey: IEEE.
31. Lowery, J. 1996. Design of hospital admissions scheduling system using simulation. In Proc. 1996 Winter Simulation Conf., ed. J. M. Charnes, D. J. Morrice, D. T. Brunner, and J. J. Swain, 1199-1204. Piscataway, New Jersey: IEEE.
32. Naylor, T. H. and J. M. Finger. 1967. Verification of computer simulation models. Management Science 14 (2): B92-B101.
33. Rao, M. J. and R. G. Sargent. 1988. An advisory system for operational validity. In Artificial Intelligence and Simulation: the Diversity of Applications, ed. T. Hensen, 245-250. San Diego, CA: Society for Computer Simulation.
34. Robinson, S. 2004. Simulation: The Practice of Model Development and Use. Chichester, West Sussex, England: John Wiley.
35. Rueß, H. and L. de Moura. 2003. From simulation to verification (and back). In Proc. 2003 Winter Simulation Conf., ed. S. Chick, P. J. Sanchez, E. Ferrin, and D. J. Morrice, 888-896. Piscataway, New Jersey: IEEE.
36. Sargent, R. G. 1979. Validation of simulation models. In: Proc. 1979 Winter Simulation Conf., ed. H. J. Highland, M. F. Spiegel, and R. E. Shannon, 497-503. Piscataway, New Jersey: IEEE.
37. Sargent, R. G. 1981. An assessment procedure and a set of criteria for use in the evaluation of computerized models and computer-based modeling tools. Final Technical Report RADC-TR-80-409, U.S. Air Force.
38. Sargent, R. G. 1982. Verification and validation of simulation Models. Chapter IX in Progress in Modelling and Simulation, ed. F. E. Cellier, 159-169. London: Academic Press.
39. Sargent, R. G. 1984a. Simulation model validation. Chapter 19 in Simulation and Model-Based Methodologies: An Integrative View, ed. T. I. Oren, B. P. Zeigler, and M. S. Elzas, 537-555. Heidelberg, Germany: Springer-Verlag.
40. Sargent, R. G. 1984b. A tutorial on verification and validation of simulation models. In Proc. 1984 Winter Simulation Conf., ed. S. Sheppard, U. W. Pooch, and C. D. Pegden, 114-121. Piscataway, NJ: IEEE.
41. Sargent, R. G. 1985. An expository on verification and validation of simulation models. In Proc. 1985 Winter Simulation Conf., ed. D. T. Gantz, G. C. Blais, and S. L. Solomon, 15-22. Piscataway, NJ: IEEE.
42. Sargent, R. G. 1986. The use of graphical models in model validation. In Proc. 1986 Winter Simulation Conf., ed. J. R. Wilson, J. O. Henriksen, and S. D. Roberts, 237-241. Piscataway, New Jersey: IEEE.
43. Sargent, R. G. 1990. Validation of mathematical models. In Proc. of Geoval-90: Symposium on Validation of Geosphere Flow and Transport Models, 571-579. Stockholm, Sweden.
44. Sargent, R. G. 1994. Verification and validation of simulation models. In Proc. 1994 Winter Simulation Conf., ed. J. D. Tew, M. S. Manivannan, D. A. Sadowski, and A. F. Seila, 77-87. Piscataway, New Jersey: IEEE.
45. Sargent, R. G. 1996a. Some subjective validation methods using graphical displays of data. In Proc. 1996 Winter Simulation Conf., ed. J. M. Charnes, D. J. Morrice, D. T. Brunner, and J. J. Swain, 345-351. Piscataway, New Jersey: IEEE.
46. Sargent, R. G. 1996b. Verifying and validating simulation models. In Proc. 1996 Winter Simulation Conf., ed. J. M. Charnes, D. J. Morrice, D. T. Brunner, and J. J. Swain, 55-64. Piscataway, New Jersey: IEEE.
47. Sargent, R. G. 2000. Verification, validation, and accreditation of simulation models. In Proc. 2000 Winter Simulation Conf., ed. J. A. Joines, R. R. Barton, K. Kang, and P. A. Fishwick, 50-59. Piscataway, New Jersey: IEEE.
48. th St. Petersburg Workshop on Simulation), ed. S. M. Ermakor, Yu. N. Kashtanov, and V. B. Melas, 109-118. Publisher: Chemistry Research Institute of St. Petersburg University.
49. Sargent, R. G. 2001b. Some approaches and paradigms for verifying and validating simulation models. In Proc. 2001 Winter Simulation Conf., ed. B. A. Peters, J. S. Smith, D. J Medeiros, and M. W. Rohrer, 106-114. Piscataway, New Jersey: IEEE.
50. Sargent, R. G. 2003. Verification and validation of simulation models. In Proc. 2003 Winter Simulation Conf., ed. S. Chick, P. J. Sanchez, E. Ferrin, and D. J. Morrice, 37-48. Piscataway, New Jersey: IEEE.
51. Sargent, R. G. 2005. Verification and validation of simulation models. In Proc. 2005 Winter Simulation Conf., ed. M. E. Kuhl, N. M. Steiger, F. B. Armstrong, and J. A. Joines, 130-143. Piscataway, New Jersey: IEEE.
52. Sargent, R. G. 2007. Verification and validation of simulation models. In Proc. 2007 Winter Simulation Conf., ed. S. G. Henderson, B. Biller, M.-H. Hsieh, J. D. Tew, and R. R. Barton, 124-137. Piscataway, New Jersey: IEEE.
53. Sargent, R. G. 2008. A new statistical procedure for validation of simulation models. International Federation of Operational Research Societies Conference, Sandton, South Africa, July 2008.
54. Sargent, R. G., P. A. Glasow, J. P. C. Kleijnen, A. M. Law, I. McGregor, and S. Youngblood. 2000. Strategic directions in verification, validation, and accreditation research. In Proc. 2000 Winter Simulation Conf., ed. J. A. Joines, R. R. Barton, K. Kang, and P. A. Fishwick, 909-916. Piscataway, New Jersey: IEEE.
55. Schlesinger, S. 1979. Terminology for model credibility. Simulation 32 (3): 103-104.
56. Schruben, L.W. 1980. Establishing the credibility of simulation models. Simulation 34 (3): 101-105.
57. U. S. General Accounting Office, PEMD-88-93 1987. DOD simulations: improved assessment procedures would increase the credibility of results.
58. Whitner, R. G. and O. Balci. 1989. Guideline for selecting and using simulation model verification techniques. In Proc. 1989 Winter Simulation Conf., ed. E. A. MacNair, K. J. Musselman, and P. Heidelberger, 559-568. Piscataway, New Jersey: IEEE.
59. Wood, D. O. 1986. MIT model analysis program: what we have learned about policy model review. In Proc. 1986 Winter Simulation Conf., ed. J. R. Wilson, J. O. Henriksen, and S. D. Roberts, 248-252. Piscataway, New Jersey: IEEE.
60. Zeigler, B. P. 1976. Theory of modelling and simulation. New York: John Wiley and Sons, Inc.