Verification and validation of simulation models

Proceedings - Winter Simulation Conference

Volumn , Issue , 2010, Pages 166-183

  Sargent, Robert G ^a  

^a Syracuse University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONCEPTUAL MODEL; DATA VALIDITY; MODEL ACCREDITATION; MODEL DEVELOPMENT; MODEL VALIDATION; MODEL VALIDITY; MODEL VERIFICATION; SIMULATION MODEL; VERIFICATION AND VALIDATION;

ACCREDITATION;

COMPUTER SIMULATION;

EID: 79951596982     PISSN: 08917736     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/WSC.2010.5679166     Document Type: Conference Paper

References (61)

1. Cheng, R. C. H. 1994. Selecting input models. In Proceedings of the 1994 Winter Simulation Conference, ed. J. D. Tew, S. Manivannan, D. A. Sadowski, and A. F. Seila, 184-191. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
2. Anderson, H. A. and R. G. Sargent. 1974. An investigation into scheduling for an interactive computer system. IBM Journal of Research and Development 18 (2): 125-137.
3. Balci, O. 1989. How to assess the acceptability and credibility of simulation results. In Proceedings of the 1989 Winter Simulation Conference, ed. E. A. MacNair, K. J. Musselman, and P. Heidelberger, 62-71. Piscataway, New Jersey: IEEE.
4. Balci, O. 2001. A methodology for certification of modeling and simulation applications. ACM Trans. on Modeling and Computer Simulation 11 (4), 352-377.
5. Balci, O. 2003. Validation, verification, and certification of modeling and simulation applications. In Proceedings of the 2003 Winter Simulation Conference, ed. S. Chick, P. J. Sanchez, E. Ferrin, and D. J.Morrice, 150-158. Piscataway, New Jersey: IEEE.
6. Balci, O. 2004. Quality assessment, verification, and validation of modeling and simulation applications. In Proceedings of the 2004 Winter Simulation Conference, ed. R. G. Ingalls, M. D. Rossetti, J. S. Smith, and H. A. Peters, 122-129. Piscataway, New Jersey: IEEE.
7. Balci, O., R. E. Nance, J. D. Arthur, and W. F. Ormsby. 2002. Expanding our horizons in verification, validation, and accreditation research and practice. In Proceedings of the 2001 Winter Simulation Conference, ed. B. A. Peters, J. S. Smith, D. J Medeiros, and M. W. Rohrer, 653-663. Piscataway, New Jersey: IEEE.
8. Balci, O. and R. G. Sargent. 1981. A methodology for cost-risk analysis in the statistical validation of simulation models. Comm. of the ACM 24 (6): 190-197.
9. Balci, O. and R. G. Sargent. 1982a. Validation of multivariate response simulation models by using Hotelling's two-sample T2 test. Simulation 39(6): 185-192.
10. Balci, O. and R. G. Sargent. 1982b. Some examples of simulation model validation using hypothesis testing. In Proceedings of the 1982 Winter Simulation Conference, ed. H. J. Highland, Y. W. Chao, and O. S. Madrigal, 620-629. Piscataway, New Jersey: IEEE.
11. Balci, O. and R. G. Sargent. 1983. Validation of multivariate response trace-driven simulation models. In Performance 83, ed. Agrawada and Tripathi, 309-323. North Holland.
12. Balci, O. and R. G. Sargent. 1984a. A bibliography on the credibility assessment and validation of simulation and mathematical models. Simuletter 15 (3): 15-27.
13. Balci, O. and R. G. Sargent. 1984b. Validation of simulation models via simultaneous confidence intervals. American Journal of Mathematical and Management Science 4 (3): 375-406.
14. Banks, J., J. S. Carson II, B. L. Nelson, and D. Nicol. 2005. Discrete-event system simulation. 4th ed. Englewood Cliffs, NJ: Prentice-Hall.
15. Banks, J., D. Gerstein, and S. P. Searles. 1988. Modeling processes, validation, and verification of complex simulations: a survey. In Methodology and Validation, Simulation Series, Vol. 19, No. 1, The Society for Computer Simulation, 13-18. San Diego, CA: Society for Modeling and Simulation International.
16. Carson II, J. S. 2002. Model verification and validation. In Proceedings of the 2002 Winter Simulation Conference, ed. E. Yucesan, C.-H. Chen, J.L. Snowdon, J. M. Charnes, 52-58. Piscataway, New Jersey: IEEE.
17. DoDI 5000.61. 2003. DoD modeling and simulation verification, validation, and accreditation, May 13, 2003.
18. Fairley, R. E. 1976. Dynamic testing of simulation software. In Proceedings of the 1976 Summer Computer Simulation Conference, 40-46. San Diego, CA: Society for Modeling and Simulation International.
19. Gass, S. I. 1983. Decision-aiding models: validation, assessment, and related issues for policy analysis. Operations Research 31 (4): 601-663.
20. Gass, S. I. 1984. Documenting a computer-based model. Interfaces 14 (3): 84-93.
21. Gass, S. I. 1993. Model accreditation: a rationale and process for determining a numerical rating. European Journal of Operational Research 66 (2): 250-258.
22. Gass, S. I. and L. Joel. 1987. Concepts of model confidence. Computers and Operations Research 8 (4): 341-346.
23. Gass, S. I. and B. W. Thompson. 1980. Guidelines for model evaluation: an abridged version of the U. S. general accounting office exposure draft. Operations Research 28 (2): 431-479.
24. Johnson, R. A. 2005. Miller and Freund's probability and statistics for engineers, 7th ed. Englewood Cliffs, NJ: Prentice-Hall.
25. Kleijnen, J. P. C. 1987. Statistical tools for simulation practitioners. New York: Marcel Dekker.
26. Kleijnen, J. P. C. 1999. Validation of models: statistical techniques and data availability. In Proceedings of the 1999 Simulation Conference, ed. P. A. Farrington, H. Black Nembhard, D. T. Sturrock, and G. W. Evans, 647-654. Piscataway, New Jersey: IEEE.
27. Kleindorfer, G. B. and R. Ganeshan. 1993. The philosophy of science and validation in simulation. In Proceedings of the 1993 Winter Simulation Conference, ed. G. W. Evans, M. Mollaghasemi, E. C. Russell, and W. E. Biles, 50-57. Piscataway, New Jersey: IEEE.
28. Knepell, P. L. and D. C. Arangno. 1993. Simulation validation: a confidence assessment methodology. IEEE Computer So-ciety Press.
29. Landry, M, J. L. Malouin, and M. Oral. 1983. Model validation in operations research. European Journal of Operational Re-search 14: 207-220.
30. Law, A. M. 2007. Simulation modeling and analysis. 4th ed. New York: McGraw-Hill.
31. Law, A. M. 2008. How to build valid and credible simulation models. In Proceedings of the 2008 Winter Simulation Conference, ed. S. J. Mason, R. Hill, L. Moench, and O. Rose, 39-47. Piscataway, New Jersey: IEEE.
32. Lowery, J. 1996. Design of hospital admissions scheduling system using simulation. In Proceedings of the 1996 Winter Simulation Conference, ed. J. M. Charnes, D. J. Morrice, D. T. Brunner, and J. J. Swain, 1199-1204. Pisctaway, New Jersey: IEEE.
33. Naylor, T. H. and J. M. Finger. 1967. Verification of computer simulation models. Management Science 14 (2): B92-B101.
34. Rao, M. J. and R. G. Sargent. 1988. An advisory system for operational validity. In Artificial Intelligence and Simulation: the Diversity of Applications, ed. T. Hensen, 245-250. San Diego, CA: Society for Computer Simulation.
35. Robinson, S. 2004. Simulation: The Practice of Model Development and Use. Chichester, West Sussex, England: John Wiley.
36. Rueß, H. and L. de Moura. 2003. From simulation to verification (and back). In Proceedings of the 2003 Winter Simu-lation Conference, ed. S. Chick, P. J. Sanchez, E. Ferrin, and D. J. Morrice, 888-896. Piscataway, New Jesey: IEEE.
37. Sargent, R. G. 1979. Validation of simulation models. In Proceedings of the 1979 Winter Simulation Conference, ed. H. J. Highland, M. F. Spiegel, and R. E. Shannon, 497-503. Piscataway, New Jersey: IEEE.
38. Sargent, R. G. 1981. An assessment procedure and a set of criteria for use in the evaluation of computerized models and computer-based modeling tools. Final Technical Report RADC-TR-80-409, U.S. Air Force.
39. Sargent, R. G. 1982. Verification and validation of simulation Models. Chapter IX in Progress in Modelling and Simulation, ed. F. E. Cellier, 159-169. London: Academic Press.
40. Sargent, R. G. 1984a. Simulation model validation. Chapter 19 in Simulation and Model-Based Methodologies: An Integrative View, ed. T. I. Oren, B. P. Zeigler, and M. S. Elzas, 537-555. Heidelberg, Germany: Springer-Verlag.
41. Sargent, R. G. 1984b. A tutorial on verification and validation of simulation models. In Proceedings of the 1984 Win-ter Simulation Conference, ed. S. Sheppard, U. W. Pooch, and C. D. Pegden, 114-121. Piscataway, NJ: IEEE.
42. Sargent, R. G. 1985. An expository on verification and validation of simulation models. In Proceedings of the 1985 Winter Simulation Conference, ed. D. T. Gantz, G. C. Blais, and S. L. Solomon, 15-22. Piscataway, NJ: IEEE.
43. Sargent, R. G. 1986. The use of graphical models in model validation. In Proceedings of the 1986 Winter Simulation Conference, ed. J. R. Wil-son, J. O. Henriksen, and S. D. Roberts, 237-241. Piscataway, New Jersey: IEEE.
44. Sargent, R. G. 1990. Validation of mathematical models. In Proc. of Geoval-90: Symposium on Validation of Geosphere Flow and Transport Models, 571-579. Stockholm, Sweden.
45. Sargent, R. G. 1994. Verification and validation of simulation models. In Proc. 1994 Winter Simulation Conference, ed. J. D. Tew, M. S. Manivannan, D. A. Sadowski, and A. F. Seila, 77-87. Piscataway, New Jersey: IEEE.
46. Sargent, R. G. 1996a. Some subjective validation methods using graphical displays of data. In Proceedings of the 1996 Winter Simulation Conference, ed. J. M. Charnes, D. J. Morrice, D. T. Brunner, and J. J. Swain, 345-351. Piscataway, New Jersey: IEEE.
47. Sargent, R. G. 1996b. Verifying and validating simulation models. In Proceedings of the 1996 Winter Simulation Conference, ed. J. M. Charnes, D. J. Morrice, D. T. Brunner, and J. J. Swain, 55-64. Piscataway, New Jer-sey: IEEE.
48. Sargent, R. G. 2000. Verification, validation, and accreditation of simulation models. In Proceedings of the 2000 Winter Simulation Conference, ed. J. A. Joines, R. R. Barton, K. Kang, and P. A. Fishwick, 50-59. Piscataway, New Jersey: IEEE.
49. Sargent, R. G. 2001a. Graphical displays of simulation model data as statistical references. In Simulation 2001 (Proc. of the 4th St. Petersburg Workshop on Simulation), ed. S. M. Ermakor, Yu. N. Kashtanov, and V. B. Melas, 109-118. Publisher: Chemistry Research Institute of St. Petersburg University.
50. Sargent, R. G. 2001b. Some approaches and paradigms for verifying and validating simulation models. In Proceedings of the 2001 Winter Simulation Conference, ed. B. A. Peters, J. S. Smith, D. J Medeiros, and M. W. Rohrer, 106-114. Piscataway, New Jersey: IEEE.
51. Sargent, R. G. 2003. Verification and validation of simulation models. In Proc. 2003 Winter Simulation Conference, ed. S. Chick, P. J. Sanchez, E. Ferrin, and D. J. Morrice, 37-48. Piscataway, New Jersey: IEEE.
52. Sargent, R. G. 2005. Verification and validation of simulation models. In Proc. 2005 Winter Simulation Conference, ed. M. E. Kuhl, N. M. Steiger, F. B. Armstrong, and J. A. Joines, 130-143. Piscataway, New Jersey: IEEE.
53. Sargent, R. G. 2007. Verification and validation of simulation models. In Proceedings of the 2007 Winter Simulation Conference, ed. S. G. Henderson, B. Biller, M.-H. Hsieh, J. D. Tew, and R. R. Barton, 124-137. Piscata-way, New Jersey: IEEE.
54. Sargent, R. G. 2008. A new statistical procedure for validation of simulation models. International Federation of Operational Research Societies Conference, Sandton, South Africa, July 2008.
55. Sargent, R. G. 2009. Verification and validation of simulation models. In Proceedings of the 2009 Winter Simulation Conference, ed. M. D. Rossetti, R. R. Hill, B. Johansson, A. Dunkin, and R. G. Ingalls, 162-176. Piscata-way, New Jersey: IEEE.
56. Sargent, R. G., P. A. Glasow, J. P. C. Kleijnen, A. M. Law, I. McGregor, and S. Youngblood. 2000. Strategic directions in verification, validation, and accreditation research. In Proceedings of the 2000 Winter Simulation Conference, ed. J. A. Joines, R. R. Barton, K. Kang, and P. A. Fishwick, 909-916. Piscataway, New Jersey: IEEE.
57. Schlesinger, S. 1979 el al. Terminology for model credibility. Simulation 32 (3): 103-104.
58. Schruben, L.W. 1980. Establishing the credibility of simulation models. Simulation 34 (3): 101-105. U. S. General Accounting Office, PEMD-88-3. 1987. DOD simulations: improved assessment procedures would increase the credibility of results.
59. Whitner, R. G. and O. Balci. 1989.Guideline for selecting and using simulation model verification techniques. In Proceedings of the 1989 Winter Simulation Conference, ed. E. A. MacNair, K. J. Musselman, and P. Heidelberger, 559-568. Piscataway, New Jersey: IEEE.
60. Wood, D. O. 1986. MIT model analysis program: what we have learned about policy model review. In Proceedings of the 1986 Winter Simulation Conference, ed. J. R. Wilson, J. O. Henriksen, and S. D. Roberts, 248-252. Piscataway, New Jersey: IEEE.
61. Zeigler, B. P. 1976. Theory of modelling and simulation. New York: John Wiley and Sons, Inc.