Statistical analysis of simulation output: State of the art

Proceedings - Winter Simulation Conference

Volumn , Issue , 2007, Pages 150-161

  Alexopoulos, Christos ^a  

^a Georgia Institute of Technology   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL METHODS; COMPUTER SIMULATION; ESTIMATION; PARAMETER ESTIMATION; TIME SERIES ANALYSIS;

BATCH MEANS; COMPUTER SIMULATIONS; ESTIMATION TECHNIQUES; OUTPUT DATA; STATE OF THE ARTS; STATISTICAL ANALYSIS; STEADY-STATE SYSTEMS; TIME-SERIES;

STATISTICAL METHODS;

EID: 49749131186     PISSN: 08917736     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/WSC.2007.4419597     Document Type: Conference Paper

References (69)

1. Aktaran-Kalayci T., C. Alexopoulos, N. T. Argon, D. Goldsman, and J. R. Wilson. 2006. Exact expected values of variance vstimators in steady-state simulation. Naval Research Logistics Quarterly 54(4):397-410.
2. Alexopoulos, C. 2006. Statistical estimation in computer simulation. In Handbooks in Operations Research and Management Science, Vol. 13, eds. S. C. Henderson, and B. L. Nelson, 193-223. Amsterdam, Netherlands: Elsevier B. V.
3. Alexopoulos, C., S. Andradóttir, N. T. Argon, and D. Goldsman. 2006a. Replicated batch means variance estimators in the presence of an initial transient. ACM Transactions on Modeling and Computer Simulation 16(4):317-328.
4. Alexopoulos, C., N. T. Argon, D. Goldsman, N. M. Steiger, G. Tokol, and J. R. Wilson. 2007a. Efficient computation of overlapping variance estimators for simulation. INFORMS Journal on Computing 19(3):314-327.
5. Alexopoulos, C., N. T. Argon, D. Goldsman, G. Tokol, and J. R. Wilson. 2007b. Overlapping variance estimators for simulation. Operations Research to appear. Available online via [accessed June 17, 2007].
6. Alexopoulos, C., and D. Goldsman. 2004. To batch or not to batch? ACM Transactions on Modeling and Computer Simulation 14:76-114.
7. Alexopoulos, C., D. Goldsman, and R. F. Serfozo. 2006b. Stationary processes: Statistical estimation. In Encyclopedia of Statistical Sciences, 2d Ed., eds. N. Balakrishnan, C. Read, and B. Vidakovic 7991-8006. Hoboken, New Jersey: John Wiley & Sons.
8. Alexopoulos, C., and A. F. Seila. 1998. Output data analysis. In Handbook of Simulation, ed. J. Banks, Chapter 7. New York: John Wiley & Sons.
9. Anderson, T. W. 1984. An introduction to multivariate statistical analysis. New York: John Wiley & Sons.
10. Antonini, C., C. Alexopoulos, D. Goldsman, and J. R. Wilson. 2007. Area variance estimators for simulation using folded standardized time series. Technical report, H. Milton Stewart School of Industrial and systems Engineering, Georgia Institute of Technology, Atlanta, GA. Available online via [accessed July 9, 2007].
11. Argon, N. T., and S. Andradóttir. 2006. Replicated batch means for steady-state simulations. Naval Research Logistics 53(6):508-524.
12. Billingsley, P. 1968. Convergence of probability measures. New York: John Wiley & Sons.
13. Calvin, J. M. 2007. Simulation output analysis using integrated paths. ACM Transactions on Modeling and Computer Simulation 17(3) to appear.
14. Calvin, J. M., and M. K. Nakayama. 2006. Permuted sandardized time series estimators for steady-state simulation. Mathematics of Operations Research 31(2):351-368.
15. Chang, B.-Y. 2004. Estimation techniques for nonlinear functions of the steady-state mean in computer simulation. Ph.D. Thesis, School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia.
16. Charnes, J. M. 1990. Power comparisons for the multivariate batch-means method. In Proceedings of the 1990 Winter Simulation Conference, 281-287. Piscataway, New Jersey: IEEE.
17. Charnes, J. M. 1991. Multivariate simulation output analysis. In Proceedings of the 1991 Winter Simulation Conference, 187-193. Piscataway, New Jersey: IEEE.
18. Chen, E. J., and W. D. Kelton. 2006. Quantile and toleranceinterval estimation in simulation. European Journal of Operational Research 168(2)2, 520-540.
19. Chen, R. D., and A. F. Seila. 1987. Multivariate inference in stationary simulation using batch means. In Proceedings of the 1987 Winter Simulation Conference, 302-304. Piscataway, New Jersey: IEEE.
20. Chien, C.-H. 1989. Small sample theory for steady state confidence intervals. Technical Report No. 37, Department of Operations Research, Stanford University, Palo Alto, California.
21. Chien, C.-H., D. Goldsman, and B. Melamed. 1997. Largesample results for batch means. Management Science 43:1288-1295.
22. Chow, Y. S., and H. Robbins. 1965. On the asymptotic theory of fixed-width sequential confidence intervals for the mean. Annals of Mathematical Statistics 36:457-462.
23. Conway, R. W. 1963. Some tactical problems in digital simulation. Management Science 10:47-61.
24. Crane, M. A., and D. L. Iglehart. 1975. Simulating stable stochastic systems III: Regenerative processes and discrete-event simulations. Operations Research 23:33-45.
25. Damerdji, H. 1994a. Strong consistency of the variance estimator in steady-state simulation output analysis. Mathematics of Operations Research 19:494-512.
26. Damerdji, H. 1994b. On the batch means and area variance estimators. In Proceedings of the 1994 Winter Simulation Conference, 340-344. Piscataway, New Jersey: IEEE.
27. Damerdji, H. 1995. Mean-square consistency of the variance estimator in steady-state simulation output analysis. Operations Research 43:282-291.
28. Durrett, R. 2005. Probability: Theory and examples, 3d Ed. Belmont, California: Brooks/Cole-Thomson Learning.
29. Fishman, G. S. 1973. Statistical analysis for queueing simulations. Management Science 20:363-369.
30. Fishman, G. S. 1974. Estimation of multiserver queueing simulations. Operations Research 22:72-78.
31. Fishman, G. S. 1978. Principles of discrete event simulation. New York: John Wiley & Sons.
32. Fishman, G. S. 2001. Discrete-event simulation. New York: Springer-Verlag.
33. Fishman, G. S., and L. S. Yarberry. 1997. An implementation of the batch means method. INFORMS Journal on Computing 9:296-310.
34. Foley, R. D., and D. Goldsman. 1999. Confidence intervals using orthonormally weighted standardized time series. ACM Transactions on Modeling and Simulation 9:297-325.
35. Glynn, P. W., and D. L. Iglehart. 1985. Large-sample theory for standardized time series: An overview. In Proceedings of the 1985 Winter Simulation Conference, 129-134. Piscataway, New Jersey: IEEE.
36. Glynn, P. W., and D. L. Iglehart. 1990. Simulation analysis using standardized time series. Mathematics of Operations Research 15:1-16.
37. Glynn, P. W., and W. Whitt. 1991. Estimating the asymptotic variance with batch means. Operations Research Letters 10:431-435.
38. Goldsman, D., K. Kang, and A. F. Seila. 1999. Cramér-von Mises variance estimators for simulations. Operations Research 47, 299-309.
39. Goldsman, D., M. Meketon, and L. W. Schruben. 1990. Properties of standardized time series weighted area variance estimators. Management Science 36:602-612.
40. Goldsman, D., and L. W. Schruben. 1990. New confidence interval estimators using standardized time series. Management Science 36:393-397.
41. Hald, A. 1952. Statistical theory with engineering applications. New York: John Wiley & Sons.
42. Heidelberger, P., and P. A. W. Lewis. 1984. Quantile estimation in dependent sequences. Operations Research 32:185-209.
43. Heidelberger, P., and P. D. Welch. 1981. A spectral method for confidence interval generation and run length control in simulations. Communications of the ACM 24:233-245.
44. Heidelberger, P., and P. D. Welch. 1983. Simulation run length control in the presence of an initial transient. Operations Research 31(6):1109-1144.
45. Henderson, S. C., and B. L. Nelson, eds. 2006. Handbooks in operations research and management science, Vol. 13. Amsterdam, Netherlands: Elsevier B. V.
46. Iglehart, D. L. 1975. Simulating stable stochastic systems, V: Comparison of ratio estimators. Naval Research Logistics Quarterly 22:553-565.
47. Iglehart, D. L. 1976. Simulating stable stochastic systems, VI: Quantile estimation. Journal of the ACM 23:347-360.
48. Lada, E. K., N. M. Steiger, and J. R. Wilson. 2006. Performance evaluation of recent procedures for steadystate simulation analysis. HE Transactions 38:711-727.
49. Lada, E. K., and J. R. Wilson. 2006. A wavelet-based spectral procedure for steady-state simulation analysis. European Journal of Operations Research 174:1769-1801.
50. Lada, E. K., J. R. Wilson, N. M. Steiger, and J. A. Joines. 2007. Performance of a wavelet-based spectral procedure for steady-satte simulation analysis. INFORMS Journal on Computing 19(2): 150-160.
51. Law, A. M. 2006. Simulation modeling and analysis, 4th Ed. New York: McGraw-Hill.
52. Law, A. M., and J. S. Carson. 1979. A sequential procedure for determining the length of a steady-state simulation. Operations Research 27:1011-1025.
53. Meketon, M. S., and B. W. Schmeiser. 1984. Overlapping batch means: Something for nothing? In Proceedings of the 1984 Winter Simulation Conference, 227-230. Piscataway, New Jersey: IEEE.
54. Mufioz, D. F., and P. W. Glynn. 1997. A batch means methodology for estimation of a nonlinear function of a steady-state mean. Management Science 43(8): 1121-1135.
55. Philipp, W., and W. Stout. 1975. Almost sure invariance principles for partial sums of weakly dependent random variables. Memoirs of the American Mathematical Society 2.
56. Sargent, R. G., K. Kang, and D. Goldsman. 1992. An investigation of finite-sample behavior of confidence interval estimators. Operations Research 40:898-913.
57. Satterthwaite, F. E. 1941. Synthesis of variance. Psychometrika 6:309-316.
58. Schmeiser, B.W. 1982. Batch size effects in the analysis of simulation output. Operations Research 30:556-568.
59. Schruben, L. W. 1983. Confidence interval estimation using standardized time series. Operations Research 31:1090-1108.
60. Seila, A. F. 1982a. A batching approach to quantile estimation in regenerative simulations. Management Science 28:573-581.
61. Seila, A. F. 1982b. Percentile estimation in discrete event simulation. Simulation 39:193-200.
62. Song, W.-M. T. 1996. On the estimation of optimal batch sizes in the analysis of simulation output analysis. European Journal of Operations Research 88:304-309.
63. Song, W.-M. T., and B. W. Schmeiser. 1995. Optimal mean-squared-error batch sizes. Management Science 41:110-123.
64. Steiger, N. M., E. K. Lada, J. R. Wilson, J. A. Joines, C. Alexopoulos, and D. Goldsman. 2005, ASAP3: An batch means procedure for steady-state simulation analysis. ACM Transactions on Modeling and Computer Simulation 15(1):39-73.
65. von Neumann, J. 1941. Distribution of the ratio of the mean square successive difference and the variance. Annals of Mathematical Statistics 12:367-395.
66. Welch, P. D. 1983. The statistical analysis of simulation results. In The Computer Performance Modeling Handbook, ed. S. Lavenberg, 268-328. New York: Academic Press.
67. Welch, P. D. 1987. On the relationship between batch means, overlapping batch means and spectral estimation, In Proceedings of the 1987 Winter Simulation Conference, 320-323. Piscataway, New Jersey: IEEE.
68. Wilson, J. R., and A. A. B. Pritsker. 1978a. A survey of research on the simulation startup problem. Simulation 31:55-58.
69. Wilson, J. R., and A. A. B. Pritsker. 1978b. Evaluation of startup policies in simulation experiments. Simulation 31:79-89.