Review of advanced methods for simulation output analysis

Proceedings - Winter Simulation Conference

Volumn 2005, Issue , 2005, Pages 188-201

  Alexopoulos, Christos ^a   Kim, Seong Hee ^a  

^a Georgia Institute of Technology   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

PARAMETER ESTIMATION; STATISTICAL METHODS; TIME SERIES ANALYSIS;

REGENERATIVE METHODS; SIMULATION OUTPUT; STATISTICAL PROCEDURES;

COMPUTER SIMULATION;

EID: 33846654834     PISSN: 08917736     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/WSC.2005.1574250     Document Type: Conference Paper

References (77)

1. Alexopoulos, C., N. T. Argon, D. Goldsman, G. Tokol, and J. R. Wilson. 2005a. Overlapping variance estimators for simulation: I. Theory. Technical Report, School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia.
2. Alexopoulos, C., N. T Argon, D. Goldsman, N. M. Steiger, G. Tokol, and J. R. Wilson. 2005b. Overlapping variance estimators for simulation: II. Implementation and evaluation. Technical Report, School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia.
3. Alexopoulos, C., and D. Goldsman. 2004. To batch or not to batch? ACM Transactions on Modeling and Computer Simulation 14:76-114.
4. Alexopoulos, C., and A. F. Seila. 1998. Output data analysis. In Handbook of Simulation, ed. J. Banks, Chapter 7. New York: John Wiley & Sons.
5. Anderson, T. W. 1984. An Introduction to multivariate statistical analysis. New York: John Wiley & Sons.
6. Bechhofer, R. E., T. J. Santner, and D. Goldsman. 1995. Design and analysis of experiments for statistical selection, screening and multiple comparisons. New York: John Wiley & Sons.
7. Billingsley, P. 1968. Convergence of probability measures. New York: John Wiley & Sons.
8. Charnes, J. M. 1989. Statistical analysis of multivariate discrete-event simulation output. Ph.D. Thesis, Department of Operations and Management Science, University of Minnesota, Minneapolis, Minnesota.
9. Charnes, J. M. 1990. Power comparisons for the multivariate batch-means method. In Proceedings of the 1990 Winter Simulation Conference, 281-287. Piscataway, New Jersey: IEEE.
10. Charnes, J. M. 1991. Multivariate simulation output analysis. In Proceedings of the 1991 Winter Simulation Conference, 187-193. Piscataway, New Jersey: IEEE.
11. Chen, H.-C., C.-H. Chen, L. Dai, and E. Yücesan. 1997. New development of optimal computing budget allocation for discrete event simulation, In Proceedings of the 1997 Winter Simulation Conference, 334-341. Piscataway, New Jersey: IEEE.
12. Chen, R. D., and A. F. Seila. 1987. Multivariate inference in stationary simulation using batch means. In Proceedings of the 1987 Winter Simulation Conference, 302-304. Piscataway, New Jersey: IEEE.
13. Chick, S. E. 1997. Selecting the best system: A decision-theoretic approach. In Proceedings of the 1997 Winter Simulation Conference, 326-333. Piscataway, New Jersey: IEEE.
14. Chick, S. E., and K. Inoue. 200la. New two-stage and sequential procedures for selecting the best simulated system. Operations Research 49:732-743.
15. Chick, S. E., and K. Inoue. 2001b. New procedures for identifying the best simulated system using common random numbers. Management Science 47:1133-1149.
16. Chien, C.-H. 1989. Small sample theory for steady state confidence intervals. Technical Report No. 37, Department of Operations Research, Stanford University, Palo Alto, California.
17. Chien, C.-H., D. Goldsman, and B. Melamed. 1997. Large-sample results for batch means. Management Science 43:1288-1295.
18. Chow, Y. S., and H. Robbins. 1965. On the asymptotic theory of fixed-width sequential confidence intervals for the mean. Annals of Mathematical Statistics 36:457-462.
19. Conway, R. W. 1963. Some tactical problems in digital simulation. Management Science 10:47-61.
20. Crane, M. A., and D. L. Iglehart. 1975. Simulating stable stochastic systems III: Regenerative processes and discrete-event simulations. Operations Research 23:33-45.
21. Damerdji, H. 1994a. Strong consistency of the variance estimator in steady-state simulation output analysis. Mathematics of Operations Research 19:494-512.
22. Damerdji, H. 1994b. On the batch means and area variance estimators. In Proceedings of the 1994 Winter Simulation Conference, 340-344. Piscataway, New Jersey: IEEE.
23. Damerdji, H. and M. K. Nakayama. 1999. Two-stage multiple-comparison procedures for steady-state simulation. ACM TOMACS 9:1-30.
24. Durrett, R. 2005. Probability: Theory and examples, 3d Ed. Belmont, California: Brooks/Cole-Thomson Learning.
25. Fishman, G. S. 1973. Statistical analysis for queueing simulations. Management Science 20:363-369.
26. Fishman, G. S. 1974. Estimation of multiserver queueing simulations. Operations Research 22:72-78.
27. Fishman, G. S. 1978. Principles of discrete event simulation. New York: John Wiley & Sons.
28. Fishman, G. S. 2001. Discrete-event simulation. New York: Springer-Verlag.
29. Fishman, G. S., and L. S. Yarberry. 1997. An implementation of the batch means method. INFORMS Journal on Computing 9:296-310.
30. Foley, R. D., and D. Goldsman. 2000. Confidence intervals using orthonormally weighted standardized time series. ACM Transactions on Modeling and Simulation 9:297-325.
31. Glynn, P. W., and D. L. Iglehart. 1985. Large-sample theory for standardized time series: An overview. In Proceedings of the 1985 Winter Simulation Conference, 129-134. Piscataway, New Jersey: IEEE.
32. Glynn, P. W., and D. L. Iglehart. 1990. Simulation analysis using standardized time series. Mathematics of Operations Research 15:1-16.
33. Glynn, P. W., and W. Whitt. 1991. Estimating the asymptotic variance with batch means. Operations Research Letters 10:431-435.
34. Goldsman, D., K. Kang, and A. F. Seila. 1999. Cramér-von Mises variance estimators for simulations. Operations Research 47, 299-309.
35. Goldsman, D., S.-H. Kim, W. S. Marshall, and B. L. Nelson. 2000. Ranking and selection for steady-state simulation. In Proceedings of the 2000 Winter Simulation Conference, 544-553, Piscataway, New Jersey: IEEE. Goldsman, D., S.-H. Kim, W. Marshall, and B. L. Nelson.
36. Ranking and selection procedures for steady-state simulation: Perspectives and procedures. INFORMS Journals on Computing 14:2-19.
37. Goldsman, D., M. Meketon, and L. W. Schruben. 1990. Properties of standardized time series weighted area variance estimators. Management Science 36:602-612.
38. Goldsman, D., and B. L. Nelson. 1998. Comparing systems via simulation. In Handbook of Simulation, ed. J. Banks, Chapter 8. New York: John Wiley & Sons.
39. Goldsman, D., and L. W. Schruben. 1990. New confidence interval estimators using standardized time series. Management Science 36:393-397.
40. Gupta, S. S. 1965. On some multiple decision (selection and ranking) rules. Technometrics 7:225-245.
41. Gupta, S. S., and D.-Y. Huang. 1976. Subset selection procedures for the means and variances of normal populations: Unequal sample sizes case, Sankhyā B, 38:112-128.
42. Heidelberger, P., and P. A. W. Lewis. 1984. Quantile estimation in dependent sequences. Operations Research 32:185-209.
43. Hong, L. J., and B. L. Nelson. 2005. The tradeoff between sampling and switching: New sequential procedures for indifference-zone selection. IIE Transactions 37:623-634.
44. Iglehart, D. L. 1975. Simulating stable stochastic systems, V: Comparison of ratio estimators. Naval Research Logistics Quarterly 22:553-565.
45. Iglehart, D. L. 1976. Simulating stable stochastic systems, VI: Quantile estimation. Journal of the ACM 23:347-360.
46. Inoue, K., S. E. Chick, and C.-H. Chen. 1999. An empirical evaluation of several methods to select the best system. ACM TOMACS 9:381-407.
47. Kim, S.-H., and B. L. Nelson. 2001. A fully sequential procedure for indifference-zone selection in simulation. ACM TOMACS 11:251-273.
48. Kim, S.-H. and B. L. Nelson. 2003. Selecting the best system: Theory and methods. In Proceedings of the 2003 Winter Simulation Conference, 101-112. Piscataway, New Jersey: IEEE.
49. Kim, S.-H., and B. L. Nelson. 2005. On the asymptotic validity of fully sequential selection procedures for steady-state simulation. Operations Research, forthcoming.
50. Koenig, L. W., and A. M. Law. 1985. A procedure for selecting a subset of size m containing the l best of k independent normal populations, with applications to simulation. Communications in Statistics - Simulation and Computation B 14:719-734.
51. Law, A. M., and J. S. Carson. 1979. A sequential procedure for determining the length of a steady-state simulation. Operations Research 27:1011-1025.
52. Law, A. M., and W. D. Kelton. 2000. Simulation modeling and analysis, 3d Ed. New York: McGraw-Hill.
53. Meketon, M. S., and B. W. Schmeiser. 1984. Overlapping batch means: Something for nothing? In Proceedings of the 1984 Winter Simulation Conference, 227-230. Piscataway, New Jersey: IEEE.
54. Moore, L. W. 1980. Quantile estimation in regenerative processes. Ph.D. Thesis, Curriculum in Operations Research and Systems Analysis, University of North Carolina, Chapel Hill, North Carolina.
55. Nakayama, M. K. 1997. Multiple-comparison procedures for steady-state simulations. Annals of Statistics 25:2433-2450.
56. Nakayama, M. K. 1995. Selecting the best system in steady-state simulations using batch means. In Proceedings of the 1995 Winter Simulation Conference, 362-366. Piscataway, New Jersey: IEEE.
57. Nelson, B. L., J. Swann, D. Goldsman, and W.-M. T. Song. 2001. Simple procedures for selecting the best system when the number of alternatives is large. Operations Research 49:950-963.
58. Ockerman, D. H. 1995. Initialization bias tests for stationary stochastic processes based upon standardized time series techniques. PhD. Thesis, School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia.
59. Philipp, W., and W. Stout. 1975. Almost sure invariance principles for partial sums of weakly dependent random variables. Memoirs of the American Mathematical Society 2.
60. Rinott, Y. 1978. On two-stage selection procedures and related probability-inequalities. Communications in Statistics - Theory and Methods A 7:799-811.
61. Sargent, R. G., K. Kang, and D. Goldsman. 1992. An investigation of finite-sample behavior of confidence interval estimators. Operations Research 40:898-913.
62. Schmeiser, B.W. 1982. Batch size effects in the analysis of simulation output. Operations Research 30:556-568.
63. Schmeiser, B. W. 1986. Personal communication.
64. Schruben, L. W. 1983. Confidence interval estimation using standardized time series. Operations Research 31:1090-1108.
65. Seila, A. F. 1982a. A batching approach to quantile estimation in regenerative simulations. Management Science 28:573-581.
66. Seila, A. F. 1982b. Percentile estimation in discrete event simulation. Simulation 39:193-200.
67. Song, W.-M. T., and B. W. Schmeiser. 1995. Optimal mean-squared-error batch sizes. Management Science 41:110-123.
68. Song, W.-M. T. 1996. On the estimation of optimal batch sizes in the analysis of simulation output analysis. European Journal of Operations Research 88:304-309.
69. Steiger, N. M., E. K. Lada, J. R. Wilson, J. A. Joines, C. Alexopoulos, and D. Goldsman. 2004. ASAP3: An batch means procedure for steady-state simulation analysis. ACM Transactions on Modeling and Computer Simulation 15(1):39-73
70. von Neumann, J. 1941a. The mean square difference. Annals of Mathematical Statistics 12:153-162.
71. von Neumann, J. 1941b. Distribution of the ratio of the mean square successive difference and the variance. Annals of Mathematical Statistics 12:367-395.
72. Welch, P. D. 1983. The statistical analysis of simulation results. In The Computer Performance Modeling Handbook, ed. S. Lavenberg, 268-328. New York: Academic Press.
73. Welch, P. D. 1987. On the relationship between batch means, overlapping batch means and spectral estimation, In Proceedings of the 1987 Winter Simulation Conference, 320-323. Piscataway, New Jersey: IEEE.
74. Wilcox, R. R. 1984. A table for Rinott's selection procedure. Journal of Quality Technology 16:97-100.
75. Wilson, J. R., and A. A. B. Pritsker. 1978a. A survey of research on the simulation startup problem. Simulation 31:55-58.
76. Wilson, J. R., and A. A. B. Pritsker. 1978b. Evaluation of startup policies in simulation experiments. Simulation 31:79-89.
77. Young, L. C. 1941. On randomness of order sequences. Annals of Mathematical Statistics 12:293-300.