Calibration, validation, and prediction in random simulation models:Gaussian process metamodels and a bayesian integrated solution

ACM Transactions on Modeling and Computer Simulation

Volumn 25, Issue 3, 2015, Pages

  Yuan, Jun ^a   Ng, Szu Hui ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Computer model calibration; Gaussian process; Sequential experimental design; Stochastic computer simulation

Indexed keywords

BAYESIAN NETWORKS; FORECASTING; GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); INTEGRATED CONTROL; MONTE CARLO METHODS; STATISTICS; STOCHASTIC SYSTEMS;

COMPUTER MODELING; ENTROPY-BASED CRITERION; GAUSSIAN PROCESSES; MATHEMATICS OF COMPUTING; MODEL CALIBRATION AND VALIDATION; PROBABILITY AND STATISTICS; SEQUENTIAL EXPERIMENTAL DESIGN; SIMULATION OUTPUT ANALYSIS;

STOCHASTIC MODELS;

EID: 84928521567     PISSN: 10493301     EISSN: 15581195     Source Type: Journal    
DOI: 10.1145/2699713     Document Type: Article

References (30)

1. B. Ankenman, B. L. Nelson, and J. Staum. 2010. Stochastic kriging for simulation metamodeling. Operations Research 58, 371-382.
2. P. D. Arendt, D. W. Apley, and W. Chen. 2012. Quantification of model uncertainty: Calibration, model discrepancy, and identifiability. Journal of Mechanical Design, 134.
3. L. S. Bastos and A. O'Hagan. 2009. Diagnostics for Gaussian process emulators. Technometrics 51, 425-438.
4. X. Chen, B. Ankenman, and B. L. Nelson. 2012. The effects of common random numbers on stochastic kriging metamodels. ACM Transactions on Modeling and Computer Simulation 22, 7/1-7/20.
5. N. A. C. Cressie. 1993. Statistics for Spatial Data. John Wiley, New York.
6. P. Hall, L. Peng, and N. Tajvidi. 1999. On prediction intervals based on predictive likelihood or bootstrap methods. Biometrika 86, 871-880.
7. G. Han, T. J. Santner, and J. J. Rawlinson. 2009. Simultaneous determination of tuning and calibration parameters for computer experiments. Technometrics 51, 464-474.
8. D. A. Henderson, R. J. Boys, K. J. Krishnan, C. Lawless, and D. J. Wilkinson. 2009. Bayesian emulation and calibration of a stochastic computer model of mitochondrial DNA deletions in substantia nigra neurons. Journal of the American Statistical Association 104, 76-87.
9. A. S. Hernandez, T. W. Lucas, and P. J. Sanchez. 2012. Selecting random Latin hypercube dimensions and designs through estimation of maximum absolute pairwise correlation. In Proceedings of the 2012Winter Simulation Conference. 1-12.
10. M. C. Kennedy and A. O'Hagan. 2001. Bayesian calibration of computer models. Journal of the Royal Statistical Society Series B-Statistical Methodology 63, 425-450.
11. J. P. C. Kleijnen and E. Mehdad. 2014. Multivariate versus univariate Kriging metamodels formulti-response simulation models. European Journal of Operational Research 236, 573-582.
12. A. Kumar. 2008. Sequential Calibration of Computer Models. Ph.D. dissertation, Ohio State University.
13. th ed.). McGraw-Hill, New York.
14. D. V. Lindley. 1956. On a measure of information provided by an experiment. Annals of Mathematical Statistics 27, 986-1006.
15. J. Loeppky, D. Bingham, and W. Welch. 2006. Computer Model Calibration or Tuning in Practice. Retrieved from http://www.stat.ubc.ca/Research/TechReports/techreports/221.pdf.
16. J. Loeppky, J. Sacks, and W. Welch. 2009. Choosing the sample size of a computer experiment: A practical guide. Technometrics 51, 366-376.
17. S. Luna and Young A. 2002. The bootstrap and kriging prediction intervals. Scandinavian Journal of Statistics 29, 1-18.
18. S. H. Ng and J. Yin. 2012. Bayesian kriging analysis and design for stochastic simulations. ACMTransactions on Modeling and Computer Simulation 22, 17/1-17/26.
19. J. Oakley. 2002. Eliciting Gaussian process priors for complex computer codes. Journal of the Royal Statistical Society Series D-the Statistician 51, 81-97.
20. W. L. Oberkampf and C. J. Roy. 2010. Verification and Validation in Scientific Computing. Cambridge University Press, Cambridge.
21. P. Z. G. Qian and C. F. J. Wu. 2008. Bayesian hierarchical modeling for integrating low-accuracy and highaccuracy experiments. Technometrics 50, 192-204.
22. T. J. Santner, B. J. Williams, and W. Notz. 2003. The Design and Analysis of Computer Experiments. New York: Springer.
23. R. G. Sargent. 2007. Verification and validation of simulation models. In Proceedings of the 2007 Winter Simulation Conference. 124-137.
24. M. C. Shewry and H. P. Wynn. 1987. Maximum entropy sampling. Journal of Applied Statistics 14, 165-170.
25. K. S. Song. 2002. Goodness-of-fit tests based on Kullback-Leibler discrimination information. IEEE Transactions on Information Theory 48, 1103-1117.
26. A. Vasicek. 1976. A test for normality based on sample entropy. Journal of the Royal Statistical Society Series B-Statistical Methodology 38, 54-59.
27. S. C. Wang, W. Chen, and K. L. Tsui. 2009. Bayesian validation of computer models. Technometrics 51, 439-451.
28. B. J. Williams, J. L. Loeppky, L. M. Moore, and M. S. Macklem. 2011. Batch sequential design to achieve predictive maturity with calibrated computermodels. Reliability Engineering&System Safety 96, 1208-1219.
29. J. Yuan and S. H. Ng. 2013a. A sequential approach for stochastic computermodel calibration and prediction. Reliability Engineering & Systems Safety 111, 273-286.
30. J. Yuan and S. H. Ng. 2013b. An entropy based sequential calibration approach for stochastic computer models. In Proceedings of the 2013 Winter Simulation Conference. 589-600.