Validating simulation models: A general framework and four applied examples

Computational Economics

Volumn 30, Issue 3, 2007, Pages 265-290

  Marks, Robert Ernest ^a  

^a Australian Graduate School of Management   (Australia)

Author keywords

Agent based computational economics; Antitrust; Daubert criteria; Functional complexity; Schelling; Simulation; Sufficiency; Validation

Indexed keywords

EID: 34748812452     PISSN: 09277099     EISSN: 15729974     Source Type: Journal    
DOI: 10.1007/s10614-007-9101-7     Document Type: Article

References (61)

1. Arthur, W. B. (2006). Out-of-equilibrium economics and agent-based modeling. In Tesfatsion, Judd, K.L. (2006), pp. 1551-1564.
2. Axelrod, R. (2006). Simulation in the social sciences, in Rennard (2006), pp. 90-100.
3. Axtell R., Axelrod R., Epstein J., Cohen M.D. (1996) Aligning simulation models: A case study and results. Computational and Mathematical Organization Theory 1: 123-141
4. Barreteau, O., and others (2003). Our companion modelling approach. Journal of Artificial Societies and Social Simulation, 6(1), http://jasss.soc.surrey.ac.uk/6/2/1.html (accessed 2006/07/30).
5. Bar-Yam, Y. (1997). Dynamics of complex systems. New York: Perseus Press. http://www.necsi.org/publications/dcs.
6. Bar-Yam, Y. (2003). Unifying principles in complex systems. In M.C. Roco & W.S. Bainbridge (Eds.), Converging technologies for improving human performance: Nanotechnology, biotechnology, information technology and cognitive science (pp. 380-409). New York: Springer. http://www.necsi.org/projects/yaneer/ComplexSystems.pdf (accessed 2006/07/30)
7. Black B., Ayala F.J., Saffran-Brinks C. (1994) Science and the law in the wake of Daubert a new search for scientific knowledge. Texas Law Review 72: 715-751
8. Brenner, T., & Werker, C. (2006). A practical guide to inference in simulation models, Max Planck Institute of Economics, Evolutionary Economics Group, Jena, #0602. https://papers.econ.mpg.de/evo/ discussionpapers/2006-02.pdf (accessed 2006/06/23).
9. Bunn D.W., Oliveira F.S. (2003) Evaluating individual market power in electricity markets via agent-based simulation. Annals of Operations Research 121: 57-77
10. Burton R.M. (2003) Computational laboratories for Organizational Science: Questions, validity and docking. Computational & Mathematical Organizational Theory 9: 91-108
11. Cook, C. (2006). Home page: The schelling segregation model demo http://www.econ.iastate.edu/tesfatsi/demos/schelling/schellhp.htm (accessed 2006/07/31).
12. Courtois P.-J. (1985) On time and space decomposition of complex structures. Communications of the ACM 28(6): 590-603
13. Durlauf S. (2005) Complexity and empirical economics. The Economic Journal 115(June): F225-F243
14. Epstein, J. M. (2006). Generative social science: Studies in agent-based Computational modeling. Princeton: P.U.P.
15. Fagiolo, G., Windrum, P., & Moneta, A. (2006). Empirical validation of agent-based models: A critical survey, LEM working paper 2006/14, Pisa, Italy: Laboratory of Economics and Management, Sant-Anna School of Advanced Studies, May. http://www.lem.sssup.it/WPLem/files/2006-14.pdf (accessed 2006/07/30).
16. Forrester J.W. (1961) Industrial dynamics. MIT Press, Camb
17. Frankel, D. M., & Volij, O. (2005). Measuring segregation, mimeo, Iowa State, http://www.econ.iastate.edu/faculty/frankel/ segindex_all20.pdf (accessed 2006/07/31)
18. Friedman, M. (1953). Essays in positive economics. University of Chicago Press.
19. Froeb L., Hosken D., Pappalardo J. (2004) Economic research at the FTC: Information, retrospectives and retailing. Review of Industrial Organization 25: 353-374
20. Gilbert N., Troitzsch K.G. (2005) Simulation for the Social Scientist, (2nd ed.). Open University Press, Buckingham
21. Gutowitz, H. (1990). Cellular automata: Theory and experiment. In Proceedings of a workshop sponsored by The Center for Nonlinear Studies. Los Alamos (pp. 7-14). The Hague: North-Holland.
22. Haefner J.W. (2005) Modeling biological systems: Principles and applications (2nd ed.). Springer, New York
23. Huntington H.G., Weyant J.P., Sweeney J.L. (1982) Modeling for insights, not numbers: The experiences of the Energy Modeling Forum. OMEGA: The International Journal of the Management Sciences 10(5): 449-462
24. Judd, K. L. (2006). Computationally intensive analyses in economics, in Tesfatsion and Judd (2006), pp. 881-893.
25. Kaldor N. (1961) Capital accumulation and economic growth. In: Lutz F.A., Hague D.C. (eds). The theory of capital. Macmillan, London, pp. 177-222
26. Klein E.E, Herskovitz P. (2005) Philosophical foundations of computer simulation validation. Simulation & Gaming 36: 303-329
27. Leombruni R., Richiardi M. (2005) Why are economists sceptical about agent-based simulations?. Physica A 355: 103-109
28. Leombruni, R., Richiardi, M., Saam, N. J., & Sonnessa, M. (2006). A common protocol for agent-based social simulation. Journal of Artificial Societies and Social Simulation, 9(1). http://jasss.soc.surrey.ac.uk/9/1/ 15.html (accessed 2006/06/22).
29. Malerba F., Nelson R., Orsenigo L., Winter S. (1999) History-friendly models of industry evolution: The computer industry. Industrial and Corporate Change 8(1): 3-40
30. Mankin, J. B., O'Neill, R. V., Shugart, H. H., & Rust, B. W. (1977). The importance of validation in ecosystem analysis. In G.S. Innis (Ed.), New directions in the analysis of ecological systems, part 1 simulation council proceedings series (Vol. 5: Pp. 63-71). California: Simulation Councils, La Jolla. Reprinted in H.H. Shugart and R.V. O'Neill, eds. Systems ecology Dowden, Hutchinson and Ross, Stroudsburg, Pennsylvania, 1979, pp. 309-317.
31. Manson S.M. (2002) Validation and verification of multi-agent systems. In: Janssen M.A. (eds). Complexity and ecosystem management. Edward Elgar, Cheltenham
32. Marks R.E. (1992) Breeding optimal strategies: Optimal behaviour for oligopolists. Journal of Evolutionary Economics 2: 17-38
33. Marks, R. E. (2006). Market design using agent-based models, in Tesfatsion and Judd (2006), pp. 1339-1380.
34. Marks, R. E., Midgley, D. F., & Cooper, L. G. (2006). Co-evolving better strategies in oligopolistic price wars, in Rennard (2006), pp. 806-821.
35. Midgley D.F., Marks R.E., Cooper L.G. (1997) Breeding competitive strategies. Management Science 43(3): 257-275
36. Midgley, D. F., Marks, R. E., & Kunchamwar, D. (2007). The building and assurance of agent-based models: An example and challenge to the field. Journal of Business Research, 60, 884-893. http://www.agsm.edu.au/ ~bobm/papers/Midgley-Marks-Kunchamwar.pdf.
37. Miller J.H. (1996) The coevolution of automata in the repeated prisoner's dilemma. Journal of Economic Behavior and Organizations 29: 87-113
38. Miller J.H. (1998) Active nonlinear tests (ANTs) of complex simulations models. Management Science 44(6): 820-830
39. Moss S., Edmonds B. (2005) Sociology and simulation: Statistical and quantitative cross-validation. American Journal of Sociology 110(4): 1095-1131
40. Rennard J.-P., eds. (2006) Handbook on research on nature-inspired computing for economics and management. Idea Group, Hershey, PA
41. Resnick, M. (1994). Turtles, termites, and traffic jams: Explorations in massively parallel microworlds. MIT Press.
42. Rosen R. (1985) Anticipatory systems: Philosophical, mathematical, and methodological foundations. Pergamon, Oxford
43. Rubinstein, A. (1998). Modeling bounded rationality. MIT Press.
44. Schelling T.C. (1971) Dynamic models of segregation. Journal of Mathematical Psychology 1: 143-186
45. Schelling T.C. (1978) Micromotives and macrobehavior. Norton, New York
46. Schelling, T. C. (2006). Some fun, thirty-five years ago, in Tesfatsion & Judd (2006), pp. 1639-1644.
47. Shannon C.E. (1948) A mathematical theory of communication. Bell Systems Technical Journal 27: 379-423 July, 623 August.
48. Sharpe, W. F. (2007). APSIM: An asset price and portfolio choice simulator. In Investors and markets: Portfolio choices, asset prices and investment advice. Princeton University Press. http://www.stanford.edu/ ∼wfsharpe/apsim/apsim.pdf (accessed 2006/07/30).
49. Simon H.A., Ando A. (1961) Aggregation of variables in dynamic systems. Econometrica 29: 111-138
50. Simon H.A., Wallach D. (1999) Cognitive modeling in perspective. Kognitionswissenschaft 8: 1-4
51. Stephens C.A., Graham A.K., Lyneis J.M. (2005) System dynamics modeling in the legal arena: Meeting the challenges of expert witness admissibility. System Dynamics Review 27(2): 95-122
52. Tesfatsion, L. (2006). Conducting experiments with Chris Cook's Schelling Demo, mimeo, Iowa State, http://www.econ.iastate.edu/classes/ econ308/tesfatsion/segex2.VIITrento.pdf (accessed 2007/05/31).
53. Tesfatsion, L., & Judd, K. L. (Eds.) (2006). In K. Arrow & M. D. Intriligator (Eds.), Handbook of computational economics, volume 2: Agent-based computational economics in the series handbooks in economics. Amsterdam: Elsevier Science.
54. Tukey J.W. (1962) The future of data analysis. Annals of Mathematical Statistics 33: 13-14
55. van Damme, E. E. C., & Pinkse, J. (2005). Merger simulation analysis: An academic perspective, TILEC Discussion paper No. 2005-013 http://ssrn.com/abstract=869737 (accessed 2006/07/30).
56. Vriend N. (2000) An illustration of the essential difference between individual and social learning and its consequences for computational analyses. Journal of Economic Dynamics and Control 24: 1-19
57. Watson J.D., Crick F.H.C. (1953) Molecular structure of nucleic acids: A structure of deoxyribose nucleic acid. Nature 4356: 737-738, April 25.
58. Werden G.J., Froeb L.M., Scheffman D.T. (2004) A Daubert discipline for merger simulation. Antitrust Magazine 18(3): 89-95
59. Weyant, J. P., & Hill, J. (1999). Introduction and overview, The costs of the Kyoto protocol: A multi-model evaluation, The Energy Journal, 20 (Special Issue) vii-xliv, May.
60. Wilensky, U. (1998). NetLogo segregation model. Center for connected learning and computer-based modeling, Northwestern University, Evanston, IL. http://ccl.northwestern.edu/netlogo/models/Segregation.
61. Woodbury, J. R. (Ed.). (2004). Whither merger simulation? The Antitrust Source, May, http://www.abanet.org/antitrust/source/05-04/whither.pdf (accessed 2006/07/30).