How you can tell if the simulations in computational criminology are any good

Journal of Experimental Criminology

Volumn 4, Issue 3, 2008, Pages 289-308

  Berk, Richard ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Agent based models; Computational criminology; Model validation; Simulation

Indexed keywords

EID: 51649094498     PISSN: 15733750     EISSN: 15728315     Source Type: Journal    
DOI: 10.1007/s11292-008-9053-5     Document Type: Article

References (66)

1. Austin, J., Naro, W., & Fabelo, T. (2007). Public safety, public spending: Forecasting America's prison population 2007-2011. Washington, DC: The Pew Charitable Trust, Public Safety Performance Project.
2. Auerhahn, K. (2007). Do you know who your probationers are? Using simulation modeling to estimate the composition of California's probation population, 1980-2000. Justice Quarterly, 23(4), 28-47.
3. Bayarri, M. J., Berger, J. O., Cafeo, J. A., Cavendish, J., Lin, J. C., Paulo, R., et al. (2007a). A framework for validation of computer models. Technometrics, 49(2), 138-154.
4. Bayarri, M. J., Walsh, D.,Berger, J. O., Cafeo, J., Garcia-Donato, G., Liu, F., et al. (2007b). Computer model validation with functional output. Annals of Statistics, 35(5), 1874-1906.
5. Berk, R. A., & MacDonald, J. (2007). The dynamics of crime regimes. Working paper, Department of Criminology, University of Pennsylvania.
6. Berk, R. A., Messinger, S. L., Rauma, D., & Berecochea, J. E. (1983). Prisons as self-regulating systems: A comparison of historical patterns in california for male and female offenders. Law and Society Review, 17(4), 547-586.
7. Berk, R. A., Bond, J., Lu, R., Turco, R., & Weiss, R. E. (2000a). Computer simulations as experiments: Using program evaluation tools to assess the validity of interventions in virtual worlds. In L. Bickman (Ed.), Donald Campbell's legacy (Volume II): Contributions to research design. Newbury Park, CA: Sage Publications.
8. Berk, R. A., Schoenberg, F., Fovell, R., Lu, R., & Weiss, R. (2000b). Approximation and inversion of a complex meteorological system via local linear filters. Journal of Applied Meteorology, 40, 446-458.
9. Berk, R. A., Fovell, R. G., Schoenberg, F., & Weiss, R. E. (2001). The use of statistical tools for evaluating computer simulations. Climatic Change, 51, 119-130.
10. Berk, R. A., Bickel, P., Campbell, K., Keller-McNulty, S., Kelly, E., & Sacks, J. (2002). Workshop on statistical approaches for the evaluation of complex computer models. Statistical Science, 17, 173-192.
11. Blumstein, A. (1967). Systems analysis of the criminal justice system. Annals of the American Academy of Political and Social Science, 374, 92-100.
12. Breiman, L. (2001a). Statistical modeling: Two cultures (with discussion). Statistical Science, 16, 199-231.
13. Breiman, L. (2001b). Random forests. Machine Learning, 45, 5-32.
14. Carley, K. M. (1996). Validating computational models. Pittsburgh, PA: Carnegie Mellon University, Department of Sociology.
15. Clark, M., & Holm, E. (1987). Microsimulation methods and spatial analysis planning. Geografiska Annaler, Series B, Human Geography, 69(2), 145-164.
16. Cellier, F. E. (1991). Continuous system modeling. New York: Springer-Verlag.
17. Cressie, N. A. (1993). Statistics for spatial data. New York: John Wiley.
18. Edmunds, B. (2006). How are physical and social spaces related? - Cognitive agents as necessary glue. In F. C. Billari, T. Fent, A. Prskawetz, & J. Scheffran (Eds.), Agent-based computational modeling: Applications in demography, social, economic, and environmental sciences. New York: Physica-Verlag, Springer.
19. Gass, S. I., & Thompson, B. W. (1980). Guidelines for model evaluation: An abridged version of the US General Accounting Office Exposure Draft. Operations Research, 28(2), 431-439.
20. Fawcett, K. R., Anderson, M. G., Bates, P. D., Jordon, J.-P. & Bathhurst, J. J. (1995). The importance of internal validation in the assessment of physically based distributed models. Transactions of the Institute of British Geographers, New Series, 20(2), 248-265.
21. Friedman, J. H. (2002). Stochastic gradient boosting. Computational Statistics and Data Analysis, 38, 367-378.
22. Gilbert, N. (2008). Agent-based models. Los Angeles: Sage Publications.
23. Gilbert, N., & Troitzsch, K. G. (2005). Simulation for the Social Scientist (2nd ed.). New York: Open University Press.
24. Goldstein, J. (1999). Emergence as a construct: History and issues. Emergence: Complexity, and Organization, 1, 49-82.
25. Grassl, H. (2000). Status and improvement of coupled circulation models. Science, 288, 1991-1997.
26. Groff, L. R. (2007). Simulation for theory testing and experimentation: An example using routine activity theory and street robbery. Journal of Quantitative Criminology, 23, 75-103.
27. Hänninen, H. (1995). Assessing ecological implications of climate change: Can we rely on our simulation models? Climate Change, 31, 1-4.
28. Hastie, T., Tibshirani, R., & Friedman, J. (2001). The elements of statistical learning: Data mining, inference, and prediction. New York: Springer-Verlag.
29. Haylock, R. G., & O'Hagan, A. (1996). On inference for outputs of computationally expensive algorithms with uncertainty on the inputs. In J. M. Bernardo, et al. (Eds.), Bayesian statistics 5 (pp. 629-637). London: Oxford University Press.
30. Hamilton, J. D. (1994). Time series analysis. Princeton: Princeton University Press.
31. Hanneman, R. A. (1995). Simulation modeling and theoretical analysis in sociology. Sociological Perspectives, 38(4), 457-462.
32. Jones, G. T., Mezic, I., Gruenwald, P. J., & Gorman, D. M. (2007). Agent-based modeling: Use with necessary caution. American Journal of Public Health, 97(5), 780-781.
33. Kelter, D. (1973). A random walk epidemic simulation. Journal of the American Statistical Association, 68(344), 821-823.
34. Kennedy, M. C., & O'Hagan, A. (2001). Bayesian calibration of computer models (with discussion). Journal of the Royal Statistical Society, Series B, 63, 425-464.
35. Kirman, A. P., & Vriend, N. J. (2001). Evolving market structure: An ACE Model of price dispersion and loyalty. Journal of Economic Dynamics and Control, 25(3-4), 459-502.
36. Kleindorfer, G. B., O'Neill, L., & Ganeshan, R. (1998). Validation in simulation: Various positions in the philosophy of science. Management Science, 44(8), 1087-1099.
37. Kleijen, J. P. C. (1995). Verification and validation of simulation models. European Journal of Operations Research, 82, 145-162.
38. Leik, R. K., & Meeker, B. F. (1995). Computer simulation for exploring theories: Models of interpersonal cooperation and competition. Sociological Perspectives, 38(4), 463-482.
39. Lim, Y. B., Sacks, J., Studden, W. J., & Welch, W. J. (2002). Design and analysis of computer experiments when the output is highly correlated over the input space. The Canadian Journal of Statistics, 30(1), 109-126.
40. Linkletter, C., Bingham, D., Hengartner, N., Higdon, D., & Ye, K. Q. (2006). Variable selection for Gaussian process models in computer experiments. Technometrics, 48(4), 478-489.
41. Love, G., & Back, G. (2000). Model verification and validation for rapidly developed simulation models: Balancing cost and theory. In Proceedings of the 18th international conference of the system dynamics society, Bergen, Norway.
42. McKay, M. D., Conover, W. J., & Beckman, R. J. (1979). A comparison of three methods for selecting values of input variables in the analysis of output from computer code. Technometrics, 21(45), 239-245.
43. Mease, D., & Bingham, D. (2006). Latin hypercube sampling for computer experiments. Technometrics, 48(4), 467-477.
44. Miller, J. H., & Page, S. E. (2007). Complex adaptive systems: An introduction to computational models of social life. Princeton: Princeton University Press.
45. Mihram, G. A. (1972). Some practical aspects of the verification and validation of simulation models. Operations Research Quarterly, 23(1), 17-29.
46. Oakley, J. E., & O'Hagan, A. (2004). Probabilistic sensitivity analysis of complex models: A Bayesian approach. Journal of the Royal Statistical Society, Series B, 66, 751-769.
47. O'Hagan, A., Kennedy, M. C., & Oakley, J. E. (1998). Uncertainty analysis and other inference tools for complex computer codes. In J. M. Bernardo, J. O. Berger, A. P. David, & A. F. M. Smith (Eds.), Bayesian statistics, 6. Oxford: University Press.
48. Perez, P., & Dray A. (2005). SimDrug: Exploring the complexity of heroin use in Melbourne. Drug Policy Modeling Project: Monograph 11. Fitzroy, Australia: Turning Point Alcohol and Drug Centre.
49. Petit, J. R., Jouzel, J., Raynaud, D., Barkov, N. I., Barnola, J. M., Basile I., et al. (1999). Climate and Atmospheric History of the past 420,000 years from the Vostok Ice Core, Antarctica. Nature, 399, 429-436.
50. Randall, D. A. (Ed.) (2000). General circulation model development: Past present and future. New York: Academic Press.
51. Sacks, J., Welch, W., Mitchell, T. J., & Wynn, H. P. (1989). Design and analysis of computer experiments. Statistical Science, 4, 409-435.
52. Sacks, J., Rouphail, N. M., Park, B. B., & Thakuriah, P. (2000). Statistically-based validation of computer simulation models in traffic operations and management. Paper submitted to the Journal of Transportation and Statistics.
53. Saltelli, A., Chan, K., & Scott, M. (Eds.) (2000). Sensitivity analysis. Chichester, UK: Wiley.
54. Santner, T. J., Williams, B. J., & Notz, W. I. (2003). The design and analysis of computer experiments. New York: Springer.
55. Schoenberg, F., Berk, R. A., Fovell, R. G., Li, C., Lu, R., & Weiss, R. E. (2000). Approximation and inversion of a complex meteorological system via local linear filters. Journal of Applied Meteorology, 40(3), 446-458.
56. Sargent, R. G. (1998). Verification and validation of simulation models. In D. J. Medekros, E. F. Watson, J. S. Carson, & M. S. Manivannan (Eds.), Proceedings of the 1998 winter simulation conference, Piscataway, New Jersey, IEEE, 500-503.
57. Tabor, C. S., & Timpone, R. J. (1996). Computational modeling. Los Angeles: Sage Publications.
58. Tesfatsion, L. (2002). Agent-based computational economics: Growing economies from the bottom up. Artificial Life, 8, 55-82.
59. Troitzsch, K. G. (2004). Validating simulation models. In G. Horton (Ed.), 18th European simulation multiconference: networked simulation and simulation networks (pp. 265-270). SCS Publishing House: Erlangen, San Diego CA.
60. van Baal, P. V. (2004). Computer simulations of criminal deterrence: From public policy to local interaction to individual behavior. Den Hagg, Netherlands: BJU.
61. Van Dyke, P., Savit, R., & Riolo, R. L. (1998). Agent-based modeling vs. equation-based modeling: A case study and user's guide. In J. A. Sichman, R. Conte, & N. Gilbert (Eds.), Multi-agent systems and agent-based simulation. First International Workshop, MABS 98, Paris, France, July 4-6, 1998. Proceedings. Heidelberg: Springer.
62. Vicek, T. (2000). Complexity: The bigger picture. Nature, 418, 131.
63. Vilaand, B. J., & Cohen, L. E. (1993). Crime as strategy: Testing an evolutionary ecological theory of expropriative behavior. American Journal of Sociology, 98(4), 873-912.
64. Windrum, P., Fagiolo G., & Moneta, A. (2007). Empirical validation of agent-based models: Alternatives and prospects. Journal of Artificial Societies and Social Simulations, 10(2), 8. http://jasss.soc.surrey.ac.uk/10/2/8.html.
65. Younger, S (2005). Violence and revenge in Egalitarian Societies. Journal of Artificial Societies and Social Simulation, 8(4), 11. http://jasss.soc. surrey.ac.uk/8/4/11.html.
66. Ziegler, B. P., Praefofer, H., & Kim, T. G. (2000). Theory of modeling and simulation: Integrating discrete event and continuous complex dynamic systems (2nd ed.). New York: Academic Press.