Prediction under uncertainty as a boundary problem: A general formulation using Iterative Closed Question Modelling

Environmental Modelling and Software

Volumn 70, Issue , 2015, Pages 97-112

  Guillaume, Joseph H A ^a,b   Kummu, Matti ^b   Räsänen, Timo A ^b   Jakeman, Anthony J ^a  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^b AALTO UNIVERSITY   (Finland)

Author keywords

Boundary critique; Iterative Closed Question Modelling; Process philosophy; Uncertainty analysis

Indexed keywords

FORECASTING; ITERATIVE METHODS;

BOUNDARY CRITIQUE; BOUNDARY PROBLEMS; ENVIRONMENTAL SYSTEMS; FLOW DURATION CURVE; GENERIC APPROACH; TWO PARAMETER; UNCERTAINTY ANALYSIS TECHNIQUE;

UNCERTAINTY ANALYSIS;

CLIMATE MODELING; CLIMATE PREDICTION; MODELING; PHILOSOPHY; TECHNOLOGICAL CHANGE; UNCERTAINTY ANALYSIS;

EID: 84929193167     PISSN: 13648152     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.envsoft.2015.04.004     Document Type: Article

References (56)

1. J. Alcamo (Ed.). Environmental Futures: the Practice of Environmental Scenario Analysis 2008, Elsevier Science, Amsterdam.
2. Alhabeeb M.J. Break-even analysis. Mathematical Finance 2012, 247-273. John Wiley & Sons, Inc. 10.1002/9781118106907.ch13.
3. Andrews F. hydromad: Hydrological Model Assessment and Development 2010, Available at:, (accessed 14.01.14.). http://hydromad.catchment.org/.
4. Andrews F.T., Croke B.F.W., Jakeman A.J. An open software environment for hydrological model assessment and development. Environ. Modell. Softw. 2011, 26(10):1171-1185. 10.1016/j.envsoft.2011.04.006.
5. Belia E., Amerlinck Y., Benedetti L., Johnson B., Sin G., Vanrolleghem P., Gernaey K., Gillot S., Neumann M., Rieger L. Wastewater treatment modelling: dealing with uncertainties. Water Sci. Technol. 2009, 60(8):1929. 10.2166/wst.2009.225.
6. Bennett N.D., Croke B.F.W., Guariso G., Guillaume J.H.A., Hamilton S.H., Jakeman A.J., Marsili-Libelli S., Newham L.T.H., Norton J.P., Perrin C., Pierce S.A., Robson B., Seppelt R., Voinov A.A., Fath B.D., Andreassian V. Characterising performance of environmental models. Environ. Modell. Softw. 2013, 40:1-20. 10.1016/j.envsoft.2012.09.011.
7. Berger J. Statistical Decision Theory and Bayesian Analysis 1985, Springer. second ed.
8. Beven K. A manifesto for the equifinality thesis. J. Hydrol. 2006, 320:18-36. 10.1016/j.jhydrol.2005.07.007.
9. Bryant B.P. sdtoolkit: Scenario Discovery Tools to Support Robust Decision Making, R Package Version 2.31 2009, http://CRAN.R-project.org/package=sdtoolkit.
10. Bryant B.P., Lempert R.J. Thinking inside the box: a participatory, computer-assisted approach to scenario discovery. Technol. Forecast. Soc. Change 2010, 77(1):34-49. 10.1016/j.techfore.2009.08.002.
11. Carpenter S.R., Folke C., Scheffer M., Westley F. Resilience: accounting for the noncomputable. Ecol. Soc. 2009, 14(1):13.
12. Churchman C.W. Operations research as a profession. Manage. Sci. 1970, 17:B37-B53. 10.2307/2629214.
13. Deb K., Pratap A., Agarwal S., Meyarivan T. A fast and elitist multiobjective genetic algorithm: NSGA-II. Evol. Comput. IEEE Trans. 2002, 6(2):182-197. 10.1109/4235.996017.
14. Dewar J.A., Builder C.H., Hix W.M., Levin M. Assumption-Based Planning: a Planning Tool for Very Uncertain Times 1993, RAND Corporation, Santa Monica, CA, Available at:, (accessed 14.01.14.). http://www.rand.org/pubs/monograph_reports/MR114.
15. Eldred M.S., Swiler L.P., Tang G. Mixed aleatory-epistemic uncertainty quantification with stochastic expansions and optimization-based interval estimation. Reliab. Eng. Syst. Saf. 2011, 96:1092-1113. 10.1016/j.ress.2010.11.010.
16. Glen G., Isaacs K. Estimating Sobol sensitivity indices using correlations. Environ. Modell. Softw. 2012, 37:157-166. 10.1016/j.envsoft.2012.03.014.
17. Guillaume J.H., Jakeman A.J. Providing Scientific Certainty in Predictive Decision Support: the Role of Closed Questions International Environmental Modelling and Software Society (IEMSs) 2012 International Congress on Environmental Modelling and Software. Managing Resources of a Limited Planet: Pathways and Visions under Uncertainty, Sixth Biennial Meeting 2012, International Environmental Modelling and Software Society, Leipzig, Available at:, (accessed 14.01.14.). http://www.iemss.org/iemss2012/proceedings/E2_0432_Guillaume_Jakeman.pdf.
18. Guillaume J.H.A., Andrews F. dream: DiffeRential Evolution Adaptive Metropolis. R Package Version 0.4-2 2012, Available at:, (accessed 14.01.14.). http://dream.r-forge.r-project.org/.
19. Guillaume J.H.A., Pierce S.A., Jakeman A.J. Managing Uncertainty in Determining Sustainable Aquifer Yield 2010, National Groundwater Conference 2010, Canberra, Australia, Available at:, (accessed 14.01.14.). http://www.groundwater2010.com/documents/GuillaumeJoseph_000.pdf.
20. Guillaume J.H.A., Qureshi M.E., Jakeman A.J. A structured analysis of uncertainty surrounding modeled impacts of groundwater extraction rules. Hydrogeology J. 2012, 205(5):915-932. 10.1007/s10040-012-0864-0.
21. Guillaume Joseph H.A., El Sawah Sondoss Fostering assumption-based stress-test thinking in managing groundwater systems: learning to avoid failures due to basic dynamics. Hydrogeology J. 2014, June. 10.1007/s10040-014-1153-x.
22. Haasnoot M., van Deursen W.P.A., Guillaume J.H.A., Kwakkel J.H., van Beek E., Middelkoop H. Fit for purpose? Building and evaluating a fast, integrated model for exploring water policy pathways. Environ. Modell. Softw. 2014, 60(October):99-120. 10.1016/j.envsoft.2014.05.020.
23. Hinman P.G. Fundamentals of Mathematical Logic 2005, AK Peters, USA.
24. Hoeting J.A., Madigan D., Raftery A.E., Volinsky C.T. Bayesian model averaging: a tutorial. Stat. Sci. 1999, 14(4):382-417. 10.1214/ss/1009212519.
25. Joseph J.F., Guillaume J.H.A. Using a parallelized MCMC algorithm in R to identify appropriate likelihood functions for SWAT. Environ. Modell. Softw. 2013, 46(0):292-298. 10.1016/j.envsoft.2013.03.012.
26. Keesman K., van Straten G. Set membership approach to identification and prediction of lake eutrophication. Water Resour. Res. 1990, 26(11):2643-2652. 10.1029/WR026i011p02643.
27. Keesman K.J., Koskela J., Guillaume J.H.A., Norton J.P., Croke B., Jakeman A.J. Uncertainty modelling and analysis of environmental systems: concepts, tools and examples. 19th International Congress on Modelling and Simulation MODSIM2011, Perth, Australia 2011, Available at:, (accessed 14.01.14.). http://www.mssanz.org.au/modsim2011/I12/keesman.pdf.
28. Keesman K.J., Norton J.P., Croke B.F.W., Newham L.T.H., Jakeman A.J. Set-membership approach for identification of parameter and prediction uncertainty in power-law relationships: the case of sediment yield. Environ. Modell. Softw. 2013, 40(0):171-180. 10.1016/j.envsoft.2012.09.007.
29. Lempert R.J., Popper S.W., Bankes S.C. Shaping the Next One Hundred Years: New Methods for Quantitative, Long-term Policy Analysis 2003, Rand Corporation, USA.
30. Matott L.S., Babendreier J.E., Purucker S.T. Evaluating uncertainty in integrated environmental models: a review of concepts and tools. Water Resour. Res. 2009, 45(6). W06421. 10.1029/2008wr007301.
31. Midgley G. Systemic Intervention: Philosophy, Methodology, and Practice 2000, Kluwer Academic/Plenum Publishers, New York.
32. Moore C., Wöhling T., Doherty J. Efficient regularization and uncertainty analysis using a global optimization methodology. Water Resour. Res. 2010, 46(8). W08527. 10.1029/2009wr008627.
33. Nazemi A., Wheater H.S., Chun K.P., Elshorbagy A. A stochastic reconstruction framework for analysis of water resource system vulnerability to climate-induced changes in river flow regime. Water Resour. Res. 2013, 49(1):291-305. 10.1029/2012wr012755.
34. Neyman J., Pearson E.S. On the problem of the most efficient tests of statistical hypotheses. Philosophical Trans. R. Soc. Lond. Ser. A 1933, 231:289-337. Containing Papers of a Mathematical or Physical Character.
35. Norton J.P. Roles for deterministic bounding in environmental modelling. Ecol. Model. 1996, 86(2-3):157-161. 10.1016/0304-3800(95)00045-3.
36. Norton J.P., Brown J.D., Mysiak J. To what extent, and how, might uncertainty be defined? Comments engendered by "Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support": Walker et al., Integrated Assessment 4: 1, 2003. Integr. Assess. J. 2006, 6(1):83-88.
37. Oreskes N., Shrader-Frechette K., Belitz K. Verification, validation, and confirmation of numerical models in the earth sciences. Science 1994, 263(5147):641-646. 10.1126/science.263.5147.641.
38. Pittock J., Finlayson C.M., Howitt J. Beguiling and risky: 'environmental works and measures' for wetland conservation under a changing climate. Hydrobiologia 2013, 708(1):111-131. 10.1007/s10750-012-1292-9.
39. Popper K. The Logic of Scientific Discovery 1959, Hutchinson, London.
40. R Development Core Team R: a Language and Environment for Statistical Computing 2010.
41. Ravalico J., Maier H., Dandy G. Sensitivity analysis for decision-making using the MORE method-a Pareto approach. Reliab. Eng. Syst. Saf. 2009, 94(7):1229-1237. 10.1016/j.ress.2009.01.009.
42. Refsgaard J.C., van der Sluijs J.P., Højberg A.L., Vanrolleghem P.A. Uncertainty in the environmental modelling process - a framework and guidance. Environ. Modell. Softw. 2007, 22(11):1543-1556. 10.1016/j.envsoft.2007.02.004.
43. Saltelli A., Annoni P. How to avoid a perfunctory sensitivity analysis. Environ. Modell. Softw. 2010, 25(12):1508-1517. 10.1016/j.envsoft.2010.04.012.
44. Saltelli A., Chan K., Scott E. Sensitivity Analysis 2004, Wiley, New York.
45. Schoups G., Vrugt J.A. A formal likelihood function for parameter and predictive inference of hydrologic models with correlated, heteroscedastic, and non-Gaussian errors. Water Resour. Res. 2010, 46(10). 10.1029/2009WR008933.
46. Schwanitz Valeria Jana Evaluating integrated assessment models of global climate change. Environ. Modell. Softw. 2013, 50(December):120-131. 10.1016/j.envsoft.2013.09.005.
47. Sterman John D. All models are wrong: reflections on becoming a systems scientist. Syst. Dyn. Rev. 2002, 18(4):501-531. 10.1002/sdr.261.
48. Thyer M., Renard B., Kavetski D., Kuczera G., Franks S.W., Srikanthan S. Critical evaluation of parameter consistency and predictive uncertainty in hydrological modeling: a case study using Bayesian total error analysis. Water Resour. Res. 2009, 45. 10.1029/2008wr006825.
49. Trautmann H., Steuer D., Mersmann O. mco: Multi Criteria Optimization Algorithms and Related Functions. R Package Version 1.0.9 2010, Available at:, (accessed 21.06.13.). http://CRAN.R-project.org/package=mco.
50. Ulrich W. Critical Heuristics of Social Planning: a New Approach to Practical Philosophy 1983, J. Wiley & Sons.
51. Voinov A., Bousquet F. Modelling with stakeholders. Environ. Modell. Softw. 2010, 25(11):1268-1281. 10.1016/j.envsoft.2010.03.007.
52. Vrugt J.A., Ter Braak C., Diks C., Robinson B.A., Hyman J.M., Higdon D. Accelerating Markov chain Monte Carlo simulation by differential evolution with self-adaptive randomized subspace sampling. Int. J. Nonlinear Sci. Numer. Simul. 2009, 10(3):273-290. 10.1515/IJNSNS.2009.10.3.273.
53. Walker W.E., Harremoës P., Rotmans J., van der Sluijs J.P., van Asselt M.B.A., Janssen P., von Krauss M.P.K. Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support. Integr. Assess. 2003, 4(1):5-17. 10.1076/iaij.4.1.5.16466.
54. Walter E., Piet-Lahanier H. Estimation of parameter bounds from bounded-error data: a survey. Math. Comput. Simul. 1990, 32(5-6):449-468. 10.1016/0378-4754(90)90002-Z.
55. Warmink J.J., Janssen J.A.E.B., Booij M.J., Krol M.S. Identification and classification of uncertainties in the application of environmental models. Environ. Modell. Softw. 2010, 25(12):1518-1527. 10.1016/j.envsoft.2010.04.011.
56. Yeh W.W.-G. Review of parameter identification procedures in groundwater hydrology: the inverse problem. Water Resour. Res. 1986, 22(2):95-108. 10.1029/WR022i002p00095.