Sensitivity analysis of environmental models: A systematic review with practical workflow

Environmental Modelling and Software

Volumn 79, Issue , 2016, Pages 214-232

  Pianosi, Francesca ^a   Beven, Keith ^d   Freer, Jim ^a   Hall, Jim W ^b   Rougier, Jonathan ^a   Stephenson, David B ^c   Wagener, Thorsten ^a  

^a UNIVERSITY OF BRISTOL   (United Kingdom)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c UNIVERSITY OF EXETER   (United Kingdom)

^d LANCASTER UNIVERSITY   (United Kingdom)

Author keywords

Calibration; Evaluation; Robust decision making; Sensitivity Analysis; Uncertainty Analysis

Indexed keywords

CALIBRATION; DECISION MAKING; UNCERTAINTY ANALYSIS;

ENVIRONMENTAL MODEL; ENVIRONMENTAL MODELLING; EVALUATION; MODEL CALIBRATION; MODEL IDENTIFICATION; PRACTICAL GUIDELINES; ROBUST DECISIONS; UNCERTAINTY ASSESSMENT;

SENSITIVITY ANALYSIS;

CALIBRATION; DECISION MAKING; ENVIRONMENTAL MODELING; NUMERICAL MODEL; PLANNING METHOD; SENSITIVITY ANALYSIS; UNCERTAINTY ANALYSIS;

EID: 84958522250     PISSN: 13648152     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.envsoft.2016.02.008     Document Type: Review

References (114)

1. Anderson B., Borgonovo E., Galeotti M., Roson R. Uncertainty in climate change modeling: can global sensitivity analysis be of help?. Risk Anal. 2014, 34(2):271-293.
2. Andres T. Sampling methods and sensitivity analysis for large parameter sets. J. Stat. Comput. Simul. 1997, 57(1-4):77-110.
3. Bai Y., Wagener T., Reed P. A top-down framework for watershed model evaluation and selection under uncertainty. Environ. Model. Softw. 2009, 24(8):901-916.
4. Baroni G., Tarantola S. A general probabilistic framework for uncertainty and global sensitivity analysis of deterministic models: a hydrological case study. Environ. Model. Softw. 2014, 51:26-34.
5. Beven K. Prophecy, reality and uncertainty in distributed hydrological modelling. Adv. Water Resour. 1993, 16:41-51.
6. Beven K., Freer J. Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology. J. Hydrol. 2001, 249(1-4):11-29.
7. Borgonovo E. A new uncertainty importance measure. Reliab. Eng. Syst. Saf. 2007, 92:771-784.
8. Borgonovo E. Sensitivity analysis of model output with input constraints: a generalized rationale for local methods. Risk Anal. 2008, 28:667-680.
9. Borgonovo E. A methodology for determining interactions in probabilistic safety assessment models by varying one parameter at a time. Risk Anal. 2010, 30(3):385-399.
10. Borgonovo E. Transformation and invariance in the sensitivity analysis of computer experiments. J. R. Stat. Soc. Ser. B 2014, 76(5):925-947.
11. Borgonovo E., Plischke E. Sensitivity analysis: a review of recent advances. Eur. J. Oper. Res. 2016, 248(3):869-887.
12. Brown C., Werick W., Leger W., Fay D. A decision-analytic approach to managing climate risks: application to the upper great lakes. J. Am. Water Resour. Assoc. 2011, 47(3):524-534.
13. Butler M.P., Reed P.M., Fisher-Vanden K., Keller K., Wagener T. Identifying parametric controls and dependencies in integrated assessment models using global sensitivity analysis. Environ. Model. Softw. 2014, 59:10-29.
14. Campolongo F., Cariboni J., Saltelli A. An effective screening design for sensitivity analysis of large models. Environ. Model. Softw. 2007, 22(10):1509-1518.
15. Campolongo F., Saltelli A. Sensitivity analysis of an environmental model: an application of different analysis methods. Reliab. Eng. Syst. Saf. 1997, 57(1):49-69.
16. Campolongo F., Saltelli A., Cariboni J. From screening to quantitative sensitivity analysis. A unified approach. Comput. Phys. Commun. 2011, 182(4):978-988.
17. Castaings W., Borgonovo E., Morris M., Tarantola S. Sampling strategies in density-based sensitivity analysis. Environ. Model. Softw. 2012, 38(0):13-26.
18. Cloke H., Pappenberger F., Renaud J. Multi-method global sensitivity analysis (mmgsa) for modelling floodplain hydrological processes. Hydrol. Process. 2008, 22(11):1660-1674.
19. Collins M., Chandler R., Cox P., Huthnance J., Rougier J., Stephenson D. Quantifying future climate change. Nat. Clim. Change 2012, 2:403-409.
20. Cukier R.I., Fortuin C.M., Shuler K.E., Petschek A.G., Schaibly J.H. Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory. J. Chem. Phys. 1973, 59(8):3873-3878.
21. Demaria E.M., Nijssen B., Wagener T. Monte Carlo sensitivity analysis of land surface parameters using the Variable Infiltration Capacity model. J. Geophys. Res. Atmos. 2007, 112(D11).
22. Devenish B., Francis P.N., Johnson B.T., Sparks R.S.J., Thomson D.J. Sensitivity analysis of dispersion modeling of volcanic ash from Eyjafjallajokull in May 2010. J. Geophys. Res. 2012, 117.
23. EC Impact Assessment Guidelines 2009, European Commission, Technical Report 92. http://ec.europa.eu/governance/impact/docs/key_docs/iag_2009_en.pdf.
24. Efron B., Tibshirani R. An Introduction to the Bootstrap 1993, Chapman & Hall/CRC.
25. EPA Guidance on the Development, Evaluation, and Application of Environmental Models 2009, Technical Report EPA/100/K-09/003, Environmental Protection Agency. http://www.epa.gov/crem/library/cred_guidance:0309.pdf.
26. Forrester A., Sobester A., Keane A. Engineering Design via Surrogate Modelling: a Practical Guide 2008, John Wiley & Sons.
27. Freer J., Beven K., Ambroise B. Bayesian estimation of uncertainty in runoff prediction and the value of data: an application of the GLUE approach. Water Resour. Res. 1996, 32(7):2161-2173.
28. Gupta H.V., Wagener T., Liu Y. Reconciling theory with observations: elements of a diagnostic approach to model evaluation. Hydrol. Process. 2008, 22(18):3802-3813.
29. Guse B., Reusser D.E., Fohrer N. How to improve the representation of hydrological processes in SWAT for a lowland catchment - temporal analysis of parameter sensitivity and model performance. Hydrol. Process. 2014, 28(4):2651-2670.
30. Hall J., Boyce S., Wang Y., Dawson R., Tarantola S., Saltelli A. Sensitivity analysis of hydraulic models. ASCE J. Hydraul. Eng. 2009, 135(11):959-969.
31. Hamm N., Hall J., Anderson M. Variance-based sensitivity analysis of the probability of hydrologically induced slope instability. Comput. Geosci. 2006, 32(6):803-817.
32. Harper E., Stella J.C., Fremier A. Global sensitivity analysis for complex ecological models: a case study of riparian cottonwood population dynamics. Ecol. Appl. 2011, 21(4):1225-1240.
33. Helton J. Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal. Reliab. Eng. Syst. Saf. 1993, 42(2-3):327-367.
34. Helton J., Davis F. Illustration of sampling-based methods for uncertainty and sensitivity analysis. Risk Anal. 2002, 22(3):591-622.
35. Herman J.D., Kollat J.B., Reed P.M., Wagener T. From maps to movies: high-resolution time-varying sensitivity analysis for spatially distributed watershed models. Hydrol. Earth Syst. Sci. 2013, 17(12):5109-5125.
36. Hill M., Tiedeman C. Effective Groundwater Model Calibration: with Analysis of Data, Sensitivities, Predictions, and Uncertainty 2007, John Wiley & Sons.
37. Homma T., Saltelli A. Importance measures in global sensitivity analysis of nonlinear models. Reliab. Eng. Syst. Saf. 1996, 52(1):1-17.
38. Howard R. Decision analysis: practice and promise. Manag. Sci. 1988, 34(6):679-695.
39. Iman R., Helton J. An investigation of uncertainty and sensitivity analysis techniques for computer models. Risk Anal. 1988, 8:71-90.
40. Iman R., Hora S. A robust measure of uncertainty importance for use in fault tree system analysis. Risk Anal. 1990, 10:401-406.
41. Kelleher C., Wagener T. Ten guidelines for effective data visualization in scientific publications. Environ. Model. Softw. 2011, 26(6):822-827.
42. Kelleher C., Wagener T., McGlynn B., Ward A.S., Gooseff M.N., Payn R.A. Identifiability of transient storage model parameters along a mountain stream. Water Resour. Res. 2013, 49(9):5290-5306.
43. Kleijnen J., Helton J. Statistical analyses of scatterplots to identify important factors in large-scale simulations, 1: review and comparison of techniques. Reliab. Eng. Syst. Saf. 1999, 65(2):147-185.
44. Kleijnen J., Helton J. Statistical analyses of scatterplots to identify important factors in large-scale simulations, 2: robustness of techniques. Reliab. Eng. Syst. Saf. 1999, 65(2):187-197.
45. Krykacz-Hausmann B. Epistemic sensitivity analysis based on the concept of entropy. Proceedings of SAMO2001, CIEMAT, Madrid 2001, 31-35.
46. Kucherenko S., Tarantola S., Annoni P. Estimation of global sensitivity indices for models with dependent variables. Comput. Phys. Commun. 2012, 183(4):937-946.
47. Lempert R., Bryant B., Bankes S. Comparing Algorithms for Scenario Discovery 2008, Working Papers WR-557-NSF, RAND Corp., Santa Monica, USA.
48. Lempert R., Popper S., Bankes S. Shaping the Next One Hundred Years: New Methods for Quantitative, Long-term Policy Analysis 2003, Monograph Reports MR-1626-RPC, RAND Corp., Santa Monica, USA.
49. Liu H., Sudjianto A., Chen W. Relative entropy based method for probabilistic sensitivity analysis in engineering design. J. Mech. Des. 2006, 128:326-336.
50. Ljung L. System Identification: Theory for the User 1999, PTR Prentice Hall, Upper Saddle River, NJ. second ed.
51. Massmann C., Wagener T., Holzmann H. A new approach to visualizing time-varying sensitivity indices for environmental model diagnostics across evaluation time-scales. Environ. Model. Softw. 2014, 51:190-194.
52. McMillan H., Krueger T., Freer J. Benchmarking observational uncertainties for hydrology: rainfall, river discharge and water quality. Hydrol. Process. 2012, 26(26):4078-4111.
53. Minunno F., van Oijen M., Cameron D., Pereira J. Selecting parameters for bayesian calibration of a process-based model: a methodology based on canonical correlation analysis. SIAM/ASA J. Uncertain. Quantif. 2013, 1(1):370-385.
54. Moore C., Doherty J. Role of the calibration process in reducing model predictive error. Water Resour. Res. 2005, 41:W05020.
55. Morris D.E., Oakley J.E., Crowe J.A. A web-based tool for eliciting probability distributions from experts. Environ. Model. Softw. 2014, 52:1-4.
56. Morris M. Factorial sampling plans for preliminary computational experiments. Technometrics 1991, 33(2):161-174.
57. Nguyen T., de Kok J. Systematic testing of an integrated systems model for coastal zone management using sensitivity and uncertainty analyses. Environ. Model. Softw. 2007, 22(11):1572-1587.
58. Norton J. Algebraic sensitivity analysis of environmental models. Environ. Model. Softw. 2008, 23(8):963-972.
59. Norton J. An introduction to sensitivity assessment of simulation models. Environ. Model. Softw. 2015, 69:166-174.
60. Nossent J., Elsen P., Bauwens W. Sobol sensitivity analysis of a complex environmental model. Environ. Model. Softw. 2011, 26(12):1515-1525.
61. Oakley J., O'Hagan A. Probabilistic sensitivity analysis of complex models: a bayesian approach. J. R. Stat. Soc. Ser. B Stat. Methodol. 2004, 66:751-769.
62. O'Hagan A., Buck C., Daneshkhah A., Eiser J.R., Garthwaite P.H., Jenkinson D.J., Oakley J.E., Rakow T. Uncertain Judgements: Eliciting Experts' Probabilities 2006, Wiley.
63. Pappenberger F., Beven K., Ratto M., Matgen P. Multi-method global sensitivity analysis of flood inundation models. Adv. Water Resour. 2008, 31(1):1-14.
64. Park C., Ahn K. A new approach for measuring uncertainty importance and distributional sensitivity in probabilistic safety assessment. Reliab. Eng. Syst. Saf. 1994, 46:253-261.
65. Pastres R., Chan K., Solidoro C., Dejak C. Global sensitivity analysis of a shallow-water 3D eutrophication model. Comput. Phys. Commun. 1999, 117:62-74.
66. Paton F.L., Maier H.R., Dandy G.C. Relative magnitudes of sources of uncertainty in assessing climate change impacts on water supply security for the southern Adelaide water supply system. Water Resour. Res. 2013, 49(3):1643-1667.
67. Peeters L., Podger G., Smith T., Pickett T., Bark R., Cuddy S. Robust global sensitivity analysis of a river management model to assess nonlinear and interaction effects. Hydrol. Earth Syst. Sci. 2014, 18(9):3777-3785.
68. Pianosi F., Wagener T. A simple and efficient method for global sensitivity analysis based on cumulative distribution functions. Environ. Model. Softw. 2015, 67:1-11.
69. Pianosi F., Wagener T., Rougier J., Freer J., Hall J. Sensitivity analysis of environmental models: a systematic review with practical workflow. Vulnerability, Uncertainty, and Risk 2014, 290-299.
70. Powell S., Baker K. Management Science, the Art of Modeling with Spreadsheets 1992, John Wiley & Sons, Hoboken, NJ, USA. fourth ed.
71. Press W., Teukolsky S., Vetterling W., Flannery B. Numerical Recipes in C 1992, Cambridge University Press. second ed.
72. Prudhomme C., Kay A., Crooks S., Reynard N. Climate change and river flooding: part 2 sensitivity characterisation for British catchments and example vulnerability assessments. Clim. Change 2013, 119(3-4):949-964.
73. Rakovec O., Hill M.C., Clark M.P., Weerts A.H., Teuling A.J., Uijlenhoet R. Distributed Evaluation of Local Sensitivity analysis (DELSA), with application to hydrologic models. Water Resour. Res. 2014, 50(1):409-426.
74. Ratto M., Castelletti A., Pagano A. Emulation techniques for the reduction and sensitivity analysis of complex environmental models. Environ. Model. Softw. 2012, 34:1-4.
75. Reusser D.E., Zehe E. Inferring model structural deficits by analyzing temporal dynamics of model performance and parameter sensitivity. Water Resour. Res. 2011, 47(7).
76. Romano J., Shaikh A. On the uniform asymptotic validity of subsampling and the bootstrap. Ann. Stat. 2012, 40(6):2789-2822.
77. Rosolem R., Gupta H., Shuttleworth W., Zeng X., deGoncalves L. A fully multiple-criteria implementation of the Sobol' method for parameter sensitivity analysis. J. Geophys. Res. 2012, 117(D07103).
78. Saltelli A., Annoni P., Azzini I., Campolongo F., Ratto M., Tarantola S. Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Comput. Phys. Commun. 2010, 181(2):259-270.
79. Saltelli A., D'Hombres B. Sensitivity analysis didn't help. A practitioner's critique of the stern review. Glob. Environ. Change 2010, 20(2):298-302.
80. Saltelli A., Marivoet J. Non-parametric statistics in sensitivity analysis for model output: a comparison of selected techniques. Reliab. Eng. Syst. Saf. 1990, 28(2):229-253.
81. Saltelli A., Ratto M., Andres T., Campolongo F., Cariboni J., Gatelli D., Saisana M., Tarantola S. Global Sensitivity Analysis 2008, The Primer. Wiley.
82. Saltelli A., Ratto M., Tarantola S., Campolongo F. Sensitivity analysis for chemical models. Chem. Rev. 2005, 105:2811-2828.
83. Saltelli A., Ratto M., Tarantola S., Campolongo F. Update 1 of: sensitivity analysis for chemical models. Chem. Rev. 2012, 112:1-21.
84. Saltelli A., Tarantola S., Chan K.P.-S. A quantitative model-independent method for global sensitivity analysis of model output. Technometrics 1999, 41(1):39-56.
85. Shin M.-J., Guillaume J.H., Croke B.F., Jakeman A.J. Addressing ten questions about conceptual rainfall-runoff models with global sensitivity analyses in R. J. Hydrol. 2013, 503(0):135-152.
86. Sieber A., Uhlenbrook S. Sensitivity analyses of a distributed catchment model to verify the model structure. J. Hydrol. 2005, 310(1-4):216-235.
87. Singh R., Wagener T., Crane R., Mann M.E., Ning L. A vulnerability driven approach to identify adverse climate and land use change combinations for critical hydrologic indicator thresholds: application to a watershed in Pennsylvania, USA. Water Resour. Res. 2014, 50:3409-3427.
88. Sobol' I. Sensitivity analysis for non-linear mathematical models. Mat. Model. 1993, 2(1990):112-118. Mathematical Modelling and Computational Experiment 1, 407-414, translated from Russian: I.M. Sobol', Sensitivity estimates for nonlinear mathematical models.
89. Sobol' I., Kucherenko S. Derivative based global sensitivity measures and their link with global sensitivity indices. Math. Comput. Simul. 2009, 79(10):3009-3017.
90. Sorooshian S., Farid A. Response surface parameter sensitivity analysis methods for postcalibration studies. Water Resour. Res. 1982, 18(5):1531-1538.
91. Sorooshian S., Gupta V.K. The analysis of structural identifiability: theory and application to conceptual rainfall-runoff models. Water Resour. Res. 1985, 21(4):487-495.
92. Spear R., Hornberger G. Eutrophication in peel inlet. II. Identification of critical uncertainties via generalized sensitivity analysis. Water Res. 1980, 14(1):43-49.
93. Spear R.C., Grieb T.M., Shang N. Parameter uncertainty and interaction in complex environmental models. Water Resour. Res. 1994, 30(11):3159-3169.
94. Stephenson D., Doblas-Reyes F. Statistical methods for interpreting Monte Carlo ensemble forecasts. Tellus A 2000, 52(3):300-322.
95. Storlie C.B., Swiler L.P., Helton J.C., Sallaberry C.J. Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models. Reliab. Eng. Syst. Saf. 2009, 94(11):1735-1763.
96. Sudret B. Global sensitivity analysis using polynomial chaos expansions. Reliab. Eng. Syst. Saf. 2008, 93(7):964-979.
97. Tang Y., Reed P., van Werkhoven K., Wagener T. Advancing the identification and evaluation of distributed rainfall-runoff models using global sensitivity analysis. Water Resour. Res. 2007, 43(6).
98. Tang Y., Reed P., Wagener T., van Werkhoven K. Comparing sensitivity analysis methods to advance lumped watershed model identification and evaluation. Hydrol. Earth Syst. Sci. 2007, 11:793-817.
99. van Griensven A., Meixner T., Grunwald S., Bishop T., Diluzio M., Srinivasan R. A global sensitivity analysis tool for the parameters of multi-variable catchment models. J. Hydrol. 2006, 324(1-4):10-23.
100. van Werkhoven K., Wagener T., Reed P., Tang Y. Characterization of watershed model behavior across a hydroclimatic gradient. Water Resour. Res. 2008, 44(1).
101. van Werkhoven K., Wagener T., Reed P., Tang Y. Sensitivity-guided reduction of parametric dimensionality for multi-objective calibration of watershed models. Adv. Water Resour. 2009, 32(8):1154-1169.
102. Vanuytrecht E., Raes D., Willems P. Global sensitivity analysis of yield output from the water productivity model. Environ. Model. Softw. 2014, 51:323-332.
103. Vautard R., Beekmann M., Menut L. Applications of adjoint modelling in atmospheric chemistry: sensitivity and inverse modelling. Environ. Model. Softw. 2000, 15(6-7):703-709.
104. Vrugt J.A., ter Braak C., Diks C., Robinson B., Hyman J., Higdon D. Accelerating Markov Chain Monte Carlo simulation by differential evolution with self-adaptive randomized subspace sampling. Int. J. Nonlinear Sci. Numer. Simul. 2009, 10:273-290.
105. Wagener T., Boyle D., Lees M., Wheater H., Gupta H., Sorooshian S. A framework for development and application of hydrological models. Hydrol. Earth Syst. Sci. 2001, 5:13-26.
106. Wagener T., Gupta H.V. Model identification for hydrological forecasting under uncertainty. Stoch. Environ. Res. Risk Assess. 2005, 19(6):378-387.
107. Wagener T., McIntyre N., Lees M.J., Wheater H.S., Gupta H.V. Towards reduced uncertainty in conceptual rainfall-runoff modelling: dynamic identifiability analysis. Hydrol. Process. 2003, 17(2):455-476.
108. Wagner B.J., Harvey J.W. Experimental design for estimating parameters of rate-limited mass transfer: analysis of stream tracer studies. Water Resour. Res. 1997, 33(7):1731-1741.
109. Wang J., Li X., Lu L., Fang F. Parameter sensitivity analysis of crop growth models based on the extended Fourier Amplitude Sensitivity Test method. Environ. Model. Softw. 2013, 48:171-182.
110. Wilby R.L., Dessai S. Robust adaptation to climate change. Weather 2010, 65(7):180-185.
111. Yang J. Convergence and uncertainty analyses in Monte-Carlo based sensitivity analysis. Environ. Model. Softw. 2011, 26(4):444-457.
112. Yilmaz K.K., Gupta H.V., Wagener T. A process-based diagnostic approach to model evaluation: application to the NWS distributed hydrologic model. Water Resour. Res. 2008, 44(9).
113. Young P.C., Spear R.C., Hornberger G.M. Modeling badly defined systems: some further thoughts. Proceedings SIMSIG Conference, Canberra 1978, 24-32.
114. Ziliani L., Surian N., Coulthard T., Tarantola S. Reduced-complexity modeling of braided rivers: assessing model performance by sensitivity analysis, calibration, and validation. J. Geophys. Res. Earth Surf. 2013, 18:1-20.