A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model

Environmental Modelling and Software

Volumn 51, Issue , 2014, Pages 269-285

  Gan, Yanjun ^a   Duan, Qingyun ^a   Gong, Wei ^a   Tong, Charles ^b   Sun, Yunwei ^b   Chu, Wei ^c   Ye, Aizhong ^a   Miao, Chiyuan ^a   Di, Zhenhua ^a  

^a BEIJING NORMAL UNIVERSITY   (China)

^b LAWRENCE LIVERMORE NATIONAL LABORATORY   (United States)

^c University of California at Irvine   (United States)

Author keywords

Parameter screening; PSUADE; Sensitivity analysis; Space filling sampling; Uncertainty quantification

Indexed keywords

EFFECTIVENESS AND EFFICIENCIES; FOURIER AMPLITUDE SENSITIVITY TESTS; MULTIVARIATE ADAPTIVE REGRESSION SPLINES; PARAMETER SCREENING; PROBLEM SOLVING ENVIRONMENTS; PSUADE; SPACE-FILLING; UNCERTAINTY QUANTIFICATIONS;

IMPORTANCE SAMPLING; REGRESSION ANALYSIS; SCREENING; SENSITIVITY ANALYSIS; SOIL MOISTURE; UNCERTAINTY ANALYSIS;

MONTE CARLO METHODS;

FOURIER TRANSFORM; HYDROLOGICAL MODELING; NUMERICAL METHOD; PARAMETERIZATION; SAMPLING; SENSITIVITY ANALYSIS; SOIL MOISTURE; UNCERTAINTY ANALYSIS;

MISSOURI; POTOMAC BASIN; SPRINGFIELD; UNITED STATES;

EID: 84887343895     PISSN: 13648152     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.envsoft.2013.09.031     Document Type: Article

References (68)

1. Beven K. Rainfall-Runoff Modelling: The Primer 2004, John Wiley & Sons, Ltd, Chichester, West Sussex.
2. Beven K. Amanifesto for the equifinality thesis. J.Hydrol. 2006, 320(1-2):18-36. 10.1016/j.jhydrol.2005.07.007.
3. Beven K., Binley A. The future of distributed models: model calibration and uncertainty prediction. Hydrol. Process. 1992, 6(3):279-298. 10.1002/hyp.3360060305.
4. Blöschl G., Sivapalan M. Scale issues in hydrological modelling: a review. Hydrol. Process. 1995, 9(3-4):251-290. 10.1002/hyp.3360090305.
5. Borgonovo E., Castaings W., Tarantola S. Model emulation and moment-independent sensitivity analysis: an application to environmental modelling. Environ. Model. Softw. 2012, 34:105-115. 10.1016/j.envsoft.2011.06.006.
6. Box G.E.P., Behnken D.W. Some new three level designs for the study of quantitative variables. Technometrics 1960, 2(4):455-475.
7. k-p fractional factorial designs: part I. Technometrics 1961, 3(3):311-351.
8. Box G.E.P., Wilson K.B. On the experimental attainment of optimum conditions. J.R. Stat. Soc. Ser. B (Methodol.) 1951, 13(1):1-45.
9. Boyle D.P., Gupta H.V., Sorooshian S. Toward improved calibration of hydrologic models: combining the strengths of manual and automatic methods. Water Resour. Res. 2000, 36(12):3663-3674. 10.1029/2000WR900207.
10. Brazil L.E. Multilevel Calibration Strategy for Complex Hydrologic Simulation Models 1988, Ph.D. Thesis, Department of Civil Engineering, Colorado State University, Fort Collins, CO.
11. Breiman L., Friedman J.H., Olshen R.A., Stone C.J. Classification and Regression Trees 1984, Wadsworth & Brooks/Cole Advanced Books & Software, Monterey, CA.
12. Burnash R.J.C. The NWS river forecast system-catchment modeling. Computer Models of Watershed Hydrology 1995, 311-366. Water Resources Publications, Highlands Ranch, CO. V.P. Singh (Ed.).
13. Burnash R.J.C., Ferral R.L., McGuire R.A. AGeneralized Streamflow Simulation System: Conceptual Modeling for Digital Computers 1973, US Department of Commerce, National Weather Service, Sacramento, CA.
14. Campolongo F., Cariboni J., Saltelli A. An effective screening design for sensitivity analysis of large models. Environ. Model. Softw. 2007, 22(10):1509-1518. 10.1016/j.envsoft.2006.10.004.
15. Chipman H.A., George E.I., McCulloch R.E. BART: Bayesian additive regression trees. Ann. Appl. Stat. 2010, 4(1):266-298. 10.1214/09-AOAS285.
16. Clark M.P., Kavetski D., Fenicia F. Pursuing the method of multiple working hypotheses for hydrological modeling. Water Resour. Res. 2011, 47:W09301. 10.1029/2010WR009827.
17. Crawford S.L. Correlation and regression. Circulation 2006, 114(19):2083-2088. 10.1161/CIRCULATIONAHA.105.586495.
18. Cukier R.I., Fortuin C.M., Shuler K.E., Petschek A.G., Schaibly J.H. Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I theory. J.Chem. Phys. 1973, 59(8):3873-3878. 10.1063/1.1680571.
19. Duan Q., Schaake J., Andreassian V., Franks S., Goteti G., Gupta H.V., Gusev Y.M., Habets F., Hall A., Hay L., Hogue T., Huang M., Leavesley G., Liang X., Nasonova O.N., Noilhan J., Oudin L., Sorooshian S., Wagener T., Wood E.F. Model parameter estimation experiment (MOPEX): an overview of science strategy and major results from the second and third workshops. J.Hydrol. 2006, 320(1-2):3-17. 10.1016/j.jhydrol.2005.07.031.
20. Duan Q., Sorooshian S., Gupta V.K. Optimal use of the SCE-UA global optimization method for calibrating watershed models. J.Hydrol. 1994, 158(3-4):265-284. 10.1016/0022-1694(94)90057-4.
21. Duan Q., Sorooshian S., Gupta V. Effective and efficient global optimization for conceptual rainfall-runoff models. Water Resour. Res. 1992, 28(4):1015-1031. 10.1029/91WR02985.
22. Eirola E., Liitiäinen E., Lendasse A., Corona F., Verleysen M. Using the delta test for variable selection. ESANN 2008 Proceedings, European Symposium on Artificial Neural Networks, Bruges, Belgium 2008, 25-30.
23. Friedman J.H. Multivariate adaptive regression splines. Ann. Stat. 1991, 19(1):1-67. 10.1214/aos/1176347963.
24. Galton F. Regression towards mediocrity in hereditary stature. J.Anthropol. Inst. G B Irel. 1886, 15:246-263.
25. Gibbs M., MacKay D.J.C. Efficient Implementation of Gaussian Processes 1997, (Unpublished manuscript).
26. Gupta H.V., Sorooshian S., Yapo P.O. Toward improved calibration of hydrologic models: multiple and noncommensurable measures of information. Water Resour. Res. 1998, 34(4):751-763. 10.1029/97WR03495.
27. Gutiérrez Á G., Schnabel S., Contador J.F.L. Using and comparing two nonparametric methods (CART and MARS) to model the potential distribution of gullies. Ecol. Model 2009, 220(24):3630-3637. 10.1016/j.ecolmodel.2009.06.020.
28. Homma T., Saltelli A. Importance measures in global sensitivity analysis of nonlinear models. Reliab. Eng. Syst. Saf. 1996, 52(1):1-17. 10.1016/0951-8320(96)00002-6.
29. Hornberger G.M., Spear R.C. An approach to the preliminary analysis of environmental systems. J.Environ. Manag. 1981, 12(1):7-18.
30. Hsieh H. Application of the PSUADE Tool for Sensitivity Analysis of an Engineering Simulation 2006, Lawrence Livermore National Laboratory (LLNL), Livermore, CA.
31. Jakeman A.J., Hornberger G.M. How much complexity is warranted in a rainfall-runoff model?. Water Resour. Res. 1993, 29(8):2637-2649. 10.1029/93WR00877.
32. Kahng A.B., Lin B., Samadi K. Improved on-chip router analytical power and area modeling. 2010 Asia and South Pacific Design Automation Conference, Taipei, Taiwan 2010, 241-246.
33. Karypis G., Kumar V. METIS: a Software Package for Partitioning Unstructured Graphs, Partitioning Meshes, and Computing Fill-Reducing Orderings of Sparse Matrices (Version 4.0) 1998, University of Minnesota, Minneapolis, MN.
34. Kavetski D., Clark M.P. Ancient numerical daemons of conceptual hydrological modeling: 2. Impact of time stepping schemes on model analysis and prediction. Water Resour. Res. 2010, 46(10):W10511. 10.1029/2009WR008896.
35. Kavetski D., Kuczera G., Franks S.W. Semi-distributed hydrological modeling: a "saturation path" perspective on TOPMODEL and VIC. Water Resour. Res. 2003, 39(9):1246. 10.1029/2003WR002122.
36. Levy S., Steinberg D.M. Computer experiments: a review. Adv. Stat. Anal. 2010, 94(4):311-324. 10.1007/s10182-010-0147-9.
37. Liu Y., Gupta H.V., Sorooshian S., Bastidas L.A., Shuttleworth W.J. Exploring parameter sensitivities of the land surface using a locally coupled land-atmosphere model. J.Geophys. Res. 2004, 109:D21101. 10.1029/2004JD004730.
38. Loeppky J.L., Sacks J., Welch W.J. Choosing the sample size of a computer experiment: a practical guide. Technometrics 2009, 51(4):366-376. 10.1198/TECH.2009.08040.
39. McKay M.D. Evaluating Prediction Uncertainty 1995, Los Alamos National Laboratory (LANL), Los Alamos, NM.
40. McKay M.D., Beckman R.J., Conover W.J. Acomparison of three methods forselecting values of input variables in the analysis of output from a computercode. Technometrics 1979, 21(2):239-245. 10.1080/00401706.2000.10485979.
41. Meteopolis N., Ulam S. The Monte Carlo method. J.Am. Stat. Assoc. 1949, 44(247):335-341.
42. Morris M.D. Factorial sampling plans for preliminary computational experments. Technometrics 1991, 33(2):161-174. 10.1080/00401706.1991.10484804.
43. Owen A.B. Orthogonal arrays for computer experiments, integration and visualization. Stat. Sinica 1992, 2(2):439-452.
44. Pi H., Peterson C. Finding the embedding dimension and variable dependencies in time series. Neural Comput. 1994, 6(3):509-520. 10.1162/neco.1994.6.3.509.
45. Plackett R.L., Burman J.P. The design of optimum multifactorial experiments. Biometrika 1946, 33(4):305-325.
46. Ratto M., Pagano A., Young P. State dependent parameter metamodelling and sensitivity analysis. Comput. Phys. Commun. 2007, 177(11):863-876. 10.1016/j.cpc.2007.07.011.
47. Renard B., Kavetski D., Kuczera G., Thyer M., Franks S.W. Understanding predictive uncertainty in hydrologic modeling: the challenge of identifying input and structural errors. Water Resour. Res. 2010, 46:W05521. 10.1029/2009WR008328.
48. Sacks J., Welch W.J., Mitchell T.J., Wynn H.P. Design and analysis of computer experiments. Stat. Sci. 1989, 4(4):409-423. 10.1214/ss/1177012413.
49. Saltelli A. Making best use of model evaluations to compute sensitivity indices. Comput. Phys. Commun. 2002, 145(2):280-297. 10.1016/S0010-4655(02)00280-1.
50. Saltelli A., Ratto M., Andres T., Campolongo F. Global Sensitivity Analysis: the Primer 2008, John Wiley & Sons, Ltd, Chichester, West Sussex.
51. Saltelli A., Tarantola S., Chan K.P.S. Aquantitative model-independent method for global sensitivity analysis of model output. Technometrics 1999, 41(1):39-56. 10.2307/1270993.
52. Shahsavani D., Grimvall A. Variance-based sensitivity analysis of model outputs using surrogate models. Environ. Model. Softw. 2011, 26(6):723-730. 10.1016/j.envsoft.2011.01.002.
53. Snow M.G., Bajaj A.K. Uncertainty Quantification Study for a Comprehensive Electrostatic MEMS Switch Model. PRISM 2010, NNSA Center for Prediction of Reliability, Integrity and Survivability of Microsystems, West Lafayette, IN.
54. Sobol' I.M. Quasi-Monte Carlo methods. Progr. Nucl. Energy 1990, 24(1-3):55-61. 10.1016/0149-1970(9090022-W).
55. Sobol' I.M. Sensitivity analysis for nonlinear mathematical models. Math. Model. Comput. Exp. 1993, 1:407-414.
56. Sobol' I.M. Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math. Comput. Simul. 2001, 55(1-3):271-280. 10.1016/S0378-4754(0000270-6).
57. Sorooshian S., Gupta V.K. Automatic calibration of conceptual rainfall-runoff models: the question of parameter observability and uniqueness. Water Resour. Res. 1983, 19(1):260-268. 10.1029/WR019i001p00260.
58. Spearman C. The proof and measurement of association between two things. Am. J. Psychol. 1904, 15(1):72-101.
59. Statnikov R.B., Matusov J.B. Multicriteria Analysis in Engineering: Using the PSI Method with MOVI 1.0 2002, Kluwer Academic Publishers, Dordrecht.
60. Steinberg D., Colla P.L., Martin K. MARS User Guide 1999, Salford Systems, San Diego, CA.
61. Tang B. Orthogonal array-based Latin hypercubes. J.Am. Stat. Assoc. 1993, 88(424):1392-1397. 10.2307/2291282.
62. Tang Y., Reed P., Wagener T., van Werkhoven K. Comparing sensitivity analysis methods to advance lumped watershed model identification and evaluation. Hydrol. Earth Syst. Sci. 2007, 11(2):793-817. 10.5194/hess-11-793-2007.
63. Tong C. PSUADE User's Manual 2005, Lawrence Livermore National Laboratory (LLNL), Livermore, CA.
64. Tong C. Quantifying Uncertainties of a Soil-foundation Structure-interaction System under Seismic Excitation 2008, Lawrence Livermore National Laboratory (LLNL), Livermore, CA.
65. Tong C., Graziani F. Apractical global sensitivity analysis methodology for multi-physics applications. Computational Methods in Transport: Verification and Validation 2008, 277-299. Springer-Verlag, Berlin. F. Graziani (Ed.).
66. van Griensven A., Meixner T., Grunwald S., Bishop T., Diluzio M., Srinivasan R. Aglobal sensitivity analysis tool for the parameters of multi-variable catchment models. J.Hydrol. 2006, 324(1):10-23. 10.1016/j.jhydrol.2005.09.008.
67. van Werkhoven K., Wagener T., Reed P., Tang Y. Characterization of watershed model behavior across a hydroclimatic gradient. Water Resour. Res. 2008, 44(1):W01429. 10.1029/2007WR006271.
68. Wemhoff A.P., Hsieh H. TNT Prout-Tompkins Kinetics Calibration with PSUADE 2007, Lawrence Livermore National Laboratory (LLNL), Livermore, CA.