Variable importance analysis: A comprehensive review

Reliability Engineering and System Safety

Volumn 142, Issue , 2015, Pages 399-432

  Wei, Pengfei ^a   Lu, Zhenzhou ^a   Song, Jingwen ^a  

^a NORTHWESTERN POLYTECHNICAL UNIVERSITY   (China)

Author keywords

Difference based; Graphic variable importance measures; Moment independent; Random forest; Regression technique; Variable importance analysis; Variance based

Indexed keywords

APPLICATION PROGRAMS; DECISION TREES; REGRESSION ANALYSIS;

DIFFERENCE-BASED; MOMENT INDEPENDENTS; RANDOM FORESTS; REGRESSION TECHNIQUES; VARIABLE IMPORTANCES; VARIANCE-BASED;

NUMERICAL METHODS;

EID: 84933518861     PISSN: 09518320     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ress.2015.05.018     Document Type: Review

References (198)

1. A Saltelli, M Ratto, T Andres, F Campolongo, J Cariboni, and D Gatelli Global sensitivity analysis. The primer 2008 John Wiley & Sons Chichester
2. JC Helton, CW Hansen, and CJ. Sallaberry Conceptual structure and computational organization of the 2008 performance assessment for the proposed high-level radioactive waste repository at Yucca mountain, Nevada Reliab Eng Syst Saf 122 2014 223 248
3. M Ionescu-Bujor, and DG. Cacuci A comparative review of sensitivity and uncertainty analysis of large-scale systems - I: Deterministic methods Nucl Sci Eng 147 3 2004 139 203
4. DG Cacuci, and A Ionescu-Bujor comparative review of sensitivity and uncertainty analysis of large-scale systems - II: Statistical methods Nucl Sci Eng 147 3 2004 204 217
5. DG Cacuci, and M. Ionescu-Bujor A comparative review of sensitivity and uncertainty analysis of large-scale systems - II: Statistical methods Nucl Sci Eng 147 3 2004 204 217
6. A Saltelli, M Ratto, S Tarantola, and F. Campolongo Sensitivity analysis for chemical models Chem Rev 105 7 2005 2811 2828
7. E. Borgonovo Measuring uncertainty importance: investigation and comparison of alternative approaches Risk Anal 26 5 2006 1349 1362
8. JW Hall, SA Boyce, Y Wang, RJ Dawson, S Tarantola, and A. Saltelli Sensitivity analysis for hydraulic models J Hydraul Eng - ASCE 135 11 2009 959 969
9. W. Tian A review of sensitivity analysis methods in building energy analysis Renewable Sustainable Energy Rev 20 2013 411 419
10. E. Borgonovo Sensitivity analysis in decision making 2013 Wiley Encyclopedia of Operational Research and Management Science 1 11
11. JC. Helton Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal Reliab Eng Syst Saf 42 2-3 1993 327 367
12. HC Frey, and SR. Patil Identification and review of sensitivity analysis methods Risk Anal 22 3 2002 553 578
13. JC Helton, and FJ. Davis Illustration of sampling-based methods for uncertainty and sensitivity analysis Risk Anal 22 2 2002 591 622
14. A Saltelli, and J. Marivoet Non-parametric statistics in sensitivity analysis for model output: a comparison of selected techniques Reliab Eng Syst Saf 28 2 1990 229 253
15. JPC Kleijnen, and JC. Helton Statistical analyses of scatterplots to identify important factors in large-scale simulation, I: Review and comparison of techniques Reliab Eng Syst Saf 65 2 1999 147 185
16. JC Helton, JD Johnson, CJ Sallaberry, and CB. Storlie Survey of sampling-based methods for uncertainty and sensitivity analysis Reliab Eng Syst Saf 91 10-11 2006 1175 1209
17. CB Storlie, and JC. Helton Multiple predictor smoothing methods for sensitivity analysis: description of techniques Reliab Eng Syst Saf 93 1 2008 28 54
18. CB Storlie, and JC. Helton Multiple predictor soothing methods for sensitivity analysis: example results Reliab Eng Syst Saf 93 1 2008 55 77
19. CB Storlie, LP Swiler, JC Helton, and CJ. Sallaberry Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models Reliab Eng Syst Saf 94 11 2009 1735 1763
20. CB Storlie, BJ Reich, JC Helton, LP Swiler, and CJ. Sallaberry Analysis of computationally demanding models with continuous and categorical inputs Reliab Eng Syst Saf 113 1 2013 30 41
21. JW Johnson, and JM. Lebreton History and use of relative importance indices on organizational research Organ Res Methods 7 3 2004 238 257
22. J. Bi A review of statistical methods for determination of relative importance of correlated predictors and identification of drivers of consumer liking J Sens Stud 27 2 2012 87 101
23. C Strobl, J Malley, and G. Tutz An introduction to recursive portioning: rationale, application and characteristics of classification and regression trees, bagging and random forest Psychol Methods 14 4 2009 323 348
24. A-L Boulesteix, S Janitza, J Kruppa, and IR. König Overview of random forest methodology and practical guidance with emphasis on computational biology and bioinformatics WIREs Data Min Knowledge Discovery 2 6 2012 493 507
25. DS. Siroky Navigating random forest and related advances in algorithmic modeling Stat Surv 3 2009 147 163
26. U. Grömping Variable importance assessment in regression: linear regression versus random forest Am Stat 63 4 2009 308 319
27. JC Helton, and DE. Burmaster Guest Editorial: treatment of aleatory and epistemic uncertainty in performance assessments for complex systems Reliab Eng Syst Saf 54 2-3 1996 91 94
28. ME. Paté-Cornell Uncertainties in risk analysis: six levels of treatment Reliab Eng Syst Saf 54 2-3 1996 95 111
29. GW. Parry The characterization of uncertainty in probabilistic risk assessments of complex systems Reliab Eng Syst Saf 54 2-3 1996 119 126
30. SC. Hora Aleatory and epistemic uncertainty in probability elicitation with an example from hazardous waste management Reliab Eng Syst Saf 54 2-3 1996 217 223
31. AD Kiureghian, and O. Ditlevsen Aleatory or epistemic? Does it matter? Struct Saf 31 2 2009 105 112
32. JC. Helton Uncertainty and sensitivity analysis in the presence of stochastic and subjective uncertainty J Stat Comput Simul 57 1-4 1997 3 76
33. JC Helton, and CJ. Sallaberry Uncertainty and sensitivity analysis: from regulatory requirements to conceptual structure and computational implementation. IFIP Advances in Information and Communication Technology AICT 2012 60 76
34. VK. Rohatg An introduction to probability theory and mathematical statistics 1976 Wiley New York, NY
35. K Campbell, MD KcKay, and BJ. Williams Sensitivity analysis when model outputs are functions Reliab Eng Syst Saf 91 2006 1468 1472
36. M Lamboni, D Makowski, Lehuger, B Gabrielle, and H. Monod Multivariate global sensitivity analysis for dynamic crop models Field Crops Res 113 2009 312 320
37. M Lamboni, H Monod, and D. Makowski multivariate sensitivity analysis to measure global contribution of input factors in dynamic models Reliab Eng Syst Saf 96 2011 450 459
38. O Garcia-Cabrejo, and A. Valocchi Global sensitivity analysis for multivariate output using polynomial chaos expansion Reliab Eng Syst Saf 126 2014 25 36
39. J Cao, FR Du, and ST. Ding Global sensitivity analysis for dynamic systems with stochastic input processes Reliab Eng Syst Saf 118 2013 106 117
40. JC Helton, and FJ. Davis Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems Reliab Eng Syst Saf 81 1 2003 23 69
41. AB. Owen Latin supercube sampling for very high-dimensional simulations ACM Trans Modell Comput Simul 8 1 1998 71 102
42. IM. Sobol' On the distribution of points in a cube and the approximate evaluation of integrals USSR Comput Math Math Phys 7 4 1967 86 112
43. IM. Sobol' Uniformly distributed sequences with an additional uniform property USSR Comput Math Math Phys 16 5 1976 236 242
44. S Kucherenko, B Feil, N Shah, and W. Mauntz The identification of model effective dimensions using global sensitivity analysis Reliab Eng Syst Saf 96 4 2011 440 449
45. S Tarantola, W Becker, and D. Zeitz A comparison of two sampling methods for global sensitivity analysis Comput Phys Commun 183 2012 1061 1072
46. Chalabi Y, Dutang C, Savicky P, Wuertz D. randtoolbox: toolbox for pseudo and quasi random number generation and RNG tests. Available at: ; 2013 (accessed 14 January 2014).
47. R Development Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL ; 2010
48. P Bratley, and BL. Fox Algorithm 659: implementing Sobol's quasi-random sequence generator ACM Trans Math Software 14 1988 88 100
49. RL Iman, and WJ. Conover A distribution-free approach to inducing rank correlation among input variables Commun Stat-Simul Comput 11 3 1982 311 334
50. E. Borgonovo Sensitivity analysis of model output with input constraints: a generalized rationale for local methods Risk Anal 28 3 2008 667 680
51. E Borgonovo, and GE. Apostolakis A new importance measure for risk-informed decision making Reliab Eng Syst Saf 72 2 2001 193 212
52. E. Borgonovo Differential, criticality and Birnbaum importance measures: an application to basic event, groups and SSCs in event trees and binary decision diagrams Reliab Eng Syst Saf 92 10 2007 1458 1467
53. E. Borgonovo Sensitivity analysis with finite changes: an application to modified EOQ models Eur J Oper Res 200 1 2010 127 138
54. A Griewank, and A. Walther Evaluating derivatives: principles and techniques of algorithmic differentiation 2nd ed. 2008 SIAM Philadelphia
55. AM. Dunker Efficient calculation of sensitivity coefficients for complex atmospheric models Atmos Environ 15 7 1981 1155 1161
56. AM. Dunker The decoupled direct method for calculating sensitivity coefficients in chemical kinetics J Chem Phys 81 5 1984 2385 2393
57. MA Kramer, and JM. Calo An improved computational method for sensitivity analysis: green's function method with 'AIM' Appl Math Modell 5 6 1981 432 441
58. M Bartholomew-Biggs, S Brown, B Christianson, and L. Dixon Automatic differentiation of algorithms J Comput Appl Math 124 2000 171 190
59. DG. Cacuci Sensitivity theory for nonlinear systems. I. Nonlinear functional analysis approach J Math Phys 22 12 1981 2794 2802
60. DG. Cacuci Sensitivity theory for nonlinear systems. II. Extensions to additional classes of responses J Math Phys 22 12 1981 2803 2812
61. MD. Morris Factorial sampling plans for preliminary computational experiments Technometrics 33 2 1991 161 174
62. F Campolongo, J Cariboni, and A. Saltelli An effective screening design for sensitivity analysis of large models Environ Modell Software 22 10 2007 1509 1518
63. MV Ruano, J Ribes, A Seco, and J. Ferrer An improved sampling strategy design for application of Morris method to systems with many input factors Environ Modell Software 37 10 2012 103 109
64. MJW. Jansen Analysis of variance designs for model output Comput Phys Commun 117 1 1999 35 43
65. MJW Jansen, WAH Rossing, and RA. Daamen Monte Carlo estimation of uncertainty contributions from several independent multivariate sources J Grasman, G Van Straten, Predictability and nonlinear modeling in natural sciences and economics 1994 Kluwer Dordrecht 334 343
66. Jansen MJW. Wingding stairs sample analysis program WINDINGS 2.0. Technical report, Private communication.
67. K Chan, A Saltelli, and S. Tarantola Winding stairs: a sampling tool to compute sensitivity indices Stat Comput 10 3 2000 187 196
68. A. Saltelli Making best use of model evaluations to compute sensitivity indices Comput Phys Commun 145 2 2002 280 297
69. A Saltelli, P Annoni, I Azzini, F Campolongo, M Ratto, and S. Tarantola Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index Comput Phys Commun 181 2 2010 259 270
70. F Campolongo, A Saltelli, and J. Cariboni From screening to quantitative sensitivity analysis. A unified approach Comput Phys Commun 182 4 2011 978 988
71. A Saltelli, F Campolongo, and J. Cariboni Screening important inputs in models with strong interaction properties Reliab Eng Syst Saf 94 7 2009 1149 1155
72. F Campolongo, and R. Braddock The use of graph theory in the sensitivity analysis of the model output: a second order screening method Reliab Eng Syst Saf 64 1 1999 1 12
73. RA Cropp, and RD. Braddock The new Morris method: an efficient second-order screening method Reliab Eng Syst Saf 78 1 2002 77 83
74. IM Sobol', and S. Kucherenko Derivative based global sensitivity measures and their link with global sensitivity indices Math Comput Simul 79 10 2009 3009 3017
75. IM Sobol', and S. Kucherenko A new derivative based importance criterion for groups of variables and its link with the global sensitivity indices Comput Phys Commun 181 7 2010 1212 1217
76. S Kucherenko, M Rodriguez-Fernandez, C Pantelides, and N. Shah Monte Carlo evaluation of derivative-based global sensitivity measures Reliab Eng Syst Saf 94 7 2009 1135 1148
77. M Lambonia, B Iooss, A-L Popelin, and F. Gamboa Derivative-based global sensitivity measures: general links with Sobol' indices and numerical tests Math Comput Simul 87 2013 45 54
78. RA Johnson, and DW. Wichern Applied multivariate statistical analysis 6th ed. 2007 Pearson Prentice Hall Upper Saddle River, NJ
79. S Chatterjee, and AS. Hadi Regression analysis by example 4th ed. 2006 John Wiley & Son Hoboken, NJ
80. JC Helton, and FJ. Davis Sampling-based methods A Saltelli, K Shan, EM. Scott, Sensitivity analysis 2000 Wiley New York, NY 101 153
81. C Xu, and GZ. Gertner Uncertainty and sensitivity analysis for models with correlated parameters Reliab Eng Syst Saf 93 10 2008 1563 1573
82. W Hao, Z Lu, and L. Tian A novel method for analyzing variance based importance measures of correlated input variables Acta Aeronaut Astronaut Sin 32 9 2011 1637 1643
83. W Hao, Z Lu, P Wei, J Feng, and B. Wang A new method on ANN for variance importance measure analysis of correlated input variables Struct Saf 38 2012 56 63
84. RL Iman, and WJ. Conover The sue of the rank transform in regression Technometrics 21 4 1978 499 509
85. U. Grömping Estimators of relative importance for linear regression based on variance decomposition Am Stat 61 2 2007 139 147
86. RH Lindeman, PF Merenda, and RZ. Gold Introduction to bivariate and multivariate analysis 1980 Scott, Foresman Glenview, IL
87. W. Kruskal Relative importance by averaging over orderings Am Stat 41 1 1987 6 10
88. W. Kruskal Correction to "Relative importance by averaging over orderings" Am Stat 41 1987 341
89. U. Grömping Relative importance for linear regression in R: the package relaimpo J Stat Software 17 1 2006 1 27
90. Feldman B. Relative importance and value. Unpublished manuscript, downloadable at ; 2013 (accessed November 25, 2013).
91. DV. Budescu Dominance analysis: a new approach to the problem of relative importance of predictors in multiple regression Psychol Bull 114 3 1993 542 551
92. R Azen, and DV. Budescu The dominance analysis approach for comparing predictors in multiple regression Psychol Methods 8 2 2003 129 148
93. DV Budescu, and R. Azen Beyond global measures of relative importance: some insights from dominance analysis Organ Res Methods 7 3 2004 341 350
94. JW. Johnson A heuristic method for estimating the relative weight of predictor variables in multiple regression Multivariate Behav Res 35 1 2000 1 19
95. V Zuber, and K. Strimmer High-dimensioanl regression and variable selection using CAR Scores Stat Appl Genet Mol Biol 10 1 2011 Article 34
96. SN. Wood Modelling and smoothing parameter estimation with multiple quadratic penalties J R Stat Soc B 62 2000 413 428
97. C. Loader Local regression and likelihood 1999 Springer New York, NY
98. J. Fox Nonparametric regression: appendix to an R and S-PLUS companion to applied regression Encycl Stat Behav Sci 2002
99. JH Fredman, and W. Stuetzle Projection pursuit regression J Am Stat Assoc 76 376 1981 817 823
100. DF. Specht A general regression neural network IEEE Trans Neural Networks 2 2 1991 568 576
101. D Basak, S Pal, and DC. Patranabis Support vector regression Neural Inf Process - Lett Rev 11 10 2007 203 224
102. SM Clarke, JH Griebsch, and TW. Simpson Analysis of support vector regression for approximation of complex engineering analyses J Mech Des 127 6 2005 1077 1087
103. B. Sudret Global sensitivity analysis using polynomial chaos expansion Reliab Eng Syst Saf 93 7 2008 964 979
104. M Ratto, A Pagano, and PC. Young State dependent parameter metamodelling and sensitivity analysis Comput Phys Commun 177 11 2007 863 876
105. M Ratto, A Pagano, and PC. Young Non-parametric estimation of conditional moments for sensitivity analysis Reliab Eng Syst Saf 94 2 2009 237 243
106. L. Breiman Random forest Mach Learn 45 1 2001 5 32
107. L Breiman, J Friedman, CJ Stone, and RA. Olshen Classification and regression trees 1984 Wadsworth Belmont, CA
108. A Fielding, and CA. O'Muircheartaigh Binary segmentation in survey analysis with particular reference to AID Statistician 25 1977 17 28
109. JR. Quinlan Introduction of decision trees Mach Learn 1 1 1986 81 106
110. JR. Quilan C4.5: programs for machine learning 1993 Morgan Kaufman Publishers Inc San Francisco, CA
111. T Hothorn, K Hornik, and A. Zeileis Unbiased recursive portioning: a conditional inference framework J Comput Graph Stat 15 3 2006 651 674
112. AP White, and WZ. Liu Bias in information-based measures in decision tree induction Mach Learn 15 3 1994 321 329
113. Y-S Shih, and H-W. Tsai Variable selection bias in regression trees with constant fits Comput Stat Data Anal 45 3 2004 595 607
114. C Strobl, A-L Boulesteix, A Zeileis, and T. Hothorn Bias in random forest variable importance measures: illustrations, sources and a solution BMC Bioinf 8 1 2007 25
115. Liaw A, Wiener M. randomForest: Breiman and Cutler's random forests for classification and regression. Available at: ; 2012 (accessed 13 December 2013).
116. Hothorn T, Hornik K, Strobl C, Zeileis A. party: a laboratory for recursive partytioning. Available at: ; 2013 (accessed 13 December 2013).
117. Jaiantilal A. randomforest-matlab: random forest (regression, classification and clustering) implementation for MATLAB (and Standalone)., Available at: ; 2010 (accessed 14 December 2013).
118. LE Raileanu, and K. Stoffel Theoretical comparison between the Gini Index and Information Gain criteria Ann Math Artif Intell 41 1 2004 77 93
119. C Strobl, A-L Boulesteix, T Kneib, T Augustin, and A. Zeileis Conditional variable importance for random forest BMC Bioinf 9 1 2008 307
120. KK Nicodemus, JD Malley, C Strobl, and A. Ziegler The behaviour of random forest permutation-based variable importance measures under predictor correlation BMC Bioinf 11 1 2010 110
121. KJ Archer, and RV. Kimes Empirical characterization of random forest variable importance measures Comput Stat Data Anal 52 4 2008 2249 2260
122. AL Boulesteix, and M. Slawski Stability and aggregation of ranked gene lists Brief Bioinf 10 5 2009 556 568
123. ML Calle, and V. Urrea Letter to the editor: stability of random forest importance measures Brief Bioinf 12 1 2010 86 89
124. KK. Nicoswmus Letter to the editor: on the stability and ranking of predictors from random forest variable importance measures Brief Bioinf 12 4 2011 369 373
125. R Díaz-Uriarte, and SA. de Andrés Gene selection and classification of microarray data using random forest BMC Bioinf 1 2006 7:3
126. BA Goldstein, EC Polley, and FBS. Briggs Random forests for genetic association studies Stat Appl Genet Mol Biol 10 1 2011 1 34
127. SJ Winham, CL Colby, RR Freimuth, X Wang, M de Andrade, and M Huebner SNP interaction detection with random forests in high-dimensional genetic data BMC Bioinf 13 2012 164
128. C Gtanger, and J-L. Lin Using the mutual information coefficient to identify lags in nonlinear models J Time Ser Anal 15 4 1994 371 384
129. WH. Conover Practical nonparametric statistics 3rd ed. 1999 Wiley New York, NY
130. SC Hora, and JC. Helton A distribution-free test for the relationship between model input and output when using Latin hypercube sampling Reliab Eng Syst Saf 79 3 2003 333 339
131. BI. Winer Statistical principles in experimental design 2nd ed. 1971 McGraw New York, NY
132. JA. Peacock Two-dimensional goodness-of-fit testing in astronomy Mon Not R Astron Soc 202 2 1983 615 627
133. PJ Clark, and FC. Evans Distance to nearest neighbor as a measure of sparial relationships in populations Ecology 35 1954 23 30
134. PJ Diggle, and TF. Cox Some distance-based tests of independence for sparsely sampled multivariate spatial point patterns Int Stat Rev 51 1 1983 11 23
135. Arya S, Mount D, Kemp SE, Jefferis G. RANN: fast neighbor search (wraps Arya and Mount's ANN library). ; 2014 (accessed 22 September 2014).
136. IM. Sobol' Sensitivity analysis for non-linear mathematical models Math Modell Comput Exp 1 1993 407 414 Translated from Russian
137. IM. Sobol' Sensitivity estimates for nonlinear mathematical models Matematicheskoe Modelirovanie 2 1990 112 118
138. T Homma, and A. Saltelli Importance measures in global sensitivity analysis of nonlinear models Reliab Eng Syst Saf 52 1 1996 1 17
139. RI Cukier, CM Fortuin, KE Shuler, AG Petschek, and JH. Schaibly Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I. Theory J Chem Phys 59 8 1973 3873 3878
140. RI Cukier, HB Levine, and KE. Shuler Nonlinear sensitivity analysis of multiparameter model systems J Comput Phys 26 1 1978 1 42
141. A Saltelli, S Tarantola, and KPS. Chan A quantitative model-independent method for global sensitivity analysis of model output Technometrics 41 1 1999 39 56
142. S Tarantola, D Gatelli, and T. Mara Random balance designs for the estimation of first order global sensitivity indices Reliab Eng Syst Saf 91 6 2006 717 727
143. C. Xu, and G. Gertner Understanding and comparison of different sampling approaches for the Fourier amplitudes sensitivity test (FAST) Comput Stat Data Anal 55 1 2011 184 198
144. TA. Mara Extension of the RBD-FAST method to the computation of global sensitivity indices Reliab Eng Syst Saf 94 8 2009 1274 1281
145. JE Oakley, and A. O'Hagan Probabilistic sensitivity analysis of complex models: a Bayesian approach J R Stat Soc: Ser B 66 3 2004 751 769
146. GT Buzzard, and DB. Xiu Variance-based global sensitivity analysis via sparse-grid interpolation and cubature Commun Comput Phys 9 3 2011 542 567
147. S. Rahman Global sensitivity analysis by polynomial dimensional decomposition Reliab Eng Syst Saf 96 7 2011 825 837
148. G Li, C Rosenthal, and H. Rabitz High dimensional model representations J Phys Chem A 105 33 2001 7765 7777
149. G Li, S-W Wang, and H. Rabitz Practical approaches to construct RS-HDMR component functions J Phys Chem A 106 37 2002 8721 8733
150. M Marseguerra, R Masini, E Zio, and C. Glacomo Variance decomposition-based sensitivity analysis via neural networks Reliab Eng Syst Saf 79 2 2003 229 238
151. JPC. Kleijnen Kriging metamodeling in simulation: a review Eur J Oper Res 192 1 2009 707 716
152. IM. Sobol' Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates Math Comput Simul 55 1-3 2001 271 280
153. S Tarantola, W Becker, and D. Zeitz A comparison of two sampling methods for global sensitivity analysis Comput Phys Commun 183 5 2012 1061 1072
154. S Kucherenko, S Tarantola, and P. Annoni Estimation of global sensitivity indices for models with dependent variables Comput Phys Commun 183 4 2012 937 946
155. W Hao, Z Lu, and L. Tian Importance measure of correlated normal variables and its sensitivity analysis Reliab Eng Syst Saf 99 2012 151 160
156. W Hao, Z Lu, and L. Li A new interpretation and validation of variance based importance measures for models with correlated inputs Comput Phys Commun 184 5 2013 1401 1413
157. G Li, H Rabitz, PE Yelvington, O Oluwole, F Bacon, CE Kolb, and J. Schoendorf Global sensitivity analysis for systems with independent and/or correlated inputs J Phys Chem A 114 19 2010 6022 6032
158. TA Mara, and S. Tarantola Variance-based sensitivity indices for models with dependent inputs Reliab Eng Syst Saf 107 2012 115 121
159. C. Xu Decoupling correlated and uncorrelated parameter uncertainty contributions for nonlinear models Appl Math Modell 37 24 2013 9950 9969
160. C Xu, and GZ. Gertner Extending a global sensitivity analysis technique to models with correlated parameter Comput Stat Data Anal 51 12 2007 5579 5590
161. C Xu, and GZ. Gertner A general first-order global sensitivity analysis method Reliab Eng Syst Saf 93 7 2008 1060 1071
162. Most T. Variance-based sensitivity analysis in the presence of correlated input variables. In: Fifth international conference on reliable engineering computing (REC), Brno, Czech Republic; 2012
163. C Zhou, Z Lu, L Li, J Feng, and B. Wang A new algorithm for variance based importance analysis of models with correlated inputs Appl Math Modell 37 3 2013 864 875
164. L Li, Z Lu, and C. Zhou Importance analysis for models with correlated input variables by the state dependent parameter method Comput Math Appl 62 12 2011 4547 4556
165. P Wei, Z Lu, W Hao, J Feng, and B. Wang Efficient sampling methods for global reliability sensitivity analysis Comput Phys Commun 183 8 2012 1728 1743
166. IM Sobol', S Tarantola, D Gatelli, S Kucherenko, and W. Mauntz Estimating the approximation error when fixing unessential factors in global sensitivity analysis Reliab Eng Syst Saf 92 7 2007 957 960
167. DL Allairea, and KE. Willcox Distributional sensitivity analysis Procedia-Soc Behav Sci 2 6 2010 7595 7596
168. P Wei, Z Lu, and J. Song A new variance-based global sensitivity analysis technique Comput Phys Commun 184 11 2013 2540 2551
169. CK Park, and KI. Ahn A new approach for measuring uncertainty importance and distributional sensitivity in probabilistic safety assessment Reliab Eng Syst Saf 46 3 1994 253 261
170. M-H Chun, S-J Han, and N-I. Tak An uncertainty importance measure using a distance metric for the change in a cumulative distribution function Reliab Eng Syst Saf 70 3 2000 313 321
171. Z Tang, Z Lu, B Jiang, P Wang, and F. Zhang Entropy-based importance measure for uncertainty model inputs AIAA J 51 10 2013 2319 2334
172. M Baucells, and E. Borgonovo Invariant probabilistic sensitivity analysis Manage Sci 59 11 2013 2536 2549
173. E. Borgonovo A new uncertainty importance measure Reliab Eng Syst Saf 92 6 2007 771 784
174. P Wei, Z Lu, and J. Song Moment-independent sensitivity analysis using copula Risk Anal 34 2 2014 210 222
175. P Wei, Z Lu, and X. Yuan Monte Carlo simulation for moment-independent sensitivity analysis Reliab Eng Syst Saf 110 2013 60 67
176. E Plischke, E Borgonovo, and CL. Smith Global sensitivity measures from given data Eur J Oper Res 226 3 2012 536 550
177. RB. Nelsen An Introduction to Copulas 2nd ed. 2006 Springer New York, NY
178. C Genest, and A-C. Favre Everything you always wanted to know about copula modeling but were afraid to ask J Hydrol Eng 12 4 2007 347 368
179. EF. Wolff N-dimensional measures of dependence Stochastica 4 3 1980 175 188
180. B Schweizer, and EF. Wolff On nonparametric measures of dependence for random variables Ann Stat 9 1981 879 885
181. ZI Botev, JF Grotowski, and DP. Kroese Kernel density estimation via diffusion Ann Stat 38 5 2010 2916 2957
182. Botev ZI. Kernel density estimation using Matlab. Available at ; 2012 (accessed April 5, 2012).
183. E Borgonovo, S Tarantola, E Plischke, and MD. Morris Transformations and invariance in the sensitivity analysis of computer experiments J R Stat Soc B 76 5 2014 925 947
184. J. Sinclair Response to the PSACOIN Level S exercise. PSACOIN Level S intercomparison Nucl Energy Agency. Organ Econ Cooperation Dev 1993
185. R Bolado-Lavin, W Castings, and S. Tarantola Contribution to the sample mean plot for graphical and numerical sensitivity analysis Reliab Eng Syst Saf 94 6 2009 1041 1049
186. S Tarantola, V Kopustinskas, R Bolado-Lavin, A Kaliatka, E Ušpuras, and M. Vaišnoras Sensitivity analysis using contribution to sample variance plot: application to a water hammer model Reliab Eng Syst Saf 99 2012 62 73
187. P Wei, Z Lu, D Wu, and C. Zhou Moment-independent regional sensitivity analysis: application to an environmental model Environ Modell Software 47 2013 55 63
188. P Wei, Z Lu, W Ruan, and J. Song Regional sensitivity analysis using revised mean and variance ratio functions Reliab Eng Syst Saf 121 2014 121 135
189. P Wei, Z Lu, and J. Song Uncertainty importance analysis using parametric moment ratio function Risk Anal 34 2 2014 223 234
190. A Hapfelmeier, T Hothorn, K Ulm, and C. Strobl A new variable importance measure for random forests with missing data Stat Comput 24 6 2014 21 34
191. PC. Young Data-based mechanistic modeling, generalized sensitivity and dominant model analysis Comput Phys Commun 117 1 1999 113 129
192. S Janitza, C Strobl, and A-L. Boulesteix An AUC-based permutation variable importance measure for random forests BMC Bioinf 14 1 2013 19
193. CC Zhou, ZZ Lu, LG Zhang, and JX. Hu Moment-independent sensitivity analysis with corelations Appl Math Modell 38 19-20 2014 4885 4896
194. Joint Research Centre of European Commission. Simlab: a free development framework for sensitivity and uncertainty analysis. Available at: ; 2013 (accessed 15 December 2013).
195. T Ziehn, and AS Tomlin GUI-HDMR - a software tool for global sensitivity analysis of complex models Environ Modell Software 24 7 2009 775 785
196. Ziehn T, Tomlin A. GUI-HDMR: a software tool for global sensitivity analysis. Available at: ; 2011 by contacting Tilo Ziehn or Alison Tomlin (accessed 21 November 2011).
197. Pujol G, Iooss B, Janon A. sensitivity: a collection of functions for factor screening, global sensitivity analysis and reliability sensitivity analysis of model output. Available at: ; 2013 (accessed 22 December 2013).
198. DF Schwarz, IR König, and A. Ziegler On safari to Random Jungle: a fast implementation of Random Forests for high-dimensional data accessed 15 March 2014 Bioinformatics 26 2010 1752 1758 Package available at: