Variable importance in regression models (WIREs Comp Stat, (2015), 7, (137-152), 10.1002/wics.1346);Variable importance in regression models

Wiley Interdisciplinary Reviews: Computational Statistics

Volumn 7, Issue 2, 2015, Pages 137-152

  Grömping, Ulrike ^a  

^a BEUTH UNIVERSITY OF APPLIED SCIENCES   (Germany)

Author keywords

Correlated regressors; Hierarchical partitioning; Multiple regression; Relative importance

Indexed keywords

LEARNING SYSTEMS; NONLINEAR ANALYSIS;

CORRELATED REGRESSORS; HIERARCHICAL PARTITIONING; MACHINE LEARNING METHODS; MULTIPLE REGRESSIONS; NONLINEAR PARAMETRIC MODEL; RELATIVE IMPORTANCE; VARIABLE IMPORTANCES; VARIANCE DECOMPOSITION;

REGRESSION ANALYSIS;

EID: 84922770098     PISSN: 19395108     EISSN: 19390068     Source Type: Journal    
DOI: 10.1002/wics.1381     Document Type: Erratum

References (74)

1. Fabbris L. Measures of regressor importance in multiple regression: an additional suggestion. Qual Quant 1980, 4:787-792.
2. 2 in multiple regression with correlated regressors. Stat Sin 1993, 3:407-420.
3. Johnson JW. A heuristic method for estimating the relative weight of regressors in multiple regression. Multivar Behav Res 2000, 35:1-19.
4. Hoffman PJ. The paramorphic representation of clinical judgment. Psychol Bull 1960, 57:116-131.
5. Pratt JW. Dividing the indivisible: using simple symmetry to partition variance explained. In: Pukkila T, Puntanen S, eds. Proceedings of Second Tampere Conference in Statistics. Finland: University of Tampere; 1987, 245-260.
6. Chevan A, Sutherland M. Hierarchical Partitioning. Am Stat 1991, 45:90-96.
7. Core Team R. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2014.
8. Ward JH. Comments on "The paramorphic representation of clinical judgment". Psychol Bull 1962, 59:74-76.
9. Darlington RB. Multiple regression in psychological research and practice. Psychol Bull 1968, 69:161-182.
10. Bring J. A geometric approach to compare variables in a regression model. Am Stat 1996, 50:57-62.
11. Thomas DR, Hughes E, Zumbo BD. On variable importance in linear regression. Soc Indic Res 1998, 45:253-275.
12. Fox J, Weisberg S. An R Companion to Applied Regression. Thousand Oaks, CA: Sage; 2011.
13. Achen CH. Interpreting and Using Regression. Newbury Park, CA: Sage; 1982.
14. Budescu DV. Dominance analysis: a new approach to the problem of relative importance in multiple regression. Psychol Bull 1993, 114:542-551.
15. Johnson JW, Lebreton JM. History and use of relative importance indices in organizational research. Organ Res Methods 2004, 7:238-257.
16. Holgersson HET, Norman T, Tavassoli S. In the quest for economic significance: assessing variable importance through mean value decomposition. Appl Econ Lett 2014, 21:545-549.
17. Theil H, Chung C-F. Information-theoretic measures of fit for univariate and multivariate linear regressions. Am Stat 1988, 42:249-252.
18. Silber JH, Rosenbaum PR, Ross RN. Comparing the contributions of groups of predictors: which outcomes vary with hospital rather than patient characteristics? J Am Stat Assoc 1995, 90:7-18.
19. Firth D. relimp: Relative Contribution of Effects in a Regression Model. R package version 1.0-3. In: R Core Team 2014, 2011.
20. Ortmann KM. A cooperative value in a multiplicative model. Cent Eur J Oper Res 2013, 21:561-583.
21. Land M, Gefeller O. A multiplicative variant of the Shapley value for factorizing the risk of disease. In: Parrone F, Garcia-Jurado I, Tijs S, eds. Game Practice: Contributions from Applied Game Theory (Theory and Decision Library 23). New York, NY: Springer; 2000, 143-158.
22. Eide GE, Gefeller O. Sequential and average attributable fractions as aids in the selection of preventive strategies. J Clin Epidemiol 1995, 48:645-655.
23. van der Laan M. Statistical inference for variable importance. Int J Biostat 2006, 2:1-33.
24. Ritter SJ, Jewell NP, Hubbard AE. R Package multiPIM: a causal inference approach to variable importance analysis. J Stat Softw 2014, 57:1-29.
25. Gevrey M, Dimopoulos I, Lek S. Review and comparison of methods to study the contribution of variables in artificial neural network models. Ecol Model 2003, 160:249-264.
26. Kuhn M. caret: Classification and Regression Training. R package version 6.0-30. In: R Core Team, 2014.
27. Pedhazur EJ. Multiple Regression in Behavioral Research: Explanation and Prediction. 2nd ed. New York: Holt, Rinehart and Winston; 1982.
28. Ortmann KM. The proportional value of a positive cooperative game. Math Methods Oper Res 2000, 51:235-248.
29. Feldman B. The proportional value of a cooperative game. Manuscript for a Contributed Paper at the Econometric Society World Congress 2000, 1999. Available at: http://fmwww.bc.edu/RePEc/es2000/1140.pdf. (Accessed August 13, 2014).
30. Gibson WA. Orthogonal predictors: a possible resolution of the Hoffman-Ward controversy. Psychol Rep 1962, 11:32-34.
31. Zuber V, Strimmer K. High-dimensional regression and variable selection using CAR scores. Stat Appl Genet Mol Biol 2011, 10:Article 34.
32. Lindeman RH, Merenda PF, Gold RZ. Introduction to Bivariate and Multivariate Analysis. Glenview, IL: Scott, Foresman; 1980.
33. Kruskal W. Relative importance by averaging over orderings. Am Stat 1987, 41:6-10.
34. Kruskal W. Correction to "Relative importance by averaging over orderings". Am Stat 1987b, 41:341.
35. Kruskal W, Majors R. Concepts of relative importance in recent scientific literature. Am Stat 1989, 43:2-6.
36. MacNally R. Regression and model building in conservation biology, biogeography and ecology: the distinction between and reconciliation of 'predictive' and 'explanatory' models. Biodivers Conserv 2000, 9:655-671.
37. MacNally R. Multiple regression and inference in conservation biology and ecology: further comments on identifying important regressors. Biodivers Conserv 2002, 11:1397-1401.
38. MacNally R, Walsh C. Hierarchical partitioning public-domain software. Biodivers Conserv 2004, 13:659-660.
39. Soofi ES, Retzer JJ, Yasai-Ardekani M. A framework for measuring the importance of variables with applications to management research and decision models. Decis Sci 2000, 31:1-31.
40. Lebreton JM, Ployhart RE, Ladd RT. A Monte Carlo comparison of relative importance methodologies. Organ Res Methods 2004, 7:258-282.
41. Nimon KF, Oswald FL. Understanding the results of multiple linear regression: beyond standardized regression coefficients. Organ Res Methods 2013, 16:650-674.
42. Bi J, Chung J. Identification of drivers of overall liking -determination of relative importances of regressor variables. J Sens Stud 2011, 26:245-254.
43. Bi J. A review of statistical methods for determination of relative importance of correlated predictors and identification of drivers of consumer liking. J Sens Stud 2012, 27:87-101.
44. Lipovetsky S, Conklin M. Analysis of regression in game theory approach. Appl Stoch Models Bus Ind 2001, 17:319-330.
45. Grömping U, Landau S. Do not adjust coefficients in Shapley value regression. Appl Stoch Models Bus Ind 2010, 26:194-202. doi:10.1002/asmb.773.
46. Green PE, Carroll JD, DeSarbo WS. A new measure of regressor importance in multiple regression. J Marketing Res 1978, 15:356-360.
47. Azen R, Budescu DV. The dominance analysis approach for comparing predictors in multiple regression. Psychol Methods 2003, 8:129-148.
48. Budescu DV, Azen R. Beyond global measures of relative importance: some insights from dominance analysis. Organ Res Methods 2004, 7:341-350.
49. Christensen R. Comment on Chevan and Sutherland. Am Stat 1992, 46:74.
50. Stufken J. On hierarchical partitioning. Am Stat 1992, 46:70-71.
51. Shapley L. A value for n-person games. In: Roth A, ed. The Shapley Value: Essays in Honor of Lloyd S. Shapley. Cambridge: Cambridge University Press(game-theoretic background for lmg); Reprinted in 1988, ed.; 1953.
52. Feldman B. Relative importance and value. Manuscript (latest version), 2005. Available at: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2255827. (Accessed August 13, 2014).
53. Grömping U. Estimators of relative importance in linear regression based on variance decomposition. Am Stat 2007, 61:139-147.
54. Grömping U. Relative importance for linear regression in R: the package relaimpo. J Stat Softw 2006, 17:1-27.
55. Ray-Mukherjee J, Nimon K, Mukherjee S, Morris DW, Slotow R, Hamer M. Using commonality analysis in multiple regressions: a tool to decompose regression effects in the face of multicollinearity. Methods Ecol Evol 2014, 5:320-328.
56. Walsh C, Mac Nally R. The hier.part Package: Hierarchical Partitioning. (Part of: Documentation for R: A language and environment for statistical computing.) R Foundation for Statistical Computing, Vienna, Austria, 2013. Available at: http://cran.r-project.org/web/packages/hier.part/hier.part.pdf. (Accessed August 13, 2014).
57. Azen R. Dominance Analysis SAS Macros, 2003. Available at: www.uwm.edu/azen/damacro.html. (Accessed August 13, 2014).
58. Fickel N. Sequenzialregression: Eine neodeskriptive Lösung des Multikollinearitätsproblems mittels stufenweise bereinigter und synchronisierter Variablen. Habilitationsschrift, University of Erlangen-Nuremberg. Berlin: VWF; 2001.
59. Fickel N. Measuring supplementary influence by using sequential linear regression. Math Preprint Arch 2003, 2003:554-573.
60. Johnson RM. The minimal transformation to orthonormality. Psychometrika 1966, 31:61-66.
61. Hoffman PJ. Assessment of the independent contributions of predictors. Psychol Bull 1962, 59:77-80.
62. Thomas DR, Zumbo BD, Kwan E, Schweitzer L. On Johnson's (2000) relative weights method for assessing variable importance: a reanalysis. Multivar Behav Res 2014, 49:329-338.
63. Schäfer J, Opgen-Rhein R, Zuber V, Ahdesmäki M, Silva APD, Strimmer K. corpcor: efficient estimation of covariance and (partial) correlation. R package version 1.6.6. In: R Core Team 2014, 2013.
64. Office of Population Research at Princeton University (no year given). Switzerland Socio-economic variables 1870 to 1930. Website of the Princeton European Fertiliy Project. Available at: http://opr.princeton.edu/archive/fileList.aspx?studyid=10&substudy=x (file efswitz.dat1888). (Accessed August 13, 2014).
65. Grömping U. Variable importance assessment in regression: linear regression versus random forest. Am Stat 2009, 63:308-319.
66. Retzer JJ, Soofi ES, Soyer R. Information importance of predictors: concept, measures, Bayesian inference, and applications. Comput Stat Data Anal 2009, 53:2363-2377.
67. Breiman L. Random forests. Mach Learn 2001, 45:5-32.
68. Strobl C, Boulesteix A, Kneib T, Augustin T, Zeileis A. Conditional variable importance for random forests. BMC Bioinformatics 2008, 9:307.
69. Breiman L, Friedman JH, Olshen RA, Stone CJ. Classification and Regression Trees. Boca Raton: Chapman and Hall; 1984.
70. Hothorn T, Hornik K, van de Wiel MA, Zeileis A. A lego system for conditional inference. Am Stat 2006, 60:257-263.
71. Hothorn T, Hornik K, Zeileis A. Unbiased recursive partitioning: a conditional inference framework. J Comput Graph Stat 2006, 15:651-674.
72. Liaw A, Wiener M. Classification and regression by random forest. R News 2002, 2:18-22.
73. Wang X, Duverger P, Bansal HS. Bayesian inference of predictors relative importance in linear regression model using dominance hierarchies. Int J Pure Appl Math 2013, 88:321-339.
74. Ehrenberg ASC. The unimportance of relative importance. Am Stat 1990, 44:260.