Meta-analyses: Some very good steps toward a bit longer journey

Organizational Research Methods

Volumn 11, Issue 1, 2008, Pages 127-147

  Dalton, Dan R ^a   Dalton, Catherine M ^a  

^a INDIANA UNIVERSITY   (United States)

Author keywords

Bayesian approaches; Fixed random effects models; Meta analysis; Moderating effects

Indexed keywords

EID: 37149009077     PISSN: 10944281     EISSN: 15527425     Source Type: Journal    
DOI: 10.1177/1094428107304409     Document Type: Article

References (82)

1. Aguinis, H. (1995). Statistical power problems with moderated multiple regression in management research. Journal of Management, 21, 1141-1158.
2. Aguinis, H. (2001). Estimation of sampling variance of correlations in meta-analysis. Personnel Psychology, 54, 569-590.
3. Aguinis, H. (2002). Estimation of interaction effects in organization studies. Organizational Research Methods, 5, 207-21l.
4. Aguinis, H. (2004). Regression analysis for categorical moderators. New York: Guilford.
5. Aguinis, H., Beaty, J.C., Boik, R.J., & Pierce, C.A. (2005). Effect size and power in assessing moderating effects of categorical variables using multiple regression: A 30-year review. Journal of Applied Psychology, 9, 94-107.
6. Aguinis, H., Boik, R.J., & Pierce, C.A. (2001). A generalized solution for approximating the power to detect effects of categorical moderator variables using multiple regression. Organizational Research Methods, 4, 291-323.
7. Aguinis, H., Petersen, S.A., & Pierce, C.A. (1999). Appraisal of the homogeneity of error variance assumption and alternatives to multiple regression for estimating moderating effects of categorical variables. Organizational Research Methods, 4, 315-349.
8. Aguinis, H., & Pierce, C.A. (1998 a). Heterogeneity of error variance and the assessment of moderating effects of categorical variables: A conceptual review. Organizational Research Methods, 1, 296-314.
9. Aguinis, H., & Pierce, C.A. (1998 b). Statistical power computations for detecting dichotomous moderator variables with moderated multiple regression. Educational and Psychological Measurement, 24, 577-592.
10. Aguinis, H., & Pierce, C.A. (1998 c). Testing moderator variable hypotheses meta-analytically. Journal of Management, 24, 577-592.
11. Aguinis, H., & Pierce, C.A. (2006). Computation of effect size for moderating effects of categorical variables in multiple regression. Applied Psychological Measurement, 30, 440-442.
12. Aguinis, H., Pierce, C.A., & Stone-Romero, E.J. (1994). Estimating the power to detect dichotomous moderators with moderated multiple regression. Educational & Psychological Measurement, 54, 690-692.
13. Aguinis, H., & Stone-Romero, E.F. (1997). Methodological artifacts in moderated multiple regression and their effects on statistical power. Journal of Applied Psychology, 82, 198-206.
14. Aguinis, H., Sturman, M.C., & Pierce, C.A. (in press). Comparison of three meta-analytic procedures for estimating moderating effects of categorical variables. Organizational Research Methods.
15. Aiken, L.S., & West, S.G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage.
16. Arnold, H.J. (1982). Moderator variables: A clarification of conceptual, analytic, and psychometric issues. Organizational Behavior and Human Performance, 29, 143-174.
17. Arnold. H.J. (1984). Testing moderator variable hypotheses: A reply to Stone and Hollenbeck. Organizational Behavior and Human Performance, 34, 214-224.
18. Brannick, M.T. (2001). Implications of empirical Bayes meta-analysis for test validation. Journal of Applied Psychology, 86, 468-480.
19. Burr, D., & Doss, H. (2005). A Bayesian semiparametric model for random-effects meta-analysis. Journal of the American Statistical Association, 100, 242-251.
20. Cheung, S.F., & Chan, D.K.S. (2004). Dependent effect sizes in meta-analysis: Incorporating the degree of interdependence. Journal of Applied Psychology, 89, 780-791.
21. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2 nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
22. Cohen, J., Cohen, P., West, S.G., & Aiken, L.S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3 rd ed.). Mahwah, NJ: Lawrence Erlbaum.
23. Cooper, H. (1998). Synthesizing research (3 rd ed.). Thousand Oaks, CA: Sage.
24. Cronbach, L.J. (1987). Statistical tests for moderator variables: Flaws in analyses recently proposed. Psychological Bulletin, 102, 414-417.
25. Dalton, D.R., & Dalton, C.M. (2005). Strategic management as a special case for meta-analysis. In D. Ketchen & D. Bergh (Eds.), Research methodology in strategy and management (Vol. 3, pp. 31-63). New York: Elsevier.
26. Erez, A., Bloom, M.C., & Wells, M.T. (1996). Using random rather than fixed effects models in meta-analysis: Implications for situational specificity and validity generalization. Personnel Psychology, 49, 275-306.
27. Field, A.P. (2001). Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixed- and random-effects methods. Psychological Methods, 6, 161-180.
28. Hagan, R.L. (1997). In praise of the null hypothesis statistical test. American Psychologist, 52, 15-24.
29. Hall, S.M., & Brannick, M.T. (2002). Comparison of two random-effects methods of meta-analysis. Journal of Applied Psychology, 87, 377-389.
30. Harlow, L.L., Mulaik, S.A., & Steiger, J.H. (1997). What if there were no significance tests? Mahwah, NJ: Lawrence Erlbaum.
31. Hartung, J., & Knapp, G. (2003). An alternative test procedure for meta-analysis. In R. Schulze, J. Holling, & D. Bohning (Eds.), Meta-analysis: New developments and applications in medical and social sciences (pp. 53-69). Cambridge, MA: Hogrefe & Huber.
32. Hedges, L.V. (1994). Fixed effects models. In H. M. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 285-299). New York: Russell Sage Foundation.
33. Hedges, L.V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
34. Hedges, L.V., & Pigott, T.D. (2001). The power of statistical tests in meta-analysis. Psychological Methods, 6, 203-217.
35. Hedges, L.V., & Pigott, T.D. (2004). The power of statistical tests for moderators in meta-analysis. Psychological Methods, 9, 426-445.
36. Hedges, L.V., & Vevea, J.L. (1998). Fixed- and random effects models in meta-analysis. Psychological Methods, 3, 486-504.
37. Howard, G.S., Maxwell, S.E., & Fleming, K.J. (2000). The proof on the pudding: An illustration of the relative strengths of null hypothesis, meta-analysis, and Bayesian analysis. Psychological Methods, 5, 315-332.
38. Hunt, M. 1997. How science takes stock: The story of meta-analysis. New York: Russell Sage Foundation.
39. Hunter, J.E., & Schmidt, F.L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. New York: Academic Press.
40. Hunter, J.E., & Schmidt, F.L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2 nd ed.). New York: Academic Press.
41. Hunter, J.E., Schmidt, F.L., & Jackson, G.B. (1982). Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage.
42. Jacard, J., Turrisi, R., & Wan, C.K. (1990). Interaction effects in multiple regression. Newbury Park, CA: Sage.
43. Kemery, E.R., Mossholder, D.W., & Dunlap, W.P. (1989). Meta-analysis and moderator variables: A cautionary note on transportability. Journal of Applied Psychology, 74, 168-170.
44. Kisamore, J.L., & Brannick, M.T. (in press). An illustration of the consequences of meta-analysis model choice. Organizational Research Methods.
45. Koslowsky, M., & Sagie, A. (1993). On the efficacy of credibility intervals as indicators of moderator effects in meta-analytic research. Journal of Organizational Behavior, 14, 695-699.
46. Kraemer, H.C., Thiemann, S., & Denenberg, V.H. (1987). How many subjects: Statistical power analysis in research. Newbury Park, CA: Sage.
47. Lipsey, M.W., & Wilson, D.B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
48. Louis, T.A., & Zelterman, D. (1994). Bayesian approaches to research synthesis. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 412-422). New York: Russell Sage Foundation.
49. Lykken, D.T. (1968). Statistical significance in psychological research. Psychological Bulletin, 70, 151-159.
50. Martinussen, M., & Bjornstad, J.F. (1999). Meta-analysis calculations based on independent and nonindependent cases. Educational and Psychological Measurement, 59, 928-950.
51. McNemar, Q. (1969). Moderation of a moderator technique. Journal of Applied Psychology, 53, 69-72.
52. Moeltner, K., Boyle, K.J., & Paterson, R.W. (2007). Meta-analysis and benefit transfer for resource evaluation-addressing classical challenges with Bayesian modeling. Journal of Environmental Economics & Management, 53, 250-269.
53. Murphy, K.R., & Myors, B. (2003). Statistical power analysis. Mahwah, NJ: Lawrence Erlbaum.
54. Overton, R.C. (1998). A comparison of fixed-effects and mixed (random-effects) models for meta-analysis tests of moderator variable effects. Psychological Methods, 3, 354-379.
55. Pearlman, K., Schmidt, F.L., & Hunter, J.E. (1980). Validity generalization results for tests used to predict job proficiency and training success in clerical occupations. Journal of Applied Psychology, 65, 373-406.
56. Petitti, D.B. (2000). Meta-analysis, decision analysis, and cost-effectiveness analysis (2 nd ed.). New York: Oxford University Press.
57. Raudenbush, S.W. (1994). Random effects models. In H. M. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 301-321). New York: Russell Sage Foundation.
58. Raudenbush, S.W., & Byrk, A.S. (1985). Empirical Bayes meta-analysis. Journal of Educational Statistics, 10, 75-98.
59. Rosenthal, R., & DiMatteo, M.R. (2001). Meta-analysis: Recent developments in quantitative methods for literature reviews. In S. T. Fiske, D. L. Schacter, & C. Zahn-Waxler (Eds.), Annual review of psychology (Vol. 52, pp. 59-82). Palo Alto, CA: Annual Reviews.
60. Rosenthal, R., Rosnow, R.L., & Rubin, D.B. (2000). Contrasts and effect sizes in behavioral research. Cambridge, UK: Cambridge University Press.
61. Rothstein, H., McDaniel, M.A., & Borenstein, M. (2002). Meta-analysis: A review of quantitative cumulation methods. In F. Drasgow & N. Schmitt (Eds.), Measuring and analyzing behavior in organizations (pp. 534-570). San Francisco: Jossey-Bass.
62. Russell, C.J., & Gilliland, S.W. (1995). Why meta-analysis doesn't tell us what the data really mean: Distinguishing between moderator effects and moderator processes. Journal of Management, 21, 813-831.
63. Sackett, P.R., Harris, M.M., & Orr, J.M. (1986). On seeking moderator variables in the meta-analysis of cor-relational data: A Monte Carlo investigation of statistical power and resistance to Type I error. Journal of Applied Psychology, 71, 302-310.
64. Sagie, A., & Koslowsky, M. (1993). Detecting moderators with meta-analysis: An evaluation and comparison of techniques. Personnel Psychology, 46, 629-640.
65. Saunders, D.R. (1956). Moderator variables in prediction. Educational and Psychological Measurement, 16, 209-222.
66. Schmidt, F.L., Gast-Rosenberg, I., & Hunter, J.E. (1980). Validity generalization results for computer programmers. Journal of Applied Psychology, 65, 643-661.
67. Schmidt, F.L., & Hunter, J.E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529-540.
68. Schmidt, F.L., Hunter, J.E., Pearlman, K., & Shane, G.S. (1979). Further tests of the Schmidt-Hunter Bayesian validity generalization methods: Implications for the situational specificity hypotheses. Journal of Applied Psychology, 78, 3-12.
69. Schulze, R. (2004). Meta-analysis: A comparison of approaches. Cambridge, MA: Hogrefe & Huber.
70. Steel, P.D.G., & Kammeyer-Mueller, J. (2002). Comparing meta-analytic moderator techniques under realistic conditions. Journal of Applied Psychology, 87, 96-111.
71. Steel, P.D.G., & Kammeyer-Mueller, J. (in press). Bayesian variance estimation for meta-analysis: Quantifying our uncertainty. Organizational Research Methods.
72. Sterling, T.D. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance-or vice versa. Journal of the American Statistical Association, 54, 30-34.
73. Stone, E.E. (1988). Moderator variables in research: A review and analysis of conceptual and methodological issues. In. G. R. Free & K. M. Rowland (Eds.), Research in personnel and human resources management (Vol. 6, pp. 191-229). Greenwich, CT: JAI Press.
74. Stone, E.E., & Hollenbeck, J.R. (1984). Some issues associated with the use of moderated regression. Organizational Behavior and Human Performance, 34, 195-213.
75. Stone, E.E., & Hollenbeck, J.R. (1989). Clarifying some controversial issues surrounding statistical procedures for detecting moderator variables: Empirical evidence and related matters. Journal of Applied Psychology, 74, 3-10.
76. Stone-Romero, E.E., Alliger, G.M., & Aguinis, H. (1994). Type II error problems in the use of moderated multiple regression for the detection of moderating effects of dichotomous variables. Journal of Management, 20, 167-178.
77. Stone-Romero, E.E., & Anderson, L.E. (1994). Techniques for detecting moderating effects: Relative statistical power of multiple regression and the comparison of subgroup-based correlation coefficients. Journal of Applied Psychology, 79, 354-359.
78. Stone-Romero, E.E., & Liakhovitski, D. (2002). Strategies for detecting moderator variables: A review of conceptual and empirical issues. In G. Ferris & J. J. Martocchio (Eds.), Research in personnel and human resources management (Vol. 21, pp. 333-372). Greenwich, CT: JAI Press.
79. Visweswaran, C., & Sanchez, J.I. (1998). Moderator search in meta-analysis: A review and cautionary note of existing approaches. Educational and Psychological Measurement, 58, 77-87.
80. Whitener, E.M. (1990). Confusion of confidence intervals and credibility intervals in meta-analysis. Journal of Applied Psychology, 75, 315-321.
81. Wood, J. (in press). Methodology for dealing with duplicate study effects in a meta-analysis. Organizational Research Methods.
82. Zedeck, S. (1971). Problems with the use of ''moderator'' variables. Psychological Bulletin, 76, 296-310.