Hypothesis tests for population heterogeneity in meta-analysis

British Journal of Mathematical and Statistical Psychology

Volumn 60, Issue 1, 2007, Pages 29-60

  Viechtbauer, Wolfgang ^a,b  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b MAASTRICHT UNIVERSITY   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

ANALYSIS OF VARIANCE; ARTICLE; EPIDEMIOLOGY; EQUIPMENT; HUMAN; META ANALYSIS; MONTE CARLO METHOD; REPRODUCIBILITY; STATISTICAL ANALYSIS; STATISTICAL MODEL;

ANALYSIS OF VARIANCE; BIAS (EPIDEMIOLOGY); DATA INTERPRETATION, STATISTICAL; EFFECT MODIFIERS (EPIDEMIOLOGY); HUMANS; LIKELIHOOD FUNCTIONS; META-ANALYSIS; MODELS, STATISTICAL; MONTE CARLO METHOD; REPRODUCIBILITY OF RESULTS;

EID: 34249804164     PISSN: 00071102     EISSN: None     Source Type: Journal    
DOI: 10.1348/000711005X64042     Document Type: Article

References (69)

1. Alexander, R. A., Scozzaro, M. J., & Borodkin, L. J. (1989). Statistical and empirical examination of the chi-square test for homogeneity of correlations in meta-analysis. Psychological Bulletin, 106, 329-331.
2. Berkey, C. S., Hoaglin, D. C., Mosteller, F., & Colditz, G. A. (1995). A random-effects regression model for meta-analysis. Statistics in Medicine, 14, 395-411.
3. Bond, C. F., Jr, Wiitala, W. L., & Richard, F. D. (2003). Meta-analysis of raw mean differences. Psychological Methods, 8, 406-418.
4. Brockwell, S. E., & Gordon, I. R. (2001). A comparison of statistical methods for meta-analysis. Statistics in Medicine, 20, 825-840.
5. Callender, J. C., & Osburn, H. E. (1988). Unbiased estimation of sampling variance of correlations. Journal of Applied Psychology, 73, 312-315.
6. Cochran, W. G. (1937). Problems arising in the analysis of a series of similar experiments. Supplement to the Journal of the Royal Statistical Society, 4, 102-118.
7. Cochran, W. G. (1954). The combination of estimates from different experiments. Biometrics, 10, 101-129.
8. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
9. Corbeil, R. R., & Searle, S. R. (1976a). A comparison of variance component estimation. Biometrics, 32, 779-791.
10. Corbeil, R. R., & Searle, S. R. (1976b). Restricted maximum likelihood (REML) estimation of variance components in the mixed model. Technometrics, 18, 31-38.
11. DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7, 177-188.
12. Erez, A., Bloom, M. C., & Wells, M. T. (1996). Using random rather than fixed effects models in meta-analysis: Implications for situational specificity and validity generalization. Personnel Psychology, 49, 275-306.
13. Field, A. P. (2001). Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixed-and random-effects methods. Psychological Methods, 6, 161-180.
14. Fisher, R. A. (1915). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika, 10, 507-521.
15. Fleiss, J. L. (1994). Measures of effect size for categorical data. In H. M. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 245-260). New York: Russell Sage Foundation.
16. Friedman, L. (2000). Estimators of random effects variance components in meta-analysis. Journal of Educational and Behavioral Statistics, 25, 1-12.
17. Hardy, R. J., & Thompson, S. G. (1996). A likelihood approach to meta-analysis with random effects. Statistics in Medicine, 15, 619-629.
18. Hardy, R. J., & Thompson, S. G. (1998). Detecting and describing heterogeneity in meta-analysis. Statistics in Medicine, 17, 841-856.
19. Hartley, H. O., & Rao, J. N. K. (1967). Maximum-likelihood estimation for the mixed analysis of variance model. Biometrika, 54, 93-108.
20. Harville, D. A. (1977). Maximum likelihood approaches to variance component estimation and to related problems. Journal of the American Statistical Association, 72, 320-338.
21. Harwell, M. (1997). An empirical study of Hedges' homogeneity test. Psychological Methods, 2, 219-231.
22. Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6, 107-128.
23. Hedges, L. V. (1982a). Fitting categorical models to effect size from a series of experiments. Journal of Educational Statistics, 7, 119-137.
24. Hedges, L. V (1982b). Estimation of effect size from a series of independent experiments. Psychological Bulletin, 92, 490-499.
25. Hedges, L. V. (1983). A random effects model for effect sizes. Psychological Bulletin, 93, 388-395.
26. Hedges, L. V (1989). An unbiased correction for sampling error in validity generalization studies. Journal of Applied Psychology, 74, 469-477.
27. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press.
28. Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychological Methods, 6, 203-217.
29. Hedges, L. V., & Vevea, J. L. (1998). Fixed-versus random-effects models in meta-analysis. Psychological Methods, 3, 486-504.
30. Hotelling, H. (1953). New light on the correlation coefficient and its transforms. Journal of the Royal Statistical Society, Series B, 15, 193-232.
31. Hunter, J. E., & Schmidt, E L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
32. Hunter, J. E., & Schmidt, E L. (1994). Estimation of sampling error variance in the meta-analysis of correlations: Use of average correlation in the homogeneous case. Journal of Applied Psychology, 79, 171-177.
33. Hunter, J. E., & Schmidt, E L. (2000). Fixed effects vs. random effects meta-analysis models: Implications for cumulative research knowledge. International Journal of Selection and Assessment, 8, 275-292.
34. Johnson, B. T., Mullen, B., & Salas, E. (1995). Comparison of three major meta-analytic approaches. Journal of Applied Psychology, 80, 94-106.
35. Koslowsky, M., & Sagie, A. (1993). On the efficacy of credibility intervals as indicators of moderator effects in meta-analytic research. Journal of Organizational Behavior, 14, 695-699.
36. Lehmann, E. L. (1999). Elements of large-sample theory. New York: Springer-Verlag.
37. Lipsey M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
38. Morrell, C. H. (1998). Likelihood ratio testing of variance components in the linear mixed-effects model using restricted maximum likelihood. Biometrics, 54, 1560-1568.
39. Morris, C. N. (1983). Parametric empirical Bayes inference: Theory and practice (with discussion). Journal of the American Statistical Association, 78, 47-65.
40. Morris, S. B. (2000). Distribution of the standardized mean change effect size for meta-analysis on repeated measures. British Journal of Mathematical and Statistical Psychology, 53, 17-29.
41. National Research Council (1992). Combining information: Statistical issues and opportunities for research. Washington, DC: National Academic Press.
42. Neyman, J., & Scott, E. L. (1948). Consistent estimates based on partially consistent observations. Econometrica, 16, 1-32.
43. Normand, S. T. (1999). Meta-analysis: Formulating, evaluating, combining, and reporting. Statistics in Medicine, 18, 321-359
44. Olkin, I., & Pratt, J. W. (1958). Unbiased estimation of certain correlation coefficients. Annals of Mathematical Statistics, 29, 201-211.
45. Patterson, H. D., & Thompson, R. (1971). Recovery of inter-block information when block sizes are unequal. Biometrika, 58, 545-554.
46. Patterson, H. D., & Thompson, R. (1974). Maximum likelihood estimation of components of variance. Proceedings of the 8th International Biometrics Conference, 197-207.
47. Rao, C. R. (1973). Linear statistical inference and its application. New York: McGraw-Hill.
48. Rasmussen, J. L., & Loher, B. T. (1988). Appropriate critical percentages for the Schmidt and Hunter meta-analysis procedure: Comparative evaluation of Type I error rate and power. Journal of Applied Psychology, 73, 683-687.
49. Rosenthal, R. (1991). Meta-analytic procedures for social research. Newbury Park, CA: Sage.
50. Rosenthal, R. (1994). Parametric measures of effect size. In H. M. Cooper & L. V Hedges (Eds.), The handbook of research synthesis (pp. 231-244). New York: Russell Sage Foundation.
51. Rosenthal, R., & DiMatteo, M. R. (2001). Meta-analysis: Recent developments in quantitative methods for literature reviews. Annual Review in Psychology, 52, 59-82.
52. Rosenthal, R., & Rubin, D. B. (1982). Comparing effect sizes of independent studies. Psychological Bulletin, 92, 500-504.
53. Sackett, P. R., Harris, M. M., & Orr, J. M. (1986). On seeking moderator variables in the meta-analysis of correlational data: A Monte Carlo investigation of statistical power and resistance to Type I error. Journal of Applied Psychology, 71, 302-310.
54. Sagie, A., & Koslowsky, M. (1993). Detecting moderators with meta-analysis: An evaluation and comparison of techniques. Personnel Psychology, 46, 629-640.
55. Sánchez-Meca, J., & Mann-Martinez, F. (1997). Homogeneity tests in meta-analysis: A Monte Carlo comparison of statistical power and Type I error. Quality & Quantity, 31, 385-399.
56. Schmidt, F. L., & Hunter, J. E. (1999). Comparison of three meta-analysis methods revisited: An analysis of Johnson, Mullen, and Salas (1995). Journal of Applied Psychology, 84, 144-148.
57. Searle, S. R., Casella, G., & McCulloch, C. E. (1992). Variance components. New York: Wiley.
58. Self, S. G., & Liang, K. Y. (1987). Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. Journal of the American Statistical Association, 82, 605-610.
59. Spector, P. E., & Levine, E. L. (1987). Meta-analysis for integrating study outcomes: A Monte Carlo study of its susceptibility to Type I and Type II errors. Journal of Applied Psychology, 72, 3-9.
60. Stram, D. O., & Lee, J. W. (1994). Variance components testing in the longitudinal mixed effects model. Biometrics, 50, 1171-1177.
61. Takkouche, B., Cadarso-Suárez, C., & Spiegelman, D. (1999). Evaluation of old and new tests of heterogeneity in epidemiologic meta-analysis. American Journal of Epidemiology, 150, 206-215.
62. Thompson, S. G., & Sharp, S.J. (1999). Explaining heterogeneity in meta-analysis: A comparison of methods. Statistics in Medicine, 18, 2693-2708.
63. van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002). Advanced methods in meta-analysis: Multivariate approach and meta-regression. Statistics in Medicine, 21, 589-624.
64. Verbeke, G., & Molenberghs, G (1997). Linear mixed models in practice: A SAS-oriented approach. New York: Springer-Verlag.
65. Verbeke, G., & Molenberghs, G. (2003). The use of score tests for inference on variance components. Biometrics, 59, 254-262.
66. Viana, M. A. G. (1980). Statistical methods for summarizing independent correlational results. Journal of Educational Statistics, 5, 83-104.
67. Viechtbauer, W. (2004). Choosing between the fixed-, random-, and mixed-effects model in meta-analysis: An analysis of existing and new model selection methods. Unpublished doctoral dissertation, University of Illinois, Urbana-Champaign.
68. Viechtbauer, W. (2005). Bias and efficiency of meta-analytic variance estimators in the random-effects model. Journal of Educational and Behavioral Statistics, 30, 261-293
69. Wang, M. C., & Bushman, B. J. (1999). Integrating results through meta-analytic review using SAS software. Cary, NC: SAS Institute.