The Relative Benefits of Meta-Analysis Conducted With Individual Participant Data Versus Aggregated Data

Psychological Methods

Volumn 14, Issue 2, 2009, Pages 165-176

  Cooper, Harris ^a   Patall, Erika A ^a  

^a DUKE UNIVERSITY   (United States)

Author keywords

aggregated data; ecological fallacy; individual participant data; meta analysis; Simpson's paradox

Indexed keywords

ARTICLE; HUMAN; INFORMATION PROCESSING; META ANALYSIS; METHODOLOGY; STATISTICAL ANALYSIS; STATISTICAL MODEL;

DATA COLLECTION; DATA INTERPRETATION, STATISTICAL; HUMANS; META-ANALYSIS AS TOPIC; MODELS, STATISTICAL;

EID: 66349113990     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0015565     Document Type: Article

References (45)

1. Bauer D.J., and Hussong A.M. Psychometric approaches for developing commensurate measures across independent studies: Traditional and new models. Psychological Methods 14 (2009) 101-125
2. Berlin J.A., Santanna J., Schmid C.H., Szczech L.A., and Feldman H.I. Individual patient- versus group-level data meta-regression for the investigation of treatment effect modifiers: Ecological bias rears its ugly head. Statistics in Medicine 21 (2002) 371-387
3. Bickel P.J., Hammel E.A., and O'Connell J.W. Sex bias in graduate admissions: Data from Berkeley. Science 187 (1975) 398-404
4. Borenstein M. Effect sizes for studies with continuous outcome data. In: Cooper H., Hedges L.V., and Valentine J.C. (Eds). The handbook of research synthesis. 2nd ed. (2009), Russell Sage Foundation, New York 221-236
5. Cooper H. Editorial. Psychological Bulletin 129 (2003) 3-9
6. Cooper H.M. Research synthesis and meta-analysis: A step-by-step approach. 4th ed. (2009), Sage, Thousand Oaks, CA
7. Glass G.V. Primary, secondary, and meta-analysis. Educational Researcher 5 (1976) 3-8
8. Glass, G. V. (2000, March). The future of meta-analysis. Paper presented at the University of California, Berkeley-Stanford University Colloquium on Meta-Analysis, Department of Psychology, University of California, Berkeley.
9. Glass G.V., and Smith M.L. Meta-analysis of research on the relationship of class size and achievement (1978), Far West Laboratory for Educational Research and Development, San Francisco
10. Goldstein H., Yang M., Omar R., Turner R., and Thompson S. Meta-analysis using multilevel models with an application to the study of class size effects. Applied Statistics 49 (2000) 399-412
11. Greenwald A. Consequences of prejudice against the null hypothesis. Psychological Bulletin 82 (1975) 1-20
12. Hedges L.V., and Vevea J.L. Fixed and random effects models in meta-analysis. Psychological Methods 3 (1998) 486-504
13. Higgins J.P.T., Whitehead A., Turner R.M., Omar R.Z., and Thompson S.G. Meta-analysis of continuous outcome data from individual patients. Statistics in Medicine 20 (2001) 2219-2241
14. Jeng G.T., Scott J.R., and Burmeister L.F. A comparison of meta-analytic results using literature vs. individual patient data: Paternal cell immunization for recurrent miscarriage. Journal of the American Medical Association 274 (1995) 830-836
15. Lambert P.C., Sutton A.J., Abrams K.R., and Jones D.R. A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. Journal of Clinical Epidemiology 55 (2002) 86-94
16. Lipsey M.W., and Wilson D.B. The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist 48 (1993) 1181-1209
17. Mathew T., and Nordström K. On the equivalence of meta-analysis using literature and using individual patient data. Biometrics 55 (1999) 1221-1223
18. National Institutes of Health. Final NIH statement on sharing research data (2003), Author, Rockville, MD. http://grants.nih.gov/grants/guide/notice-files/not-od-03-032.html Retrieved December 10, 2007 from http://grants.nih.gov/grants/guide/notice-files/not-od-03-032.html
19. Olkin I. History and goals. In: Wachter K.W., and Straf M.L. (Eds). The future of meta-analysis (1990), Russell Sage Foundation, New York 3-10
20. Olkin I., and Sampson A. Comparison of meta-analysis versus analysis of variance of individual patient data. Biometrics 54 (1998) 317-322
21. Overton R.C. A comparison of fixed-effects and mixed (random-effects) models for meta-analysis tests of moderator variable effects. Psychological Methods 3 (1998) 354-379
22. Pearson K. Report on certain enteric fever inoculation statistics. British Medical Journal 3 (1904) 1243-1246
23. Riley R.D., Simmonds M.C., and Look M.P. Evidence synthesis combining individual patient data and aggregate data: A systematic review identified current practice and possible methods. Journal of Clinical Epidemiology 60 (2007) 431-439
24. Robinson W.S. Ecological correlations and the behavior of individuals. American Sociological Review 15 (1950) 351-357
25. Rosenthal R., and Rubin D.B. Interpersonal expectancy effects: The first 345 studies. The Behavioral and Brain Sciences 3 (1978) 377-386
26. Rothstein H.R., and Hopewell S. The grey literature. In: Cooper H., Hedges L.V., and Valentine J.C. (Eds). The handbook of research synthesis. 2nd ed. (2009), Russell Sage Foundation, New York 103-125
27. Rothstein H.R., Sutton A.J., and Borenstein M. Publication bias in meta-analysis: Prevention, assessment and adjustments (2005), Wiley, Chichester, United Kingdom
28. Schmid C.H., Stark P.C., Berlin J.A., Landais P., and Lau J. Meta-regression detected associations between heterogeneous treatment effects and study-level, but not patient-level, factors. Journal of Clinical Epidemiology 57 (2004) 683-697
29. Schmidt F.L., and Hunter J.E. Development of a general solution to the problem of validity generalization. Journal of Applied Psychology 62 (1977) 529-540
30. Shadish W.R., and Haddock C.K. Combining estimates of effect sizes. In: Cooper H., Hedges L.V., and Valentine J.C. (Eds). The handbook of research synthesis. 2nd ed. (2009), Russell Sage Foundation, New York 257-278
31. Shadish W.R., and Rindskopf D.M. Methods for evidence-based practice: Quantitative synthesis of single-subject designs. New Directions for Evaluation 113 (2007) 95-109
32. Sieber J.E. Sharing social science data: Advantages and challenges (1991), Sage, Newbury Park, CA
33. Simmonds M.C., and Higgins J.P.T. Covariate heterogeneity in meta-analysis: Criteria for deciding between meta-regression and individual patient data. Statistics in Medicine 26 (2007) 2982-2999
34. Simmonds M.C., Higgins J.P.T., Stewart L.A., Tierney J.F., Clarke M.J., and Thompson S.G. Meta-analysis of individual patient data from randomized trials: A review of methods used in practice. Clinical Trials 2 (2005) 209-217
35. Simpson E.H. The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society, Series B 13 (1951) 238-241
36. Smith M.L., and Glass G.V. Meta-analysis of psychotherapy outcome studies. American Psychologist 32 (1977) 752-760
37. Steinberg K.K., Smith S.J., Stroup D.F., Olkin I., Lee N.C., Williamson G.D., and Thacker S.B. Comparison of effect estimates from a meta-analysis of summary data from published studies and from meta-analysis using individual patient data for ovarian cancer studies. American Journal of Epidemiology 145 (1997) 917-925
38. Stewart L.A., and Clarke M.J. Practical methodology of meta-analyses (overviews) using updated individual patient data. Statistics in Medicine 14 (1995) 2057-2079
39. Stewart L.A., and Parmar M.K.B. Meta-analysis of the literature or of individual patient data: Is there a difference?. Lancet 341 (1993) 418-422
40. Stewart L.A., and Tierney J.F. To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data. Evaluations of Health Professions 25 (2002) 76-97
41. Sutton A.J., Kendrick D., and Coupland C.A.C. Meta-analysis of individual- and aggregate-level data. Statistics in Medicine 27 (2008) 651-669
42. Turner R.M., Omar R.Z., Yang M., Goldstein H., and Thompson S.G. A multilevel model framework for meta-analysis of clinical trials with binary outcomes. Statistics in Medicine 19 (2000) 3417-3432
43. Valentine, J. C., Pigott, T. & Rothstein, H. R. (in press). How many studies do you need? A primer on statistical power for meta-analysis. Journal of Educational and Behavioral Statistics
44. Whitehead A., Omar R.Z., Higgins J.P.T., Savaluny E., Turner R.M., and Thompson S.G. Meta-analysis of ordinal outcomes using individual patient data. Statistics in Medicine 20 (2001) 2243-2260
45. Wicherts J.M., Borsboom D., Kats J., and Molenaar D. The poor availability of psychological research data for reanalysis. American Psychologist 61 (2006) 726-728