Differentiating biases from genuine heterogeneity: Distinguishing artifactual from substantive effects

Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments

Volumn , Issue , 2006, Pages 287-302

  Ioannidis, John P A ^a,b  

^a UNIVERSITY OF IOANNINA   (Greece)

^b TUFTS MEDICAL CENTER   (United States)

Author keywords

Biases and genuine heterogeneity; Control rate effects; Control rate meta regressions; Publication bias versus control rate effects; Publication bias versus null bias; Study design effects; Time dependent effects; Time lag bias

Indexed keywords

EID: 84889319891     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/0470870168.ch15     Document Type: Chapter

References (52)

1. Balk, E.M., Bonis, P.A., Moskowitz, H., Schmid, C.H., Ioannidis, J.P., Wang, C. and Lau, J. (2002). Correlation of quality measures with estimates of treatment effect in metaanalyses of randomized controlled trials. Journal of the American Medical Association, 287, 2973-2982.
2. Breathnach, O.S., Freidlin, B., Conley, B., Green, M.R., Johnson, D.H., Gandara, D.R., O’Connell, M., Shepherd, F.A. and Johnson, B.E. (2001). Twenty-two years of phase III trials for patients with advanced non-small-cell lung cancer: Sobering results. Journal of Clinical Oncolology, 19, 1734-1742.
3. Cappelleri, J.C., Ioannidis, J.P., Schmid, C.H., de Ferranti, S.D., Aubert, M., Chalmers, T.C. and Lau, J. (1996). Large trials vs meta-analysis of smaller trials: How do their results compare? Journal of the American Medical Association, 276, 1332-1338.
4. Clarke, M. and Stewart, L. (1998). Time lag bias in publishing clinical trials. Journal of the American Medical Association, 279, 1952.
5. DECODE Study Group, on behalf of the European Diabetes Epidemiology Study Group (1998). Will new diagnostic criteria for diabetes mellitus change phenotype of patients with diabetes? Reanalysis of European epidemiological data. British Medical Journal, 317, 371-375.
6. De Gruttola, V., Fleming, T., Lin, D.Y. and Coombs, R. (1997). Perspective: Validating surrogate markers -are we being naive? Journal of Infectious Diseases, 175, 237-246.
7. Dickersin, K., Olson, C.M., Rennie, D., Cook, D., Flanagin, A., Zhu, Q., Reiling, J. and Pace, B. (2002). Association between time interval to publication and statistical significance. Journal of the American Medical Association, 287, 2829-2831.
8. Fleming, T.R. and DeMets, D.L. (1996). Surrogate end points in clinical trials: Are we being misled? Annals of Internal Medicine, 125, 605-613.
9. Glasziou, P.P. and Irwig, L.M. (1995). An evidence based approach to individualising treatment. British Medical Journal, 311, 1356-1359.
10. Goring, H.H., Terwilliger, J.D. and Blangero, J. (2001). Large upward bias in estimation of locus-specific effects from genomewide scans. American Journal of Human Genetics, 69, 1357-1369.
11. Haidich, A.B. and Ioannidis, J.P. (2001). Effect of early patient enrollment on the time to completion and publication of randomized controlled trials. Amercian Journal of Epidemiology, 154, 873-880.
12. HIV Trialists’ Collaborative Group (1999). Meta-analyses of the randomised evidence on zidovudine, didanosine and zalcitabine in the treatment of HIV infection. Lancet, 353, 2014-2025.
13. Hughes, M.D. and Pocock, S.J. (1988). Stopping rules and estimation problems in clinical trials. Statistics in Medicine, 7, 1231-1242.
14. Huwiler-Muntener, K., Jüni, P., Junker, C. and Egger, M. (2002). Quality of reporting of randomized trials as a measure of methodologic quality. Journal of the American Medical Association, 287, 2801-2804.
15. Ioannidis, J.P. (1998). Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. Journal of the American Medical Association, 279, 281-286.
16. Ioannidis, J.P. and Lau, J. (1997). The impact of high-risk patients on the results of clinical trials. Journal of Clinical Epidemiology, 50, 1089-1098.
17. Ioannidis, J.P. and Lau J. (1998a). Can quality of clinical trials and meta-analyses be quantified? Lancet, 352, 590-591.
18. Ioannidis, J.P. and Lau, J. (1998b). Heterogeneity of the baseline risk within patient populations of clinical trials: A proposed evaluation algorithm. American Journal Epidemiology, 148, 1117-1126.
19. Ioannidis, J.P. and Lau, J. (2001). Evolution of treatment effects over time: Empirical insight from recursive cumulative meta-analyses. Proceedings of the National Academy of Sciences of the United States, 98, 831-836.
20. Ioannidis, J.P., Cappelleri, J.C., Lau, J., Skolnik, P.R., Melville, B., Chalmers, T.C. and Sacks, H.S. (1995). Early or deferred zidovudine therapy in HIV-infected patients without an AIDS-defining illness. Annals of Internal Medicine, 122, 856-866.
21. Ioannidis, J.P., Cappelleri, J.C., Schmid, C.H. and Lau, J. (1996). Impact of epidemic and individual heterogeneity on the population distribution of disease progression rates. An example from patient populations in trials of human immunodeficiency virus infection. American Journal of Epidemiology, 144, 1074-1085.
22. Ioannidis, J.P., Cappelleri, J.C., Sacks, H.S. and Lau, J. (1997). The relationship between study design, results, and reporting of randomized clinical trials of HIV infection. Controlled Clinical Trials, 18, 431-444.
23. Ioannidis, J.P., Cappelleri, J.C. and Lau, J. (1998). Issues in comparisons between meta-analyses and large trials. Journal of the American Medical Association, 279, 1089-1093.
24. Ioannidis, J.P., Dixon, D.O., McIntosh, M., Albert, J.M., Bozzette, S.A. and Schnittman, S.M. (1999). Relationship between event rates and treatment effects in clinical site differences within multicenter trials: An example from primary Pneumocystis carinii prophylaxis. Controlled Clinical Trials, 20, 253-266.
25. Ioannidis, J.P., Ntzani, E.E., Trikalinos, T.A. and Contopoulos-Ioannidis, D.G. (2001). Replication validity of genetic association studies. Nature Genetics, 29, 306-309.
26. Ioannidis, J.P., Rosenberg, P.S., Goedert, J.J. and O’Brien, T.R. (2002). Commentary: Meta-analysis of individual participants’ data in genetic epidemiology. American Journal of Epidemiology, 156, 204-210.
27. Ioannidis, J.P., Polycarpou, A., Ntais, C., and Pavlidis, N. (2003). Randomized trials comparing chemotherapy regimens for advanced non-small cell lung cancer: Biases and evolution over time. European Journal of Cancer, 39, 2278-2287.
28. Jüni, P., Altman, D.G. and Egger, M. (2001). Systematic reviews in health care: Assessing the quality of controlled clinical trials. British Medical Journal, 323, 42-46.
29. Kanfer, R. and Ackerman, P.L. (1989). Motivation and cognitive abilities -an integrative aptitude treatment interaction approach to skill acquisition. Journal of Applied Psychology, 74, 657-690.
30. Kjaergard, L.L., Villumsen, J. and Gluud, C. (2001). Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Annals of Internal Medicine, 135, 982-989.
31. Lan, K.K., Rosenberger, W.F. and Lachin, J.M. (1993). Use of spending functions for occasional or continuous monitoring of data in clinical trials. Statistics in Medicine, 12, 2219-2231.
32. LeLorier, J., Gregoire, G., Benhaddad, A., Lapierre, J. and Derderian, F. (1997). Discrepancies between meta-analyses and subsequent large randomized, controlled trials. New England Journal of Medicine, 337, 536-542.
33. Lohmueller, K.E., Pearce, C.L., Pike, M., Lander, E.S. and Hirschhorn, J.N. (2003). Meta-analysis of genetic association studies supports a contribution of common variants to susceptibility to common disease. Nature Genetics, 33, 177-182.
34. McIntosh, M.W. (1996). The population risk as an explanatory variable in research synthesis of clinical trials. Statistics in Medicine, 15, 1713-1728.
35. Moher, D., Jadad, A.R., Nichol, G., Penman, M., Tugwell, P. and Walsh, S. (1995). Assessing the quality of randomized controlled trials: An annotated bibliography of scales and checklists. Controlled Clinical Trials, 16, 62-73.
36. Moher, D., Pham, B., Jones, A., Cook, D.J., Jadad, A.R., Moher, M., Tugwell, P. and Klassen, T.P. (1998). Does quality of reports of randomized trials affect estimates of intervention efficacy reported in meta-analyses? Lancet, 352, 609-613.
37. Mulherin, S.A., O’Brien, T.R., Ioannidis, J.P., Goedert, J.J., Buchbinder, S.P., Coutinho, R.A. et al. (2003). International Meta-Analysis of HIV Host Genetics. Effects of CCR5-Delta32 and CCR2-64I alleles on HIV-1 disease progression: The protection varies with duration of infection. AIDS, 17, 377-387.
38. Oxman, A.D. and Guyatt, G.H. (1992). A consumer’s guide to subgroup analyses. Annals of Internal Medicine, 116, 78-84.
39. Rosenberg, P.S. (1995). Hazard function estimation using B-splines. Biometrics, 51, 874-887.
40. Schmid, C.H., Lau, J., McIntosh, M.W. and Cappelleri, J.C. (1998). An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials. Statistics in Medicine, 17, 1923-1942.
41. Schulz, K.F., Chalmers, I., Hayes, R.J. and Altman, D.G. (1995). Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Journal of the American Medical Association, 273, 408-412.
42. Sharp, S.J. and Thompson, S.G. (2000). Analyzing the relationship between treatment effect and underlying risk in meta-analysis: Comparison and development of approaches. Statistics in Medicine, 19, 3251-3274.
43. Smith, M.L. and Glass, G.V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychology, 32, 752-760.
44. Stern, J.M. and Simes, R.J. (1997). Publication bias: Evidence of delayed publication in a cohort study of clinical research projects. British Medical Journal, 315, 640-645.
45. Sterne, J.A., Jüni, P., Schulz, K.F., Altman, D.G., Bartlett, C. and Egger, M. (2002). Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Statistics in Medicine, 21, 1513-1524.
46. Stewart, L.A. and Clarke, M.J. (1995). Practical methodology of meta-analyses (overviews) using updated individual patient data. Cochrane Working Group. Statistics in Medicine, 14, 2057-2079.
47. Stewart, L.A. and Parmar, M.K. (1993). Meta-analysis of the literature or of individual patient data: Is there a difference? Lancet, 341, 418-422.
48. Thompson, S.G. and Higgins, J.P. (2002). How should meta-regression analyses be undertaken and interpreted? Statistics in Medicine, 21, 1559-1573.
49. Trikalinos, T.A. and Ioannidis, J.P. (2001). Predictive modeling and heterogeneity of baseline risk in meta-analysis of individual patient data. Journal of Clinical Epidemiology, 54, 245-252.
50. Villar, J., Carroli, G. and Belizan, J.M. (1995). Predictive ability of meta-analyses of randomized controlled trials. Lancet, 345, 772-776.
51. Woods, K.L. (1995). Mega-trials and management of acute myocardial infarction. Lancet, 346, 611-614.
52. Yusuf, S., Collins, R. and Peto, R. (1984). Why do we need some large, simple randomized trials? Statistics in Medicine, 3, 409-422.