Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis

Journal of Clinical Epidemiology

Volumn 86, Issue , 2017, Pages 39-50

  Mayo Wilson, Evan ^a   Fusco, Nicole ^a   Li, Tianjing ^a   Dickersin, Kay ^a   Hong, Hwanhee ^a   Canner, Joseph K ^b  

^a JOHNS HOPKINS BLOOMBERG SCHOOL OF PUBLIC HEALTH   (United States)

^b JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

Clinical trials; Meta analysis; Outcomes; Selective outcome reporting; Systematic reviews

Indexed keywords

GABAPENTIN; QUETIAPINE; 4 AMINOBUTYRIC ACID; AMINE; ANALGESIC AGENT; CYCLOHEXANECARBOXYLIC ACID DERIVATIVE; NEUROLEPTIC AGENT;

ANALGESIA; BIPOLAR DEPRESSION; CLINICAL OUTCOME; CLINICAL RESEARCH; HUMAN; MCGILL PAIN QUESTIONNAIRE; META ANALYSIS; NEUROPATHIC PAIN; OUTCOME ASSESSMENT; PAIN INTENSITY; PRIORITY JOURNAL; PSYCHOPHARMACOTHERAPY; RANDOMIZED CONTROLLED TRIAL (TOPIC); REVIEW; SYSTEMATIC REVIEW; BIPOLAR DISORDER; NEURALGIA; STATISTICS AND NUMERICAL DATA; TREATMENT OUTCOME;

AMINES; ANALGESICS; ANTIPSYCHOTIC AGENTS; BIPOLAR DISORDER; CYCLOHEXANECARBOXYLIC ACIDS; GAMMA-AMINOBUTYRIC ACID; HUMANS; NEURALGIA; OUTCOME ASSESSMENT (HEALTH CARE); QUETIAPINE FUMARATE; RANDOMIZED CONTROLLED TRIALS AS TOPIC; TREATMENT OUTCOME;

EID: 85020419684     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2017.05.007     Document Type: Review

References (56)

1. Zarin, D.A., Tse, T., Williams, R.J., Califf, R.M., Ide, N.C., The ClinicalTrials.gov results database–update and key issues. N Engl J Med 364 (2011), 852–860.
2. Davey, J., Turner, R.M., Clarke, M.J., Higgins, J.P., Characteristics of meta-analyses and their component studies in the Cochrane Database of Systematic Reviews: a cross-sectional, descriptive analysis. BMC Med Res Methodol, 11, 2011, 160.
3. Kirkham, J.J., Dwan, K.M., Altman, D.G., Gamble, C., Dodd, S., Smyth, R., et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ, 340, 2010, c365.
4. Dwan, K., Altman, D.G., Cresswell, L., Blundell, M., Gamble, C.L., Williamson, P.R., Comparison of protocols and registry entries to published reports for randomised controlled trials. Cochrane Database Syst Rev, 2011, MR000031.
5. Furukawa, T.A., Watanabe, N., Omori, I.M., Montori, V.M., Guyatt, G.H., Association between unreported outcomes and effect size estimates in Cochrane meta-analyses. Jama 297 (2007), 468–470.
6. Moayyedi, P., Deeks, J., Talley, N.J., Delaney, B., Forman, D., An update of the Cochrane systematic review of Helicobacter pylori eradication therapy in nonulcer dyspepsia: resolving the discrepancy between systematic reviews. Am J Gastroenterol 98:12 (2003), 2621–2626.
7. Tendal, B., Nuesch, E., Higgins, J.P., Juni, P., Gotzsche, P.C., Multiplicity of data in trial reports and the reliability of meta-analyses: empirical study. Bmj, 343, 2011, d4829.
8. Schulz, K.F., Grimes, D.A., Multiplicity in randomised trials I: endpoints and treatments. Lancet 365 (2005), 1591–1595.
9. Gyawali, B., Prasad, V., Same data; different interpretations. J Clin Oncol, 2016, 10.1200/JCO.2016.68.2021.
10. Wood, A.M., White, I.R., Thompson, S.G., Are missing outcome data adequately handled? A review of published randomized controlled trials in major medical journals. Clin Trials 1 (2004), 368–376.
11. Chan, A.W., Hrobjartsson, A., Jorgensen, K.J., Gotzsche, P.C., Altman, D.G., Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols. Bmj, 337, 2008, a2299.
12. Vedula, S.S., Li, T., Dickersin, K., Differences in reporting of analyses in internal company documents versus published trial reports: comparisons in industry-sponsored trials in off-label uses of gabapentin. Plos Med, 10(1), 2013, e1001378.
13. Page, M.J., McKenzie, J.E., Chau, M., Green, S.E., Forbes, A., Methods to select results to include in meta-analyses deserve more consideration in systematic reviews. J Clin Epidemiol 68 (2015), 1282–1291.
14. Dwan, K., Altman, D.G., Clarke, M., Gamble, C., Higgins, J.P., Sterne, J.A., et al. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. Plos Med, 11(6), 2014, e1001666.
15. Dwan, K., Gamble, C., Williamson, P.R., Kirkham, J.J., Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS One, 8, 2013, e66844.
16. Chan, A.W., Hrobjartsson, A., Haahr, M.T., Gotzsche, P.C., Altman, D.G., Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 291 (2004), 2457–2465.
17. Chalmers, T.C., Frank, C.S., Reitman, D., Minimizing the three stages of publication bias. JAMA 263 (1990), 1392–1395.
18. Page, M.J., McKenzie, J.E., Forbes, A., Many scenarios exist for selective inclusion and reporting of results in randomized trials and systematic reviews. J Clin Epidemiol 66 (2013), 524–537.
19. Page, M.J., McKenzie, J.E., Kirkham, J., Dwan, K., Kramer, S., Green, S., et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev, 2014, MR000035.
20. Mayo-Wilson, E., Hutfless, S., Li, T., Gresham, G., Fusco, N., Ehmsen, J., et al. Integrating multiple data sources (MUDS) for meta-analysis to improve patient-centered outcomes research: a protocol for a systematic review. Syst Rev, 4, 2015, 143.
21. Mayo-Wilson E, Li T, Fusco N, Bertizzolo L, Canner J, Cowley T et al. Disagreements across multiple reports of clinical trials: implications for interpretation of trial quality and results. Under Rev.
22. Meinert, C.L., Clinical Trials Dictionary: Terminology and Usage Recommendations. 2nd ed., 2012, John Wiley & Sons, Hoboken, NJ.
23. Li, T., Vedula, S.S., Hadar, N., Parkin, C., Lau, J., Dickersin, K., Innovations in data collection, management, and archiving for systematic reviews. Ann Intern Med 162 (2015), 287–294.
24. Saldanha, I.J., Dickersin, K., Wang, X., Li, T., Outcomes in Cochrane systematic reviews addressing four common eye conditions: an evaluation of completeness and comparability. PLoS One, 9, 2014, e109400.
25. Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ, 349, 2015, g7647.
26. Schulz, K.F., Altman, D.G., Moher, D., Group, C., CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. Plos Med, 7(3), 2010, e1000251.
27. Chan, A.W., Tetzlaff, J.M., Altman, D.G., Dickersin, K., Moher, D., SPIRIT 2013: new guidance for content of clinical trial protocols. Lancet 381 (2013), 91–92.
28. Chan, A.W., Tetzlaff, J.M., Altman, D.G., Laupacis, A., Gotzsche, P.C., Krleza-Jeric, K., et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med 158 (2013), 200–207.
29. Patel, C.J., Burford, B., Ioannidis, J.P., Assessment of vibration of effects due to model specification can demonstrate the instability of observational associations. J Clin Epidemiol 68 (2015), 1046–1058.
30. Backonja, M., Gabapentin for painful diabetic neuropathy (in reply). JAMA 282 (1999), 133–134.
31. Kendall, T., Morriss, R., Mayo-Wilson, E., Marcus, E., Guideline development group of the National Institute for Health and Care Excellence. assessment and management of bipolar disorder: summary of updated NICE guidance. BMJ, 349, 2014, g5673.
32. Dworkin, R.H., O'Connor, A.B., Audette, J., Baron, R., Gourlay, G.K., Haanpaa, M.L., et al. Recommendations for the pharmacological management of neuropathic pain: an overview and literature update. Mayo Clin Proc 85:3 Suppl (2010), S3–S14.
33. Chan, A.W., Song, F., Vickers, A., Jefferson, T., Dickersin, K., Gotzsche, P.C., et al. Increasing value and reducing waste: addressing inaccessible research. Lancet 383 (2014), 257–266.
34. Glasziou, P., Altman, D.G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet 383 (2014), 267–276.
35. Goodman, S., Fanelli, D., Ioannidis, J.P., What does research reproducibility mean?. Sci Trans Med, 8(341), 2016, 341ps12.
36. Kaufman, J., Ryan, R., Bosch-Capblanch, X., Cartier, Y., Cliff, J., Glenton, C., et al. Outcomes mapping study for childhood vaccination communication: too few concepts were measured in too many ways. J Clin Epidemiol 72 (2016), 33–44.
37. Williamson, P., Clarke, M., The COMET (Core outcome measures in effectiveness trials) initiative: its Role in improving Cochrane reviews. Cochrane Database Syst Rev, 2012, ED000041.
38. Page, M.J., Forbes, A., Chau, M., Green, S.E., McKenzie, J.E., Investigation of bias in meta-analyses due to selective inclusion of trial effect estimates: empirical study. BMJ Open, 6(4), 2016, e011863.
39. Saldanha, I.J., Li, T., Yang, C., Owczarzak, J., Williamson, P.R., Dickersin, K., Clinical trials and systematic reviews addressing similar interventions for the same condition do not consider similar outcomes to be important: a case study in HIV/AIDS. J Clin Epidemiol 84 (2017), 85–94.
40. Altman, D.G., Missing outcomes in randomized trials: addressing the dilemma. Open Med 3:2 (2009), e51–e53.
41. Li, T., Hutfless, S., Scharfstein, D., Daniels, M., Hogan, J., Little, R., et al. Standards should be applied in the prevention and handling of missing data for patient-centered outcomes research: a systematic review and expert consensus. J Clin Epidemiol 67 (2014), 15–32.
42. Mathieu, S., Boutron, I., Moher, D., Altman, D.G., Ravaud, P., Comparison of registered and published primary outcomes in randomized controlled trials. JAMA 302 (2009), 977–984.
43. Perlmutter, A., Tran, V.T., Dechartres, A., Ravaud, P., Comparison of primary outcomes in protocols, public clinical-trial registries and publications: the example of oncology trials. Ann Oncol 28 (2017), 688–695.
44. Tendal, B., Higgins, J.P., Juni, P., Hrobjartsson, A., Trelle, S., Nuesch, E., et al. Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study. BMJ, 339, 2009, b3128.
45. Taichman, D.B., Backus, J., Baethge, C., Bauchner, H., de Leeuw, P.W., Drazen, J.M., et al. Sharing clinical trial data: a proposal from the international committee of medical journal editors. PLoS Med, 13(1), 2016, e1001950.
46. Moorthy, V.S., Karam, G., Vannice, K.S., Kieny, M.P., Rationale for WHO's new position calling for prompt reporting and public disclosure of interventional clinical trial results. PLoS Med, 12(4), 2015, e1001819.
47. Doshi, P., Dickersin, K., Healy, D., Vedula, S.S., Jefferson, T., Restoring invisible and abandoned trials: a call for people to publish the findings. BMJ, 346, 2013, f2865.
48. Nosek, B.A., Alter, G., Banks, G.C., Borsboom, D., Bowman, S.D., Breckler, S.J., et al. Scientific standards. Promoting an open research culture. Science 348 (2015), 1422–1425.
49. Weber, W.E., Merino, J.G., Loder, E., Trial registration 10 years on. BMJ, 351, 2015, h3572.
50. Zarin, D.A., Tse, T., Williams, R.J., Carr, S., Trial reporting in ClinicalTrials.gov—the final rule. N Engl J Med 375 (2016), 1998–2004.
51. Hudson, K.L., Lauer, M.S., Collins, F.S., Toward a new era of trust and transparency in clinical trials. JAMA 316 (2016), 1353–1354.
52. Simera, I., Altman, D.G., Moher, D., Schulz, K.F., Hoey, J., Guidelines for reporting health research: the EQUATOR network's survey of guideline authors. PLoS Med, 5(6), 2008, e139.
53. Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev, 4, 2015, 1.
54. Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Plos Med, 6(7), 2009, e1000097.
55. Stewart, L.A., Clarke, M., Rovers, M., Riley, R.D., Simmonds, M., Stewart, G., et al. Preferred reporting items for systematic review and meta-analyses of individual participant data: the PRISMA-IPD Statement. JAMA 313 (2015), 1657–1665.
56. Doshi, P., Jefferson, T., Del Mar, C., The imperative to share clinical study reports: recommendations from the Tamiflu experience. PLoS Med, 9(4), 2012, e1001201.