Why clinical trial outcomes fail to translate into benefits for patients

Trials

Volumn 18, Issue 1, 2017, Pages

  Heneghan, Carl ^a   Goldacre, Ben ^a   Mahtani, Kamal R ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Clinical outcomes; Composite outcomes; Core outcome sets; Publication bias; Reporting bias; Surrogate outcomes

Indexed keywords

ANTIDEPRESSANT AGENT; GLYCOSYLATED HEMOGLOBIN; PROSTATE SPECIFIC ANTIGEN;

ADVERSE OUTCOME; BONE DENSITY; CAROTID ENDARTERECTOMY; CEREBROVASCULAR ACCIDENT; CLINICAL RESEARCH; CLINICAL TRIAL (TOPIC); DECISION MAKING; DEPRESSION; EVIDENCE BASED MEDICINE; FALSE POSITIVE RESULT; GENERAL PRACTITIONER; HUMAN; NOTE; PATIENT CARE; POSTMARKETING SURVEILLANCE; PRACTICE GUIDELINE; PUBLICATION; RHEUMATOID ARTHRITIS; RISK FACTOR; SELF REPORT; BIOASSAY; METHODOLOGY; PATIENT PARTICIPATION; PROCEDURES; PUBLISHING; STATISTICAL ANALYSIS; STATISTICAL BIAS; STATISTICS AND NUMERICAL DATA; TREATMENT OUTCOME;

BIAS; CLINICAL TRIALS AS TOPIC; DATA INTERPRETATION, STATISTICAL; ENDPOINT DETERMINATION; HUMANS; PATIENT PARTICIPATION; PUBLICATION BIAS; RESEARCH DESIGN; TREATMENT OUTCOME;

EID: 85014930406     PISSN: None     EISSN: 17456215     Source Type: Journal    
DOI: 10.1186/s13063-017-1870-2     Document Type: Note

References (70)

1. Rupp T, Zuckerman D. Quality of life, overall survival, and costs of cancer drugs approved based on surrogate endpoints. JAMA Intern Med. 2017;177:276-7. doi: 10.1001/jamainternmed.2016.7761.
2. World Health Organisation (WHO). WHO handbook for guideline developers. Geneva: WHO; 2012. http://apps.who.int/iris/bitstream/10665/75146/1/9789241548441_eng.pdf?ua=1. Accessed 11 Mar 2017.
3. Tudur Smith C, Hickey H, Clarke M, Blazeby J, Williamson P. The trials methodological research agenda: results from a priority setting exercise. Trials. 2014;15:32. doi: 10.1186/1745-6215-15-32.
4. Twaddell S. Surrogate outcome markers in research and clinical practice. Aust Prescr. 2009;32:47-50. doi: 10.18773/austprescr.2009.023.
5. Yudkin JS, Lipska KJ, Montori VM. The idolatry of the surrogate. BMJ. 2011;343:d7995. doi: 10.1136/bmj.d7995.
6. la Cour JL, Brok J, Gøtzsche PC. Inconsistent reporting of surrogate outcomes in randomised clinical trials: cohort study. BMJ. 2010;341:c3653. doi: 10.1136/bmj.c3653.
7. Atkins D, Briss PA, Eccles M, Flottorp S, Guyatt GH, Harbour RT, et al. Systems for grading the quality of evidence and the strength of recommendations II: pilot study of a new system. BMC Health Serv Res. 2005;5:25. doi: 10.1186/1472-6963-5-25.
8. D'Agostino Jr RB. Debate: the slippery slope of surrogate outcomes. Curr Control Trials Cardiovasc Med. 2000;1:76-8. doi: 10.1186/cvm-1-2-076.
9. DREAM (Diabetes REduction Assessment with ramipril and rosiglitazone Medication) Trial Investigators. Effect of rosiglitazone on the frequency of diabetes in patients with impaired glucose tolerance or impaired fasting glucose: a randomised controlled trial. Lancet. 2006;368:1096-105. doi: 10.1016/S0140-6736(06)69420-8. A published erratum appears in Lancet. 2006;368:1770.
10. Cohen D. Rosiglitazone: what went wrong? BMJ. 2010;341:c4848. doi: 10.1136/bmj.c4848.
11. Ferreira-González I, Busse JW, Heels-Ansdell D, Montori VM, Akl EA, Bryant DM, et al. Problems with use of composite end points in cardiovascular trials: systematic review of randomised controlled trials. BMJ. 2007;334:786. doi: 10.1136/bmj.39136.682083.AE.
12. Freemantle N, Calvert MJ. Interpreting composite outcomes in trials. BMJ. 2010;341:c3529. doi: 10.1136/bmj.c3529.
13. Cordoba G, Schwartz L, Woloshin S, Bae H, Gøtzsche PC. Definition, reporting, and interpretation of composite outcomes in clinical trials: systematic review. BMJ. 2010;341:c3920. doi: 10.1136/bmj.c3920.
14. Lim E, Brown A, Helmy A, Mussa S, Altman DG. Composite outcomes in cardiovascular research: a survey of randomized trials. Ann Intern Med. 2008;149:612-7.
15. Ibrahim F, Tom BDM, Scott DL, Prevost AT. A systematic review of randomised controlled trials in rheumatoid arthritis: the reporting and handling of missing data in composite outcomes. Trials. 2016;17:272. doi: 10.1186/s13063-016-1402-5.
16. Moustgaard H, Bello S, Miller FG, Hróbjartsson A. Subjective and objective outcomes in randomized clinical trials: definitions differed in methods publications and were often absent from trial reports. J Clin Epidemiol. 2014;67:1327-34. doi: 10.1016/j.jclinepi.2014.06.020.
17. Molnar FJ, Man-Son-Hing M, Fergusson D. Systematic review of measures of clinical significance employed in randomized controlled trials of drugs for dementia. J Am Geriatr Soc. 2009;57:536-46. doi: 10.1111/j.1532-5415.2008.02122.x.
18. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008;336:601-5. doi: 10.1136/bmj.39465.451748.AD.
19. Hróbjartsson A, Thomsen ASS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ. 2012;344:e1119. doi: 10.1136/bmj.e1119.
20. Page MJ, Higgins JPT, Clayton G, Sterne JA, Hróbjartsson A, Savović J. Empirical evidence of study design biases in randomized trials: systematic review of meta-epidemiological studies. PLoS One. 2016;11:e0159267. doi: 10.1371/journal.pone.0159267.
21. Abraha I, Cherubini A, Cozzolino F, De Florio R, Luchetta ML, Rimland JM, et al. Deviation from intention to treat analysis in randomised trials and treatment effect estimates: meta-epidemiological study. BMJ. 2015;350:h2445. doi: 10.1136/bmj.h2445.
22. Marshall M, Lockwood A, Bradley C, Adams C, Joy C, Fenton M. Unpublished rating scales: a major source of bias in randomised controlled trials of treatments for schizophrenia. Br J Psychiatry. 2000;176:249-52. doi: 10.1192/bjp.176.3.249.
23. Gøtzsche PC. Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis. Control Clin Trials. 1989;10:31-56. https://www.ncbi.nlm.nih.gov/pubmed/2702836. A published erratum appears in Control Clin Trials. 1989;10:356.
24. Hobart JC, Cano SJ, Zajicek JP, Thompson AJ. Rating scales as outcome measures for clinical trials in neurology: problems, solutions, and recommendations. Lancet Neurol. 2007;6:1094-105. doi: 10.1016/S1474-4422(07)70290-9. A published erratum appears in Lancet Neurol. 2008;7:25.
25. Rahimi K, Malhotra A, Banning AP, Jenkinson C. Outcome selection and role of patient reported outcomes in contemporary cardiovascular trials: systematic review. BMJ. 2010;341:c5707. doi: 10.1136/bmj.c5707.
26. Rothwell PM. Factors that can affect the external validity of randomised controlled trials. PLoS Clin Trials. 2006;1:e9. doi: 10.1371/journal.pctr.0010009.
27. Make B. How can we assess outcomes of clinical trials: the MCID approach. COPD. 2007;4:191-4.
28. National Institute for Health and Care Excellence. https://cks.nice.org.uk/sore-throat-acute#!topicsummary. Accessed 11 Mar 2017.
29. National Institute for Health and Care Excellence (NICE). Amantadine, oseltamivir and zanamivir for the treatment of influenza: technology appraisal guidance. NICE guideline TA168. London: NICE; 2009. https://www.nice.org.uk/Guidance/ta168. Accessed 28 Dec 2016.
30. Heneghan CJ, Onakpoya I, Jones MA, Doshi P, Del Mar CB, Hama R, et al. Neuraminidase inhibitors for influenza: a systematic review and meta-analysis of regulatory and mortality data. Health Technol Assess. 2016;20:(42). doi: 10.3310/hta20420.
31. Siontis GCM, Ioannidis JPA. Risk factors and interventions with statistically significant tiny effects. Int J Epidemiol. 2011;40:1292-307. doi: 10.1093/ije/dyr099.
32. Wolfe F, Michaud K, Strand V. Expanding the definition of clinical differences: from minimally clinically important differences to really important differences: analyses in 8931 patients with rheumatoid arthritis. J Rheumatol. 2005;32:583-9.
33. Linde K, Kriston L, Rücker G, Jamil S, Schumann I, Meissner K, et al. Efficacy and acceptability of pharmacological treatments for depressive disorders in primary care: systematic review and network meta-analysis. Ann Fam Med. 2015;13:69-79. doi: 10.1370/afm.1687.
34. Guyatt GH, Briel M, Glasziou P, Bassler D, Montori VM. Problems of stopping trials early. BMJ. 2012;344:e3863. doi: 10.1136/bmj.e3863.
35. Akl EA, Briel M, You JJ, Sun X, Johnston BC, Busse JW, et al. Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic review. BMJ. 2012;344:e2809. doi: 10.1136/bmj.e2809.
36. Kang H. The prevention and handling of the missing data. Korean J Anesthesiol. 2013;64:402-6. doi: 10.4097/kjae.2013.64.5.402.
37. Rothwell P, Warlow C. Is self-audit reliable? Lancet. 1995;346:1623. doi: 10.1016/s0140-6736(95)91953-8.
38. Schmucker C, Schell LK, Portalupi S, Oeller P, Cabrera L, Bassler D, et al. Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries. PLoS One. 2014;9:e114023. doi: 10.1371/journal.pone.0114023.
39. Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14(8). doi: 10.3310/hta14080.
40. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev. 2009;1:MR000006. doi: 10.1002/14651858.MR000006.pub3.
41. Laine C, De Angelis C, Delamothe T, Drazen JM, Frizelle FA, Haug C, et al. Clinical trial registration: looking back and moving ahead. Ann Intern Med. 2007;147:275-7. doi: 10.7326/0003-4819-147-4-200708210-00166.
42. AllTrials. About AllTrials. http://www.alltrials.net/find-out-more/about-alltrials/. Accessed 11 Mar 2017.
43. Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365. doi: 10.1136/bmj.c365.
44. Rodgers MA, Brown JVE, Heirs MK, Higgins JPT, Mannion RJ, Simmonds MC, et al. Reporting of industry funded study outcome data: comparison of confidential and published data on the safety and effectiveness of rhBMP-2 for spinal fusion. BMJ. 2013;346:f3981. doi: 10.1136/bmj.f3981.
45. Golder S, Loke YK, Wright K, Norman G. Reporting of adverse events in published and unpublished studies of health care interventions: a systematic review. PLoS Med. 2016;13:e1002127. doi: 10.1371/journal.pmed.1002127.
46. Wieseler B, Wolfram N, McGauran N, Kerekes MF, Vervölgyi V, Kohlepp P, et al. Completeness of reporting of patient-relevant clinical trial outcomes: comparison of unpublished clinical study reports with publicly available data. PLoS Med. 2013;10:e1001526. doi: 10.1371/journal.pmed.1001526.
47. Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ. 2012;344:d7202. doi: 10.1136/bmj.d7202.
48. Schroll JB, Penninga EI, Gøtzsche PC. Assessment of Adverse Events in Protocols, Clinical Study Reports, and Published Papers of Trials of Orlistat: A Document Analysis. PLoS Med. 2016;13(8):e1002101. doi: 10.1371/journal.pmed.1002101.
49. Vedula SS, Bero L, Scherer RW, Dickersin K, et al. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361:1963-71. doi: 10.1056/nejmsa0906126.
50. Saini P, Loke YK, Gamble C, Altman DG, Williamson PR, Kirkham JJ. Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews. BMJ. 2014;349:g6501. doi: 10.1136/bmj.g6501.
51. Zorzela L, Loke YK, Ioannidis JP, Golder S, Santaguida P, Altman DG, et al. PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ. 2016;352:i157. doi: 10.1136/bmj.i157. A published erratum appears in BMJ. 2016;353:i2229.
52. Smyth RMD, Kirkham JJ, Jacoby A, Altman DG, Gamble C, Williamson PR. Frequency and reasons for outcome reporting bias in clinical trials: interviews with trialists. BMJ. 2011;342:c7153. doi: 10.1136/bmj.c7153.
53. Dwan K, Altman DG, Clarke M, Gamble C, Higgins JPT, Sterne JA, et al. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 2014;11:e1001666. doi: 10.1371/journal.pmed.1001666.
54. Weston J, Dwan K, Altman D, Clarke M, Gamble C, Schroter S, et al. Feasibility study to examine discrepancy rates in prespecified and reported outcomes in articles submitted to The BMJ. BMJ Open. 2016;6:e010075. doi: 10.1136/bmjopen-2015-010075.
55. Goldacre B, Drysdale H, Powell-Smith A, Dale A, Milosevic I, Slade E, et al. The COMPare Trials Project. 2016. http://compare-trials.org/. Accessed 11 Mar 2017.
56. King NB, Harper S, Young ME. Use of relative and absolute effect measures in reporting health inequalities: structured review. BMJ. 2012;345:e5774. doi: 10.1136/bmj.e5774.
57. Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. doi: 10.1136/bmj.c332.
58. Mahtani KR. 'Spin' in reports of clinical research. Evid Based Med. 2016;21:201-2. doi: 10.1136/ebmed-2016-110570.
59. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303:2058-64. doi: 10.1001/jama.2010.651.
60. Boutron I, Altman DG, Hopewell S, Vera-Badillo F, Tannock I, Ravaud P. Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of cancer: the SPIIN randomized controlled trial. J Clin Oncol. 2014;32:4120-6. doi: 10.1200/jco.2014.56.7503.
61. Lieb K, von der Osten-Sacken J, Stoffers-Winterling J, Reiss N, Barth J. Conflicts of interest and spin in reviews of psychological therapies: a systematic review. BMJ Open. 2016;6:e010606. doi: 10.1136/bmjopen-2015-010606.
62. Yavchitz A, Ravaud P, Altman DG, Moher D, Hróbjartsson A, Lasserson T, et al. A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity. J Clin Epidemiol. 2016;75:56-65. doi: 10.1016/j.jclinepi.2016.01.020.
63. European Agency for the Evaluation of Medicinal Products, Committee for Proprietary Medicinal Products. Points to consider on multiplicity issues in clinical trials. CPMP/EWP/908/99. 19 Sep 2002. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003640.pdf. Accessed 11 Mar 2017.
64. Sun X, Briel M, Busse JW, You JJ, Akl EA, Mejza F, et al. Credibility of claims of subgroup effects in randomised controlled trials: systematic review. BMJ. 2012;344:e1553. doi: 10.1136/bmj.e1553.
65. Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132. doi: 10.1186/1745-6215-13-132.
66. Tugwell P, Boers M, Brooks P, Simon L, Strand V, Idzerda L. OMERACT: an international initiative to improve outcome measurement in rheumatology. Trials. 2007;8:38. doi: 10.1186/1745-6215-8-38.
67. Williamson P. The COMET Initiative [abstract]. Trials. 2013;14 Suppl 1:O65. doi: 10.1186/1745-6215-14-s1-o65.
68. Gargon E, Williamson PR, Altman DG, Blazeby JM, Clarke M. The COMET Initiative database: progress and activities update (2014). Trials. 2015;16:515. doi: 10.1186/s13063-015-1038-x.
69. Grading of Recommendations Assessment, Development and Evaluation (GRADE). http://www.gradeworkinggroup.org/. Accessed 11 Mar 2017.
70. Kelly S, Lafortune L, Hart N, Cowan K, Fenton M. Brayne C; Dementia Priority Setting Partnership. Dementia priority setting partnership with the James Lind Alliance: using patient and public involvement and the evidence base to inform the research agenda. Age Ageing. 2015;44:985-93. doi: 10.1093/ageing/afv143.