Blinding in Rehabilitation Research: Empirical Evidence on the Association between Blinding and Treatment Effect Estimates

American Journal of Physical Medicine and Rehabilitation

Volumn 99, Issue 3, 2020, Pages 198-209

  Armijo Olivo, Susan ^a,b,c   Dennett, Liz ^b   Arienti, Chiara ^c   Dahchi, Mustafa ^b   Arokoski, Jari ^d,e   Heinemann, Allen W ^f   Malmivaara, Antti ^g  

^a UNIVERSITY OF APPLIED SCIENCES OSNABRÜCK   (Germany)

^b UNIVERSITY OF ALBERTA   (Canada)

^c IRCCS FONDAZIONE DON CARLO GNOCCHI   (Italy)

^d HELSINKI UNIVERSITY CENTRAL HOSPITAL   (Finland)

^e UNIVERSITY OF HELSINKI   (Finland)

^f NORTHWESTERN UNIVERSITY   (United States)

^g NATIONAL PUBLIC HEALTH INSTITUTE   (Finland)

Author keywords

Blinding; Randomized Controlled Trials; Rehabilitation

Indexed keywords

DOUBLE BLIND PROCEDURE; HUMAN; METHODOLOGY; REHABILITATION RESEARCH;

DOUBLE-BLIND METHOD; HUMANS; OUTCOME ASSESSMENT, HEALTH CARE; REHABILITATION RESEARCH; RESEARCH DESIGN;

EID: 85080866609     PISSN: 08949115     EISSN: 15377385     Source Type: Journal    
DOI: 10.1097/PHM.0000000000001377     Document Type: Article

References (89)

1. Pildal J, Hrobjartsson A, Jorgensen KJ,: Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol 2007; 36: 847-57
2. Wood L, Egger M, Gluud LL,: Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: Meta-epidemiological study. BMJ 2008; 336: 601-5
3. Hartling L, Ospina M, Liang Y,: Risk of bias versus quality assessment of randomized controlled trials: Cross sectional study. BMJ 2009; 339: 1017
4. Moher D, Pham B, Jones A,: Does quality of reports of randomized trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998; 352: 609-13
5. Schulz KF, Chalmers I, Hayes RJ,: Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995; 273: 408-12
6. Nuesch E, Trelle S, Reichenbach S,: The effects of excluding patients from the analysis in randomized controlled trials: Meta-epidemiological study. BMJ 2009; 339: 679-83
7. Nuesch E, Trelle S, Reichenbach S,: Small study effects in meta-analyses of osteoarthritis trials: Meta-epidemiological study. BMJ 2010; 341: 241
8. Savovic J, Jones HE, Altman DG,: Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med 2012; 157: 429-38
9. Nuesch E, Reichenbach S, Trelle S,: The importance of allocation concealment and patient blinding in osteoarthritis trials: A meta-epidemiologic study. Arthritis Care Res 2009; 61: 1633-41
10. Berger VW, Weinstein S,: Ensuring the comparability of comparison groups: Is randomization enough? Control Clin Trials 2004; 25: 515-24
11. Trowman R, Dumville JC, Torgerson DJ,: The impact of trial baseline imbalances should be considered in systematic reviews: A methodological case study. J Clin Epidemiol 2007; 60: 1229-33
12. Armijo-Olivo S, Saltaji H, da Costa BR,: What is the influence of randomization sequence generation and allocation concealment on treatment effects of physical therapy trials? A meta-epidemiological study. BMJ Open 2015; 5: E008562
13. Saltaji H,: Examining Bias in Oral Health Randomized Trials and Its Impact on the Magnitude of Effect Sizes. Edmonton, Department of Dentistry, University of Alberta, 2016
14. Hewitt CE, Kumaravel B, Dumville JC,: Assessing the impact of attrition in randomized controlled trials. J Clin Epidemiol 2010; 63: 1264-70
15. Bekelman JE, Li Y, Gross CP,: Scope and impact of financial conflicts of interest in biomedical research: A systematic review. JAMA 2003; 289: 454-65
16. Lexchin J, Bero LA, Djulbegovic B,: Pharmaceutical industry sponsorship and research outcome and quality: Systematic review. BMJ 2003; 326: 1167-70
17. Higgins JP, Altman DG, Gøtzsche PC,: The Cochrane Collaboration's tool for assessing risk of bias in randomized trials. BMJ 2011; 343: D5928
18. Savović J, Weeks L, Sterne JA,: Evaluation of the Cochrane Collaboration's tool for assessing the risk of bias in randomized trials: Focus groups, online survey, proposed recommendations and their implementation. Syst Rev 2014; 3: 37
19. Hartling L, Hamm MP, Milne A,: Testing the risk of bias tool showed low reliability between individual reviewers and across consensus assessments of reviewer pairs. J Clin Epidemiol 2012; 66: 973-81
20. Armijo-Olivo S, Ospina M, da Costa BR,: Poor reliability between Cochrane reviewers and blinded external reviewers when applying the Cochrane risk of bias tool in physical therapy trials. PLoS One 2014; 9: E96920
21. Armijo-Olivo S, Stiles CR, Hagen NA,: Assessment of study quality for systematic reviews: A comparison of the Cochrane collaboration risk of bias tool and the effective public health practice project quality assessment tool: Methodological research. J Eval Clin Pract 2012; 18: 12-8
22. Hempel S, Suttorp M, Miles J,: Empirical evidence of associations between trial quality and effect size. Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. Available at: Http://www.ncbi.nlm.nih.gov/books/NBK56925/. Published 2011. Accessed November 20, 2019
23. Hempel S,: Detection of associations between trial quality and effect sizes. Agency for Healthcare Research and Quality. Available at: Http://www.ncbi.nlm.nih.gov/books/NBK84249/. Published 2012. Accessed November 20, 2019
24. Coolbrandt A, Wildiers H, Aertgeerts B,: Characteristics and effectiveness of complex nursing interventions aimed at reducing symptom burden in adult patients treated with chemotherapy: A systematic review of randomized controlled trials. Int J Nurs Stud 2014; 51: 495-510
25. Kunz R, Autti-Ramo I, Anttila H,: A systematic review finds that methodological quality is better than its reputation but can be improved in physiotherapy trials in childhood cerebral palsy. J Clin Epidemiol 2006; 59: 1239-48
26. Armijo-Olivo S, Fuentes J, da Costa BR,: Blinding in physical therapy trials and its association with treatment effects: A meta-epidemiological study. Am J Phys Med Rehabil 2017; 96: 34-44
27. Boutron I, Tubach F, Giraudeau B,: Blinding was judged more difficult to achieve and maintain in nonpharmacologic than pharmacologic trials. J Clin Epidemiol 2004; 57: 543-50
28. Armijo-Olivo S, Fuentes J, Ospina M,: Inconsistency in the items included in tools used in general health research and physical therapy to evaluate the methodological quality of randomized controlled trials: A descriptive analysis. BMC Med Res Methodol 2013; 13: 116
29. Olivo SA, Macedo LG, Gadotti IC,: Scales to assess the quality of randomized controlled trials: A systematic review. Phys Ther 2008; 88: 156-75
30. Malmivaara A,: Pure intervention effect or effect in routine health care-blinded or non-blinded randomized controlled trial. BMC Med Res Methodol 2018; 18: 91
31. Malmivaara A,: Assessing the effectiveness of rehabilitation and optimizing effectiveness in routine clinical work. J Rehabil Med 2018; 50: 849-51
32. Moher D, Liberati A, Tetzlaff J,: Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Ann Intern Med 2009; 151: 264-9
33. Cheon S, Park HJ, Chae Y,: Does different information disclosure on placebo control affect blinding and trial outcomes? A case study of participant information leaflets of randomized placebo-controlled trials of acupuncture. BMC Med Res Methodol 2018; 18: 13
34. Boutron I, Altman DG, Moher D, CONSORT NPT Group: CONSORT statement for randomized trials of nonpharmacologic treatments: A 2017 update and a CONSORT extension for nonpharmacologic trial abstracts. Ann Intern Med 2017; 167: 40-7
35. Puhl AA, Reinhart CJ, Doan JB,: The quality of placebos used in randomized, controlled trials of lumbar and pelvic joint thrust manipulation-a systematic review. Spine J 2017; 17: 445-56
36. Takayama M, Yajima H, Kawase A,: The potential of double blinding with two placebo acupuncture needles: A randomized controlled pilot-trial. Medicine 2014; 2: 11-27
37. Mayo NE, Kaur N, Barbic SP,: How have research questions and methods used in clinical trials published in clinical rehabilitation changed over the last 30 years? Clin Rehabil 2016; 30: 847-64
38. Villamar MF, Contreras VS, Kuntz RE,: The reporting of blinding in physical medicine and rehabilitation randomized controlled trials: A systematic review. J Rehabil Med 2013; 45: 6-13
39. Hart T, Bagiella E,: Design and implementation of clinical trials in rehabilitation research. Arch Phys Med Rehabil 2012; 93 (8 suppl): S117-26
40. Hopton AK, Macpherson H,: Assessing blinding in randomized controlled trials of acupuncture: Challenges and recommendations. Chin J Integr Med 2011; 17: 173-6
41. Minns Lowe CJ, Wilson MS, Sackley CM,: Blind outcome assessment: The development and use of procedures to maintain and describe blinding in a pragmatic physiotherapy rehabilitation trial. Clin Rehabil 2011; 25: 264-74
42. Solari A,: Methodological aspects of randomized controlled trials on cognitive interventions. Neurol Sci 2010; 31 (suppl 2): S279-82
43. Rakel B, Cooper N, Adams HJ,: A new transient sham TENS device allows for investigator blinding while delivering a true placebo treatment. J Pain 2010; 11: 230-8
44. Machado LA, Kamper SJ, Herbert RD,: Imperfect placebos are common in low back pain trials: A systematic review of the literature. Eur Spine J 2008; 17: 889-904
45. Boutron I, Moher D, Altman DG, CONSORT Group: Methods and processes of the CONSORT group: Example of an extension for trials assessing nonpharmacologic treatments. Ann Intern Med 2008; 148: W60-6
46. Boutron I, Guittet L, Estellat C,: Reporting methods of blinding in randomized trials assessing nonpharmacological treatments. PLoS Med 2007; 4: E61
47. Licciardone JC, Russo DP,: Blinding protocols, treatment credibility, and expectancy: Methodologic issues in clinical trials of osteopathic manipulative treatment. J Am Osteopath Assoc 2006; 106: 457-63
48. Chae Y, Lee YS, Enck P,: How placebo needles differ from placebo pills? Front Psych 2018; 9: 243
49. Gagnier J,: Bias arising from the use of patient reported outcomes measures. Qual Life Res 2017; 26 (1 suppl 1): 140-1
50. Cerritelli F, Verzella M, Cicchitti L,: The paradox of sham therapy and placebo effect in osteopathy a systematic review. Medicine (United States) 2016; 95: E4728
51. Bolvig J, Juhl CB, Boutron I,: Assessing bias in osteoarthritis trials included in cochrane reviews: A meta-epidemiological study. Osteoarthr Cartil 2016; 1: S42-3
52. Losina E, Ranstam J, Collins JE,: OARSI clinical trials recommendations: Key analytic considerations in design, analysis, and reporting of randomized controlled trials in osteoarthritis. Osteoarthr Cartil 2015; 23: 677-85
53. Freed B, Williams BJ, Bang H,: Evaluation of blinding in randomized controlled trials of back pain treatment modalities. PM and R 2015; 1: S145-6
54. McIntyre A, Vermeer J, Mays R,: Motor interventions for stroke rehabilitation: Do the randomized controlled trials have good methodological quality? Stroke 2014; 45: E292
55. McIntyre A, Janzen S, Richardson M,: The methodological quality of randomized controlled trials in stroke rehabilitation literature. Stroke 2013; 44: E195
56. Bishop F, Jacobsen E, Shaw J,: Debriefing to placebo allocation: A phenomenological study of participants' experiences in a randomized clinical trial. European Journal of Integrative Medicine 2010; 2: 199
57. Relf I, Chow R, Pirotta M,: Blinding techniques in randomized controlled trials of laser therapy: An overview and possible solution. Evid Based Complement Alternat Med 2008; 5: 383-9
58. Kamper SJ,: Blinding: Linking evidence to practice. Journal of Orthopedic & Sports Physical Therapy 2018; 48: 825-6
59. Berger VW,: Assessing the success of masking in acupuncture trials: Further insight. Chin J Integr Med 2011; 17: 546
60. Bishop FL, Jacobson EE, Shaw J,: Participants' experiences of being debriefed to placebo allocation in a clinical trial. Qual Health Res 2012; 22: 1138-49
61. Bhogal SK, Teasell RW, Foley NC,: The PEDro scale provides a more comprehensive measure of methodological quality than the Jadad scale in stroke rehabilitation literature. J Clin Epidemiol 2005; 58: 668-73
62. Bo C, Xue Z, Yi G,: Assessing the quality of reports about randomized controlled trials of acupuncture treatment on diabetic peripheral neuropathy. PLoS One 2012; 7: E38461
63. Boutron I, Tubach F, Giraudeau B,: Methodological differences in clinical trials evaluating nonpharmacological and pharmacological treatments of hip and knee osteoarthritis. JAMA 2003; 290: 1062-70
64. Bradt J,: Randomized controlled trials in music therapy: Guidelines for design and implementation. J Music Ther 2012; 49: 120-49
65. Li X, Wang R, Shi X,: Reporting characteristics and risk of bias in randomized controlled trials of acupuncture analgesia published in PubMed-listed journals. Acupunct Med 2017; 35: 259-67
66. Palmer S, Domaille M, Cramp F,: Blinding of transcutaneous electrical nerve stimulation (TENS): Insights from a randomized placebo controlled trial. Physiotherapy (United Kingdom) 2011; 1: ES958-9
67. Bang H, Flaherty SP, Kolahi J,: Blinding assessment in clinical trials: A review of statistical methods and a proposal of blinding assessment protocol. Clin Res Regul Affairs 2010; 27: 42-51
68. Charlesworth JEG, Petkovic G, Kelley JM,: Effects of placebos without deception compared with no treatment: A systematic review and meta-analysis. J Evid Based Med 2017; 10: 97-107
69. Jacobs WC, Kruyt MC, Moojen WA,: No evidence for intervention-dependent influence of methodological features on treatment effect. J Clin Epidemiol 2013; 66: 1347-55
70. Page SJ, Persch AC,: Recruitment, retention, and blinding in clinical trials. Am J Occup Ther 2013; 67: 154-61
71. Rosner AL,: Evidence-based medicine: Revisiting the pyramid of priorities. J Bodywork Mov Ther 2012; 16: 42-9
72. Manheimer E, Ezzo J, Hadhazy V,: Published reports of acupuncture trials showed important limitations. J Clin Epidemiol 2006; 59: 107-13
73. Takakura N, Yajima H,: A double-blind placebo needle for acupuncture research. BMC Complement Altern Med 2007; 7: 31
74. Cheon S, Lee H,: What information on sham acupuncture is given to trial participants and how does it affect blinding and outcomes? Integr Med Res 2015; 1: 40
75. Haller H, Ostermann T, Lauche R,: Controlling for placebo effects in clinical trials of craniosacral therapy: Blinding success and credibility of a new sham-control protocol. J Altern Complement Med 2014; 20: A130
76. Boyd R,: Lessons learnt from RCTs of interventions for severe cerebral palsy. Dev Med Child Neurol 2013; 2: 6-7
77. Berger VW,: On the insufficiency of reporting masking. J Rehabil Med 2013; 45: 221-2
78. Chaibi A, Saltyte Benth J, Tuchin P,: EHMTI-0266. Validation of a sham manipulative procedure: A new protocol for conducting placebo-control clinical trials in manual therapy. Journal of Headache and Pain Conference: 4th European Headache and Migraine Trust International Congress, EHMTIC 2014; 15 (suppl 1)
79. Kawchuk GN, Haugen R, Fritz J,: A true blind for subjects who receive spinal manipulation therapy. Arch Phys Med Rehabil 2009; 90: 366-8
80. Sterne JAC, Juni P, Schulz KF,: Statistical methods for assessing the influence of study characteristics on treatment effects in 'meta-epidemiological' research. Statistics in Medicine 2002; 21: 1513-24
81. Moroz A, Freed B, Tiedemann L,: Blinding measured: A systematic review of randomized controlled trials of acupuncture. Evid Based Complement Alternat Med 2013; 2013: 708251
82. Liu CJ, LaValley M, Latham NK,: Do unblinded assessors bias muscle strength outcomes in randomized controlled trials of progressive resistance strength training in older adults? Am J Phys Med Rehabil 2011; 90: 190-6
83. Hrobjartsson A, Emanuelsson F, Skou Thomsen AS,: Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies. Int J Epidemiol 2014; 43: 1272-83
84. Braithwaite FA, Walters JL, Li LSK,: Effectiveness and adequacy of blinding in the moderation of pain outcomes: Systematic review and meta-analyses of dry needling trials. PeerJ 2018; 6: E5318
85. Bolvig J, Juhl CB, Boutron I,: Some Cochrane risk-of-bias items are not important in osteoarthritis trials: A meta-epidemiological study based on Cochrane reviews. J Clin Epidemiol 2018; 95: 143-56
86. Bang H, Ni L, Davis CE,: Assessment of blinding in clinical trials. Controlled Clinical Trials 2006; 25: 128-36
87. Berkman ND, Santaguida PL, Viswanathan M,: The Empirical Evidence of Bias in Trials Measuring Treatment Differences. Report No.: 14-EHC050-EF. Rockville MD, Agency for Healthcare Research and Quality (US), 2014
88. Vos T, Flaxman AD, Naghavi M,: Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990-2010: A systematic analysis for the global burden of disease study 2010. Lancet 2012; 380: 2163-96
89. Moseley AM, Herbert RD, Maher CG,: Reported quality of randomized controlled trials of physiotherapy interventions has improved over time. J Clin Epidemiol 2011; 64: 594-601