CONSORT-SPI 2018 Explanation and Elaboration: Guidance for reporting social and psychological intervention trials

Trials

Volumn 19, Issue 1, 2018, Pages

  Grant, Sean ^a   Mayo Wilson, Evan ^b,u   Montgomery, Paul ^c   Macdonald, Geraldine ^d,t   Michie, Susan ^e   Hopewell, Sally ^f   Moher, David ^g   Lawrence Aber, J ^h   Altman, Doug ^f   Bhui, Kamaldeep ⁱ   Booth, Andrew ^j   Clark, David ^k   Craig, Peter ^l   Eisner, Manuel ^m   Sherman, Lawrence ^m   Fraser, Mark W ⁿ   Gardner, Frances ^k   Hedges, Larry ^o   Hollon, Steve ^p   Kaplan, Robert ^q   Kaufmann, Peter ^r   Konstantopoulos, Spyros ^s   McLeroy, Kenneth ^v   Mittman, Brian ^w   Nezu, Arthur ^x   Sonuga Barke, Edmund ^y   Thomas, James ^z   VandenBos, Gary ^aa   Waters, Elizabeth ^ab   West, Robert ^ac   Yaffe, Joanne ^ad   more..

^a RAND CORPORATION   (United States)

^b JOHNS HOPKINS BLOOMBERG SCHOOL OF PUBLIC HEALTH   (United States)

^c UNIVERSITY OF BIRMINGHAM   (United Kingdom)

^d Centre for Exercise Nutrition and Health Sciences School for Policy Studies   (United Kingdom)

^e UNIVERSITY COLLEGE LONDON   (United Kingdom)

^f UNIVERSITY OF OXFORD   (United Kingdom)

^g OTTAWA HOSPITAL RESEARCH INSTITUTE   (Canada)

^h NEW YORK UNIVERSITY   (United States)

ⁱ QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

^j UNIVERSITY OF SHEFFIELD   (United Kingdom)

^k UNIVERSITY OF OXFORD   (United Kingdom)

^l UNIVERSITY OF GLASGOW   (United Kingdom)

^m UNIVERSITY OF CAMBRIDGE   (United Kingdom)

ⁿ UNIVERSITY OF NORTH CAROLINA   (United States)

^o NORTHWESTERN UNIVERSITY   (United States)

^p VANDERBILT UNIVERSITY   (United States)

^q UNIVERSITY OF CALIFORNIA   (United States)

^r NATIONAL INSTITUTES OF HEALTH   (United States)

^s MICHIGAN STATE UNIVERSITY   (United States)

^t UNIVERSITY OF BRISTOL   (United Kingdom)

^u JOHNS HOPKINS UNIVERSITY   (United States)

^v TEXAS A AND M UNIVERSITY   (United States)

^w KAISER PERMANENTE   (United States)

^x DREXEL UNIVERSITY COLLEGE OF MEDICINE   (United States)

^y KING'S COLLEGE LONDON   (United Kingdom)

^z UNIVERSITY OF LONDON   (United Kingdom)

^aa AMERICAN PSYCHOLOGICAL ASSOCIATION   (United States)

^ab UNIVERSITY OF MELBOURNE   (Australia)

^ac Health Behaviour Research Centre   (United States)

^ad UNIVERSITY OF UTAH   (United States)

more..

Author keywords

CONSORT; Randomised controlled trial; Reporting guideline; Reporting standards; Transparency

Indexed keywords

ADULT; ARTICLE; CHECKLIST; CONSENSUS; DELPHI STUDY; EDITOR; FEMALE; HUMAN; HUMAN EXPERIMENT; INTERVENTION STUDY; MALE; PRACTICE GUIDELINE; RANDOMIZED CONTROLLED TRIAL (TOPIC); SCIENTIST; STAKEHOLDER ENGAGEMENT; SYSTEMATIC REVIEW; BEHAVIOR THERAPY; BIOASSAY; CONSENSUS DEVELOPMENT; MEASUREMENT ACCURACY; METHODOLOGY; OUTCOME ASSESSMENT; SOCIAL ENVIRONMENT; STANDARDS; TREATMENT OUTCOME;

BEHAVIOR THERAPY; CHECKLIST; CONSENSUS; DATA ACCURACY; DELPHI TECHNIQUE; ENDPOINT DETERMINATION; HUMANS; OUTCOME ASSESSMENT (HEALTH CARE); RANDOMIZED CONTROLLED TRIALS AS TOPIC; RESEARCH DESIGN; SOCIAL ENVIRONMENT; TREATMENT OUTCOME;

EID: 85050756494     PISSN: None     EISSN: 17456215     Source Type: Journal    
DOI: 10.1186/s13063-018-2735-z     Document Type: Article

References (117)

1. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276:637-9.
2. Moher D, Jones A, Lepage L, for the CONSORT Group. Use of the CONSORT statement and quality of reports of randomized trials: A comparative before-and-after evaluation. JAMA. 2001;285:1992-5.
3. Plint AC, Moher D, Morrison A, et al. Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust. 2006;185:263-7.
4. Turner L, Shamseer L, Altman DG, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev. 2012;11:MR000030.
5. Devereaux PJ, Manns BJ, Ghali WA, Quan H, Guyatt GH. The reporting of methodological factors in randomized controlled trials and the association with a journal policy to promote adherence to the consolidated standards of reporting trials (CONSORT) checklist. Control Clin Trials. 2002;23:380-8.
6. Shamseer L, Hopewell S, Altman DG, Moher D, Schulz KF. Update on the endorsement of CONSORT by high impact factor journals: a survey of journal 'instructions to authors' in 2014. Trials. 2016;17(1):301.
7. Hopewell S, Altman DG, Moher D, Schulz KF. Endorsement of the CONSORT statement by high impact factor medical journals: a survey of journal editors and journal 'Instructions to Authors'. Trials. 2008;9:20.
8. Campbell MK, Piaggio G, Elbourne DR, Altman DG. Consort 2010 statement: extension to cluster randomised trials. BMJ. 2012;345:e5661.
9. Piaggio G, Elbourne DR, Pocock SJ, Evans SJW, Altman DG, for the CONSORT Group. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA. 2012;308(24):2594-604.
10. Zwarenstein M, Treweek S, Gagnier JJ, et al. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ. 2008;337:a2390.
11. Vohra S, Shamseer L, Sampson M, et al. CONSORT extension for reporting N-of-1 trials (CENT) 2015 statement. BMJ. 2015;350:h1738.
12. Calvert M, Blazeby J, Altman DG, et al. Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. JAMA. 2013;309(8):814-22.
13. Hopewell S, Clarke M, Moher D, et al. CONSORT for reporting randomised trials in journal and conference abstracts. Lancet. 2008;371:281-3.
14. Ioannidis JP, Evans SJ, Gotzsche PC, et al. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med. 2004;141:781-8.
15. Boutron I, Moher D, Altman DG, Schulz K, Ravaud P, for the CONSORT group. Methods and processes of the CONSORT group: example of an extension for trials assessing nonpharmacologic treatments. Ann Intern Med. 2008;148(4):W60-6.
16. Gagnier JJ, Boon H, Rochon P, Moher D, Barnes J, Bombardier C. Reporting randomized, controlled trials of herbal interventions: an elaborated CONSORT statement. Ann Intern Med. 2006;144:364-7.
17. MacPherson H, Altman DG, Hammerschlag R, et al. Revised STandards for reporting interventions in clinical trials of acupuncture (STRICTA): extending the CONSORT statement. PLoS Med. 2010;7(6):e1000261.
18. Grant SP, Mayo-Wilson E, Melendez-Torres GJ, Montgomery P. Reporting quality of social and psychological intervention trials: a systematic review of reporting guidelines and trial publications. PLoS One. 2013;8(5):e65442.
19. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.
20. Michie S, Fixsen D, Grimshaw J, Eccles M. Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci. 2009;4:40.
21. Mayo-Wilson E. Reporting implementation in randomized trials: proposed additions to the consolidated standards of reporting trials statement. Am J Public Health. 2007;97(4):630-3.
22. Glasziou P, Meats E, Heneghan C, Shepperd S. What is missing from descriptions of treatment in trials and reviews? BMJ. 2008;336(7659):1472-4.
23. Open Science Collaboration. Estimating the reproducibility of psychological science. Science. 2015;349(6251):aac4716.
24. Nosek BA, Alter G, Banks GC, et al. Promoting an open research culture. Science. 2015;348:1422-5.
25. McNutt M. Reproducibility. Science. 2014;343(6168):229.
26. Open Science Collaboration. An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspect Psychol Sci. 2012;7(6):657-60.
27. Tajika A, Ogawa Y, Takeshima N, Hayasaka Y, Furukawa TA. Replication and contradiction of highly cited research papers in psychiatry: 10-year follow-up. Br J Psychiatry. 2015; Available online July 2015: https://doi.org/10.1192/bjp.bp.1113.143701.
28. Michie S, Wood CE, Johnston M, Abraham C, Francis JJ, Hardeman W. Behaviour change techniques: the development and evaluation of a taxonomic method for reporting and describing behaviour change interventions (a suite of five studies involving consensus methods, randomised controlled trials and analysis of qualitative data). Health Technol Assess. 2015;19(99):1-187.
29. Glasziou P, Altman DG, Bossuyt P, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267-76.
30. Moher D, Glasziou P, Chalmers I, et al. Increasing value and reducing waste in biomedical research: Who's listening? Lancet. 2015; Available online 27 September 2015:doi: 10.1016/S0140-6736(1015)00307-00304
31. Driessen E, Hollon SD, Bockting CLH, Cuijpers P, Turner EH. Does publication bias inflate the apparent efficacy of psychological treatment for major depressive disorder? A systematic review and meta-analysis of us national institutes of health-funded trials. PLoS One. 2015;10(9):e0137864.
32. Ioannidis JP, Munafo MR, Fusar-Poli P, Nosek BA, David SP. Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention. Trends Cogn Sci. 2014;18(5):235-41.
33. Cybulski L, Mayo-Wilson E, Grant S. Improving transparency and reproducibility through registration: the status of intervention trials published in clinical psychology journals. J Consult Clin Psychol. 2016;84(9):753.
34. Mayo-Wilson E, Grant S, Hopewell S, Macdonald G, Moher D, Montgomery P. Developing a reporting guideline for social and psychological intervention trials. Trials. 2013;14(1):242.
35. Fraser MW, Galinsky MJ, Richman JM, Day SH. Intervention research: developing social programs. New York: Oxford University Press; 2009.
36. Medical Research Council. Developing and evaluating complex interventions: New guidance. London: Medical Research Council; 2008. www.mrc.ac.uk/documents/pdf/complex-interventions-guidance.
37. Grant S. Development of a CONSORT extension for social and psychological interventions. Oxford: Social Policy & Intervention, University of Oxford; 2014.
38. Institute of Medicine. Psychosocial interventions for mental and substance use disorders: a framework for establishing evidence-based standards. Washington, DC: The National Academies Press; 2015.
39. Riley W. New NIH Clinical Trials Policies: Implications for Behavioral and Social Science Researchers. 2016; https://obssr.od.nih.gov/new-nih-clinical-trials-policies-implications-for-behavioral-and-social-science-researchers/.
40. Möhler R, Köpke S, Meyer G. Criteria for reporting the development and evaluation of complex interventions in healthcare: revised guideline (CReDECI 2). Trials. 2015;16:204.
41. Montgomery P, Grant S, Hopewell S, et al. Protocol for CONSORT-SPI: an extension for social and psychological interventions. Implement Sci. 2013;8(1):99.
42. Montgomery P, Grant S, Mayo-Wilson E, et al. Reporting randomised trials of social and psychological interventions: The CONSORT-SPI 2017 Extension. TRLS-D-18-00052.
43. Moher D, Hopewell S, Schultz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.
44. Des Jarlais DC, Lyles C, Crepaz N, the TREND Group. Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. Am J Public Health. 2004;94(3):361-6.
45. Petticrew M, Roberts H. Systematic reviews in the social sciences: a practical guide. Oxford: Blackwell Publishing Ltd; 2006.
46. Cooper H. Reporting research in psychology: how to meet journal article reporting standards. Washington, DC: APA; 2011.
47. Hopewell S, Clarke M, Moher D, et al. CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration. PLoS Med. 2008;5(1):e20.
48. Davidson KW, Goldstein M, Kaplan RM, et al. Evidence-based behavioral medicine: what is it and how do we achieve it? Ann Behav Med. 2003;26(3):161-71.
49. Moberg-Mogren E, Nelson DL. Evaluating the quality of reporting occupational therapy randomized controlled trials by expanding the CONSORT criteria. Am J Occup Ther. 2006;60:226-35.
50. Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials. gov results database-update and key issues. N Engl J Med. 2011;364(9):852-60.
51. Moore GF, Audrey S, Barker M, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.
52. Breuer E, Lee L, De Silva M, Lund C. Using theory of change to design and evaluate public health interventions: a systematic review. Implement Sci. 2016;11(1):63.
53. Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf. 2015;24(3):228-38.
54. Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol. 2007;27:379-87.
55. Chan AW, Tetzlaff JM, Altman DG, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200-7.
56. Bonell C, Jamal F, Melendez-Torres GJ, Cummins S. 'Dark logic': theorising the harmful consequences of public health interventions. J Epidemiol Community Health. 2015;69(1):95-8.
57. American Educational Research Association. Standards for reporting on empirical social science research in AERA publications. Educ Res. 2006;35(6):33-40.
58. Goodman SN, Fanelli D, Ioannidis JPA. What does research reproducibility mean? Sci Transl Med. 2016;8(341):341.
59. Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJW, for the CONSORT Group. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA. 2006;294(10):1152-60.
60. Shamseer L, Sampson M, Bukutu C, et al. CONSORT extension for N-of-1 trials (CENT) guidelines. BMC Complement Altern Med. 2012;12(Suppl 1):P140.
61. Zarin DA, Tse T, Williams RJ, Carr S. Trial reporting in ClinicalTrials. gov-the final rule. N Engl J Med. 2016;375(20):1998-2004.
62. Hudson KL, Lauer MS, Collins FS. Toward a new era of trust and transparency in clinical trials. JAMA. 2016;316(13):1353-4.
63. National Institutes of Health. NIH policy on dissemination of NIH-funded clinical trial information. Fed Regist. 2016;81:183.
64. Simmons JP, Nelson LD, Simonsohn U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci. 2011;22(11):1359-66.
65. Scott A, Rucklidge JJ, Mulder RT. Is mandatory prospective trial registration working to prevent publication of unregistered trials and selective outcome reporting? An observational study of five psychiatry journals that mandate prospective clinical trial registration. PLoS One. 2015;10(8):e0133718.
66. Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. Belmont: Wadsworth; 2002.
67. Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials. 2012;13(1):95.
68. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.
69. Montgomery P, Underhill K, Gardner F, Operario D, Mayo-Wilson E. The Oxford implementation index: a new tool for incorporating implementation data into systematic reviews and meta-analyses. J Clin Epidemiol. 2013;66(8):874-82.
70. Hoffmann TC, Glasziou P, Boutron I, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687.
71. Appelbaum M, Cooper H, Kline RB, Mayo-Wilson E, Nezu AM, Rao SM. Journal article reporting standards for quantitative research in psychology: the APA publications and communications board task force report. Am Psychol. 2018;73(1):3
72. Perera R, Heneghan C, Yudkin PA. Graphical method for depicting randomised trials of complex interventions. BMJ. 2007;334:127-9.
73. Torgerson DJ. Contamination in trials: is cluster randomisation the answer? BMJ. 2001;322:355-7.
74. Moore G, Audrey S, Barker M, et al. Process evaluation in complex public health intervention studies: the need for guidance. J Epidemiol Community Health. 2013;68(2):101-2.
75. West R. Providing full manuals and intervention descriptions: addiction policy. Addiction. 2008;103:1411.
76. Mayo-Wilson E, Fusco N, Li T, et al. Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis. J Clin Epidemiol. 2017;86:39-50.
77. Altman DG, Moher D, Schulz KF. Harms of outcome switching in reports of randomised trials: CONSORT perspective. BMJ. 2017;356:j396.
78. Chalder T, Deary V, Husain K, Walwyn R. Family-focused cognitive behaviour therapy versus psycho-education for chronic fatigue syndrome in 11- to 18-year-olds: a randomized controlled treatment trial. Psychol Med. 2010;40:1269-79.
79. Slutsky AS. Data safety and monitoring boards. N Engl J Med. 2004;350(11):1143-7.
80. Schulz KF, Grimes DA. Allocation concealment in randomised trials: defending against deciphering. Lancet. 2002;359(9306):614-8.
81. Cuijpers P, Cristea IA. How to prove that your therapy is effective, even when it is not: a guideline. Epidemiol Psychiatr Sci. 2016;25(5):428-35.
82. Wilson DB. Comment on 'developing a reporting guideline for social and psychological intervention trials'. J Exp Criminol. 2013;9:375-7.
83. Little RJ, Rubin DB. Statistical analysis with missing data. 2nd ed. Hoboken: Wiley; 2002.
84. Li T, Hutfless S, Scharfstein DO, et al. Standards should be applied in the prevention and handling of missing data for patient-centered outcomes research: a systematic review and expert consensus. J Clin Epidemiol. 2014;67(1):15-32.
85. Gardner F, Hutchings J, Bywater T, Whitaker C. Who benefits and how does it work? Moderators and mediators of outcome in an effectiveness trial of a parenting intervention. J Clin Child Adolesc Psychol. 2010;39(4):568-80.
86. Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol. 2012;12(1):181.
87. Boutron I, Altman D, Moher D, Schulz KF, Ravaud P. CONSORT statement for randomized trials of nonpharmacologic treatments: a 2017 update and a CONSORT extension for nonpharmacologic trial abstracts. Ann Intern Med. 2017;167(1):40-7.
88. Welch V, Jull J, Petkovic J, et al. Protocol for the development of a CONSORT-equity guideline to improve reporting of health equity in randomized trials. Implement Sci. 2015;10(1):146.
89. Welch V, Petticrew M, Tugwell P, et al. PRISMA-equity 2012 extension: reporting guidelines for systematic reviews with a focus on health equity. PLoS Med. 2012;9(10):e1001333.
90. O'Neill J, Tabish H, Welch V, et al. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J Clin Epidemiol. 2014;67(1):56-64.
91. Knol MJ, Groenwold R, Grobbee DE. P-values in baseline tables of randomised controlled trials are inappropriate but still common in high impact journals. Eur J Prev Cardiol. 2012;19:231-2.
92. Piwowar HA, Day RS, Fridsma DB. Sharing detailed research data is associated with increased citation rate. PLoS One. 2007;2:e308.
93. Taichman DB, Sahni P, Pinborg A, et al. Data sharing statements for clinical trials: a requirement of the International Committee of Medical Journal Editors. JAMA. 2017;317(24):2491-2.
94. Lipsey MW, Wilson DB. Practical meta-analysis. Thousand Oaks: Sage; 2001.
95. McCord J. Cures that harm: unanticipated outcomes of crime prevention programs. Ann Am Acad Polit Soc Sci. 2003;587:16-30.
96. Chalmers I. Trying to do more good than harm in policy and practice: the role of rigorous, transparent, up-to-date evaluations. Ann Am Acad Polit Soc Sci. 2003;589(22):22-40.
97. Fletcher A, Jamal F, Moore G, Evans RE, Murphy S, Bonell C. Realist complex intervention science: applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions. Evaluation. 2016;22(3):286-303.
98. Moore GF, Evans RE. What theory, for whom and in which context? Reflections on the application of theory in the development and evaluation of complex population health interventions. SSM-Popul Health. 2017;3(132-5.
99. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303(20):2058-64.
100. Chiu K, Grundy Q, Bero L. 'Spin' in published biomedical literature: a methodological systematic review. PLoS Biol. 2017;15(9):e2002173.
101. World Health Organization. WHO Trial Registration Data Set (Version 12.1). 2017; http://www.who.int/ictrp/network/trds/en/. Accessed 26 Feb 2017.
102. Meinert CL. Toward prospective registration of clinical trials. Control Clin Trials. 1988;9(1):1-5.
103. Simes R. Publication bias: the case for an international registry of clinical trials. J Clin Oncol. 1986;4(10):1529-41.
104. Dickersin K. Report from the panel on the case for registers of clinical trials at the eighth annual meeting of the Society for Clinical Trials. Control Clin Trials. 1988;9(1):76-81.
105. De Angelis C, Drazen JM, Frizelle FA, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med. 2004;351(12):1250-1.
106. International Clinical Trials Registry Platform (ICTRP). Joint statement on public disclosure of results from clinical trials 2017.
107. Harrison BA, Mayo-Wilson E. Trial registration: understanding and preventing reporting bias in social work research. Res Soc Work Pract. 2014;24(3):372-6.
108. Cybulski L, Mayo-Wilson E, Grant S. Improving transparency and reproducibility through registration: the status of intervention trials published in clinical psychology journals. J Consult Clin Psychol. 2016;84(9):753-67.
109. Huth EJ. Guidelines on authorship of medical papers. Ann Intern Med. 1986;104(2):269-74.
110. Petrosino A, Soydan H. The impact of program developers as evaluators on criminal recidivism: results from meta-analyses of experimental and quasi-experimental research. J Exp Criminol. 2005;1(4):435-50.
111. Eisner M. No effects in independent prevention trials: can we reject the cynical view? J Exp Criminol. 2009;5(2):163-83.
112. Concannon TW, Meissner P, Grunbaum JA, et al. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Intern Med. 2012;27(8):985-91.
113. Keown K, Van Eerd D, Irvin E. Stakeholder engagement opportunities in systematic reviews: knowledge transfer for policy and practice. J Contin Educ Health Prof. 2008;28(2):67-72.
114. Concannon TW, Fuster M, Saunders T, et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J Gen Intern Med. 2014;29(12):1692-701.
115. Staniszewska S, Brett J, Simera I, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ. 2017;358:j3453.
116. Bower P, Brueton V, Gamble C, et al. Interventions to improve recruitment and retention in clinical trials: a survey and workshop to assess current practice and future priorities. Trials. 2014;15(1):399.
117. Gottfredson D, Cook T, Gardner F, et al. Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: next generation. Prev Sci. 2015;16(7):893-926.