Improving transparency and reproducibility through registration: The status of intervention trials published in clinical psychology journals

Journal of Consulting and Clinical Psychology

Volumn 84, Issue 9, 2016, Pages 753-767

  Cybulski, Lukasz ^a   Mayo Wilson, Evan ^b   Grant, Sean ^c  

^a KAROLINSKA INSTITUTE   (Sweden)

^b JOHNS HOPKINS BLOOMBERG SCHOOL OF PUBLIC HEALTH   (United States)

^c RAND CORPORATION   (United States)

Author keywords

intervention research; randomized controlled trials; reporting bias; reporting guidelines; trial registration

Indexed keywords

ARTICLE; BEHAVIOR THERAPY; CLINICAL PSYCHOLOGY; DRUG DEPENDENCE TREATMENT; EATING DISORDER; GERONTOPSYCHIATRY; HUMAN; INTERVENTION STUDY; JOURNAL IMPACT FACTOR; MEDICAL LITERATURE; POLICY; PREVALENCE; PSYCHOLOGIC ASSESSMENT; PSYCHOLOGY; PSYCHOTHERAPY; PUBLICATION; RANDOMIZED CONTROLLED TRIAL (TOPIC); REGISTRATION; REPRODUCIBILITY; UNITED STATES; VALIDITY; CLINICAL TRIAL (TOPIC); REGISTER;

CLINICAL TRIALS AS TOPIC; HUMANS; PSYCHOLOGY, CLINICAL; REGISTRIES; REPRODUCIBILITY OF RESULTS;

EID: 84973502226     PISSN: 0022006X     EISSN: 19392117     Source Type: Journal    
DOI: 10.1037/ccp0000115     Document Type: Article

References (116)

1. Altman, D. G., Furberg, C. D., Grimshaw, J. M., & Shanahan, D. R. (2014). Linked publications from a single trial: A thread of evidence. Trials, 15, 369. http://dx.doi.org/10.1186/1745-6215-15-369
2. American Psychological Association Publications and the APA Publications and Communications Board Working Group on Journal Article Reporting Standards. (2008). Reporting standards for research in psychology: Why do we need them? What might they be? American Psychologist, 63, 839-851. http://dx.doi.org/10.1037/0003-066X.63.9.839
3. Ames, D. (2006). Registration of clinical trials submitted for publication in International Psychogeriatrics. International Psychogeriatrics, 18, 191-193. http://dx.doi.org/10.1017/S1041610206003413
4. Anderson, M. L., Chiswell, K., Peterson, E. D., Tasneem, A., Topping, J., & Califf, R. M. (2015). Compliance with results reporting at Clinical-Trials.gov. The New England Journal of Medicine, 372, 1031-1039. http://dx.doi.org/10.1056/NEJMsa1409364
5. Antes, G., & Chalmers, I. (2003). Under-reporting of clinical trials is unethical. Lancet, 361, 978-979. http://dx.doi.org/10.1016/S0140-6736(03)12838-3
6. APA Data Sharing Working Group. (2015). Data sharing: Principles and considerations for policy development. Retrieved from www.apa.org/science/about/psa/2015/07/data-sharing.aspx
7. Azar, M., Riehm, K. E., McKay, D., & Thombs, B. D. (2015). Transparency of outcome reporting and trial registration of randomized controlled trials published in the journal of consulting and clinical psychology. PLoS ONE, 10, e0142894. http://dx.doi.org/10.1371/journal.pone.0142894
8. Bakker, M., van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7, 543-554. http://dx.doi.org/10.1177/1745691612459060
9. Begg, C., Cho, M., Eastwood, S., Horton, R., Moher, D., Olkin, I., ⋯. Stroup, D. F. (1996). Improving the quality of reporting of randomized controlled trials: The CONSORT statement. JAMA: The Journal of the American Medical Association, 276, 637-639. http://dx.doi.org/10.1001/jama.1996.03540080059030
10. Bergman, R. L., Gonzalez, A., Piacentini, J., & Keller, M. L. (2013). Integrated behavior therapy for selective mutism: A randomized controlled pilot study. Behaviour Research and Therapy, 51, 680-689. http://dx.doi.org/10.1016/j.brat.2013.07.003
11. Borek, A. J., Abraham, C., Smith, J. R., Greaves, C. J., & Tarrant, M. (2015). A checklist to improve reporting of group-based behaviourchange interventions. BMC Public Health, 15, 963. http://dx.doi.org/10.1186/s12889-015-2300-6
12. Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., & Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, 365-376. http://dx.doi.org/10.1038/nrn3475
13. Center for Open Science. (2015a). Badges to acknowledge open practice. Retrieved from http://osf.io/tvyxz/
14. Center for Open Science. (2015b). Registered reports. Retrieved from https://osf.io/8mpji/wiki/home
15. Chalmers, I. (1990). Underreporting research is scientific misconduct. Journal of the American Medical Association, 263, 1405-1408. http://dx.doi.org/10.1001/jama.1990.03440100121018
16. Chan, A. W., & Altman, D. G. (2005). Identifying outcome reporting bias in randomised trials on PubMed: Review of publications and survey of authors. BMJ: British Medical Journal, 330, 753. http://dx.doi.org/10.1136/bmj.38356.424606.8F
17. Chan, A. W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004). Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles. JAMA: The Journal of the American Medical Association, 291, 2457-2465. http://dx.doi.org/10.1001/jama.291.20.2457
18. Chan, A. W., Krleza-Jerić, K., Schmid, I., & Altman, D. G. (2004). Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. Canadian Medical Association Journal, 171, 735-740. http://dx.doi.org/10.1503/cmaj.1041086
19. Chan, A. W., Song, F., Vickers, A., Jefferson, T., Dickersin, K., Gøtzsche, P. C., ⋯. van der Worp, H. B. (2014). Increasing value and reducing waste: Addressing inaccessible research. Lancet, 383, 257-266. http://dx.doi.org/10.1016/S0140-6736(13)62296-5
20. Chan, A. W., Tetzlaff, J. M., Altman, D. G., Laupacis, A., Gøtzsche, P. C., Krleža-Jerić, K., ⋯ Moher, D. (2013). SPIRIT 2013 statement: Defining standard protocol items for clinical trials. Annals of Internal Medicine, 158, 200-207. http://dx.doi.org/10.7326/0003-4819-158-3-201302050-00583
21. Chavalarias, D., Wallach, J. D., Li, A. H., & Ioannidis, J. P. A. (2016). Evolution of reporting p values in the biomedical literature, 1990-2015. Journal of the American Medical Association, 315, 1141-1148. http://dx.doi.org/10.1001/jama.2016.1952
22. Cooper, H., DeNeve, K., & Charlton, K. (1997). Finding the missing science: The fate of studies submitted for review by a human subjects committee. Psychological Methods, 2, 447-452. http://dx.doi.org/10.1037/1082-989X.2.4.447
23. Council of Science Editors. (2006). ICMJE's statement on clinical trial registration. Retrieved from http://www.councilscienceeditors.org/resource-library/editorial-policies/cse-policies/other-supported-statements/icmjes-statement-onclinical-trial-registration/
24. Cuijpers, P., Smit, F., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2010). Efficacy of cognitive-behavioural therapy and other psychological treatments for adult depression: Meta-analytic study of publication bias. The British Journal of Psychiatry, 196, 173-178. http://dx.doi.org/10.1192/bjp.bp.109.066001
25. Cuijpers, P., van Straten, A., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2010). The effects of psychotherapy for adult depression are overestimated: A meta-analysis of study quality and effect size. Psychological Medicine, 40, 211-223. http://dx.doi.org/10.1017/S003 3291709006114
26. Dal-Ré, R., Bracken, M. B., & Ioannidis, J. P. (2015). Call to improve transparency of trials of non-regulated interventions. British Medical Journal, 350, h1323. http://dx.doi.org/10.1136/bmj.h1323
27. De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., ⋯ Van Der Weyden, M. B. (2004). Clinical trial registration: A statement from the International Committee of Medical Journal Editors. The New England Journal of Medicine, 351, 1250-1251. http://dx.doi.org/10.1056/NEJMe048225
28. De Angelis, C. D., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., ⋯ Van Der Weyden, M. B. (2005). Is this clinical trial fully registered?-A statement from the International Committee of Medical Journal Eds. The New England Journal of Medicine, 352, 2436-2438. http://dx.doi.org/10.1056/NEJMe058127
29. Dickersin, K. (1992). Why register clinical trials?-Revisited. Controlled Clinical Trials, 13, 170-177. http://dx.doi.org/10.1016/0197-2456(92)90022-R
30. Dickersin, K., Manheimer, E., Wieland, S., Robinson, K. A., Lefebvre, C., & McDonald, S. (2002). Development of the Cochrane Collaboration's CENTRAL Register of controlled clinical trials. Evaluation & the Health Professions, 25, 38-64. http://dx.doi.org/10.1177/0163 278702025001004
31. Dickersin, K., Min, Y. I., & Meinert, C. L. (1992). Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. Journal of the American Medical Association, 267, 374-378. http://dx.doi.org/10.1001/jama.1992.034 80030052036
32. Driessen, E., Hollon, S. D., Bockting, C. L., Cuijpers, P., & Turner, E. H. (2015). Does publication bias inflate the apparent efficacy of psychological treatment for major depressive disorder? A systematic review and meta-analysis of U.S. National Institutes of Health-Funded Trials. PLoS ONE, 10, e0137864. http://dx.doi.org/10.1371/journal.pone.0137864
33. Dwan, K., Gamble, C., Williamson, P. R., & Kirkham, J. J. (2013). Systematic review of the empirical evidence of study publication bias and outcome reporting bias: An updated review. PLoS ONE, 8, e66844. http://dx.doi.org/10.1371/journal.pone.0066844
34. Eich, E. (2014). Business not as usual. Psychological Science, 25, 3-6. http://dx.doi.org/10.1177/0956797613512465
35. Emerson, G. B., Warme, W. J., Wolf, F. M., Heckman, J. D., Brand, R. A., & Leopold, S. S. (2010). Testing for the presence of positive-outcome bias in peer review: A randomized controlled trial. Archives of Internal Medicine, 170, 1934-1939. http://dx.doi.org/10.1001/archinternmed.2010.406
36. EQUATOR Network. (2015). Tookits. Retrieved from www.equatornetwork.org/toolkits/
37. Faggiano, F., Allara, E., Giannotta, F., Molinar, R., Sumnall, H., Wiers, R., ⋯ Conrod, P. (2014). Europe needs a central, transparent, and evidencebased approval process for behavioural prevention interventions. PLoS Medicine, 11, e1001740. http://dx.doi.org/10.1371/journal.pmed.1001740
38. Fanelli, D. (2012). Negative results are disappearing from most disciplines and countries. Sociometrics, 90, 901-904.
39. Food and Drug Administration Amendments Act, 42 U.S. C. § 801 (2007).
40. Food and Drug Administration Modernization Act, 21 U.S. C. § 301 (1997).
41. Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., ⋯ Wager, E. (2014). Reducing waste from incomplete or unusable reports of biomedical research. Lancet, 383, 267-276. http://dx.doi.org/10.1016/S0140-6736(13)62228-X
42. Goldacre, B. (2015). How to get all trials reported: Audit, better data, and individual accountability. PLoS Medicine, 12, e1001821. http://dx.doi.org/10.1371/journal.pmed.1001821
43. Gonzales, J. E., & Cunningham, C. A. (2015). The promise of preregistration in psychological research. Retrieved from http://www.apa.org/science/about/psa/2015/08/pre-registration.aspx
44. Gorman, D. M. (2015). "Everything works:" The need to address confirmation bias in evaluations of drug misuse prevention interventions for adolescents. Addiction, 110, 1539-1540. http://dx.doi.org/10.1111/add.12954
45. Gottfredson, D. C., Cook, T. D., Gardner, F. E., Gorman-Smith, D., Howe, G. W., Sandler, I. N., & Zafft, K. M. (2015). Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: Next generation. Prevention Science, 16, 893-926. http://dx.doi.org/10.1007/s11121-015-0555-x
46. Grant, S. P. (2014). Developing of a CONSORT extension for social and psychological interventions. Retrieved from http://ora.ox.ac.uk/objects/uuuid:c1bd46df-eb3f-4dc6-9cc1-38c26a5661a9
47. Grant, S. P., Mayo-Wilson, E., Melendez-Torres, G. J., & Montgomery, P. (2013). Reporting quality of social and psychological intervention trials: A systematic review of reporting guidelines and trial publications. PLoS ONE, 8, e65442. http://dx.doi.org/10.1371/journal.pone.0065442
48. Institute of Medicine. (2015a). Psychosocial interventions for mental and substance use disorders: A framework for establishing evidence-based standards. Washington, DC: Institute of Medicine.
49. Institute of Medicine. (2015b). Sharing clinical trial data: maximizing benefits. Washington, DC: Minimizing Risk.
50. Ioannidis, J. P. (2005). Why most published research findings are false. PLoS Medicine, 2, e124. http://dx.doi.org/10.1371/journal.pmed.0020124
51. Ioannidis, J. P. (2008). Why most discovered true associations are inflated. Epidemiology, 19, 640-648. http://dx.doi.org/10.1097/EDE.0b0 13e31818131e7
52. Ioannidis, J. P. (2012). Why science is not necessarily self-correcting. Perspectives on Psychological Science, 7, 645-654. http://dx.doi.org/10.1177/1745691612464056
53. Ioannidis, J. P., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014). Publication and other reporting biases in cognitive sciences: Detection, prevalence, and prevention. Trends in Cognitive Sciences, 18, 235-241. http://dx.doi.org/10.1016/j.tics.2014.02.010
54. Ironson, G., O'Cleirigh, C., Leserman, J., Stuetzle, R., Fordiani, J., Fletcher, M., & Schneiderman, N. (2013). Gender-specific effects of an augmented written emotional disclosure intervention on posttraumatic, depressive, and HIV-disease-related outcomes: A randomized, controlled trial. Journal of Consulting and Clinical Psychology, 81, 284-298. http://dx.doi.org/10.1037/a0030814
55. Journal Citation Reports. (2013). Retrieved from http://login.webofknowledge.com
56. Kaplan, R. M., & Irvin, V. L. (2015). Likelihood of null effects of large NHLBI clinical trials has increased over time. PLoS ONE, 10, e0132382. http://dx.doi.org/10.1371/journal.pone.0132382
57. Kim, C., & Prasad, V. (2015). Cancer drugs approved on the basis of a surrogate end point and subsequent overall survival: An analysis of 5 years of U.S. Food and Drug Administration approvals. Journal of the American Medical Association: Internal Medicine, 175, 1992-1994.
58. Knüppel, H., Metz, C., Meerpohl, J. J., & Strech, D. (2013). How psychiatry journals support the unbiased translation of clinical research. A cross-sectional study of editorial policies. PLoS ONE, 8, e75995. http://dx.doi.org/10.1371/journal.pone.0075995
59. Kühberger, A., Fritz, A., & Scherndl, T. (2014). Publication bias in psychology: A diagnosis based on the correlation between effect size and sample size. PLoS ONE, 9, e105825. http://dx.doi.org/10.1371/journal.pone.0105825
60. Kyriacou, D. N. (2016). The enduring evolution of the p value. Journal of the American Medical Association, 315, 1113-1115. http://dx.doi.org/10.1001/jama.2016.2152
61. Lee, S. W., & Kwon, J. H. (2013). The efficacy of imagery rescripting (IR) for social phobia: A randomized controlled trial. Journal of Behavior Therapy and Experimental Psychiatry, 44, 351-360. http://dx.doi.org/10.1016/j.jbtep.2013.03.001
62. Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P., ⋯ Glasziou, P. (2014). Biomedical research: Increasing value, reducing waste. Lancet, 383, 101-104. http://dx.doi.org/10.1016/S0140-6736(13)62329-6
63. Marin Dos Santos, D. H., & Atallah, A. N. (2015). FDAAA legislation is working, but methodological flaws undermine the reliability of clinical trials: A cross-sectional study. Peer J, 3, e1015. http://dx.doi.org/10.7717/peerj.1015
64. Masicampo, E. J., & Lalande, D. R. (2012). A peculiar prevalence of p values just below.05. Quarterly Journal of Experimental Psychology, 65, 2271-2279. http://dx.doi.org/10.1080/17470218.2012.711335
65. Mathieu, S., Boutron, I., Moher, D., Altman, D. G., & Ravaud, P. (2009). Comparison of registered and published primary outcomes in randomized controlled trials. JAMA: The Journal of the American Medical Association, 302, 977-984. http://dx.doi.org/10.1001/jama.2009.1242
66. Mayo-Wilson, E., Montgomery, P., Hopewell, S., Macdonald, G., Moher, D., & Grant, S. (2013). Developing a reporting guideline for social and psychological intervention trials. The British Journal of Psychiatry, 203, 250-254. http://dx.doi.org/10.1192/bjp.bp.112.123745
67. Meinert, C. L. (1988). Toward prospective registration of clinical trials. Controlled Clinical Trials, 9, 1-5. http://dx.doi.org/10.1016/0197-2456(88)90002-5
68. Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., ⋯ Van der Laan, M. (2014). Social science. Promoting transparency in social science research. Science, 343, 30-31. http://dx.doi.org/10.1126/science.1245317
69. Milette, K., Roseman, M., & Thombs, B. D. (2011). Transparency of outcome reporting and trial registration of randomized controlled trials in top psychosomatic and behavioral health journals: A systematic review. Journal of Psychosomatic Research, 70, 205-217. http://dx.doi.org/10.1016/j.jpsychores.2010.09.015
70. Miller, J. E., Korn, D., & Ross, J. S. (2015). Clinical trial registration, reporting, publication and FDAAA compliance: A cross-sectional analysis and ranking of new drugs approved by the FDA in 2012. British Medical Journal Open, 5, e009758. http://dx.doi.org/10.1136/bmjopen- 2015-009758
71. Moher, D., Glasziou, P., Chalmers, I., Nasser, M., Bossuyt, P. M., Korevaar, D. A., ⋯ Boutron, I. (2015). Increasing value and reducing waste in biomedical research: Who's listening? Lancet. Advance online publication. http://dx.doi.org/10.1016/S0140-6736(15)00307-4
72. Montgomery, P., Grant, S., Hopewell, S., Macdonald, G., Moher, D., Michie, S., & Mayo-Wilson, E. (2013). Protocol for CONSORT-SPI: An extension for social and psychological interventions. Implementation Science; IS, 8, 99. http://dx.doi.org/10.1186/1748-5908-8-99
73. Montgomery, P., Underhill, K., Gardner, F., Operario, D., & Mayo-Wilson, E. (2013). The Oxford Implementation Index: A new tool for incorporating implementation data into systematic reviews and meta-analyses. Journal of Clinical Epidemiology, 66, 874-882. http://dx.doi.org/10.1016/j.jclinepi.2013.03.006
74. Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., ⋯ Baird, J. (2015). Process evaluation of complex interventions: Medical Research Council guidance. British Medical Journal, 350, h1258. http://dx.doi.org/10.1136/bmj.h1258
75. Moorthy, V. S., Karam, G., Vannice, K. S., & Kieny, M. P. (2015). Rationale for WHO's new position calling for prompt reporting and public disclosure of interventional clinical trial results. PLoS Medicine, 12, e1001819. http://dx.doi.org/10.1371/journal.pmed.1001819
76. Motulsky, H. J. (2014). Common misconceptions about data analysis and statistics. The Journal of Pharmacology and Experimental Therapeutics, 351, 200-205. http://dx.doi.org/10.1124/jpet.114.219170
77. National Institute of Health. (2014a). NIH request for public comments on the draft NIH Policy on Dissemination of NIH-Funded Clinical Trial Information. Retrieved from grants.nih.gov/grants/guide/notice-files/NOT-OD-15-019.html
78. National Institute of Health. (2014b). Notice of revised NIH definition of "clinical trial". Retrieved from grants.nih.gov/grants/guide/notice-files/NOT-OD-15-015.html
79. National Library of Medicine. (2015a). Submit studies. Retrieved from https://www.clinicaltrials.gov/ct2/manage-recs
80. National Library of Medicine. (2015b). Training materials. Retrieved from https://clinicaltrials.gov/ct2/manage-recs/present
81. Niemeyer, H., Musch, J., & Pietrowsky, R. (2012). Publication bias in meta-analyses of the efficacy of psychotherapeutic interventions for schizophrenia. Schizophrenia Research, 138, 103-112. http://dx.doi.org/10.1016/j.schres.2012.03.023
82. Niemeyer, H., Musch, J., & Pietrowsky, R. (2013). Publication bias in meta-analyses of the efficacy of psychotherapeutic interventions for depression. Journal of Consulting and Clinical Psychology, 81, 58-74. http://dx.doi.org/10.1037/a0031152
83. Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., ⋯ Yarkoni, T. (2015). Promoting an open research culture. Science, 348, 1422-1425. http://dx.doi.org/10.1126/science.aab2374
84. Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7, 615-631. http://dx.doi.org/10.1177/1745691612459058
85. Olson, C. M., Rennie, D., Cook, D., Dickersin, K., Flanagin, A., Hogan, J.W., ⋯ Pace, B. (2002). Publication bias in editorial decision making. JAMA: The Journal of the American Medical Association, 287, 2825-2828. http://dx.doi.org/10.1001/jama.287.21.2825
86. Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, aac4716. http://dx.doi.org/10.1126/science.aac4716
87. Oransky, I. (2015). Author retracts study of changing minds on same-sex marriage after colleague admits data were faked. Retrieved from http://retractionwatch.com/2015/05/20/author-retracts-study-of-changingminds-on-same-sex-marriage-after-colleague-admits-data-were-faked/
88. Pandis, N., Shamseer, L., Kokich, V. G., Fleming, P. S., & Moher, D. (2014). Active implementation strategy of CONSORT adherence by a dental specialty journal improved randomized clinical trial reporting. Journal of Clinical Epidemiology, 67, 1044-1048. http://dx.doi.org/10.1016/j.jclinepi.2014.04.001
89. Pigott, T. D., Valentine, J. C., Polanin, J. R., Williams, R. T., & Canada, D. D. (2013). Outcome-reporting bias in education research. Educational Researcher, 42, 424-432. http://dx.doi.org/10.3102/0013189X13507104
90. Pocock, S. J., Collier, T. J., Dandreo, K. J., de Stavola, B. L., Goldman, M. B., Kalish, L. A., ⋯ McCormack, V. A. (2004). Issues in the reporting of epidemiological studies: A survey of recent practice. British Medical Journal, 329, 883. http://dx.doi.org/10.1136/bmj.38250.571088.55
91. Riehm, K. E., Azar, M., & Thombs, B. D. (2015). Transparency of outcome reporting and trial registration of randomized controlled trials in top psychosomatic and behavioral health journals: A 5-year followup. Journal of Psychosomatic Research, 79, 1-12. http://dx.doi.org/10.1016/j.jpsychores.2015.04.010
92. Rosenthal, R. (1979). The "file drawer problem" and tolerance for null results. Psychological Bulletin, 86, 638-641. http://dx.doi.org/10.1037/0033-2909.86.3.638
93. Saitz, R. (2014). The best evidence for alcohol screening and brief intervention in primary care supports efficacy, at best, not effectiveness: You say tomāto, I say tomǍto? That's not all it's about. Addiction Science and Clinical Practice, 9, 14. http://dx.doi.org/10.1186/1940-0640-9-14
94. Saldanha, I. J., Dickersin, K., Wang, X., & Li, T. (2014). Outcomes in Cochrane systematic reviews addressing four common eye conditions: An evaluation of completeness and comparability. PLoS ONE, 9, e109400. http://dx.doi.org/10.1371/journal.pone.0109400
95. Scherer, R. W., Langenberg, P., & von Elm, E. (2007). Full publication of results initially presented in abstracts. Cochrane Database of Systematic Reviews, 18, MR000005. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/17443628
96. Schulz, K. F., Altman, D. G., & Moher, D. (2010). CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. British Medical Journal, 340, c332. http://dx.doi.org/10.1136/bmj.c332
97. Scott, A., Rucklidge, J. J., & Mulder, R. T. (2015). Is mandatory prospective trial registration working to prevent publication of unregistered trials and selective outcome reporting? An observational study of five psychiatry journals that mandate prospective clinical trial registration. PLoS ONE, 10, e0133718. http://dx.doi.org/10.1371/journal.pone.0133718
98. Simera, I., Moher, D., Hirst, A., Hoey, J., Schulz, K. F., & Altman, D. G. (2010). Transparent and accurate reporting increases reliability, utility, and impact of your research: Reporting guidelines and the EQUATOR Network. BMC Medicine, 8, 24. http://dx.doi.org/10.1186/1741-7015-8-24
99. Simes, R. J. (1986). Publication bias: The case for an international registry of clinical trials. Journal of Clinical Oncology, 4, 1529-1541.
100. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366. http://dx.doi.org/10.1177/0956797611417632
101. Simons, D. J., Holcombe, A. O., & Spellman, B. A. (2014). An introduction to registered replication reports at perspectives on psychological science. Perspectives on Psychological Science, 9, 552-555. http://dx.doi.org/10.1177/1745691614543974
102. Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). p-curve: A key to the file-drawer. Journal of Experimental Psychology: General, 143, 534-547. http://dx.doi.org/10.1037/a0033242
103. Song, F., Parekh, S., Hooper, L., Loke, Y. K., Ryder, J., Sutton, A. J., ⋯ Harvey, I. (2010). Dissemination and publication of research findings: An updated review of related biases. Health Technology Assessment, 14, 1-193. http://dx.doi.org/10.3310/hta14080
104. Staines, G. L., & Cleland, C. M. (2007). Bias in meta-analytics estimates of the absolute efficacy of psychotherapy. Review of General Psychology, 11, 329-347. http://dx.doi.org/10.1037/1089-2680.11.4.329
105. Stice, E., Rohde, P., Durant, S., Shaw, H., & Wade, E. (2013). Effectiveness of peer-led dissonance-based eating disorder prevention groups: Results from two randomized pilot trials. Behaviour Research and Therapy, 51, 197-206. http://dx.doi.org/10.1016/j.brat.2013.01.004
106. Svensson, S., Menkes, D. B., & Lexchin, J. (2013). Surrogate outcomes in clinical trials: A cautionary tale. Journal of the American Medical Association: Internal Medicine, 173, 611-612. http://dx.doi.org/10.1001/jamainternmed.2013.3037
107. Thaler, K., Kien, C., Nussbaumer, B., Van Noord, M. G., Griebler, U., Klerings, I., & Gartlehner, G. (2015). Inadequate use and regulation of interventions against publication bias decreases their effectiveness: A systematic review. Journal of Clinical Epidemiology, 68, 792-802. http://dx.doi.org/10.1016/j.jclinepi.2015.01.008
108. van Assen, M. A., van Aert, R. C., Nuijten, M. B., & Wicherts, J. M. (2014). Why publishing everything is more effective than selective publishing of statistically significant results. PLoS ONE, 9, e84896. http://dx.doi.org/10.1371/journal.pone.0084896
109. Viergever, R. F., Karam, G., Reis, A., & Ghersi, D. (2014). The quality of registration of clinical trials: Still a problem. PLoS ONE, 9, e84727.
110. Wagenmakers, E. J., Wetzels, R., Borsboom, D., van der Maas, H. L., & Kievit, R. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7, 632-638. http://dx.doi.org/10.1177/1745691612463078
111. Wager, E., & Kleiniert, S. (2010). Responsible research publication: International standards for authors. Retrieved from publicationethics.org/resources/international-standards-for-Eds.-and-authors
112. Wasserstein, R. L., & Lazar, N. A. (2016). The ASA's statement on p-values: Context, process, and purpose. The American Statistician. Advance online publication. http://dx.doi.org/10.1080/00031305.2016.1154108
113. World Health Organization. (2015). WHO statement on public disclosure of clinical trial results. Retrieved from http://www.who.int/ictrp/results/WHO-Statement-results-reporting-clinical-trials.pdf?ua-1
114. Zarin, D. A. (2014). ClinicalTrials.gov webinar series: Outcome measures and statistical analyses module. Retrieved from http://prsinfo.clinicaltrials.gov/webinars/module7/index.html
115. Zarin, D. A., & Keselman, A. (2007). Registering a clinical trial in ClinicalTrials.gov. Chest, 131, 909-912. http://dx.doi.org/10.1378/chest.06-2450
116. Zarin, D. A., Tse, T., Williams, R. J., Califf, R. M., & Ide, N. C. (2011). The ClinicalTrials.gov results database-Update and key issues. The New England Journal of Medicine, 364, 852-860. http://dx.doi.org/10.1056/NEJMsa1012065