The researchers' view of scientific rigor-survey on the conduct and reporting of in vivo research

PLoS ONE

Volumn 11, Issue 12, 2016, Pages

  Reichlin, Thomas S ^a   Vogt, Lucile ^a   Würbel, Hanno ^a  

^a UNIVERSITY OF BERN   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CONSTRUCT VALIDITY; CONTROLLED STUDY; EXTERNAL VALIDITY; HUMAN; IN VIVO STUDY; INTERVIEW; MEDICAL EDUCATION; MEDICAL RESEARCH; PRACTICE GUIDELINE; RELIABILITY; REPRODUCIBILITY; SELF REPORT; SWITZERLAND; ANIMAL; BIOLOGICAL MODEL; PUBLICATION;

ANIMALS; BIOMEDICAL RESEARCH; GUIDELINES AS TOPIC; HUMANS; MODELS, BIOLOGICAL; PERIODICALS AS TOPIC; SWITZERLAND;

EID: 85000949464     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0165999     Document Type: Article

References (63)

1. Prinz F, Schlange T, Asadullah K. Believe it or not: How much can we rely on published data on potential drug targets? Nat. Rev. Drug Discov. [Internet]. Nature Publishing Group; 2011 [cited 2013 Feb 27]; 10:712. Available: http://www.ncbi.nlm.nih.gov/pubmed/21892149 doi: 10.1038/nrd3439-c1 PMID: 21892149
2. Begley CG, Ellis LM. Raise standards for preclinical cancer research. Nature. 2012; 483:531-3. doi: 10. 1038/483531a PMID: 22460880
3. Howells DW, Sena ES, Macleod MR. Bringing rigour to translational medicine. Nat. Rev. Neurol. [Internet]. Nature Publishing Group; 2014 [cited 2014 Jan 29]; 10:37-43. Available: http://www.ncbi.nlm.nih. gov/pubmed/24247324 doi: 10.1038/nrneurol.2013.232 PMID: 24247324
4. Kola I, Landis J. Can the pharmaceutical industry reduce attrition rates? Nat. Rev. Drug Discov. [Internet]. 2004; 3:711-5. Available: http://www.ncbi.nlm.nih.gov/pubmed/15286737 doi: 10.1038/nrd1470 PMID: 15286737
5. van der Worp HB, Howells DW, Sena ES, Porritt MJ, Rewell S, O'Collins V, et al. Can animal models of disease reliably inform human studies? PLoS Med. [Internet]. 2010; 7:e1000245. Available: http://www. plosmedicine.org/article/fetchObject.action?uri=info%253Adoi%252F10.1371%252Fjournal.pmed. 1000245&representation=PDF doi: 10.1371/journal.pmed.1000245 PMID: 20361020
6. Editor. Further confirmation needed. Nat. Biotechnol. 2012; 30:806. doi: 10.1038/nbt.2335 PMID: 22965029
7. McNutt M. Reproducibility. Science (80-.). 2014; 343:229.
8. Richter SH, Garner JP, Würbel H. Environmental standardization: Cure or cause of poor reproducibility in animal experiments? Nat. Methods. 2009; 6:257-61. doi: 10.1038/nmeth.1312 PMID: 19333241
9. Bailoo JD, Reichlin TS, Würbel H. Refinement of experimental design and conduct in laboratory animal research. ILAR J. [Internet]. 2014 [cited 2014 Dec 29]; 55:383-91. Available: http://ilarjournal. oxfordjournals.org/cgi/doi/10.1093/ilar/ilu037 doi: 10.1093/ilar/ilu037 PMID: 25541540
10. Collins FS, Tabak LA. Policy: NIH plans to enhance reproducibility. Nature [Internet]. 2014 [cited 2014 Jun 3]; 505:612-3. Available: http://europepmc.org/abstract/MED/24482835 PMID: 24482835
11. Festing MFW. We are not born knowing how to design and analyse scientific experiments. Altern. to Lab. Anim. ATLA. 2013; 41:19-21.
12. Fanelli D. Do pressures to publish increase scientists' bias? An empirical support from US States data. PLoS One. 2010; 5:e10271. doi: 10.1371/journal.pone.0010271 PMID: 20422014
13. Fang FC, Steen RG, Casadevall A. Misconduct accounts for the majority of retracted scientific publications. Proc. Natl. Acad. Sci. [Internet]. 2012 [cited 2013 Jun 7]; 110:1136-7. Available: http://www.pnas. org/cgi/doi/10.1073/pnas.1220833110
14. Ioannidis JPA. Why most published research findings are false. PLoS Med. [Internet]. 2005 [cited 2013May 21]; 2:e124. Available: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1182327&tool= pmcentrez&rendertype=abstract
15. Macleod MR. Why animal research needs to improve. Nature. 2011; 477:511. doi: 10.1038/477511a PMID: 21956292
16. Ransohoff DF. Bias as a threat to the validity of cancer molecular-marker research. Nat. Rev. Cancer. 2005; 5:142-9. doi: 10.1038/nrc1550 PMID: 15685197
17. Henderson VC, Kimmelman J, Fergusson D, Grimshaw JM, Hackam DG. Threats to validity in the design and conduct of preclinical efficacy studies: A systematic review of guidelines for in vivo animal experiments. PLoS Med. [Internet]. 2013 [cited 2014 Feb 20]; 10:e1001489. Available: http://www. pubmedcentral.nih.gov/articlerender.fcgi?artid=3720257&tool=pmcentrez&rendertype=abstract doi: 10.1371/journal.pmed.1001489 PMID: 23935460
18. Steckler T. Editorial: preclinical data reproducibility for R&D-the challenge for neuroscience. Springerplus [Internet]. 2015; 4:1. Available: http://www.ncbi.nlm.nih.gov/pubmed/25674489\nhttp://www.ncbi. nlm.nih.gov/pmc/articles/PMC4320139/pdf/40064-2014-Article-1534.pdf doi: 10.1186/2193-1801-4-1 PMID: 25674489
19. Freedman LP, Cockburn IM, Simcoe TS. The economics of reproducibility in preclinical research. PLoS Biol. [Internet]. 2015; 13:e1002165. Available: http://dx.plos.org/10.1371/journal.pbio.1002165 doi: 10. 1371/journal.pbio.1002165 PMID: 26057340
20. Freedman LP, Gibson MC. The impact of pharmacogenomics research on drug development. Clin. Pharmacol. Ther. 2015; 97:16-8. doi: 10.1002/cpt.9 PMID: 25670378
21. O'Collins VE, Macleod MR, Donnan GA, Horky LL, van der Worp HB, Howells DW. 1,026 experimental treatments in acute stroke. Ann. Neurol. [Internet]. 2006 [cited 2013 Jun 6]; 59:467-77. Available: http:// www.ncbi.nlm.nih.gov/pubmed/16453316 doi: 10.1002/ana.20741 PMID: 16453316
22. Scannell JW, Blanckley A, Boldon H, Warrington B. Diagnosing the decline in pharmaceutical R&D efficiency. Nat. Rev. Drug Discov. [Internet]. Nature Publishing Group; 2012; 11:191-200. Available: http:// www.ncbi.nlm.nih.gov/pubmed/22378269 doi: 10.1038/nrd3681 PMID: 22378269
23. Henderson VC, Demko N, Hakala A, MacKinnon N, Federico CA, Fergusson D, et al. A meta-analysis of threats to valid clinical inference in preclinical research of sunitinib. Elife [Internet]. 2015; 4:1-13.Available: http://elifesciences.org/lookup/doi/10.7554/eLife.08351
24. Sena ES, van der Worp HB, Howells DW, Macleod MR. How can we improve the pre-clinical development of drugs for stroke? Trends Neurosci. [Internet]. 2007 [cited 2013 May 15]; 30:433-9. Available: http://www.ncbi.nlm.nih.gov/pubmed/17765332 doi: 10.1016/j.tins.2007.06.009 PMID: 17765332
25. McCann SK, Irvine C, Mead GE, Sena ES, Currie GL, Egan KE, et al. Efficacy of antidepressants in animal models of ischemic stroke: A systematic review and meta-analysis. Stroke [Internet]. 2014; 45:3055-63. Available: http://stroke.ahajournals.org/cgi/doi/10.1161/STROKEAHA.114.006304 doi: 10.1161/STROKEAHA.114.006304 PMID: 25184357
26. Macleod MR, Lawson McLean A, Kyriakopoulou A, Serghiou S, de Wilde A, Sherratt N, et al. Risk of bias in reports of in vivo research: A focus for improvement. PLoS Biol. [Internet]. 2015; 13:e1002273. Available: http://dx.plos.org/10.1371/journal.pbio.1002273 doi: 10.1371/journal.pbio.1002273 PMID: 26460723
27. Frantzias J, Sena ES, Macleod MR, Al-Shahi Salman R. Treatment of intracerebral hemorrhage in animal models: meta-analysis. Ann. Neurol. [Internet]. 2011 [cited 2013 Jun 28]; 69:389-99. Available: http://www.ncbi.nlm.nih.gov/pubmed/21387381 doi: 10.1002/ana.22243 PMID: 21387381
28. Rooke EDM, Vesterinen HM, Sena ES, Egan KJ, Macleod MR. Dopamine agonists in animal models of Parkinson's disease: A systematic review and meta-analysis. Parkinsonism Relat. Disord. [Internet]. Elsevier Ltd; 2011 [cited 2013 Jun 26]; 17:313-20. Available: http://www.ncbi.nlm.nih.gov/pubmed/ 21376651 doi: 10.1016/j.parkreldis.2011.02.010 PMID: 21376651
29. Vesterinen HM, Sena ES, Ffrench-Constant C, Williams A, Chandran S, Macleod MR. Improving the translational hit of experimental treatments in multiple sclerosis. Mult. Scler. 2010; 16:1044-55. doi: 10. 1177/1352458510379612 PMID: 20685763
30. Currie GL, Delaney A, Bennett MI, Dickenson AH, Egan KJ, Vesterinen HM, et al. Animal models of bone cancer pain: Systematic review and meta-analyses. Pain [Internet]. International Association for the Study of Pain; 2013 [cited 2013 Aug 7]; 154:917-26. Available: http://www.ncbi.nlm.nih.gov/ pubmed/23582155 doi: 10.1016/j.pain.2013.02.033 PMID: 23582155
31. Hirst JA, Howick J, Aronson JK, Roberts N, Perera R, Koshiaris C, et al. The need for randomization in animal trials: An overview of systematic reviews. PLoS One. 2014; 9.
32. Macleod MR, O'Collins T, Howells DW, Donnan GA. Pooling of animal experimental data reveals influence of study design and publication bias. Stroke [Internet]. 2004 [cited 2013 Apr 29]; 35:1203-8. Available: http://www.ncbi.nlm.nih.gov/pubmed/15060322 doi: 10.1161/01.STR.0000125719.25853.20 PMID: 15060322
33. Kilkenny C, Parsons N, Kadyszewski E, Festing MFW, Cuthill IC, Fry D, et al. Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLoS One [Internet]. 2009 [cited 2013 May 22]; 4:e7824. Available: http://www.pubmedcentral.nih.gov/articlerender.fcgi? artid=2779358&tool=pmcentrez&rendertype=abstract doi: 10.1371/journal.pone.0007824 PMID: 19956596
34. Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research. PLoS Biol. [Internet]. 2010 [cited 2013 May 24]; 8:e1000412. Available: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2893951&tool= pmcentrez&rendertype=abstract doi: 10.1371/journal.pbio.1000412 PMID: 20613859
35. Baker D, Lidster K, Sottomayor A, Amor S. Two years later: Journals are not yet enforcing the ARRIVE guidelines on reporting standards for pre-clinical animal studies. Eisen JA, editor. PLoS Biol. [Internet]. Public Library of Science; 2014 [cited 2014 May 26]; 12:e1001756. Available: http://dx.plos.org/10. 1371/journal.pbio.1001756 doi: 10.1371/journal.pbio.1001756 PMID: 24409096
36. Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, Robinson ESJ, et al. Power failure: Why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. [Internet]. Nature Publishing Group; 2013 [cited 2013 Nov 6]; 14:365-76. Available: http://www.ncbi.nlm.nih.gov/pubmed/23571845 doi: 10.1038/nrn3475 PMID: 23571845
37. Nuzzo R. Statistical errors. Nature. 2014; 506:150-2. doi: 10.1038/506150a PMID: 24522584
38. Vaux DL. Know when your numbers are significant. Nature. 2012; 492:180-1. doi: 10.1038/492180a PMID: 23235861
39. Begley CG, Ioannidis JPA. Reproducibility in science: Improving the standard for basic and preclinical research. Circ. Res. [Internet]. 2015; 116:116-26. Available: http://circres.ahajournals.org/cgi/doi/10. 1161/CIRCRESAHA.114.303819 doi: 10.1161/CIRCRESAHA.114.303819 PMID: 25552691
40. Macleod MR, van der Worp HB, Sena ES, Howells DW, Dirnagl U, Donnan GA. Evidence for the efficacy of NXY-059 in experimental focal cerebral ischaemia is confounded by study quality. Stroke [Internet]. 2008 [cited 2013 Mar 11]; 39:2824-9. Available: http://www.ncbi.nlm.nih.gov/pubmed/18635842 doi: 10.1161/STROKEAHA.108.515957 PMID: 18635842
41. Pound P, Bracken MB. Is animal research sufficiently evidence based to be a cornerstone of biomedical research? BMJ Br. Med. J. [Internet]. 2014 [cited 2014 Sep 4]; 348:g3387. Available: http://www.ncbi. nlm.nih.gov/pubmed/24879816
42. McNutt M. Journals unite for reproducibility. Nature. 2014; 515:7.
43. LimeSurvey Project Team. LimeSurvey: An open source survey tool [Internet]. Schmitz C, editor. Hamburg, Germany: LimeSurvey Project; 2012. Available: http://www.limesurvey.org
44. Burnham KP, Anderson DR. Multimodel inference: Understanding AIC and BIC in model selection. Sociol. Methods Res. 2004; 33:261-304.
45. Aho K, Derryberry D, Peterson T. Model selection for ecologists: The worldviews of AIC and BIC. Ecology. 2014; 95:631-6. PMID: 24804445
46. Barton K. MuMIn: Multi-model inference [Internet]. 2014. Available: http://cran.r-project.org/package= MuMIn
47. R Core Team. R: A language and environment for statistical computing [Internet]. Vienna, Austria: R Foundation for Statistical Computing; 2014. Available: http://www.r-project.org/
48. Minnerup J, Zentsch V, Schmidt A, Fisher M, Schäbitz W- R. Methodological quality of experimental stroke studies published in the Stroke Journal. Stroke [Internet]. 2016; 47:267-72. Available: http:// stroke.ahajournals.org/lookup/doi/10.1161/STROKEAHA.115.011695 doi: 10.1161/STROKEAHA.115. 011695 PMID: 26658439
49. Ehrlinger J, Dunning D. How chronic self-views influence (and potentially mislead) estimates of performance. J. Pers. Soc. Psychol. 2003; 84:5-17. PMID: 12518967
50. Kruger J, Dunning D. Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments. J. Pers. Soc. Psychol. 1999; 7:1121-34.
51. Bello S, Krogsbøll LT, Gruber J, Zhao ZJ, Fischer D, Hróbjartsson A. Lack of blinding of outcome assessors in animal model experiments implies risk of observer bias. J. Clin. Epidemiol. [Internet]. 2014 [cited 2014 Aug 25]; 67:973-83. Available: http://www.ncbi.nlm.nih.gov/pubmed/24972762 doi: 10.1016/j. jclinepi.2014.04.008 PMID: 24972762
52. Howells DW, Sena ES, O'Collins V, Macleod MR. Improving the efficiency of the development of drugs for stroke. Int. J. Stroke [Internet]. 2012 [cited 2013 Jun 4]; 7:371-7. Available: http://www.ncbi.nlm.nih. gov/pubmed/22712738 doi: 10.1111/j.1747-4949.2012.00805.x PMID: 22712738
53. Landis SC, Amara SG, Asadullah K, Austin CP, Blumenstein R, Bradley EW, et al. A call for transparent reporting to optimize the predictive value of preclinical research. Nature [Internet]. Nature Publishing Group; 2012 [cited 2013 Feb 27]; 490:187-91. Available: http://www.ncbi.nlm.nih.gov/pubmed/ 23060188 doi: 10.1038/nature11556 PMID: 23060188
54. Cressey D. Surge in support for animal-research guidelines. Nature. News [Internet]. 2016; Available: http://www.nature.com/doifinder/10.1038/nature.2016.19274
55. Kontinen VK. From clear reporting to better research models. Scand. J. Pain [Internet]. Elsevier B.V.; 2013 [cited 2013 Oct 9]; 4:57. Available: http://linkinghub.elsevier.com/retrieve/pii/ S1877886013000074
56. Editor. Reducing our irreproducibility. Nature. 2013; 496:198.
57. Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet [Internet]. Elsevier Ltd; 2014 [cited 2014 Jan 20]; 383:267-76. Available: http://www.ncbi.nlm.nih.gov/pubmed/24411647 doi: 10.1016/S0140-6736 (13)62228-X PMID: 24411647
58. Kontinen V. Raising the standards of preclinical pain studies. Scand. J. Pain [Internet]. Elsevier B.V.; 2015; 7:38-9. Available: http://linkinghub.elsevier.com/retrieve/pii/S1877886015000063
59. Mullane K, Enna SJ, Piette J, Williams M. Guidelines for manuscript submission in the peer-reviewed pharmacological literature. Biochem. Pharmacol. [Internet]. 2015; 97:225-35. Available: http://www. sciencedirect.com/science/article/pii/S0006295215003585 doi: 10.1016/j.bcp.2015.06.023 PMID: 26208784
60. Würbel H. Behaviour and the standardization fallacy. Nat. Genet. [Internet]. 2000; 26:263. Available: http://www.ncbi.nlm.nih.gov/pubmed/11062457 doi: 10.1038/81541 PMID: 11062457
61. Würbel H. Behavioral phenotyping enhanced-beyond (environmental) standardization. Genes, Brain Behav. [Internet]. 2002; 1:3-8. Available: http://www.ncbi.nlm.nih.gov/pubmed/12886944
62. Richter SH, Garner JP, Auer C, Kunert J, Würbel H. Systematic variation improves reproducibility of animal experiments. Nat. Methods [Internet]. Nature Publishing Group; 2010 [cited 2013 Apr 4]; 7:167-8. Available: http://www.ncbi.nlm.nih.gov/pubmed/20195246 doi: 10.1038/nmeth0310-167 PMID: 20195246
63. Voelkl B, Würbel H. Reproducibility crisis: Are we ignoring reaction norms? Trends Pharmacol. Sci. 2016; 37:509-10. doi: 10.1016/j.tips.2016.05.003 PMID: 27211784