A call for transparent reporting to optimize the predictive value of preclinical research

Nature

Volumn 490, Issue 7419, 2012, Pages 187-191

  Landis, Story C ^a   Amara, Susan G ^b   Asadullah, Khusru ^c   Austin, Chris P ^d   Blumenstein, Robi ^e   Bradley, Eileen W ^f   Crystal, Ronald G ^g   Darnell, Robert B ^h   Ferrante, Robert J ^b   Fillit, Howard ⁱ   Finkelstein, Robert ^a   Fisher, Marc ^j   Gendelman, Howard E ^k   Golub, Robert M ^l   Goudreau, John L ^m   Gross, Robert A ⁿ   Gubitz, Amelie K ^a   Hesterlee, Sharon E ^o   Howells, David W ^p   Huguenard, John ^q   Kelner, Katrina ^r   Koroshetz, Walter ^a   Krainc, Dimitri ^s   Lazic, Stanley E ^t   Levine, Michael S ^u   MacLeod, Malcolm R ^v   McCall, John M ^w   Iii, Richard T Moxley ^x   Narasimhan, Kalyani ^y   Noble, Linda J ^z   Perrin, Steve ^aa   Porter, John D ^a   Steward, Oswald ^ab   Unger, Ellis ^ac   Utz, Ursula ^a   Silberberg, Shai D ^a   more..

^a NATIONAL INSTITUTE OF NEUROLOGICAL DISORDERS AND STROKE   (United States)

^b UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

^c BAYER PHARMA AG   (Germany)

^d NATIONAL INSTITUTES OF HEALTH   (United States)

^e CHDI MANAGEMENT CHDI FOUNDATION   (United States)

^f NATIONAL INSTITUTES OF HEALTH   (United States)

^g WEILL CORNELL MEDICAL COLLEGE   (United States)

^h ROCKEFELLER UNIVERSITY   (United States)

ⁱ Alzheimers Drug Discovery Foundation   (United States)

^j UNIVERSITY OF MASSACHUSETTS MEDICAL SCHOOL   (United States)

^k UNIVERSITY OF NEBRASKA MEDICAL CENTER   (United States)

^l JAMA   (United States)

^m East Lansing   (United States)

ⁿ UNIVERSITY OF ROCHESTER MEDICAL CENTER   (United States)

^o Parent Project Muscular Dystrophy   (United States)

^p UNIVERSITY OF MELBOURNE   (Australia)

^q STANFORD UNIVERSITY   (United States)

^r AAAS   (United States)

^s MASSACHUSETTS GENERAL HOSPITAL   (United States)

^t F HOFFMANN LA ROCHE LTD   (Switzerland)

^u UNIVERSITY OF CALIFORNIA   (United States)

^v UNIVERSITY OF EDINBURGH   (United Kingdom)

^w PharMac LLC   (United States)

^x UNIVERSITY OF ROCHESTER SCHOOL OF MEDICINE AND DENTISTRY   (United States)

^y Nature Neuroscience   (United States)

^z UNIVERSITY OF CALIFORNIA   (United States)

^aa ALS Therapy Development Institute   (United States)

^ab UNIVERSITY OF CALIFORNIA   (United States)

^ac CENTER FOR DRUG EVALUATION AND RESEARCH   (United States)

more..

Author keywords

[No Author keywords available]

Indexed keywords

NEUROLOGY; OPTIMIZATION; RESEARCH WORK; SIZE EFFECT; STAKEHOLDER; VISION;

CLINICAL RESEARCH; HUMAN; INCIDENT REPORT; NONHUMAN; PREDICTIVE VALUE; PRIORITY JOURNAL; REVIEW; STANDARDIZATION; TRANSPARENT REPORTING;

ANIMALS; PUBLISHING; RANDOM ALLOCATION; RESEARCH DESIGN; SAMPLE SIZE; STATISTICS AS TOPIC;

EID: 84867302438     PISSN: 00280836     EISSN: 14764687     Source Type: Journal    
DOI: 10.1038/nature11556     Document Type: Review

References (64)

1. Begley, C. G. & Ellis, L. M. Raise standards for preclinical cancer research. Nature 483, 531-533 (2012).
2. Hess, K. R.Statistical design considerations in animalstudiespublishedrecently in Cancer Research. Cancer Res. 71, 625 (2011).
3. Kilkenny, C. et al. Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLoS ONE 4, e7824 (2009).
4. Moher, D., Simera, I., Schulz, K. F., Hoey, J. & Altman, D. G. Helping editors, peer reviewers and authors improve the clarity, completeness and transparency of reporting health research. BMC Med. 6, 13 (2008).
5. Prinz, F., Schlange, T. & Asadullah, K. Believe it or not: how much can we rely on published data on potential drug targets? Nature Rev. Drug Discov. 10, 712 (2011).
6. Sena, E., van der Worp, H. B., Howells, D. & Macleod, M. How can we improve the pre-clinical development of drugs for stroke? Trends Neurosci. 30, 433-439 (2007).
7. Steward, O., Popovich, P. G., Dietrich, W. D. & Kleitman, N. Replication and reproducibility in spinal cord injury research. Exp. Neurol. 233, 597-605 (2012).
8. van der Worp, H. B. & Macleod, M. R. Preclinical studies of human disease: time to take methodological quality seriously. J. Mol. Cell. Cardiol. 51, 449-450 (2011).
9. Hackam, D. G. & Redelmeier, D. A. Translation of research evidence from animals to humans. J. Am. Med. Assoc. 296, 1727-1732 (2006).
10. Macleod, M. R. et al. Evidence for the efficacy of NXY-059 in experimental focal cerebral ischaemia is confounded by study quality. Stroke39, 2824-2829 (2008).
11. Bebarta, V., Luyten, D.& Heard, K.Emergencymedicine animal research: does use of randomization and blinding affect the results? Acad. Emerg. Med. 10, 684-687 (2003).
12. Crossley, N. A. et al.Empirical evidence of bias in the design of experimental stroke studies-A metaepidemiologic approach. Stroke 39, 929-934 (2008).
13. Rooke, E. D., Vesterinen, H. M., Sena, E. S., Egan, K. J. & Macleod, M. R. Dopamine agonists in animal models of Parkinson's disease: a systematic review and metaanalysis. Parkinsonism Relat. Disord. 17, 313-320 (2011).
14. Vesterinen, H. M. et al. Improving the translational hit of experimental treatments in multiple sclerosis. Mult. Scler. J. 16, 1044-1055 (2010).
15. Stroke Therapy Academic Industry Roundtable (STAIR). Recommendations for standards regarding preclinical neuroprotective and restorative drug development. Stroke 30, 2752-2758 (1999).
16. Fanelli, D. "Positive" results increase downthe hierarchy of the sciences. PLoSONE 5, e10068 (2010).
17. Jerndal, M. et al. A systematic review and meta-analysis of erythropoietin in experimental stroke. J. Cereb. Blood Flow Metab. 30, 961-968 (2010).
18. Macleod, M. R., O'Collins, T., Horky, L. L., Howells, D. W. & Donnan, G. A. Systematic review and metaanalysis of the efficacy of FK506 in experimental stroke. J. Cereb. Blood Flow Metab. 25, 713-721 (2005).
19. Sena, E. S. et al. Factors affecting the apparent efficacy and safety of tissue plasminogen activator in thrombotic occlusion models of stroke: systematic review and meta-analysis. J. Cereb. Blood Flow Metab. 30, 1905-1913 (2010).
20. Wheble, P. C. R., Sena, E. S.&Macleod, M. R.A systematic reviewandmeta-analysis of the efficacy of piracetam and piracetam-like compounds in experimental stroke. Cerebrovasc. Dis. 25, 5-11 (2008).
21. Festing, M. F. & Altman, D. G. Guidelines for the design and statistical analysis of experiments using laboratory animals. ILAR J. 43, 244-258 (2002).
22. Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M. & Altman, D. G. Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS Biol. 8, e1000412 (2010).
23. van der Worp, H. B. et al. Can animal models of disease reliably inform human studies? PLoS Med. 7, e1000245 (2010).
24. Fisher, M. et al. Update of the stroke therapy academic industry roundtable preclinical recommendations. Stroke 40, 2244-2250 (2009).
25. Ludolph, A. C. et al. Guidelines for preclinical animal research in ALS/MND: a consensus meeting. Amyotroph. Lateral Scler. 11, 38-45 (2010).
26. Shineman, D. W. et al. Accelerating drug discovery for Alzheimer's disease: best practices for preclinical animal studies. Alzheimers Res. Ther. 3, 28 (2011).
27. Unger, E. F. All is not well in the world of translational research. J. Am. Coll. Cardiol. 50, 738-740 (2007).
28. Ioannidis, J. P. A. Why most published research findings are false. PLoS Med. 2, e124 (2005).
29. Dienes, Z. Bayesian versus orthodox statistics: which side are you on? Perspect. Psychol. Sci. 6, 274-290 (2011).
30. Simmons, J. P., Nelson, L. D. & Simonsohn, U. False-positive psychology: undisclosed flexibility indata collectionandanalysis allowspresentinganythingas significant. Psychol. Sci. 22, 1359-1366 (2011).
31. Beal, K. G. & Khamis, H. J. A problem in statistical-analysis: simultaneous inference. Condor 93, 1023-1025 (1991).
32. Lazic, S. E. The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis? BMC Neurosci. 11, 5 (2010).
33. Scott, S. et al. Design, power, and interpretation of studies in the standard murine model of ALS. Amyotroph. Lateral Scler. 9, 4-15 (2008).
34. Proschan, M. A. & Waclawiw, M. A. Practical guidelines for multiplicity adjustment in clinical trials. Control. Clin. Trials 21, 527-539 (2000).
35. Festing, M. F. W. Design and statistical methods in studies using animalmodels of development. ILAR J. 47, 5-14 (2006).
36. Nakagawa, S. & Cuthill, I. C. Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol. Rev. Camb. Philos. Soc. 82, 591-605 (2007).
37. Chalmers, T. C., Celano, P., Sacks, H. S. &Smith, H. Bias in treatment assignment in controlled clinical-trials. N. Engl. J. Med. 309, 1358-1361 (1983).
38. Jüni, P., Altman, D. G.& Egger, M. Systematic reviews in health care-assessing the quality of controlled clinical trials. Br. Med. J. 323, 42 (2001).
39. Pildal, J. et al. Impact of allocation concealment on conclusions drawn from metaanalyses of randomized trials. Int. J. Epidemiol. 36, 847-857 (2007).
40. Pocock, S. J., Hughes, M. D. & Lee, R. J. Statistical problems in the reporting of clinical-trials. A survey of three medical journals. N. Engl. J. Med. 317, 426-432 (1987).
41. Schulz, K. F., Chalmers, I., Hayes, R. J. & Altman, D. G. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. J. Am. Med. Assoc. 273, 408-412 (1995).
42. Wood, L. et al.Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. Br. Med. J. 336, 601-605 (2008).
43. Moher, D. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. Br. Med. J. 340, c869 (2011).
44. Moher, D., Schulz, K. F. & Altman, D. G. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 357, 1191-1194 (2001).
45. Schulz, K. F., Altman, D. G. & Moher, D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. PLoS Med. 7, e1000251 (2010).
46. Plint, A. C. et al. Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med. J. Aust. 185, 263-267 (2006).
47. Kane, R. L., Wang, J. & Garrard, J. Reporting in randomized clinical trials improved after adoption of the CONSORT statement. J. Clin. Epidemiol. 60, 241-249 (2007).
48. Prady, S. L., Richmond, S. J., Morton, V. M. & Macpherson, H. A systematic evaluation of the impact of STRICTA and CONSORT recommendations on quality of reporting for acupuncture trials. PLoS ONE 3, e1577 (2008).
49. Smith, B. A. et al. Quality of reporting randomized controlled trials (RCTs) in nursing literature: application of the consolidated standards reporting trials (CONSORT). Nurs. Outlook 56, 31-37 (2008).
50. Macleod, M. R., O'Collins, T., Howells, D. W. & Donnan, G. A. Pooling of animal experimental data reveals influence of study design and publication bias. Stroke 35, 1203-1208 (2004).
51. Macleod, M. R., O'Collins, T., Horky, L. L., Howells, D. W. & Donnan, G. A. Systematic review and meta-analysis of the efficacy of melatonin in experimental stroke. J. Pineal Res. 38, 35-41 (2005).
52. Gallo, J. M. Pharmacokinetic/pharmacodynamic-driven drug development. Mt. Sinai J. Med. 77, 381-388 (2010).
53. Moher, D. et al. Describing reporting guidelines for health research: a systematic review. J. Clin. Epidemiol. 64, 718-742 (2011).
54. Callaham, M. L., Wears, R. L., Weber, E. J., Barton, C. & Young, G. Positive-outcome bias and other limitations in the outcome of research abstracts submitted to a scientific meeting. J. Am. Med. Assoc. 280, 254-257 (1998).
55. Dickersin, K.&Chalmers, I. Recognizing, investigation and dealingwith incomplete and biased reporting of clinical research: fromFrancis Bacon to theWHO. J. R. Soc. Med. 104, 532-538 (2011).
56. Fanelli, D. Negative results are disappearing from most disciplines and countries. Scientometrics 90, 891-904 (2012).
57. Kyzas, P. A., Denaxa-Kyza, D. & Ioannidis, J. P. A. Almost all articles on cancer prognostic markers report statistically significant results. Eur. J. Cancer 43, 2559-2579 (2007).
58. Liu, S. Dealing with publication bias in translational stroke research. J. Exp. Stroke Transl. Med. 2, 16-21 (2009).
59. Rockwell, S., Kimler, B. E. & Moulder, J. E. Publishing negative results: the problem of publication bias. Radiat. Res. 165, 623-625 (2006).
60. Rosenthal, R. The file drawer problem and tolerance for null results. Psychol. Bull. 86, 638-641 (1979).
61. Sterling, T. D. Publication decisions and their possible effects on inferences drawn from tests of significance-or vice versa. J. Am. Stat. Assoc. 54, 30-34 (1959).
62. Song, F. et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol. Assess. 14, 1-220 (2010).
63. Sena, E. S., van der Worp, H. B., Bath, P. M. W., Howells, D. W. & Macleod, M. R. Publication bias in reports of animal stroke studies leads to major overstatement of efficacy. PLoS Biol. 8, e1000344 (2010).
64. Fanelli, D. Do pressures to publish increase scientists' bias? An empirical support from US states data. PLoS ONE 5, e10271 (2010).