Replication validity of initial association studies: A comparison between psychiatry, neurology and four somatic diseases

PLoS ONE

Volumn 11, Issue 6, 2016, Pages

  Dumas Mallet, Estelle ^a,b   Button, Katherine ^c   Boraud, Thomas ^a,b,e   Munafo, Marcus ^d   Gonon, François ^a,b  

^a CNRS   (France)

^b UNIVERSITÉ DE BORDEAUX   (France)

^c University of Bath   (United Kingdom)

^d UNIVERSITY OF BRISTOL   (United Kingdom)

^e BORDEAUX UNIVERSITY HOSPITAL   (France)

Author keywords

[No Author keywords available]

Indexed keywords

ALZHEIMER DISEASE; ARTICLE; ATTENTION DEFICIT DISORDER; AUTISM; BEHAVIOR DISORDER; BREAST CANCER; COGNITIVE DEFECT; DISEASE MARKER; EFFECT SIZE; EPILEPSY; GENETIC ANALYSIS; GLAUCOMA; HUMAN; MAJOR DEPRESSION; MENTAL DISEASE; META ANALYSIS (TOPIC); MULTIPLE SCLEROSIS; NEUROIMAGING; NEUROLOGIC DISEASE; PARKINSON DISEASE; PROBABILITY; PSORIASIS; RELIABILITY; RHEUMATOID ARTHRITIS; RISK ASSESSMENT; SAMPLE SIZE; SCHIZOPHRENIA; SENSITIVITY AND SPECIFICITY; DISEASE PREDISPOSITION; HEALTH SURVEY; MENTAL DISORDERS; META ANALYSIS; NERVOUS SYSTEM DISEASES; PUBLICATION; RECEIVER OPERATING CHARACTERISTIC; REPRODUCIBILITY; RISK FACTOR;

BIOLOGICAL MARKER;

BIOMARKERS; DISEASE SUSCEPTIBILITY; HUMANS; MENTAL DISORDERS; NERVOUS SYSTEM DISEASES; POPULATION SURVEILLANCE; PUBLICATIONS; REPRODUCIBILITY OF RESULTS; RISK FACTORS; ROC CURVE; SAMPLE SIZE;

EID: 84976548608     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0158064     Document Type: Article

References (46)

1. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005; 2(8):e124. PMID: 16060722.
2. Sarewitz D. Beware the creeping cracks of bias. Nature. 2012; 485(7397):149. doi:10.1038/485149aPMID: 22575922.
3. Munafo M, Noble S, Browne WJ, Brunner D, Button K, Ferreira J, et al. Scientific rigor and the art of motorcycle maintenance. Nature biotechnology. 2014; 32(9):871-3. doi:10.1038/nbt.3004PMID: 25203032.
4. Begley CG, Ioannidis JP. Reproducibility in science: Improving the standard for basic and preclinical research. Circulation research. 2015; 116(1):116-26. doi:10.1161/CIRCRESAHA.114.303819 PMID: 25552691.
5. Freedman LP, Cockburn IM, Simcoe TS. The Economics of Reproducibility in Preclinical Research. PLoS biology. 2015; 13(6):e1002165. doi:10.1371/journal.pbio.1002165 PMID: 26057340.
6. DiMasi JA, Feldman L, Seckler A, Wilson A. Trends in risks associated with new drug development: success rates for investigational drugs. Clinical pharmacology and therapeutics. 2010; 87(3):272-7. doi:10.1038/clpt.2009.295 PMID: 20130567.
7. Gan HK, You B, Pond GR, Chen EX. Assumptions of expected benefits in randomized phase III trials evaluating systemic treatments for cancer. Journal of the National Cancer Institute. 2012; 104(8):590- 8. doi:10.1093/jnci/djs141 PMID: 22491345.
8. Ioannidis J, Lau J. Evolution of treatment effects over time: Empirical insight from recursive cumulative metaanalyses. Proceedings of the National Academy of Sciences of the United States of America. 2001; 98(3):831-6. PMID: 11158556.
9. Trikalinos TA, Churchill R, Ferri M, Leucht S, Tuunainen A, Wahlbeck K, et al. Effect sizes in cumulative meta- Analyses of mental health randomized trials evolved over time. J Clin Epidemiol. 2004; 57 (11):1124-30. PMID: 15612138.
10. Ioannidis JP. Contradicted and initially stronger effects in highly cited clinical research. JAMA. 2005; 294(2):218-28. PMID: 16014596.
11. Pereira TV, Horwitz RI, Ioannidis JP. Empirical evaluation of very large treatment effects of medical interventions. JAMA. 2012; 308(16):1676-84. doi:10.1001/jama.2012.13444 PMID: 23093165.
12. Tajika A, Ogawa Y, Takeshima N, Hayasaka Y, Furukawa TA. Replication and contradiction of highly cited research papers in psychiatry: 10-year follow-up. The British journal of psychiatry: The journal of mental science. 2015; 207(4):357-62.
13. Ioannidis JP, Ntzani EE, Trikalinos TA, Contopoulos-Ioannidis DG. Replication validity of genetic association studies. Nat Genet. 2001; 29(3):306-9. PMID: 11600885.
14. Hirschhorn JN, Lohmueller K, Byrne E, Hirschhorn K. A comprehensive review of genetic association studies. Genetics in medicine: official journal of the American College of Medical Genetics. 2002; 4 (2):45-61.
15. Trikalinos TA, Ntzani EE, Contopoulos-Ioannidis DG, Ioannidis JP. Establishment of genetic associations for complex diseases is independent of early study findings. European journal of human genetics: EJHG. 2004; 12(9):762-9. PMID: 15213707.
16. Ioannidis JP, Panagiotou OA. Comparison of effect sizes associated with biomarkers reported in highly cited individual articles and in subsequent meta- Analyses. JAMA. 2011; 305(21):2200-10. doi:10.1001/jama.2011.713 PMID: 21632484.
17. Button KS, Ioannidis JP, Mokrysz C, Nosek BA, Flint J, Robinson ES, et al. Power failure: Why small sample size undermines the reliability of neuroscience. Nature reviews Neuroscience. 2013; 14 (5):365-76. doi:10.1038/nrn3475 PMID: 23571845.
18. Wallace BC, Schmid CH, Lau J, Trikalinos TA. Meta-Analyst: software for meta- Analysis of binary, continuous and diagnostic data. BMC medical research methodology. 2009; 9:80. doi:10.1186/1471-2288-9-80PMID: 19961608.
19. Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Statistics in medicine. 1998; 17(8):857-72. PMID: 9595616.
20. Pereira TV, Ioannidis JP. Statistically significant meta- Analyses of clinical trials have modest credibility and inflated effects. J Clin Epidemiol. 2011; 64(10):1060-9. doi:10.1016/j.jclinepi.2010.12.012 PMID: 21454050.
21. Open-Science-Collaboration. PSYCHOLOGY. Estimating the reproducibility of psychological science. Science (New York, NY). 2015; 349(6251):4716.
22. Ioannidis JP. Effect of formal statistical significance on the credibility of observational associations. American journal of epidemiology. 2008; 168(4):374-83; discussion 84-90. doi:10.1093/aje/kwn156PMID: 18611956.
23. Johnson VE. Revised standards for statistical evidence. Proceedings of the National Academy of Sciences of the United States of America. 2013; 110(48):19313-7. doi:10.1073/pnas.1313476110 PMID: 24218581.
24. Dienes Z. Using Bayes to get the most out of non-significant results. Frontiers in psychology. 2014; 5:781. doi:10.3389/fpsyg.2014.00781 PMID: 25120503.
25. Munafo MR, Stothart G, Flint J. Bias in genetic association studies and impact factor. Molecular psychiatry. 2009; 14(2):119-20doi:10.1038/mp.2008.77 PMID: 19156153.
26. Ioannidis JP. Why most discovered true associations are inflated. Epidemiology (Cambridge, Mass). 2008; 19(5):640-8.
27. Ioannidis JP, Trikalinos TA, Ntzani EE, Contopoulos-Ioannidis DG. Genetic associations in large versus small studies: An empirical assessment. Lancet (London, England). 2003; 361(9357):567-71.
28. Ioannidis JP. Excess significance bias in the literature on brain volume abnormalities. Archives of general psychiatry. 2011; 68(8):773-80. doi:10.1001/archgenpsychiatry.2011.28 PMID: 21464342.
29. David SP, Ware JJ, Chu IM, Loftus PD, Fusar-Poli P, Radua J, et al. Potential reporting bias in fMRI studies of the brain. PloS one. 2013; 8(7):e70104. doi:10.1371/journal.pone.0070104 PMID: 23936149.
30. Fusar-Poli P, Radua J, Frascarelli M, Mechelli A, Borgwardt S, Di Fabio F, et al. Evidence of reporting biases in voxel-based morphometry (VBM) studies of psychiatric and neurological disorders. Human brain mapping. 2014; 35(7):3052-65. doi:10.1002/hbm.22384 PMID: 24123491.
31. Lohmueller KE, Pearce CL, Pike M, Lander ES, Hirschhorn JN. Meta- Analysis of genetic association studies supports a contribution of common variants to susceptibility to common disease. Nat Genet. 2003; 33(2):177-82. PMID: 12524541.
32. Ioannidis JP, Munafo MR, Fusar-Poli P, Nosek BA, David SP. Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention. Trends in cognitive sciences. 2014; 18 (5):235-41doi:10.1016/j.tics.2014.02.010 PMID: 24656991.
33. Prinz F, Schlange T, Asadullah K. Believe it or not: how much can we rely on published data on potential drug targets? Nature reviews Drug discovery. 2011; 10(9):712. doi:10.1038/nrd3439-c1PMID: 21892149.
34. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta- Analysis detected by a simple, graphical test. BMJ (Clinical research ed). 1997; 315(7109):629-34.
35. Sterne JA, Egger M, Smith GD. Systematic reviews in health care: Investigating and dealing with publication and other biases in meta- Analysis. BMJ (Clinical research ed). 2001; 323(7304):101-5.
36. Thompson S, Ekelund U, Jebb S, Lindroos AK, Mander A, Sharp S, et al. A proposed method of bias adjustment for meta- Analyses of published observational studies. International journal of epidemiology. 2011; 40(3):765-77. doi:10.1093/ije/dyq248 PMID: 21186183.
37. Ioannidis JP. Commentary: Adjusting for bias: A user's guide to performing plastic surgery on metaanalyses of observational studies. International journal of epidemiology. 2011; 40(3):777-9. doi:10.1093/ije/dyq265 PMID: 21233141.
38. Sterne JA, Sutton AJ, Ioannidis JP, Terrin N, Jones DR, Lau J, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta- Analyses of randomised controlled trials. BMJ (Clinical research ed). 2011; 343:d4002.
39. Papatheodorou SI, Tsilidis KK, Evangelou E, Ioannidis JP. Application of credibility ceilings probes the robustness of meta- Analyses of biomarkers and cancer risk. J Clin Epidemiol. 2015; 68(2):163-74. doi:10.1016/j.jclinepi.2014.09.004 PMID: 25433443.
40. Gonon F, Konsman JP, Cohen D, Boraud T. Why most biomedical findings echoed by newspapers turn out to be false: The case of Attention Deficit Hyperactivity Disorder. PloS one. 2012; 7(9):e44275. doi:10.1371/journal.pone.0044275 PMID: 22984483.
41. Hirtz D, Thurman DJ, Gwinn-Hardy K, Mohamed M, Chaudhuri AR, Zalutsky R. How common are the "common" neurologic disorders? Neurology. 2007; 68(5):326-37. PMID: 17261678.
42. Iqbal SA, Wallach JD, Khoury MJ, Schully SD, Ioannidis JP. Reproducible Research Practices and Transparency across the Biomedical Literature. PLoS biology. 2016; 14(1):e1002333. doi:10.1371/journal.pbio.1002333PMID: 26726926.
43. Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet (London, England). 2014; 383(9913):267-76.
44. Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gulmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. Lancet (London, England). 2014; 383 (9912):156-65.
45. Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet (London, England). 2014; 383 (9912):166-75.
46. Lewandowsky S, Bishop D. Research integrity: Don't let transparency damage science. Nature. 2016; 529(7587):459-61. doi:10.1038/529459aPMID: 26819029.