Transparency in Ecology and Evolution: Real Problems, Real Solutions

Trends in Ecology and Evolution

Volumn 31, Issue 9, 2016, Pages 711-719

  Parker, Timothy H ^a   Forstmeier, Wolfgang ^b   Koricheva, Julia ^c   Fidler, Fiona ^d   Hadfield, Jarrod D ^e   Chee, Yung En ^d   Kelly, Clint D ^f   Gurevitch, Jessica ^g   Nakagawa, Shinichi ^h  

^a WHITMAN COLLEGE   (United States)

^b MAX PLANCK INSTITUTE FOR ORNITHOLOGY   (Germany)

^c UNIVERSITY OF LONDON   (United Kingdom)

^d UNIVERSITY OF MELBOURNE   (Australia)

^e UNIVERSITY OF EDINBURGH   (United Kingdom)

^f UNIVERSITÉ DU QUÉBEC À MONTRÉAL   (Canada)

^g STONY BROOK UNIVERSITY   (United States)

^h UNIVERSITY OF NEW SOUTH WALES   (Australia)

Author keywords

confirmation bias; inflated effect size; P hacking; preregistration; replication; selective reporting

Indexed keywords

DATA INTERPRETATION; ECOLOGY; EVOLUTION; LITERATURE REVIEW; PUBLISHING; RESEARCH WORK;

ECOLOGY; HUMAN; INTERPERSONAL COMMUNICATION; PUBLISHING;

DISCLOSURE; ECOLOGY; EDITORIAL POLICIES; HUMANS;

EID: 84990817527     PISSN: 01695347     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tree.2016.07.002     Document Type: Review

References (58)

1. 1 Smaldino, P.E., McElreath, R., The natural selection of bad science. arXiv, 2016 ArXiv:1605.09511v1.
2. 2 Møller, A.P., Jennions, M.D., Testing and adjusting for publication bias. Trends Ecol. Evol. 16 (2001), 580–586.
3. 3 Godefroid, S., et al. How successful are plant species reintroductions?. Biol. Conserv. 144 (2011), 672–682.
4. 4 Head, M.L., et al. The extent and consequences of P-hacking in science. PLoS Biol., 13, 2015, e1002106.
5. 5 Simonsohn, U., et al. P-curve: a key to the file drawer. J. Exp. Psychol. Gen. 143 (2014), 534–547.
6. 6 Kerr, N.L., HARKing: hypothesizing after the results are known. Pers. Soc. Psychol. Rev. 2 (1998), 196–217.
7. 7 Nickerson, R.S., Confirmation bias: a ubiquitous phenomenon in many guises. Rev. Gen. Psychol. 2 (1998), 175–220.
8. 8 Gelman, A., Working through some issues. Significance 12 (2015), 33–35.
9. 9 Rothstein, H.R., et al. (eds.) Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments, 2005, John Wiley & Sons.
10. 10 Fidler, F., et al. Impact of criticism of null-hypothesis significance testing on statistical reporting practices in conservation biology. Conserv. Biol. 20 (2006), 1539–1544.
11. 11 Koricheva, J., Gurevitch, J., Uses and misuses of meta-analysis in plant ecology. J. Ecol. 102 (2014), 828–844.
12. 12 Parker, T.H., What do we really know about the signalling role of plumage colour in blue tits?. A case study of impediments to progress in evolutionary biology. Biol. Rev. 88 (2013), 511–536.
13. 13 Ferreira, V., et al. A meta-analysis of the effects of nutrient enrichment on litter decomposition in streams. Biol. Rev. 90 (2015), 669–688.
14. 14 Menge, D.N.L., Field, C.B., Simulated global changes alter phosphorus demand in annual grassland. Global Change Biol. 13 (2007), 2582–2591.
15. 15 Zhang, Y., et al. Forest productivity increases with evenness, species richness and trait variation: a global meta-analysis. J. Ecol. 100 (2012), 742–749.
16. 16 Leisner, C.P., Ainsworth, E.A., Quantifying the effects of ozone on plant reproductive growth and development. Global Change Biol. 18 (2012), 606–616.
17. 17 Moles, A.T., et al. Assessing the evidence for latitudinal gradients in plant defence and herbivory. Functional Ecol. 25 (2011), 380–388.
18. 18 Cassey, P., et al. A survey of publication bias within evolutionary ecology. Proc. R. Soc. Lond. B Biol. Sci. 271 (2004), S451–S454.
19. 19 Bruns, S.B., Ioannidis, J.P.A., P-curve and p-hacking in observational research. PLoS ONE, 11, 2016, e0149144.
20. 20 Bishop, D.V.M., Thompson, P.A., Problems in using p-curve analysis and text-mining to detect rate of p-hacking and evidential value. PeerJ, 4, 2016, e1715.
21. 21 Ridley, J., et al. An unexpected influence of widely used significance thresholds on the distribution of reported P-values. J. Evol. Biol. 20 (2007), 1082–1089.
22. 22 Gelman, A., O'Rourke, K., Discussion: difficulties in making inferences about scientific truth from distributions of published p-values. Biostatistics 15 (2014), 18–23.
23. 23 Fanelli, D., ‘Positive’ results increase down the hierarchy of the sciences. PLoS ONE, 5, 2010, e10068.
24. 24 Csada, R.D., et al. The ‘file drawer problem’ of non-significant results: does it apply to biological research?. Oikos 76 (1996), 591–593.
25. 25 Møller, A.P., Jennions, M.D., How much variance can be explained by ecologists and evolutionary biologists?. Oecologia 132 (2002), 492–500.
26. 26 Hereford, J., et al. Comparing strengths of directional selection: how strong is strong?. Evolution 58 (2004), 2133–2143.
27. 27 Jennions, M.D., Møller, A.P., A survey of the statistical power of research in behavioral ecology and animal behavior. Behav. Ecol. 14 (2003), 438–445.
28. 28 Smith, D.R., et al. Power rangers: no improvement in the statistical power of analyses published in Animal Behaviour. Animal Behav. 81 (2011), 347–352.
29. 29 Button, K.S., et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14 (2013), 365–376.
30. 30 Gelman, A., Weakliem, D., Of beauty, sex, and power. American Scientist 97 (2009), 310–316.
31. 31 Eberhardt, L.L., Thomas, J.M., Designing environmental field studies. Ecol. Monogr. 61 (1991), 53–73.
32. 32 Murtaugh, P.A., In defense of P values. Ecology 95 (2014), 611–617.
33. 33 Barto, E.K., Rillig, M.C., Dissemination biases in ecology: effect sizes matter more than quality. Oikos 121 (2012), 228–235.
34. 34 Murtaugh, P.A., Journal quality, effect size, and publication bias in meta-analysis. Ecology 83 (2002), 1162–1166.
35. 35 Pike, N., Using false discovery rates for multiple comparisons in ecology and evolution. Methods Ecol. Evol. 2 (2011), 278–282.
36. 36 Forstmeier, W., Schielzeth, H., Cryptic multiple hypotheses testing in linear models: overestimated effect sizes and the winner's curse. Behav. Ecol. Sociobiol. 65 (2011), 47–55.
37. 37 Nakagawa, S., Parker, T.H., Replicating research in ecology and evolution: feasibility, incentives, and the cost–benefit conundrum. BMC Biol., 13, 2015, 88.
38. 38 Kelly, C.D., Replicating empirical research in behavioral ecology: how and why it should be done but rarely ever is. Q. Rev. Biol. 81 (2006), 221–236.
39. 39 Birkhead, T.R., Of moths and men. International Society for Behavioral Ecology Newsletter 14 (2002), 15–16.
40. 40 Nakagawa, S., Cuthill, I.C., Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol. Rev. 82 (2007), 591–605.
41. 41 Belovsky, G.E., et al. Ten suggestions to strengthen the science of ecology. BioScience 54 (2004), 345–351.
42. 42 Baker, M., 1,500 scientists lift the lid on reproducibility. Nature 533 (2016), 452–454.
43. 43 Parker, T.H., Nakagawa, S., Mitigating the epidemic of type I error: ecology and evolution can learn from other disciplines. Front. Ecol. Evol., 2, 2014, 76.
44. 44 Open Science Collaboration, Estimating the reproducibility of psychological science. Science, 349, 2015, aac4716.
45. 45 Nosek, B.A., et al. Promoting an open research culture. Science 348 (2015), 1422–1425.
46. 46 Whitlock, M.C., Data archiving in ecology and evolution: best practices. Trends Ecol. Evol. 26 (2011), 61–65.
47. 47 Mislan, K.A.S., et al. Elevating the status of code in ecology. Trends Ecol. Evol. 31 (2016), 4–7.
48. 48 Kidwell, M.C., et al. Badges to acknowledge open practices: a simple, low cost, effective method for increasing transparency. PLOS Biol., 14, 2016, e1002456.
49. 49 Roche, D.G., et al. Public data archiving in ecology and evolution: how well are we doing?. PLoS Biol., 13, 2015, e1002295.
50. 50 Mills, J.A., et al. Archiving primary data: solutions for long-term studies. Trends Ecol. Evol. 30 (2015), 581–589.
51. 51 Ross, J.S., et al. Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med., 6, 2009, e1000144.
52. 52 Wagenmakers, E-J., et al. An agenda for purely confirmatory research. Perspect. Psychol. Sci. 7 (2012), 632–638.
53. 53 Chambers, C.D., Registered reports: a new publishing initiative at Cortex. Cortex 49 (2013), 609–610.
54. 54 Huizenga, J.R., Cold Fusion: The Scientific Fiasco of the Century. 1994, Oxford University Press.
55. 55 van Wilgenburg, E., Elgar, M.A., Confirmation bias in studies of nestmate recognition: a cautionary note for research into the behaviour of animals. PLoS ONE, 8, 2013, e53548.
56. 56 Holman, L., et al. Evidence of experimental bias in the life sciences: why we need blind data recording. PLoS Biol., 13, 2015, e1002190.
57. 57 Kardish, M.R., et al. Blind trust in unblinded observation in ecology, evolution and behavior. Front. Ecol. Evol., 3, 2015, 51.
58. 58 Wacholder, S., et al. Assessing the probability that a positive report is false: an approach for molecular epidemiology studies. J. Natl. Cancer Inst. 96 (2004), 434–442.