A Practical Guide for Improving Transparency and Reproducibility in Neuroimaging Research

PLoS Biology

Volumn 14, Issue 7, 2016, Pages

  Gorgolewski, Krzysztof J ^a   Poldrack, Russell A ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CAREER; HUMAN; NEUROIMAGING; NEUROSCIENCE; PSYCHOLOGY; PUBLICATION; REPRODUCIBILITY; SCIENTIST; METHODOLOGY; PRACTICE GUIDELINE; PROCEDURES; RESEARCH; SCIENCE; STANDARDS;

GUIDELINES AS TOPIC; HUMANS; NEUROIMAGING; PUBLICATIONS; REPRODUCIBILITY OF RESULTS; RESEARCH; RESEARCH DESIGN; RESEARCH REPORT; SCIENCE;

EID: 84982851880     PISSN: 15449173     EISSN: 15457885     Source Type: Journal    
DOI: 10.1371/journal.pbio.1002506     Document Type: Article

References (39)

1. Carp J, On the plurality of (methodological) worlds: estimating the analytic flexibility of FMRI experiments. Front Neurosci. 2012;6: 149. doi: 10.3389/fnins.2012.0014923087605
2. Simmons JP, Nelson LD, Simonsohn U, False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci. 2011;22: 1359–1366. doi: 10.1177/095679761141763222006061
3. Ioannidis JPA, Munafò MR, Fusar-Poli P, Nosek BA, David SP, Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention. Trends Cogn Sci. 2014;18: 235–241. doi: 10.1016/j.tics.2014.02.01024656991
4. Open Science CollaborationEstimating the reproducibility of psychological science. Science. 2015;349. doi: 10.1126/science.aac4716
5. Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, Robinson ESJ, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013;14: 365–376. doi: 10.1038/nrn347523571845
6. Halchenko YO, Hanke M, Four aspects to make science open “by design” and not as an after-thought. Gigascience. 2015;4: 31. doi: 10.1186/s13742-015-0072-726191404
7. Pernet C, Poline J-B, Improving functional magnetic resonance imaging reproducibility [Internet]. Gigascience. 2015. p. 15. doi: 10.1186/s13742-015-0055-825830019
8. Eglen S, Marwick B, Halchenko Y, Hanke M, Sufi S, Gleeson P, et al. Towards standard practices for sharing computer code and programs in neuroscience [Internet]. bioRxiv. 2016. p. 045104. doi: 10.1101/045104
9. Piwowar HA, Day RS, Fridsma DB, Sharing detailed research data is associated with increased citation rate. PLoS ONE. 2007;2: e308. doi: 10.1371/journal.pone.000030817375194
10. Piwowar HA, Vision TJ, Data reuse and the open data citation advantage. PeerJ. 2013;1: e175. doi: 10.7717/peerj.17524109559
11. Wicherts JM, Bakker M, Molenaar D, Tractenberg RE, Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results. PLoS ONE. 2011;6: e26828. doi: 10.1371/journal.pone.002682822073203
12. Brakewood B, Poldrack RA, The ethics of secondary data analysis: considering the application of Belmont principles to the sharing of neuroimaging data. Neuroimage. 2013;82: 671–676. doi: 10.1016/j.neuroimage.2013.02.04023466937
13. Poline J-B, Breeze JL, Ghosh S, Gorgolewski K, Halchenko YO, Hanke M, et al. Data sharing in neuroimaging research. Front Neuroinform. 2012;6: 9. doi: 10.3389/fninf.2012.0000922493576
14. Poldrack RA, Gorgolewski KJ, Making big data open: data sharing in neuroimaging. Nat Neurosci. Nature Publishing Group; 2014;17: 1510–1517. doi: 10.1038/nn.381825349916
15. Nooner KB, Colcombe SJ, Tobe RH, Mennes M, Benedict MM, Moreno AL, et al. The NKI-Rockland Sample: A Model for Accelerating the Pace of Discovery Science in Psychiatry. Front Neurosci. 2012;6: 152. doi: 10.3389/fnins.2012.0015223087608
16. Van Essen DC, Smith SM, Barch DM, Behrens TEJ, Yacoub E, Ugurbil K, et al. The WU-Minn Human Connectome Project: an overview. Neuroimage. 2013;80: 62–79. doi: 10.1016/j.neuroimage.2013.05.04123684880
17. Gorgolewski KJ, Auer T, Calhoun VD, Craddock RC, Das S, Duff EP, et al. The Brain Imaging Data Structure: a standard for organizing and describing outputs of neuroimaging experiments. Scientific Data. 2016; 3:160044. doi: 10.1038/sdata.2016.44
18. Poldrack RA, Barch DM, Mitchell JP, Wager TD, Wagner AD, Devlin JT, et al. Toward open sharing of task-based fMRI data: the OpenfMRI project. Front Neuroinform. 2013;7: 1–12. doi: 10.3389/fninf.2013.00012
19. Mennes M, Biswal BB, Castellanos FX, Milham MP, Making data sharing work: The FCP/INDI experience. Neuroimage. Elsevier Inc.; 2012; doi: 10.1016/j.neuroimage.2012.10.064
20. Gadde S, Aucoin N, Grethe JS, Keator DB, Marcus DS, Pieper S, XCEDE: An Extensible Schema for Biomedical Data. Neuroinformatics. 2012;10: 19–32. doi: 10.1007/s12021-011-9119-921479735
21. Gorgolewski KJ, Margulies DS, Milham MP, Making data sharing count: a publication-based solution. Front Neurosci. Frontiers; 2013;7: 9. doi: 10.3389/fnins.2013.0000923390412
22. Gorgolewski KJ, Varoquaux G, Rivera G, Schwarz Y, Ghosh SS, Maumet C, et al. NeuroVault.org: a web-based repository for collecting and sharing unthresholded statistical maps of the human brain. Front Neuroinform. Frontiers; 2015;9. doi: 10.3389/fninf.2015.00008
23. Brown JA, Rudie JD, Bandrowski A, Van Horn JD, Bookheimer SY, The UCLA multimodal connectivity database: a web-based platform for brain connectivity matrix sharing and analysis. Front Neuroinform. 2012;6: 28. doi: 10.3389/fninf.2012.0002823226127
24. Barnes N, Publish your computer code: it is good enough. Nature. 2010;467: 753. doi: 10.1038/467753a20944687
25. Vandewalle P, Code Sharing Is Associated with Research Impact in Image Processing. Comput Sci Eng. AIP Publishing; 2012;14: 42–47. doi: 10.1109/MCSE.2012.63
26. Blischak JD, Davenport ER, Wilson G, A Quick Introduction to Version Control with Git and GitHub. PLoS Comput Biol. 2016;12: e1004668. doi: 10.1371/journal.pcbi.100466826785377
27. Gorgolewski K, Burns CD, Madison C, Clark D, Halchenko YO, Waskom ML, et al. Nipype: a flexible, lightweight and extensible neuroimaging data processing framework in python. Front Neuroinform. 2011;5: 13. doi: 10.3389/fninf.2011.0001321897815
28. Bellec P, Courchesne SL, Dickinson P, Lerch J, Zijdenbos A, Evans AC, The pipeline system for Octave and Matlab (PSOM): a lightweight scripting framework and execution engine for scientific workflows. Front Neuroinform. 2012;6. doi: 10.3389/fninf.2012.00007
29. Cusack R, Vicente-Grabovetsky A, Mitchell DJ, Wild CJ, Auer T, Linke AC, et al. Automatic analysis (aa): efficient neuroimaging workflows and parallel processing using Matlab and XML. Front Neuroinform. 2014;8: 90. doi: 10.3389/fninf.2014.0009025642185
30. Askren MK, McAllister-Day TK, Koh N, Mestre Z, Dines JN, Korman BA, et al. Using Make for Reproducible and Parallel Neuroimaging Workflow and Quality-Assurance. Front Neuroinform. Frontiers; 2016;10. doi: 10.3389/fninf.2016.00002
31. Poldrack RA, Laumann TO, Koyejo O, Gregory B, Hover A, Chen M-Y, et al. Long-term neural and physiological phenotyping of a single human. Nat Commun. 2015;6: 8885. doi: 10.1038/ncomms988526648521
32. Knuth DE, Literate programming. CSLI Lecture Notes, Stanford, CA: Center for the Study of Language and Information (CSLI), 1992. http://adsabs.harvard.edu/abs/1992lipr.book...K
33. The future of publishing: A new page. Nature. 2013;495: 425. doi: 10.1038/495425a23538807
34. Popova K. Evolution of Open Access Policies and Business Models: Which Way Leads to The Future? In: Scicasts [Internet]. [cited 5 Feb 2016]. https://scicasts.com/insights/2123-open-science/10333-evolution-of-open-access-policies-and-business-models-which-way-to-the-future/
35. Aarts AA, Bosco F, Carp JM, Field JG, IJzerman H, Lewis M, et al. Maximizing the reproducibility of your research. In: Lilienfeld SO, Waldman ID, editors. Psychological Science Under Scrutiny: Recent Challenges and Proposed Solutions. 2014. http://ar.ascb.org/publicpolicy/Articles/OSC2014final.pdf
36. Boekel W, Wagenmakers E-J, Belay L, Verhagen J, Brown S, Forstmann BU, A purely confirmatory replication study of structural brain-behavior correlations. Cortex. 2015;66: 115–133. doi: 10.1016/j.cortex.2014.11.01925684445
37. Chambers CD, Feredoes E, Muthukumaraswamy SD, Etchells PJ, Instead of “playing the game” it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond. doi: 10.3934/Neuroscience
38. Fischer A, van Reekum C, Editorial. Cognition and Emotion. 2015;29: 765–766. doi: 10.1080/02699931.2015.1026223
39. Chambers CD, Registered reports: a new publishing initiative at Cortex. Cortex. 2013;49: 609–610. doi: 10.1016/j.cortex.2012.12.01623347556