Improving functional magnetic resonance imaging reproducibility

GigaScience

Volumn 4, Issue 1, 2015, Pages

  Pernet, Cyril ^a   Poline, Jean Baptiste ^b  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Code; Functional MRI; Open science; Reproducibility; Scripts; Workflows

Indexed keywords

ANALYTIC METHOD; ARTICLE; CLINICAL PROTOCOL; DATA PROCESSING; DOCUMENTATION; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; NEUROIMAGING; PRIORITY JOURNAL; REPRODUCIBILITY; SCIENCE; SCIENTIST; INFORMATION DISSEMINATION; NUCLEAR MAGNETIC RESONANCE IMAGING; PROCEDURES; STANDARDS; TRENDS; WORKFLOW;

INFORMATION DISSEMINATION; MAGNETIC RESONANCE IMAGING; NEUROIMAGING; REPRODUCIBILITY OF RESULTS; WORKFLOW;

EID: 84945497733     PISSN: None     EISSN: 2047217X     Source Type: Journal    
DOI: 10.1186/s13742-015-0055-8     Document Type: Article

References (78)

1. Galton F. Biometry. Biometrika 1901, 1(1):7-10. 10.1093/biomet/1.1.7.
2. Irreproducible LJ, Results E. Causes, (Mis)interpretations, and Consequences. Circulation 2012, 125:1211-4. 10.1161/CIRCULATIONAHA.112.098244.
3. Stodden V, Leisch F, Peng RD. Implementing Reproducible Research 2014, Taylor and Francis group CRC Press, Victoria.
4. Ioannidis JPA. Why Most Published Research Findings Are False. PLoS Med 2005, 2(8):e124. 10.1371/journal.pmed.0020124, 16060722.
5. Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, Robinson ESJ, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 2013, 14(5):365-76. 10.1038/nrn3475, 23571845.
6. Simonsohn U, Nelson LD, Simmons JP. P-curve: A key to the file-drawer. J Exp Psychol Gen 2014, 143:534-47. 10.1037/a0033242, 23855496.
7. Carp J. On the plurality of (methodological) worlds: Estimating the analytic flexibility of fMRI experiments. Front Neurosci. 2012;6(149)..
8. Aurich NK, Alves Filho JO, MarquesdaSilva AM, Franco AR. Evaluating the Reliability of Different Preprocessing Steps to Estimate Graph Theoretical Measures in Resting State fMRI data. Front Neurosci 2015, 9:48. 10.3389/fnins.2015.00048, 25745384.
9. Simmons JP, Nelson LD, Simonsohn U. False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychol Sci 2011, 22(11):1359-66. 10.1177/0956797611417632, 22006061.
10. Donoho DL, Maleki A, Rahman I, Shahram M, Stodden V. Reproducible Research in Computational Harmonic Analysis. Comput Sci Eng 2009, 11(1):8-18. 10.1109/MCSE.2009.15.
11. Monogan J. The Controversy of Preregistration in Social Research [Internet]. [cited 2015 Mar 13]. Available from: http://bitss.org/2014/06/13/preregistration-controversy/..
12. Rosenthal R. The file drawer problem and tolerance for null results. Psychol Bull 1979, 86:638. 10.1037/0033-2909.86.3.638.
13. Marwick B. Reproducible Research: A Primer for the Social Sciences [Internet] 2014, Reproducibility, Rpres.
14. Drummond C. Replicability is not reproducibility: nor is it good science. Evaluation Methods for Machine Learning Workshop [Internet]. Montreal, Quebec, CA; 2009. Available from: http://cogprints.org/7691/7/ICMLws09.pdf..
15. Peng RD. Reproducible Research in Computational Science. Science 2011, 334:1226-7. 10.1126/science.1213847, 22144613.
16. Reproducibility [Internet]. Wikipedia. [cited 2013 Mar 13]. Available from: http://en.wikipedia.org/wiki/Reproducibility..
17. Dryad [Internet]. [cited 2015 Mar 13]. Available from: http://datadryad.org/..
18. FigShare [Internet]. [cited 2015 Mar 13]. Available from: http://figshare.com/..
19. OpenScience framework [Internet]. [cited 2015 Mar 13]. Available from: https://osf.io/..
20. Zenodo [Internet]. [cited 2015 Mar 13]. Available from: http://zenodo.org/..
21. Poldrack RA, Fletcher PC, Henson RN, Worsley KJ, Brett M, Nichols TE. Guidelines for reporting an fMRI study. Neuroimage 2008, 40(2):409-14. 10.1016/j.neuroimage.2007.11.048, 18191585.
22. Ince DC, Hatton L, Graham-Cumming J. The case for open computer programs. Nature 2012, 482:485-8. 10.1038/nature10836, 22358837.
23. Osborne JM, Bernabeu MO, Bruna M, Calderhead B, Cooper J, Dalchau N, et al. Ten Simple Rules for Effective Computational Research. PLoS Comput Biol 2014, 10(3):e1003506. 10.1371/journal.pcbi.1003506, 24675742.
24. Sandve GK, Nekrutenko A, Taylor J, Hovig E. Ten Simple Rules for Reproducible Computational Research. PLoS Comput Biol 2013, 9(10):e1003285. 10.1371/journal.pcbi.1003285, 24204232.
25. Wellcome Trust Centre for Neuroimaging. Statistical Parametric Mapping [Internet]. [cited 2015 Mar 13]. Available from: http://www.fil.ion.ucl.ac.uk/spm/..
26. Flandin G, Friston KJ. Statistical parametric mapping (SPM). Scholarpedia 2008, 3(4):6332. 10.4249/scholarpedia.6232.
27. Ghosh S, Gorgolewski K. Neuroimaging in Pythom Pipelines and Interfaces [Internet]. [cited 2015 Mar 13]. Available from: http://nipy.sourceforge.net/nipype/..
28. Gorgolewski K, Burns CD, Madison C, Clark D, Halchenko YO, Waskom ML, et al. Nipype: A flexible, lightweight and extensible neuroimaging data processing framework. Front Neuroinformatics. 2011;5(13)..
29. Cox RW. Analysis of Functional NeuroImages [Internet]. [cited 2015 Mar 13]. Available from: http://afni.nimh.nih.gov/afni/..
30. Cox RW. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput Biomed Res. 1996;28:162-73..
31. FMRIB, Analysis Group. FMRIB Software Library [Internet]. [cited 2015 Mar 13]. Available from: http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/..
32. Jenkinson M, Beckmann CF, Behrens T, Woolrich MW, Smith SM. FSL. Neuroimage 2012, 62:782-90. 10.1016/j.neuroimage.2011.09.015, 21979382.
33. NeuroImaging Tool and Resources Clearinghouse [Internet]. [cited 2015 Mar 13]. Available from: http://www.nitrc.org/..
34. Gronenschild EHBM, Habets P, Jacobs HIL, Mengelers R, Rozendaal N, van Os J, et al. The Effects of FreeSurfer Version, Workstation Type, and Macintosh Operating System Version on Anatomical Volume and Cortical Thickness Measurements. PLoS One 2012, 7(6):e38234. 10.1371/journal.pone.0038234, 22675527.
35. Halchenko Y, Hanke M. Open is not enough. Let's take the next step: An integrated, community-driven computing platform for neuroscience. Front. Neuroinformatics 2012, 6:22.
36. snapshot.debian.org [Internet]. [cited 2013 Mar 13]. Available from: http://snapshot.debian.org/..
37. Comparison of revision control software [Internet]. Wikipedia. [cited 2013 Mar 13]. Available from: http://en.wikipedia.org/wiki/Comparison_of_revision_control_software..
38. Stodden V. The scientific method in practice: Reproducibility in the computational sciences. MIT Sloan Res Pap. 2010;4773-10..
39. Barnes N. Publish your computer code: it is good enough. Nature 2010, 467:753. 10.1038/467753a, 20944687.
40. git [Internet]. [cited 2015 Mar 13]. Available from: http://git-scm.com/..
41. Subversion [Internet]. [cited 2015 Mar 13]. Available from: http://subversion.apache.org/..
42. Github [Internet]. [cited 2015 Mar 13]. Available from: https://github.com/..
43. Bitbucket [Internet]. [cited 2015 Mar 13]. Available from: https://bitbucket.org/..
44. Workflow [Internet]. Wikipedia. [cited 2015 Mar 13]. Available from: http://en.wikipedia.org/wiki/Workflow..
45. Bellec P. PSOM [Internet]. [cited 2015 Mar 13]. Available from: https://github.com/SIMEXP/psom..
46. Bellec P, Lavoie-Courchesne S, Dickinson P, Lerch J, Zijdenbos A, Evans AC. The pipeline system for Octave and Matlab (PSOM): a lightweight scripting framework and execution engine for scientific workflows. Front Neuroinformatics. 2012;6(7)..
47. Mitchell D, Auer T. Automatic Analysis pipeline [Internet]. [cited 2015 Mar 13]. Available from: http://imaging.mrc-cbu.cam.ac.uk/imaging/AA..
48. Cusack R, Vicente-Grabovetsky A, Mitchell DJ, Wild CJ, Auer T, Linke A, et al. Automatic analysis (aa): efficient neuroimaging workflows and parallel processing using Matlab and XML. Front Neuroinformatics. 2015;8(90)..
49. IPython Notebook [Internet]. [cited 2015 Mar 13]. Available from: http://ipython.org/notebook.html..
50. Taverna [Internet]. [cited 2015 Mar 13]. Available from: http://www.taverna.org.uk/..
51. Kepler [Internet]. [cited 2015 Mar 13]. Available from: https://kepler-project.org/..
52. Laboratory of Neuro Imaging. LONI pipeline [Internet]. [cited 2015 Mar 13]. Available from: http://pipeline.bmap.ucla.edu/..
53. Torgerson CM, Quinn C, Dinov I, Liu Z, Petrosyan P, Pelphrey K, et al. Interacting with the National Database for Autism Research (NDAR) via the LONI Pipeline workflow environment. Brain Imaging Behav. 2015;9;89-103..
54. Poline J-B, Breeze JL, Ghosh S, Gorgolewski K, Halchenko YO, Hanke M, et al. Data sharing in neuroimaging research. Front Neuroinformatics. 2012..
55. Gorgolewski K, Storkey AJ, Bastin ME, Whittle I, Wardlaw J, Pernet CR. A test-retest fMRI dataset for motor, language and spatial attention functions. Gigascience 2013, 2:6. 10.1186/2047-217X-2-6, 23628139.
56. Hanke M, Baumgartner FJ, Ibe P, Kaule FR, Pollmann S, Speck O, et al. A high-resolution 7-Tesla fMRI dataset from complex natural stimulation with an audio movie. Sci Data. 2014 May 27;1..
57. Poldrack RA. OpenfMRI [Internet]. [cited 2015 Mar 13]. Available from: https://openfmri.org/..
58. Poldrack RA, Barch DM, Mitchell JP, Wager TD, Wagner AD, Devlin JT, et al. Toward open sharing of task-based fMRI data: the OpenfMRI project. Front Neuroinformatics. 2013;7..
59. fMRI Data Center [Internet]. [cited 2015 Mar 13]. Available from: http://databib.org/repository/371..
60. Van Horn JD, Gazzaniga MS. Why share data? Lessons learned from the fMRIDC. Neuroimage 2013, 82:677-82. 10.1016/j.neuroimage.2012.11.010, 23160115.
61. Calhoun VD. A spectrum of sharing: maximization of information content for brain imaging data. GigaScience. 2015 Dec;4(1)..
62. Laboratory for Computational Neuroimaging. MRI Deface. [cited 2015 Mar 13]. Available from: http://www.nitrc.org/projects/mri_deface/..
63. Bischoff-Grethe A, Ozyurt IB, Busa E, Quinn BT, Fennema-Notestine C, Clark CP, et al. A Technique for the Deidentification of Structural Brain MR Images. Hum Brain Mapp 2007, 28(9):892-903. 10.1002/hbm.20312, 17295313.
64. Goodman A, Pepe A, Blocker AW, Borgman C, Cranner K, Crosas M, et al. Ten Simple Rules for the Care and Feeding of Scientific Data. PLoS Comput Biol 2014, 10(4):e1003542. 10.1371/journal.pcbi.1003542, 24763340.
65. World Wide Web Consortium Provenance Group [Internet]. [cited 2015 Mar 13]. Available from: http://www.w3.org/2011/prov/wiki/Main_Page..
66. Poldrack RA, Gorgolewski KJ. Making big data open: data sharing in neuroimaging. Nat Neurosci 2014, 17:11. 10.1038/nn.3818.
67. International Neuroinformatics Coordinating Facility [Internet]. Available from: http://www.incf.org/..
68. Neuroinformatics Coordinating Facility data sharing task force [Internet]. [cited 2015 Mar 13]. Available from: http://wiki.incf.org/mediawiki/index.php/Neuroimaging_Task_Force..
69. Neuro-Imaging Data Model [Internet]. [cited 2015 Mar 13]. Available from: http://nidm.nidash.org/..
70. Johnson VE. Revised Standards for Statistical Evidence. Proc Natl Acad Sci U S A 2013, 110(48):19313-7. 10.1073/pnas.1313476110, 24218581.
71. Jernigan TL, Gamst AC, Fennema-Notestine C, Ostergaard AL. More " mapping" in brain mapping: Statistical comparison of effects. Hum Brain Mapp 2003, 19(2):90-5. 10.1002/hbm.10108, 12768533.
72. Gorgolewski K. NeuroVault [Internet]. [cited 2015 Mar 13]. Available from: http://neurovault.org/..
73. Gorgolewski K, Varoquaux G, Rivers G, Schwartz Y, Ghosh SS, Maumet C, et al. NeuroVault.org: A web-based repository for collecting and sharing unthresholded statistical maps of the human brain. Bio Arch X. 2014;pre-print..
74. Rousselet GA, Pernet CR. Improving standards in brain-behavior correlation analyses. Front Hum Neurosci. 2012;6..
75. Creative Commons organization [Internet]. [cited 2015 Mar 13]. Available from: http://creativecommons.org/choose/..
76. Open Data Commons [Internet]. [cited 2015 Mar 13]. Available from: http://opendatacommons.org/licenses/pddl/..
77. Piwowar HA, Day RS, Fridsma DB. Sharing Detailed Research Data Is Associated with Increased Citation Rate. PLoS One 2007, 2(3):e308. 10.1371/journal.pone.0000308, 17375194.
78. Harnad S. Publish or Perish - Self-Archive to Flourish: The Green Route to Open Access. Eur Res Consort Inform Math. 2006;64..