On the plurality of (methodological) worlds: Estimating the analytic flexibility of fmri experiments

Frontiers in Neuroscience

Volumn , Issue OCT, 2012, Pages

  Carp, Joshua ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Analysis flexibility; Data analysis; False positive results; fMRI; Selective reporting

Indexed keywords

ARTICLE; DIAGNOSTIC ACCURACY; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUNCTIONAL NEUROIMAGING; IMAGE ANALYSIS; IMAGE PROCESSING; KERNEL METHOD; MATHEMATICAL MODEL; MONTE CARLO METHOD;

EID: 84870979883     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: None     Document Type: Article

References (45)

1. Aron, A., Behrens, T., Smith, S., Frank, M., and Poldrack, R. (2007). Triangulating a cognitive control network using diffusion-weighted magnetic resonance imaging (MRI) and functional MRI. J. Neurosci. 27, 3743-3752.
2. Aron, A., Robbins, T., and Poldrack, R. (2004). Inhibition and the right inferior frontal cortex. Trends Cogn. Sci. (Regul. Ed.) 8, 170-177.
3. Bossuyt, P., Reitsma, J., Bruns, D., Gat-sonis, C., Glasziou, P., Irwig, L., et al. (2003). Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Br. Med. J. 326, 41-44.
4. Carp, J. (2011). Optimizing the order of operations for movement scrubbing: comment on Power et al. Neuroimage. doi: 10.1016/j.neuroimage.2011.12.061
5. Carp, J. (2012). The secret lives of experiments: methods reporting in the fMRI literature. Neuroimage 63, 289-300.
6. Chan, A.-W., Hróbjartsson, A., Haahr, (2004a). Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. J. Am. Med. Assoc. 291, 2457-2465.
7. Chan, A.-W., Krlẽa-Jeric, K., Schmid, I., andAltman,D.(2004b). Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. Can. Med. Assoc. J. 171, 735-740.
8. Churchill, N., Oder, A., Abdi, H., Tam, F., Lee, W., Thomas, C., et al. (2012a). Optimizing preprocessing and analysis pipelines for single-subject fMRI. I. Standard temporal motion and physiological noise correction methods. Hum. Brain Mapp. 33, 609-627.
9. Churchill, N., Yourganov, G., Oder, A., Tam, F., Graham, S., and Strother, S. (2012b). Optimizing preprocessing and analysis pipelines for single-subject fMRI: 2. Interactions with ICA, PCA, task contrast and inter-subject heterogeneity. PLoS ONE 7, e31147. doi:10.1371/journal.pone.0031147
10. Churchill, N., Yourganov, G., Spring, R., Rasmussen, P., Lee, W., Ween, J., et al. (2012c). PHYCAA: data-driven measurement and removal of physiological noise in BOLD fMRI. Neuroimage 59, 1299-1314.
11. Della-Maggiore, V., Chau, W., Peres-Neto, P., and McIntosh, A. (2002). An empirical comparison of SPM preprocessing parameters to the analysis of fMRI data. Neuroimage 17, 19-28.
12. Dwan, K., Altman, D.,Arnaiz, J., Bloom, J., Chan, A.-W., Cronin, E., et al. (2008). Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE 3, e3081. doi:10.1371/journal.pone.0003081
13. Friston, K. J., Williams, S., Howard, R., Frackowiak, R. S., and Turner, R. (1996). Movement-related effects in fMRI time-series.Magn. Reson. Med. 35, 346-355.
14. Genovese, C., Lazar, N., and Nichols, T. (2002). Thresholding of statistical maps in functional neuroimag-ing using the false discovery rate. Neuroimage 15, 870-878.
15. Glover, G. H., Li, T. Q., and Ress, D. (2000). Image-based method for retrospective correction of physiological motion effects in fMRI: RETROICOR. Magn. Reson. Med. 44, 162-167.
16. Hopfinger, J. B., Büchel, C., Holmes, A. P., and Friston, K. J. (2000). A study of analysis parameters that influence the sensitivity of event-related fMRI analyses. Neuroimage 11, 326-333.
17. Ince, D., Hatton, L., and Graham-Cumming, J. (2012). The case for open computer programs. Nature 482, 485-488.
18. Ioannidis, J. (2005a). Contradicted and initially stronger effects in highly cited clinical research. J. Am. Med. Assoc. 294, 218-228.
19. Ioannidis, J. (2005b). Why most published research findings are false. PLoS Med. 2, e124. doi:10.1371/journal.pmed.0020124
20. Ioannidis, J. (2011). Excess significance bias inthe literatureon brain volume abnormalities. Arch. Gen. Psychiatry 68, 773-780.
21. John, L., Loewenstein, G., and Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol. Sci. 23, 524-532.
22. Lieberman, M., and Cunningham, W. (2009). TypeIand Type IIerror concerns in fMRI research: re-balancing the scale. Soc. Cogn. Affect. Neurosci. 4, 423-428.
23. Lund, T., Nørgaard, M., Rostrup, E., Rowe, J., and Paulson, O. (2005). Motion or activity: their role in intra- and inter-subject variation in fMRI. Neuroimage 26, 960-964.
24. Mathieu, S., Boutron, I., Moher, D., Altman, D., and Ravaud, P. (2009). Comparison of registered and published primary outcomes in randomized controlled trials. J. Am. Med. Assoc. 302, 977-984.
25. Moher, D., Schulz, K. F., Altman, D., and Group, C. (2001). The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. J. Am. Med. Assoc. 285, 1987-1991.
26. Murphy, K., Birn, R., Handwerker, D., Jones, T., and Bandettini, P. (2009). The impact of global signal regression on resting state correlations: are anti-correlated networks introduced? Neuroimage 44, 893-905.
27. Nichols, T., and Hayasaka, S. (2003). Controlling the familywise error rate in functional neuroimaging: a comparative review. Stat. Methods Med. Res. 12, 419-446.
28. Plint, A., Moher, D., Morrison, A., Schulz, K., Altman, D., Hill, C., et al. (2006). Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med. J. Aust. 185, 263-267.
29. Poldrack, R., Fletcher, P., Henson, R., Worsley, K., Brett, M., and Nichols, T. (2008). Guidelines for reporting an fMRI study. Neuroimage 40, 409-414.
30. Poldrack, R., Mumford, J., and Nichols, T. (2011). Handbook of Functional MRI Data Analysis. Cambridge: Cambridge University Press.
31. Poline, J.-B., Strother, S., Dehaene-Lambertz, G., Egan, G., and Lancaster, J. (2006). Motivation and synthesis of the FIAC experiment: reproducibility of fMRI results across expert analyses. Hum. Brain Mapp. 27, 351-359.
32. Power, J., Barnes, K., Snyder, A., Schlag-gar, B., and Petersen, S. (2012). Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion. Neuroimage 59, 2142-2154.
33. Purdon, P. L., and Weisskoff, R. M. (1998). Effect of temporal autocorrelation due to physiological noise and stimulus paradigm on voxel-level false-positive rates in fMRI. Hum. Brain Mapp. 6, 239-249.
34. Simmons, J., Nelson, L., and Simon-sohn, U. (2011). False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol. Sci. 22, 1359-1366.
35. Skudlarski, P., Constable, R. T., and Gore, J. C. (1999). ROC analysis of statistical methods used in functional MRI: individual subjects. Neuroimage 9, 311-329.
36. Sladky, R., Friston, K., Tröstl, J., Cunnington, R., Moser, E., and Windischberger, C. (2011). Slice-timing effects and their correction in functional MRI. Neuroimage 58, 588-594.
37. Smith, S., Beckmann, C., Ramnani, N., Woolrich, M., Bannister, P., Jenkinson, M., et al. (2005). Variability in fMRI: a re-examination of inter-session differences. Hum. Brain Mapp. 24, 248-257.
38. Song, X. W., Dong, Z. Y., Long, X. Y., Li., S. F., Zuo, X. N., Zhu, C. Z., et al. (2011). REST: a toolkit for resting-state functional magnetic resonance imaging data processing. PLoS ONE 6, e25031. doi:10.1371/journal.pone.0025031
39. Strother, S., La Conte, S., Hansen, L., Anderson, J., Zhang, J., Pulapura, S., et al. (2004). Optimizing the fMRI data-processing pipeline using prediction and repro-ducibility performance metrics: I. A preliminary group analysis. Neuroimage 23(Suppl. 1), S196-S207.
40. Tohka, J., Foerde, K., Aron, A., Tom, S., Toga, A., and Poldrack, R. (2008). Automatic independent component labeling for artifact removalinfMRI. Neuroimage 39, 1227-1245.
41. Tzourio-Mazoyer, N., Landeau, B., Pap-athanassiou, D., Crivello, F., Etard, O., Delcroix, N., et al. (2002). Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroim-age 15, 273-289.
42. von Elm, E., Altman, D., Egger, M., Pocock, S., Gøtzsche, P., Vanden-broucke, J., et al. (2007). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. PLoS Med. 4, e296. doi:10.1371/journal.pmed.0040296
43. Wager, T., Lindquist, M., Nichols, T., Kober, H., and Van Snellenberg, J. (2009). Evaluating the consistency and specificityof neuroimaging data using meta-analysis. Neuroimage 45, S210-S221.
44. Weissenbacher, A., Kasess, C., Gerstl, F., Lanzenberger, R., Moser, E., and Windischberger, C. (2009). Correlations and anticorrelations in resting-state functional connectivity MRI: a quantitative comparison of preprocessing strategies. Neuroimage 47, 1408-1416.
45. Woolrich, M. W., Ripley, B. D., Brady, M., and Smith, S. M. (2001). Temporal autocorrelation in univariate linear modeling of FMRI data. Neu-roimage 14, 1370-1386.