Circular analysis in systems neuroscience: The dangers of double dipping

Nature Neuroscience

Volumn 12, Issue 5, 2009, Pages 535-540

  Kriegeskorte, Nikolaus ^a   Simmons, W Kyle ^a   Bellgowan, Patrick S ^a   Baker, Chris I ^a  

^a NATIONAL INSTITUTE OF MENTAL HEALTH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL; ARTIFACT; EPIDEMIOLOGY; HUMAN; IMAGE PROCESSING; INFORMATION PROCESSING; METHODOLOGY; NEUROSCIENCE; NUCLEAR MAGNETIC RESONANCE IMAGING; REPRODUCIBILITY; REVIEW; STANDARD; STATISTICAL ANALYSIS; SYSTEMS BIOLOGY;

ANIMALS; ARTIFACTS; DATA COLLECTION; DATA INTERPRETATION, STATISTICAL; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; NEUROSCIENCES; REPRODUCIBILITY OF RESULTS; SELECTION BIAS; SYSTEMS BIOLOGY;

EID: 67649135107     PISSN: 10976256     EISSN: 15461726     Source Type: Journal    
DOI: 10.1038/nn.2303     Document Type: Article

References (24)

1. Baker, C.I., Hutchison, T.L. & Kanwisher, N. Does the fusiform face area contain subregions highly selective for nonfaces? Nat. Neurosci. 10, 3–4 (2007).
2. Simmons, W.K. et al. Imaging the context-sensitivity of ventral temporal category representations using high-resolution fMRI. Soc. Neurosci. Abstr. 262.12 (2006).
3. Baker, C.I., Simmons, W.K., Bellgowan, P.S. & Kriegeskorte, N. Circular inference in neuroscience: the dangers of double dipping. Soc. Neurosci. Abstr. 104.11 (2007).
4. Vul, E. & Kanwisher, N. Begging the question: the non-independence error in fMRI data analysis. in Foundations and Philosophy for Neuroimaging (eds. Hanson, S. & Bunzl, M.) (in the press).
5. Vul, E., Harris, C., Winkielman, P. & Pashler, H. Puzzlingly high correlations in fMRI studies of emotion, personality and social cognition. Perspect. Psychol. Sci. (in the press).
6. Benjamini, Y. Comment: microarrays, empirical bayes and the two-groups model. Statist. Sci. 23, 23–28 (2008).
7. Worsley, K.J., Evans, A.C., Marrett, S. & Neelin, P. A three-dimensional statistical analysis for CBF activation studies in human brain. J. Cereb. Blood Flow Metab. 12, 900–918 (1992).
8. Friston, K.J., Jezzard, P. & Turner, R. Analysis of functional MRI time-series. Hum. Brain Mapp. 1, 153–171 (1994).
9. Worsley, K.J. & Friston, K.J. Analysis of fMRI time-series revisited– again. Neuroimage 2, 173–181 (1995).
10. Nichols, T. & Hayasaka, S. Controlling the family-wise error rate in functional neuroimaging: a comparative review. Stat. Methods Med. Res. 12, 419–446 (2003).
11. Genovese, C.R., Lazar, N.A. & Nichols, T. Thresholding of statistical maps in functional neuroimaging using the false discovery rate. Neuroimage 15, 870–878 (2002).
12. Friston, K.J., Rotshtein, P., Geng, J.J., Sterzer, P. & Henson, R.N. A critique of functional localisers. Neuroimage 30, 1077–1087 (2006).
13. Saxe, R., Brett, M. & Kanwisher, N. Divide and conquer: a defense of functional localizers. Neuroimage 30, 1088–1096 (2006).
14. Haxby, J.V. et al. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293, 2425–2430 (2001).
15. Cox, D.D. & Savoy, R.L. Functional magnetic resonance imaging (fMRI) ‘brain reading’: detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 19, 261–270 (2003).
16. Mitchell, T.M., Hutchinson, R., Niculescu, R.S., Pereira, F. & Wang, X. Learning to decode cognitive states from brain images. Mach. Learn. 57, 145–175 (2004).
17. Kriegeskorte, N., Goebel, R. & Bandettini, P. Information-based functional brain mapping. Proc. Natl. Acad. Sci. USA 103, 3863–3868 (2006).
18. Norman, K.A., Polyn, S.M., Detre, G.J. & Haxby, J.V. Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn. Sci. 10, 424–430 (2006).
19. Strother, S.C. et al. The quantitative evaluation of functional neuroimaging experiments: the NPAIRS data analysis framework. Neuroimage 15, 747–771 (2002).
20. Strother, S. et al. Optimizing the fMRI data-processing pipeline using prediction and reproducibility performance metrics. I. A preliminary group analysis. Neuroimage 23, S196–S207 (2004).
21. Mitchell, T.M. Machine Learning (McGraw Hill, Portland, Oregon, 1997).
22. Duda, R.O., Hart, P.E. & Stork, D.G. Pattern Classification. 2nd edn. (Wiley, New York, 2001).
23. Bishop, C.M. Pattern Recognition and Machine Learning (Springer, New York, 2006).
24. Kant, I. Kritik der reinen Vernunft. 2nd edn. http://www.gutenberg.org/dirs/etext04/8ikc210.txt (1781). 540