Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): Random permutations and cluster size control

NeuroImage

Volumn 65, Issue , 2013, Pages 69-82

  Stelzer, Johannes ^a   Chen, Yi ^a,b   Turner, Robert ^a  

^a MAX PLANCK INSTITUTE FOR HUMAN COGNITIVE AND BRAIN SCIENCES   (Germany)

^b CHARITÉ UNIVERSITÄTSMEDIZIN BERLIN   (Germany)

Author keywords

Cluster size control; FMRI; Multiple testing; MVPA; Second level analysis; Statistics

Indexed keywords

ACCURACY; ARTICLE; CLASSIFICATION; CLUSTER ANALYSIS; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; INTERMETHOD COMPARISON; MULTIVOXEL PATTERN ANALYSIS; NEUROIMAGING; NORMAL HUMAN; PRIORITY JOURNAL; SENSITIVITY AND SPECIFICITY; SIMULATION; STATISTICAL ANALYSIS; STATISTICAL SIGNIFICANCE; STUDENT T TEST; VALIDITY;

BRAIN MAPPING; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MODELS, NEUROLOGICAL; MODELS, THEORETICAL; SENSITIVITY AND SPECIFICITY;

EID: 84868260207     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2012.09.063     Document Type: Article

References (33)

1. Arndt S., Cizadlo T., Andreasen N.C., Heckel D., Gold S., O'Leary D.S. Tests for comparing images based on randomization and permutation methods. J. Cereb. Blood Flow Metab. 1996, 16:1271-1279.
2. Benjamini Y. A step-down multiple hypotheses testing procedure that controls the false discovery rate under independence. Journal of Statistical Planning and Inference 1999, (Journal of Statistical Planning and Inference | ScienceDirect.com.). 10.1016/S0378-3758(99)00040-3.
3. Björnsdotter M., Rylander K., Wessberg J. A Monte Carlo method for locally multivariate brain mapping. NeuroImage 2011, 56(2):508-516. 10.1016/j.neuroimage.2010.07.044.
4. Bode S., Haynes J.D. Decoding sequential stages of task preparation in the human brain. NeuroImage 2009, 45:606-613.
5. Carlin J.D., Rowe J.B., Kriegeskorte N., Thompson R., Calder A.J. Direction-Sensitive Codes for Observed Head Turns in Human Superior Temporal Sulcus. Cereb. Cortex 2012, 22(4):735-744.
6. Chang Chih-Chung, Lin Chih-Jen LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2011, 2:27:1-27:27. (Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm).
7. Chen Y., Namburi P., Elliott L.T., Heinzle J., Soon C.S., Chee M.W., Haynes J.D. Cortical surface-based searchlight decoding. NeuroImage 2011, 56:582-592.
8. Chumbley J.R., Friston K.J. False discovery rate revisited: FDR and topological inference using Gaussian random fields. NeuroImage 2009, 44:62-70.
9. Efron A., Tibshirani R.J. An Introduction to the Bootstrap 1993, Chapmann & Hall.
10. Forman S.D., Cohen J.D., Fitzgerald M., Eddy W.F., Mintun M.A., Noll D.C. Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn. Reson. Med. 1995, 33:636-647.
11. Golland P., Fischl B. Permutation tests for classification: towards statistical significance in image-based studies. Inf. Process. Med. Imaging 2003, 18:330-341.
12. Good P. Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses 2006, 2nd edition.
13. Guillot A., Collet C., Nguyen V.A., Malouin F., Richards C., Doyon J. Brain activity during visual versus kinesthetic imagery: an fMRI study. Hum. Brain Mapp. 2009, 30(7):2157-2172. 10.1002/hbm.20658.
14. Hanke M., Halchenko Y.O., Sederberg P.B., Hanson S.J., Haxby J.V., Pollmann S. PyMVPA: a Python Toolbox for Multivariate Pattern Analysis of fMRI data. Neuroinformatics 2009, 7(1):37-53. 10.1007/s12021-008-9041-y.
15. Hayasaka S., Nichols T.E. Validating cluster size inference: random field and permutation methods. NeuroImage 2003, 20:2343-2356.
16. Hefny A., Atiya A.F. A New Monte Carlo-Based Error Rate Estimator 2010.
17. Heller R., Stanley D., Yekutieli D., Rubin N., Benjamini Y. Cluster-based analysis of FMRI data. NeuroImage 2006, 33:599-608.
18. Holmes A.P., Blair R.C., Watson J.D., Ford I. Nonparametric analysis of statistic images from functional mapping experiments. J. Cereb. Blood Flow Metab. 1996, 16:7-22.
19. Ibe O. Fundamentals of Applied Probability and Random Processes 2005, Academic Press.
20. Isaksson A., Wallman M., Göransson H., Gustafsson M.G. Cross-validation and bootstrapping are unreliable in small sample classification. Pattern Recognit. Lett. 2008, 29(14):1960-1965.
21. Kahnt T., Heinzle J., Park S.Q., Haynes J.D. The neural code of reward anticipation in human orbitofrontal cortex. Proc. Natl. Acad. Sci. U. S. A. 2010, 107:6010-6015.
22. Kriegeskorte N. Pattern-information analysis: from stimulus decoding to computational-model testing. NeuroImage 2011, 56:411-421.
23. Kriegeskorte N., Goebel R., Bandettini P. Information-based functional brain mapping. Proc. Natl. Acad. Sci. U. S. A. 2006, 103:3863-3868.
24. Krueger F., Fischer R., Heinecke A., Hagendorf H. An fMRI investigation into the neural mechanisms of spatial attentional selection in a location-based negative priming task. Brain Res. 2007, 1174:110-119. 10.1016/j.brainres.2007.08.016.
25. Langford J. Tutorial on practical prediction theory for classification. J. Mach. Learn. Res. 2005, 6:273-306.
26. Lohmann G., Müller K., Bosch V., Mentzel H., Hessler S., Chen L., Zysset S., et al. LIPSIA-a new software system for the evaluation of functional magnetic resonance images of the human brain. Comput. Med. Imaging Graph. 2001, 25(6):449-457.
27. Nichols T.E., Holmes A.P. Nonparametric permutation tests for functional neuroimaging: a primer with examples. Hum. Brain Mapp. 2002, 15:1-25.
28. Pereira F., Botvinick M. Information mapping with pattern classifiers: a comparative study. NeuroImage 2011, 56(2):476-496.
29. Prado J., Weissman D.H. Spatial attention influences trial-by-trial relationships between response time and functional connectivity in the visual cortex. NeuroImage 2011, 54(1):465-473. 10.1016/j.neuroimage.2010.08.038.
30. Sato W., Kochiyama T., Uono S., Yoshikawa S. Commonalities in the neural mechanisms underlying automatic attentional shifts by gaze, gestures, and symbols. NeuroImage 2009, 45(3):984-992. 10.1016/j.neuroimage.2008.12.052.
31. Thakral P.P., Slotnick S.D. The role of parietal cortex during sustained visual spatial attention. Brain Res. 2009, 1302(C):157-166. 10.1016/j.brainres.2009.09.031.
32. Wickenberg-Bolin U., Goransson H., Fryknas M., Gustafsson M.G., Isaksson A. Improved variance estimation of classification performance via reduction of bias caused by small sample size. BMC Bioinforma. 2006, 7:127.
33. Zarahn E., Aguirre G.K., D'Esposito M. Empirical analyses of BOLD fMRI statistics. I. Spatially unsmoothed data collected under null-hypothesis conditions. NeuroImage 1997, 5:179-197.