A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives

Neuron

Volumn 87, Issue 2, 2015, Pages 257-270

  Haynes, John Dylan ^a,b,c  

^a CHARITÉ UNIVERSITÄTSMEDIZIN BERLIN   (Germany)

^b HUMBOLDT UNIVERSITY   (Germany)

^c DRESDEN UNIVERSITY OF TECHNOLOGY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN FUNCTION; BRAIN MAPPING; BRAIN REGION; COGNITION; EVOKED RESPONSE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUNCTIONAL NEUROIMAGING; HUMAN; IMAGE ANALYSIS; IMAGE PROCESSING; IMAGE QUALITY; NERVE CELL NETWORK; NERVE FUNCTION; PRIORITY JOURNAL; REVIEW; SIGNAL PROCESSING; BIOLOGICAL MODEL; BLOOD; BRAIN; COMPUTER ASSISTED DIAGNOSIS; COMPUTER SIMULATION; MENTAL FUNCTION; NUCLEAR MAGNETIC RESONANCE IMAGING; VASCULARIZATION;

OXYGEN;

BRAIN; BRAIN MAPPING; COMPUTER SIMULATION; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MENTAL PROCESSES; MODELS, NEUROLOGICAL; OXYGEN;

EID: 84937390759     PISSN: 08966273     EISSN: 10974199     Source Type: Journal    
DOI: 10.1016/j.neuron.2015.05.025     Document Type: Review

References (124)

1. Adams D.L., Sincich L.C., Horton J.C. Complete pattern of ocular dominance columns in human primary visual cortex. J.Neurosci. 2007, 27:10391-10403.
2. Albright T.D., Stoner G.R. Contextual influences on visual processing. Annu. Rev. Neurosci. 2002, 25:339-379.
3. Alink A., Krugliak A., Walther A., Kriegeskorte N. fMRI orientation decoding in V1 does not require global maps or globally coherent orientation stimuli. Front. Psychol. 2013, 4:493.
4. Allefeld C., Haynes J.D. Searchlight-based multi-voxel pattern analysis of fMRI by cross-validated MANOVA. Neuroimage 2014, 89:345-357.
5. Allefeld C., Haynes J.D. Multi-Voxel Pattern Analysis. Brain Mapping: An Encyclopedic Reference 2015, 641-646. Academic Press, Waltham. A.W. Toga (Ed.).
6. Anders S., Heinzle J., Weiskopf N., Ethofer T., Haynes J.D. Flow of affective information between communicating brains. Neuroimage 2011, 54:439-446.
7. Attneave F. Some informational aspects of visual perception. Psychol. Rev. 1954, 61:183-193.
8. Bishop C.M. Neural Networks for Pattern Recognition 1995, OUP, Oxford.
9. Boynton G.M. Imaging orientation selectivity: decoding conscious perception in V1. Nat. Neurosci. 2005, 8:541-542.
10. Braver T.S., Cohen J.D., Nystrom L.E., Jonides J., Smith E.E., Noll D.C. A parametric study of prefrontal cortex involvement in human working memory. Neuroimage 1997, 5:49-62.
11. Brouwer G.J., Heeger D.J. Decoding and reconstructing color from responses in human visual cortex. J.Neurosci. 2009, 29:13992-14003.
12. Carandini M., Heeger D.J. Normalization as a canonical neural computation. Nat. Rev. Neurosci. 2011, 13:51-62.
13. Chaimow D., Yacoub E., Ugurbil K., Shmuel A. Modeling and analysis of mechanisms underlying fMRI-based decoding of information conveyed in cortical columns. Neuroimage 2011, 56:627-642.
14. Chen Y., Namburi P., Elliott L.T., Heinzle J., Soon C.S., Chee M.W., Haynes J.D. Cortical surface-based searchlight decoding. Neuroimage 2011, 56:582-592.
15. Chu C., Ni Y., Tan G., Saunders C.J., Ashburner J. Kernel regression for fMRI pattern prediction. Neuroimage 2011, 56:662-673.
16. Cichy R.M., Chen Y., Haynes J.D. Encoding the identity and location of objects in human LOC. Neuroimage 2011, 54:2297-2307.
17. Cichy R.M., Heinzle J., Haynes J.D. Imagery and perception share cortical representations of content and location. Cereb. Cortex 2012, 22:372-380.
18. Cichy R.M., Pantazis D., Oliva A. Resolving human object recognition in space and time. Nat. Neurosci. 2014, 17:455-462.
19. Cox D.D., Savoy R.L. Functional magnetic resonance imaging (fMRI) "brain reading": detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 2003, 19:261-270.
20. De Baene W., Vogels R. Effects of adaptation on the stimulus selectivity of macaque inferior temporal spiking activity and local field potentials. Cereb. Cortex 2010, 20:2145-2165.
21. De Martino F., Valente G., Staeren N., Ashburner J., Goebel R., Formisano E. Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns. Neuroimage 2008, 43:44-58.
22. Downing P.E., Chan A.W., Peelen M.V., Dodds C.M., Kanwisher N. Domain specificity in visual cortex. Cereb. Cortex 2006, 16:1453-1461.
23. Duda R.O., Hart P.E., Stork D.G. Pattern Classification 2000, Wiley, New York. Second Edition.
24. Dumoulin S.O., Wandell B.A. Population receptive field estimates in human visual cortex. Neuroimage 2008, 39:647-660.
25. Etzel J.A., Zacks J.M., Braver T.S. Searchlight analysis: promise, pitfalls, and potential. Neuroimage 2013, 78:261-269.
26. Fisher R.A. The use of multiple measurements in taxonomic problems. Ann. Eugen. 1936, 7:179-188.
27. Freeman J., Brouwer G.J., Heeger D.J., Merriam E.P. Orientation decoding depends on maps, not columns. J.Neurosci. 2011, 31:4792-4804.
28. Freeman J., Heeger D.J., Merriam E.P. Coarse-scale biases for spirals and orientation in human visual cortex. J.Neurosci. 2013, 33:19695-19703.
29. Friston K.J., Frith C.D., Frackowiak R.S., Turner R. Characterizing dynamic brain responses with fMRI: a multivariate approach. Neuroimage 1995, 2:166-172.
30. Friston K.J., Holmes A.P., Poline J.B., Grasby P.J., Williams S.C., Frackowiak R.S., Turner R. Analysis of fMRI time-series revisited. Neuroimage 1995, 2:45-53.
31. Furmanski C.S., Engel S.A. An oblique effect in human primary visual cortex. Nat. Neurosci. 2000, 3:535-536.
32. Gardner J.L. Is cortical vasculature functionally organized?. Neuroimage 2010, 49:1953-1956.
33. Goldfine A.M., Bardin J.C., Noirhomme Q., Fins J.J., Schiff N.D., Victor J.D. Reanalysis of "Bedside detection of awareness in the vegetative state: a cohort study". Lancet 2013, 381:289-291.
34. Gratton C., Sreenivasan K.K., Silver M.A., D'Esposito M. Attention selectively modifies the representation of individual faces in the human brain. J.Neurosci. 2013, 33:6979-6989.
35. Grill-Spector K., Kushnir T., Edelman S., Avidan G., Itzchak Y., Malach R. Differential processing of objects under various viewing conditions in the human lateral occipital complex. Neuron 1999, 24:187-203.
36. Hackmack K., Paul F., Weygandt M., Allefeld C., Haynes J.D. Multi-scale classification of disease using structural MRI and wavelet transform. Neuroimage 2012, 62:48-58. Alzheimer's Disease Neuroimaging Initiative.
37. Harvey B.M., Klein B.P., Petridou N., Dumoulin S.O. Topographic representation of numerosity in the human parietal cortex. Science 2013, 341:1123-1126.
38. Haufe S., Meinecke F., Görgen K., Dähne S., Haynes J.D., Blankertz B., Bießmann F. On the interpretation of weight vectors of linear models in multivariate neuroimaging. Neuroimage 2014, 87:96-110.
39. Haxby J.V., Gobbini M.I., Furey M.L., Ishai A., Schouten J.L., Pietrini P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 2001, 293:2425-2430.
40. Haxby J.V., Connolly A.C., Guntupalli J.S. Decoding neural representational spaces using multivariate pattern analysis. Annu. Rev. Neurosci. 2014, 37:435-456.
41. Haynes J.D., Rees G. Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nat. Neurosci. 2005, 8:686-691.
42. Haynes J.D., Rees G. Decoding mental states from brain activity in humans. Nat. Rev. Neurosci. 2006, 7:523-534.
43. Haynes J.D., Sakai K., Rees G., Gilbert S., Frith C., Passingham R.E. Reading hidden intentions in the human brain. Curr. Biol. 2007, 17:323-328.
44. Heinzle J., Kahnt T., Haynes J.D. Topographically specific functional connectivity between visual field maps in the human brain. Neuroimage 2011, 56:1426-1436.
45. Huth A.G., Nishimoto S., Vu A.T., Gallant J.L. A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron 2012, 76:1210-1224.
46. Ioannidis J.P., Ntzani E.E., Trikalinos T.A., Contopoulos-Ioannidis D.G. Replication validity of genetic association studies. Nat. Genet. 2001, 29:306-309.
47. Josephs O., Henson R.N. Event-related functional magnetic resonance imaging: modelling, inference and optimization. Philos. Trans. R. Soc. Lond. B Biol. Sci. 1999, 354:1215-1228.
48. Kahnt T., Heinzle J., Park S.Q., Haynes J.D. The neural code of reward anticipation in human orbitofrontal cortex. Proc. Natl. Acad. Sci. USA 2010, 107:6010-6015.
49. Kamitani Y., Sawahata Y. Spatial smoothing hurts localization but not information: pitfalls for brain mappers. Neuroimage 2010, 49:1949-1952.
50. Kamitani Y., Tong F. Decoding the visual and subjective contents of the human brain. Nat. Neurosci. 2005, 8:679-685.
51. Kay K.N., Naselaris T., Prenger R.J., Gallant J.L. Identifying natural images from human brain activity. Nature 2008, 452:352-355.
52. Kohonen T. Self-Organization and Asssociative Memory 1989, Springer, Berlin.
53. Kriegeskorte N. Relating population-code representations between man, monkey, and computational models. Front. Neurosci. 2009, 3:363-373.
54. Kriegeskorte N. Pattern-information analysis: from stimulus decoding to computational-model testing. Neuroimage 2011, 56:411-421.
55. Kriegeskorte N., Kievit R.A. Representational geometry: integrating cognition, computation, and the brain. Trends Cogn. Sci. 2013, 17:401-412.
56. Kriegeskorte N., Goebel R., Bandettini P. Information-based functional brain mapping. Proc. Natl. Acad. Sci. USA 2006, 103:3863-3868.
57. Kriegeskorte N., Mur M., Bandettini P. Representational similarity analysis - connecting the branches of systems neuroscience. Front. Syst. Neurosci. 2008, 2:4.
58. Kriegeskorte N., Mur M., Ruff D.A., Kiani R., Bodurka J., Esteky H., Tanaka K., Bandettini P.A. Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron 2008, 60:1126-1141.
59. Kriegeskorte N., Simmons W.K., Bellgowan P.S., Baker C.I. Circular analysis in systems neuroscience: the dangers of double dipping. Nat. Neurosci. 2009, 12:535-540.
60. Kriegeskorte N., Cusack R., Bandettini P. How does an fMRI voxel sample the neuronal activity pattern: compact-kernel or complex spatiotemporal filter?. Neuroimage 2010, 49:1965-1976.
61. Lee S.H., Kravitz D.J., Baker C.I. Goal-dependent dissociation of visual and prefrontal cortices during working memory. Nat. Neurosci. 2013, 16:997-999.
62. Liu T.T., Frank L.R., Wong E.C., Buxton R.B. Detection power, estimation efficiency, and predictability in event-related fMRI. Neuroimage 2001, 13:759-773.
63. Logothetis N.K. What we can do and what we cannot do with fMRI. Nature 2008, 453:869-878.
64. Logothetis N.K., Wandell B.A. Interpreting the BOLD signal. Annu. Rev. Physiol. 2004, 66:735-769.
65. Marquand A.F., Brammer M., Williams S.C., Doyle O.M. Bayesian multi-task learning for decoding multi-subject neuroimaging data. Neuroimage 2014, 92:298-311.
66. Mitchell T.M., Hutchinson R., Just M.A., Niculescu R.S., Pereira F., Wang X. Classifying instantaneous cognitive states from FMRI data. AMIA Annu. Symp. Proc. 2003, 2003:465-469.
67. Mitchell T.M., Shinkareva S.V., Carlson A., Chang K.M., Malave V.L., Mason R.A., Just M.A. Predicting human brain activity associated with the meanings of nouns. Science 2008, 320:1191-1195.
68. Miyawaki Y., Uchida H., Yamashita O., Sato M.A., Morito Y., Tanabe H.C., Sadato N., Kamitani Y. Visual image reconstruction from human brain activity using a combination of multiscale local image decoders. Neuron 2008, 60:915-929.
69. Mountcastle V.B. Modality and topographic properties of single neurons of cat's somatic sensory cortex. J.Neurophysiol. 1957, 20:408-434.
70. Mourão-Miranda J., Bokde A.L., Born C., Hampel H., Stetter M. Classifying brain states and determining the discriminating activation patterns: Support Vector Machine on functional MRI data. Neuroimage 2005, 28:980-995.
71. Mourão-Miranda J., Friston K.J., Brammer M. Dynamic discrimination analysis: a spatial-temporal SVM. Neuroimage 2007, 36:88-99.
72. Mumford J.A., Davis T., Poldrack R.A. The impact of study design on pattern estimation for single-trial multivariate pattern analysis. Neuroimage 2014, 103:130-138.
73. Mur M., Bandettini P.A., Kriegeskorte N. Revealing representational content with pattern-information fMRI-an introductory guide. Soc. Cogn. Affect. Neurosci. 2009, 4:101-109.
74. Naselaris T., Prenger R.J., Kay K.N., Oliver M., Gallant J.L. Bayesian reconstruction of natural images from human brain activity. Neuron 2009, 63:902-915.
75. Naselaris T., Kay K.N., Nishimoto S., Gallant J.L. Encoding and decoding in fMRI. Neuroimage 2011, 56:400-410.
76. Nevado A., Young M.P., Panzeri S. Functional imaging and neural information coding. Neuroimage 2004, 21:1083-1095.
77. Nichols T.E., Holmes A.P. Nonparametric permutation tests for functional neuroimaging: a primer with examples. Hum. Brain Mapp. 2002, 15:1-25.
78. Nili H., Wingfield C., Walther A., Su L., Marslen-Wilson W., Kriegeskorte N. A toolbox for representational similarity analysis. PLoS Comput. Biol. 2014, 10:e1003553.
79. Nishimoto S., Vu A.T., Naselaris T., Benjamini Y., Yu B., Gallant J.L. Reconstructing visual experiences from brain activity evoked by natural movies. Curr. Biol. 2011, 21:1641-1646.
80. Noirhomme Q., Lesenfants D., Gomez F., Soddu A., Schrouff J., Garraux G., Luxen A., Phillips C., Laureys S. Biased binomial assessment of cross-validated estimation of classification accuracies illustrated in diagnosis predictions. Neuroimage Clin. 2014, 4:687-694.
81. Norman K.A., Polyn S.M., Detre G.J., Haxby J.V. Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn. Sci. 2006, 10:424-430.
82. O'Craven K.M., Downing P.E., Kanwisher N. fMRI evidence for objects as the units of attentional selection. Nature 1999, 401:584-587.
83. Ogawa S., Lee T.M., Kay A.R., Tank D.W. Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc. Natl. Acad. Sci. USA 1990, 87:9868-9872.
84. Oosterhof N.N., Wiggett A.J., Diedrichsen J., Tipper S.P., Downing P.E. Surface-based information mapping reveals crossmodal vision-action representations in human parietal and occipitotemporal cortex. J.Neurophysiol. 2010, 104:1077-1089.
85. Op de Beeck H.P. Against hyperacuity in brain reading: spatial smoothing does not hurt multivariate fMRI analyses?. Neuroimage 2010, 49:1943-1948.
86. Pereira F., Botvinick M. Information mapping with pattern classifiers: a comparative study. Neuroimage 2011, 56:476-496.
87. Pereira F., Mitchell T., Botvinick M. Machine learning classifiers and fMRI: a tutorial overview. Neuroimage 2009, 45(1, Suppl):S199-S209.
88. Poldrack R.A. Can cognitive processes be inferred from neuroimaging data?. Trends Cogn. Sci. 2006, 10:59-63.
89. Polyn S.M., Natu V.S., Cohen J.D., Norman K.A. Category-specific cortical activity precedes retrieval during memory search. Science 2005, 310:1963-1966.
90. Pouget A., Dayan P., Zemel R. Information processing with population codes. Nat. Rev. Neurosci. 2000, 1:125-132.
91. Raichle M.E. Two views of brain function. Trends Cogn. Sci. 2010, 14:180-190.
92. Ramírez F.M., Cichy R.M., Allefeld C., Haynes J.D. The neural code for face orientation in the human fusiform face area. J.Neurosci. 2014, 34:12155-12167.
93. Reddy L., Kanwisher N.G., VanRullen R. Attention and biased competition in multi-voxel object representations. Proc. Natl. Acad. Sci. USA 2009, 106:21447-21452.
94. Reverberi C., Görgen K., Haynes J.D. Compositionality of rule representations in human prefrontal cortex. Cereb. Cortex 2012, 22:1237-1246.
95. Riesenhuber M., Poggio T. Hierarchical models of object recognition in cortex. Nat. Neurosci. 1999, 2:1019-1025.
96. Sasaki Y., Rajimehr R., Kim B.W., Ekstrom L.B., Vanduffel W., Tootell R.B. The radial bias: a different slant on visual orientation sensitivity in human and nonhuman primates. Neuron 2006, 51:661-670.
97. Schoenmakers S., Barth M., Heskes T., van Gerven M. Linear reconstruction of perceived images from human brain activity. Neuroimage 2013, 83:951-961.
98. Schönwiesner M., Zatorre R.J. Spectro-temporal modulation transfer function of single voxels in the human auditory cortex measured with high-resolution fMRI. Proc. Natl. Acad. Sci. USA 2009, 106:14611-14616.
99. Schreiber K., Krekelberg B. The statistical analysis of multi-voxel patterns in functional imaging. PLoS ONE 2013, 8:e69328.
100. Scott D.W. Multivariate density estimation: Theory, practice, visualization 1992, Wiley, New York.
101. Serences J.T., Saproo S. Computational advances towards linking BOLD and behavior. Neuropsychologia 2012, 50:435-446.
102. Sereno M.I., Dale A.M., Reppas J.B., Kwong K.K., Belliveau J.W., Brady T.J., Rosen B.R., Tootell R.B. Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science 1995, 268:889-893.
103. Shmuel A., Chaimow D., Raddatz G., Ugurbil K., Yacoub E. Mechanisms underlying decoding at 7 T: ocular dominance columns, broad structures, and macroscopic blood vessels in V1 convey information on the stimulated eye. Neuroimage 2010, 49:1957-1964.
104. Smola A.J., Schölkopf B. A tutorial on support vector regression. Stat. Comput. 2004, 14:199-222.
105. Soon C.S., Brass M., Heinze H.J., Haynes J.D. Unconscious determinants of free decisions in the human brain. Nat. Neurosci. 2008, 11:543-545.
106. Sprague T.C., Serences J.T. Attention modulates spatial priority maps in the human occipital, parietal and frontal cortices. Nat. Neurosci. 2013, 16:1879-1887.
107. Sprague T.C., Ester E.F., Serences J.T. Reconstructions of information in visual spatial working memory degrade with memory load. Curr. Biol. 2014, 24:2174-2180.
108. Stelzer J., Chen Y., Turner R. Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): random permutations and cluster size control. Neuroimage 2013, 65:69-82.
109. Swisher J.D., Gatenby J.C., Gore J.C., Wolfe B.A., Moon C.H., Kim S.G., Tong F. Multiscale pattern analysis of orientation-selective activity in the primary visual cortex. J.Neurosci. 2010, 30:325-330.
110. Thirion B., Duchesnay E., Hubbard E., Dubois J., Poline J.B., Lebihan D., Dehaene S. Inverse retinotopy: inferring the visual content of images from brain activation patterns. Neuroimage 2006, 33:1104-1116.
111. Thomas J.M., Huber E., Stecker G.C., Boynton G.M., Saenz M., Fine I. Population receptive field estimates of human auditory cortex. Neuroimage 2015, 105:428-439.
112. Todd M.T., Nystrom L.E., Cohen J.D. Confounds in multivariate pattern analysis: Theory and rule representation case study. Neuroimage 2013, 77:157-165.
113. Tong F., Pratte M.S. Decoding patterns of human brain activity. Annu. Rev. Psychol. 2012, 63:483-509.
114. Tononi G., Srinivasan R., Russell D.P., Edelman G.M. Investigating neural correlates of conscious perception by frequency-tagged neuromagnetic responses. Proc. Natl. Acad. Sci. USA 1998, 95:3198-3203.
115. Tzourio-Mazoyer N., Landeau B., Papathanassiou D., Crivello F., Etard O., Delcroix N., Mazoyer B., Joliot M. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 2002, 15:273-289.
116. Viswanathan, S., Cieslak, M., and Grafton, S.T. (2012). On the geometric structure of fMRI searchlight-based information maps. arXiv:1210.6317.
117. Wang H.X., Merriam E.P., Freeman J., Heeger D.J. Motion direction biases and decoding in human visual cortex. J.Neurosci. 2014, 34:12601-12615.
118. Wisniewski D., Reverberi C., Tusche A., Haynes J.-D. The neural representation of voluntary task-set selection in dynamic environments. Cereb. Cortex 2014, 10.1093/cercor/bhu155.
119. Woolgar A., Golland P., Bode S. Coping with confounds in multivoxel pattern analysis: what should we do about reaction time differences? A comment on Todd, Nystrom & Cohen 2013. Neuroimage 2014, 98:506-512.
120. Yacoub E., Harel N., Ugurbil K. High-field fMRI unveils orientation columns in humans. Proc. Natl. Acad. Sci. USA 2008, 105:10607-10612.
121. Yamashita O., Sato M.A., Yoshioka T., Tong F., Kamitani Y. Sparse estimation automatically selects voxels relevant for the decoding of fMRI activity patterns. Neuroimage 2008, 42:1414-1429.
122. Yarkoni T., Poldrack R.A., Nichols T.E., Van Essen D.C., Wager T.D. Large-scale automated synthesis of human functional neuroimaging data. Nat. Methods 2011, 8:665-670.
123. Zimmermann J., Goebel R., De Martino F., van de Moortele P.F., Feinberg D., Adriany G., Chaimow D., Shmuel A., Uğurbil K., Yacoub E. Mapping the organization of axis of motion selective features in human area MT using high-field fMRI. PLoS ONE 2011, 6:e28716.
124. Zuiderbaan W., Harvey B.M., Dumoulin S.O. Modeling center-surround configurations in population receptive fields using fMRI. J.Vis. 2012, 12:10.