MEG-based decoding of the spatiotemporal dynamics of visual category perception

NeuroImage

Volumn 83, Issue , 2013, Pages 1063-1073

  Van de Nieuwenhuijzen, M E ^a   Backus, A R ^a   Bahramisharif, A ^a   Doeller, C F ^a   Jensen, O ^a   van Gerven, M A J ^a  

^a RADBOUD UNIVERSITY   (Netherlands)

Author keywords

MEG; Multivariate analysis; Perception; Spatiotemporal dynamics

Indexed keywords

ACCURACY; ADULT; ARTICLE; BRAIN REGION; CLASSIFIER; COGNITION; CONTROLLED STUDY; FEMALE; HUMAN; HUMAN EXPERIMENT; IMAGE ANALYSIS; INFERIOR TEMPORAL GYRUS; LINGUAL GYRUS; MAGNETOENCEPHALOGRAPHY; MALE; MULTIVARIATE ANALYSIS; NEUROLOGICAL DIAGNOSTIC DEVICE; NORMAL HUMAN; OCCIPITAL CORTEX; OCCIPITAL GYRUS; PARIETAL GYRUS; PATTERN RECOGNITION; PRIORITY JOURNAL; SPATIOTEMPORAL ANALYSIS; TASK PERFORMANCE; TEMPORAL CORTEX; VISUAL DISCRIMINATION; VISUAL STIMULATION;

MEG; MULTIVARIATE ANALYSIS; PERCEPTION; SPATIOTEMPORAL DYNAMICS;

ADULT; ALGORITHMS; BRAIN; BRAIN MAPPING; FEMALE; HUMANS; MAGNETOENCEPHALOGRAPHY; MALE; PHOTIC STIMULATION; SIGNAL PROCESSING, COMPUTER-ASSISTED; VISUAL PERCEPTION;

EID: 84886445574     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2013.07.075     Document Type: Article

References (85)

1. Ales J.M., Appelbaum L.G., Cottereau B.R., Norcia A.M. The time course of shape discrimination in the human brain. NeuroImage 2012, 67:77-88.
2. Allison T., Puce A., Spencer D.D., McCarthy G. Electrophysiological studies of human face perception. I: potentials generated in occipitotemporal cortex by face and non-face stimuli. Cereb. Cortex 1999, 9(5):415-430.
3. Almeida J., Mahon B.Z., Nakayama K., Caramazza A. Unconscious processing dissociates along categorical lines. Proc. Natl. Acad. Sci. U. S. A. 2008, 105(39):15214-152148.
4. Almeida J., Mahon B.Z., Caramazza A. The role of the dorsal visual processing stream in tool identification. Psychol. Sci. 2010, 21(6):772-778.
5. Amano K., Goda N., Nishida S., Ejima Y., Takeda T., Ohtani Y. Estimation of the timing of human visual perception from magnetoencephalography. J. Neurosci. 2006, 26(15):3981-3991.
6. Bacon-Macé N., Macé M.J.-M., Fabre-Thorpe M., Thorpe S.J. The time course of visual processing: backward masking and natural scene categorisation. Vis. Res. 2005, 45(11):1459-1469.
7. Bahramisharif A., Van Gerven M., Schoffelen J.-M., Ghahramani Z., Heskes T. The dynamic beamformer. Machine learning and interpretation in neuroimaging. Lect. Notes Comput. Sci. 2012, 7263:148-155.
8. Bentin S., Allison T., Puce A., Perez E., McCarthy G. Electrophysiological studies of face perception in humans. J. Cogn. Neurosci. 1996, 8(6):551-565.
9. Bießmann F., Dähne S., Meinecke F. On the interpretability of linear multivariate neuroimaging analyses: filters, patterns and their relationship. NIPS Workshop on Machine Learning and Inference in Neuroimaging 2012, 1-6.
10. Bode S., Sewell D.K., Lilburn S., Forte J.D., Smith P.L., Stahl J. Predicting perceptual decision biases from early brain activity. J. Neurosci. 2012, 32(36):12488-12498.
11. Brodeur M.B., Dionne-Dostie E., Montreuil T., Lepage M. The Bank of Standardized Stimuli (BOSS), a new set of 480 normative photos of objects to be used as visual stimuli in cognitive research. PLoS One 2010, 5(5):e10773.
12. Carlson T., Schrater P., He S. Patterns of activity in the categorical representations of objects. J. Cogn. Neurosci. 2003, 15(5):704-717.
13. Carlson T., Hogendoorn H., Kanai R., Mesik J., Turret J. High temporal resolution decoding of object position and category. J. Vis. 2011, 11(10):1-17.
14. Carroll M.K., Cecchi G.A., Rish I., Garg R., Rao A.R. Prediction and interpretation of distributed neural activity with sparse models. NeuroImage 2009, 44:112-122.
15. Chan A.M., Halgren E., Marinkovic K., Cash S.S. Decoding word and category-specific spatiotemporal representations from MEG and EEG. NeuroImage 2011, 54(4):3028-3039.
16. Connolly A.C., Guntupalli J.S., Gors J., Hanke M., Halchenko Y.O., Wu Y.-C., Abdi H., Haxby J.V. The representation of biological classes in the human brain. J. Neurosci. 2012, 32(8):2608-2618.
17. Cox D.D., Savoy R.L. Functional magnetic resonance imaging (fMRI) "brain reading": detecting and classifying distributed patterns of fMRI activity in human visual cortex. NeuroImage 2003, 19(2):261-270.
18. Crouzet S.M., Thorpe S.J. Low-level cues and ultra-fast face detection. Front. Psychol. 2011, 2(342):1-9.
19. Crouzet S.M., Kirchner H., Thorpe S. Fast saccades towards faces: face detection in just 100ms. J. Vis. 2010, 10(4):1-17.
20. De Gelder B., Van den Stock J. The bodily expressive action stimulus test (BEAST). Construction and validation of a stimulus basis for measuring perception of whole body expression of emotions. Front. Psychol. 2011, 2(181):1-6.
21. Downing P.E., Chan A.W.-Y., Peelen M.V., Dodds C.M., Kanwisher N. Domain specificity in visual cortex. Cereb. Cortex 2006, 16(10):1453-1461.
22. Epstein R., Kanwisher N. A cortical representation of the local visual environment. Nature 1998, 392:598-601.
23. Farah M.J., Wilson K.D., Drain M., Tanaka J.N. What is "special" about face perception?. Psychol. Rev. 1998, 105(3):482-498.
24. Fei-Fei L., Perona P. A Bayesian hierarchical model for learning natural scene categories. IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2005, 2:524-531.
25. Fellinger R., Gruber W., Zauner A., Freunberger R., Klimesch W. Evoked traveling alpha waves predict visual-semantic categorization-speed. NeuroImage 2012, 59(4):3379-3388.
26. Friedman J., Hastie T., Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 2010, 33(1):1-22.
27. Fuentemilla L., Penny W.D., Cashdollar N., Bunzeck N., Düzel E. Theta-coupled periodic replay in working memory. Curr. Biol. 2010, 20(7):606-612.
28. Gauthier I., Tarr M.J., Moylan J., Skudlarski P., Gore J.C., Anderson A.W. The fusiform "face area" is part of a network that processes faces at the individual level. J. Cogn. Neurosci. 2000, 12(3):495-504.
29. Goeleven E., De Raedt R., Leyman L., Verschuere B. The Karolinska directed emotional faces: a validation study. Cogn. Emot. 2008, 22(6):1094-1118.
30. Guimaraes M.P., Wong D.K., Uy E.T., Grosenick L., Suppes P. Single-trial classification of MEG recordings. IEEE Trans. Biomed. Eng. 2007, 54(3):436-443.
31. Haxby J.V., Gobbini M.I., Furey M.L., Ishai A., Schouten J.L., Pietrini P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 2001, 293(5539):2425-2430.
32. Haynes J.-D., Rees G. Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nat. Neurosci. 2005, 8(5):686-691.
33. Itier R.J., Taylor M.J. Inversion and contrast polarity reversal affect both encoding and recognition processes of unfamiliar faces: a repetition study using ERPs. NeuroImage 2002, 15(2):353-372.
34. Itier R.J., Taylor M.J. N170 or N1? Spatiotemporal differences between object and face processing using ERPs. Cereb. Cortex 2004, 14(2):132-142.
35. Jafarpour A., Horner A.J., Fuentemilla L., Penny W.D., Düzel E. Decoding oscillatory representations and mechanisms in memory. Neuropsychologia 2013, 51(4):772-780.
36. Kamitani Y., Tong F. Decoding the visual and subjective contents of the human brain. Nat. Neurosci. 2005, 8(5):679-685.
37. Kay K.N., Naselaris T., Prenger R.J., Gallant J.L. Identifying natural images from human brain activity. Nature 2008, 452(7185):352-355.
38. Kirchner H., Thorpe S.J. Ultra-rapid object detection with saccadic eye movements: visual processing speed revisited. Vis. Res. 2006, 46(11):1762-1776.
39. Kirchner H., Barbeau E.J., Thorpe S.J., Régis J., Liégeois-Chauvel C. Ultra-rapid sensory responses in the human frontal eye field region. J. Neurosci. 2009, 29(23):7599-7606.
40. Kriegeskorte N. Pattern-information analysis: from stimulus decoding to computational-model testing. NeuroImage 2011, 56:411-421.
41. Kriegeskorte N., Goebel R., Bandettini P. Information-based functional brain mapping. Proc. Natl. Acad. Sci. U. S. A. 2006, 103(10).
42. Lange N., Strother S.C., Anderson J.R., Nielsen F.A., Holmes A.P., Kolenda T., Savoy R., Hansen L.K. Plurality and resemblance in fMRI data analysis. NeuroImage 1999, 10:282-303.
43. Liu H., Agam Y., Madsen J.R., Kreiman G. Timing, timing, timing: fast decoding of object information from intracranial field potentials in human visual cortex. Neuron 2009, 62(2):281-290.
44. Lundqvist D., Flykt A., Öhman A. The Karolinska Directed Emotional Faces - KDEF 1998, Department of Neurosciences Karolinska Hospital, Stockholm.
45. McCarthy G., Puce A., Belger A., Allison T. Electrophysiological studies of human face perception. II: response properties of face-specific potentials generated in occipitotemporal cortex. Cereb. Cortex 1999, 9(5):431-444.
46. Mishkin M., Ungerleider L., Macko K. Object vision and spatial vision: two cortical pathways. Trends Neurosci. 1983, 6:414-417.
47. Mitchell T.M., Shinkareva S.V., Carlson A., Chang K.-M., Malave V.L., Mason R.A., Just M.A. Predicting human brain activity associated with the meanings of nouns. Science 2008, 320(5880):1191-1195.
48. Miyawaki Y., Uchida H., Yamashita O., Sato M., Morito Y., Tanabe H.C., Sadato N., et al. Visual image reconstruction from human brain activity using a combination of multiscale local image decoders. Neuron 2008, 60(5):915-929.
49. Murphy B., Baroni M., Poesio M. EEG responds to conceptual stimuli and corpus semantics. Proc. ACL/EMNLP 2009, 619-627.
50. Murphy B., Poesio M., Bovolo F., Bruzzone L., Dalponte M., Lakany H. EEG decoding of semantic category reveals distributed representations for single concepts. Brain Lang. 2011, 117(1):12-22.
51. Naselaris T., Prenger R.J., Kay K.N., Oliver M., Gallant J.L. Bayesian reconstruction of natural images from human brain activity. Neuron 2009, 63(6):902-915.
52. Nichols D.F., Betts L.R., Wilson H.R. Decoding of faces and face components in face-sensitive human visual cortex. Front. Psychol. 2010, 1(28):1-13.
53. Noguchi Y., Tanabe H.C., Sadato N., Hoshiyama M., Kakigi R. Voluntary attention changes the speed of perceptual neural processing. Eur. J. Neurosci. 2007, 25:3163-3172.
54. Nolte G. The magnetic lead field theorem in the quasi-static approximation and its use for magnetoencephalography forward calculation in realistic volume conductors. Phys. Med. Biol. 2003, 48(22):3637-3652.
55. Oostenveld R., Fries P., Maris E., Schoffelen J.-M. FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput. Intell. Neurosci. 2011, 2011:1-9.
56. Petrov Y., Nador J., Qian J. VEP correlates of feedback in human cortex. PLoS One 2012, 7(12):e51791.
57. Ramkumar P., Jas M., Pannasch S., Hari R., Parkkonen L. Feature-specific information processing precedes concerted activation in human visual cortex. J. Neurosci. 2013, 33(18):7691-7699.
58. Rees G., Russell C., Frith C., Driver J. Inattentional blindness versus inattentional amnesia for fixated but ignored words. Science 1999, 286(5449):2504-2507.
59. Reynolds J.H., Pasternak T., Desimone R. Attention increases sensitivity of V4 neurons. Neuron 2000, 26:703-714.
60. Riesenhuber M., Poggio T. Hierarchical models of object recognition in cortex. Nat. Neurosci. 1999, 2(11):1019-1025.
61. Rossion B., Joyce C.A., Cottrell G.W., Tarr M.J. Early lateralization and orientation tuning for face, word, and object processing in the visual cortex. NeuroImage 2003, 20(3):1609-1624.
62. Rousselet G.A., Husk J.S., Bennett P.J., Sekuler A.B. Single-trial EEG dynamics of object and face visual processing. NeuroImage 2007, 36(3):843-862.
63. Sakuraba S., Sakai S., Yamanaka M., Yokosawa K., Hirayama K. Does the human dorsal stream really process a category for tools?. J. Neurosci. 2012, 32(11):3949-3953.
64. Salzberg S. On comparing classifiers: pitfalls to avoid and a recommended approach. Data Min. Knowl. Discov. 1997, 1:317-328.
65. Sandberg K., Bahrami B., Kanai R., Barnes G.R., Overgaard M., Rees G. Early visual responses predict conscious face perception within and between subjects during binocular rivalry. J. Cogn. Neurosci. 2013, 25(6):969-985.
66. Seeck M., Michel C.M., Mainwaring N., Cosgrove R., Blume H., Ives J., Landis T., Schomer D.L. Evidence for rapid face recognition from human scalp and intracranial electrodes. Neuroreport 1997, 8(12):2749-2754.
67. Serre T., Oliva A., Poggio T. A feedforward architecture accounts for rapid categorization. Proc. Natl. Acad. Sci. U. S. A. 2007, 104(15):6424-6429.
68. Simanova I., Van Gerven M., Oostenveld R., Hagoort P. Identifying object categories from event-related EEG: toward decoding of conceptual representations. PLoS One 2010, 5(12):e14465.
69. Stolk A., Todorovic A., Schoffelen J.-M., Oostenveld R. Online and offline tools for head movement compensation in MEG. NeuroImage 2013, 68:39-48.
70. Sudre G., Pomerleau D., Palatucci M., Wehbe L., Fyshe A., Salmelin R., Mitchell T. Tracking neural coding of perceptual and semantic features of concrete nouns. NeuroImage 2012, 62:451-463.
71. Suppes P., Lu Z.L., Han B. Brain wave recognition of words. Proc. Natl. Acad. Sci. U. S. A. 1997, 94(26):14965-14969.
72. Suppes P., Han B., Epelboim J., Lu Z.L. Invariance of brain-wave representations of simple visual images and their names. Proc. Natl. Acad. Sci. U. S. A. 1999, 96(25):14658-14663.
73. Taylor M.J. Non-spatial attentional effects on P1. Clin. Neurophysiol. 2002, 113(12):1903-1908.
74. Thirion B., Duchesnay E., Hubbard E., Dubois J., Poline J.-B., Lebihan D., Dehaene S. Inverse retinotopy: inferring the visual content of images from brain activation patterns. NeuroImage 2006, 33(4):1104-1116.
75. Thorpe S.J., Fabre-Thorpe M. Seeking categories in the brain. Science 2001, 291:260-262.
76. Thorpe S.J., Fize D., Marlot C. Speed of processing in the human visual system. Nature 1996, 381:520-522.
77. Van Gerven M.A.J., Cseke B., De Lange F.P., Heskes T. Efficient Bayesian multivariate fMRI analysis using a sparsifying spatio-temporal prior. NeuroImage 2010, 50(1):150-161.
78. Van Gerven M.A.J., De Lange F.P., Heskes T. Neural decoding with hierarchical generative models. Neural Comput. 2010, 22(12):3127-3142.
79. Van Gerven M.A.J., Maris E., Sperling M., Sharan A., Litt B., Anderson C., Baltuch G., et al. Decoding the memorization of individual stimuli with direct human brain recordings. NeuroImage 2013, 70:223-232.
80. Van Veen B.D., Van Drongelen W., Yuchtman M., Suzuki A. Localization of brain electrical activity via linearly constrained minimum variance spatial filtering. IEEE Trans. Biomed. Eng. 1997, 44(9):867-880.
81. VanRullen R., Thorpe S.J. The time course of visual processing: from early perception to decision-making. J. Cogn. Neurosci. 2001, 13(4):454-461.
82. Willenbockel V., Sadr J., Fiset D., Horne G.O., Gosselin F., Tanaka J.W. Controlling low-level image properties: the SHINE toolbox. Behav. Res. Methods 2010, 42(3):671-684.
83. Wokke M.E., Sligte I.G., Steven Scholte H., Lamme V.A.F. Two critical periods in early visual cortex during figure-ground segregation. Brain Behav. 2012, 2(6):763-777.
84. Wu W., Gao S. Learning event-related potentials (ERPs) from multichannel EEG recordings: a spatio-temporal modeling framework with a fast estimation algorithm. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2011, 2011:6959-6962.
85. Yamashita O., Sato M., Yoshioka T., Tong F., Kamitani Y. Sparse estimation automatically selects voxels relevant for the decoding of fMRI activity patterns. NeuroImage 2008, 42:1414-1429.