Decoding complex flow-field patterns in visual working memory

NeuroImage

Volumn 91, Issue , 2014, Pages 43-51

  Christophel, Thomas B ^a   Haynes, John Dylan ^a,b  

^a CHARITÉ UNIVERSITÄTSMEDIZIN BERLIN   (Germany)

^b HUMBOLDT UNIVERSITY   (Germany)

Author keywords

Human MT+; Motion flow fields; Multivariate decoding; Posterior parietal cortex; Primary somatosensory cortex; Visual working memory

Indexed keywords

ACCURACY; ARTICLE; BOLD SIGNAL; CLASSIFICATION; COMPLEX FLOW FIELD PATTERN STIMULATION; CONTROLLED STUDY; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUNCTIONAL NEUROIMAGING; HEMISPHERIC DOMINANCE; HUMAN; HUMAN EXPERIMENT; MEMORY CONSOLIDATION; NEUROANATOMY; NORMAL HUMAN; PARIETAL CORTEX; PRIMARY SOMATOSENSORY CORTEX; PRIORITY JOURNAL; SUPPORT VECTOR MACHINE; TASK PERFORMANCE; VISUAL AREA MT+; VISUAL MEMORY; VISUAL NERVOUS SYSTEM; VISUAL STIMULATION; VISUAL WORKING MEMORY; WORKING MEMORY;

HUMAN MT+; MOTION FLOW-FIELDS; MULTIVARIATE DECODING; POSTERIOR PARIETAL CORTEX; PRIMARY SOMATOSENSORY CORTEX; VISUAL WORKING MEMORY;

ADULT; BRAIN MAPPING; CEREBROVASCULAR CIRCULATION; CUES; FEMALE; HEMODYNAMICS; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; MEMORY, SHORT-TERM; MOTION PERCEPTION; PSYCHOMOTOR PERFORMANCE; REPRODUCIBILITY OF RESULTS; VISUAL CORTEX; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84896703745     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2014.01.025     Document Type: Article

References (88)

1. Andersen R.A., Snyder L.H., Bradley D.C., Xing J. Multimodal representation of space in the posterior parietal cortex and its use in planning movements. Annu. Rev. Neurosci. 1997, 20(1):303-330.
2. Ashburner J., Friston K.J. Unified segmentation. Neuroimage 2005, 26(3):839-851.
3. Baddeley A.D., Hitch G.J. Working memory. The Psychology of Learning and Motivation 1974, vol. 8:47-89. Academic Press, New York.
4. Bisley J.W., Goldberg M.E. Attention, intention, and priority in the parietal lobe. Annu. Rev. Neurosci. 2010, 33:1-21.
5. Blake R., Sobel K.V., James T.W. Neural synergy between kinetic vision and touch. Psychol. Sci. 2004, 15(6):397-402.
6. Blakemore S.-J., Bristow D., Bird G., Frith C., Ward J. Somatosensory activations during the observation of touch and a case of vision-touch synaesthesia. Brain 2005, 128(7):1571-1583.
7. Bogler C., Bode S., Haynes J.-D. Decoding successive computational stages of saliency processing. Curr. Biol. 2011, 21:1667-1671.
8. Brouwer G.J., van Ee R. Visual cortex allows prediction of perceptual states during ambiguous structure-from-motion. J. Neurosci. 2007, 27(5):1015-1023.
9. Cappe C., Barone P. Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. Eur. J. Neurosci. 2005, 22(11):2886-2902.
10. Christophel T.B., Hebart M.N., Haynes J.-D. Decoding the contents of visual short-term memory from human visual and parietal cortex. J. Neurosci. 2012, 32(38):12983-12989.
11. Constantinidis C., Procyk E. The primate working memory networks. Cogn. Affect. Behav. Neurosci. 2004, 4(4):444-465.
12. Corbetta M., Kincade J.M., Shulman G.L. Neural systems for visual orienting and their relationships to spatial working memory. J. Cogn. Neurosci. 2002, 14(3):508-523.
13. Courtney S.M. Attention and cognitive control as emergent properties of information representation in working memory. Cogn. Affect. Behav. Neurosci. 2004, 4(4):501-516.
14. Courtney S.M., Ungerleider L.G., Keil K., Haxby J.V. Transient and sustained activity in a distributed neural system for human working memory. Nature 1997, 386(6625):608-611.
15. Cox D.D., Savoy R.L. Functional magnetic resonance imaging (fMRI) "brain reading": detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 2003, 19(2):261-270.
16. Culham J., He S., Dukelow S., Verstraten F.A. Visual motion and the human brain: what has neuroimaging told us?. Acta Psychol. 2001, 107(1-3):69-94.
17. Curtis C.E., D'Esposito M. Persistent activity in the prefrontal cortex during working memory. Trends Cogn. Sci. 2003, 7(9):415-423.
18. Day S. Some demographic and socio-cultural aspects of synesthesia. Synesthesia: Perspectives From Cognitive Neuroscience 2005, 11-33. Oxford University Press. L.C. Robertson, N. Sagiv (Eds.).
19. Dumoulin S.O., Bittar R.G., Kabani N.J., Baker C.L., Goualher G.L., Pike G.B., Evans A.C. A new anatomical landmark for reliable identification of human area V5/MT: a quantitative analysis of sulcal patterning. Cereb. Cortex 2000, 10(5):454-463.
20. Edin F., Klingberg T., Johansson P., McNab F., Tegnér J., Compte A. Mechanism for top-down control of working memory capacity. Proc. Natl. Acad. Sci. 2009, 106(16):6802-6807.
21. Eickhoff S.B., Stephan K.E., Mohlberg H., Grefkes C., Fink G.R., Amunts K., Zilles K. A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. Neuroimage 2005, 25(4):1325-1335.
22. Emrich S.M., Riggall A.C., LaRocque J.J., Postle B.R. Distributed patterns of activity in sensory cortex reflect the precision of multiple items maintained in visual short-term memory. J. Neurosci. 2013, 33(15):6516-6523.
23. Field D.T., Wann J.P. Perceiving time to collision activates the sensorimotor cortex. Curr. Biol. 2005, 15(5):453-458.
24. Funahashi S. Prefrontal cortex and working memory processes. Neuroscience 2006, 139(1):251-261.
25. Geyer S., Schleicher A., Zilles K. Areas 3a, 3b, and 1 of human primary somatosensory cortex: 1. Microstructural organization and interindividual variability. Neuroimage 1999, 10(1):63-83.
26. Geyer S., Schormann T., Mohlberg H., Zilles K. Areas 3a, 3b, and 1 of human primary somatosensory cortex: 2. Spatial normalization to standard anatomical space. Neuroimage 2000, 11(6):684-696.
27. Goldman-Rakic P. Cellular basis of working memory. Neuron 1995, 14(3):477-485.
28. Good P.I. Permutation, Parametric, and Bootstrap Tests of Hypotheses 2004, Springer. 3rd ed.
29. Hagler D.J., Sereno M.I. Spatial maps in frontal and prefrontal cortex. NeuroImage 2006, 29(2):567-577.
30. Hagler D.J., Riecke L., Sereno M.I. Parietal and superior frontal visuospatial maps activated by pointing and saccades. Neuroimage 2007, 35(4):1562-1577.
31. Harrison S.A., Tong F. Decoding reveals the contents of visual working memory in early visual areas. Nature 2009, 458(7238):632-635.
32. Haxby J.V., Gobbini M.I., Furey M.L., Ishai A., Schouten J.L., Pietrini P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 2001, 293(5539):2425-2430.
33. Haynes J.D., Rees G. Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nat. Neurosci. 2005, 8(5):686-691.
34. Haynes J.-D., Rees G. Decoding mental states from brain activity in humans. Nat. Rev. Neurosci. 2006, 7(7):523-534.
35. Haynes J.D., Sakai K., Rees G., Gilbert S., Frith C., Passingham R.E. Reading hidden intentions in the human brain. Curr. Biol. 2007, 17(4):323-328.
36. Hebart M.N., Donner T.H., Haynes J.-D. Human visual and parietal cortex encode visual choices independent of motor plans. NeuroImage 2012, 63(3):1393-1403.
37. Hong S.W., Tong F., Seiffert A.E. Direction-selective patterns of activity in human visual cortex suggest common neural substrates for different types of motion. Neuropsychologia 2012, 50(4):514-521.
38. Hsu C.W., Lin C.J. A comparison of methods for multiclass support vector machines. IEEE Trans. Neural Networks 2002, 13(2):415-425.
39. Hubbard E., Ramachandran V.S. The phenomenology of synaesthesia. J. Conscious. Stud. 2003, 10(8):49-57.
40. Indovina I., Maffei V., Bosco G., Zago M., Macaluso E., Lacquaniti F. Representation of visual gravitational motion in the human vestibular cortex. Science 2005, 308(5720):416-419.
41. Jerde T.A., Merriam E.P., Riggall A.C., Hedges J.H., Curtis C.E. Prioritized maps of space in human frontoparietal cortex. J. Neurosci. 2012, 32(48):17382-17390.
42. Jonides J., Lewis R.L., Nee D.E., Lustig C.A., Berman M.G., Moore K.S. The mind and brain of short-term memory. Annu. Rev. Psychol. 2008, 59:193-224.
43. Kamitani Y., Tong F. Decoding the visual and subjective contents of the human brain. Nat. Neurosci. 2005, 8(5):679-685.
44. Kamitani Y., Tong F. Decoding seen and attended motion directions from activity in the human visual cortex. Curr. Biol. 2006, 16(11):1096-1102.
45. Keysers C., Wicker B., Gazzola V., Anton J.-L., Fogassi L., Gallese V. A touching sight: SII/PV activation during the observation and experience of touch. Neuron 2004, 42(2):335-346.
46. Kiebel S.J., Poline J.B., Friston K.J., Holmes A.P., Worsley K.J. Robust smoothness estimation in statistical parametric maps using standardized residuals from the general linear model. Neuroimage 1999, 10(6):756-766.
47. Konen C.S., Kastner S. Two hierarchically organized neural systems for object information in human visual cortex. Nat. Neurosci. 2008, 11(2):224-231.
48. Kriegeskorte N., Goebel R., Bandettini P. Information-based functional brain mapping. Proc. Natl. Acad. Sci. U. S. A. 2006, 103(10):3863-3868.
49. Lewis-Peacock J.A., Postle B.R. Temporary activation of long-term memory supports working memory. J. Neurosci. 2008, 28(35):8765-8771.
50. Lewis-Peacock J.A., Postle B.R. Decoding the internal focus of attention. Neuropsychologia 2012, 50(4):470-478.
51. Lewis-Peacock J.A., Drysdale A.T., Oberauer K., Postle B.R. Neural evidence for a distinction between short-term memory and the focus of attention. J. Cogn. Neurosci. 2012, 24(1):61-79.
52. Ludwig, V.U., Simner, J., n.d. What colour does that feel? Tactile-visual mapping and the development of cross-modality. Cortex, (0).
53. Macaluso E., Driver J. Multisensory spatial interactions: a window onto functional integration in the human brain. Trends Neurosci. 2005, 28(5):264-271.
54. Malikovic A., Amunts K., Schleicher A., Mohlberg H., Eickhoff S.B., Wilms M., Palomero-Gallagher N., et al. Cytoarchitectonic analysis of the human extrastriate cortex in the region of V5/MT+: a probabilistic, stereotaxic map of area hOc5. Cereb. Cortex 2007, 17(3):562-574.
55. Martino G., Marks L.E. Cross-modal interaction between vision and touch: the role of synesthetic correspondence. Perception 2000, 29(6):745-754.
56. McCarthy G., Blamire A.M., Puce A., Nobre A.C., Bloch G., Hyder F., Goldman-Rakic P., et al. Functional magnetic resonance imaging of human prefrontal cortex activation during a spatial working memory task. Proc. Natl. Acad. Sci. 1994, 91(18):8690-8694.
57. Motulsky H., Christopoulos A. Fitting Models to Biological Data Using Linear and Nonlinear Regression: A Practical Guide to Curve Fitting 2004, Oxford University Press, U.S.A.
58. Ollinger J.M., Shulman G.L., Corbetta M. Separating processes within a trial in event-related functional MRI: I. Method Neuroimage 2001, 13(1):210-217.
59. Postle B.R. Working memory as an emergent property of the mind and brain. Neuroscience 2006, 139(1):23-38.
60. Postle B.R., Druzgal T.J., D'Esposito M. Seeking the neural substrates of visual working memory storage. Cortex 2003, 39(4-5):927-946.
61. Ramachandran V.S., Hubbard E.M. Synaesthesia-a window into perception, thought and language. J. Conscious. Stud. 2001, 8(12):3-34.
62. Ranganath C., Cohen M.X., Dam C., D'Esposito M. Inferior temporal, prefrontal, and hippocampal contributions to visual working memory maintenance and associative memory retrieval. J. Neurosci. 2004, 24(16):3917-3925.
63. Ranganath C., DeGutis J., D'Esposito M. Category-specific modulation of inferior temporal activity during working memory encoding and maintenance. Cogn. Brain Res. 2004, 20(1):37-45.
64. Riggall A.C., Postle B.R. The relationship between working memory storage and elevated activity as measured with functional magnetic resonance imaging. J. Neurosci. 2012, 32(38):12990-12998.
65. Schaefer M., Heinze H.-J., Rotte M. Seeing the hand being touched modulates the primary somatosensory cortex. Neuroreport 2005, 16(10):1101-1105.
66. Schluppeck D., Glimcher P., Heeger D.J. Topographic organization for delayed saccades in human posterior parietal cortex. J. Neurophysiol. 2005, 94(2):1372-1384.
67. Serences J.T., Boynton G.M. Feature-based attentional modulations in the absence of direct visual stimulation. Neuron 2007, 55(2):301-312.
68. Serences J.T., Boynton G.M. The representation of behavioral choice for motion in human visual cortex. J. Neurosci. 2007, 27(47):12893-12899.
69. Serences J.T., Ester E.F., Vogel E.K., Awh E. Stimulus-specific delay activity in human primary visual cortex. Psychol. Sci. 2009, 20(2):207-214.
70. Sereno A.B., Maunsell J.H.R. Shape selectivity in primate lateral intraparietal cortex. Nature 1998, 395(6701):500-503.
71. Sereno M.I., Pitzalis S., Martinez A. Mapping of contralateral space in retinotopic coordinates by a parietal cortical area in humans. Science 2001, 294(5545):1350-1354.
72. Simner J., Ludwig V.U. The color of touch: a case of tactile-visual synaesthesia. Neurocase 2012, 18(2):167-180.
73. Sneve M.H., Alnæs D., Endestad T., Greenlee M.W., Magnussen S. Visual short-term memory: activity supporting encoding and maintenance in retinotopic visual cortex. Neuroimage 2012, 63(1):166-178.
74. Soon C.S., Brass M., Heinze H.J., Haynes J.D. Unconscious determinants of free decisions in the human brain. Nat. Neurosci. 2008, 11(5):543-545.
75. Sperling G. The information available in brief visual presentations. Psychol. Monogr.: Gen. Appl. 1960, 74(11):1-29.
76. Stokes M.G., Kusunoki M., Sigala N., Nili H., Gaffan D., Duncan J. Dynamic coding for cognitive control in prefrontal cortex. Neuron 2013, 78(2):364-375.
77. Tootell R.B., Hadjikhani N., Hall E.K., Marrett S., Vanduffel W., Vaughan J.T., Dale A.M. The retinotopy of visual spatial attention. Neuron 1998, 21(6):1409-1422.
78. Ungerleider L.G., Courtney S.M., Haxby J.V. A neural system for human visual working memory. Proc. Natl. Acad. Sci. 1998, 95(3):883-890.
79. Van Essen D.C. Organization of visual areas in macaque and human cerebral cortex. The Visual Neurosciences 2004, 1:507-521. MIT Press Cambridge, MA. L. Chalupa, J. Werner (Eds.).
80. Van Kemenade B.M., Seymour K., Wacker E., Spitzer B., Blankenburg F., Sterzer P. Tactile and visual motion direction processing in hMT+/V5. NeuroImage 2014, 84:420-427.
81. Wacker E., Spitzer B., Lützkendorf R., Bernarding J., Blankenburg F. Tactile motion and pattern processing assessed with high-field fMRI. PLoS ONE 2011, 6(9):e24860.
82. Wager T.D., Smith E.E. Neuroimaging studies of working memory: a meta-analysis. Cogn. Affect. Behav. Neurosci. 2003, 3(4):255-274.
83. Wandell B.A., Dumoulin S.O., Brewer A.A. Visual field maps in human cortex. Neuron 2007, 56(2):366-383.
84. Wilms M., Eickhoff S., Specht K., Amunts K., Shah N., Malikovic A., Fink G. Human V5/MT+: comparison of functional and cytoarchitectonic data. Anat. Embryol. 2005, 210(5):485-495.
85. Woloszyn L., Sheinberg D.L. Neural dynamics in inferior temporal cortex during a visual working memory task. J. Neurosci. 2009, 29(17):5494-5507.
86. Zhou Y.-D., Fuster J.M. Neuronal activity of somatosensory cortex in a cross-modal (visuo-haptic) memory task. Exp. Brain Res. 1997, 116(3):551-555.
87. Zhou Y.-D., Fuster J.M. Visuo-tactile cross-modal associations in cortical somatosensory cells. Proc. Natl. Acad. Sci. 2000, 97(17):9777-9782.
88. Zimmer H.D. Visual and spatial working memory: from boxes to networks. Neurosci. Biobehav. Rev. 2008, 32(8):1373-1395.