A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes

NeuroImage

Volumn 105, Issue , 2015, Pages 215-228

  Naselaris, Thomas ^a   Olman, Cheryl A ^b   Stansbury, Dustin E ^c   Ugurbil, Kamil ^b   Gallant, Jack L ^c,d  

^a MEDICAL UNIVERSITY OF SOUTH CAROLINA   (United States)

^b UNIVERSITY OF MINNESOTA   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Decoding; FMRI; Mental imagery; Perception; Vision; Voxel wise encoding models

Indexed keywords

ACCURACY; ADULT; ART; ARTICLE; ARTIFICIAL NEURAL NETWORK; BOLD SIGNAL; EVOKED CORTICAL RESPONSE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; IMAGERY; IMAGINATION; MEMORY CONSOLIDATION; NORMAL HUMAN; SENSITIVITY AND SPECIFICITY; SIGNAL DETECTION; VISION; VISUAL CORTEX; VISUAL STIMULATION; VOXEL WISE ENCODING MODEL; WAVELET ANALYSIS; BRAIN MAPPING; NUCLEAR MAGNETIC RESONANCE IMAGING; PATTERN RECOGNITION; PHYSIOLOGY; PROCEDURES;

ADULT; BRAIN MAPPING; HUMANS; IMAGINATION; MAGNETIC RESONANCE IMAGING; PATTERN RECOGNITION, VISUAL; VISUAL CORTEX;

EID: 84910017372     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2014.10.018     Document Type: Article

References (74)

1. Adriany G., Waks M., Tramm B., Schillak S., Yacoub E., de Martino F., Van de Moortele P., Naselaris T., Olman C., Vaughan T., Ugurbil K. An open faced 4 ch. loop transmit/16 ch. receive array coil for HiRes fMRI at 7tesla. Int. Soc. Magn. Reson. Med. 2012, 20:429.
2. Albers A.M., Kok P., Toni I., Dijkerman H.C., de Lange F.P. Shared representations for working memory and mental imagery in early visual cortex. Curr. Biol. 2013, 23:1427-1431.
3. Albright T.D. On the perception of probable things: neural substrates of associative memory, imagery, and perception. Neuron 2012, 74:227-245.
4. Amedi A., Malach R., Pascual-Leone A. Negative BOLD differentiates visual imagery and perception. Neuron 2005, 48:859-872.
5. Angelucci A., Levitt J.B., Walton E.J., Hupe J.M., Bullier J., Lund J.S. Circuits for local and global signal integration in primary visual cortex. J. Neurosci. 2002, 22:8633-8646.
6. Bastos A.M., Usrey W.M., Adams R.A., Mangun G.R., Fries P., Friston K.J. Canonical microcircuits for predictive coding. Neuron 2012, 76:695-711.
7. Berkes P., Orbán G., Lengyel M., Fiser J. Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science 2011, 331:83-87.
8. Bullier J. What is fed back?. Twenty-three Problems in Systems Neuroscience 2006, Oxford University Press. T. Sejnowski, J. Hemmen (Eds.).
9. Chen W., Kato T., Zhu X.H., Ogawa S., Tank D.W., Ugurbil K. Human primary visual cortex and lateral geniculate nucleus activation during visual imagery. Neuroreport 1998, 9:3669-3674.
10. Cichy R.M., Heinzle J., Haynes J.D. Imagery and perception share cortical representations of content and location. Cereb. Cortex 2011, 22:372-380.
11. Cui X., Jeter C.B., Yang D., Montague P.R., Eagleman D.M. Vividness of mental imagery: individual variability can be measured objectively. Vis. Res. 2007, 47:474-478.
12. Çukur T., Huth A.G., Nishimoto S., Gallant J.L. Functional subdomains within human FFA. J. Neurosci. 2013, 33:16748-16766.
13. D'Esposito M., Detre J.A., Aguirre G.K., Stallcup M., Alsop D.C., Tippet L.J., Farah M.J. A functional MRI study of mental image generation. Neuropsychologia 1997, 35:725-730.
14. Daselaar S.M., Porat Y., Huijbers W., Pennartz C.M.A. Modality-specific and modality-independent components of the human imagery system. NeuroImage 2010, 52:677-685.
15. Engel S.A., Glover G.H., Wandell B.A. Retinotopic organization in human visual cortex and the spatial precision of functional MRI. Cereb. Cortex 1997, 181-192.
16. Felleman D.J., Van Essen D.C. Distributed hierarchical processing in the primate. Cereb. Cortex 1991, 1:1-47.
17. Formisano E., Linden D.E.J., Di Salle F., Trojano L., Esposito F., Sack A.T., et al. Tracking the mind's image in the brain I: time-resolved fMRI during visuospatial mental imagery. Neuron 2002, 35:185-194.
18. Ganis G., Thompson W.L., Kosslyn S.M. Brain areas underlying visual mental imagery and visual perception: an fMRI study. Cogn. Brain Res. 2004, 20:226-241.
19. Goebel R., Khorram-Sefat D., Muckl L., Hacker H., Singer W. The constructive nature of vision: direct evidence from functional magnetic resonance imaging studies of apparent motion and motion imagery. Eur. J. Neurosci. 1998, 10:1563-1573.
20. Gorgolewski K., Burns C.D., Madison C., Clark D., Halchenko Y.O., Waskom M.L., Ghosh S.S. Nipype: a flexible, lightweight and extensible neuroimaging data processing framework in Python. Front. Neuroinform 2011, 5:13. (Available: http://www.frontiersin.org/Neuroinformatics/10.3389/fninf.2011.00013/full Accessed 10 September 2013).
21. Gregory R.L. Perceptions as hypotheses. Philos. Trans. R. Soc. B 1980, 290:181-197.
22. Handy T.C., Miller M., Schott B., Shroff N., Janata P., Van Horn J., Inati S., Grafton S., Gazzaniga M. Visual imagery and memory: do retrieval strategies affect what the mind's eye sees?. Eur. J. Cogn. Psychol. 2004, 16:631-652.
23. Harrison S.A., Tong F. Decoding reveals the contents of visual working memory in early visual areas. Nature 2009, 458:632-635.
24. Horikawa T., Tamaki M., Miyawaki Y., Kamitani Y. Neural decoding of visual imagery during sleep. Science 2013, 340:639-642.
25. Ishai A., Ungerleider L.G., Haxby J.V. Distributed neural systems for the generation of visual images. Neuron 2000, 28:979-990.
26. Ishai A., Haxby J.V., Ungerleider L.G. Visual imagery of famous faces: effects of memory and attention revealed by fMRI. NeuroImage 2002, 17:1729-1741.
27. Johnson M.R., Johnson M.K. Decoding individual natural scene representations during perception and imagery. Front. Hum. Neurosci. 2014, 8.
28. Joo S.J., Boynton G.M., Murray S.O. Long-range, pattern-dependent contextual effects in early human visual cortex. Curr. Biol. 2012, 22:781-786.
29. Kamitani Y., Tong F. Decoding the visual and subjective contents of the human brain. Nat. Neurosci. 2005, 8:679-685.
30. Kay K.N., Naselaris T., Prenger R.J., Gallant J.L. Identifying natural images from human brain activity. Nature 2008, 452:352-355.
31. Klein I., Paradis A.L., Poline J.B., Kosslyn S.M., Le Bihan D. Transient activity in the human calcarine cortex during visual-mental imagery: an event-related fMRI study. J. Cogn. Neurosci. 2000, 12:15-23.
32. Knauff M., Kassubek J., Mulack T., Greenlee M.W. Cortical activation evoked by visual mental imagery as measured by fMRI. Neuroreport 2000, 11:3957-3962.
33. Kosslyn S.M., Thompson W.L. When is early visual cortex activated during visual mental imagery. Psychol. Bull. 2003, 129:723-746.
34. Kosslyn S.M., Ball T.M., Reiser B.J. Visual images preserve metric spatial information: evidence from studies of image scanning. J. Exp. Psychol. Hum. Percept. Perform. 1978, 4:47-60.
35. Kosslyn S.M., Thompson W., Ganis G. The Case for Mental Imagery 2009, Oxford University Press, Oxford, (248pp.).
36. Lambert S., Sampaio E., Scheiber C., Mauss Y. Neural substrates of animal mental imagery: calcarine sulcus and dorsal pathway involvement-an fMRI study. Brain Res. 2002, 924:176-183.
37. Le Bihan D., Turner R., Zeffiro T.A., Cuénod C.A., Jezzard P., Bonnerot V. Activation of human primary visual cortex during visual recall: a magnetic resonance imaging study. PNAS 1993, 90:11802-11805.
38. Lee T.S., Mumford D. Hierarchical Bayesian inference in the visual cortex. J. Opt. Soc. Am. A 2003, 20:1434-1448.
39. Lee S.H., Kravitz D.J., Baker C.I. Disentangling visual imagery and perception of real-world objects. NeuroImage 2012, 59:4064-4073.
40. Markov N.T., Vezoli J., Chameau P., Falchier A., Quilodran R., Huissoud C., Lamy C., et al. The anatomy of hierarchy: feedforward and feedback pathways in macaque visual cortex. J. Comp. Neurol. 2014, 522:225-259.
41. Markov N.T., Ercsey-Ravasz M., Van Essen D.C., Knoblauch K., Toroczkai Z., Kennedy H. Cortical high-density counterstream architectures. Science 2014, 342. 1238406.
42. Murray S.O., Kersten D., Olshausen B.A., Schrater P., Woods D.L. Shape perception reduces activity in human primary visual cortex. PNAS 2002, 99:15164-15169.
43. Naselaris T., Prenger R.J., Kay K.N., Oliver M., Gallant J.L. Bayesian reconstruction of natural images from human brain activity. Neuron 2009, 63:902-915.
44. Naselaris T., Kay K.N., Nishimoto S., Gallant J.L. Encoding and decoding in fMRI. NeuroImage 2011, 56:400-410.
45. Nishimoto S., Vu A.T., Naselaris T., Benjamini Y., Yu B., Gallant J.L. Reconstructing visual experiences from brain activity evoked by natural movies. Curr. Biol. 2011, 21:1641-1646.
46. O'Craven K.M., Kanwisher N. Mental imagery of faces and places activates corresponding stimulus-specific brain regions. J. Cogn. Neurosci. 2000, 12:1013-1023.
47. Olman C.A., Yacoub E. High-field FMRI for human applications: an overview of spatial resolution and signal specificity. Open Neuroimaging J. 2011, 5:74-89.
48. Olshausen B.A., Field D.J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 1996, 381:607-609.
49. Podgorny P., Shepard R.N. Functional representations common to visual perception and imagination. J. Exp. Psychol. Hum. Percept. Perform. 1978, 4:21-35.
50. Pylyshyn Z.W. What the mind's eye tells the mind's brain: a critique of mental imagery. Psychol. Bull. 1973, 80:1-24.
51. Pylyshyn Z.W. Mental imagery: in search of a theory. Behav. Brain Sci. 2002, 25:157-237.
52. Rao R.P.N., Ballard D.H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 1999, 2:79-87.
53. Reddy L., Tsuchiya N., Serre T. Reading the mind's eye: decoding category information during mental imagery. NeuroImage 2010, 50:818-825.
54. Rockland K.S., Pandya D.N. Laminar origins and terminations of cortical connections of the occipital lobe in the rhesus monkey. Brain Res. 1979, 179:3-20.
55. Ryle G. The Concept of Mind 1949, Hutchinson, London.
56. Sabbah P., Simond G., Levrier O., Habib M., Trabaud V., Murayama N., et al. Functional magnetic resonance imaging at 1.5T during sensorimotor and cognitive task. Eur. Neurol. 1995, 35:131-136.
57. Sack A.T., Sperling J.M., Prvulovic D., Formisano E., Goebel R., Di Salle F., et al. Tracking the mind's image in the brain II: transcranial magnetic stimulation reveals parietal asymmetry in visuospatial imagery. Neuron 2002, 35:195-204.
58. Sartre J.P. Imagination: A Psychological Critique 1936, Trans. F. Williams, University of Michigan Press, Ann Arbor.
59. Serences J.T. Value-based modulations in human visual cortex. Neuron 2008, 60:1169-1181.
60. Slotnick S.D., Thompson W.L., Kosslyn S.M. Visual mental imagery induces retinotopically organized activation of early visual areas. Cereb. Cortex 2005, 15:1570-1583.
61. Sorger B., Reithler J., Dahmen B., Goebel R. A real-time fMRI-based spelling device immediately enabling robust motor-independent communication. Curr. Biol. 2012, 22:1333-1338.
62. Stokes M., Thompson R., Cusack R., Duncan J. Top-down activation of shape-specific population codes in visual cortex during mental imagery. J. Neurosci. 2009, 29:1565-1572.
63. Thirion B., Duchesnay E., Hubbard E., Dubois J., Poline J.B., Le Bihan D., Dehaene S. Inverse retinotopy: inferring the visual content of images from brain activation patterns. NeuroImage 2006, 33:1104-1116.
64. Thomas N.J.T. Mental imagery. The Stanford Encyclopedia of Philosophy 2014, (URL=). Spring 2014 edition. E.N. Zalta (Ed.).
65. Thorndike E.L. On the function of mental imagery. J. Philos. Psychol. Sci. Methods 1907, 4:324-327.
66. Tong F. Imagery and visual working memory: one and the same?. Trends Cogn. Sci. 2013, 17:489-490.
67. Trojano L., Grossi D., Linden D.E.J., Formisano E., Hacker H., Zanella F.E., et al. Matching two imagined clocks: the functional anatomy of spatial analysis in the absence of visual stimulation. Cereb. Cortex 2000, 10:473-481.
68. Vetter P., Smith F.W., Muckli L. Decoding sound and imagery content in early visual cortex. Curr. Biol. 2014, 11:1256-1262.
69. Watson J.B. Psychology as the behaviorist views it. Psychol. Rev. 1913, 20:158-177.
70. Wheeler M.E., Petersen S.E., Buckner R.L. Memory's echo: vivid remembering reactivates sensory-specific cortex. PNAS 2000, 97(20):11125-11129.
71. Wittgenstein L. Philosophical Investigations 1953, Trans. G.E.M. Anscombe., Blackwell, Oxford.
72. Xing Y., Ledgeway T., McGraw P.V., Schluppeck D. Decoding working memory of stimulus contrast in early visual cortex. J. Neurosci. 2013, 33:10301-10311.
73. Yacoub E., Shmuel A., Pfeuffer J., Van De Moortele P.F., Adriany G., Andersen P., Vaughan J.T., Merkle H., Ugurbil K., Hu X. Imaging brain function in humans at 7tesla. Magn. Reson. Med. 2001, 45:588-594.
74. Yuille A., Kersten D. Vision as Bayesian inference: analysis by synthesis?. Trends Cogn. Sci. 2006, 10:301-308.