Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders

Neuron

Volumn 60, Issue 5, 2008, Pages 915-929

  Miyawaki, Yoichi ^a,b   Uchida, Hajime ^b,c   Yamashita, Okito ^b   Sato, Masa aki ^b   Morito, Yusuke ^d,e   Tanabe, Hiroki C ^d,e   Sadato, Norihiro ^d,e   Kamitani, Yukiyasu ^b,c  

^a NATIONAL INSTITUTE OF INFORMATION AND COMMUNICATIONS TECHNOLOGY   (Japan)

^b ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

^c NARA INSTITUTE OF SCIENCE AND TECHNOLOGY   (Japan)

^d GRADUATE UNIVERSITY FOR ADVANCED STUDIES   (Japan)

^e NATIONAL INSTITUTE FOR PHYSIOLOGICAL SCIENCES   (Japan)

Author keywords

SYSNEURO

Indexed keywords

ADULT; ANALYSIS OF VARIANCE; ARTICLE; CONTROLLED STUDY; ELECTROENCEPHALOGRAM; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; IMAGE QUALITY; IMAGE RECONSTRUCTION; MALE; NEUROIMAGING; NORMAL HUMAN; PRIORITY JOURNAL; RETINA IMAGE;

BRAIN; BRAIN MAPPING; CONTRAST SENSITIVITY; FEMALE; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; OXYGEN; PATTERN RECOGNITION, VISUAL; PHOTIC STIMULATION;

EID: 57049118473     PISSN: 08966273     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neuron.2008.11.004     Document Type: Article

References (40)

1. Arsenault A.S., Wilkinson F., and Kingdom F.A. Modulation frequency and orientation tuning of second-order texture mechanisms. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 16 (1999) 427-435
2. Averbeck B.B., Latham P.E., and Pouget A. Neural correlations, population coding and computation. Nat. Rev. Neurosci. 7 (2006) 358-366
3. Bishop C.M. Pattern Recognition and Machine Learning (2006), Springer, New York
4. Campbell F.W., and Robson J.G. Application of Fourier analysis to the visibility of gratings. J. Physiol. 197 (1968) 551-566
5. Chen Y., Geisler W.S., and Seidemann E. Optimal decoding of correlated neural population responses in the primate visual cortex. Nat. Neurosci. 9 (2006) 1412-1420
6. Cox D.D., and Savoy R.L. Functional magnetic resonance imaging (fMRI) "brain reading": detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 19 (2003) 261-270
7. De Valois R.L., Albrecht D.G., and Thorell L.G. Spatial frequency selectivity of cells in macaque visual cortex. Vision Res. 22 (1982) 545-559
8. Donoghue J.P. Connecting cortex to machines: recent advances in brain interfaces. Nat. Neurosci. 5 (2002) 1085-1088
9. Dougherty R.F., Koch V.M., Brewer A.A., Fischer B., Modersitzki J., and Wandell B.A. Visual field representations and locations of visual areas V1/2/3 in human visual cortex. J. Vis. 3 (2003) 586-598
10. Dumoulin S.O., and Wandell B.A. Population receptive field estimates in human visual cortex. Neuroimage 39 (2008) 647-660
11. Duncan R.O., and Boynton G.M. Cortical magnification within human primary visual cortex correlates with acuity thresholds. Neuron 38 (2003) 659-671
12. Engel S.A., Rumelhart D.E., Wandell B.A., Lee A.T., Glover G.H., Chichilnisky E.J., and Shadlen M.N. fMRI of human visual cortex. Nature 369 (1994) 525
13. Engel S.A., Glover G.H., and Wandell B.A. Retinotopic organization in human visual cortex and the spatial precision of functional MRI. Cereb. Cortex 7 (1997) 181-192
14. Hansen K.A., David S.V., and Gallant J.L. Parametric reverse correlation reveals spatial linearity of retinotopic human V1 BOLD response. Neuroimage 23 (2004) 233-241
15. Haxby J.V., Gobbini M.I., Furey M.L., Ishai A., Schouten J.L., and Pietrini P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293 (2001) 2425-2430
16. Haynes J.D., and Rees G. Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nat. Neurosci. 8 (2005) 686-691
17. Hubel D.H., and Wiesel T.N. Receptive fields and functional architecture of monkey striate cortex. J. Physiol. 195 (1968) 215-243
18. Kamitani Y., and Tong F. Decoding the visual and subjective contents of the human brain. Nat. Neurosci. 8 (2005) 679-685
19. Kamitani Y., and Tong F. Decoding seen and attended motion directions from activity in the human visual cortex. Curr. Biol. 16 (2006) 1096-1102
20. Kapadia M.K., Ito M., Gilbert C.D., and Westheimer G. Improvement in visual sensitivity by changes in local context: parallel studies in human observers and in V1 of alert monkeys. Neuron 15 (1995) 843-856
21. Kay K.N., Naselaris T., Prenger R.J., and Gallant J.L. Identifying natural images from human brain activity. Nature 452 (2008) 352-355
22. Kosslyn S.M., Thompson W.L., Kim I.J., and Alpert N.M. Topographical representations of mental images in primary visual cortex. Nature 378 (1995) 496-498
23. Kraft A., Schira M.M., Hagendorf H., Schmidt S., Olma M., and Brandt S.A. fMRI localizer technique: efficient acquisition and functional properties of single retinotopic positions in the human visual cortex. Neuroimage 28 (2005) 453-463
24. Landy M.S., and Oruc I. Properties of second-order spatial frequency channels. Vision Res. 42 (2002) 2311-2329
25. Meng M., Remus D.A., and Tong F. Filling-in of visual phantoms in the human brain. Nat. Neurosci. 8 (2005) 1248-1254
26. Mitchell T.M., Shinkareva S.V., Carlson A., Chang K.M., Malave V.L., Mason R.A., and Just M.A. Predicting human brain activity associated with the meanings of nouns. Science 320 (2008) 1191-1195
27. Olshausen B.A., and Field D.J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381 (1996) 607-609
28. Poggio T., and Bizzi E. Generalization in vision and motor control. Nature 431 (2004) 768-774
29. Sasaki Y., and Watanabe T. The primary visual cortex fills in color. Proc. Natl. Acad. Sci. USA 101 (2004) 18251-18256
30. Sereno M.I., Dale A.M., Reppas J.B., Kwong K.K., Belliveau J.W., Brady T.J., Rosen B.R., and Tootell R.B. Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science 268 (1995) 889-893
31. Shmuel A., Yacoub E., Chaimow D., Logothetis N.K., and Ugurbil K. Spatio-temporal point-spread function of fMRI signal in human gray matter at 7 Tesla. Neuroimage 35 (2007) 539-552
32. Singh K.D., Smith A.T., and Greenlee M.W. Spatiotemporal frequency and direction sensitivities of human visual areas measured using fMRI. Neuroimage 12 (2000) 550-564
33. Smith A.T., Singh K.D., Williams A.L., and Greenlee M.W. Estimating receptive field size from fMRI data in human striate and extrastriate visual cortex. Cereb. Cortex 11 (2001) 1182-1190
34. Stanley G.B., Li F.F., and Dan Y. Reconstruction of natural scenes from ensemble responses in the lateral geniculate nucleus. J. Neurosci. 19 (1999) 8036-8042
35. Thirion B., Duchesnay E., Hubbard E., Dubois J., Poline J.B., Lebihan D., and Dehaene S. Inverse retinotopy: inferring the visual content of images from brain activation patterns. Neuroimage 33 (2006) 1104-1116
36. Tipping M. Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn. Res. 1 (2001) 211-244
37. Vanni S., Henriksson L., and James A.C. Multifocal fMRI mapping of visual cortical areas. Neuroimage 27 (2005) 95-105
38. Wolpaw J.R., and McFarland D.J. Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Proc. Natl. Acad. Sci. USA 101 (2004) 17849-17854
39. Yamashita O., Sato M.A., Yoshioka T., Tong F., and Kamitani Y. Sparse estimation automatically selects voxels relevant for the decoding of fMRI activity patterns. Neuroimage 42 (2008) 1414-1429
40. Zipser K., Lamme V.A., and Schiller P.H. Contextual modulation in primary visual cortex. J. Neurosci. 16 (1996) 7376-7389