Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream

Journal of Neuroscience

Volumn 35, Issue 27, 2015, Pages 10005-10014

  Güçlü, Umut ^a   van Gerven, Marcel A J ^a  

^a RADBOUD UNIVERSITY   (Netherlands)

Author keywords

Deep learning; Functional magnetic resonance imaging; Neural coding

Indexed keywords

ARTICLE; BRAIN FUNCTION; BRAIN MAPPING; CONTROLLED STUDY; ELECTROENCEPHALOGRAM; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; MALE; MEASUREMENT ACCURACY; NERVE CELL NETWORK; NERVE POTENTIAL; NERVE STIMULATION; NUCLEAR MAGNETIC RESONANCE IMAGING; OCCIPITAL CORTEX; PRIORITY JOURNAL; STRIATE CORTEX; TASK PERFORMANCE; TRAINING; VISUAL STIMULATION; VOXEL BASED MORPHOMETRY; BIOLOGICAL MODEL; BLOOD; BRAIN; IMAGE PROCESSING; PHOTOSTIMULATION; PHYSIOLOGY; VASCULARIZATION; VISUAL SYSTEM;

OXYGEN;

BRAIN; BRAIN MAPPING; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; MODELS, NEUROLOGICAL; OXYGEN; PHOTIC STIMULATION; VISUAL PATHWAYS;

EID: 84936878573     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.5023-14.2015     Document Type: Article

References (62)

1. Agrawal P, Stansbury D, Malik J, Gallant JL (2014) Pixels to voxels: modeling visual representation in the human brain. arXiv 1407. 5104 [q-bio. NC].
2. Aquino KM, Robinson PA, Drysdale PM (2014) Spatiotemporal hemodynamic response functions derived from physiology. J Theor Biol 347: 118-136. CrossRef Medline.
3. Cadieu CF, Hong H, Yamins DL, Pinto N, Ardila D, Solomon EA, Majaj NJ, DiCarlo JJ (2014) Deep neural networks rival the representation of primate IT cortex for core visual object recognition. PLoS Comput Biol 10: e1003963. CrossRef Medline.
4. Carandini M, Demb JB, Mante V, Tolhurst DJ, Dan Y, Olshausen BA, Gallant JL, Rust NC (2005) Do we know what the early visual system does? J Neurosci 25: 10577-10597. CrossRef Medline.
5. Chatfield K, Simonyan K, Vedaldi A, ZissermanA (2014) Return of the devil in the details: delving deep into convolutional nets. arXiv 1405. 3531. [cs. CV].
6. Cox DD (2014) Do we understand high-level vision? Curr Opin Neurobiol 25: 187-193. CrossRef Medline.
7. Çukur T, Nishimoto S, Huth AG, Gallant JL (2013) Attention during natural vision warps semantic representation across the human brain. Nat Neurosci 16: 763-770. CrossRef Medline.
8. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) ImageNet: a largescale hierarchical image database. IEEE Conference on Computer Vision and Pattern Recognition, pp 248-255.
9. Desimone R, Albright TD, Gross CG, Bruce C (1984) Stimulus-selective properties of inferior temporal neurons in the macaque. J Neurosci 4: 2051-2062. Medline.
10. DiCarlo JJ, CoxDD (2007) Untangling invariant object recognition. Trends Cogn Sci 11: 333-341. CrossRef Medline.
11. Dumoulin SO, Wandell BA (2008) Population receptive field estimates in human visual cortex. Neuroimage 39: 647-660. CrossRef Medline.
12. Emadi N, Esteky H (2014) Behavioral demand modulates object category representation in the inferior temporal cortex. J Neurophysiol 112: 2628-2637. CrossRef Medline.
13. Felsen G, Dan Y (2005) A natural approach to studying vision. Nat Neurosci 8: 1643-1646. CrossRef Medline.
14. Forstmann BU, Wagenmakers E-J (2015) Model-based cognitive neuroscience. New York: Springer.
15. Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern 36: 193-202. CrossRef Medline.
16. Goodale MA, Milner AD (1992) Separate visual pathways for perception and action. Trends Neurosci 15: 20-25. CrossRef Medline.
17. Grill-Spector K, Weiner KS (2014) The functional architecture of the ventral temporal cortex and its role in categorization. Nat Rev Neurosci 15: 536-548. CrossRef Medline.
18. Grill-Spector K, Kushnir T, Hendler T, Edelman S, Itzchak Y, Malach R (1998) A sequence of object-processing stages revealed by fMRI in the human occipital lobe. Hum Brain Mapp 6: 316-328. CrossRef Medline.
19. Gross CG, Rocha-Miranda CE, Bender DB (1972) Visual properties of neurons in inferotemporal cortex of the macaque. J Neurophysiol 35: 96-111. Medline.
20. Güçlü U, van Gerven MAJ (2014) Unsupervised feature learning improves prediction of human brain activity in response to natural images. PLoS Comput Biol 10: e1003724. CrossRef Medline.
21. Harrison SA, Tong F (2009) Decoding reveals the contents of visual working memory in early visual areas. Nature 458: 632-635. CrossRef Medline.
22. Haxby JV, Guntupalli JS, Connolly AC, Halchenko YO, Conroy BR, Gobbini MI, Hanke M, Ramadge PJ (2011) A common, high-dimensional model of the representational space in human ventral temporal cortex. Neuron 72: 404-416. CrossRef Medline.
23. Hegdé J, Van Essen DC (2007) A comparative study of shape representation in macaque visual areas V2 and V4. Cereb Cortex 17: 1100-1116. CrossRef Medline.
24. Hinton GE (2007) Learning multiple layers of representation. Trends Cogn Sci 11: 428-434. CrossRef Medline.
25. Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv 2070. 580v1 [cs. NE].
26. Hochstein S, Ahissar M (2002) View from the top: hierarchies and reverse hierarchies in the visual system. Neuron 36: 791-804. CrossRef Medline.
27. Horikawa T, Tamaki M, Miyawaki Y, Kamitani Y (2013) Neural decoding of visual imagery during sleep. Science 340: 639-642. CrossRef Medline.
28. Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J Physiol 160: 106-154. CrossRef Medline.
29. Hung CP, Kreiman G, Poggio T, DiCarlo JJ (2005) Fast readout of object identity from macaque inferior temporal cortex. Science 310: 863-866. CrossRef Medline.
30. Huth AG, Nishimoto S, Vu AT, Gallant JL (2012) A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron 76: 1210-1224. CrossRef Medline.
31. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. arXiv 1408. 5093 [csCV].
32. Jones JP, Palmer LA (1987) An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. J Neurophysiol 58: 1233-1258. Medline.
33. Kay KN, Naselaris T, Prenger RJ, Gallant JL (2008) Identifying natural images from human brain activity. Nature 452: 352-355. CrossRef Medline.
34. Khaligh-Razavi SM, Kriegeskorte N (2014) Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS Comput Biol 10: e1003915. CrossRef Medline.
35. Kok P, de Lange FP (2014) Shape perception simultaneously up-and downregulates neural activity in the primary visual cortex. Curr Biol 24: 1531-1535. CrossRef Medline.
36. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Advances in neural information processing systems, NIPS Proceedings, pp 1097-1105.
37. Larsson J, Heeger DJ (2006) Two retinotopic visual areas in human lateral occipital cortex. J Neurosci 26: 13128-13142. CrossRef Medline.
38. Logothetis NK, WandellBA (2004) Interpreting theBOLDsignal. Annu Rev Physiol 66: 735-769. CrossRef Medline.
39. Markov NT, Vezoli J, Chameau P, Falchier A, Quilodran R, Huissoud C, Lamy C, Misery P, Giroud P, Ullman S, Barone P, Dehay C, Knoblauch K, Kennedy H (2014) Anatomy of hierarchy: feedforward and feedback pathways in macaque visual cortex. J Comp Neurol 522: 225-259. CrossRef Medline.
40. McKee JL, Riesenhuber M, Miller EK, Freedman DJ (2014) Task dependence of visual and category representations in prefrontal and inferior temporal cortices. J Neurosci 34: 16065-16075. CrossRef Medline.
41. Meeds E, Roweis S (2007) Nonparametric Bayesian biclustering. Technical report. Department of Computer Science, University of Toronto.
42. Mesulam MM (1998) From sensation to cognition. Brain 121: 1013-1052. CrossRef Medline.
43. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D (2015) Human-level control through deep reinforcement learning. Nature 518: 529-533. CrossRef Medline.
44. Naselaris T, Prenger RJ, Kay KN, Oliver M, Gallant JL (2009) Bayesian reconstruction of natural images from human brain activity. Neuron 63: 902-915. CrossRef Medline.
45. Naselaris T, Kay KN, Nishimoto S, Gallant JL (2011) Encoding and decoding in fMRI. Neuroimage 56: 400-410. CrossRef Medline.
46. Norris DG (2006) Principles of magnetic resonance assessment of brain function. J Magn Reson Imaging 23: 794-807. CrossRef Medline.
47. Olshausen BA, Field DJ (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381: 607-609. CrossRef Medline.
48. Patterson K, Nestor PJ, Rogers TT (2007) Where do you know what you know? The representation of semantic knowledge in the human brain. Nat Rev Neurosci 8: 976-987. CrossRef Medline.
49. Pedregosa F, Eickenberg M, Ciuciu P, Thirion B, Gramfort A (2014) Datadriven HRF estimation for encoding and decoding models. arXiv 1402. 7015 [csCE].
50. Rainer G, Lee H, Logothetis NK (2004) The effect of learning on the function of monkey extrastriate visual cortex. PLoS Biol 2: E44. CrossRef Medline.
51. Rust NC, Movshon JA (2005) In praise of artifice. Nat Neurosci 8: 1647-1650. CrossRef Medline.
52. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275: 1593-1599. CrossRef Medline.
53. Serre T, Wolf L, Bileschi S, Riesenhuber M, Poggio T (2007) Robust object recognition with cortex-like mechanisms. IEEE Trans Pattern Anal Mach Intell 29: 411-426. CrossRef Medline.
54. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv 1409. 1556 [csCV].
55. Smith AT, Singh KD, Williams AL, GreenleeMW (2001) Estimating receptive field size from fMRI data in human striate and extrastriate visual cortex. Cereb Cortex 11: 1182-1190. CrossRef Medline.
56. Tanaka K (1996) Inferotemporal cortex and object vision. Annu Rev Neurosci 19: 109-139. CrossRef Medline.
57. Tanigawa H, Lu HD, RoeAW (2010) Functional organization for color and orientation in macaque V4. Nat Neurosci 13: 1542-1548. CrossRef Medline.
58. Thirion B, Duchesnay E, Hubbard E, Dubois J, Poline JB, Lebihan D, Dehaene S (2006) Inverse retinotopy: inferring the visual content of images from brain activation patterns. Neuroimage 33: 1104-1116. CrossRef Medline.
59. van Gerven MAJ, de Lange FP, Heskes T (2010) Neural decoding with hierarchical generative models. Neural Comput 22: 3127-3142. CrossRef Medline.
60. Wu MC, David SV, Gallant JL (2006) Complete functional characterization of sensory neurons by system identification. Annu Rev Neurosci 29: 477-505. CrossRef Medline.
61. Yamins DL, Hong H, Cadieu CF, Solomon EA, Seibert D, DiCarlo JJ (2014) Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc Natl Acad Sci U S A 111: 8619-8624. CrossRef Medline.
62. Zeiler MD, Fergus R (2012) Visualizing and understanding convolutional networks. arXiv 1311. 2901 [csCV].