Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition

PLoS Computational Biology

Volumn 10, Issue 12, 2014, Pages

  Cadieu, Charles F ^a   Hong, Ha ^a,b   Yamins, Daniel L K ^a   Pinto, Nicolas ^a   Ardila, Diego ^a   Solomon, Ethan A ^a   Majaj, Najib J ^a   DiCarlo, James J ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b HARVARD MIT DIVISION OF HEALTH SCIENCES AND TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPLEX NETWORKS; MATHEMATICAL TRANSFORMATIONS; OBJECT RECOGNITION;

BACKGROUND VARIATION; BIOINSPIRED MODELS; GEOMETRIC TRANSFORMATIONS; INFERIOR TEMPORAL CORTICES; MACHINE-LEARNING; NEURAL RECORDINGS; OBJECTS RECOGNITION; PERFORMANCE; VISUAL OBJECT RECOGNITION; VISUAL SYSTEMS;

DEEP NEURAL NETWORKS;

ADULT; ANIMAL EXPERIMENT; ARTICLE; ARTIFICIAL NEURAL NETWORK; CLASSIFIER; COMPARATIVE STUDY; CORE VISUAL OBJECT RECOGNITION; DEEP NEURAL NETWORK; INFERIOR TEMPORAL CORTEX; KERNEL METHOD; MALE; MATHEMATICAL COMPUTING; MENTAL PERFORMANCE; NERVE CELL; NOISE; NONHUMAN; PREDICTION; VISUAL MEMORY; VISUAL SYSTEM; ALGORITHM; ANIMAL; BIOLOGICAL MODEL; NERVE CELL NETWORK; PATTERN RECOGNITION; PHYSIOLOGY; RHESUS MONKEY; TEMPORAL LOBE;

PRIMATES;

ALGORITHMS; ANIMALS; MACACA MULATTA; MALE; MODELS, NEUROLOGICAL; NERVE NET; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, VISUAL; TEMPORAL LOBE;

EID: 84919607718     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1003963     Document Type: Article

References (69)

1. Thorpe S, Fize D, Marlot C, (1996) Speed of processing in the human visual system. Nature 381: 520–522.
2. Fabre-Thorpe M, Richard G, Thorpe SJ, (1998) Rapid categorization of natural images by rhesus monkeys. Neuroreport 9: 303–308.
3. Keysers C, Xiao D, Perrett D, (2001) The Speed of Sight. Journal of Cognitive Neuroscience 13: 90–101.
4. Potter MC, Wyble B, Hagmann CE, McCourt ES, (2013) Detecting meaning in RSVP at 13 ms per picture. Attention, Perception, & Psychophysics 76: 270–279.
5. Andrews TJ, Coppola DM, (1999) Idiosyncratic characteristics of saccadic eye movements when viewing different visual environments. Vision Research 39: 2947–2953.
6. DiCarlo JJ, Zoccolan D, Rust NC, (2012) How Does the Brain Solve Visual Object Recognition? Neuron 73: 415–434.
7. Desimone R, Albright TD, Gross CG, Bruce C, (1984) Stimulus-selective properties of inferior temporal neurons in the macaque. Journal of Neuroscience 4: 2051–2062.
8. Kobatake E, Tanaka K, (1994) Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. Journal of Neurophysiology 71: 856–867.
9. Hung CP, Kreiman G, Poggio T, DiCarlo JJ, (2005) Fast readout of object identity from macaque inferior temporal cortex. Science 310: 863–866.
10. Rust NC, DiCarlo JJ, (2010) Selectivity and tolerance (“invariance”) both increase as visual information propagates from cortical area V4 to IT. Journal of Neuroscience 30: 12978–12995.
11. DiCarlo JJ, Cox DD, (2007) Untangling invariant object recognition. Trends in Cognitive Sciences 11: 333–341.
12. Fukushima K, (1980) Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics 36: 193–202.
13. Riesenhuber M, Poggio T, (1999) Hierarchical models of object recognition in cortex. Nature Neuroscience 2: 1019–1025.
14. Stringer SM, Rolls ET, (2002) Invariant Object Recognition in the Visual System with Novel Views of 3D Objects. Neural Computation 14: 2585–2596.
15. Serre T, Wolf L, Bileschi S, Riesenhuber M, Poggio T (2007) Robust Object Recognition with Cortex-Like Mechanisms. IEEE Transactions on Pattern Analysis and Machine Intelligence: 411–426.
16. Pinto N, Doukhan D, DiCarlo JJ, Cox DD, (2009) A High-Throughput Screening Approach to Discovering Good Forms of Biologically Inspired Visual Representation. PLoS Computational Biology 5: e1000579.
17. Hubel DH, Wiesel TN, (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. The Journal of Physiology 160: 106–154.
18. Hubel DH, Wiesel TN, (1968) Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology 195: 215–243.
19. Perrett DI, Oram MW, (1993) Neurophysiology of shape processing. Image and Vision Computing 11: 317–333.
20. Mel BW, (1997) SEEMORE: Combining Color, Shape, and Texture Histogramming in a Neurally Inspired Approach to Visual Object Recognition. Neural Computation 9: 777–804.
21. Wallis G, Rolls ET, (1997) Invariant Face and Object Recognition in the Visual System. Progress in Neurobiology 51: 167–194.
22. Serre T, Kreiman G, Kouh M, Cadieu C, Knoblich U, et al. (2007) A quantitative theory of immediate visual recognition. In: Progress in Brain Research, Elsevier. pp.33–56.
23. Le QV, Monga R, Devin M, Chen K, Corrado GS, et al. (2012) Building high-level features using large scale unsupervised learning. In: ICML 2012: 29th International Conference on Machine Learning. pp.1–11.
24. Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25. pp.1106–1114.
25. Zeiler MD, Fergus R (2013) Visualizing and Understanding Convolutional Networks. ArXiv.org, arXiv: 1311.2901[cs.CV]
26. Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, et al. (2014) OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks. In: International Conference on Learning Representations. pp.1–16.
27. Yamins DLK, Hong H, Cadieu CF, Solomon EA, Seibert D, et al. (2014) Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences 111: 8619–8624.
28. Braun ML, (2006) Accurate Error Bounds for the Eigenvalues of the Kernel Matrix. The Journal of Machine Learning Research 7: 2303–2328.
29. Braun ML, Buhmann JM, Müller KR, (2008) On relevant dimensions in kernel feature spaces. The Journal of Machine Learning Research 9: 1875–1908.
30. Montavon G, Braun ML, Müller KR, (2011) Kernel Analysis of Deep Networks. The Journal of Machine Learning Research 12: 2563–2581.
31. Kriegeskorte N, Mur M, Ruff DA, Kiani R, Bodurka J, et al. (2008) Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey. Neuron 60: 1126–1141.
32. Kriegeskorte N, Mur M, Bandettini P (2008) Representational Similarity Analysis – Connecting the Branches of Systems Neuroscience. Frontiers in Systems Neuroscience 2.
33. Mur M, Ruff DA, Bodurka J, De Weerd P, Bandettini PA, et al. (2012) Categorical, yet graded–single-image activation profiles of human category-selective cortical regions. The Journal of Neuroscience 32: 8649–8662.
34. Yamins D, Hong H, Cadieu CF, DiCarlo JJ, (2013) Hierarchical Modular Optimization of Convolutional Networks Achieves Representations Similar to Macaque IT and Human Ventral Stream. Advances in Neural Information Processing Systems0020: 3093–3101.
35. Pinto N, Cox DD, DiCarlo JJ, (2008) Why is Real-World Visual Object Recognition Hard? PLoS Computational Biology 4: e27.
36. Pinto N, Barhomi Y, Cox DD, DiCarlo JJ, (2011) Comparing state-of-the-art visual features on invariant object recognition tasks. IEEE Workshop on Applications of Computer Vision (WACV 2011): 463–470.
37. Weiskrantz L, Saunders RC, (1984) Impairments of Visual Object Transforms in Monkeys. Brain 107: 1033–1072.
38. Oliva A, Torralba A, (2007) The role of context in object recognition. Trends in Cognitive Sciences 11: 520–527.
39. Pinto N, Majaj N, Barhomi Y, Solomon E, DiCarlo JJ (2010) Human versus machine: comparing visual object recognition systems on a level playing field. Cosyne Abstracts 2010, Salt Lake City USA.
40. Keerthi SS, Lin CJ, (2003) Asymptotic behaviors of support vector machines with Gaussian kernel. Neural Computation 15: 1667–1689.
41. Mutch J, Lowe DG, (2008) Object Class Recognition and Localization Using Sparse Features with Limited Receptive Fields. International Journal of Computer Vision 80: 45–57.
42. Freeman J, Simoncelli EP, (2011) Metamers of the ventral stream. Nature Neuroscience 14: 1195–1201.
43. Cadieu CF, Hong H, Yamins D, Pinto N, Majaj NJ, et al. (2013) The Neural Representation Benchmark and its Evaluation on Brain and Machine. In: International Conference on Learning Representations. pp.1–16.
44. Tolhurst DJ, Movshon JA, Dean AF (1983) The statistical reliability of signals in single neurons in cat and monkey visual cortex. Vision research.
45. Shadlen MN, Newsome WT (1998) The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding. Journal of Neuroscience: 3870–3896.
46. Evgeniou T, Pontil M, Poggio T, (2000) Regularization Networks and Support Vector Machines. Advances in Computational Mathematics 13: 1–50.
47. Endres D, Földiák P, (2007) Bayesian binning for maximising information rate of rapid serial presentation for sensory neurons. BMC Neuroscience 8: P151.
48. Vogels R, Sáry G, Orban GA, (1995) How task-related are the responses of inferior temporal neurons? Visual Neuroscience 12: 207–214.
49. Koida K, Komatsu H, (2006) Effects of task demands on the responses of color-selective neurons in the inferior temporal cortex. Nature Neuroscience 10: 108–116.
50. Suzuki W, Matsumoto K, Tanaka K, (2006) Neuronal Responses to Object Images in the Macaque Inferotemporal Cortex at Different Stimulus Discrimination Levels. Journal of Neuroscience 26: 10524–10535.
51. Op de Beeck HP, Baker CI, (2010) Informativeness and learning: Response to Gauthier and colleagues. Trends in Cognitive Sciences 14: 236–237.
52. Kobatake E, Wang G, Tanaka K, (1998) Effects of shape-discrimination training on the selectivity of inferotemporal cells in adult monkeys. Journal of Neurophysiology 80: 324–330.
53. Baker CI, Behrmann M, Olson CR, (2002) Impact of learning on representation of parts and wholes in monkey inferotemporal cortex. Nature Neuroscience 5: 1210–1216.
54. Sigala N, Logothetis NK, (2002) Visual categorization shapes feature selectivity in the primate temporal cortex. Nature 415: 318–320.
55. Li N, DiCarlo JJ, (2010) Unsupervised Natural Visual Experience Rapidly Reshapes Size-Invariant Object Representation in Inferior Temporal Cortex. Neuron 67: 1062–1075.
56. Stevenson IH, London BM, Oby ER, Sachs NA, (2012) Functional Connectivity and Tuning Curves in Populations of Simultaneously Recorded Neurons. PLoS Computational Biology 8: e1002775.
57. LeCun Y, Bottou L, Bengio Y, Haffner P, (1998) Gradient-based learning applied to document recognition. Proceedings of the IEEE 86: 2278–2324.
58. Rumelhart DE, Hinton GE, Williams RJ, (1986) Learning representations by back-propagating errors. Nature 323: 533–536.
59. Mallat S, (2012) Group Invariant Scattering. Communications on Pure and Applied Mathematics 65: 1331–1398.
60. Majaj N, Hong H, Solomon E, DiCarlo JJ (2012) A unified neuronal population code fully explains human object recognition. Cosyne Abstracts 2012, Salt Lake City USA.
61. Churchland MM, Cunningham JP, Kaufman MT, Foster JD, Nuyujukian P, et al. (2012) Neural population dynamics during reaching. Nature 487: 51–56.
62. Canolty RT, Ganguly K, Kennerley SW, Cadieu CF, Koepsell K, et al. (2010) Oscillatory phase coupling coordinates anatomically dispersed functional cell assemblies. Proceedings of the National Academy of Sciences 107: 17356–17361.
63. Frey BJ, Dueck D, (2007) Clustering by Passing Messages Between Data Points. Science 315: 972–976.
64. Quiroga RQ, Nadasdy Z, Ben-Shaul Y, (2004) Unsupervised Spike Detection and Sorting with Wavelets and Superparamagnetic Clustering. Neural Computation 16: 1661–1687.
65. Rifkin RM, Lippert RA, (2007) Notes on Regularized Least Squares. MIT-CSAIL Technical Report 2007-025: 1–8.
66. Rasmussen CE, Williams CKI (2006) Gaussian Processes for Machine Learning. MIT Press.
67. Smola AJ, Schölkopf B, Müller KR, (1998) The connection between regularization operators and support vector kernels. Neural Networks 11: 637–649.
68. Montavon G, Müller KR (2012) Deep Boltzmann Machines and the Centering Trick. Neural Networks: Tricks of the Trade: 621–637.
69. Wu MC, David SV, Gallant JL, (2006) Complete functional characterization of sensory neurons by system identification. Annual review of neuroscience 29: 477.