Natural image sequences constrain dynamic receptive fields and imply a sparse code

Brain Research

Volumn 1536, Issue , 2013, Pages 53-67

  Häusler, Chris ^a,c   Susemihl, Alex ^b,c   Nawrot, Martin P ^a,c  

^a FREIE UNIVERSITÄT BERLIN   (Germany)

^b TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

^c BERNSTEIN CENTER FOR COMPUTATIONAL NEUROSCIENCE   (Germany)

Author keywords

Autoencoding; Lifetime sparseness; Machine learning; Population sparseness; Restricted Boltzmann Machine; Visual cortex

Indexed keywords

ALGORITHM; ANIMAL STRUCTURES; ARTIFICIAL NEURAL NETWORK; CONFERENCE PAPER; NERVE CELL; NERVE CELL NETWORK; NONHUMAN; PRIORITY JOURNAL; PROCESS MODEL; RECEPTIVE FIELD; RESTRICTED BOLTZMANN MACHINE; SPATIOTEMPORAL ANALYSIS; STOCHASTIC MODEL; STRIATE CORTEX; TEMPORAL AUTOENCODING; VISUAL STIMULATION;

ANIMALIA; MAMMALIA;

AUTOENCODING; LIFETIME SPARSENESS; MACHINE LEARNING; POPULATION SPARSENESS; RESTRICTED BOLTZMANN MACHINE; VISUAL CORTEX;

NEURAL NETWORKS (COMPUTER); NEURONS; TIME FACTORS; VISUAL CORTEX; VISUAL FIELDS;

EID: 84887159638     PISSN: 00068993     EISSN: 18726240     Source Type: Journal    
DOI: 10.1016/j.brainres.2013.07.056     Document Type: Conference Paper

References (90)

1. C. Assisi, M. Stopfer, G. Laurent, and M. Bazhenov Adaptive regulation of sparseness by feedforward inhibition Nat. Neurosci. 10 9 2007 1176 1184
2. A.J. Bell, and T.J. Sejnowski The independent components of natural scenes are edge filters Vision Res. 37 23 1997 3327 3338
3. J. Benda, and A.V. Herz A universal model for spike-frequency adaptation Neural Comput. 15 11 2003 2523 2564
4. Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle Greedy layer-wise training of deep networks Adv. Neural Inf. Process. Syst. 19 2007 153
5. Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., Bengio, Y., 2010. Theano: a CPU and GPU math expression compiler. In: Proceedings of the Python for Scientific Computing Conference (SciPy). Oral Presentation.
6. W. Bosking, Y. Zhang, B. Schofield, and D. Fitzpatrick Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex J. Neurosci. 17 6 1997 2112 2127
7. M.F. Brill, T. Rosenbaum, I. Reus, C.J. Kleineidam, M.P. Nawrot, and W. Rössler Parallel processing via a dual olfactory pathway in the honeybee J. Neurosci. 33 6 2013 2443 2456
8. C. Cadieu, and B. Olshausen Learning intermediate-level representations of form and motion from natural movies Neural Comput. 2012 1 40
9. Cadieu, C.F., Olshausen, B.A., 2008. Learning transformational invariants from natural movies. In: Advances in Neural Information Processing Systems 21, pp. 1-8.
10. N.L. Carlson, V.L. Ming, and M.R. DeWeese Sparse codes for speech predict spectrotemporal receptive fields in the inferior colliculus PLoS Comput. Biol. 8 7 2012 e1002594
11. Caron, S.J., Ruta, V., Abbott, L., Axel, R., 2013. Random convergence of olfactory inputs in the drosophila mushroom body. Nature 497, 113-117.
12. Carreira-Perpinan, M., Hinton, G., 2005. On contrastive divergence learning. In: Artificial Intelligence and Statistics, vol. 2005, p. 17.
13. S. Cassenaer, and G. Laurent Conditional modulation of spike-timing-dependent plasticity for olfactory learning Nature 482 7383 2012 47 52
14. M.J. Chacron, A. Longtin, and L. Maler Negative interspike interval correlations increase the neuronal capacity for encoding time-dependent stimuli J. Neurosci. 21 14 2001 5328 5343
15. M.J. Chacron, L. Maler, and J. Bastian Electroreceptor neuron dynamics shape information transmission Nat. Neurosci. 8 5 2005 673 678
16. C. Chen, H. Read, and M. Escabí Precise feature based time scales and frequency decorrelation lead to a sparse auditory code J. Neurosci. 32 25 2012 8454 8468
17. D. Coppola, L. White, D. Fitzpatrick, and D. Purves Unequal representation of cardinal and oblique contours in ferret visual cortex Proc. Natl. Acad. Sci. 95 5 1998 2621 2623
18. D. Crisan, and B. Rozovskii The Oxford Handbook of Nonlinear Filtering 2011 Oxford University Press Oxford
19. Y. Dan, J. Atick, and R. Reid Efficient coding of natural scenes in the lateral geniculate nucleus: experimental test of a computational theory J. Neurosci. 16 10 1996 3351 3362
20. Farkhooi, F., Froese, A., Muller, E., Menzel, R., Nawrot, M.P., 2012. Cellular adaptation accounts for the sparse and reliable sensory stimulus representation. http://arxiv.org/abs/1210.7165.
21. F. Farkhooi, E. Muller, and M.P. Nawrot Adaptation reduces variability of the neuronal population code Phys. Rev. E 83 5 2011 050905
22. F. Farkhooi, M.F. Strube-Bloss, and M.P. Nawrot Serial correlation in neural spike trains experimental evidence, stochastic modeling, and single neuron variability Phys. Rev. E 79 2 2009 021905
23. D. Field, and A. Hayes Contour integration and the lateral connections of v1 neurons Visual Neurosci. 2 2004 1069 1079
24. R.W. Friedrich, and G. Laurent Dynamics of olfactory bulb input and output activity during odor stimulation in zebrafish J. Neurophysiol. 91 6 2004 2658 2669
25. A.P. Georgopoulos, J.F. Kalaska, R. Caminiti, and J.T. Massey On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex J. Neurosci. 2 11 1982 1527 1537
26. B. Haider, M.R. Krause, A. Duque, Y. Yu, J. Touryan, J.A. Mazer, and D.A. McCormick Synaptic and network mechanisms of sparse and reliable visual cortical activity during nonclassical receptive field stimulation Neuron 65 1 2010 107
27. R. Herikstad, J. Baker, J.-P. Lachaux, C.M. Gray, and S.-C. Yen Natural movies evoke spike trains with low spike time variability in cat primary visual cortex J. Neurosci. 31 44 2011 15844 15860
28. A.V. Herz, T. Gollisch, C.K. Machens, and D. Jaeger Modeling single-neuron dynamics and computations a balance of detail and abstraction Science 314 5796 2006 80 85
29. G. Hinton A practical guide to training restricted Boltzmann machines Momentum 9 2010 1
30. G. Hinton, and R. Salakhutdinov Reducing the dimensionality of data with neural networks Science 313 5786 2006 504 507
31. Hinton, G., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R., 2012. Improving neural networks by preventing co-adaptation of feature detectors. Arxiv preprint arXiv:1207.0580.
32. K.S. Honegger, R.A. Campbell, and G.C. Turner Cellular-resolution population imaging reveals robust sparse coding in the Drosophila mushroom body J. Neurosci. 31 33 2011 11772 11785
33. T. Hromádka, M. DeWeese, and A. Zador Sparse representation of sounds in the unanesthetized auditory cortex PLoS Biol. 6 1 2008 e16
34. Huerta, R., 2013. Learning pattern recognition and decision making in the insect brain. In: American Institute of Physics Conference Series, vol. 1510, pp. 101-119.
35. R. Huerta, and T. Nowotny Fast and robust learning by reinforcement signals: explorations in the insect brain Neural Comput. 21 8 2009 2123 2151
36. R. Huerta, T. Nowotny, M. García-Sanchez, H. Abarbanel, and M. Rabinovich Learning classification in the olfactory system of insects Neural Comput. 16 8 2004 1601 1640
37. I. Ito, R.C.-y. Ong, B. Raman, and M. Stopfer Sparse odor representation and olfactory learning Nat. Neurosci. 11 10 2008 1177 1184
38. S. Jadhav, J. Wolfe, and D. Feldman Sparse temporal coding of elementary tactile features during active whisker sensation Nat. Neurosci. 12 6 2009 792 800
39. R.A. Jortner, S.S. Farivar, and G. Laurent A simple connectivity scheme for sparse coding in an olfactory system J. Neurosci. 27 7 2007 1659 1669
40. A. Knoblauch, and G. Palm Pattern separation and synchronization in spiking associative memories and visual areas Neural Networks Off. J. Int. Neural Network Soc. 14 6-7 2001 763
41. S. Krofczik, R. Menzel, and M.P. Nawrot Rapid odor processing in the honeybee antennal lobe network Front. Comput. Neurosci. 2008 2
42. Larochelle, H., Bengio, Y., 2008. Classification using discriminative restricted Boltzmann machines. In: Proceedings of the 25th International Conference on Machine Learning, ACM, pp. 536-543.
43. S.B. Laughlin Energy as a constraint on the coding and processing of sensory information Curr. Opin. Neurobiol. 11 4 2001 475 480
44. Le, Q., Karpenko, A., Ngiam, J., Ng, A., 2011. Ica with reconstruction cost for efficient overcomplete feature learning. In: NIPS.
45. H. Lee, C. Ekanadham, and A. Ng Sparse deep belief net model for visual area v2 Adv. Neural Inf. Process. Syst. 20 2008 873 880
46. H. Lee, Y. Largman, P. Pham, and A. Ng Unsupervised feature learning for audio classification using convolutional deep belief networks Adv. Neural Inf. Process. Syst. 22 2009 1096 1104
47. P. Lennie The cost of cortical computation Curr. Biol. 13 6 2003 493 497
48. W.B. Levy, and R.A. Baxter Energy efficient neural codes Neural Comput. 8 3 1996 531 543
49. P. Maldonado, C. Babul, W. Singer, E. Rodriguez, D. Berger, and S. Grün Synchronization of neuronal responses in primary visual cortex of monkeys viewing natural images J. Neurophysiol. 100 3 2008 1523 1532
50. Marszalek, M., Laptev, I., Schmid, C., 2009. Actions in context. In: IEEE Conference on Computer Vision and Pattern Recognition.
51. K.A. Martin, and S. Schröder Functional heterogeneity in neighboring neurons of cat primary visual cortex in response to both artificial and natural stimuli J. Neurosci. 33 17 2013 7325 7344
52. Mohamed, A., Sainath, T., Dahl, G., Ramabhadran, B., Hinton, G., Picheny, M., 2011. Deep belief networks using discriminative features for phone recognition. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, pp. 5060-5063.
53. M. Nawrot, A. Aertsen, and S. Rotter Single-trial estimation of neuronal firing rates: from single-neuron spike trains to population activity J. Neurosci. Methods 94 1 1999 81 92
54. Nawrot, M.P., 2010. Analysis and interpretation of interval and count variability in neural spike trains. In: Analysis of Parallel Spike Trains, Springer, pp. 37-58.
55. M.P. Nawrot Dynamics of sensory processing in the dual olfactory pathway of the honeybee Apidologie 43 3 2012 269 291
56. M.P. Nawrot, C. Boucsein, V. Rodriguez-Molina, A. Aertsen, S. Grün, and S. Rotter Serial interval statistics of spontaneous activity in cortical neurons in vivo and in vitro Neurocomputing 70 10 2007 1717 1722
57. M.P. Nawrot, C. Boucsein, V. Rodriguez Molina, A. Riehle, A. Aertsen, and S. Rotter Measurement of variability dynamics in cortical spike trains J. Neurosci. Methods 169 2 2008 374 390
58. B. Olshausen Emergence of simple-cell receptive field properties by learning a sparse code for natural images Nature 381 6583 1996 607 609
59. B.A. Olshausen, and D.J. Field Sparse coding of sensory inputs Curr. Opin. Neurobiol. 14 4 2004 481 487
60. G. Palm On associative memory Biol. Cybern. 36 1 1980 19 31
61. M. Papadopoulou, S. Cassenaer, T. Nowotny, and G. Laurent Normalization for sparse encoding of odors by a wide-field interneuron Science 332 6030 2011 721 725
62. J. Perez-Orive, O. Mazor, G.C. Turner, S. Cassenaer, R.I. Wilson, and G. Laurent Oscillations and sparsening of odor representations in the mushroom body Science 297 5580 2002 359 365
63. D.H. Perkel, G.L. Gerstein, and G.P. Moore Neuronal spike trains and stochastic point processes I. the single spike train Biophys. J. 7 4 1967 391 418
64. T. Pfeil, A. Grübl, S. Jeltsch, E. Müller, P. Müller, M.A. Petrovici, M. Schmuker, D. Brüderle, J. Schemmel, and K. Meier Six networks on a universal neuromorphic computing substrate Front. Neurosci. 2013 7
65. C. Poo, and J.S. Isaacson Odor representations in olfactory cortex sparse coding, global inhibition and oscillations Neuron 62 6 2009 850
66. D.S. Reich, F. Mechler, and J.D. Victor Independent and redundant information in nearby cortical neurons Science 294 5551 2001 2566 2568
67. P. Reinagel How do visual neurons respond in the real world? Curr. Opin. Neurobiol. 11 4 2001 437 442
68. P. Reinagel, and R. Reid Precise firing events are conserved across neurons J. Neurosci. 22 16 2002 6837 6841
69. J. Rickert, A. Riehle, A. Aertsen, S. Rotter, and M.P. Nawrot Dynamic encoding of movement direction in motor cortical neurons J. Neurosci. 29 44 2009 13870 13882
70. A. Riehle, S. Grün, M. Diesmann, and A. Aertsen Spike synchronization and rate modulation differentially involved in motor cortical function Science 278 5345 1997 1950 1953
71. Saxe, A., Bhand, M., Mudur, R., Suresh, B., Ng, A., 2011. Unsupervised learning models of primary cortical receptive fields and receptive field plasticity. In: Advances in Neural Information Processing Systems.
72. H. Shimazaki, and S. Shinomoto A method for selecting the bin size of a time histogram Neural Comput. 19 6 2007 1503 1527
73. E. Simoncelli, and B. Olshausen Natural image statistics and neural representation Annu. Rev. Neurosci. 24 1 2001 1193 1216
74. M.F. Strube-Bloss, M.P. Nawrot, and R. Menzel Mushroom body output neurons encode odor-reward associations J. Neurosci. 31 8 2011 3129 3140
75. Sutskever, I., Hinton, G., 2007. Learning multilevel distributed representations for high-dimensional sequences. In: Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics, pp. 544-551.
76. I. Sutskever, G. Hinton, and G. Taylor The recurrent temporal restricted Boltzmann machine Adv. Neural Inf. Process. Syst. 2008 21
77. Taylor, G., 2009. Composable, Distributed-state Models for High-dimensional Time Series. Ph.D. Thesis.
78. G. Taylor, G. Hinton, and S. Roweis Modeling human motion using binary latent variables Adv. Neural Inf. Process. Syst. 19 2007 1345
79. Tuckwell, H.C., 2005. Introduction to Theoretical Neurobiology: vol. 2, Nonlinear and Stochastic Theories, vol. 8. Cambridge University Press.
80. J.H. van Hateren, and D.L. Ruderman Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex Proc. R. Soc. London Ser. B Biol. Sci. 265 1412 1998 2315 2320
81. S. Van Hooser Similarity and diversity in visual cortex is there a unifying theory of cortical computation? The Neuroscientist 13 6 2007 639 656
82. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P. Manzagol Stacked denoising autoencoders learning useful representations in a deep network with a local denoising criterion J. Mach. Learn. Res. 11 2010 3371 3408
83. W. Vinje, and J. Gallant Sparse coding and decorrelation in primary visual cortex during natural vision Science 287 5456 2000 1273 1276
84. G. Wang, S. Ding, and K. Yunokuchi Difference in the representation of cardinal and oblique contours in cat visual cortex Neurosci. Lett. 338 1 2003 77 81
85. B. Willmore, and D.J. Tolhurst Characterizing the sparseness of neural codes Network Comput. Neural Syst. 12 3 2001 255 270
86. B.D. Willmore, J.A. Mazer, and J.L. Gallant Sparse coding in striate and extrastriate visual cortex J. Neurophysiol. 105 6 2011 2907 2919
87. R.I. Wilson, G.C. Turner, and G. Laurent Transformation of olfactory representations in the Drosophila antennal lobe Sci. Signal. 303 5656 2004 366
88. J. Wolfe, A. Houweling, and M. Brecht Sparse and powerful cortical spikes Curr. Opin. Neurobiol. 20 3 2010 306
89. S. Yen, J. Baker, and C. Gray Heterogeneity in the responses of adjacent neurons to natural stimuli in cat striate cortex J. Neurophysiol. 97 2 2007 1326 1341
90. C. Zetzsche, and U. Nuding Nonlinear and higher-order approaches to the encoding of natural scenes Network Comput. Neural Syst. 16 2-3 2005 191 221