Sparse Coding Models Can Exhibit Decreasing Sparseness while Learning Sparse Codes for Natural Images

PLoS Computational Biology

Volumn 9, Issue 8, 2013, Pages

  Zylberberg, Joel ^a   DeWeese, Michael Robert ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CODES (SYMBOLS); IMAGE CODING; NEURONS;

CODING MODELS; NATURAL IMAGES; NATURAL SCENES; NEURON ACTIVITY; PRIMARY VISUAL CORTEX; RECEPTIVE FIELDS; RESPONSE PROPERTIES; SIMPLE CELL; SPARSE CODES; SPARSE CODING;

PHYSIOLOGICAL MODELS;

ARTICLE; CONTROLLED STUDY; HOMEOSTASIS; IMAGE PROCESSING; LEARNING; MACACA; MATHEMATICAL MODEL; NERVE CELL NETWORK; NERVE FUNCTION; NEUROTRANSMISSION; NONHUMAN; NONPARAMETRIC TEST; RECEPTIVE FIELD; SPARSE CODING MODEL; THEORETICAL MODEL; VISUAL CORTEX;

MUSTELA;

EID: 84883360856     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1003182     Document Type: Article

References (34)

1. Rehn M, Sommer FT, (2007) A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields. J Comput Neurosci 22: 135-146.
2. Simoncelli E, Olshausen B, (2001) Natural image statistics and neural representation. Annu Rev Neurosci 24: 1193-1216.
3. Olshausen BA, Field DJ, (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381: 607-609.
4. Vinje WE, Gallant JL, (2002) Natural stimulation of the nonclassical receptive field increases information transmission efficiency in V1. J Neurosci 22: 2904-2915.
5. Vinje WE, Gallant JL, (2000) Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287: 1273-1276.
6. Haider BA, Krause MR, Duque A, Yu Y, Touryan J, et al. (2010) Synaptic and network mechanisms of sparse and reliable visual cortical activity during nonclassical receptive field stimulation. Neuron 65: 107-121.
7. Baddeley R, Abbott LF, Booth MCA, Sengpiel F, Freeman T, et al. (1997) Responses of neurons in primary and inferior temporal visual cortices. Proc R Soc Lon B 264: 1775-1783.
8. Olshausen BA, Cadieu CF, Warland DK, (2009) Learning real and complex overcomplete representations from the statistics of natural images. Proc SPIE 7446: 74460S-1-74460S-11.
9. Willmore B, Tolhurst D, (2001) Characterizing the sparseness of neural codes. Network: Comput Neural Syst 12: 255-270.
10. Hromádka T, DeWeese MR, Zador AM, (2008) Sparse representation of sounds in the unanesthetized auditory cortex. PLoS Biol 6: 124-137.
11. Tolhurst DJ, Smyth D, Thompson ID, (2009) The sparseness of neuronal responses in ferret primary visual cortex. J Neurosci 29: 2355-2370.
12. Zylberberg J, Murphy JT, DeWeese MR, (2011) A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of V1 simple cell receptive fields. PLoS Comp Biol 7: e1002250-1-e1002250-12.
13. Bell AJ, Sejnowski TJ, (1997) The "independent components" of natural scenes are edge filters. Vision Res 37: 3327-3328.
14. Berkes P, White BL, Fiser J (2009) No evidence for active sparsification in the visual cortex. In: Bengio Y, Schuurmans D, Lafferty JD, Williams CKI, Culotta A, editors. Advances in Neural Information Processing Systems 22, San Mateo: Morgan Kaufmann. pp. 108-116.
15. Berkes P, Orbán G, Lengyel M, Fiser J, (2011) Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science 331: 83-87.
16. King P, Zylberberg J, DeWeese M, (2013) Inhibitory interneurons decorrelate excitatory cells to drive sparse code formation in a spiking model of v1. J Neurosci 33: 5475-5485.
17. Abbott L, Nelson S, (2000) Synaptic plasticity: taming the beast. Nat Neurosci 3: 1178-1183.
18. Marder E, Goaillard J, (2006) Variability, compensation and homeostasis in neuron and network function. Nat Rev Neurosci 7: 563-574.
19. Turrigiano G, (2011) Too many cooks? intrinsic and synaptic homeostatic mechanisms in cortical circuit refinement. Annu Rev Neurosci 34: 89-103.
20. Treves A, Rolls ET, (1991) What determines the capacity of autoassociative memories in the brain? Network: Comput Neural Syst 2: 371-397.
21. Olshausen BA, Field DJ, (1997) Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Res 37: 3311-3325.
22. Carlson NL, Ming VL, DeWeese MR, (2012) Sparse codes for speech predict spectrotemporal receptive fields in the inferior colliculus. PLoS Comput Biol 8: e1002594.
23. Puerta G, Bornschein J, Lücke J (2010) The maximal causes of natural scenes are edge filters. In: Lafferty J, Williams CKI, Shawe-Taylor J, Zemel R, Culotta A, editors. Advances in Neural Information Processing 23, San Mateo: Morgan Kaufmann. pp. 1939-1947.
24. White L, Fitzpatrick D, (2007) Vision and cortical map development. Neuron 56: 327-338.
25. Li Y, Fitzpatrick D, White L, (2006) The development of direction selectivity in ferret visual cortex depends on early visual experience. Nat Neurosci 9: 676-681.
26. Imbert M, Buisseret P, (1975) Receptive field characteristics and plastic properties of visual cortical cells in kittens reared with or without visual experience. Exp Brain Res 22: 25-36.
27. Fagiolini M, Pizzorusso T, Berardi N, Domenici L, Maffei L, (1994) Functional postnatal development of the rat primary visual cortex and the role of visual experience: Dark rearing and monocular deprivation. Vision Res 34: 709-720.
28. Cynader M, Berman N, Hein A, (1973) Cats reared in stroboscopic illumination: effects on receptive fields in visual cortex. Proc Natl Acad Sci USA 70: 1353-1354.
29. Rochefort NL, Garaschuk O, Milos R, Narushima M, Marandi N, et al. (2009) Sparsification of neuronal activity in the visual cortex at eye-opening. Proc Natl Acad Sci USA 106: 15049-15054.
30. Perrinet L, (2010) Role of homeostasis in learning sparse representations. Neural Comput 22: 1812-1836.
31. Miller K, Mackay D, (1994) The role of constraints in hebbian learning. Neural Comput 6: 100-126.
32. Sullivan T, de Sa V, (2006) Homeostatic synaptic scaling in self-organizing maps. Neural Netw 19: 734-743.
33. MATLAB (2012) version R2012a. Natick, Massachusetts: The MathWorks Inc.
34. Ringach D, (2002) Spatial structure and asymmetry of simple-cell receptive fields in macaque primary visual cortex. J Neurophysiol 88: 455-463.