Stable learning of functional maps in self-organizing spiking neural networks with continuous synaptic plasticity

Frontiers in Computational Neuroscience

Volumn , Issue FEB, 2013, Pages

  Srinivasa, Narayan ^a   Jiang, Qin ^a  

^a Hughes Aircraft Co   (United States)

Author keywords

Functional maps; Learning; Ocular dominance; Orientation selectivity; Spiking networks; Stability; STDP

Indexed keywords

LEARNING; OCULAR DOMINANCE; ORIENTATION SELECTIVITY; SPIKING NETWORKS; STDP;

CONVERGENCE OF NUMERICAL METHODS; MAINTENANCE; NEURAL NETWORKS; NEURONS;

PLASTICITY;

ARTICLE; EXCITATORY POSTSYNAPTIC POTENTIAL; EYE DOMINANCE; FUNCTIONAL MAP; INHIBITORY POSTSYNAPTIC POTENTIAL; LATERAL GENICULATE BODY; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NEUROMODULATION; NEUROSCIENCE; OCULAR DOMINANCE MAP; ORIENTATION SELECTIVITY MAP; RETINA GANGLION CELL; SPIKE TIMING DEPENDENT PLASTICITY; STRIATE CORTEX; VISUAL CORTEX; VISUAL ORIENTATION;

EID: 84873896514     PISSN: 16625188     EISSN: None     Source Type: Journal    
DOI: 10.3389/fncom.2013.00010     Document Type: Article

References (125)

1. Antonini, A. and Stryker, M. P. (1978). Plasticity of geniculocortical afferents following brief or prolonged monocular occlusion in the cat. Journal of Computational Neurology 369, 64-82.
2. Bartsch, A. P., and van Hemmen, J. L. (2000). Combined Hebbian development of geniculocortical and lateral connectivity in a model of primary visual cortex. Biological Cybernetics 84, 41-55.
3. Basole, A., White, L. E., and Fitzpatrick, D. (2003). Mapping multiple stimulus features in the population response of visual cortical neurons. Nature 423, 986-990.
4. Basole, A., Kraft-Kerekes, V., White, L. E., and Fitzpatrick, D. (2006). Cortical cartography revisited: a frequency perspective on the functional architecture of visual cortex. Progress in Brain Research 154, 121-134.
5. Bauer, C., Burger, T., Stetter, M., and Lang, E. W. (2000). A neural network model for the selforganization of cortical grating cells. Z. Naturforsch 55c, 282-291.
6. Bednar, J., and Miikkulainen, R. (2004). Prenatal and postnatal development of laterally connected orientation maps. Neurocomputing 58-60, 985-992.
7. Bi, Q. Q. and Poo, M. M. (1998). Activity-induced synaptic modification in hippocampal culture, dependence on spike timing, synaptic strength and cell type, Journal of Neuroscience 18, 10464-10472.
8. Billings, G., and van Rossum, M. C. W. (2009). Memory retention and spike-timing dependent plasticity. Journal of Neurophysiology 101, 2755-2788.
9. Binzegger, T., Douglas, R. J., and Martin, K. A. C. (2004). A quantitative map of the circuit of the cat primary visual cortex. Journal of Neuroscience 39, 8441-8453.
10. Blakemore, C. and Cooper, G. F. (1970). Development of brain depends on visual environment. Nature 228, 477-478.
11. Blasdel, G. G. (1992). Orientation selectivity, preference, and continuity in monkey striate cortex. Journal of Neuroscience 12(8), 3139-13161.
12. Buonamano, D., and Merzenich, M. (1998). Cortical plasticity: from synapses to maps. Annual Review of Neuroscience 21, 149-156.
13. Buchs, N. J., and Senn, W. (2002). Spike-based synaptic plasticity and the emergence of direction selective simple cells: simulation results. Journal of Computational Neuroscience 13, 167-186.
14. Butts, D. (2002). Retinal Waves: Implications for synaptic learning rules during development. Neuroscientist 8, 243-253.
15. Caporale, N., and Dan, Y. (2008). Spike timing-dependent plasticity: a Hebbian learning rule. Annual Review Neuroscience 31, 25-46.
16. Carver, S., Roth E., Cowan, N. J., and Fortune, E. S. (2008). Synaptic plasticity can produce and enhance direction selectivity. PLoS Computational Biology 4, e32, doi:10.1371/journal.pcbi.0040032.
17. Cassenaer, S., and Laurent, G. (2007). Hebbian STDP in mushroom bodies facilitates the synchronous flow of olfactory information in locusts. Nature 448(7154), 709-713.
18. Chapman, B., Stryler, M. P. and Bonhoeffer, T. (1996). Development of orientation-preference maps in ferret primary visual cortex. Journal of Neuroscience 16, 6443-6453.
19. Chen, C., and Regehr, W. G. (2000). Developmental remodeling of the retinogeniculate synapse. Neuron 28, 955-966.
20. Chiu, C., and Weliky, M. (2001). Spontaneous activity in developing ferret visual cortex in vivo. Journal of Neuroscience 21, 8906-8914.
21. Coppola, D. M., and White, L. E. (2004). Visual experience promotes the isotropic representation of orientation preference. Visual Neuroscience 21, 39-51.
22. Crair, M., Ruthazer, E., Gillespie, D., and Stryker, M. (1997). Relationship between the ocular dominance and orientation maps in visual cortex of monocularly deprived cats. Neuron 19, 307-318.
23. Crair, M., Gillespie, D., and Stryker, M. (1998). The role of visual experience in the development of columns in cat visual cortex. Science 279, 565-570.
24. Crowley, J. C., and Katz, L. C. (1999). Development of ocular dominance columns in the absence of retinal input. Nature Neuroscience 2, 1125-1130.
25. Crowley, J. C. and Katz. L. C. (2002). Ocular dominance development revisited. Current Opinion of Neurobiology 12, 104-109.
26. Debbane, D., Gahwiler, B. H., and Thompson, S. (1998). Long-term synaptic plasticity between pairs of individual CA3 pyramidal cells in rat hippocampal slice cultures. Journal of Physiology 507, 237-247.
27. Feller, M. B. (2009). Retinal waves are likely to instruct the formation of eye-specific retinogeniculate projections. Neural Development 4, 24-28.
28. Fei, L., Fergus, R., and Perona, P. (2006). One-shot learning of object categories. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 594-611.
29. Ferster, D., and Miller, K. D. (2000). Neural mechanisms of orientation selectivity in the visual cortex. Annual Review Neuroscience 23, 441-471.
30. Finelli, L. A., Haney, S., Bazhenov, M., Stopfer, M., and Sejnowski, T. J. (2008). Synaptic learning rules and sparse coding in a model sensory system. PloS Comput. Biol., 4(4), e1000062.
31. Fiser, J., Chiu, C., and Weliky, M. (2004). Small modulation of ongoing cortical dynamics of sensory input during natural vision. Nature 431, 573-578.
32. Froemke, R. C., Letzkus, J. J., Kampa, B. M., Hang, G. B., Stuart, G. J. (2010). Dendritic synapse location and neocortical spike-timing dependent plasticity. Frontiers in Synaptic Neuroscience 2:29, doi:10.3389/fnsyn.2010.00029.
33. Frund, I., Ohl, F. W., and Hermann, C. S. (2009). Spike-timing-dependent plasticity leads to gamma band responses in a neural network. Biological Cybernetics 101, 227-240.
34. Goard, M., and Dan, Y. (2009). Basal forebrain activation enhances cortical coding of natural scenes. Nature Neuroscience 12, 1444-1449.
35. Godfrey, K. B., and Swindale, N. V. (2007). Retinal wave behavior through activity-dependent refractory periods. PLoS Comput. Biol 3(11): e245. Doi:10.137/journal.pcbi.0030245.
36. Hartmann, K., Bruehl, C., Golovko, T., and Draguhn, A. (2008). Fast homeostatic plasticity of inhibition via activity-dependent vesicular filling. PloS One 3, 2979-2987.
37. Hensch, T. K. (2005). Critical period plasticity in local cortical circuits. Nature Neuroscience 6, 877-888.
38. Hodgkin A. L., and Huxley, A. F. (1952). A quantitative description of membrane current and application to conduction and excitation in nerve. Journal of Physiology 117, 500-544.
39. Hubel, D., and Wiesel, T. N. (1962). Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. Journal of Physiology 160, 106-154.
40. Hubel, D. H., and Wiesel, T. N. (1963). Receptive fields of striate cortex of very young, visually inexperienced kittens. Journal of Neurophysiology 26, 994-1002.
41. Hubel, D. H., and Wiesel, T. N. (1968). Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology 195, 215-243.
42. Hubel, D., and Wiesel, T. N, (2005). Brain and Visual Perception. New York: Oxford University Press.
43. Huberman, A. D., Speer, C. M., and Chapman, B. (2006). Spontaneous retinal activity mediates development of ocular dominance columns and binocular receptive fields in V1. Neuron 52, 247-254.
44. Huberman, A. D., Feller, M. B., and Chapman, B. (2008). Mechanisms underlying development of visual maps and receptive fields. Annual Review Neuroscience 31, 479-509.
45. Izhikevich, E. (2004). Which model to use for cortical spiking neurons. IEEE Transactions on Neural Networks 15, 1063-1070.
46. Jacob, V., Brasier, D. J., Erchova, I., Feldman, D., and Shulz, D. E. (2007). Spike timingdependent synaptic depression in the in vivo barrel cortex of the rat. J. Neurosci., 27(6), 1271-1284.
47. Jutras, M. J., and Buffalo, E. A. (2010). Synchronous neural activity and memory formation. Current Opinion in Neurobiology 20, 1-6.
48. Kang, K., Shelley, M., and Sompolinsky, H. (2003). Mexican hats and pinwheels in visual cortex, Proceedings of National Academy of Sciences 100(5), 2848-2853.
49. Katz, L. C., and Crowley, J. C. (2002). Development of cortical circuits: Lessons from ocular dominance columns. Nature Review Neuroscience 3, 34-42.
50. Katz, L., and Shatz, C. (1996). Synaptic activity and the construction of cortical circuits, Science 274, 1133-1138.
51. Krelle A. K., Bonhoeffer, T., and Hubener, M. (2011). Altered visual experience induces instructive changes of orientation preference in mouse visual cortex. The Journal of Neuroscience, 31(39), 13911-13920.
52. Koehler, C. L., Akimov, N. P., and Renteria, R. C. (2011). Receptive field center size decreases and firing properties mature in ON and OFF retinal ganglion cells after eye opening in the mouse. J. Neurophysiol. 106(2), 895-904.
53. Kremkow, J., Kumar, A., Rotter, S., and Aertsen, A. (2007). Emergence of population synchrony in a layered network of the cat visual cortex. Neurocomputing 70, 2069-2073.
54. Kumar, A., Schraeder. S., and Aertsen, A. (2008). High conductance state of cortical networks. Neural Computation 20, 1-43.
55. Kisvarday, Z. F., Buzas, P., and Eysel, U. T. (2001). Calculating direction maps from intrinsic signals revealed by optical imaging. Cerebral Cortex 11, 636-647.
56. Kullback, S. (1987). The kullback-leibler distance. The American Statistician 41(4):340-341.
57. Levy, W. B., and Steward, D. (1983). Temporal contiguity requirements for long-term associative potentiation/depression in the hippocampus. Neuroscience 8, 791-797.
58. Li, Y., Fitzpatrick, D., and White, L. E. (2006). The development of direction selectivity in ferret visual cortex requires early visual experience. Nature Neuroscience 9, 676-681.
59. Magee, J. C., and Johnston, D. (1997). A synaptically controlled, associative signal for Hebbian plasticity in hippocampal neurons. Science 275, 209-213.
60. Maldonado, P. E., Godecke, I., Gray, C. M. and Bonhoeffer, T. (1997). Orientation selectivity in pinwheel centers in cat striate cortex. Science 276, 1551-1555.
61. Mark, S. and Tsodyks, M. (2012). Population spikes in cortical networks during different functional states. Frontiers in Computational Neuroscience 6:43. Doi: 10.3389/fncom.2012.00043.
62. Markram, H., Lubke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275, 213-215.
63. Markram, H., Gerstner, W., and Sjostrom, P. J. (2011). A history of spike-timing-dependent plasticity. Frontiers in Synaptic Neuroscience 3, 1-24.
64. Meliza, C. D., and Dan, Y. (2006). Receptive-field modification in rat visual cortex induced by paired visual stimulation and single-cell spiking. Neuron 49(2), 183-189.
65. Miikkulainen, R., Bednar, J. A., Choe, Y., and Shirosh, J. (2005). Computational Maps in the Visual Cortex. New York, Springer.
66. Miller, K., (1996). Receptive fields and maps in the visual cortex: models of ocular dominance and orientation columns. In Models of Neural Networks, III, E. Domany, J. van Hemmen, and K. Schulten, eds., New York: Springer-Verlag, 55-78.
67. Miller, K. D., Erwin, E., and Kayser, A. (1999). Is the development of orientation selectivity instructed by activity? Journal of Neurobiology 41, 44-57.
68. Miller, K. D., Keller, J. B., and Stryker, M. P. (1989). Ocular dominance column development: analysis and simulation. Science 245, 605-615.
69. Minkovich, K., Thibeault, C. M., Nogin, A., Cho, Y. K., O'Brien, M. J., and Srinivasa, N. (2012). HRLSim: A high performance spiking neural network simulator for GPGPU clusters, IEEE Trans. of Neural Networks and Learning Systems, under review.
70. Mooney, R., Penn, A. A., Gallego, R., Shatz, C. J. (1996). Thalamic relay of spontaneous retinal activity prior to vision. Neuron 17, 863-874.
71. Mu, Y., and Poo, M. M. (2006). Spike timing-dependent LTP/LTD mediates visual experiencedependent plasticity in a developing retinotectal system. Neuron 50(1), 115-125.
72. Muezzinoglu, M. K., Tristan, I., Heurta, R., Afraimovich, V. S., and Rabinovich, M. I. (2010). Transients versus attractors in complex networks. International Journal of Bifurcations and Chaos 20, 1653-1675.
73. Myhr, K. L., Lukasiewicz, P. D., and Wong, R. O. L. (2001). Mechanisms underlying developmental changes in the firing patterns of ON and OFF retinal ganglion cells during refinement of their central projections. Journal of Neuroscience, 21(21), 8664-8671.
74. Nageswaran, J.M., Dutt, N., Krichmar, J.L., Nicolau, A., and Veidenbaum, A.V. (2009). A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors. Neural Networks 22, 791-800.
75. Ohki, K., Chung, S, Kara, P., Hubener, M., Bonhoeffer, T., and Reid, R. C. (2006). Highly ordered arrangement of single neurons in orientation pinwheels. Nature 442, 925-928.
76. Ohshiro, T., Hussain, S., Weliky, M. (2011). Development of cortical orientation selectivity in the absence of visual experience with contour. J. Neurophysiol. 106(4), 1923-1932.
77. Okun, M., Lampl, I. (2008). Instantaneous correlation of excitation and inhibition during ongoing and sensory-evoked activities. Nature Neuroscience 11, 535-537.
78. Perin, R., Berger, T. K., and Markram, H. (2011). A synaptic organizing principle for cortical neuronal groups. Proceedings of National Academy of Sciences 108(13), 5419-5424.
79. Pouille, F., Burgin, A. M., Adesnik, H., Atallah, B. V., and Scanziani, M. (2009). Input normalization by global feedforward inhibition expands cortical dynamic range. Nature Neuroscience 12, 1577-1585.
80. Purves, D., and Lichtman, J. (1985). Principles of Neural Development. Sunderland, MA: Sinauer Associates.
81. Rao, S. C., Toth L. J. and Sur, M. (1997). Optical imaged maps of orientation preference in primary visual cortex of cats and ferrets. Journal of Computational Neurology 387, 358-370.
82. Ruthazer, E., and Stryker. M. (1996). The role of activity in the development of long-range horizontal connections in area 17 of the ferret. Journal of Neuroscience 18, 7253-7269.
83. Sengpiel, F., Stawinski, P., Bonhoeffer, T. (1999). Influence of experience on orientation maps in cat visual cortex. Nature Neuroscience 2, 727-732.
84. Schiller, P. H. (1992). The ON and OFF channels of the visual system. Trends in Neurosciences 15, 86-92.
85. Schummers, J., Marino, J., and Sur, M. (2004). Local networks in visual cortex and their influence on neuronal responses and dynamics. Journal of Physiology 98, 429-441.
86. Schummers, J., Sharma, J., and Sur, M. (2005). Bottom-up and top-down dynamics in visual cortex. Progress in Brain Research 149, 65-81.
87. Shapley, R., Hawken, M. and Ringach, D. L. (2003). Dynamics of orientation selectivity in the primary visual cortex and the importance of cortical inhibition. Neuron 38, 689-699.
88. Shapley, R., Hawken, M., and Xing, D. (2007). The dynamics of visual responses in the primary visual cortex. Progress in Brain Research 165, 21-32.
89. Shu, Y., Hasenstaub, A., and McCormick, D. A. (2003). Turning on and off recurrent balanced cortical activity. Nature 423, 288-293.
90. Sirosh, J., and Miikkulainen, R., (1995). Self-Organization and functional role of lateral connections and multisize receptive fields in the primary visual cortex. University of Texas at Austin, Technical Report AI95-230, May 1995.
91. Sjostrom. P. J., Rancz, E. A., Roth, A., and Hausser, M. (2008). Dendritic excitability and synaptic plasticity. Physiological Reviews 88, 769-840.
92. Smith, S. L., and Trachtenberg, J. T. (2007). Experience-dependent binocular competition in the visual cortex begins at eye opening. Nature Neuroscience 10, 370-375.
93. Somers, D. C., Nelson, S. B., and Sur, M. (1995). An emergent model of orientation selectivity in cat visual cortical simple cells. Journal of Neuroscience 15, 773-786.
94. Somers, D., Dragoi, V. and Sur, M. (2002). Orientation selectivity and its modulation by local and long-range connections in visual cortex. The Cat Primary Visual Cortex, A. Peters and B. Payne, eds., Academic Press, 471-520.
95. Song, S., and Abbott, L. F. (2001). Cortical remapping through spike timing-dependent plasticity. Neuron 32, 1-20.
96. Song, S., Miller, K., Abbott, L. F. (2000). Competitive Hebbian learning through spike timingdependent plasticity. Nature Neuroscience 3, 919-925.
97. Song, S., Sjostrom, P. J., Reigl, M., Nelson, S., and Chklovskii, D, B. (2005). Highly nonrandom features of synaotic connectivity in local cortical circuits. PLoS Biology 3:e68. doi:10.1371/journal.pbio.0030068.
98. Sporns, O., and Zwi, J. D. (2004). The small world of the cerebral cortex. Neuroinformatics 2, 145-162.
99. Swindale, N. V. (2000). How many maps are there in visual cortex? Cerebral Cortex 10, 633-643.
100. Taha, S. A., and Stryker M. P. (2005). Molecular substrates of plasticity in the developing visual cortex. Progress in Brain Research 147, 101-114.
101. Trachtenberg, J. T., Trepel, C., and Stryker, M. P. (2000). Rapid extra-granular plasticity in the absence of thalamocortical plasticity in the developing visual cortex. Science 287, 2029-2032.
102. Tsodyks, M., Pawelzik, K., and Markram, H. (1998). Neural networks with dynamic synapses. Neural Computation 10, 821-835.
103. Tsodyks, M., and Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proceedings of National Academy of Sciences 94, 719-723.
104. Turrigiano, G. G. and Nelson, S. B. (2004). Homeostatic plasticity in the developing nervous system. Nature Review Neuroscience 5, 97-107.
105. Ursey, W. M., Reppan, J. B. and Reid, R. C. (1999). Specificity and strength of retinogeniculate connections. Journal of Neurophysiology 82, 3527-3540.
106. Voges, N. and Perrinet, L. (2012). Complete dynamics in recurrent cortical networks based on spatially realistic connectivities. Frontiers in Computational Neuroscience 6:41. doi:10.3389/fncom.2012.00041.
107. Vogels, T. P., Rajan, K., and Abbott, L. F. (2005). Neural network dynamics. Annual Review Neuroscience 28, 357-376.
108. Vogels, T. P., Sprekeler, H., Zenke, F., Clopath, C., and Gerstner, W. (2011). Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks. Science 334, 1569-1573.
109. Vislay-Meltzer, R. L., Kampff, A. R., and Engert, F. (2006). Spatiotemporal specificity of neuronal activity directs the modification of receptive fields in the developing retinotectal system. Neuron 50(1), 101-114.
110. Watt, A. J., and Desai, N. S. (2010). Homeostatic plasticity and STDP: Keeping a neuron's cool in a fluctuating world. Frontiers in Synaptic Neuroscience 2, 5: doi: 10.3389/fnsyn.2010.00005.
111. Weliky, M., Bosking, W. H., and Fitzpatrick, D. (1996). A systematic map of direction preference in primary visual cortex. Nature 379(6567):725-728.
112. Wenisch, O. G., Noll, J., and van Hemmen, J. L. (2005). Spontaneously emerging direction selectivity maps in visual cortex through STDP. Biological Cybernetics 93, 239-247.
113. White, L. E., Coppola, D. M., Fitzpatrick, D. (2001). The contribution of sensory experience to the maturation of orientation selectivity in ferret visual cortex. Nature 411, 1049-1052.
114. White, L. E., and Fitzpatrick, D. (2007). Vision and cortical map development. Neuron 56, 327-338, 2007.
115. Wiesel, T. N., and Hubel, D. H. (1974). Ordered arrangement of orientation columns in monkeys lacking visual experience, Journal Computational Neurology 158, 307-318.
116. Williams, S. R., Wozny, C., and Mitchell, S. J. (2007). The back and forth of dendritic plasticity. Neuron 56, 947-953.
117. Wong, O. L. R. (1999). Retinal waves and visual system development, Annual Review of Neuroscience 22, 29-47.
118. Woodin, M. A., Ganguly, K., Poo, M. M. (2003). Coincident pre-and postsynaptic activity modifies GABAergic synapses by postsynaptic changes in Clâtransporter activity. Neuron 39, 807-820.
119. Yang, Q., Qi, Xianglin and Wang, Y. (2012). A neural network model on self-organizing emergence of simple-cell receptive field with orientation selectivity in visual cortex. Science in China Series C Life Sciences, 44(5), 469-478.
120. Yao., H., and Dan, Y. (2001). Stimulus timing-dependent plasticity in cortical processing of orientation. Neuron 32, 315-323.
121. Young, J. M., Waleszcyk, W. J., Wang, C., Calford, M. B., Dreher, B., and Obermayer, K. (2007). Cortical reorganization consistent with spike timing but not correlation-dependent plasticity. Nature Neuroscience 10, 887-895.
122. Yu, H., Farley, B. J., Jin, D. Z., and Sur, M. (2005). The coordinated mapping of visual space and response features in visual cortex. Neuron 47, 267-280.
123. Yuste, R., and Sur, M. (1999). Development and plasticity of the cerebral cortex: from molecules to maps. Journal of Neuroscience 41, 1-6.
124. Zaghloul, K. A., Boahen, K., and Demb, J. B. (2003). Different circuits for ON and OFF retinal ganglion cells cause different contrast sensitivities. The Journal of Neuroscience 23, 2645-2654.
125. Zhang, Z., and Xu, G. (1998). A unified theory of uncalibrated stereo for both perspective and affine cameras. Journal of Mathematical Imaging and Vision 9, 213-229.