Is a 4-bit synaptic weight resolution enough? - constraints on enabling spike-timing dependent plasticity in neuromorphic hardware

Frontiers in Neuroscience

Volumn , Issue JULY, 2012, Pages 1-19

  Pfeil, Thomas ^a   Potjans, Tobias C ^b,c   Schrader, Sven ^a,b,d   Schemmel, Johannes ^a   Diesmann, Markus ^b,c,d,e   Meier, Karlheinz ^a  

^a UNIVERSITY OF HEIDELBERG   (Germany)

^b FORSCHUNGSZENTRUM JÜLICH   (Germany)

^c RIKEN   (Japan)

^d RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

^e RWTH AACHEN UNIVERSITY   (Germany)

Author keywords

Dependent plasticity, Synaptic weight resolution, Circuit variations, PyNN, NEST; Large scale spiking neural networks; Neuromorphic hardware; Spike timing; Wafer scale integration

Indexed keywords

ACCURACY; ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER; COMPUTER PROGRAM; COMPUTER SIMULATION; HUMAN; LARGE SCALE PRODUCTION; NERVE CELL PLASTICITY; NEUMORPHIC HARDWARE; NEUROSCIENCE; OPTICAL RESOLUTION; QUALITY CONTROL; SPIKE WAVE; SYNAPSE; SYNAPTIC WEIGHT RESOLUTION;

EID: 84865180086     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2012.00090     Document Type: Article

References (78)

1. Amit, D., and Fusi, S. (1994). Learn-ing in neural networks with mate-rial synapses. Neural Comput. 6, 957-982.
2. Arthur, J. V., and Boahen, K. (2006). "Learning in silicon: timing is every-thing," in Advances in Neural Infor-mation Processing Systems (NIPS), Vol. 18. Vancouver: MIT Press, 75-82.
3. Badoni, D., Giulioni, M., Dante, V., and Del Giudice, P. (2006). "An aVLSI recurrent network of spiking neu-rons with reconfigurable and plastic synapses," in Proceedings of the 2006 International Symposium on Circuits and Systems (ISCAS) (Island of Kos: IEEE Press), 4.
4. Bi, G., and Poo, M. (1998). Synap-tic modifications in cultured hip-pocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464-10472.
5. Bi, G., and Poo, M. (2001). Synap-tic modification by correlated activ-ity: Hebb's postulate revisited. Annu. Rev. Neurosci. 24, 139-166.
6. Bill, J., Schuch, K., Brüderle, D., Schemmel, J., Maass, W., and Meier, K. (2010). Compen-sating inhomogeneities of neuromorphic VLSI devices via short-term synaptic plasticity. Front. Comput. Neurosci. 4:129. doi: 10.3389/fncom.2010.00129
7. Brader, J. M., Senn, W., and Fusi, S. (2007). Learning real world stimuli in a neural network with spike-driven synaptic dynamics. Neural Comput. 19, 2881-2912.
8. Brette, R., Rudolph, M., Carnevale, T., Hines, M., Beeman, D., Bower, J. M., Diesmann, M., Morrison, A., Goodman, P. H., Harris, F. C. Jr., Zirpe, M., Natschläger, T., Pecevski, D., Ermentrout, B., Djurfeldt, M., Lansner, A., Rochel, O., Vieville, T., Muller, E., Davison, A. P., El Bous-tani, S., and Destexhe, A. (2007). Simulation of networks of spik-ing neurons: a review of tools and strategies. J. Comput. Neurosci. 23, 349-398.
9. Brüderle, D., Petrovici, M., Vogginger, B., Ehrlich, M., Pfeil, T., Millner, S., Grübl, A., Wendt, K., Müller, E., Schwartz, M.-O., Husmann de Oliveira, D., Jeltsch, S., Fieres, J., Schilling, M., Müller, P., Breitwieser, O., Petkov, V., Muller, L., Davison, A. P., Krishnamurthy, P., Kremkow, J., Lundqvist, M., Muller, E., Partzsch, J., Scholze, S., Zühl, L., Destexhe, A., Diesmann, M., Potjans, T. C., Lansner, A., Schüffny, R., Schemmel, J., and Meier, K. (2011). A com-prehensive workflow for general-purpose neural modeling with highly configurable neuromorphic hardware systems. Biol. Cybern. 104, 263-296.
10. Brunel, N. (2000). Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J. Comput. Neurosci. 8, 183-208.
11. Butts, D. A., Weng, C., Jin, J., Yeh, C.-I., Lesica, N. A., Alonso, J.-M., and Stanley, G. B. (2007). Temporal pre-cision in the neural code and the timescales of natural vision. Nature 449, 92-95.
12. Cassenaer, S., and Laurent, G. (2007). Hebbian STDP in mushroom bod-ies facilitates the synchronous flow of olfactory information in locusts. Nature 448, 709-713.
13. Cassenaer, S., and Laurent, G. (2012). Conditional modulation of spike-timing-dependent plas-ticity for olfactory learning. Nature 482, 47-52.
14. Clopath, C., Ziegler, L., Vasilaki, E., Büsing, L., and Gerstner, W. (2008). Tag-Trigger-Consolidation: a model of early and late long-term-potentiation and depression. PLoS Comput. Biol. 4, e1000248. doi:10.1371/journal.pcbi.1000248
15. Daouzli, A., Saighi, S., Buhry, L., Bor-nat, Y., and Renaud, S. (2008). "Weights convergence and spikes correlation in an adaptive neural network implemented on VLSI," in Proceedings of the International Conference on Bio-inspired Sys-tems and Signal Processing, Funchal, 286-291.
16. Davison, A., Brüderle, D., Eppler, J. M., Kremkow, J., Muller, E., Pecevski, D., Perrinet, L., and Yger, P. (2009). PyNN: a common inter-face for neuronal network simula-tors. Front. Neuroinformatics 2:11. doi: 10.3389/neuro.11.011.2008.
17. Davison, A. P., and Frégnac, Y. (2006). Learning cross-modal spatial trans-formations through spike timing-dependent plasticity. J. Neurosci. 26, 5604-5615.
18. Desbordes, G., Jin, J., Alonso, J.-M., and Stanley, G. B. (2010). Modulation of temporal precision in thalamic pop-ulation responses to natural visual stimuli. Front. Syst. Neurosci. 4:151. doi:10.3389/fnsys.2010.00151
19. Desbordes, G., Jin, J., Weng, C., Lesica, N. A., Stanley, G. B., and Alonso, J.-M. (2008). Tim-ing precision in population cod-ing of natural scenes in the early visual system. PLoS Biol. 6, e324. doi:10.1371/journal.pbio.0060324
20. Eckhorn, R., Reitböck, H.-J., Arndt, M., and Dicke, P. (1990). Feature linking via synchronization among distributed assemblies: results from cat visual cortex and from simula-tions. Neural Comput. 2, 293-307.
21. El Boustani, S., Yger, P., Frégnac, Y., and Destexhe, A. (2012). Stable learn-ing in stochastic network states. J. Neurosci. 32, 194-214.
22. Esmaeilzadeh, H., Blem, E., St Amant, R.,Sankaralingam,K.,and Burger,D. (2011). "Dark silicon and the end of multicore scaling," in Proceedings of the 2011 International Symposium on Computer Architecture (ISCA) (San Jose: ACM Press), 365-376.
23. Facets. (2010). Fast Analog Computing with Emergent Transient States. Available at: http://www.facets-project.org
24. Fromherz, P. (2002). Electrical inter-facing of nerve cells and semi-conductor chips. Chemphyschem 3, 276-284.
25. Fusi, S., Drew, P. J., and Abbott, L. F. (2005). Cascade models of synapti-cally stored memories. Neuron 45, 599-611.
26. Gardiner, C. (2009). Stochastic Meth-ods: A Handbook for the Natural and Social Sciences (4th Edn.). Berlin: Springer.
27. Gerstner, W., Kempter, R., Van Hem-men, J. L., and Wagner, H. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature 383, 76-78.
28. Gewaltig, M.-O., and Diesmann, M. (2007). NEST (NEural Simulation Tool). Scholarpedia J. 2, 1430.
29. Gütig, R., Aharonov, R., Rotter, S., and Sompolinsky, H. (2003). Learn-ing input correlations through non-linear temporally asymmetric Heb-bian plasticity. J. Neurosci. 23, 3697-3714.
30. Indiveri, G., Linares-Barranco, B., Hamilton, T. J., Van Schaik, A., Etienne-Cummings, R., Delbruck, T., Liu, S. -C., Dudek, P., Häfliger, P., Renaud, S., Schemmel, J., Cauwen-berghs, G., Arthur, J., Hynna, K., Folowosele, F., Saighi, S., Serrano-Gotarredona, T., Wijekoon, J., Wang, Y., and Boahen, K. (2011). Neuromorphic silicon neuron circuits. Front. Neurosci. 5:73. doi: 10.3389/fnins.2011.00073
31. Indiveri, G., Stefanini, F., and Chicca, E. (2010). "Spike-based learning with a generalized integrate and fire silicon neuron," in Proceedings of the 2010 International Symposium on Circuits and Systems (ISCAS) (Paris: IEEE Press), 1951-1954.
32. Itti, L., and Koch, C. (2001). Computa-tional modeling of visual attention. Nat. Rev. Neurosci. 2, 194-203.
33. Jin, X., Lujan, M., Plana, L., Davies, S., Temple, S., and Furber, S. (2010a). Modeling spiking neural networks on SpiNNaker. Comput. Sci. Eng. 12, 91-97.
34. Jin, X., Rast, A., Galluppi, F., Davies, S., and Furber, S. (2010b)."Implement-ing spike-timing-dependent plas-ticity on SpiNNaker neuromor-phic hardware," in Proceedings of the 2010 International Joint Confer-ence on Neural Networks (IJCNN) (Barcelona: IEEE Press), 1-8.
35. Johansson, C., and Lansner, A. (2007). Towards cortex sized artificial neural systems. Neural Netw. 20, 48-61.
36. Kempter, R., Gerstner, W., and Van Hemmen, J. L. (1999). Hebbian learning and spiking neurons. Phys. Rev. E 59, 4498-4514.
37. Kuba, H., Koyano, K., and Ohmori, H. (2002). Synaptic depression improves coincidence detection in the nucleus laminaris in brainstem slices of the chick embryo. Eur. J. Neurosci. 15, 984-990.
38. Kuhn, A., Aertsen, A., and Rotter, S. (2003). Higher-order statistics of input ensembles and the response of simple model neurons. Neural Comput. 1, 67-101.
39. Kunkel, S., Diesmann, M., and Morri-son,A. (2011). Limits to the develop-ment of feed-forward structures in large recurrent neuronal networks. Front. Comput. Neurosci. 4:160. doi:10.3389/fncom.2010.00160
40. Levi, T., Lewis, N., Saighi, S., Tomas, J., Bornat, Y., and Renaud, S. (2008). VLSI Circuits for Biomedical Appli-cations, Chap. 12. Norwood: Artech House, 241-264.
41. Levy, W. B., and Steward, D. (1983). Temporal contiguity require-ments for long-term associative potentiation/depression in the hippocampus. Neuroscience 8, 791-797.
42. Linares-Barranco, B., Serrano-Gotarredona, T., Camunas-Mesa, L. A.,Perez-Carrasco,J.A.,Zamarreno-Ramos, C., and Masquelier, T. (2011). On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front. Neurosci. 5:26. doi:10.3389/fnins.2011.00026
43. Mann, H. B., and Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. Ann. Stat. 18, 50-60.
44. Markram, H. (2006). The blue brain project. Nat. Rev. Neurosci. 7, 153-160.
45. Markram, H., Lübke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275, 213-215.
46. Marre, O., Yger, P., Davison, A. P., and Frégnac, Y. (2009). Reliable recall of spontaneous activity patterns in cortical networks. J. Neurosci. 29, 14596-14606.
47. Merolla, P., and Boahen, K. (2006). "Dynamic computation in a recur-rent network of heterogeneous silicon neurons," in Proceedings of the 2006 International Sym-posium on Circuits and Systems (ISCAS) (Island of Kos: IEEE Press), 4539-4542.
48. Millner, S., Grübl, A., Schemmel, J., Meier, K., and Schwartz, M.-O. (2010). "A VLSI implementation of the adaptive exponential integrate-and-fire neuron model," in Advances in Neural Information Processing Sys-tems (NIPS), Vol. 23, Vancouver, 1642-1650.
49. Morrison, A., Aertsen, A., and Dies-mann, M. (2007). Spike-timing dependent plasticity in balanced random networks. Neural Comput. 19, 1437-1467.
50. Morrison, A., Diesmann, M., and Ger-stner, W. (2008). Phenomenological models of synaptic plasticity based on spike-timing. Biol. Cybern. 98, 459-478.
51. Morrison, A., Mehring, C., Geisel, T., Aertsen, A., and Diesmann, M. (2005). Advancing the bound-aries of high connectivity net-work simulation with distributed computing. Neural Comput. 17, 1776-1801.
52. Mu, Y., and Poo, M. (2006). Spike timing-dependent LTP/LTD medi-ates visual experience-dependent plasticity in a developing retinotectal system. Neuron 50, 115-125.
53. Nordlie, E., Gewaltig, M.-O., and Plesser, H. E. (2009). Towards reproducible descriptions of neu-ronal network models. PLoS Comput. Biol. 5, e1000456. doi:10.1371/journal.pcbi.1000456
54. Pelgrom, M., Duinmaijer, A., and Welbers, A. (1989). Matching properties of MOS transistors. IEEE J. Solid-State Circuits 24, 1433-1439.
55. Perrin,D. (2011). Complexity and high-end computing in biology and med-icine. Adv. Exp. Med. Biol. 696, 377-384.
56. Plana, L. A., Furber, S. B., Temple, S., Khan, M., Shi, Y., Wu, J., and Yang, S. (2007). A GALS infrastruc-ture for a massively parallel multi-processor.IEEEDes.TestComput.24, 454-463.
57. Ramakrishnan, S., Hasler, P., and Gordon, C. (2011). Floating gate synapses with spike-time-dependent plasticity. IEEE Trans. Biomed. Cir-cuits Syst. 5, 244-252.
58. Reynolds, J. N., and Wickens, J. R. (2002). Dopamine-dependent plas-ticity of corticostriatal synapses. Neural Netw. 15, 507-521.
59. Rubin, J., Lee, D., and Sompolin-sky, H. (2001). Equilibrium proper-ties of temporally asymmetric Heb-bian plasticity. Phys. Rev. Lett. 86, 364-367.
60. Schemmel, J., Brüderle, D., Grübl, A., Hock, M., Meier, K., and Millner, S. (2010). "A wafer-scale neuromor-phic hardware system for large-scale neural modeling," in Proceedings of the 2010 International Symposium on Circuits and Systems (ISCAS) (Paris: IEEE Press), 1947-1950.
61. Schemmel, J., Brüderle, D., Meier, K., and Ostendorf, B. (2007). "Model-ing synaptic plasticity within net-works of highly accelerated I&F neu-rons," in Proceedings of the 2007 International Symposium on Circuits and Systems (ISCAS) (New Orleans: IEEE Press).
62. Schemmel, J., Fieres, J., and Meier, K. (2008). "Wafer-scale integration of analog neural networks," in Pro-ceedings of the 2008 International Joint Conference on Neural Net-works (IJCNN) (Hong Kong: IEEE Press).
63. Schemmel, J., Gruebl, A., Meier, K., and Mueller, E. (2006). "Implementing synaptic plasticity in a VLSI spik-ing neural network model," in Pro-ceedings of the 2006 International Joint Conference on Neural Networks (IJCNN) (Vancouver: IEEE Press), 1-6.
64. Senn, W., Segev, I., and Tsodyks, M. (1998). Reading neuronal synchrony with depressing synapses. Neural Comput. 10, 815-819.
65. Shmuel, A., Korman, M., Sterkin, A., Harel, M., Ullman, S., Malach, R., and Grinvald, A. (2005). Retinotopic axis specificity and selective clustering of feedback projections from V2 to V1 in the owl monkey. J. Neurosci. 25, 2117-2131.
66. Sjöström, P., Turrigiano, G., and Nel-son, S. (2001). Rate, timing, and cooperativity jointly determine cor-tical synaptic plasticity. Neuron 32, 1149-1164.
67. Song, S., Miller, K. D., and Abbott, L. F. (2000). Competitive Heb-bian learning through spike-timing-dependent synaptic plasticity. Nat. Neurosci. 3, 919-926.
68. Thompson, S. E., and Parthasarathy, S. (2006). Moore's law: the future of si microelectronics. Mater. Today 9, 20-25.
69. Trachtenberg, J. T., Chen, B. E., Knott, G. W., Feng, G., Sanes, J. R., Welker, E., and Svoboda, K. (2002). Long-term in vivo imaging of experience-dependent synaptic plas-ticity in adult cortex. Nature 420, 788-794.
70. Tsodyks, M. V., and Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proc. Natl. Acad. Sci. U.S.A. 94, 719-723.
71. Turrigiano, G. G., Leslie, K. R., Desai, N. S., Rutherford, L. C., and Nel-son,S. B. (1998).Activity-dependent scaling of quantal amplitude in neocortical neurons. Nature 391, 892-896.
72. Van Rossum, M. C. W., Bi, G., and Tur-rigiano, G. G. (2000). Stable Heb-bian learning from spike timing-dependent plasticity. J. Neurosci. 20, 8812-8821.
73. Vogels, T., Sprekeler, H., Zenke, F., Clopath, C., and Gerstner, W. (2011). Inhibitoryplasticity balances excitation and inhibition in sensory pathways and memory networks. Science 334, 1569-1573.
74. Vogels, T. P., Rajan, K., and Abbott, L. F. (2005). Neural network dynamics. Annu. Rev. Neurosci. 28, 357-376.
75. Vogelstein, R., Tenore, F., Guevre-mont, L., Etienne-Cummings, R., and Mushahwar, V. (2008). A silicon central pattern generator controls locomotion in vivo. IEEE Trans. Bio-med. Circ. Syst. 2, 212-222.
76. Vogelstein, R. J., Mallik, U., Culur-ciello, E., Cauwenberghs, G., and Etienne-Cummings, R. (2007). A multichip neuromorphic system for spike-based visual information processing. Neural Comput. 19, 2281-2300.
77. Yger, P., El Boustani, S., Destexhe, A., and Frégnac, Y. (2011). Topo-logically invariant macroscopic sta-tistics in balanced networks of conductance-based integrate-and-fire neurons. J. Comput. Neurosci. 31, 229-245.
78. Zou, Q., Bornat, Y., Saighi, S., Tomas, J., Renaud, S., and Destexhe, A. (2006). Analog-digital simulations of full conductance-based networks of spiking neurons with spike tim-ingdependentplasticity. Network 17, 211-233.