A framework for plasticity implementation on the SpiNNaker neural architecture

Frontiers in Neuroscience

Volumn 9, Issue JAN, 2015, Pages

  Galluppi, Francesco ^a   Lagorce, Xavier ^a   Stromatias, Evangelos ^b   Pfeiffer, Michael ^c   Plana, Luis A ^b   Furber, Steve B ^b   Benosman, Ryad B ^a  

^a INSTITUT DE LA VISION   (France)

^b UNIVERSITY OF MANCHESTER   (United Kingdom)

^c UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

BCM; Learning; Neuromorphic hardware; Plasticity; SpiNNaker; STDP

Indexed keywords

ARTICLE; COMPUTER; EXCITATION; HUMAN; LEARNING; MEMBRANE POTENTIAL; MEMORY; NERVE CELL; NERVE CELL PLASTICITY; PLASTICITY; SIMULATION; SPIKE; SPIKE TIMING DEPENDENT PLASTICITY; VOLTAGE DEPENDENT SPIKE TIMING DEPENDENT PLASTICITY; WEIGHT;

EID: 84921798570     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2014.00429     Document Type: Article

References (87)

1. Abbott, L. F., and Nelson, S. B. (2000). Synaptic plasticity: taming the beast. Nat. Neurosci. 3, 1178-1183. doi: 10.1038/81453
2. Bartolozzi, C., Nikolayeva, O., and Indiveri, G. (2008). "Implementing homeostatic plasticity in VLSI networks of spiking neurons," in Proceedings of the 15th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2008 (St. Julien's), 682-685.
3. Beyeler, M., Dutt, N. D., and Krichmar, J. L. (2013). Categorization and decision-making in a neurobiologically plausible spiking network using a STDP-like learning rule. Neural Netw. 48, 109-124. doi: 10.1016/j.neunet.2013.07.012
4. Bi, G., and Poo, M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464-10472.
5. Bienenstock, E. L., Cooper, L. N., and Munro, P. W. (1982). Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. J. Neurosci. 2, 32-48.
6. Binas, J., Rutishauser, U., Indiveri, G., and Pfeiffer, M. (2014). Learning and stabilization of winner-take-all dynamics through interacting excitatory and inhibitory plasticity. Front. Comput. Neurosci. 8:68. doi: 10.3389/fncom.2014.00068
7. Brader, J. M., Senn, W., and Fusi, S. (2007). Learning real-world stimuli in a neural network with spike-driven synaptic dynamics. Neural Comput. 19, 2881-2912. doi: 10.1162/neco.2007.19.11.2881
8. Brette, R., and Goodman, D. F. (2012). Simulating spiking neural networks on gpu. Netw. Comput. Neural Syst. 23, 167-182. doi: 10.3109/0954898X.2012.730170
9. Brette, R., Rudolph, M., Carnevale, T., Hines, M., Beeman, D., Bower, J., et al. (2007). Simulation of networks of spiking neurons: a review of tools and strategies. J. Comput. Neurosci. 23, 349-398. doi: 10.1007/s10827-007-0038-6
10. Cassenaer, S., and Laurent, G. (2012). Conditional modulation of spike-timing-dependent plasticity for olfactory learning. Nature 482, 47-52. doi: 10.1038/nature10776
11. Clopath, C., Busing, L., Vasilaki, E., and Gerstner, W. (2010). Connectivity reflects coding: a model of voltage-based STDP with homeostasis. Nature Neurosci. 13, 344-352. doi: 10.1038/nn.2479
12. Coath, M., Sheik, S., Chicca, E., Indiveri, G., Denham, S. L., and Wennekers, T. (2013). A robust sound perception model suitable for neuromorphic implementation. Front. Neurosci. 7:278. doi: 10.3389/fnins.2013.00278
13. Davies, S., Galluppi, F., Rast, A., and Furber, S. (2012). A forecast-based STDP rule suitable for neuromorphic implementation. Neural Netw. 32, 3-14. doi: 10.1016/j.neunet.2012.02.018
14. Davison, A. A. P., Brüderle, D., Eppler, J. J., Kremkow, J., Muller, E., Pecevski, D., et al. (2008). PyNN: a common interface for neuronal network simulators. Front. Neuroinformat. 2:11. doi: 10.3389/neuro.11.011.2008
15. Diehl, P. U., and Cook, M. (2014). "Efficient implementation of STDP rules on SpiNNaker neuromorphic hardware," in International Conference on Neural Networks (IJCNN) 2014 (Beijing), 4288-4295.
16. Fidjeland, A., Roesch, E., Shanahan, M., and Luk, W. (2009). "NeMo: a platform for neural modelling of spiking neurons using GPUs," in 2009 20th IEEE International Conference on Application-Specific Systems, Architectures and Processors (Boston, MA: IEEE), 137-144.
17. Fidjeland, A., and Shanahan, M. (2010). "Accelerated simulation of spiking neural networks using GPUs," in Neural Networks, International Joint Conference on (Barcelona), 1-8.
18. Friedrich, J., Urbanczik, R., and Senn, W. (2011). Spatio-temporal credit assignment in neuronal population learning. PLoS Computat. Biol. 7:e1002092. doi: 10.1371/journal.pcbi.1002092
19. Furber, S., Galluppi, F., Temple, S., and Plana, A. (2014). The SpiNNaker project. Proc. IEEE 102, 652-665. doi: 10.1109/JPROC.2014.2304638
20. Furber, S., and Temple, S. (2008). Neural systems engineering. Computat. Intell. 4, 763-796. doi: 10.1098/rsif.2006.0177
21. Furber, S., Temple, S., and Brown, A. (2006). "High-performance computing for systems of spiking neurons," in The AISB06 Workshop on GC5: Architecture of Brain and Mind (Bristol).
22. Galluppi, F., Davies, S., Rast, A., Sharp, T., Plana, L., and Furber, S. (2012). "A hierarchical configuration system for a massively parallel neural hardware platform," in CF '12 Proceedings of the 9th Conference on Computing Frontiers, ed ACM (Cagliari), 183-192.
23. Galluppi, F., Denk, C., Meiner, M. C., Stewart, T., Plana, L. A., Eliasmith, C., et al. (2014). "Event-based neural computing on an autonomous mobile platform," in Proceedings of IEEE International Conference on Robotics and Automation (ICRA) (Hong Kong) 2862-2867.
24. Gerstner, W., Kempter, R., van Hemmen, J. L., and Wagner, H. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature 383, 76-78. doi: 10.1038/383076a0
25. Gerstner, W., Sprekeler, H., and Deco, G. (2012). Theory and simulation in neuroscience. Science 338, 60-65. doi: 10.1126/science.1227356
26. Giulioni, M., Camilleri, P., Dante, V., Badoni, D., Indiveri, G., Braun, J., et al. (2008). "A VLSI network of spiking neurons with plastic fully configurable learning synapses," in 2008 15th IEEE International Conference on Electronics, Circuits and Systems (St. Julien's).
27. Giulioni, M., Pannunzi, M., Badoni, D., Dante, V., and Del Giudice, P. (2009). Classification of correlated patterns with a configurable analog VLSI neural network of spiking neurons and self-regulating plastic synapses. Neural Comput. 21, 3106-3129. doi: 10.1162/neco.2009.08-07-599
28. Guyonneau, R., Van Rullen, R., and Thorpe, S. J. (2005). Neurons tune to the earliest spikes through STDP. Neural Comput. 17, 859-879. doi: 10.1162/0899766053429390
29. Hartline, H. K., Wagner, H. G., and Ratliff, F. (1956). Inhibition in the eye of the limulus. J. Gen. Physiol. 39, 651-673. doi: 10.1085/jgp.39.5.651
30. Hebb, D. O. (1949). The Organization of Behavior: A Neuropsychological Theory. New York, NY: Wiley-Interscience.
31. Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. U.S.A. 79, 2554-2558. doi: 10.1073/pnas.79.8.2554
32. Indiveri, G., Chicca, E., and Douglas, R. (2006). A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity. IEEE Trans. Neural Netw. 17, 211-221. doi: 10.1109/TNN.2005.860850
33. Indiveri, G., and Fusi, S. (2007). "Spike-based learning in VLSI networks of integrate-and-fire neurons," in Circuits and Systems, 2007. ISCAS 2007. IEEE International Symposium on (New Orleans, LA), 3371-3374.
34. Indiveri, G., Linares-Barranco, B., Hamilton, T. J., Van Schaik, A., Etienne-Cummings, R., Delbruck, T., et al. (2011). Neuromorphic silicon neuron circuits. Front. Neurosci. 5:73. doi: 10.3389/fnins.2011.00073
35. Indiveri, G., Linares-Barranco, B., Legenstein, R., Deligeorgis, G., and Prodromakis, T. (2013). Integration of nanoscale memristor synapses in neuromorphic computing architectures. IOP Nanotechnol. 24:384010. doi: 10.1088/0957-4484/24/38/384010
36. Izhikevich, E. (2006). Polychronization: computation with spikes. Neural Comput. 18, 245-282. doi: 10.1162/089976606775093882
37. Izhikevich, E. (2007). Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb. Cortex 17, 2443-2452. doi: 10.1093/cercor/bhl152
38. Jeyabalaratnam, J., Bharmauria, V., Bachatene, L., Cattan, S., Angers, A., and Molotchnikoff, S. (2013). Adaptation shifts preferred orientation of tuning curve in the mouse visual cortex. PLoS ONE 8:e64294. doi: 10.1371/journal.pone.0064294
39. Jin, X., Galluppi, F., Patterson, C., Rast, A., Davies, S., Temple, S., et al. (2010a). "Algorithm and software for simulation of spiking neural networks on the multi-chip SpiNNaker system," in Neural Networks, 2010. IJCNN 2010. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on (Barcelona), 1-8.
40. Jin, X., Rast, A., Galluppi, F., Davies, S., and Furber, S. (2010b). "Implementing spike-timing-dependent plasticity on SpiNNaker neuromorphic hardware," in Neural Networks, 2010. IJCNN 2010. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on (Barcelona: IEEE), 1-8.
41. Jin, X., Rast, A., Galluppi, F., Khan, M., and Furber, S. (2009). "Implementing learning on the SpiNNaker universal neural chip multiprocessor," in Neural Information Processing, Volume 5863, Chapter 48, eds C. S. Leung, M. Lee, and J. H. Chan (Berlin; Heidelberg: Springer), 425-432.
42. Kappel, D., Nessler, B., and Maass, W. (2014). STDP installs in winner-take-all circuits an online approximation to hidden markov model learning. PLoS Computat. Biol. 10:e1003511. doi: 10.1371/journal.pcbi.1003511
43. Kempter, R., Gerstner, W., and van Hemmen, J. L. (2001). Intrinsic stabilization of output rates by spike-based hebbian learning. Neural Comput. 13, 2709-2741. doi: 10.1162/089976601317098501
44. Kleberg, F. I., Fukai, T., and Gilson, M. (2014). Excitatory and inhibitory STDP jointly tune feedforward neural circuits to selectively propagate correlated spiking activity. Front. Comput. Neurosci. 8:53. doi: 10.3389/fncom.2014.00053
45. Kullmann, D. M., Moreau, A. W., Bakiri, Y., and Nicholson, E. (2012). Plasticity of inhibition. Neuron 75, 951-962. doi: 10.1016/j.neuron.2012.07.030
46. Lazar, A., Pipa, G., and Triesch, J. (2009). {SORN}: a self-organizing recurrent neural network. Front. Comput. Neurosci. 3:23. doi: 10.3389/neuro.10.023.2009
47. Le, Q. V., Monga, R., Devin, M., Corrado, G., Chen, K., Ranzato, M., et al. (2012). "Building high-level features using large scale unsupervised learning," in Proceedings of the 29th International Conference on Machine Learning (ICML) (Vancouver, BC), 1-11.
48. Legenstein, R., Pecevski, D., and Maass, W. (2008). A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Comput. Biol. 4:e1000180. doi: 10.1371/journal.pcbi.1000180
49. Markram, H., Lbke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275, 213-215. doi: 10.1126/science.275.5297.213
50. Mayr, C., Stärke, P., Partzsch, J., Schueffny, R., Cederstroem, L., Shuai, Y., et al. (2012). "Waveform driven plasticity in BiFeO3 memristive devices: model and implementation," in NIPS (Lake Tahoe, NV), 1709-1717.
51. Mead, C. (1989). Analog VLSI and Neural Systems. London: Addison-Wesley Longman Publishing Co., Inc.
52. Mehta, M. R., Quirk, M. C., and Wilson, M. A. (2000). Experience-dependent asymmetric shape of hippocampal receptive fields. Neuron 25, 707-715. doi: 10.1016/S0896-6273(00)81072-7
53. Min, R., Santello, M., and Nevian, T. (2012). The computational power of astrocyte mediated synaptic plasticity. Front. Comput. Neurosci. 6:93. doi: 10.3389/fncom.2012.00093
54. Mitra, S., Fusi, S., and Indiveri, G. (2009). Real-time classification of complex patterns using spike-based learning in neuromorphic VLSI. IEEE Trans. Biomed. Circ. Syst. 3, 32-42. doi: 10.1109/TBCAS.2008.2005781
55. Moore, B. D., and Freeman, R. D. (2012). Development of orientation tuning in simple cells of primary visual cortex. J. Neurophysiol. 107, 2506-2516. doi: 10.1152/jn.00719.2011
56. Morrison, A., Diesmann, M., and Gerstner, W. (2008). Phenomenological models of synaptic plasticity based on spike timing. Biol. Cybern. 98, 459-478. doi: 10.1007/s00422-008-0233-1
57. Morrison, A., Mehring, C., and Geisel, T. (2005). Advancing the boundaries of high-connectivity network simulation with distributed computing. Neural Comput. 17, 1776-1801. doi: 10.1162/0899766054026648
58. Mostafa, H., Corradi, F., Stefanini, F., and Indiveri, G. (2014). "A hybrid analog/digital spike-timing dependent plasticity learning circuit for neuromorphic VLSI multi-neuron architectures," in International Symposium on Circuits and Systems (ISCAS) 2014 (IEEE) 854-857.
59. Nageswaran, J., Dutt, N., Krichmar, J., and Nicolau, A. (2007). A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors. Neural Netw. 22, 791-800. doi: 10.1016/j.neunet.2009.06.028
60. Neftci, E., Das, S., Pedroni, B., Kreutz-Delgado, K., and Cauwenberghs, G. (2014). Event-driven contrastive divergence for spiking neuromorphic systems. Front. Neurosci. 7:272. doi: 10.3389/fnins.2013.00272
61. Neil, D., and Liu, S.-C. (2014). "Minitaur, an event-driven FPGA-based spiking network accelerator," in IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 1-1.
62. Nessler, B., Pfeiffer, M., Buesing, L., and Maass, W. (2013). Bayesian computation emerges in generic cortical microcircuits through spike-timing-dependent plasticity. PLoS Comput. Biol. 9:e1003037. doi: 10.1371/journal.pcbi.1003037
63. Oja, E. (1982). Simplified neuron model as a principal component analyzer. J. Math. Biol. 15, 267-273. doi: 10.1007/BF00275687
64. Painkras, E., Plana, L., Garside, J., Temple, S., Galluppi, F., Patterson, C., et al. (2013). "SpiNNaker: a 1W 18-core system-on-chip for massively-parallel neural net simulation," in The IEEE Journal of Solid State Circuits, VV, 8, 1-13.
65. Pawlak, V., Wickens, J. R., Kirkwood, A., and Kerr, J. N. D. (2010). Timing is not everything: neuromodulation opens the STDP gate. Front. Synaptic Neurosci. 2:146. doi: 10.3389/fnsyn.2010.00146
66. Pfeil, T., Potjans, T. C., Schrader, S., Potjans, W., Schemmel, J., Diesmann, M., et al. (2012). Is a 4-Bit synaptic weight resolution enough? constraints on enabling spike-timing dependent plasticity in neuromorphic hardware. Front. Neurosci. 6:90. doi: 10.3389/fnins.2012.00090
67. Plana, L., Furber, S., Temple, S., Khan, M., Shi, Y., Wu, J., et al. (2007). A GALS infrastructure for a massively parallel multiprocessor. IEEE Des. Test Comput. 24, 454-463. doi: 10.1109/MDT.2007.149
68. Plesser, H., Eppler, J., Morrison, A., Diesmann, M., and Gewaltig, M. (2007). "Efficient parallel simulation of large-scale neuronal networks on clusters of multiprocessor computers," in Euro-Par 2007 Parallel Processing (Berlin; Heidelberg: Springer), 672-681.
69. Potjans, W., Diesmann, M., and Morrison, A. (2011). An imperfect dopaminergic error signal can drive temporal-difference learning. PLoS Comput. Biol. 7:e1001133. doi: 10.1371/journal.pcbi.1001133
70. Rahimi Azghadi, M., Iannella, N., Al-Sarawi, S. F., Indiveri, G., and Abbott, D. (2014). Spike-based synaptic plasticity in silicon: design, implementation, application, and challenges. Proc. IEEE 102, 717-737. doi: 10.1109/JPROC.2014.2314454
71. Savin, C., Joshi, P., and Triesch, J. (2010). Independent component analysis in spiking neurons. PLoS Comput. Biol. 6:e1000757. doi: 10.1371/journal.pcbi.1000757
72. Schemmel, J., Bruderle, D., Meier, K., and Ostendorf, B. (2007). "Modeling synaptic plasticity within networks of highly accelerated I&F neurons," in 2007 IEEE International Symposium on Circuits and Systems (New Orleans, LA). doi: 10.1109/ISCAS.2007.378289
73. Sharp, T., and Furber, S. (2013). "Correctness and performance of the SpiNNaker architecture," in Neural Networks (IJCNN), The 2013 International Joint Conference on (Dallas, TX), 1-8.
74. Sharp, T., Plana, L., Galluppi, F., and Furber, S. (2011). "Event-driven simulation of arbitrary spiking neural networks on SpiNNaker," in The International Conference on Neural Information Processing (ICONIP), Volume 2011 (Shanghai: Springer), 424-430. doi: 10.1007/978-3-642-24965-5/48
75. Sheik, S., Coath, M., Indiveri, G., Denham, S. L., Wennekers, T., and Chicca, E. (2012). Emergent auditory feature tuning in a real-time neuromorphic VLSI system. Front. Neurosci. 6:17. doi: 10.3389/fnins.2012.00017
76. Shouval, H., Intrator, N., and Cooper, L. N. (1997). BCM network develops orientation selectivity and ocular dominance in natural scene environment. Vis. Res. 37, 3339-3342. doi: 10.1016/S0042-6989(97)00087-4
77. Sjöström, P. J., Turrigiano, G. G., and Nelson, S. B. (2001). Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron 32, 1149-1164. doi: 10.1016/S0896-6273(01)00542-6
78. Song, S., and Abbott, L. F. (2001). Cortical development and remapping through spike timing-dependent plasticity. Neuron 32, 339-350. doi: 10.1016/S0896-6273(01)00451-2
79. Song, S., Miller, K. D., and Abbott, L. F. (2000). Competitive hebbian learning through spike-timing-dependent synaptic plasticity. Nat. Neurosci. 3, 919-926. doi: 10.1038/78829
80. Stewart, T. C., Tripp, B., and Eliasmith, C. (2009). Python scripting in the nengo simulator. Front. Neuroinformat. 3:7. doi: 10.3389/neuro.11.007.2009
81. Stromatias, E., Galluppi, F., Patterson, C., and Furber, S. (2013). "Power analysis of large-scale, real-time neural networks on SpiNNaker," in The International Joint Conference on Neural Networks - ICJNN 2013 (Dallas, TX), 1570-1577.
82. Tapson, J. C., Cohen, G. K., Afshar, S., Stiefel, K. M., Buskila, Y., Wang, R. M., et al. (2013). Synthesis of neural networks for spatio-temporal spike pattern recognition and processing. Front. Neurosci. 7:153. doi: 10.3389/fnins.2013.00153
83. van Schaik, A., and Liu, S. (2005). "AER EAR: a matched silicon cochlea pair with address event representation interface," in IEEE International Symposium on Circuits and Systems ISCAS (Kobe), 4213-4216.
84. Vogelstein, R. J., Tenore, F., Philipp, R., Adlerstein, M. S., Goldberg, D. H., and Cauwenberghs, G. (2002). "Spike timing-dependent plasticity in the address domain," in Advances in Neural Information Processing Systems (Vancouver, CA), 1147-1154.
85. Wang, R., Cohen, G., Stiefel, K. M., Hamilton, T. J., Tapson, J., and van Schaik, A. (2013). An FPGA implementation of a polychronous spiking neural network with delay adaptation. Front. Neurosci. 7:14. doi: 10.3389/fnins.2013.00014
86. Wong, T., Preissl, R., Datta, P., Flickner, M., Singh, R., Esser, S., et al. (2013). 10∧14. Technical report, IBM.
87. Zamarreño Ramos, C., Camuñas Mesa, L. A., Pérez-Carrasco, J. A., Masquelier, T., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011). On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front. Neurosci 5:26. doi: 10.3389/fnins.2011.00026