A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses

Frontiers in Neuroscience

Volumn 9, Issue APR, 2015, Pages

  Qiao, Ning ^a   Mostafa, Hesham ^a   Corradi, Federico ^a   Osswald, Marc ^a   Stefanini, Fabio ^a   Sumislawska, Dora ^a   Indiveri, Giacomo ^a  

^a UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

Analog VLSI; Asynchronous; Attractor network; Brain inspired computing; Real time; Spike based learning; Spike timing dependent plasticity (STDP); Winner Take All (WTA)

Indexed keywords

ARTICLE; CONNECTOME; CONTROLLED STUDY; INTEGRATED CIRCUIT; LEARNING ALGORITHM; MACHINE LEARNING; NEGATIVE FEEDBACK; NERVE CELL PLASTICITY; PATTERN RECOGNITION; SPIKE; SYNAPSE; WORKING MEMORY;

EID: 84928671707     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2015.00141     Document Type: Article

References (77)

1. Abbott, L., and Nelson, S. (2000). Synaptic plasticity: taming the beast. Nat. Neurosci. 3, 1178-1183. doi: 10.1038/81453
2. Amit, D. (1992). Modeling Brain Function: The World of Attractor Neural Networks. New York, NY: Cambridge University Press.
3. Amit, D., and Mongillo, G. (2003). Spike-driven synaptic dynamics generating working memory states. Neural Comput. 15, 565-596. doi: 10.1162/089976603321192086
4. Barak, O., and Rigotti, M. (2011). A simple derivation of a bound on the perceptron margin using singular value decomposition. Neural Comput. 23, 1935-1943. doi: 10.1162/NECO_a_00152
5. Barak, O., Rigotti, M., and Fusi, S. (2013). The sparseness of mixed selectivity neurons controls the generalization-discrimination trade-off. J. Neurosci. 33, 3844-3856. doi: 10.1523/JNEUROSCI.2753-12.2013
6. Bartolozzi, C., and Indiveri, G. (2007a). "A selective attention multi-chip system with dynamic synapses and spiking neurons," in Advances in Neural Information Processing Systems (NIPS), Neural Information Processing Systems Foundation, Vol. 19, eds B. Schölkopf, J. Platt, and T. Hofmann (Cambridge, MA: MIT Press), 19, 113-120.
7. Bartolozzi, C., and Indiveri, G. (2007b). Synaptic dynamics in analog VLSI. Neural Comput. 19, 2581-2603. doi: 10.1162/neco.2007.19.10.2581
8. Benjamin, B. V., Gao, P., McQuinn, E., Choudhary, S., Chandrasekaran, A. R., Bussat, J., et al. (2014). Neurogrid: a mixed-analog-digital multichip system for large-scale neural simulations. Proc. IEEE 102, 699-716. doi: 10.1109/JPROC.2014.2313565
9. Billings, G., and van Rossum, M. (2009). Memory retention and spike-timing-dependent plasticity. J. Neurophysiol. 101, 2775-2788. doi: 10.1152/jn.91007.2008
10. Boahen, K. (2000). Point-to-point connectivity between neuromorphic chips using address-events. IEEE Trans. Circ. Syst. II 47, 416-434. doi: 10.1109/82.842110
11. Boegerhausen, M., Suter, P., and Liu, S.-C. (2003). Modeling short-term synaptic depression in silicon. Neural Comput. 15, 331-348. doi: 10.1162/089976603762552942
12. Brader, J., Senn, W., and Fusi, S. (2007). Learning real world stimuli in a neural network with spike-driven synaptic dynamics. Neural Comput. 19, 2881-2912. doi: 10.1162/neco.2007.19.11.2881
13. Breiman, L. (2001). Random forests. Mach. Learn. 45, 5-32. doi: 10.1023/A:1010933404324
14. Brette, R., and Gerstner, W. (2005). Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. J. Neurophysiol. 94, 3637-3642. doi: 10.1152/jn.00686.2005
15. Bruederle, D., Petrovici, M., Vogginger, B., Ehrlich, M., Pfeil, T., Millner, S., et al. (2011). A comprehensive workflow for general-purpose neural modeling with highly configurable neuromorphic hardware systems. Biol. Cybern. 104, 263-296. doi: 10.1007/s00422-011-0435-9
16. Chicca, E., Stefanini, F., Bartolozzi, C., and Indiveri, G. (2014). Neuromorphic electronic circuits for building autonomous cognitive systems. Proc. IEEE 102, 1367-1388. doi: 10.1109/JPROC.2014.2313954
17. Costas-Santos, J., Serrano-Gotarredona, T., Serrano-Gotarredona, R., and Linares-Barranco, B. (2007). A spatial contrast retina with on-chip calibration for neuromorphic spike-based AER vision systems. IEEE Trans. Circ. Syst. I, 54, 1444-1458. doi: 10.1109/TCSI.2007.900179
18. Del Giudice, P., Fusi, S., and Mattia, M. (2003). Modeling the formation of working memory with networks of integrate-and-fire neurons connected by plastic synapses. J. Physiol. Paris 97, 659-681. doi: 10.1016/j.jphysparis.2004.01.021
19. Deiss, S., Douglas, R., and Whatley, A. (1998), "A pulse-coded communications infrastructure for neuromorphic systems," in Pulsed Neural Networks chapter 6 eds W. Maass and C. Bishop (Cambridge: MIT Press), 157-178.
20. Delbruck, T., Berner, R., Lichtsteiner, P., and Dualibe, C. (2010). "32-bit configurable bias current generator with sub-off-current capability," in IEEE International Symposium on Circuits and Systems (ISCAS) (Paris: IEEE), 1647-1650.
21. Delbruck, T., and Lang, M. (2013). Robotic goalie with 3MS reaction time at 4% CPU load using event-based dynamic vision sensor. Front. Neurosci. 7:223. doi: 10.3389/fnins.2013.00223
22. Douglas, R., Mahowald, M., and Mead, C. (1995). Neuromorphic analogue VLSI. Annu. Rev. Neurosci. 18, 255-281.
23. Farabet, C., Couprie, C., Najman, L., and Le Cun, Y. (2013). Learning hierarchical features for scene labeling, IEEE Trans. Patt. Anal. Mach. Intell. 35, 1915-1929. doi: 10.1109/TPAMI.2012.231
24. Farquhar, E., and Hasler, P. (2005). A bio-physically inspired silicon neuron. IEEE Trans. Circ. Syst. 52, 477-488. doi: 10.1109/TCSI.2004.842871
25. Furber, S., Galluppi, F., Temple, S., and Plana, L. (2014). The spinnaker project. Proc. IEEE 102, 652-665. doi: 10.1109/JPROC.2014.2304638
26. Giulioni, M., Camilleri, P., Dante, V., Badoni, D., Indiveri, G., Braun, J., et al. (2008). "A VLSI network of spiking neurons with plastic fully configurable "stop-learning" synapses," in International Conference on Electronics, Circuits, and Systems, ICECS (IEEE), 678-681.
27. Giulioni, M., Camilleri, P., Mattia, M., Dante, V., Braun, J., and Giudice, P. D. (2012). Robust working memory in an asynchronously spiking neural network realized in neuromorphic VLSI. Front. Neurosci. 5:149. doi: 10.3389/fnins.2011.00149
28. Hsieh, H.-Y., and Tang, K.-T. (2012). Vlsi implementation of a bio-inspired olfactory spiking neural network. IEEE Trans. Neural Netw. Learn. Syst. 23, 1065-1073. doi: 10.1109/TNNLS.2012.2195329
29. Hynna, K., and Boahen, K. (2007). Thermodynamically-equivalent silicon models of ion channels. Neural Comput. 19, 327-350. doi: 10.1162/neco.2007.19.2.327
30. Indiveri, G., Chicca, E., and Douglas, R. (2006). A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity. IEEE Trans. Neural Netw. 17, 211-221. doi: 10.1109/TNN.2005.860850
31. Indiveri, G., Linares-Barranco, B., Hamilton, T., van Schaik, A., Etienne-Cummings, R., Delbruck, T., et al. (2011). Neuromorphic silicon neuron circuits. Front. Neurosci. 5:73. doi: 10.3389/fnins.2011.00073
32. Izhikevich, E. (2003). Simple model of spiking neurons. IEEE Trans. Neural Netw. 14, 1569-1572. doi: 10.1109/TNN.2003.820440
33. Le, Q., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G., et al. (2012). "Building high-level features using large scale unsupervised learning," in International Conference in Machine Learning (Edinburgh).
34. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278-2324.
35. Linares-Barranco, B., and Serrano-Gotarredona, T. (2003). On the design and characterization of femtoampere current-mode circuits. IEEE J. Solid State Circ. 38, 1353-1363. doi: 10.1109/JSSC.2003.814415
36. Lisman, J., and Spruston, N. (2010). Questions about stdp as a general model of synaptic plasticity. Front. Synaptic Neurosci. 2:1-3. doi: 10.3389/fnsyn.2010.00140
37. Liu, S.-C. (2003). Analog VLSI circuits for short-term dynamic synapses. Eur. J. Appl. Signal Process. 7, 1-9. doi: 10.1155/S1110865703302094
38. Liu, S.-C., and Delbruck, T. (2010). Neuromorphic sensory systems. Curr. Opin. Neurobiol. 20, 288-295. doi: 10.1016/j.conb.2010.03.007
39. Liu, S.-C., Kramer, J., Indiveri, G., Delbruck, T., and Douglas, R. (2002). Analog VLSI: Circuits and Principles. Cambridge London: MIT Press.
40. Maass, W., Natschläger, T., and Markram, H. (2002). Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531-2560. doi: 10.1162/089976602760407955
41. Mahowald, M., and Douglas, R. (1991). A silicon neuron. Nature 354, 515-518.
42. Manohar, R. (2006). "Reconfigurable asynchronous logic," in Custom Integrated Circuits Conference (San Jose, CA: IEEE), 13-20.
43. Markram, H., Gerstner, W., and Sjöström, P. (2012). Spike-timing-dependent plasticity: a comprehensive overview. Front. Synaptic Neurosci. 4:2. doi: 10.3389/fnsyn.2012.00002
44. Masquelier, T., Guyonneau, R., and Thorpe, S. (2009). Competitive STDP-based spike pattern learning. Neural Comput. 21, 1259-1276. doi: 10.1162/neco.2008.06-08-804
45. Merolla, P., Arthur, J., Alvarez-Icaza, R., Cassidy, A., Sawada, J., Akopyan, F., et al. (2014). A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345, 668-673. doi: 10.1126/science.1254642
46. Mohamed, A., Dahl, G., and Hinton, G. (2012). Acoustic modeling using deep belief networks. IEEE Trans. Audio Speech Lang. Process. 20, 14-22. doi: 10.1109/TASL.2011.2109382
47. Neftci, E., Binas, J., Rutishauser, U., Chicca, E., Indiveri, G., and Douglas, R. (2013). Synthesizing cognition in neuromorphic electronic systems. Proc. Natl. Acad. Sci. U.S.A. 110, E3468-E3476. doi: 10.1073/pnas.1212083110
48. Neil, D., and Liu, S.-C. (2014). Minitaur, an event-driven FPGA-based spiking network accelerator. IEEE Trans. Very Large Scale Integr. Syst. 99, 1-1. doi: 10.1109/TVLSI.2013.2294916
49. Nessler, B., Pfeiffer, M., and Maass, W. (2009). "STDP enables spiking neurons to detect hidden causes of their inputs," in Advances in Neural Information Processing Systems (NIPS), Vol. 22, eds Y. Bengio, D. Schuurmans, J. Lafferty, C. I. Williams, and A. Culotta (Vancouver, BC: MIT Press), 1357-1365.
50. O'Connor, P., Neil, D., Liu, S.-C., Delbruck, T., and Pfeiffer, M. (2013). Real-time classification and sensor fusion with a spiking deep belief network. Front. Neurosci. 7:178. doi: 10.3389/fnins.2013.00178
51. Olshausen, B., and Field, D. (1997). Sparse coding with an overcomplete basis set: a strategy employed by vi? Vis. Res. 37, 3311-3326.
52. Petrovici, M., Vogginger, B., Müller, P., Breitwieser, O., Lundqvist, M., Muller, L., et al. (2014). Characterization and compensation of network-level anomalies in mixed-signal neuromorphic modeling platforms. PLoS ONE 9:e108590. doi: 10.1371/journal.pone.0108590
53. Ramakrishnan, S., Wunderlich, R., and Hasler, P. (2012). "Neuron array with plastic synapses and programmable dendrites," in Biomedical Circuits and Systems Conference (BioCAS) (Hsinchu: IEEE), 400-403.
54. Rasche, C., and Hahnloser, R. (2001). Silicon synaptic depression. Biol. Cybern. 84, 57-62. doi: 10.1007/s004220170004
55. Rigotti, M., Rubin, D. B. D., Morrison, S., Salzman, C., and Fusi, S. (2010). Attractor concretion as a mechanism for the formation of context representations. Neuroimage 52, 833-847. doi: 10.1016/j.neuroimage.2010.01.047
56. Ross, J., Burr, D., and Morrone, C. (1996). Suppression of the magnocellular pathway during saccades. Behav. Brain Res. 80, 1-8.
57. Rovere, G., Ning, Q., Bartolozzi, C., and Indiveri, G. (2014). "Ultra low leakage synaptic scaling circuits for implementing homeostatic plasticity in neuromorphic architectures," in International Symposium on Circuits and Systems, (ISCAS) (Melbourne VIC: IEEE), 2073-2076.
58. Rutishauser, U., and Douglas, R. (2009). State-dependent computation using coupled recurrent networks. Neural Comput. 21, 478-509. doi: 10.1162/neco.2008.03-08-734
59. Sarpeshkar, R., Lyon, R., and Mead, C. (1996). "An analog VLSI cochlea with new transconductance amplifiers and nonlinear gain control," in Proceedings of IEEE International Symposium on Circuits and Systems, Vol. 3 (Atlanta, GA: IEEE), 292-296.
60. Schapire, R., and Freund, Y. (2012). Boosting: Foundations and Algorithms. Cambridge, MA: MIT Press.
61. Schmuker, M., Pfeil, T., and Nawrot, M. (2014). A neuromorphic network for generic multivariate data classification. Proc. Natl. Acad. Sci. 111, 2081-2086. doi: 10.1073/pnas.1303053111
62. Schöner, G. (2007). Cambridge Handbook of Computational Cognitive Modeling, Chapter Dynamical Systems Approaches to Cognition. New York, NY: Cambridge University Press.
63. Senn, W. (2002). Beyond spike timing: the role of nonlinear plasticity and unreliable synapses. Biol. Cybern. 87, 344-355. doi: 10.1007/s00422-002-0350-1
64. Senn, W., and Fusi, S. (2005). Learning only when necessary: better memories of correlated patterns in networks with bounded synapses. Neural Comput. 17, 2106-2138. doi: 10.1162/0899766054615644
65. Serrano-Gotarredona, R., Serrano-Gotarredona, T., Acosta-Jimenez, A., Serrano-Gotarredona, C., Perez-Carrasco, J., Linares-Barranco, A., et al. (2008). On real-time aer 2d convolutions hardware for neuromorphic spike based cortical processing. IEEE Trans. Neural Netw. 19, 1196-1219. doi: 10.1109/TNN.2008.2000163
66. Sheik, S., Chicca, E., and Indiveri, G. (2012a). "Exploiting device mismatch in neuromorphic VLSI systems to implement axonal delays," in International Joint Conference on Neural Networks, IJCNN (Brisbane, QLD: IEEE), 1940-1945.
67. Sheik, S., Coath, M., Indiveri, G., Denham, S., Wennekers, T., and Chicca, E. (2012b). Emergent auditory feature tuning in a real-time neuromorphic VLSI system. Front. Neurosci. 6:17. doi: 10.3389/fnins.2012.00017
68. Sheik, S., Pfeiffer, M., Stefanini, F., and Indiveri, G. (2013). "Spatio-temporal spike pattern classification in neuromorphic systems," in Biomimetic and Biohybrid Systems, eds N. F. Lepora, A. Mura, H. G. Krapp, P. F. M. J. Verschure, and T. J. Prescott (Zurich: Springer), 262-273.
69. Stefanini, F., Sheik, S., Neftci, E., and Indiveri, G. (2014). Pyncs: a microkernel for high-level definition and configuration of neuromorphic electronic systems. Front. Neuroinform. 8:73. doi: 10.3389/fninf.2014.00073
70. van Schaik, A., Jin, C., and Hamilton, T. (2010). "A log-domain implementation of the Izhikevich neuron model," in International Symposium on Circuits and Systems, (ISCAS) (Paris: IEEE), 4253-4256.
71. van Schaik, A., and Meddis, R. (1999). Analog very large-scale integrated (VLSI) implementation of a model of amplitude-modulation sensitivity in the auditory brainstem. J. Acoust. Soc. Am. 105, 811.
72. Vapnik, V. (1995). The Nature of Statical Learning Theory. New York, NY: Springer-Verlag.
73. Wang, R., Cohen, G., Stiefel, K., Hamilton, T., Tapson, J., and van Schaik, A. (2013). An FPGA implementation of a polychronous spiking neural network with delay adaptation. Front. Neurosci. 7:14. doi: 10.3389/fnins.2013.00014
74. Wang, X. (1999). Synaptic basis of cortical persistent activity: the importance of NMDA receptors to working memory. J. Neurosci. 19, 9587-9603.
75. Xu, P., Horiuchi, T.-K., Sarje, A., and Abshire, P. (2007). "Stochastic synapse with short-term depression for silicon neurons," in Biomedical Circuits and Systems Conference (BIOCAS) (Montreal, QC: IEEE), 99-102.
76. Yu, T., Park, J., Joshi, S., Maier, C., and Cauwenberghs, G. (2012). "65k-neuron integrate-and-fire array transceiver with address-event reconfigurable synaptic routing," in Biomedical Circuits and Systems Conference (BioCAS) (Hsinchu: IEEE), 21-24.
77. Zaghloul, K., and Boahen, K. (2004). Optic nerve signals in a neuromorphic chip: Parts 1&2. IEEE Trans. Biomed. Circ. Syst. 51, 657-675. doi: 10.1109/TBME.2003.821039