Robust working memory in an asynchronously spiking neural network realized with neuromorphic VLSI

Frontiers in Neuroscience

Volumn , Issue FEB, 2012, Pages

  Giulioni, Massimiliano ^a   Camilleri, Patrick ^b   Dante, Maurizio Mattia ^a   Braun, Jochen ^b   Del Giudice, Paolo ^a,c  

^a ISTITUTO SUPERIORE DI SANITÀ   (Italy)

^b OTTO VON GUERICKE UNIVERSITY   (Germany)

^c INFN SEZIONE DI ROMA   (Italy)

Author keywords

Attractor dynamics; Neuromorphic chips; Spiking neurons; Stochastic dynamics; Working memory

Indexed keywords

ARTICLE; INTEGRATED CIRCUIT; NERVE CELL EXCITABILITY; NERVE CELL NETWORK; SYNAPTIC EFFICACY; WORKING MEMORY;

EID: 84862187288     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2012.00149     Document Type: Article

References (81)

1. Amit, D. J. (1989).Modeling Brain Function. Cambridge: Cambridge University Press.
2. Amit, D. J. (1995). The hebbian paradigm reintegrated, local reverberations as internal representation. Behav. Brain Sci. 18, 617-657.
3. Amit, D. J., and Brunel, N. (1997). Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cereb. Cortex 7, 237-252.
4. Amit, D. J., and Mongillo, G. (2003). Spike-driven synaptic dyanamics generating working memory states. Neural. Comput. 15, 565-596.
5. Bamford, S. A., Murray, D. A., and Willshaw, D. J. (2010). Large developing receptive fields using a distributed and locally reprogrammable address-event receiver. IEEE Trans. Neural Netw. 21, 286-304.
6. Boahen, K. A. (2007). Neurogrid Project. Available at: http://www.stanford.edu/group/brainsinsilicon/goals.html
7. Boahen, K. A. (2000). Point-to-point connectivity between neuromorphic chips using address-events. IEEE Trans. Circuits Syst. II 47, 416-434.
8. Boahen, K. A. (2005). Neuromorphic microchips. Sci. Am. 292, 56-63.
9. Brader, J. M., Senn, W., and Fusi, S. (2007). Learning real-world stimuli in a neural network with spike-driven synaptic dynamics. Neural. Comput. 19, 2881-2912.
10. Braun, J., andMattia, M. (2010).Attractors end noise: twin drivers of decisions andmultistability. Neuroimage 52, 740-751.
11. Brunel, N., and Hakim, V. (1999). Fast global oscillations in networks of integrate-and-fire neurons with low firing rates. Neural. Comput. 11, 1621-1671.
12. Brunel, N., and Sergi, S. (1998). Firing frequency of integrate-and-fire neurons with the finite synaptic time constants. J. Theor. Biol. 195, 87-95.
13. Camilleri, P., Giulioni, M., Mattia, M., Braun, J., and Del Giudice, P. (2010). "Self-sustained activity in attractor networks using neuromorphic VLSI," in Neural Networks (IJCNN), The 2010 International Joint Conference, Barcelona, 1-6.
14. Cassidy, A., Andreou, A. G., and Georgiou, J. (2011). "Design of a one million neuron single FPGA neuromorphic system for real-time multimodal scene analysis," in Information Sciences and Systems Conference (CISS), Baltimore, 1-6.
15. Chan, V., Jin, C. T., and van Schaik, A. (2010). Adaptive sound localisation with a silicon cochlea pair. Front. Neurosci. 4:196. doi:10.3389/fnins.2010.00196
16. Chan, V., Liu, S. C., and van Schaik, A. (2007). AER EAR: a matched silicon cochlea pair with address event representation interface. IEEE Trans. Circuits Syst. I 54, 48-59.
17. Chicca, E., Dante, V., Whatley, A. M., Lichtsteiner, P., Delbruck, T., Indiveri, G., Del Giudice, P., and Douglas, R. J. (2007). Multi-chip pulse based neuromorphic systems: a general communication infrastructure and a specific application example. IEEE Trans. Circuits Syst. I 54, 981-993.
18. Choi, T. Y. W., Merolla, P. A., Arthur, J. V., Boahen, K. A., and Shi, B. E. (2005).Neuromorphic implementation of orientation hypercolumns. IEEE Trans. Circuits Syst. I 56, 1049-1060.
19. Chow, S. S., Romo, R., and Brody, C. D. (2009). Context-dependent modulation of functional connectivity: secondary somatosensory cortex to prefrontal cortex connections in two-stimulus-interval discrimination tasks. J. Neurosci. 29, 7238-7245.
20. Corradi, F. (2011). Dinamica ad attrattori in una rete di neuroni spiking su chip VLSI neuromorfi (unpublished). Master's thesis, University of Rome 'Sapienza', Rome.
21. D'Andreagiovanni, M., Dante, V., Del Giudice, P., Mattia, M., and Salina, G. (2001)."Emergent asynchronous, irregular firing in a deterministic analog VLSI recurrent network," in Proceedings of the World Congress on Neuroinformatics, ed. F. Rattey (Vienna:AGESIM/ASIMVerlag), 478-486.
22. Dante, V., Del Giudice, P., and Whatley, A. M. (2005). The Neuromorphic Engineer Newsletter. Available at: http://neural.iss.infn.it/Papers/Pages_5-6_from_NME3.pdf
23. Deco, G., and Rolls, T. S. (2005). Sequential memory: a putative neural and synaptic dynamical mechanism. J. Cogn. Neurosci. 17, 294-307.
24. Del Giudice, P., Fusi, S., and Mattia, M. (2003). Modeling the formation of working memory with networks of integrate-and-fire neurons connected by plastic synapses. J. Physiol. Paris 97, 659-681.
25. Destexhe, A., and Contreras, D. (2006). Neuronal computations with stochastic network states. Science 314, 85-90.
26. Durstewitz, D., and Deco, G. (2008). Computational significance of transient dynamics in cortical networks. Eur. J. Neurosci. 27, 217-227.
27. Federici, L. (2011). Dinamica stocastica di decisione in una rete di neuroni distribuita su chip VLSI (unpublished). Master's thesis, University of Rome 'Sapienza', Rome.
28. Fox, M. D., and Raichle, M. E. (2007). Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging. Nat. Rev. Neurosci. 8, 700-711.
29. Furber, S., and Brown, A. (2009). "Biologically-inspired massivelyparallel architectures - computing beyond a million processors," in Proceedings of ASCD International Conference on Application of Concurrency to SystemDesign, Augsburg, 3-12.
30. Furman, M., and Wang, X.-J. (2008). Similarity effect and optimal control of multiple-choice decision making. Neuron 60, 1153-1168.
31. Fusi, S., Annunziato, M., Badoni, D., Salamon, A., and Amit, D. J. (2000). Spike-driven synaptic plasticity: theory, simulation, VLSI implementation. Neural. Comput. 12, 2227.
32. Fusi, S., Asaad, W. F., Miller, E. K., and Wang, X.-J. (2007). A neural circuit model of flexible sensorimotor mapping: learning and forgetting on multiple timescales. Neuron 54, 319-333.
33. Fusi, S., and Mattia, M. (1999). Collective behavior of networks with linear(VLSI) integrate-andfire neurons. Neural. Comput. 11, 633.
34. Gigante, G., Mattia, M., Braun, J., and Del Giudice, P. (2009). Bistable perception modeled as competing stochastic integrations at two levels. PLoS Comput. Biol. 5, e1000430. doi:10.1371/journal.pcbi.1000430
35. Giulioni, M. (2008). Networks of Spiking Neurons and Plastic Synapses: Implementation and Control. St. Julian's, Malta. Available at: http:// neural.iss.infn.it/Papers/PhD_thesis.pdf
36. Giulioni, M., Camilleri, P., Dante, V., Badoni, D., Indiveri, G., Braun, J., and Del Giudice, P. (2008). "A VLSI network of spiking neurons with plastic fully configurable stop-learning synapses," in Proceedings of IEEE International Conference on Electronics Circuits and Systems, 678-681.
37. Grinvald, A., Arieli, A., Tsodyks, M., and Kenet, T. (2003). Neuronal assemblies: single cortical neurons are obedient members of a huge orchestra. Biopolymers 68, 422-436.
38. Hamilton, T. J., Jin, C., van Schaik, A., and Tapson, J. (2008). An active 2-d silicon cochlea. IEEE Trans. Biomed. Circuits Syst. 2, 30-43.
39. Hasselmo, M. E., and McClelland, J. L. (1999). Neural models of memory. Curr. Opin. Neurobiol. 9, 184.
40. Holcman, D., and Tsodyks, M. (2006). The emergence of up and down states in cortical networks. PLoS Comput. Biol. 2, e23. doi:10.1371/journal.pcbi.0020023
41. Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. U.S.A. 79, 2554-2558.
42. Indiveri, G. (2008). Neuromorphic VLSI models of selective attention: from single chip vision sensors to multi-chip systems. Sensors 8, 5352-5375.
43. Indiveri, G., Chicca, E., and Douglas, R. (2006). A VLSI array of lowpower spiking neurons and bistable synapses with spike-timing dependent plasticity. IEEE Trans. Neural Netw. 17, 211-221.
44. Kim, D., Fu, Z. M., and Culurciello, E. (2009). A 1-mw CMOS temporal-difference AER sensor for wireless sensor networks. IEEE Trans. Electron Devices 56, 2586-2593.
45. Laing, C. R., and Chow, C. C. (2002). A spiking neuron model for binocular rivalry. J. Comput. Neurosci. 12, 39-53.
46. Lichtsteiner, P., Posch, C., and Delbruck, T. (2008). A 128?128 120 dB 15 ?s latency asynchronous temporal contrast vision sensor. IEEE J. Solid State Circuits 43, 566-576.
47. Liu, S. C., and Delbruck, T. (2010).Neuromorphic sensory systems. Curr. Opin. Neurobiol. 20, 1-8.
48. Lukashin, A. V., Amirikian, B. R., Mozhaev, V. L., Wilcox, G. L., and Georgopoulos, A. P. (1996). Modeling motor cortical operations by an attractor network of stochastic neurons. Biol. Cybern. 74, 255-261.
49. Machens, C. K., Romo, R., and Brody, C. D. (2005). Flexible control ofmutual inhibition: a neural model of twointerval discrimination supporting methods. Science 307, 1121-1124.
50. Mahowald, M. (1992). VLSI Analogs of Neuronal Visual Processing: A Synthesis of Form and Function. Ph.D. thesis, California Institute of Technology, Pasadena, CA.
51. Marti, D., Deco, G., Mattia, M., Gigante, G., and Del Giudice, P. (2008). A fluctuation-driven mechanism for slow decision processes in reverberant networks. PLoS ONE 3, e2534. doi:10.1371/journal.pone.0002534
52. Mascaro, M., and Amit, D. J. (1999). Effective neural response function for collective population states. Network 10, 251-373.
53. Massoud, T., and Horiuchi, T. (2011). A neuromorphic VLSI head direction cell system. IEEE Trans. Circuits Syst. 58, 150-163.
54. Mattia, M., and Del Giudice, P. (2002). Population dynamics of interacting spiking neurons. Phys. Rev. E Stat. Nonlin. SoftMatter Phys. 66, 051917.
55. Mattia, M., and Del Giudice, P. (2004). Finite-size dynamics of inhibitory and excitatory interacting spiking neurons. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 70, 052903.
56. Mattia, M., Ferraina, S., and Del Giudice, P. (2010). Dissociated multiunit activity and local field potentials: a theory inspired analysis of a motor decision task. Neuroimage 52, 812-823.
57. Mead, C. (1989). Analog VLSI and Neural Systems. Reading, Addison-Wesley.
58. Meier, K. (2011). Brain-ScaleS Project. Available at: http://brainscales.kip.uni-heidelberg.de/public/index.html
59. Merolla, P., Arthur, J., Akopyan, F., Imam, N., Manohar, R., andModha, D. S. (2011)."Adigital neurosynaptic core using embedded crossbar memory with 45pj per spike in 45nm," in IEEE Custom Integrated Circuits Conference, IEEE, San Jose.
60. Mongillo, G., Amit, D. J., and Brunel, N. (2003). Retrospective and prospective persistent activity induced by hebbian learning in a recurrent cortical network. Eur. J. Neurosci. 18, 2011-2024.
61. Neftci, E., Chicca, E., Indiveri, G., Cook, M., and Douglas, R. J. (2010). "State-dependent sensory processing in networks of VLSI spiking neurons," in International Symposium on Circuits and Systems, ISCAS 2010 (IEEE), Paris, 2789-2792.
62. Neftci, E., Chicca, E., Indiveri, G., and Douglas, R. (2011). A systematic methods for configuring VLSI networks of spiking neurons. Neural. Comput. 23, 2457-2497.
63. Renart, A., Brunel, N., and Wang, X.-J. (2004). "Mean-field theory of irregularly spiking neuronal populations and working memory in recurrent cortical networks," Computational Neuroscience A Comprehensive Approach, ed J. Feng (Boca Raton: Chapman et Hall/Crc), 431-490.
64. Ringach, D. L. (2009). Spontaneous and driven cortical activity: implications for computation. Curr.Opin.Neurobiol. 19, 439-444.
65. Risken, H. (1989). The Fokker-Planck Equation. Berlin: Springer.
66. Seo, J., Brezzo, B., Liu, Y., Parker, B. D., Esser, S. K., Montoye, R. K., Rajendran, B., Tierno, J. A., Chang, L., Modha, D. S., and Friedman, D. J. (2011). "A 45nm CMOS neuromorphic chipwith a scalable architecture for learning in networks of spiking neurons," in IEEE Custom Integrated Circuits Conference, San Jose.
67. Serrano-Gotarredona, R., Oster, M., Lichtsteiner, P., Linares-Barranco, A., Paz-Vicente, R., and Gomez-Rodriguez, F. (2009). Caviar: a 45k neuron, 5m synapse, 12g connects/s AER hardware sensory processing learning actuating system for highspeed visual object recognition and tracking. Trans. Neural Netw. 20, 1417-1438.
68. Serrano-Gotarredona, R., Serrano-Gotarredona, T., Acosta-Jimenez, A., and Linares-Barranco, B. (2006). A neuromorphic corticallayer microchip for spike- based event processing vision systems. IEEE Trans. Circuits Syst. I 53, 2548-2566.
69. Shi, R. Z., and Horiuchi, T. K. (2007). A neuromorphic VLSI model of bat interaural level difference processing for azimuthal echolocation. IEEE Trans. Circuits Syst. I 54, 74-88.
70. Shu, Y., Hasenstaub, A., and McCormick, D. A. (2003). Turning on and off recurrent balanced cortical activity. Nature 423, 288-293.
71. Skarda, C. A., and Freeman, W. J. (1987). How brains make chaos in order to make sense of the world. Behav. Brain Sci. 10, 161-195.
72. Spiridon, M., and Gerstner, W. (1999). Noise spectrum and signal transmission through a population of spiking neurons. Network 10, 257-272.
73. Trappenberg, T. (2005). "Continuous attractor neural networks," in Recent Developments in Biologically Inspired Computing, eds L. N. de Castro and F. J. Von Zuben (IdeaGroup Publishing), Oxford, 398-425.
74. van Vreeswijk, C. A., and Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274, 1724-1726.
75. Vasilaki, E., Fusi, S., Wang, X.-J., and Senn, W. (2009). Learning flexible sensori-motor mappings in a complex network. Biol. Cybern. 100, 147-158.
76. Wang, X.-J. (2002). Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36, 955-968.
77. Wang, X.-J. (2008). Decision making in recurrent neuronal circuits. Neuron 60, 215-234.
78. Wen, B., and Boahen, K. (2009). A silicon cochlea with active coupling. IEEE Trans. Biomed. Circuits Syst. 3, 444-455.
79. Wong, K., Alexander, C. H., Shadlen, M. N., and Wang, X.-J. (2007). Neural circuit dynamics underlying accumulation of time-varying evidence during perceptual decision making. Front. Comput. Neurosci. 1:6. doi:10.3389/neuro.10.006.2007
80. Zaghloul, K. A., and Boahen, K. (2006). A silicon retina that reproduces signals in the optic nerve. J. Neural Eng. 3, 257-267.
81. Zipser, D., Kehoe, B., Littlewort, G., and Fuster, J. (1993). A spiking network model of short-termactivememory. J. Neurosci. 13, 3406-3420.