Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations

Proceedings of the IEEE

Volumn 102, Issue 5, 2014, Pages 699-716

  Benjamin, Ben Varkey ^a   Gao, Peiran ^a   McQuinn, Emmett ^b   Choudhary, Swadesh ^c   Chandrasekaran, Anand R ^d   Bussat, Jean Marie ^e   Alvarez Icaza, Rodrigo ^f   Arthur, John V ^f   Merolla, Paul A ^f   Boahen, Kwabena ^a  

^a Stanford University   (United States)

^b In Depth Inc   (United States)

^c INTEL CORPORATION   (United States)

^d Mad Street Den ^*  (India)

^e APPLE INC   (United States)

^f IBM ALMADEN RESEARCH CENTER   (United States)

Author keywords

Analog circuits; application specific integrated circuits; asynchronous circuits; brain modeling; computational neuroscience; interconnection networks; mixed analog digital integrated circuits; neural network hardware; Neuromorphic electronic systems

Indexed keywords

ANALOG CIRCUITS; APPLICATION SPECIFIC INTEGRATED CIRCUITS; BRAIN MODELS; ENERGY EFFICIENCY; FORESTRY; INTEGRATED CIRCUIT INTERCONNECTS; INTERCONNECTION NETWORKS (CIRCUIT SWITCHING); MICROPROCESSOR CHIPS; NEURAL NETWORKS;

ASYNCHRONOUS CIRCUITS; COMPUTATIONAL NEUROSCIENCE; ELECTRONIC SYSTEMS; MIXED ANALOG-DIGITAL INTEGRATED CIRCUITS; NEURAL NETWORK HARDWARE;

BRAIN;

DESIGN; ELECTRIC CIRCUITS; FORESTRY; NEURAL NETWORKS;

EID: 84900521434     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2014.2313565     Document Type: Article

References (76)

1. C. Eliasmith and O. Trujillo "The use and abuse of large-scale brain models," Current Opinion Neurobiol. vol. 25 pp. 1-6 2014.
2. C. Eliasmith T. C. Stewart X. Choo T. Bekolay T. DeWolf Y. Tang C. Tang and D. Rasmussen "A large-scale model of the functioning brain," Science vol. 338 no. 6111 pp. 1202-1205 Nov. 2012.
3. S. Herculano-Houzel "Scaling of brain metabolism with a fixed energy budget per neuron: Implications for neuronal activity plasticity and evolution," PLoS One vol. 6 no. 3 2011 e17514.
4. H. Markram "The human brain project," Sci. Amer. vol. 306 no. 6 pp. 50-55 May 2012.
5. P. Kogge "The tops in flops," IEEE Spectrum vol. 48 no. 2 pp. 48-54 Feb. 2011.
6. S. B. Furber D. R. Lester L. A. Plana J. D. Garside E. Painkras S. Temple and A. D. Brown "Overview of the SpiNNaker system architecture," IEEE Trans. Comput. vol. 62 no. 12 pp. 2454-2467 Dec. 2013.
7. S. B. Furber F. Gallupi S. Temple and L. A. Plana "The SpiNNaker project: A massively-parallel computer architecture for neural simulations," Proc. IEEE vol. 102 no. 5 May 2014 DOI: 10.1109/JPROC.2014. 2304638.
8. P. Merolla J. Arthur F. Akopyan N. Imam R. Manohar and D. Modha "A digital neurosynaptic core using embedded crossbar memory with 45 pJ per spike in 45 nm," in Proc. Custom Integr. Circuits Conf. 2011 DOI: 10.1109/CICC.2011.6055294.
9. J. Schemmel D. Brüderle A. Grübl M. Hock K. Meier and S. Millner "A wafer-scale neuromorphic hardware system for large-scale neural modeling," in Proc. IEEE Int. Symp. Circuits Syst. 2010 pp. 1947-1950.
10. R. Silver K. Boahen S. Grillner N. Kopell and K. L. Olsen "Neurotech for neuroscience: Unifying concepts organizing principles emerging tools," J. Neurosci. vol. 27 no. 44 pp. 11807-11819 2007.
11. C.Mead "Neuromorphic electronic systems," Proc. IEEE vol. 78 no. 10 pp. 1629-1636 Oct. 1990.
12. C. Mead Analog VLSI and Neural Systems. Boston MA USA: Addison-Wesley 1989.
13. C. Diorio P. Hasler B. Minch and C. Mead "A single-transistor silicon synapse," IEEE Trans. Electron Devices vol. 43 no. 11 pp. 1972-1980 Nov. 1996.
14. B. Linares-Barranco T. Serrano-Gotarredona and R. Serrano-Gotarredona "Compact low-power calibration mini-DACs for neural arrays with programmable weights," IEEE Trans. Neural Netw. vol. 14 no. 5 pp. 1207-1216 Sep. 2003.
15. A. Cassidy and A. Andreou "Dynamical digital silicon neurons," in Proc. Biomed. Circuits Syst. Conf. 2008 pp. 289-292.
16. A. S. Cassidy J. Georgiou and A. G. Andreou "Design of silicon brains in the nano-CMOS era: Spiking neurons learning synapses and neural architecture optimization," Neural Netw. vol. 45 pp. 4-26 2013.
17. M. Sivilotti M. Emerling and C. Mead "A novel associative memory implemented using collective computation," in Proc. Chapel Hill Conf. Very Large Scale Integr. 1985 pp. 329-339.
18. K. A. Boahen P. O. Pouliquen A. G. Andreou and R. E. Jenkins "A heteroassociative memory using current-mode MOS analog VLSI circuits," IEEE Trans. Circuits Syst. vol. 36 no. 5 pp. 747-755 May 1989.
19. M. Sivilotti "Wiring considerations in analog VLSI systems with application to field-programmable networks," Ph.D. dissertation Comput. Science Dept. California Inst. Technol. Pasadena CA USA 1991.
20. M. Mahowald An Analog VLSI System for Stereoscopic Vision. Boston MA USA: Kluwer 1994.
21. K. Boahen "Point-to-point connectivity between neuromorphic chips using address events," IEEE Trans. Circuits Syst. II Analog Digit. Signal Process. vol. 47 no. 5 pp. 416-434 May 2000.
22. M. Yasunaga N. Masuda M. Yagyu M. Asai M. Yamada and A. Masaki "Design fabrication and evaluation of a 5-inch wafer-scale neural network LSI composed on 576 digital neurons," in Proc. Int. Joint Conf. Neural Netw. 1990 pp. 527-535.
23. D. Hammerstrom "A VLSI architecture for high-performance low-cost on-chip learning," in Proc. Int. Joint Conf. Neural Netw. 1990 pp. 537-544.
24. J. G. Elias "Artificial dendritic trees," Neural Comput. vol. 5 no. 4 pp. 648-664 1993.
25. S. R. Deiss R. J. Douglas and A. M. Whatley "A pulse-coded communications infrastructure for neuromorphic systems," Pulsed Neural Networks W. Maass and C. M. Bishop Eds. Cambridge MA USA: MIT Press 1999 ch. 6 pp. 157-178.
26. R. Vogelstein U. Mallik J. Vogelstein and G. Cauwenberghs "Dynamically reconfigurable silicon array of spiking neurons with conductance-based synapses," IEEE Trans. Neural Netw. vol. 18 no. 1 pp. 253-265 Jan. 2007.
27. J. Lin P. Merolla J. Arthur and K. Boahen "Programmable connections in neuromorphic grids," in Proc. IEEE Midwest Symp. Circuits Syst. 2006 pp. 80-84.
28. L. Camunas-Mesa C. Zamarreno-Ramos A. Linares-Barranco A. J. Acosta-Jimenez T. Serrano-Gotarredona and B. Linares-Barranco "An event-driven multi-kernel convolution processor module for event-driven vision sensors," IEEE J. Solid-State Circuits vol. 47 no. 2 pp. 504-517 Feb. 2012.
29. S. Moradi and G. Indiveri "An event-based neural network architecture with an asynchronous programmable synaptic memory," IEEE Trans. Biomed. Circuits Syst. vol. 8 no. 1 pp. 98-107 Feb. 2014.
30. R. Z. Shi and T. K. Horiuchi "A summating exponentially-decaying CMOS synapse for spiking neural systems," in Advances in Neural Information Processing Systems (NIPS). Cambridge MA USA: MIT Press 2004.
31. J. V. Arthur and K. A. Boahen "Synchrony in silicon: The gamma rhythm," IEEE Trans. Neural Netw. vol. 18 no. 6 pp. 1815-1825 Nov. 2007.
32. B. V. Benjamin J. V. Arthur P. Gao P. Merolla and K. Boahen "A superposable silicon synapse with programmable reversal potential," in Proc. Int. Conf. IEEE Eng. Med. Biol. Soc. Aug. 2012 pp. 771-774.
33. D. R. Frey "Log-domain filtering: An approach to current-mode filtering," Inst. Electr. Eng. Proc. G Circuits Devices Syst. vol. 140 no. 6 pp. 406-416 1993.
34. P. Merolla and K. A. Boahen "A recurrent model of orientation maps with simple and complex cells," in Advances in Neural Information Processing Systems (NIPS). Cambridge MA USA: MIT Press 2004 pp. 995-1002.
35. T. Choi P. Merolla J. Arthur K. Boahen and B. Shi "Neuromorphic implementation of orientation hypercolumns," IEEE Trans. Circuits Syst. I Reg. Papers vol. 52 no. 3 pp. 1049-1060 Jun. 2005.
36. A. G. Andreou and K. A. Boahen "Translinear circuits in subthreshold MOS," Analog Integr. Circuits Signal Process. vol. 9 no. 2 pp. 141-166 1996.
37. R. Sarpeshkar "Analog versus digital: Extrapolating from electronics to neurobiology," Neural Comput. vol. 10 no. 7 pp. 1601-1638 1998.
38. C. Mead and L. Conway Introduction to VLSI Systems. Reading MA USA: Addison-Wesley 1980.
39. C. Zamarreno-Ramos A. Linares-Barranco T. Serrano-Gotarredona and B. Linares-Barranco "Multicasting mesh AER: A scalable assembly approach for reconfigurable neuromorphic structured AER systems. Application to ConvNets," IEEE Trans. Biomed. Circuits Syst. vol. 7 no. 1 pp. 82-102 Feb. 2012.
40. S. Joshi S. Deiss M. Arnold J. Park T. Yu and G. Cauwenberghs "Scalable event routing in hierarchical neural array architecture with global synaptic connectivity," in Proc. Int. Workshop Cellular Nanoscale Netw. Appl. Feb. 2010 DOI: 10.1109/CNNA.2010.5430296.
41. S. Scholze S. Schiefer J. Partzsch S. Hartmann C. Mayr S. Höppner H. Eisenreich S. Henker B. Vogginger and R. Schüffny "VLSI implementation of a 2.8 Gevent/s packet based AER interface with routing and event sorting functionality," Front. Neurosci. vol. 5 no. 117 2011 DOI: 10.3389/fnins.2011.00117.
42. P. Merolla J. V. Arthur R. Alvarez-Icaza J.-M. Bussat and K. Boahen "A multicast tree router for multichip neuromorphic systems," IEEE Trans. Circuits Syst. I Reg. Papers vol. 61 no. 3 pp. 820-833 Mar. 2014.
43. W. Dally and B. Towles Principles and Practices of Interconnection Networks. San Francisco CA USA: Morgan Kaufmann 2004.
44. R. Boppana S. Chalasani and C. Raghavendra "On multicast wormhole routing in multicomputer networks," in Proc. IEEE Symp. Parallel Distrib. Process. 1994 pp. 722-729.
45. A. Despain and D. Patterson "X-tree: A structured multiprocessor computer architecture," in Proc. IEEE Symp. Comput. Architecture 1978 pp. 144-151.
46. S. Browning "The tree machine: A highly concurrent computing environment," California Inst. Technol. Pasadena CA USA Tech. Rep. Caltech-CS-TR-80-3760 1980.
47. E. Horowitz and A. Zorat "The binary tree as an interconnection network: Applications to multiprocessor systems and VLSI," IEEE Trans. Comput. vol. C-30 no. 4 pp. 247-253 Apr. 1981.
48. D. Vainbrand and R. Ginosar "Scalable network-on-chip architecture for configurable neural networks," Microprocess. Microsyst. vol. 35 no. 2 pp. 152-166 2011.
49. G. B. Ermentrout and N. Kopell "Parabolic bursting in an excitable system coupled with a slow oscillation," SIAM J. Appl. Math. vol. 46 no. 2 pp. 233-253 Apr. 1986.
50. A. Destexhe Z. F. Mainen and T. J. Sejnowski "An efficient method for computing synaptic conductances based on a kinetic model of receptor binding," Neural Comput. vol. 6 no. 1 pp. 14-18 Jan. 1994.
51. D. I. Feinstein "The hexagonal resistive network and the circular approximation," California Inst. Technol. Pasadena CA USA Tech. Rep. CaltechCSTR:1988.cs-tr-88-07 1988.
52. E. Vittoz and J. Fellrath "CMOS analog integrated circuits based on weak inversion operation," IEEE J. Solid-State Circuits vol. 12 no. 3 pp. 224-231 Jun. 1977.
53. T. Delbrück and A. V. Schaik "Bias current generators with wide dynamic range," Analog Integr. Circuits Signal Process. vol. 43 no. 3 pp. 247-268 Jun. 2005.
54. P. Gao B. V. Benjamin and K. Boahen "Dynamical system guided mapping of quantitative neuronal models onto neuromorphic hardware," IEEE Trans. Circuits Syst. I Reg. Papers vol. 59 no. 10 pp. 2383-2394 Oct. 2012.
55. J. V. Arthur and K. Boahen "Silicon-neuron design: A dynamical systems approach," IEEE Trans. Circuits Syst. I Reg. Papers vol. 58 no. 5 pp. 1034-1043 May 2011.
56. K. M. Hynna and K. Boahen "Thermodynamically equivalent silicon models of voltage-dependent ion channels," Neural Comput. vol. 19 no. 2 pp. 327-350 2007.
57. K. A. Zaghloul and K. A. Boahen "An on-off log domain circuit that recreates adaptive filtering in the retina," IEEE Trans. Circuits Syst. I Reg. Papers vol. 52 no. 1 pp. 99-107 Jan. 2005.
58. A. J. Annema B. Nauta R. van Langevelde and H. Tuinhout "Analog circuits in ultra-deep-submicron CMOS," IEEE J. Solid-State Circuits vol. 40 no. 1 pp. 132-143 Jan. 2005.
59. J. Lin and K. Boahen "A delay-insensitive address-event link," in Proc. IEEE Int. Symp. Asynchron. Circuits Syst. (ASYNC) 2009 pp. 55-62.
60. A. Martin "Programming in VLSI: From communicating processes to delay-insensitive circuits," in Developments in Concurrency and Communication. Reading MA USA: Addison-Wesley 1991 pp. 1-64.
61. A. Martin and M. Nystrom "Asynchronous techniques for system-on-chip design," Proc. IEEE vol. 94 no. 6 pp. 1089-1120 Jun. 2006.
62. K. A. Boahen "A burst-mode word-serial address-event link I: Transmitter design," IEEE Trans. Circuits Syst. I Reg. Papers vol. 51 no. 7 pp. 1269-1280 Jul. 2004.
63. K. A. Boahen "A burst-mode word-serial address-event link II: Receiver design," IEEE Trans. Circuits Syst. I Reg. Papers vol. 51 no. 7 pp. 1281-1291 Jul. 2004.
64. K. A. Boahen "A burst-mode word-serial address-event link III: Analysis and test results," IEEE Trans. Circuits Syst. I Reg. Papers vol. 51 no. 7 pp. 1292-1300 Jul. 2004.
65. C. Mead and T. Delbruck "Scanners for visualizing activity of analog VLSI circuitry," Analog Integr. Circuits Signal Process. vol. 1 no. 2 pp. 93-106 1991.
66. N. Imam and R. Manohar "Address-event communication using token-ring mutual exclusion," in Proc. IEEE Int. Symp. Asynchron. Circuits Syst. 2011 pp. 99-108.
67. A. Chandrasekaran and K. Boahen "A 1-change-in-4 delay-insensitive interchip link," in Proc. IEEE Int. Symp. Circuits Syst. May 2010 pp. 3216-3219.
68. P. B. McGee M. Y. Agyekum M. A. Mohamed and S. M. Nowick "A level-encoded transition signaling protocol for high-throughput asynchronous global communication," in Proc. IEEE Int. Symp. Asynchron. Circuits Syst. 2008 pp. 116-127.
69. S. Choudhary S. Sloan S. Fok A. Neckar E. Trautmann P. Gao T. Stewart C. Eliasmith and K. Boahen "Silicon neurons that compute," in Proc. Int. Conf. Artif. Neural Netw. 2012 pp. 121-128.
70. J. M. Rabaey A. P. Chandrakasan and B. Nikolic Digital Integrated Circuits 2nd ed. Englewood Cliffs NJ USA: Prentice-Hall 2002.
71. J. Schemmel private communication 2013.
72. J. V. Arthur and K. Boahen "Learning in silicon: Timing is everything," in Advances in Neural Information Processing Systems (NIPS). Cambridge MA USA: MIT Press 2006 pp. 75-82.
73. S. Brink S. Nease P. Hasler S. Ramakrishnan R. Wunderlich A. Basu and B. Degnan "A learning-enabled neuron array IC based upon transistor channel models of biological phenomena," IEEE Trans. Biomed. Circuits Syst. vol. 7 no. 1 pp. 71-81 Feb. 2013.
74. D. H. Goldberg G. Cauwenberghs and A. G. Andreou "Probabilistic synaptic weighting in a reconfigurable network of VLSI integrate-and-fire neurons," Neural Netw. vol. 14 no. 6 pp. 781-793 2001.
75. R. J. Vogelstein F. Tenore R. Philipp M. S. Adlerstein D. H. Goldberg and G. Cauwenberghs "Spike timing-dependent plasticity in the address domain," in Advances in Neural Information Processing Systems (NIPS). Cambridge MA USA: MIT Press 2002 pp. 1147-1154.
76. S. H. Jo T. Chang I. Ebong B. B. Bhadviya P. Mazumder and W. Lu "Nanoscale memristor device as synapse in neuromorphic systems," Nano Lett. vol. 10 no. 4 pp. 1297-1301 2010.