Performance/price estimates for cortex-scale hardware: A design space exploration

Neural Networks

Volumn 24, Issue 3, 2011, Pages 291-304

  Zaveri, Mazad S ^a   Hammerstrom, Dan ^a  

^a Portland State University   (United States)

Author keywords

Design space; Digital; Efficiency; Hardware; Hardware spectrum; Mixed signal; Performance; Price; Processing node; Spiking neuron; Time multiplexing; Virtualization

Indexed keywords

DIGITAL; MIXED SIGNAL; PERFORMANCE; PRICE; PROCESSING NODE; SPIKING NEURON; TIME-MULTIPLEXING; VIRTUALIZATIONS;

DESIGN; HARDWARE; INTELLIGENT COMPUTING; MULTIPLEXING; NANOTECHNOLOGY; NEURAL NETWORKS; SPACE RESEARCH;

COMPUTER HARDWARE;

ARTICLE; ARTIFICIAL NEURAL NETWORK; BAYES THEOREM; BRAIN CORTEX; BRAIN NERVE CELL; COMPUTER SYSTEM; MATHEMATICAL MODEL; NANOTECHNOLOGY; PRIORITY JOURNAL; SPIKE;

CEREBRAL CORTEX; COMPUTER SIMULATION; COMPUTERS; EQUIPMENT DESIGN; HUMANS; NERVE NET; NEURAL NETWORKS (COMPUTER); USER-COMPUTER INTERFACE;

EID: 79951517328     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2010.12.003     Document Type: Article

References (55)

1. Albus J.S. Reverse engineering the brain. AAAI fall symp. series on biologically inspired cognitive architectures 2008, 5-14. AAAI Press.
2. Amsinck C.J., Di-Spigna N.H., Nackashi D.P., Franzon P.D. Scaling constraints in nanoelectronic random-access memories. Nanotechnology 2005, 16(10):2251-2260.
3. Ananthanarayanan, R., & Modha, D. S. (2007). Anatomy of a cortical simulator. Paper presented at the ACM/IEEE conf. on high performance networking and computing: supercomputing.
4. Bailey, J. L. (1993). A VLSI interconnect strategy for biologically inspired artificial neural networks. Ph.D. dissertation. Oregon grad. inst. of sci. & tech. Beaverton, OR.
5. Bailey, J., & Hammerstrom, D. (1988). Why VLSI implementations of associative VLCNs require connectionmultiplexing. Paper presented at the Proc. IEEE intl. conf. on neural networks.
6. Beiu, V. (2007). Grand challenges of nanoelectronics and possible architectural solutions: what do Shannon, von Neumann, Kolmogorov, and Feynman have to do with Moore. Paper presented at the Proc. 37th IEEE intl. symp. on multiple-valued logic.
7. Beiu, V., & Ibrahim, W. (2008). Does the brain really outperform rent's rule? Paper presented at the Proc. IEEE intl. symp. on circuits and systems.
8. Boahen K.A. Point-to-point connectivity between neuromorphic chips using address events. IEEE Transactions on Circuits and Systems Part II: Analog and Digital Signal Processing 2000, 47(5):416-434.
9. Braitenberg V., Schüz A. Cortex: statistics and geometry of neuronal connectivity 1998, Springer-Verlag, Berlin.
10. Chen G., Lau F.C.M. Layout of the cube-connected cycles without long wires. The Computer Journal 2001, 44(5):374-383.
11. Chen G., Lau F.C.M. Tighter layouts of the cube-connected cycles. IEEE Transactions on Parallel and Distributed Systems 2000, 11(2):182-191.
12. Diorio C., Hasler P., Minch B.A., Mead C. A complementary pair of four-terminal silicon synapses. Analog Integrated Circuits and Signal Processing 1997, 13(1-2):153-166.
13. Gao, C. (2008). Hardware architectures and implementations for associative memories-the building blocks of hierarchically distributed memories. Ph.D. dissertation. Portland State Univ. Portland, OR.
14. Gao C., Hammerstrom D. Cortical models onto CMOL and CMOS-architectures and performance/price. IEEE Transactions on Circuits and Systems I 2007, 54(11):2502-2515.
15. Gao, C., Zaveri, M. S., & Hammerstrom, D. (2008). CMOS/CMOL architectures for spiking cortical column. Paper presented at the Proc. IEEE world congress on computational intelligence-int. joint conf. on neural networks.
16. Gelder T.J.v. Distributed versus local representation. The MIT encyclopedia of the cognitive sciences 1999, 236-238. MIT Press, Cambridge, MA. R.A. Wilson, F.C. Keil (Eds.).
17. George, D., & Hawkins, J. (2005). A hierarchical Bayesian model of invariant pattern recognition in the visual cortex. Paper presented at the Proc. int. joint. conf. on neural networks.
18. Gerstner W. Spiking neurons. Pulsed neural networks 1998, 3-53. MIT Press, Cambridge, MA. W. Maass, C.M. Bishop (Eds.).
19. Gerstner W., Kistler W.M. Spiking neuron models: single neurons, populations, plasticity 2002, Cambridge University Press, Cambridge, UK.
20. Gielen, G., Rutenbar, R., Borkar, S., Brodersen, R., Chern, J. -H., & Naviasky, E., et al. (2006). Tomorrow's analog: just dead or just different? Paper presented at the 43rd ACM/IEEE design automation conf.
21. Granger R. Brain circuit implementation: high-precision computation from low-precision components. Replacement parts for the brain 2005, 277-294. MIT Press, Cambridge, MA. T. Berger, D. Glanzman (Eds.).
22. Hammerstrom D. A survey of bio-inspired and other alternative architectures. Nanotechnology: information technology-II 2008, Vol. 4:251-285. Wiley-VCH Verlag GmbH & Co., Weinheim, Germany.
23. Haron, N. Z., Hamdioui, S., & Cotofana, S. (2009). Emerging non-CMOS nanoelectronic devices-what are they? Paper presented at the 4th IEEE intl. conf. on nano/micro engineered and molecular systems.
24. Holler, M., Tam, S., Castro, H., & Benson, R. (1989). An electrically trainable artificial neural network (ETANN) with 10240 "floating gate" synapses. Paper presented at the int. joint conf. on neural networks.
25. Hooser S.D.V. Similarity and diversity in visual cortex: is there a unifying theory of cortical computation?. The Neuroscientist 2007, 13(6):639-656.
26. Indiveri G., Chicca E., Douglas R. A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity. IEEE Transactions on Neural Networks 2006, 17:211-221.
27. ITRS (2005). Executive summary-2005 edition: international technology roadmap for semiconductors. ITRS.
28. Izhikevich E.M. Which model to use for cortical spiking neurons?. IEEE Transactions on Neural Networks 2004, 15(5):1063-1070.
29. Jahnke, A., Roth, U., & Klar, H. (1996). A SIMD/dataflow architecture for a neurocomputer for spike-processing neural networks (NESPINN). Paper presented at the Proc. of microneuro.
30. Johansson C., Lansner A. Towards cortex sized artificial neural systems. Neural Networks 2007, 20(1):48-61.
31. Johansson C., Lansner A. Towards cortex sized attractor ANN. Biologically inspired approaches to advanced information technology 2004, 63-79. Springer, Berlin. A.J. Ijspeert, M. Masayuki, W. Naoki (Eds.).
32. Jo S.H., Kim K.-H., Lu W. High-density crossbar arrays based on a Si memristive system. Nano Letters 2009, 9(2):870-874.
33. Jo S.H., Kim K.-H., Lu W. Programmable resistance switching in nanoscale two-terminal devices. Nano Letters 2009, 9(1):496-500.
34. Kempter R., Gerstner W., van Hemmen J.L.V. Hebbian learning and spiking neurons. Physical Review E 1999, 59:4498-4514.
35. Laughlin S.B., Sejnowski T.J. Communication in neuronal networks. Science 2003, 301:1870-1874.
36. Lennie P. The cost of cortical computation. Current Biology 2003, 13(6):493-497.
37. Likharev K.K., Strukov D.V. CMOL: devices, circuits, and architectures. Introduction to molecular electronics 2005, 447-478. Springer, Berlin. G. Cuniberti, G. Fagas, K. Richter (Eds.).
38. Likharev, K. K., & Strukov, D. B. (2007). Prospects for the development of digital CMOL circuits. Paper presented at the Proc. int. symp. on nanoscale architectures.
39. Madappuram, B., Beiu, V., Kelly, P. M., & McDaid, L. J. (2008). On brain-inspired connectivity and hybrid network topologies. Paper presented at the Proc. IEEE intl. symp. on nanoscale architectures.
40. Maguire L.P., McGinnity T.M., Glackin B., Ghani A., Belatreche A., Harkin J. Challenges for large-scale implementations of spiking neural networks on FPGAs. Neurocomputing 2007, 71(1-3):13-29.
41. Markram H. The blue brain project. Nature Reviews-Neuroscience 2006, 7:153-160.
42. Mehrtash N., Jung D., Hellmich H.H., Schoenauer T., Lu V.T., Klar H. Synaptic plasticity in spiking neural networks (SP2INN): a system approach. IEEE Transactions on Neural Networks 2003, 14(5):980-992.
43. Murmann B. Digitally assisted analog circuits. IEEE Micro 2006, 26(2):38-47.
44. NAE (2008). Reverse-engineer the brain grand challenges for engineering. The US National Academy of Engineering (NAE) of the National Academies.
45. Preparata F.P., Vuillemin J The cube-connected cycles: a versatile network for parallel computation. Communications of the ACM 1981, 24(5):300-309.
46. Roth G., Dicke U. Evolution of the brain and intelligence. Trends in Cognitive Sciences 2005, 9(5):250-257.
47. Schüz A., Chaimow D., Liewald D., Dortenman M. Quantitative aspects of corticocortical connections: a tracer study in the mouse. Cerebral Cortex 2006, 16(10):1474-1486.
48. Schaefer M., Schoenauer T., Wolff C., Hartmann G., Klar H., Ruckert U. Simulation of spiking neural networks-architectures and implementations. Neurocomputing 2002, 48(1):647-679.
49. Schemmel, J., Fieres, J., & Meier, K. (2008). Wafer-scale integration of analog neural networks. Paper presented at the Proc. IEEE world congress on computational intelligence-int. joint conf. on neural networks.
50. Schemmel, J., Meier, K., & Mueller, E. (2004). A new VLSI model of neural microcircuits including spike time dependent plasticity. Paper presented at the Proc. of IEEE int. joint conf. on neural networks budapest.
51. Schoenauer T., Atasoy S., Mehrtash N., Klar H. Neuropipe-chip: a digital neuro-processor for spiking neural networks. IEEE Transactions on Neural Networks 2002, 13(1):205-213.
52. Sotiriadis P.P. Information capacity of nanowire crossbar switching networks. IEEE Transactions on Information Theory 2006, 52(7):3019-3032.
53. Wittie L.D. Communication structures for large networks of microcomputers. IEEE Transactions on Computers 1981, 30(4):264-273.
54. Zaveri, M. S. (2009). CMOL/CMOS hardware architectures and performance/price for Bayesian memory-the building block of intelligent systems. Ph.D. dissertation. Portland State University. Portland, OR.
55. Zaveri M.S., Hammerstrom D. CMOL/CMOS implementations of Bayesian polytree inference: digital & mixed-signal architectures and performance/price. IEEE Transactions on Nanotechnology 2010, 9(2). 10.1109/TNANO.2009.2028342.