Scalable communications for a million-core neural processing architecture

Journal of Parallel and Distributed Computing

Volumn 72, Issue 11, 2012, Pages 1507-1520

  Patterson, Cameron ^a   Garside, Jim ^a   Painkras, Eustace ^a   Temple, Steve ^a   Plana, Luis A ^a   Navaridas, Javier ^a   Sharp, Thomas ^a   Furber, Steve ^a  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

Author keywords

GALS; HPC; Low power; Network on Chip; Neuromorphic; Parallel architecture

Indexed keywords

GALS; HPC; LOW POWER; NETWORK ON CHIP; NEUROMORPHIC;

COMPUTER SIMULATION; COMPUTER SOFTWARE SELECTION AND EVALUATION; HARDWARE; MULTIPROCESSING SYSTEMS; NETWORK ARCHITECTURE; NEURAL NETWORKS; PARALLEL ARCHITECTURES; VLSI CIRCUITS;

COMPUTER ARCHITECTURE;

EID: 84866102225     PISSN: 07437315     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jpdc.2012.01.016     Document Type: Article

References (53)

1. Y. Ajima, S. Sumimoto, and T. Shimizu Tofu: a 6d mesh/torus interconnect for exascale computers Computer 42 2009 36 40
2. 13 synapses Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC'09 2009 ACM New York, NY, USA 63:1 63:12
3. ARM, AMBA open specifications, http://www.arm.com/products/system-ip/ amba/amba-open-specifications.php, 2011, [Online; accessed 12-Dec-2011].
4. ARM, Dhrystone and MIPs performance of ARM processors, http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.faqs/ka3885.html, 2011, [Online; accessed 12-Dec-2011].
5. K. Asanovic, J. Beck, J. Feldman, N. Morgan, J. Wawrzynek, A supercomputer for neural computation, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN), vol. 1, pp. 5-9.
6. W. Bainbridge, W. Toms, D. Edwards, S. Furber, Delay-insensitive, point-to-point interconnect using m-of-n codes, in: Ninth International Symposium on Asynchronous Circuits and Systems, 2003, Proceedings, pp. 132-140.
7. M. Bhuiyan, V. Pallipuram, M. Smith, Acceleration of spiking neural networks in emerging multi-core and GPU architectures, in: 2010 IEEE International Symposium on Parallel Distributed Processing, Workshops and PhD Forum (IPDPSW), pp. 1-8.
8. T. Binzegger, R.J. Douglas, and K.A. Martin Stereotypical bouton clustering of individual neurons in cat primary visual cortex Journal of Neuroscience 27 2007 12242 12254
9. C.M. Bishop Neural Networks for Pattern Recognition first ed. 1996 Oxford University Press USA
10. A. Bland, J. Rogers, R. Kendall, D. Kothe, G. Shipman, Jaguar: the world's most powerful computer, in: Cray user group 2009, Cray Inc.
11. K. Boahen Communicating neuronal ensembles between neuromorphic chips T.S. Lande, Neuromorphic Systems Engineering The Kluwer International Series in Engineering and Computer Science vol. 447 1998 Springer US 229 259
12. K. Boahen Point-to-point connectivity between neuromorphic chips using address events IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing 47 2000 416 434
13. S.H. Bokhari On the mapping problem IEEE Transactions on Computers 30 1981 207 214
14. D.M. Chapiro, Globally-asynchronous locally-synchronous systems, Stanford University, CA, Ph.D. Thesis, 1984.
15. A.P. Davidson, D. Brüderle, J.M. Eppler, J. Kremkow, E. Muller, D. Pecevski, L. Perrinet, and P. Yger PyNN: a common interface for neuronal network simulators Frontiers in Neuroinformatics 2 2009 1 10
16. P. Dayan, and L.F. Abbott Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems first ed. 2001 The MIT Press
17. W. Feng, K. Cameron, The green 500 list: environmentally responsible supercomputing, http://www.green500.org/, 2011, [Online; accessed 12-Dec-2011].
18. J. Fieres, J. Schemmel, K. Meier, Realizing biological spiking network models in a configurable wafer-scale hardware system, in: IEEE International Joint Conference on Neural Networks, 2008, IJCNN 2008, (IEEE World Congress on Computational Intelligence), pp. 969-976.
19. S. Furber, A. Brown, Biologically-inspired massively-parallel architectures - computing beyond a million processors, in: International Conference on Application of Concurrency to System Design, pp. 3-12.
20. S. Furber, and S. Temple Neural systems engineering Journal of the Royal Society Interface 4 2006 193 206
21. S. Furber, S. Temple, A. Brown, High-performance computing for systems of spiking neurons, in: AISB Workshop on Grand Challenge 5: Architecture of Brain and Mind, pp. 29-36.
22. F. Galluppi, A. Rast, S. Davies, and S. Furber A general-purpose model translation system for a universal neural chip K. Wong, B. Mendis, A. Bouzerdoum, Neural Information Processing. Theory and Algorithms Lecture Notes in Computer Science vol. 6443 2010 Springer Berlin/Heidelberg 58 65
23. A. Gara, J. Moreira, IBM Blue Gene supercomputer, IBM Research Report, 2011.
24. M.R. Garey, and D.S. Johnson Computers and Intractability: A Guide to the Theory of NP-Completeness 1979 W.H. Freeman & Co. New York, NY, USA
25. B. Han, T. Taha, Neuromorphic models on a GPGPU cluster, in: The 2010 International Joint Conference on Neural Networks (IJCNN).
26. H. Hellmich, M. Geike, P. Griep, P. Mahr, M. Rafanelli, H. Klar, Emulation engine for spiking neurons and adaptive synaptic weights, in: 2005 IEEE International Joint Conference on Neural Networks, 2005, IJCNN'05, Proceedings. vol. 5, pp. 3261-3266.
27. S. Herculano-Houzel The human brain in numbers: a linearly scaled-up primate brain Frontiers in Human Neuroscience 3 2009
28. A.L. Hodgkin, and A.F. Huxley A quantitative description of membrane current and its application to conduction and excitation in nerve The Journal of Physiology 117 1952 500 544
29. E.M. Izhikevich Polychronization: computation with spikes Neural Computation 18 2006 245 282
30. E.M. Izhikevich Simple model of spiking neurons IEEE Transactions on Neural Networks 14 2003 1569 1572
31. E. Izhikevich Which model to use for cortical spiking neurons? IEEE Transactions on Neural Networks 15 2004 1063 1070
32. X. Jin, F. Galluppi, C. Patterson, A. Rast, S. Davies, S. Temple, S. Furber, Algorithm and software for simulation of spiking neural networks on the multi-chip SpiNNaker system, in: The 2010 International Joint Conference on Neural Networks (IJCNN).
33. X. Jin, M. Luján, L.A. Plana, A.D. Rast, S.R. Welbourne, and S.B. Furber Efficient parallel implementation of multilayer backpropagation networks on SpiNNaker Proceedings of the 7th ACM International Conference on Computing Frontiers, CF'10 2010 ACM New York, NY, USA 89 90
34. W. Maass Networks of spiking neurons: the third generation of neural network models Neural Networks 10 1996 1659 1671
35. W. Maass Fast sigmoidal networks via spiking neurons Neural Computation 9 1997 279 304
36. H. Markram The Blue Brain project Natural Review of Neuroscience 7 2006 153 160
37. W. McCulloch, and W. Pitts A logical calculus of the ideas immanent in nervous activity The Bulletin of Mathematical Biology 5 1943 115 133
38. P. Merolla, J. Arthur, F. Akopyan, N. Imam, R. Manohar, and D.S. Modha A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45 nm R. Patel, T. Andre, A. Khan, Custom Integrated Circuits Conference, CICC 2011 IEEE 1 4
39. H. Meuer, E. Strohmaier, J. Dongarra, H. Simon, Top500 supercomputing sites, http://www.top500.org/, 2011, [Online; accessed 12-Dec-2011].
40. A. Mortara, and E.A. Vittoz A communication architecture tailored for analog VLSI artificial neural networks: intrinsic performance and limitations IEEE Transactions on Neural Networks 5 1994 459 466
41. J. Navaridas, M. Luján, J. Miguel-Alonso, L.A. Plana, and S. Furber Understanding the interconnection network of SpiNNaker Proceedings of the 23rd International Conference on Supercomputing, ICS'09 2009 ACM New York, NY, USA 286 295
42. J. Navaridas, L.A. Plana, J. Miguel-Alonso, M. Luján, and S.B. Furber SpiNNaker: impact of traffic locality, causality and burstiness on the performance of the interconnection network Proceedings of the 7th ACM International Conference on Computing Frontiers, CF'10 2010 ACM New York, NY, USA 11 20
43. M. Pearson, I. Gilhespy, K. Gurney, C. Melhuish, B. Mitchinson, M. Nibouche, and A. Pipe A real-time, FPGA based, biologically plausible neural network processor Artificial Neural Networks: Formal Models and Their Applications, ICANN 2005 Lecture Notes in Computer Science vol. 3697 2005 Springer Berlin/Heidelberg 755 756
44. P. Pfaerber, K. Asanovic, Parallel neural network training on MultiSpert, in: Proceedings of IEEE 3rd Int'l Conference on Algorithms and Architectures for Parallel Processing, ICA3PP, pp. 659-666.
45. L. Plana, J. Bainbridge, S. Furber, S. Salisbury, Y. Shi, J. Wu, An on-chip and inter-chip communications network for the SpiNNaker massively-parallel neural net simulator, in: Second ACM/IEEE International Symposium on Networks-on-Chip, 2008, NoCS 2008, pp. 215-216.
46. L. Plana, S. Furber, S. Temple, M. Khan, Y. Shi, J. Wu, and S. Yang A GALS infrastructure for a massively parallel multiprocessor Design & Test of Computers, IEEE 24 2007 454 463
47. A. Rast, F. Galluppi, S. Davies, L. Plana, C. Patterson, T. Sharp, D. Lester, and S. Furber Concurrent heterogeneous neural model simulation on real-time neuromimetic hardware Neural Networks 24 2011 961 978 Multi-Scale, Multi-Modal Neural Modeling and Simulation
48. A. Rast, F. Galluppi, X. Jin, S. Furber, The leaky integrate-and-fire neuron: a platform for synaptic model exploration on the SpiNNaker chip, in: The 2010 International Joint Conference on Neural Networks (IJCNN).
49. A. Rast, X. Jin, M. Khan, and S. Furber The deferred event model for hardware-oriented spiking neural networks M. Kppen, N. Kasabov, G. Coghill, Advances in Neuro-Information Processing Lecture Notes in Computer Science vol. 5507 2009 Springer Berlin/Heidelberg 1057 1064
50. T. Sharp, C. Patterson, S. Furber, Distributed configuration of massively-parallel simulation on SpiNNaker neuromorphic hardware, in: The 2011 International Joint Conference on Neural Networks (IJCNN), pp. 1099-1105.
51. Defense Sciences Office, systems of neuromorphic adaptive plastic scalable electronics (SYNAPSE), 2011, [Online; accessed 12-Dec-2011].
52. P. Worley Comparison of Cray XT3 and XT4 Scalability 2007 Cray Inc.
53. J. Wu, S. Furber, J. Garside, A programmable adaptive router for a GALS parallel system, in: Proceedings of the 2009 15th IEEE Symposium on Asynchronous Circuits and Systems, Async 2009, pp. 23-31.