Real-Time Scalable Cortical Computing at 46 Giga-Synaptic OPS/Watt with ∼100× Speedup in Time-to-Solution and ∼100,000× Reduction in Energy-to-Solution

International Conference for High Performance Computing, Networking, Storage and Analysis, SC

Volumn 2015-January, Issue January, 2014, Pages 27-38

  Cassidy, Andrew S ^a   Alvarez Icaza, Rodrigo ^a   Akopyan, Filipp ^a   Sawada, Jun ^a   Arthur, John V ^a   Merolla, Paul A ^a   Datta, Pallab ^a   Tallada, Marc Gonzalez ^a   Taba, Brian ^a   Andreopoulos, Alexander ^a   Amir, Arnon ^a   Esser, Steven K ^a   Kusnitz, Jeff ^a   Appuswamy, Rathinakumar ^a   Haymes, Chuck ^b   Brezzo, Bernard ^b   Moussalli, Roger ^b   Bellofatto, Ralph ^b   Baks, Christian ^b   Mastro, Michael ^b   Schleupen, Kai ^b   Cox, Charles E ^b   Inoue, Ken ^b   Millman, Steve ^b   Imam, Nabil ^c   McQuinn, Emmett ^a   Nakamura, Yutaka Y ^a   Vo, Ivan ^a   Guok, Chen ^d   Nguyen, Don ^e   Lekuch, Scott ^b   Asaad, Sameh ^b   Friedman, Daniel ^b   Jackson, Bryan L ^a   Flickner, Myron D ^a   Risk, William P ^a   Manohar, Rajit ^c   Modha, Dharmendra S ^a   more..

^a IBM RESEARCH   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

^c Department of Obstetrics Gynecology   (United States)

^d IBM   (United States)

^e Posthumous ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPLEX NETWORKS; COMPUTATIONAL EFFICIENCY; NEURAL NETWORKS; RECURRENT NEURAL NETWORKS; SCALABILITY;

BRAIN-INSPIRED; COMPUTER VISION APPLICATIONS; EVENT-DRIVEN; NEUMANN ARCHITECTURE; NEURAL NETWORK SIMULATIONS; ORDERS OF MAGNITUDE; REAL TIME; TIME-TO-SOLUTION;

COMPUTER VISION;

EID: 84936934295     PISSN: 21674329     EISSN: 21674337     Source Type: Conference Proceeding    
DOI: 10.1109/SC.2014.8     Document Type: Conference Paper

References (66)

1. D. S. Modha, R. Ananthanarayanan, S. K. Esser, A. Ndirango, A. J. Sherbondy, and R. Singh, "Cognitive computing," Communications of the ACM, vol. 54, no. 8, pp. 62-71, 2011.
2. D. S. Modha and R. Singh, "Network architecture of the long distance pathways in the macaque brain," Proceedings of the National Academy of the Sciences USA, vol. 107, no. 30, pp. 13 485-13 490, 2010.
3. A. S. Cassidy, P. Merolla, J. V. Arthur, S. Esser, B. Jackson, R. Alvarez-Icaza, P. Datta, J. Sawada, T. M. Wong, V. Feldman et al., "Cognitive computing building block: A versatile and efficient digital neuron model for neurosynaptic cores," in International Joint Conference on Neural Networks (IJCNN). IEEE, 2013.
4. R. Ananthanarayanan and D. S. Modha, "Anatomy of a cortical simulator," in Supercomputing 07, 2007.
5. R. Ananthanarayanan, S. K. Esser, H. D. Simon, and D. S. Modha, "The cat is out of the bag: cortical simulations with 109 neurons, 1013 synapses," in Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis. IEEE, 2009, pp. 1-12.
6. R. Preissl, T. M. Wong, P. Datta, M. Flickner, R. Singh, S. K. Esser, W. P. Risk, H. D. Simon, and D. S. Modha, "Compass: A scalable simulator for an architecture for cognitive computing," in Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis. IEEE Computer Society Press, 2012, p. 54.
7. T. M. Wong, R. Preissl, P. Datta, M. Flickner, R. Singh, S. K. Esser, E. McQuinn, R. Appuswamy, W. P. Risk, H. D. Simon et al., "1014," IBM Research Divsion, Research Report RJ10502, 2012.
8. P. Merolla, J. Arthur, F. Akopyan, N. Imam, R. Manohar, and D. S. Modha, "A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm," in Custom Integrated Circuits Conference (CICC). IEEE, 2011.
9. E. McQuinn, P. Datta, M. Flickner, W. Risk, D. Modha, T. Wong, S. Raghav, and R. Appuswamy, "International science & engineering visualization challenge," Science, vol. 339, no. 6119, pp. 512-513, 2013.
10. C. Mead, Analog VLSI and neural systems. Boston, MA: Addison-Wesley, 1989.
11. W. D. Hillis, The connection machine. The MIT Press, 1989.
12. P. A. Merolla, J. V. Arthur, R. Alvarez-Icaza, A. S. Cassidy, J. Sawada, F. Akopyan, B. L. Jackson, N. Imam, C. Guo, Y. Nakamura et al., "A million spiking-neuron integrated circuit with a scalable communication network and interface," Science, 2014.
13. S. Borkar, "Thousand core chips: A technology perspective," in Proceedings of the 44th Annual Design Automation Conference (DAC). New York, NY, USA: ACM, 2007, pp. 746-749.
14. R. Haring, M. Ohmacht, T. Fox, M. Gschwind, D. Satterfield, K. Sugavanam, P. Coteus, P. Heidelberger, M. Blumrich, R. Wisniewski et al., "The IBM Blue Gene/Q compute chip," in IEEE Micro. IEEE Computer Society, 2012, pp. 48-60.
15. D. J. Kerbyson, K. J. Barker, D. S. Gallo, D. Chen, J. R. Brunheroto, K. D. Ryu, G. L.-T. Chiu, and A. Hoisie, "Tracking the performance evolution of Blue Gene systems," in ISC, 2013, pp. 317-329.
16. A. Gara, "The long term impact of codesign," in High Performance Computing, Networking, Storage and Analysis (SCC). IEEE, 2012, pp. 2212-2246.
17. D. J. Kerbyson, A. Vishnu, K. J. Barker, and A. Hoisie, "Codesign challenges for exascale systems: Performance, power, and reliability," Computer, vol. 44, no. 11, pp. 0037-43, 2011.
18. S. K. Esser, A. Andreopoulos, R. Appuswamy, P. Datta, D. Barch, A. Amir, J. Arthur, A. S. Cassidy, P. Merolla, S. Chandra et al., "Cognitive computing systems: Algorithms and applications for networks of neurosynaptic cores," in International Joint Conference on Neural Networks (IJCNN). IEEE, 2013.
19. A. Amir, P. Datta, W. Risk, A. S. Cassidy, J. A. Kusnitz, S. K. Esser, A. Andreopoulos, T. M. Wong, M. Flickner, R. Alvarez-Icaza et al., "Cognitive computing programming paradigm: A corelet language for composing networks of neuro-synaptic cores," in International Joint Conference on Neural Networks (IJCNN). IEEE, 2013.
20. C.-J. Wang, C.-H. Wu, and S. Sivasindaram, "Neural network simulation on shared-memory vector multiprocessors," in Proceedings of the ACM/IEEE Conference on Supercomputing, 1989, pp. 197-204.
21. S. Kunkel, T. C. Potjans, J. M. Eppler, H. E. E. Plesser, A. Morrison, and M. Diesmann, "Meeting the memory challenges of brain-scale network simulation," Frontiers in neuroinformatics, vol. 5, p. 35, 2012.
22. M. Gewaltig and M. Diesmann, "NEST (NEural Simulation Tool)," Scholarpedia, vol. 2, no. 4, p. 1430, 2007.
23. H. Miyazaki, Y. Kusano, H. Okano, T. Nakada, K. Seki, T. Shimizu, N. Shinjo, F. Shoji, A. Uno, and M. Kurokawa, "K computer: 8.162 petaflops massively parallel scalar supercomputer built with over 548k cores," in International Solid-State Circuits Conference (ISSCC). IEEE, 2012, pp. 192-194.
24. M. Djurfeldt, M. Lundqvist, C. Johansson, M. Rehn, O. Ekeberg, and A. Lansner, "Brain-scale simulation of the neocortex on the IBM Blue Gene/L supercomputer," IBM Journal of Research and Development, vol. 52, no. 1.2, pp. 31-41, Jan 2008.
25. H. Markram, "The Blue Brain Project," Nature Reviews Neuroscience, vol. 7, no. 2, pp. 153-160, February 2006.
26. R. D. Traub, D. Contreras, M. O. Cunningham, H. Murray, F. E. LeBeau, A. Roopun, A. Bibbig, W. B. Wilent, M. J. Higley, and M. A. Whittington, "Single-column thalamocortical network model exhibiting gamma oscillations, sleep spindles, and epileptogenic bursts," Journal of Neurophysiology, vol. 93, no. 4, pp. 2194-2232, 2005.
27. J. M. Nageswaran, N. Dutt, J. L. Krichmar, A. Nicolau, and A. V. Veidenbaum, "A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors," Neural Networks, vol. 22, no. 5, pp. 791-800, 2009.
28. R. Brette, M. Rudolph, T. Carnevale, M. Hines, D. Beeman, J. Bower, M. Diesmann, A. Morrison, P. Goodman, J. Harris, FrederickC. et al., "Simulation of networks of spiking neurons: A review of tools and strategies," Journal of Computational Neuroscience, vol. 23, no. 3, pp. 349-398, 2007.
29. G. Indiveri, B. Linares-Barranco, T. J. Hamilton, A. Van Schaik, R. Etienne-Cummings, T. Delbruck, S.-C. Liu, P. Dudek, P. Häfliger, S. Renaud et al., "Neuromorphic silicon neuron circuits," Frontiers in Neuroscience, vol. 5, 2011.
30. M. Mahowald and R. Douglas, "A silicon neuron," Nature, vol. 354, no. 6354, pp. 515-518, 1991.
31. R. H. Hahnloser, R. Sarpeshkar, M. A. Mahowald, R. J. Douglas, and H. S. Seung, "Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit," Nature, vol. 405, no. 6789, pp. 947-951, 2000.
32. B. Benjamin, P. Gao, E. McQuinn, S. Choudhary, A. Chandrasekaran, J.-M. Bussat, R. Alvarez-Icaza, J. Arthur, P. Merolla, and K. Boahen, "Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations," Proceedings of the IEEE, vol. 102, no. 5, pp. 699-716, May 2014.
33. R. Serrano-Gotarredona, M. Oster, P. Lichtsteiner, A. Linares-Barranco, R. Paz-Vicente, F. Gómez-Rodríguez, L. Camuñas-Mesa, R. Berner, M. Rivas-Pérez, T. Delbruck et al., "CAVIAR: A 45k neuron, 5M synapse, 12G connects/s AER hardware sensory-processing-learning-actuating system for high-speed visual object recognition and tracking," IEEE Transactions on Neural Networks, vol. 20, no. 9, 2009.
34. T. Yu, J. Park, S. Joshi, C. Maier, and G. Cauwenberghs, "65K-neuron integrate-and-fire array transceiver with address-event reconfigurable synaptic routing," in Biomedical Circuits and Systems Conference (Bio-CAS). IEEE, 2012, pp. 21-24.
35. J. Schemmel, A. Grubl, S. Hartmann, A. Kononov, C. Mayr, K. Meier, S. Millner, J. Partzsch, S. Schiefer, S. Scholze et al., "Live demonstration: A scaled-down version of the BrainScaleS wafer-scale neuromorphic system," in International Symposium on Circuits and Systems (ISCAS). IEEE, 2012, pp. 702-702.
36. E. Stromatias, F. Galluppi, C. Patterson, and S. Furber, "Power analysis of large-scale, real-time neural networks on SpiNNaker," in International Joint Conference on Neural Networks (IJCNN). IEEE, 2013, pp. 1-8.
37. T. Ishiyama, K. Nitadori, and J. Makino, "4.45 Pflops astrophysical N-body simulation on K computer: The gravitational trillion-body problem," in Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis. Los Alamitos, CA, USA: IEEE Computer Society Press, 2012.
38. D. E. Shaw, R. O. Dror, J. K. Salmon, J. Grossman, K. M. Mackenzie, J. A. Bank, C. Young, M. M. Deneroff, B. Batson, K. J. Bowers et al., "Millisecond-scale molecular dynamics simulations on Anton," in Proceedings of the International Conference on High Performance Computing Networking, Storage and Analysis. IEEE, 2009, pp. 1-11.
39. A. Rahimian, I. Lashuk, S. Veerapaneni, A. Chandramowlishwaran, D. Malhotra, L. Moon, R. Sampath, A. Shringarpure, J. Vetter, R. Vuduc et al., "Petascale direct numerical simulation of blood flow on 200k cores and heterogeneous architectures," in Proceedings of the International Conference on High Performance Computing Networking, Storage and Analysis. IEEE Computer Society, 2010, pp. 1-11.
40. Y. Hasegawa, J.-I. Iwata, M. Tsuji, D. Takahashi, A. Oshiyama, K. Minami, T. Boku, F. Shoji, A. Uno, M. Kurokawa et al., "First-principles calculations of electron states of a silicon nanowire with 100,000 atoms on the K computer," in Proceedings of the International Conference on High Performance Computing Networking, Storage and Analysis. ACM, 2011.
41. M. Eisenbach, C.-G. Zhou, D. M. Nicholson, G. Brown, J. Larkin, and T. C. Schulthess, "A scalable method for ab initio computation of free energies in nanoscale systems," in Proceedings of the International Conference on High Performance Computing Networking, Storage and Analysis. ACM, 2009, p. 64.
42. V. N. Gamezo, A. M. Khokhlov, E. S. Oran, A. Y. Chtchelkanova, and R. O. Rosenberg, "Thermonuclear supernovae: Simulations of the deflagration stage and their implications," Science, vol. 299, no. 5603, pp. 77-81, 2003.
43. A. Nakano, R. K. Kalia, P. Vashishta, T. J. Campbell, S. Ogata, F. Shimojo, and S. Saini, "Scalable atomistic simulation algorithms for materials research," Scientific Programming, vol. 10, no. 4, pp. 263-270, 2002.
44. K. Pingali and A. Rogers, "Compiling for locality." in ICPP, 1990, pp. 142-146.
45. V. B. Mountcastle, "Modality and topographic properties of single neurons of cat's somatic sensory cortex." Journal of Neurophysiology, vol. 20, no. 4, p. 408, 1957.
46. S. B. Laughlin and T. J. Sejnowski, "Communication in neuronal networks," Science, vol. 301, no. 5641, pp. 1870-1874, 2003.
47. W. Gropp, E. Lusk, and A. Skjellum, Using MPI: portable parallel programming with the message-passing interface. MIT press, 1999.
48. L. Dagum and R. Menon, "OpenMP: an industry standard API for shared-memory programming," Computational Science & Engineering, IEEE, vol. 5, no. 1, pp. 46-55, 1998.
49. W. J. Dally and C. L. Seitz, "Deadlock-free message routing in multiprocessor interconnection networks," IEEE Transactions on Computers, vol. 100, no. 5, pp. 547-553, 1987.
50. V. J. Wedeen, D. L. Rosene, R. Wang, G. Dai, F. Mortazavi, P. Hagmann, J. H. Kaas, and W.-Y. I. Tseng, "The geometric structure of the brain fiber pathways," Science, vol. 335, no. 6076, pp. 1628-1634, 2012.
51. G. S. Banavar, "An application framework for compositional modularity," Ph.D. dissertation, The University of Utah, 1995.
52. P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Computer Vision and Pattern Recognition, 2001.
53. T. Ojala, M. Pietikainen, and T. Maenpaa, "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 971-987, 2002.
54. A. Andreopoulos and J. K. Tsotsos, "50 years of object recognition: Directions forward," Computer Vision and Image Understanding, vol. 117, no. 8, pp. 827-891, 2013.
55. "NeoVision2 dataset," http://ilab.usc.edu/neo2/dataset/, 2013.
56. R. Kasturi, D. Goldgof, R. Ekambaram, G. Pratt, E. Krotkov, D. D. Hackett, Y. Ran, Q. Zheng, R. Sharma, M. Anderson et al., "Performance evaluation of neuromorphic-vision object recognition algorithms," in Proc. 22nd International Conference on Pattern Recognition (ICPR'14), Aug 2014.
57. R. W. Wisniewski, "BlueGene/Q: Architecture, codesign; path to exascale," http://projects.csail.mit.edu/caos/2012-01-25-caos-bgq-v1-ed.pdf.
58. M. Gilge et al., IBM System Blue Gene Solution Blue Gene/Q Application Development. IBM Redbooks, 2013.
59. G. Lakner and B. Knudson, "IBM system Blue Gene solution: Blue Gene/Q system administration. IBM redbooks, june 2012."
60. "Performance application programming interface," http://icl.cs.utk.edu/ papi.
61. "Intel 64 and IA-32 Architectures Software Developer's Manual, Volume 3B: System Programming Guide, Part 2," http://download.intel.com/ products/processor/manual/253669.pdf.
62. A. Gara, M. A. Blumrich, D. Chen, G. L.-T. Chiu, P. Coteus, M. E. Giampapa, R. A. Haring, P. Heidelberger, D. Hoenicke, G. V. Kopcsay et al., "Overview of the Blue Gene/L system architecture," IBM J. Res. Devel., vol. 49, pp. 195-212, 2005.
63. S. Kamil, L. Oliker, A. Pinar, and J. Shalf, "Communication requirements and interconnect optimization for high-end scientific applications," IEEE Transactions on Parallel and Distributed Systems, vol. 21, no. 2, pp. 188-202, 2010.
64. J. S. Vetter, V. Tipparaju, W. Yu, and P. C. Roth, "HPC interconnection networks: The key to exascale computing," in Advances in Parallel Computing: High Speed and Large Scale Scientific Computing. IOS Press, 2009, vol. 18, pp. 95-106.
65. C. E. Leiserson, "Fat-trees: universal networks for hardware-efficient supercomputing," IEEE Transactions on Computers, vol. 100, no. 10, pp. 892-901, 1985.
66. G. Pratt, S.Ward, J. Nguyen, and J. Pezaris, "The diamond interconnect," MIT Technical Report, Tech. Rep., 1993.