HRLSim: A high performance spiking neural network simulator for GPGPU clusters

IEEE Transactions on Neural Networks and Learning Systems

Volumn 25, Issue 2, 2014, Pages 316-331

  Minkovich, Kirill ^a,b   Thibeault, Corey M ^a,c   O'Brien, Michael John ^a,d   Nogin, Aleksey ^a   Cho, Youngkwan ^a   Srinivasa, Narayan ^a  

^a Hughes Aircraft Co   (United States)

^b MENTOR GRAPHICS CORPORATION   (United States)

^c University of Nevada   (United States)

^d UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Distributed graphical processing units (GPUs) programming; general purpose GPU (GPGPU); large scale; neuron simulation; spike timing dependent plasticity (STDP); spiking neural simulation

Indexed keywords

GENERAL PURPOSE GPU; GRAPHICAL PROCESSING UNIT (GPUS); LARGE SCALE; NEURAL SIMULATIONS; SPIKE-TIMING-DEPENDENT PLASTICITY;

INTELLIGENT AGENTS; PROGRAM PROCESSORS; SIMULATORS; TOOLS;

NEURAL NETWORKS;

ACTION POTENTIAL; ANIMAL; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; COMPUTER SIMULATION; HUMAN; NERVE CELL; NERVE CELL NETWORK; PHYSIOLOGY; SYNAPSE;

ACTION POTENTIALS; ANIMALS; COMPUTER SIMULATION; HUMANS; MODELS, NEUROLOGICAL; NERVE NET; NEURAL NETWORKS (COMPUTER); NEURONS; SYNAPSES;

EID: 84893657361     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2013.2276056     Document Type: Article

References (51)

1. G. Buzsáki, Rhythms of the Brain, 1st ed. Oxford, U.K.: Oxford Univ. Press, Aug. 2006.
2. B. A. Wilt, L. D. Burns, E. T. Wei Ho, K. K. Ghosh, E. A. Mukamel, and M. J. Schnitzer, "Advances in light microscopy for neuroscience," Annu. Rev. Neurosci., vol. 32, no. 1, pp. 435-506, 2009.
3. N. Srinivasa and J. Cruz-Albrecht, "Neuromorphic adaptive plastic scalable electronics: Analog learning systems," IEEE Pulse, vol. 3, no. 1, pp. 51-56, Jan. 2012.
4. K. Minkovich, N. Srinivasa, J. M. Cruz-Albrecht, Y. K. Cho, and A. Nogin, "Programming time-multiplexed reconfigurable hardware using a scalable neuromorphic compiler," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 6, pp. 889-901, Jun. 2012.
5. Z. Fan, F. Qiu, A. Kaufman, and S. Yoakum-Stover, "GPU cluster for high performance computing," in Proc. ACM/IEEE Conf. Supercomput., Nov. 2004, p. 47.
6. M. Hines and N. Carnevale, "Translating network models to parallel hardware in NEURON," J. Neurosci. Methods, vol. 169, no. 2, pp. 425-455, 2007.
7. M. L. Hines and N. T. Carnavale, "The neuron simulation environment," Neural Comput., vol. 9, no. 6, pp. 1179-1209, 1997. (Pubitemid 127469144)
8. J. Bower, D. Beeman, and M. Hucka, "GENESIS simulation system," The Handbook of Brain Theory and Neural Networks. Cambridge, MA, USA: MIT Press, 2002, pp. 475-478.
9. J. Bower and D. Beeman, The Book of GENESIS: Exploring Realistic Neural Models with the GEneral NEural SImulation System, 2nd ed. New York, NY, USA: Springer-Verlag, 1998.
10. M.-O. Gewaltig and M. Diesmann, "NEST (NEural Simulation Tool)," Scholarpedia, vol. 2, no. 4, p. 1430, 2007.
11. E. C. Wilson, "Parallel implementation of a large scale biologically realistic neocortical neural network simulator," M.S. thesis, Univ. Nevada, Reno, NV, USA, Aug. 2001.
12. C. Wilson, P. Goodman, and C. Harris, Jr., "Implementation of a biologically realistic parallel neocortical-neural network simulator," in Proc. SIAM Conf. Parallel Process. Sci. Comput., 2001, pp. 1-11.
13. D. Pecevski, T. Natschläger, and K. Schuch, "PCSIM: A parallel simulation environment for neural circuits fully integrated with python," Frontiers Neuroinf., vol. 3, no. 11, p. 15, May 2009.
14. R. Brette, M. Rudolph, T. Carnevale, M. Hines, D. Beeman, J. Bower, M. Diesmann, A. Morrison, P. Goodman, F. Harris, M. Zirpe, T. Natschlager, D. Pecevski, B. Ermentrout, M. Djurfeldt, A. Lansner, O. Rochel, T. Vieville, E. Muller, A. Davison, S. El Boustani, and A. Destexhe, "Simulation of networks of spiking neurons: A review of tools and strategies," J. Comput. Neurosci., vol. 23, no. 3, pp. 349-398, 2007. (Pubitemid 47566522)
15. J. M. Nageswaran, N. Dutt, J. L. Krichmar, A. Nicolau, and A. V. Veidenbaum, "A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors," Neural Netw., vol. 22, nos. 5-6, pp. 791-800, 2009.
16. M. Richert, J. M. Nageswaran, N. Dutt, and J. L. Krichmar, "An efficient simulation environment for modeling large-scale cortical processing," Frontiers Neuroinf., vol. 5, p. 19, Sep. 2011.
17. C. M. Thibeault, R. Hoang, and F. C. Harris, Jr., "A novel multi-GPU neural simulator," in Proc. 3rd BICoB, Mar. 2011.
18. E. Izhikevich, "Simple model of spiking neurons," IEEE Trans. Neural Netw., vol. 14, no. 6, pp. 1569-1572, Nov. 2003.
19. R. Scorcioni, "GPGPU implementation of a synaptically optimized, anatomically accurate spiking network simulator," in Proc. BSEC, May 2010, pp. 1-3.
20. J.-P. Tiesel and A. S. Maida, "Using parallel GPU architecture for simulation of planar I/F networks," in Proc. INNS, 2009, pp. 3118-3123.
21. B. Han and T. M. Taha, "Acceleration of spiking neural network based pattern recognition on NVIDIA graphics processors," Appl. Opt., vol. 49, no. 10, pp. B83-B91, 2010.
22. J. Igarashi, O. Shouno, T. Fukai, and H. Tsujino, "Real-time simulation of a spiking neural network model of the basal ganglia circuitry using general purpose computing on graphics processing units," Neural Netw., vol. 24, no. 9, pp. 950-960, 2011.
23. P. Richmond, B. Lars, G. Michele, and V. E. Richmond, "Democratic population decisions result in robust policy-gradient learning: A parametric study with GPU simulations," PLoS ONE, vol. 6, no. 5, p. e18539, May 2011.
24. A. Fidjeland and M. Shanahan, "Accelerated simulation of spiking neural networks using GPUs," in Proc. IJCNN, Jul. 2010, pp. 1-8.
25. D. Yudanov, M. Shaaban, R. Melton, and L. Reznik, "GPU-based simulation of spiking neural networks with real-time performance amp; high accuracy," in Proc. IJCNN, Jul. 2010, pp. 1-8.
26. A. Nere, A. Hashmi, and M. Lipasti, "Profiling heterogeneous multi-GPU systems to accelerate cortically inspired learning algorithms," in Proc. IEEE IPDPS, May 2011, pp. 906-920.
27. R. Y. de Camargo, L. Rozante, and S. W. Song, "A multi-GPU algorithm for large-scale neuronal networks," Concurrency Comput., Pract. Exper., vol. 23, no. 6, pp. 556-572, 2011.
28. N. Burkitt, "A review of the integrate-and-fire neuron model: I. Homogeneous synaptic input," Biol. Cybern., vol. 95, pp. 1-19, Jun. 2006.
29. E. M. Izhikevich, Dynamical Systems in Neuroscience. Cambridge, MA, USA: MIT Press, 2007.
30. E. Izhikevich, "Which model to use for cortical spiking neurons?" IEEE Trans. Neural Netw., vol. 15, no. 5, pp. 1063-1070, Sep. 2004.
31. S. Song, K. D. Miller, and L. F. Abbott, "Competitive Hebbian learning through spike-timing-dependent synaptic plasticity," Nature Neurosci., vol. 3, no. 9, pp. 919-926, 2000.
32. A. P. Davison, D. Brüderle, J. M. Eppler, J. Kremkow, E. Muller, D. Pecevski, L. Perrinet, and P. Yger, "PyNN: A common interface for neuronal network simulators," Frontiers Neuroinf., vol. 2, pp. 1-11, Jan. 2009.
33. J. Hickey and A. Nogin, "Omake: Designing a scalable build process," in Fundamental Approaches to Software Engineering (Lecture Notes in Computer Science), vol. 3922, L. Baresi and R. Heckel, Eds. Berlin, Germany: Springer-Verlag, 2006, pp. 63-78. (Pubitemid 44019302)
34. A. Pope and P. Langley, "CASTLE: A framework for integrating cognitive models into virtual environments," AAAI, Palo Alto, CA, USA, Tech. Rep. FS-08-04 (152), 2008.
35. O. Michel, "Webots: Symbiosis between virtual and real mobile robots," in Virtual Worlds (Lecture Notes in Computer Science), vol. 1434, J.-C. Heudin, Ed. Berlin, Germany: Springer-Verlag, 1998, pp. 254-263. (Pubitemid 128101751)
36. Python: Boost Documentation [Online]. Available: http://www.boost. org/libs/python/doc/
37. A. Morrison, C. Mehring, T. Geisel, A. Aertsen, and M. Diesmann, "Advancing the boundaries of high-connectivity network simulation with distributed computing," Neural Comput., vol. 17, no. 8, pp. 1776-1801, Aug. 2005. (Pubitemid 40864607)
38. H. Plesser, J. Eppler, A. Morrison, M. Diesmann, and M.-O. Gewaltig, "Efficient parallel simulation of large-scale neuronal networks on clusters of multiprocessor computers," in Euro-Par 2007 Parallel Processing (Lecture Notes in Computer Science), vol. 4641, A.-M. Kermarrec, L. Bougé, and T. Priol, Eds. Berlin, Germany: Springer-Verlag, 2007, pp. 672-681.
39. M. Migliore, C. Cannia, W. Lytton, H. Markram, and M. Hines, "Parallel network simulations with NEURON," J. Comput. Neurosci., vol. 21, no. 2, pp. 119-129, 2006. (Pubitemid 44249586)
40. K. A. Boahen, "Point-to-point connectivity between neuromorphic chips using address-events," IEEE Trans. Circuits Syst. II, vol. 47, no. 5, pp. 416-34, May 2000.
41. MVAPICH: MPI Over InfiniBand, 10GigE/iWARP and RoCE. Ohio State Univ., Columbus, OH, USA [Online]. Available: http://mvapich.cse.ohio-state.edu/ overview/mvapich2/
42. C. M. Thibeault, K. Minkovich, M. J. O'Brien, F. C. Harris, and N. Srinivasa, "Efficiently passing messages in distributed spiking neural network simulation," Frontiers Comput. Neurosci., vol. 7, p. 77, Jun. 2013.
43. A. Morrison, S. Straube, H. E. Plesser, and M. Diesmann, "Exact subthreshold integration with continuous spike times in discrete-time neural network simulations." Neural Comput., vol. 19, no. 1, pp. 47-79, 2007.
44. N. Srinivasa and Y. K. Cho, "A self-organizing spiking neural model for learning fault-tolerant spatio-motor transformations," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 10, pp. 1526-1538, Oct. 2012.
45. M. J. O'Brien and N. Srinivasa, "A spiking neural model for stable reinforcement of synapses based on multiple distal rewards," Neural Comput., vol. 25, no. 1, pp. 123-156, 2013.
46. A. Morrison, A. Aertsen, and M. Diesmann, "Spike-timing-dependent plasticity in balanced random networks," Neural Comput., vol. 19, no. 6, pp. 1437-1467, 2007.
47. S. Achard, R. Salvador, B. Whitcher, J. Suckling, and E. Bullmore, "A resilient, low-frequency, small-world human brain functional network with highly connected association cortical hubs." J. Neurosci., vol. 26, no. 1, pp. 63-72, 2006. (Pubitemid 43090914)
48. S. Kunkel, T. C. Potjans, J. M. Eppler, H. E. Plesser, A. Morrison, and M. Diesmann, "Meeting the memory challenges of brain-scale network simulation," Frontiers Neuroinf., vol. 5, no. 35, pp. 1-15, 2012.
49. Khronos. (2009). OpenCL-The Open Standard for Parallel Programming of Heterogeneous Systems, Beaverton, OH, USA [Online]. Available: http://www.khronous.org/opencl/
50. P. A. Merolla, J. V. Arthur, B. E. Shi, and K. A. Boahen, "Expandable networks for neuromorphic chips," IEEE Trans. Circuits Syst., vol. 54, no. 2, pp. 301-311, Feb. 2007. (Pubitemid 46444261)
51. J. Schemmel, A. Grübl, K. Meier, and E. Mueller, "Implementing synaptic plasticity in a VLSI spiking neural network model," in Proc. Int. Joint Conf. Neural Netw., 2006, pp. 1-6. (Pubitemid 351369277)