Profiling heterogeneous multi-GPU systems to accelerate cortically inspired learning algorithms

Proceedings - 25th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2011

Volumn , Issue , 2011, Pages 906-920

  Nere, Andrew ^a   Hashmi, Atif ^a   Lipasti, Mikko ^a  

^a University of Wisconsin Madison   (United States)

Author keywords

cortical learning algorithms; CUDA; GPGPU; profiling systems

Indexed keywords

CORTICAL NETWORK; CUDA; ENERGY EFFICIENT; FAULT-TOLERANT; FUNCTIONAL PROPERTIES; GPGPU; HARDWARE ACCELERATORS; INITIAL DESIGN; INTELLIGENT LEARNING; MEMORY SYSTEMS; PARALLEL LEARNING ALGORITHMS; PROFILING SYSTEMS; PROFILING TOOLS; RUN-TIME PROFILING; SINGLE-THREADED;

ARCHITECTURAL DESIGN; BRAIN; DISTRIBUTED COMPUTER SYSTEMS; DISTRIBUTED PARAMETER NETWORKS; ENERGY EFFICIENCY; MAMMALS; MEMORY ARCHITECTURE; NEUROPHYSIOLOGY; OPTIMIZATION; PARALLEL ARCHITECTURES; PROGRAM PROCESSORS;

LEARNING ALGORITHMS;

EID: 80053254903     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2011.88     Document Type: Conference Paper

References (34)

1. E. Agullo, C. Augonnet, J. Dongarra, H. Ltaief, R. Namyst, S. Thibault, and S. Tomov. Faster, cheaper, better a hybridization methodology to develop linear algebra software for gpus. GPU Computing Gems, 2, 2010.
2. Billconan and Kavinguy. A neural network on gpu. http://www.codeproject. com/KB/graphics/GPUNN.aspx.x.
3. R. E. Brown and P. M. Milner. The legacy of Donald O. Hebb: more than the Hebb synapse. Nat Rev Neurosci, 4(12):1013-1019, Dec 2003.
4. S. Che, M. Boyer, J. Meng, D. Tarjan, J. Sheaffer, and K. Skadron. A performance study of general-purpose applications on graphics processors using cuda. Journal of Parallel and Distributed Computing, 68(10):1370-1380, October 2008.
5. G. Diamos and S. Yalamanchili. Harmony: an execution model and runtime for heterogeneous many core systems. In Proceedings of the 17th international symposium on High performance distributed computing, pages 197-200. ACM, 2008.
6. R. Douglas, C. Koch, M. Mahowald, K. Martin, and H. Suarez. Recurrent excitation in neocortical circuits. Sciense, 269(5226):981-985, 1995.
7. K. Grill-Spector, T. Kushnir, T. Hendler, S. Edelman, Y. Itzchak, and R. Malach. A sequence of object-processing stages revealed by fmri in the human occipital lobe. Hum. Brain Map., 6:316-328, 1998. (Pubitemid 28364526)
8. A. Hashmi and M. Lipasti. Cortical columns: Building blocks for intelligent systems. In Proceedings of the Symposium Series on Computational Intelligence, pages 21-28, 2009.
9. A. Hashmi and M. Lipasti. Discovering cortical algorithms. In Proceedings of the International Conference on Neural Computation (ICNC 2010), 2010.
10. A. Hashmi, A. Nere, and M. Lipasti. A case for neuromorphic isas. In Proceedings of the sixteenth edition of ASPLOS on Architectural support for programming languages and operating systems, ASPLOS '11, New York, NY, USA, 2011. ACM.
11. J. Hawkins and S. Blakeslee. On Intelligence. Henry Holt &Company, Inc., 2005.
12. D. Hubel and T. Wiesel. Receptive fields, binocular interactions and functional architecture in cat's visual cortex. Journal of Physiology, 160:106-154, 1962.
13. D. Hubel and T. Wiesel. Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology, 195:215-243, 1968.
14. H. Jang, A. Park, and K. Jung. Neural network implementation using cuda and openmp. In DICTA '08: Proceedings of the 2008 Digital Image Computing: Techniques and Applications, pages 155-161, Washington, DC, USA, 2008. IEEE Computer Society.
15. E. Kandel, J. Schwartz, and T. Jessell. Principles of Neural Science. McGraw-Hill, 4 edition, 2000.
16. A. Losonczy and J. Magee. Integrative properties of radial oblique dendrites in hippocampal ca1 pyramidal neurons. Neuron, 50:291-307, 2006.
17. V. Mountcastle. An organizing principle for cerebral function: The unit model and the distributed system. In G. Edelman and V. Mountcastle, editors, The Mindful Brain. MIT Press, Cambridge, Mass., 1978.
18. V. Mountcastle. The columnar organization of the neocortex. Brain, 120:701-722, 1997. (Pubitemid 27174104)
19. J. M. Nageswaran, N. Dutt, J. L. Krichmar, A. Nicolau, and A. Veidenbaum. Efficient simulation of large-scale spiking neural networks using cuda graphics processors. In IJCNN'09: Proceedings of the 2009 international joint conference on Neural Networks, pages 3201-3208, Piscataway, NJ, USA, 2009. IEEE Press.
20. J. M. Nageswaran, N. Dutt, J. L. Krichmar, A. Nicolau, and A. V. Veidenbaum. A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors. Neural Networks, 22(5-6):791-800, 2009. Advances in Neural Networks Research: IJCNN2009, 2009 International Joint Conference on Neural Networks.
21. A. Nere and M. Lipasti. Cortical architectures on a gpgpu. In GPGPU '10: Proceedings of the 3rd Workshop on General-Purpose Computation on Graphics Processing Units, pages 12-18, New York, NY, USA, 2010. ACM.
22. NVIDIA. Nvidia's next generation cuda compute architecture: Fermi.
23. NVIDIA. CUDA 3.1 Programming Guide. NVIDIA Corporation, 2701 San Toman Expressway, Santa Clara, CA 95050, USA, 2010.
24. R. Raina, A. Madhavan, and A. Y. Ng. Largescale deep unsupervised learning using graphics processors. In International Conf. on Machine Learning, 2009.
25. K. L. Rice, T. M. Taha, and C. N. Vutsinas. Scaling analysis of a neocortex inspired cognitive model on the cray xd1. J. Supercomput., 47(1):21-43, 2009.
26. D. Ringach. Haphazard wiring of simple receptive fields and orientation columns in visual cortex. J. Neurophysiol., 92(1):468-476, Jul 2004. (Pubitemid 38802269)
27. G. Roth and U. Dicke. Evolution of brain and intelligence. TRENDS in Cognitive Sciences, 5:250-257, 2005.
28. S. J. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Pearson Education, 2003.
29. S. Ryoo, C. I. Rodrigues, S. S. Baghsorkhi, S. S. Stone, D. B. Kirk, and W.-m. W. Hwu. Optimization principles and application performance evaluation of a multithreaded gpu using cuda. In PPoPP '08: Proceedings of the 13th ACM SIGPLAN Symposium on Principles and practice of parallel programming, pages 73-82, New York, NY, USA, 2008. ACM.
30. D. Schaa and D. Kaeli. Exploring the multiple-gpu design space. In IPDPS '09: Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing, pages 1-12, Washington, DC, USA, 2009. IEEE Computer Society.
31. A. Sillito, J. Cudeiro, and H. Jones. Always returning: feedback and sensory processing in visual cortex and thalamus. Trends Neurosci., 29(6):307-316, Jun 2006. (Pubitemid 43903023)
32. L. Swanson. Mapping the human brain: past, present, and future. Trends in Neurosciences, 18(11):471-474, 1995.
33. L. G. Valiant. A bridging model for parallel computation. Commun. ACM, 33(8):103-111, 1990.
34. S. Xiao and W. chun Feng. Inter-block gpu communication via fast barrier synchronization. In Parallel Distributed Processing (IPDPS), 2010 IEEE International Symposium on, pages 1-12, 2010.