Neurostream: Scalable and Energy Efficient Deep Learning with Smart Memory Cubes

IEEE Transactions on Parallel and Distributed Systems

Volumn 29, Issue 2, 2018, Pages 420-434

  Azarkhish, Erfan ^a   Rossi, Davide ^a   Loi, Igor ^a   Benini, Luca ^a,b  

^a UNIVERSITY OF BOLOGNA   (Italy)

^b ETH ZURICH   (Switzerland)

Author keywords

convolutional neural networks; Hybrid memory cube; large scale deep learning; streaming floating point

Indexed keywords

BANDWIDTH; BUDGET CONTROL; COMPUTATION THEORY; CONVOLUTION; COST EFFECTIVENESS; DEEP LEARNING; DIGITAL ARITHMETIC; DYNAMIC RANDOM ACCESS STORAGE; EMBEDDED SYSTEMS; GEOMETRY; INTEGRATED CIRCUIT DESIGN; LEARNING SYSTEMS; NEURAL NETWORKS; RANDOM ACCESS STORAGE; STANDARDS; STORAGE ALLOCATION (COMPUTER);

CONVOLUTIONAL NEURAL NETWORK; FLOATING POINTS; HYBRID MEMORY; INDEXES; MEMORY MANAGEMENT; RANDOM ACCESS MEMORY; THREE-DIMENSIONAL DISPLAY;

ENERGY EFFICIENCY;

EID: 85030640525     PISSN: 10459219     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPDS.2017.2752706     Document Type: Article

References (61)

1. C. Szegedy, et al., "Going deeper with convolutions, "in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 1-9.
2. Y. Taigman, M. Yang, M. Ranzato, L. Wolf, "DeepFace: Closing the gap to human-level performance in face verification, "in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 1701-1708.
3. T.-Y. Lin, et al., Microsoft COCO: Common Objects in Context. Berlin, Germany: Springer, 2014.
4. S. Srinivas, et al., "A taxonomy of deep convolutional neural nets for computer vision, "Frontiers Robot. AI, vol. 2, 2016, Art. no. 36.
5. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition, "CoRR, vol. abs/1409.1556, 2014. [Online]. Available: Http://arxiv.org/abs/1409.1556
6. S. E. Kahou, et al., "Combining modality specific deep neural networks for emotion recognition in video, "in Proc. 15th ACM Int. Conf. Multimodal Interaction, 2013, pp. 543-550.
7. X. Li, et al., "Deep neural network for RFID-based activity recognition, "in Proc. 8th Wireless Students by Students Students Workshop, 2016, pp. 24-26.
8. R. Adolf, S. Rama, B. Reagen, G.-Y. Wei, D. Brooks, "Fathom: Reference workloads for modern deep learning methods, "CoRR, vol. abs/1608.06581, 2016. [Online]. Available: Http://arxiv.org/abs/1608.06581
9. S. Carrillo, et al., "Scalable hierarchical network-on-chip architecture for spiking neural network hardware implementations, "IEEE Trans. Parallel Distrib. Syst., vol. 24, no. 12, pp. 2451-2461, Dec. 2013.
10. J. Bilski and J. Smolag, "Parallel architectures for learning the RTRN and Elman dynamic neural networks, "IEEE Trans. Parallel Distrib. Syst., vol. 26, no. 9, pp. 2561-2570, Sep. 2015.
11. Y. Jia, et al., "Caffe: Convolutional architecture for fast feature embedding, "in Proc. 22nd ACM Int. Conf. Multimedia, 2014, pp. 675-678.
12. S. Chetlur, C. Woolley, P. Vandermersch, J. Cohen, J. Tran, B. Catanzaro, E. Shelhamer, "cuDNN: Efficient primitives for deep learning, "CoRR, vol. abs/1410.0759, 2014, http://arxiv.org/abs/1410.0759
13. C. Zhang, P. Li, G. Sun, Y. Guan, B. Xiao, J. Cong, "Optimizing FPGA-based accelerator design for deep convolutional neural networks, "in Proc. ACM/SIGDA Int. Symp. Field-Programmable Gate Arrays, 2015, 161-170.
14. C. Farabet, B. Martini, B. Corda, P. Akselrod, E. Culurciello, Y. LeCun, "NeuFlow: A runtime reconfigurable dataflow processor for vision, "in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Workshops, Jun. 2011, pp. 109-116.
15. C. Zhang, Z. Fang, P. Zhou, P. Pan, J. Cong, "Caffeine: Towards uniformed representation and acceleration for deep convolutional neural networks, "in Proc. IEEE/ACM Int. Conf. Comput.-Aided Des., Nov. 2016, pp. 1-8.
16. S. Han, et al., "EIE: Efficient inference engine on compressed deep neural network, "in Proc. 43rd Annu. Int. Symp. Comput. Archit., 2016, pp. 243-254.
17. T. Chen, et al., "A high-throughput neural network accelerator, "IEEE Micro, vol. 35, no. 3, pp. 24-32, May 2015.
18. L. Cavigelli and L. Benini, "A 803 GOp/s/W convolutional network accelerator, "IEEE Trans. Circuits Syst. Video Technol., vol. PP, no. 99, p. 1, 2016, doi: 10.1109/TCSVT.2016.2592330.
19. Z. Du, et al., "ShiDianNao: Shifting vision processing closer to the sensor, "SIGARCH Comput. Archit. News, vol. 43, no. 3, pp. 92-104, Jun. 2015.
20. L. Xu, D. P. Zhang, N. Jayasena, "Scaling deep learning on multiple in-memory processors, "in 3rd Workshop Near-Data Process., 2015, http://www.cs.utah.edu/wondp/tentative.html
21. D. Kim, J. Kung, S. Chai, S. Yalamanchili, S. Mukhopadhyay, "Neurocube: A programmable digital neuromorphic architecture with high-density 3D memory, "in Proc. 43rd Annu. Int. Symp. Comput. Archit., 2016, pp. 380-392.
22. P. Chi, et al., "PRIME: A novel processing-in-memory architecture for neural network computation in ReRAM-based main memory, "in Proc. 43rd Annu. Int. Symp. Comput. Architecture, 2016, pp. 27-39.
23. M. Gao, J. Pu, X. Yang, M. Horowitz, C. Kozyrakis, "TETRIS: Scalable and efficient neural network acceleration with 3D memory, "SIGARCH Comput. Archit. News, vol. 45, no. 1, pp. 751-764, Apr. 2017.
24. E. Azarkhish, D. Rossi, I. Loi, L. Benini, Design and Evaluation of a Processing-in-Memory Architecture for the Smart Memory Cube. Cham, Switzerland: Springer, 2016, pp. 19-31.
25. E. Azarkhish, C. Pfister, D. Rossi, I. Loi, L. Benini, "Logic-base interconnect design for near memory computing in the Smart Memory Cube, "IEEE Trans. Very Large Scale Integr. Syst., vol. 25, no. 1, pp. 210-223, Jan. 2017.
26. Hybrid Memory Cube Specification 2.1, Hybrid Memory Cube Consortium Std., 2015, http://hybridmemorycube.org/files/SiteDownloads/HMC-30GVSR-HMCC-Specification-Rev2.1-20151105.pdf
27. D. Kang, et al., "256Gb 3b/cell V-NAND flash memory with 48 stacked WL layers, "in Proc. IEEE Int. Solid-State Circuits Conf., Jan. 2016, pp. 130-131.
28. S. Williams, A. Waterman, D. Patterson, "Roofline: An insightful visual performance model for multicore architectures, "Commun. ACM, vol. 52, no. 4, pp. 65-76, Apr. 2009.
29. I. Goodfellow, Y. Bengio, A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016. [Online]. Available: Http://www.deeplearningbook.org.
30. L. Cavigelli, M. Magno, L. Benini, "Accelerating real-time embedded scene labeling with convolutional networks, "in Proc. 52nd ACM/EDAC/IEEE Des. Autom. Conf., 2015, pp. 1-6.
31. K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition, "CoRR, vol. abs/1512.03385, 2015, http://arxiv.org/abs/1512.03385
32. K. He, X. Zhang, S. Ren, J. Sun, "Identity mappings in deep residual networks, "CoRR, vol. abs/1603.05027, 2016, http://arxiv.org/abs/1603.05027
33. Large scale visual recognition challenge. [Online]. Available: Http://image-net.org/challenges/LSVRC/
34. P. H. Pham, D. Jelaca, C. Farabet, B. Martini, Y. LeCun, E. Culurciello, "NeuFlow: Dataflow vision processing system-on-achip, "in Proc. IEEE 55th Int. Midwest Symp. Circuits Syst., 2012, pp. 1044-1047.
35. M. Peemen, R. Shi, S. Lal, B. Juurlink, B. Mesman, H. Corporaal, "The neuro vector engine: Flexibility to improve convolutional net efficiency for wearable vision, "in Proc. Design, Autom. Test Eur. Conf. Exhib, Mar. 2016, pp. 1604-1609.
36. A. Haidar, J. Kurzak, P. Luszczek, "An improved parallel singular value algorithm and its implementation for multicore hardware, "in Proc. Int. Conf. High Perform. Comput., Netw., Storage Anal., 2013, pp. 1-12.
37. J. Gmez-Luna, et al., "In-place matrix transposition on GPUs, "IEEE Tran. Parallel Distrib. Syst., vol. 27, no. 3, pp. 776-788, Mar. 2016.
38. Y. H. Chen, T. Krishna, J. S. Emer, V. Sze, "Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks, "IEEE J. Solid-State Circuits, vol. 52, no. 1, pp. 127-138, Jan. 2017.
39. Caffe performance measurements on NVIDIA GPUs. (2016). [Online]. Available: Http://git.caffe.help/performance-hardware. html
40. S. Liu, et al., "Cambricon: An instruction set architecture for neural network, "in Proc. 43rd Annu. Int. Symp. Comput. Archit., 2016, pp. 393-405.
41. J. Murphy, "Deep learning benchmarks of NVIDIA tesla P100 PCIe, tesla K80, tesla M40 GPUs, "Jan. 2017, https://www. microway.com/hpc-tech-tips/deep-learning-benchmarksnvidiatesla-p100-16gb-pcie-tesla-k80-tesla-m40-gpus/
42. M. Abadi, et al., "TensorFlow: Large-scale machine learning on heterogeneous distributed systems, "CoRR, vol. abs/1603.04467, 2016, http://arxiv.org/abs/1603.04467
43. Rethinking computation: A processor architecture for machine intelligence. (2016). [Online]. Available: Https://goo.gl/gAWZGp
44. X. Meng, et al., "MLlib: Machine learning in apache spark, "J. Mach. Learn. Res., vol. 17, no. 1, pp. 1235-1241, Jan. 2016.
45. M. Grossman, M. Breternitz, V. Sarkar, "HadoopCL2: Motivating the design of a distributed, heterogeneous programming system with machine-learning applications, "IEEE Trans. Parallel Distrib. Syst., vol. 27, no. 3, pp. 762-775, Mar. 2016.
46. S. Mittal and J. S. Vetter, "A survey of software techniques for using non-volatile memories for storage and main memory systems, "IEEE Trans. Parallel Distrib. Syst., vol. 27, no. 5, pp. 1537-1550, May 2016.
47. M. Gautschi, et al., "A near-threshold RISC-V core with DSP extensions for scalable IoT endpoint devices, "CoRR, vol. abs/1608.08376, 2016. [Online]. Available: Http://arxiv.org/abs/1608.08376
48. D. Rossi, et al., "A 60 GOPS/W,-1.8 V to 0.9 V body bias ULP cluster in 28nm UTBB FD-SOI technology, "Solid-State Electron., vol. 117, pp. 170-184, 2016.
49. The parallel ultra low power (PULP) processing platform. (2016). [Online]. Available: Http://www.pulp-platform.org/
50. I. Loi, D. Rossi, G. Haugou, M. Gautschi, L. Benini, "Exploring multi-banked shared-L1 program cache on ultra-low power, tightly coupled processor clusters, "in Proc. 12th ACM Int. Conf. Comput. Frontiers, 2015, Art. no. 64.
51. E. Azarkhish, D. Rossi, I. Loi, L. Benini, "A modular shared L2 memory design for 3-D integration, "IEEE Trans. Very Large Scale Integr. Syst., vol. 23, no. 8, pp. 1485-1498, Aug. 2015.
52. SMCSim: A simulation framework for the smart memory cube (SMC). (2016). [Online]. Available: Https://iis-git.ee.ethz. ch/erfan.azarkhish/SMCSim
53. A. Giusti, D. C. Ciresan, J. Masci, L. M. Gambardella, J. Schmidhuber, "Fast image scanning with deep max-pooling convolutional neural networks, "CoRR, vol. abs/1302.1700, 2013. [Online]. Available: Http://arxiv.org/abs/1302.1700
54. Berkeley SoftFloat library. (2017). [Online]. Available: Http://www.jhauser.us/arithmetic/SoftFloat.html
55. G. Kim, J. Kim, J. H. Ahn, J. Kim, "Memory-centric system interconnect design with hybrid memory cubes, "in Proc. 22Nd Int. Conf. Parallel Architectures Compilation Techn., 2013, pp. 145-156.
56. J. Jeddeloh and B. Keeth, "Hybrid memory cube new DRAM architecture increases density and performance, "in Proc. Symp. VLSI Technol., Jun. 2012, pp. 87-88.
57. E. Azarkhish, I. Loi, L. Benini, "A case for three-dimensional stacking of tightly coupled data memories over multi-core clusters using low-latency interconnects, "IET Comput. Digital Techn., vol. 7, no. 5, pp. 191-199, Sep. 2013.
58. H. Kimura, et al., "28Gb/s 560mW multi-standard SerDes with single-stage analog front-end and 14-tap decision-feedback equalizer in 28nm CMOS, "in Proc. IEEE Int. Solid-State Circuits Conf. Digest Tech. Papers, Feb. 2014, pp. 38-39.
59. P. M. Kogge and D. R. Resnick, "Yearly update: Exascale projections for 2013, "Sandia National Laboratoris, Albuquerque, NM, USA, Tech. Rep. SAND2013-9229, Oct. 2013.
60. K. Sohn, et al., "A 1.2V 20nm 307GB/s HBM DRAM with at-speed wafer-level I/O test scheme and adaptive refresh considering temperature distribution, "in 2016 Proc. IEEE Int. Solid-State Circuits Conf., Jan. 2016, pp. 316-317.
61. "GPU-based deep learning inference: A performance and power analysis, "White Paper, NVIDIA, 2015.