EIE: Efficient Inference Engine on Compressed Deep Neural Network

Proceedings - 2016 43rd International Symposium on Computer Architecture, ISCA 2016

Volumn , Issue , 2016, Pages 243-254

  Han, Song ^a   Liu, Xingyu ^a   Mao, Huizi ^a   Pu, Jing ^a   Pedram, Ardavan ^a   Horowitz, Mark A ^a   Dally, William J ^b  

^a STANFORD UNIVERSITY   (United States)

^b NVIDIA   (United States)

Author keywords

Algorithm Hardware co Design; ASIC; Deep Learning; Hardware Acceleration; Model Compression

Indexed keywords

APPLICATION SPECIFIC INTEGRATED CIRCUITS; BUDGET CONTROL; COMPUTER ARCHITECTURE; COMPUTER HARDWARE; DYNAMIC RANDOM ACCESS STORAGE; EMBEDDED SYSTEMS; ENERGY CONSERVATION; ENGINES; HARDWARE; NETWORK ARCHITECTURE; RECONFIGURABLE HARDWARE; STATIC RANDOM ACCESS STORAGE;

CO-DESIGNS; DEEP LEARNING; DEEP NEURAL NETWORKS; HARDWARE ACCELERATION; MODEL COMPRESSION; MULTIPLE CONNECTIONS; REDUNDANT CONNECTIONS; SPARSE MATRIX-VECTOR MULTIPLICATION;

ENERGY EFFICIENCY;

EID: 84988443578     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISCA.2016.30     Document Type: Conference Paper

References (45)

1. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in NIPS, 2012.
2. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions," arXiv:1409.4842, 2014.
3. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv:1409.1556, 2014.
4. T. Mikolov, M. Karafiát, L. Burget, J. Cernocky, and S. Khudanpur, "Recurrent neural network based language model." in INTERSPEECH, September 26-30, 2010, 2010, pp. 1045-1048.
5. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
6. Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, "Deepface: Closing the gap to human-level performance in face verification," in CVPR. IEEE, 2014, pp. 1701-1708.
7. A. Karpathy and L. Fei-Fei, "Deep visual-semantic alignments for generating image descriptions," arXiv:1412.2306, 2014.
8. A. Coates, B. Huval, T. Wang, D. Wu, B. Catanzaro, and N. Andrew, "Deep learning with cots hpc systems," in 30th ICML, 2013.
9. M. Horowitz. Energy table for 45nm process, Stanford VLSI wiki. [Online]. Available: https://sites.google.com/site/seecproject
10. T. Chen, Z. Du, N. Sun, J. Wang, C. Wu, Y. Chen, and O. Temam, "Diannao: a small-footprint high-throughput accelerator for ubiquitous machine-learning," in ASPLOS, 2014.
11. Y. Chen, T. Luo, S. Liu, S. Zhang, L. He, J. Wang, L. Li, T. Chen, Z. Xu, N. Sun, and O. Temam, "Dadiannao: A machine-learning supercomputer," in MICRO, December 2014.
12. Z. Du, R. Fasthuber, T. Chen, P. Ienne, L. Li, T. Luo, X. Feng, Y. Chen, and O. Temam, "Shidiannao: shifting vision processing closer to the sensor," in ISCA. ACM, 2015, pp. 92-104.
13. C. Farabet, C. Poulet, J. Y. Han, and Y. LeCun, "Cnp: An FPGA-based processor for convolutional networks," in FPL, 2009.
14. J. Qiu, J. Wang, S. Yao, K. Guo, B. Li, E. Zhou, J. Yu, T. Tang, N. Xu, S. Song, Y. Wang, and H. Yang, "Going deeper with embedded FPGA platform for convolutional neural network," in FPGA, 2016.
15. A. Shafiee and et al., "ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars," ISCA, 2016.
16. S. Han, J. Pool, J. Tran, and W. J. Dally, "Learning both weights and connections for efficient neural networks," in Proceedings of Advances in Neural Information Processing Systems, 2015.
17. R. Girshick, "Fast R-CNN," arXiv:1504.08083, 2015.
18. S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural computation, 1997.
19. A. Graves and J. Schmidhuber, "Framewise phoneme classification with bidirectional lstm and other neural network architectures," Neural Networks, 2005.
20. N. D. Lane and P. Georgiev, "Can deep learning revolutionize mobile sensing" in International Workshop on Mobile Computing Systems and Applications. ACM, 2015, pp. 117-122.
21. Richard Dorrance and Fengbo Ren and Dejan Markovíc, "A Scalable Sparse Matrix-vector Multiplication Kernel for Energy-efficient Sparse-blas on FPGAs," in FPGA, 2014.
22. V. Nair and G. E. Hinton, "Rectified linear units improve restricted boltzmann machines," in ICML, 2010.
23. S. Han, H. Mao, and W. J. Dally, "Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding," International Conference on Learning Representations 2016.
24. R. W. Vuduc, "Automatic performance tuning of sparse matrix kernels," Ph.D. dissertation, UC Berkeley, 2003.
25. N. Muralimanohar, R. Balasubramonian, and N. P. Jouppi, "Cacti 6.0: A tool to model large caches," HP Laboratories, pp. 22-31, 2009.
26. NVIDIA. Technical brief: NVIDIA jetson TK1 development kit bringing GPU-accelerated computing to embedded systems.
27. NVIDIA. Whitepaper: GPU-based deep learning inference: A performance and power analysis.
28. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell, "Caffe: Convolutional architecture for fast feature embedding," arXiv:1408.5093, 2014.
29. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image database," in Computer Vision and Pattern Recognition. 2009.
30. F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, "Squeezenet: Alexnet-level accuracy with 50x fewer parameters and 0.5mb model size," arXiv:1602.07360, 2016.
31. Ling Zhuo and Viktor K. Prasanna, "Sparse Matrix-Vector Multiplication on FPGAs," in FPGA, 2005.
32. V. Eijkhout, LAPACK working note 50: Distributed sparse data structures for linear algebra operations, 1992.
33. A. Lavin, "Fast algorithms for convolutional neural networks," arXiv:1509.09308, 2015.
34. S. J. Hanson and L. Y. Pratt, "Comparing biases for minimal network construction with back-propagation," in NIPS, 1989.
35. Y. LeCun, J. S. Denker, S. A. Solla, R. E. Howard, and L. D. Jackel, "Optimal brain damage." in NIPs, vol. 89, 1989.
36. B. Hassibi, D. G. Stork et al., "Second order derivatives for network pruning: Optimal brain surgeon," Advances in neural information processing systems, pp. 164-164, 1993.
37. E. L. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus, "Exploiting linear structure within convolutional networks for efficient evaluation," in NIPS 2014.
38. X. Zhang, J. Zou, X. Ming, K. He, and J. Sun, "Efficient and accurate approximations of nonlinear convolutional networks," arXiv:1411.4229, 2014.
39. B. Reagen, P. Whatmough, R. Adolf, S. Rama, H. Lee, S. K. Lee, J. M. Hernndez-Lobato, G.-Y. Wei, and D. Brooks, "Minerva: Enabling low-power, highly-accurate deep neural network accelerators," ISCA, 2016.
40. S. K. Esser and et al., "Convolutional networks for fast, energyefficient neuromorphic computing," arXiv:1603.08270, 2016.
41. C. Zhang, P. Li, G. Sun, Y. Guan, B. Xiao, and J. Cong, "Optimizing FPGA-based accelerator design for deep convolutional neural networks," in FPGA, 2015.
42. Alexander Monakov and Anton Lokhmotov and Arutyun Avetisyan, "Automatically tuning sparse matrix-vector multiplication for GPU architectures," in HiPEAC, 2010.
43. N. Bell and M. Garland, "Efficient sparse matrix-vector multiplication on cuda," Nvidia Technical Report NVR-2008-004, Tech. Rep., 2008.
44. Bell, Nathan and Garland, Michael, "Implementing Sparse Matrixvector Multiplication on Throughput-oriented Processors," in High Performance Computing Networking, Storage and Analysis, 2009.
45. J. Fowers and K. Ovtcharov and K. Strauss and E.S. Chung and G. Stitt, "A high memory bandwidth FPGA accelerator for sparse matrixvector multiplication," in FCCM, 2014.