Accelerating binarized neural networks: Comparison of FPGA, CPU, GPU, and ASIC

Proceedings of the 2016 International Conference on Field-Programmable Technology, FPT 2016

Volumn , Issue , 2017, Pages 77-84

  Nurvitadhi, Eriko ^a   Sheffield, David ^a   Sim, Jaewoong ^a   Mishra, Asit ^a   Venkatesh, Ganesh ^a   Marr, Debbie ^a  

^a INTEL CORPORATION   (United States)

Author keywords

ASIC; Binarized neural networks; CPU; Data analytics; Deep learning; FPGA; GPU; Hardware accelerator

Indexed keywords

APPLICATION SPECIFIC INTEGRATED CIRCUITS; BINS; COMPUTATIONAL EFFICIENCY; DEEP LEARNING; DEEP NEURAL NETWORKS; EFFICIENCY; FIELD PROGRAMMABLE GATE ARRAYS (FPGA); GRAPHICS PROCESSING UNIT; HARDWARE; PROGRAM PROCESSORS;

ALGORITHM EFFICIENCY; BIT-LEVEL OPERATIONS; COMPUTATIONAL DEMANDS; DATA ANALYTICS; EFFICIENCY IMPROVEMENT; HARDWARE ACCELERATION; HARDWARE ACCELERATORS; THEORETICAL PERFORMANCE;

COMPUTER HARDWARE;

EID: 85016000557     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/FPT.2016.7929192     Document Type: Conference Paper

References (16)

1. M. Courbariaux, Y. Bengio, J-P. David, "BinaryConnect: Training Deep Neural Networks with binary weights during propagations," Neural Information Processing Systems (NIPS), 2015.
2. M. Courbariaux, I. Hubara, D. Soudry, et al., "Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1," arXiv:1602.02830 [cs.LG].
3. D. Amodei, R. Anubhai, E. Battenberg, "Deep Speech 2: End-to-End Speech Recognition in English and Mandarin," arXiv:1512.02595 [cs.CL].
4. S. Kestur, J. D. Davis, O. Williams, "BLAS Comparison on FPGA, CPU and GPU," ISVLSI, 2010.
5. D. Mukonoki, T. Imamura, D. Takahashi, "Fast implementation of General Matrix-Vector Multiplication (GEMV) on Kepler GPUs," Euromicro International Conference on Parallel, Distributed, and Network-based Processing, 2015.
6. C. Farabet, C. Poulet, J. Y. Han, and Y. LeCun. "Cnp: An FPGA-based processor for convolutional networks," in Field Programmable Logic and Applications (FPL), 2009.
7. T. Chen, Z. Du, N. Sun, et al. "Diannao: A small-footprint high-throughput accelerator for ubiquitous machine-learning," Architectural Support for Programming Languages and Operating Systems, 2014.
8. A. Krizhevsky, et al, "Imagenet classification with deep convolutional neural networks," NIPS, 2012.
9. M. Horowitz. Energy table for 45nm process, Stanford VLSI wiki.
10. S. Ioffe, C. Szegedy "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," arXiv:1502.03167 [cs.LG].
11. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
12. A. Karpathy and L. Fei-Fei, "Deep visual-semantic alignments for generating image descriptions," arXiv preprint arXiv:1412.2306, 2014.
13. Altera's PowerPlay Early Power Estimators (EPE) and Power Analyzer. URL: https://www.altera.com/support/support-resources/operation-and-testing/power/pow-powerplay.tablet.html
14. E. Nurvitadhi, A. Mishra, D. Marr "A sparse matrix vector multiply accelerator for support vector machine," CASES, 2015.
15. E. Nurvitadhi, A. Mishra, Y. Wang, G. Venkatesh, D. Marr, "Hardware accelerator for analytics of sparse data," DATE, 2016.
16. E. Nurvitadhi, et al, "Accelerating recurrent neural networks in analytics servers: Comparison of FPGA, CPU, GPU, and ASIC," FPL, 2016.