Can FPGAs beat GPUs in accelerating next-generation deep neural networks?

FPGA 2017 - Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays

Volumn , Issue , 2017, Pages 5-14

  Nurvitadhi, Eriko ^a   Venkatesh, Ganesh ^a   Sim, Jaewoong ^a   Marr, Debbie ^a   Huang, Randy ^a   Ong, Jason Gee Hock ^a   Liew, Yeong Tat ^a   Srivatsan, Krishnan ^a   Moss, Duncan ^a   Subhaschandra, Suchit ^a   Boudoukh, Guy ^a  

^a INTEL CORPORATION   (United States)

Author keywords

Accelerator; Deep learning; GPU; Intel Stratix 10 FPGA

Indexed keywords

COMPUTER ARCHITECTURE; DEEP LEARNING; DEEP NEURAL NETWORKS; DIGITAL ARITHMETIC; ENERGY EFFICIENCY; GRAPHICS PROCESSING UNIT; LOGIC GATES; NEXT GENERATION NETWORKS; PARTICLE ACCELERATORS; PROGRAM PROCESSORS; RANDOM ACCESS STORAGE;

ALGORITHMIC EFFICIENCIES; CURRENT GENERATION; CUSTOMIZABILITY; FLOATING POINT UNITS; FLOATING POINTS; FPGA TECHNOLOGY; HIGH BANDWIDTH; MEMORY BLOCKS;

FIELD PROGRAMMABLE GATE ARRAYS (FPGA);

EID: 85016004196     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/3020078.3021740     Document Type: Conference Paper

References (30)

1. M. Courbariaux, Y. Bengio, J-P. David "BinaryConnect: Training Deep Neural Networks with binary weights during propagations," NIPS 2015.
2. M. Courbariaux, I. Hubara, et al., "Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1," arXiv:1602.02830 [cs.LG].
3. M. Rastegari, V. Ordonez, J. Redmon, A. Farhadi "XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks," arXiv:1603.05279 [cs.CV]
4. F. Li, B. Liu. "Ternary Weight Networks," arXiv:1605.04711 [cs.CV]
5. G. Venkatesh, E. Nurvitadhi, D. Marr, ".Accelerating Deep Convolutional Networks Using Low-Precision and Sparsity," ICASSP, 2017.
6. S. Han, H. Mao, W. J. Dally, "Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization, and Huffman Coding," ICLR 2016.
7. P. Gysel, et al., "Hardware-Oriented Approximation of Convolutional Neural Networks," ICLR Workshop 2016.
8. J. Albericio, P. Judd, T. Hetherington, et al, "Cnvlutin: Ineffectual-Neuron-Free Deep Convolutional Neural Network Computing," ISCA 2016.
9. S. Han, X. Liu, et al., "EIE: Efficient Inference Engine on Compressed Deep Neural Network," ISCA 2016.
10. N. Suda, V. Chandra, et al., "Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks," ISFPGA 2016.
11. J. Qiu, et al., "Going Deeper with Embedded FPGA Platform for Convolutional Neural Network," ISFPGA 2016.
12. P.K. Gupta, "Accelerating Datacenter Workloads," Keynote at FPL 2016. Slides available at www.fpl2016.org.
13. A. Putnam, A. M. Caulfield, et al., "A Reconfigurable Fabric for Accelerating Large-Scale Datacenter Services," ISCA 2014.
14. S. Y. Kung, "VLSI Array Processors," Prentice-Hall, Inc. Upper Saddle River, NJ, USA, 1987.
15. A. Pedram, et al., "A High-Performance, Low-Power Linear Algebra Core," ASAP 2011.
16. Altera Arria 10 Website. https://www.altera.com/products/fpga/arria-series/arria-10/overview.html
17. Altera Stratix 10 Website. https://www.altera.com/products/fpga/stratix-series/stratix-10/overview.html
18. Nvidia Titan X Website. http://www.geforce.com/hardware/10series/titan-x-pascal
19. Altera's PowerPlay Early Power Estimators (EPE) and Power Analyzer, https://www.altera.com/support/support-resources/operation-and-testing/power/pow-powerplay.html
20. S. Gross, M. Wilber, "Training and investigating Residual Nets," http://torch.ch/blog/2016/02/04/resnets.html
21. J. C. Johnson, "cnn-benchmarks",available at https://github.com/jcjohnson/cnn-benchmarks
22. G. Baeckler, "HyperPipelining of High-Speed Interface Logic," ISFPGA Tutorial, 2016.
23. A. Lavin, S. Gray, "Fast Algorithms for Convolutional Neural Networks," arXiv:1509.09308 [cs.NE].
24. P. D'Alberto, P. A. Milder, et al., "Generating FPGA Accelerated DFT Libraries," FCCM 2007.
25. W. Chen, J. Wilson, et al., "Compressing Neural Networks with the Hashing Trick," ICML 2015.
26. Y. Chen, T. Luo, S. Liu, et al., "Dadiannao: A machine-learning supercomputer," Int. Symposium on Microarchitecture (MICRO), 2014.
27. S. Kestur, et al., "BLAS Comparison on FPGA, CPU and GPU," IEEE Annual Sym. on VLSI (ISVLSI), 2010
28. E. Nurvitadhi, J. Sim, D. Sheffield, et al, "Accelerating Recurrent Neural Networks in Analytics Servers: Comparison of FPGA, CPU, GPU, and ASIC," FPL 2016.
29. MAGMA: Matrix Algebra on GPU and Multicore Architectures.Website: http://icl.cs.utk.edu/magma/
30. E. Nurvitadhi, D. Sheffield, J. Sim, et al, "Accelerating Binarized Neural Networks: Comparison of FPGA, CPU, GPU, and ASIC," FPT 2016.