FPGA-accelerated deep convolutional neural networks for high throughput and energy efficiency

Concurrency and Computation: Practice and Experience

Volumn 29, Issue 20, 2017, Pages

  Qiao, Yuran ^a   Shen, Junzhong ^a   Xiao, Tao ^a   Yang, Qianming ^a   Wen, Mei ^a   Zhang, Chunyuan ^a  

^a NATIONAL UNIVERSITY OF DEFENSE TECHNOLOGY   (China)

Author keywords

Accelerator; Caffe; CNN; FPGA; Matrix Multiplier

Indexed keywords

ACCELERATION; APPLICATION PROGRAMS; COMPUTER PROGRAMMING; CONVOLUTION; DEEP NEURAL NETWORKS; ENERGY EFFICIENCY; FIELD PROGRAMMABLE GATE ARRAYS (FPGA); MATRIX ALGEBRA; MEMORY ARCHITECTURE; PARTICLE ACCELERATORS; PROGRAM PROCESSORS; SYSTEM-ON-CHIP;

ACCELERATOR ARCHITECTURES; APPLICATION DEVELOPERS; CAFFE; COMPUTATIONAL PERFORMANCE; COMPUTATIONAL WORKLOAD; HIGH ENERGY EFFICIENCY; PERFORMANCE PORTABILITY; PRACTICE AND EXPERIENCE;

CONVOLUTIONAL NEURAL NETWORKS;

EID: 84966447574     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.3850     Document Type: Conference Paper

References (28)

1. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, Curran Associates, Inc.: Lake Tahoe, Nevada, USA, 2012; 1097–1105.
2. Hinton G, Deng L, Yu D, Dahl GE, Mohamed Ar, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN, Kingsbury B. Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. Signal Processing Magazine, IEEE 2012; 29(6):82–97.
3. Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR): IEEE, Columbus, OH, USA, 2014; 580–587.
4. Gokhale V, Jin J, Dundar A, Martini B, Culurciello E. A 240 G-ops/s mobile coprocessor for deep neural networks. In 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW): IEEE, Columbus, OH, USA, 2014; 696–701.
5. Ovtcharov K, Ruwase O, Kim JY, Fowers J, Strauss K, Chung ES. Accelerating deep convolutional neural networks using specialized hardware. Microsoft Research Whitepaper: Microsoft Research, 2015.
6. Wan L, Li K, Liu J, Li K. GPU implementation of a parallel two-list algorithm for the subset-sum problem. Concurrency Computation Practice Experience 2015; 27(1):119–145.
7. Tang G, Yang W, Li K, Ye Y, Xiao G, Li K. An iteration-based hybrid parallel algorithm for tridiagonal systems of equations on multi-core architectures. Concurrency Computation Practice Experience 2015; 27(17):5076–5095.
8. NVIDIA. TESLA K20 GPU accelerator board specification, July 2013.
9. Farabet C, Poulet C, Han JY, LeCun Y. CNP: an FPGA-based processor for convolutional networks. FPL 2009. International Conference on Field Programmable Logic and Applications, 2009: IEEE, Prague, Czech Republic, 2009; 32–37.
10. Zhang C, Li P, Sun G, Guan Y, Xiao B, Cong J. Optimizing FPGA-based accelerator design for deep convolutional neural networks. Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays: ACM, Monterey, CA, USA, 2015; 161–170.
11. Cadambi S, Majumdar A, Becchi M, Chakradhar S, Graf HP. A programmable parallel accelerator for learning and classification. Proceedings of the 19th International Conference on Parallel Architectures and Compilation Techniques: ACM, New York, NY, USA, 2010; 273–284.
12. Gupta S, Agrawal A, Gopalakrishnan K, Narayanan P. Deep learning with limited numerical precision. arXiv preprint 2015. arXiv:1502.02551.
13. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T. Caffe: convolutional architecture for fast feature embedding. Proceedings of the ACM International Conference on Multimedia: ACM, Orlando, Florida, USA, 2014; 675–678.
14. Collobert R, Kavukcuoglu K, Farabet C. Torch7: a matlab-like environment for machine learning. BigLearn, NIPS Workshop, Granada, Spain, 2011.
15. Farabet C. Towards real-time image understanding with convolutional networks. Ph.D. Thesis, Université Paris-Est, 2013.
16. Dou Y, Vassiliadis S, Kuzmanov GK, Gaydadjiev GN. 64-bit floating-point FPGA matrix multiplication. Proceedings of the 2005 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays: ACM, New York, NY, USA, 2005; 86–95.
17. Gorman Mel. Understanding the Linux Virtual Memory Manager. Prentice Hall Upper Saddle River: New Jersey, USA, 2004.
18. Xiao T, Qiao Y, Shen J, Yang Q, Wen M. Unified virtual memory support for deep CNN accelerator on SoC FPGA. In Algorithms and Architectures for Parallel Processing. Springer: Zhangjiajie, China 2015; 64–76.
19. Farabet C, Martini B, Corda B, Akselrod P, Culurciello E, LeCun Y. Neuflow: a runtime reconfigurable dataflow processor for vision. 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW): IEEE, Colorado Springs, CO, USA, 2011; 109–116.
20. Yi D, Lei Z, Liao S, Li SZ. Learning face representation from scratch. arXiv preprint 2014: arXiv:1411.7923.
21. Chakradhar S, Sankaradas M, Jakkula V, Cadambi S. A dynamically configurable coprocessor for convolutional neural networks. ACM SIGARCH Computer Architecture News 2010; 38(3):247–257.
22. Peemen M, Setio AAA, Mesman B, Corporaal H. Memory-centric accelerator design for convolutional neural networks. 2013 IEEE 31st International Conference on Computer Design (ICCD): IEEE, 2013; 13–19.
23. Chen T, Du Z, Sun N, Wang J, Wu C, Chen Y, Temam O. Diannao: a small-footprint high-throughput accelerator for ubiquitous machine-learning. Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems: ACM, 2014; 269–284.
24. Chetlur S, Woolley C, Vandermersch P, Cohen J, Tran J, Catanzaro B, Shelhamer E. CuDNN: Efficient primitives for deep learning. arXiv preprint 2014: arXiv:1410.0759.
25. Jiang J, Hu R, Mikel L, Dou Y. Accuracy evaluation of deep belief networks with fixed-point arithmetic. Computer Modelling & New Technologies 2014; 18(6):7–14.
26. Han S, Pool J, Tran J, Dally WJ. Learning both weights and connections for efficient neural networks. arXiv preprint 2015: arXiv:1506.02626.
27. Li K, Yang W, Li K. Performance analysis and optimization for SPMV on GPU using probabilistic modeling. IEEE Transactions on Parallel and Distributed Systems 2015; 26(1):196–205.
28. Yang W, Li K, Mo Z, Li K. Performance optimization using partitioned SPMV on GPUs and multicore CPUs. IEEE Transactions on Computers 2015; 64(9):2623–2636.