DSCNN: Hardware-oriented optimization for Stochastic Computing based Deep Convolutional Neural Networks

Proceedings of the 34th IEEE International Conference on Computer Design, ICCD 2016

Volumn , Issue , 2016, Pages 678-681

  Li, Zhe ^a   Ren, Ao ^a   Li, Ji ^b   Qiu, Qinru ^a   Wang, Yanzhi ^a   Yuan, Bo ^c  

^a Syracuse University   (United States)

^b University of Southern California   (United States)

^c City College of the City University of New York   (United States)

Author keywords

Deep Convolutional Neural Networks; Deep Learning; Hardware oriented Co optimization; Stochastic Computing

Indexed keywords

COMPLEX NETWORKS; COMPUTER HARDWARE; CONVOLUTION; EXTRACTION; FEATURE EXTRACTION; HARDWARE; INTERNET OF THINGS; NEURAL NETWORKS; PROGRAM PROCESSORS; SOFTWARE TESTING;

CALCULATION PRECISION; CO-OPTIMIZATION; COMPUTATION COMPLEXITY; CONVOLUTIONAL NEURAL NETWORK; DEEP LEARNING; INTERNET OF THINGS (IOT); LARGE-SCALE APPLICATIONS; STOCHASTIC COMPUTING;

STOCHASTIC SYSTEMS;

EID: 85006716849     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICCD.2016.7753357     Document Type: Conference Paper

References (21)

1. Y. Le Cun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015.
2. Y. Le Cun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
3. P. Y. Simard, D. Steinkraus, and J. C. Platt, "Best practices for convolutional neural networks applied to visual document analysis." in ICDAR, vol. 3, 2003, pp. 958-962.
4. S. Lawrence, C. L. Giles, A. C. Tsoi, and A. D. Back, "Face recognition: A convolutional neural-network approach," IEEE transactions on neural networks, vol. 8, no. 1, pp. 98-113, 1997.
5. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, 2012, pp. 1097-1105.
6. D. Ciregan, U. Meier, and J. Schmidhuber, "Multi-column deep neural networks for image classification," in Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2012, pp. 3642-3649.
7. D. C. Ciresan, U. Meier, J. Masci, L. Maria Gambardella, and J. Schmidhuber, "Flexible, high performance convolutional neural networks for image classification," in IJCAI Proceedings-International Joint Conference on Artificial Intelligence, vol. 22, no. 1, 2011, p. 1237.
8. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, "Large-scale video classification with convolutional neural networks," in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2014, pp. 1725-1732.
9. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 580-587.
10. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1-9.
11. B. Catanzaro, "Deep learning with cots hpc systems," 2013.
12. J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, M. Mao, A. Senior, P. Tucker, K. Yang, Q. V. Le et al., "Large scale distributed deep networks," in Advances in neural information processing systems, 2012, pp. 1223-1231.
13. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell, "Caffe: Convolutional architecture for fast feature embedding," in Proceedings of the 22nd ACM international conference on Multimedia. ACM, 2014, pp. 675-678.
14. J. Bergstra, F. Bastien, O. Breuleux, P. Lamblin, R. Pascanu, O. Delalleau, G. Desjardins, D. Warde-Farley, I. Goodfellow, A. Bergeron et al., "Theano: Deep learning on gpus with python," in NIPS 2011, Big Learning Workshop, Granada, Spain. Citeseer, 2011.
15. R. Courtland, "Gordon moore: The man whose name means progress," IEEE Spectrum, vol. 30, 2015.
16. W. Qian, X. Li, M. D. Riedel, K. Bazargan, and D. J. Lilja, "An architecture for fault-tolerant computation with stochastic logic," IEEE Transactions on Computers, vol. 60, no. 1, pp. 93-105, 2011.
17. J. Li and J. Draper, "Accelerating soft-error-rate (ser) estimation in the presence of single event transients," in Proceedings of the 53rd Annual Design Automation Conference. ACM, 2016, p. 55.
18. B. R. Gaines, "Stochastic computing systems," in Advances in information systems science. Springer, 1969, pp. 37-172.
19. B. D. Brown and H. C. Card, "Stochastic neural computation. i. computational elements," IEEE Transactions on computers, vol. 50, no. 9, pp. 891-905, 2001.
20. A. Ren, Z. Li, Y. Wang, Q. Qiu, and B. Yuan, "Designing reconfigurable large-scale deep learning systems using stochastic computing," in 2016 IEEE International Conference on Rebooting Computing. IEEE, 2016.
21. K. Kim, J. Kim, J. Yu, J. Seo, J. Lee, and K. Choi, "Dynamic energyaccuracy trade-off using stochastic computing in deep neural networks," in Proceedings of the 53rd Annual Design Automation Conference. ACM, 2016, p. 124.