Fast convolutional nets with fbfft: A GPU performance evaluation

3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings

Volumn , Issue , 2015, Pages

  Vasilache, Nicolas ^a   Johnson, Jeff ^a   Mathieu, Michael ^a   Chintala, Soumith ^a   Piantino, Serkan ^a   LeCun, Yann ^a  

^a FACEBOOK AI RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER GRAPHICS; COMPUTER HARDWARE; CONVOLUTION; FAST FOURIER TRANSFORMS; FREQUENCY DOMAIN ANALYSIS; NEURAL NETWORKS; PROGRAM PROCESSORS;

ALGORITHMIC APPLICATIONS; CONVOLUTIONAL NEURAL NETWORK; CURRENT GENERATION; FREQUENCY DOMAINS; NVIDIA GRAPHICS; OPEN SOURCES; PERFORMANCE PROFILE; TIME DOMAIN CONVOLUTIONS;

GRAPHICS PROCESSING UNIT;

EID: 85083953426     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Bergstra, James, Breuleux, Olivier, Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Desjardins, Guillaume, Turian, Joseph, Warde-Farley, David, and Bengio, Yoshua. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy), June 2010. Oral Presentation.
2. Bluestein, Leo I. A linear filtering approach to the computation of discrete Fourier transform. Audio and Electroacoustics, IEEE Transactions on, 18(4):451–455, December 1970. ISSN 0018-9278.
3. Brosch, Tom and Tam, Roger C. Efficient training of convolutional deep belief networks in the frequency domain for application to high-resolution 2d and 3d images. Neural Computation, 27 (1):211–227, 2015. doi: 10.1162/NECO_a_00682. URL http://dx.doi.org/10.1162/NECO_a_00682.
4. Burrus, C. Sidney. Fast fourier transforms, 2008. URL http://cnx.org/contents/16e8e5e8-4f22-4b53-9cd6-a15b14f01ce4@5.6:16/Fast_Fourier_Transforms_(6x9_V.
5. Chellapilla, Kumar, Puri, Sidd, and Simard, Patrice. High Performance Convolutional Neural Networks for Document Processing. In Lorette, Guy (ed.), Tenth International Workshop on Frontiers in Handwriting Recognition, La Baule (France), October 2006. Université de Rennes 1, Suvisoft. URL https://hal.inria.fr/inria-00112631. http://www.suvisoft.com.
6. Chetlur, Sharan, Woolley, Cliff, Vandermersch, Philippe, Cohen, Jonathan, Tran, John, Catanzaro, Bryan, and Shelhamer, Evan. cudnn: Efficient primitives for deep learning. CoRR, abs/1410.0759, 2014. URL http://arxiv.org/abs/1410.0759.
7. Collobert, R., Kavukcuoglu, K., and Farabet, C. Torch7: A matlab-like environment for machine learning. In BigLearn, NIPS Workshop, 2011a.
8. Collobert, Ronan, Weston, Jason, Bottou, Léon, Karlen, Michael, Kavukcuoglu, Koray, and Kuksa, Pavel. Natural language processing (almost) from scratch. J. Mach. Learn. Res., 12:2493–2537, November 2011b. ISSN 1532-4435. URL http://dl.acm.org/citation.cfm?id=1953048.2078186.
9. Cooley, James W. and Tukey, John W. An algorithm for the machine calculation of complex fourier series. Mathematics of computation, 19(90):297–301, 1965.
10. Garland, Michael, Le Grand, Scott, Nickolls, John, Anderson, Joshua, Hardwick, Jim, Morton, Scott, Phillips, Everett, Zhang, Yao, and Volkov, Vasily. Parallel computing experiences with cuda. IEEE Micro, 28(4):13–27, July 2008. ISSN 0272-1732. doi: 10.1109/MM.2008.57. URL http://dx.doi.org/10.1109/MM.2008.57.
11. Giles, Mike. Course on cuda programming on nvidia gpus, lecture 3, 2014. URL http://people.maths.ox.ac.uk/gilesm/cuda/lecs/lec3.pdf.
12. Gunnels, John A., Henry, Greg M., and van de Geijn, Robert A. A family of high-performance matrix multiplication algorithms. In Proceedings of the International Conference on Computational Sciences-Part I, ICCS’01, pp. 51–60, London, UK, UK, 2001. Springer-Verlag. ISBN 3-540-42232-3. URL http://dl.acm.org/citation.cfm?id=645455.653765.
13. Irigoin, F. and Triolet, R. Supernode partitioning. In Proceedings of the 15th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL’88, pp. 319–329, New York, NY, USA, 1988. ACM. ISBN 0-89791-252-7. doi: 10.1145/73560.73588. URL http://doi.acm.org/10.1145/73560.73588.
14. Jia, Yangqing, Shelhamer, Evan, Donahue, Jeff, Karayev, Sergey, Long, Jonathan, Girshick, Ross, Guadarrama, Sergio, and Darrell, Trevor. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093, 2014.
15. Krizhevsky, Alex. cuda-convnet2, 2014. URL https://code.google.com/p/cudaconvnet2/.
16. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q. (eds.), Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc., 2012. URL http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf.
17. Lavin, Andrew. maxdnn: An efficient convolution kernel for deep learning with maxwell gpus. CoRR, abs/1501.06633, 2015. URL http://arxiv.org/abs/1501.06633.
18. Lyons, Richard G. Understanding Digital Signal Processing. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1st edition, 1996. ISBN 0201634678.
19. Mathieu, Michaël, Henaff, Mikael, and LeCun, Yann. Fast training of convolutional networks through ffts. CoRR, abs/1312.5851, 2013. URL http://arxiv.org/abs/1312.5851.
20. Ragan-Kelley, Jonathan, Barnes, Connelly, Adams, Andrew, Paris, Sylvain, Durand, Frédo, and Amarasinghe, Saman P. Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. In ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI’13, Seattle, WA, USA, June 16-19, 2013, pp. 519–530, 2013. doi: 10.1145/2462156.2462176. URL http://doi.acm.org/10.1145/2462156.2462176.
21. Ruetsch, Greg and Micikevicius, Paulius. Optimizing matrix transpose in cuda. Technical report, NVIDIA Corp., January 2009.
22. Sermanet, Pierre, Eigen, David, Zhang, Xiang, Mathieu, Michael, Fergus, Rob, and LeCun, Yann. Overfeat: Integrated recognition, localization and detection using convolutional networks. In International Conference on Learning Representations (ICLR 2014). CBLS, April 2014. URL http://openreview.net/document/d332e77d-459a-4af8-b3ed-55ba.
23. Shin, Jaewook, Hall, Mary W., Chame, Jacqueline, Chen, Chun, Fischer, Paul F., and Hovland, Paul D. Speeding up nek5000 with autotuning and specialization. In Proceedings of the 24th ACM International Conference on Supercomputing, ICS’10, pp. 253–262, New York, NY, USA, 2010. ACM. ISBN 978-1-4503-0018-6.
24. Valiant, Leslie G. A bridging model for parallel computation. Commun. ACM, 33(8):103–111, August 1990. ISSN 0001-0782. doi: 10.1145/79173.79181. URL http://doi.acm.org/10.1145/79173.79181.
25. Volkov, V. Better performance at lower occupancy. In GPU Technology Conference, 2010. URL http://www.cs.berkeley.edu/~volkov/volkov10-GTC.pdf.