Speeding-up convolutional neural networks using fine-tuned CP-decomposition

3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings

Volumn , Issue , 2015, Pages

  Lebedev, Vadim ^a,b   Ganin, Yaroslav ^a   Rakhuba, Maksim ^a,c   Oseledets, Ivan ^a,d   Lempitsky, Victor ^a  

^a SKOLKOVO INSTITUTE OF SCIENCE AND TECHNOLOGY   (Russian Federation)

^b YANDEX   (Russian Federation)

^c MOSCOW INSTITUTE OF PHYSICS AND TECHNOLOGY   (Russian Federation)

^d INSTITUTE OF NUMERICAL MATHEMATICS   (Russian Federation)

Author keywords

[No Author keywords available]

Indexed keywords

CONVOLUTION; DROPS; TENSORS;

CHARACTER CLASSIFICATION; CLASSIFICATION ERRORS; CONVOLUTION KERNEL; CONVOLUTIONAL NEURAL NETWORK; NON-LINEAR LEAST SQUARES; TENSOR DECOMPOSITION; TRAINING DATA; TWO-STEP APPROACH;

MULTILAYER NEURAL NETWORKS;

EID: 85083952441     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. Chellapilla, Kumar, Puri, Sidd, and Simard, Patrice. High performance convolutional neural networks for document processing. In Tenth International Workshop on Frontiers in Handwriting Recognition, 2006.
2. Chintala, Soumith. Convnet-benchmarks. https://github.com/soumith/ convnet-benchmarks, 2014. Accessed: 2014-12-19.
3. De Silva, Vin and Lim, Lek-Heng. Tensor rank and the ill-posedness of the best low-rank approximation problem. SIAM J. Matrix Anal. Appl., 30(3):1084–1127, 2008.
4. Dean, Jeffrey, Corrado, Greg, Monga, Rajat, Chen, Kai, Devin, Matthieu, Mao, Mark, Senior, Andrew, Tucker, Paul, Yang, Ke, Le, Quoc V, et al. Large scale distributed deep networks. In Advances in Neural Information Processing Systems, pp. 1223–1231, 2012.
5. Denton, Emily, Zaremba, Wojciech, Bruna, Joan, LeCun, Yann, and Fergus, Rob. Exploiting linear structure within convolutional networks for efficient evaluation. arXiv preprint arXiv:1404.0736, 2014.
6. Farabet, Clément, LeCun, Yann, Kavukcuoglu, Koray, Culurciello, Eugenio, Martini, Berin, Akselrod, Polina, and Talay, Selcuk. Large-scale FPGA-based convolutional networks. Machine Learning on Very Large Data Sets, 2011.
7. Glorot, Xavier and Bengio, Yoshua. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2010, Chia Laguna Resort, Sardinia, Italy, May 13-15, 2010, pp. 249–256, 2010.
8. Jaderberg, Max, Vedaldi, Andrea, and Zisserman, Andrew. Speeding up convolutional neural networks with low rank expansions. In Proceedings of the British Machine Vision Conference (BMVC), 2014a.
9. Jaderberg, Max, Vedaldi, Andrea, and Zisserman, Andrew. Deep features for text spotting. In Computer Vision–ECCV 2014, pp. 512–528. Springer, 2014b.
10. Jia, Yangqing, Shelhamer, Evan, Donahue, Jeff, Karayev, Sergey, Long, Jonathan, Girshick, Ross, Guadarrama, Sergio, and Darrell, Trevor. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093, 2014.
11. Kofidis, Eleftherios and Regalia, Phillip A. On the best rank-1 approximation of higher-order super-symmetric tensors. SIAM J. Matrix Anal. Appl., 23(3):863–884, 2002.
12. Kolda, T. G. and Bader, B. W. Tensor decompositions and applications. SIAM Rev., 51(3):455–500, 2009.
13. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097–1105, 2012.
14. LeCun, Yann, Boser, Bernhard, Denker, John S, Henderson, Donnie, Howard, Richard E, Hubbard, Wayne, and Jackel, Lawrence D. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541–551, 1989.
15. Lin, Min, Chen, Qiang, and Yan, Shuicheng. Network in network. arXiv preprint arXiv:1312.4400, 2013.
16. Mathieu, Michael, Henaff, Mikael, and LeCun, Yann. Fast training of convolutional networks through FFTs. arXiv preprint arXiv:1312.5851, 2013.
17. Rigamonti, Roberto, Sironi, Amos, Lepetit, Vincent, and Fua, Pascal. Learning separable filters. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pp. 2754–2761. Ieee, 2013.
18. Simonyan, Karen and Zisserman, Andrew. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556, 2014. URL http://arxiv.org/abs/1409.1556.
19. Sorber, L., Van Barel, M., and De Lathauwer, L. Tensorlab v2.0. Available online, 2014. URL http://tensorlab.net.
20. Stegeman, Alwin and Comon, Pierre. Subtracting a best rank-1 approximation may increase tensor rank. Linear Algebra Appl., 433(7):1276–1300, 2010.
21. Taigman, Yaniv, Yang, Ming, Ranzato, Marc’Aurelio, and Wolf, Lior. Deepface: Closing the gap to human-level performance in face verification. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pp. 1701–1708. IEEE, 2014.
22. Tomasi, Giorgio and Bro, Rasmus. A comparison of algorithms for fitting the parafac model. Comp. Stat. Data An., 50(7):1700–1734, 2006.