Towards convolutional neural networks compression via global error reconstruction

IJCAI International Joint Conference on Artificial Intelligence

Volumn 2016-January, Issue , 2016, Pages 1753-1759

  Lin, Shaohui ^a   Ji, Rongrong ^a   Guo, Xiaowei ^b   Li, Xuelong ^c  

^a XIAMEN UNIVERSITY   (China)

^b TENCENT AI LAB   (China)

^c XI AN INSTITUTE OF OPTICS AND PRECISION MECHANICS   (China)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; ARTIFICIAL INTELLIGENCE; BACKPROPAGATION; COMPACTION; CONVOLUTION; DATA COMPRESSION; DIGITAL STORAGE; ELECTRIC DISTORTION; ERRORS; IMAGE CODING; NEURAL NETWORKS; SIGNAL DISTORTION;

CONVOLUTIONAL NEURAL NETWORK; DISCRIMINABILITY; GLOBAL PERSPECTIVE; LOW RANK APPROXIMATIONS; OUTPUT RECONSTRUCTION ERRORS; RATE DISTORTION PERFORMANCE; RATE DISTORTIONS; STATE OF THE ART;

IMAGE CLASSIFICATION;

EID: 85006100269     PISSN: 10450823     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. I. Sutskever A. Krizhevsky and G. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, pages 1097-1105, 2012.
2. A. Osokin A. Novikov, D. Podoprikhin and D. Vetrov. Tensorizing neural networks. In NIPS, pages 442-450, 2015.
3. L. Ba and R. Caruana. Do deep nets really need to be deep In NIPS, pages 2654-2662, 2014.
4. Y. Jia P. Sermanet S. Reed D. Anguelov D. Erhan V. Vanhoucke C. Szegedy, W. Liu and A. Rabinovich. Going deeper with convolutions. In CVPR, 2015.
5. K. Fukushima. Neocognitron: A selforganizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological cybernetics, 36(4): 193-202, 1980.
6. G. Golub and C. Van Loan. Matrix computations, volume 3. JHU Press, 2012.
7. J. Gower and G. Dijksterhuis. Procrustes problems, volume 3. Oxford University Press Oxford, 2004.
8. R. Socher L. Li K. Li J. Deng, W. Dong and F. Li. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255, 2009.
9. K. Yu F. Lv T. Huang J. Wang, J. Yang and Y. Gong. Locality-constrained linear coding for image classification. In CVPR, pages 3360-3367, 2010.
10. S. Ren K. He, X. Zhang and J. Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. In ECCV, pages 346-361. 2014.
11. S. Ren K. He, X. Zhang and J. Sun. Deep residual learning for image recognition. arXiv preprint arXiv: 1512. 03385, 2015.
12. L. Dinh M. Ranzato M. Denil, B. Shakibi and N. Freitas. Predicting parameters in deep learning. In NIPS, pages 2148-2156, 2013.
13. V. Nair and G. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, pages 807-814, 2010.
14. X. Zhang M. Mathieu R. Fergus P. Sermanet, D. Eigen and Y. LeCun. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv: 1312. 6229, 2013.
15. T. Darrell R. Girshick, J. Donahue and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, pages 580-587, 2014.
16. J. Tran S. Han, J. Pool and W. Dally. Learning both weights and connections for efficient neural network. In NIPS, pages 1135-1143, 2015.
17. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409. 1556, 2014.
18. S. Srinivas and R. Babu. Datafree parameter pruning for deep neural networks. arXiv preprint arXiv: 1507. 06149, 2015.
19. Y. Takane and H. Hwang. Regularized linear and kernel redundancy analysis. Computational Statistics & Data Analysis, 52(1): 394-405, 2007.
20. Y. Takane and S. Jung. Generalized constrained redundancy analysis. Behaviormetrika, 33(2): 179-192, 2006.
21. S. Tyree K. WeinbergerW. Chen, J. Wilson and Y. Chen. Compressing neural networks with the hashing trick. In ICML, 2015.
22. X. Ming K. He X. Zhang, J. Zou and J. Sun. Efficient and accurate approximations of nonlinear convolutional networks. In CVPR, 2015.
23. R. Feris S. Kumar A. Choudhary Y. Cheng, F. Yu and S. Chang. An exploration of parameter redundancy in deep networks with circulant projections. In ICCV, pages 2857-2865, 2015.
24. M. Yang Y. Gong, L. Liu and L. Bourdev. Compressing deep convolutional networks using vector quantization. arXiv preprint arXiv: 1412. 6115, 2014.
25. R. Guo Y. Gong, L. Wang and S. Lazebnik. Multi-scale orderless pooling of deep convolutional activation features. In ECCV, pages 392-407. 2014.
26. J. Donahue S. Karayev J. Long R. Girshick S. Guadarrama Y. Jia, E. Shelhamer and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. In ACM Multimedia, pages 675-678, 2014.
27. Y. Bengio Y. LeCun, L. Bottou and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11): 2278-2324, 1998.
28. G. Orr Y. LeCun, L. Bottou and K. Müller. Efficient backprop. In Neural networks: Tricks of the trade, pages 9-48. Springer, 2012.
29. M. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In ECCV, pages 818-833, 2014.