Less is more: Towards compact CNNs

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 9908 LNCS, Issue , 2016, Pages 662-677

  Zhou, Hao ^a   Alvarez, Jose M ^b   Porikli, Fatih ^b,c  

^a UNIVERSITY OF MARYLAND   (United States)

^b CSIRO   (Australia)

^c AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

Convolutional neural network; Neuron reduction; Sparsity

Indexed keywords

CONVOLUTION; LARGE DATASET; NEURONS;

CLASSIFICATION ACCURACY; HIGH CAPACITY; LARGE-SCALE DATASETS; LESS IS MORES; MEMORY FOOTPRINT; OBJECTIVE FUNCTIONS; SPARSITY; TEST TIME;

CONVOLUTIONAL NEURAL NETWORKS;

EID: 84990032200     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-319-46493-0_40     Document Type: Conference Paper

References (40)

1. Ba, J., Caruana, R.: Do deep nets really need to be deep?. In: NIPS (2014)
2. Bucila, C., Caruana, R., Niculescu-Mizil, A.: Model compression. In: ACM SIGKDD (2006)
3. Chen, L., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected CRFS. In: ICLR (2014)
4. Chintala, S.: Soumith/imagenet-multigpu.torch (2015). https://github.com/soumith/imagenet-multiGPU.torch
5. Collins, M.D., Kohli, P.: Memory bounded deep convolutional networks. CoRR abs/1412.1442 (2014)
6. Collobert, R., Kavukcuoglu, K., Farabet, C.: Torch7: a matlab-like environment for machine learning. In: BigLearn, NIPS Workshop (2011)
7. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: CVPR (2009)
8. Deng, W., Yin, W., Zhang, Y.: Group sparse optimization by alternating direction method. In: SPIE. vol. 8858 (2013)
9. Denil, M., Shakibi, B., Dinh, L., Ranzato, M., Freitasa, N.D.: Predicting parameters in deep learning. In: NIPS (2013)
10. Denton, E.L., Zaremba, W., Bruna, J., LeCun, Y., Fergus, R.: Exploiting linear structure within convolutional networks for efficient evaluation. In: NIPS (2014)
11. Duchi, J., Singer, Y.: Efficient online and batch learning using forward backward splitting. J. Mach. Learn. Res. 10, 2899-2934 (2009)
12. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR (2014)
13. Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: AISTATS (2011)
14. Goldstein, T., Studer, C., Baraniuk, R. G.: A field guide to forward-backward splitting with a FASTA implementation. arXiv eprint arXiv.abs/1411.3406 (2014)
15. Gong, Y., Liu, L., Yang, M., Bourdev, L.D.: Compressing deep convolutional networks using vector quantization. CoRR abs/1412.6115 (2014)
16. Han, S., Mao, H., Dally, W. J.: Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding. In: ICLR (2016)
17. Han, S., Pool, J., Tran, J., Dally, W.J.: Learning both weights and connections for efficient neural networks. In: NIPS (2015)
18. Hassibi, B., Stork, D.G.: Second order derivatives for network pruning: Optimal brain surgeon. In: NIPS (1993)
19. Hinton, G.E., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. In: NIPS 2014 Deep Learning Workshop (2014)
20. Jaderberg, M., Vedaldi, A., Zisserman, A.: Speeding up convolutional neural networks with low rank expansions. In: BMVC (2014)
21. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding. arXiv preprint (2014). arXiv:1408.5093
22. Krizhevsky, A.: Learning multiple layers of features from tiny images. Technical report, Department of Computer Science, University of Toronto (2009)
23. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)
24. Lebedev, V., Ganin, Y., Rakhuba, M., Oseledets, I.V., Lempitsky, V.S.: Speedingup convolutional neural networks using fine-tuned cp-decomposition. In: ICLR (2015)
25. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278-2324 (1998)
26. LeCun, Y., Denker, J.S., Solla, S.A.: Optimal brain damage. In: NIPS (1990)
27. Liu, B., Wang, M., Foroosh, H., Tappen, M., Penksy, M.: Sparse convolutional neural networks. In: CVPR (2015)
28. Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. IEEE Trans. PAMI 35(1), 208-220 (2013)
29. Mathieu, M., Henaff, M., LeCun, Y.: Fast training of convolutional networks through FFTS. CoRR abs/1312.5851, http://arxiv.org/abs/1312.5851 (2013)
30. Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: ICML (2010)
31. Novikov, A., Podoprikhin, D., Osokin, A., Vetrov, D.P.: Tensorizing neural networks. In: NIPS (2015)
32. Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., Bengio, Y.: Fitnets: Hints for thin deep nets. In: ICLR (2015)
33. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: Imagenet large scale visual recognition challenge. IJCV 115(3), 211-252 (2015)
34. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)
35. Snoek, J., Larochelle, H., Adams, R.P.: Practical bayesian optimization of machine learning algorithms. In: NIPS (2012)
36. Srinivas, S., Babu, R.V.: Data-free parameter pruning for deep neural networks. In: BMVC (2015)
37. Tsuruoka, Y., Tsujii, J., Ananiadou, S.: Stochastic gradient descent training for l1-regularized log-linear models with cumulative penalty. In: ACL-IJCNLP (2009)
38. Vedaldi, A., Lenc, K.: Matconvnet - convolutional neural networks for matlab. In: ACM MM (2014)
39. Yu, D., Seide, F., Li, G., Deng, L.: Exploiting sparseness in deep neural networks for large vocabulary speech recognition. In: ICASSP (2012)
40. Zeiler, M.D., Ranzato, M., Monga, R., Mao, M., Yang, K., Le, Q., Nguyen, P., Senior, A., Vanhoucke, V., Dean, J., Hinton, G.: On rectified linear units for speech processing. In: ICASSP (2013)