Blockout: Dynamic Model Selection for Hierarchical Deep Networks

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 2016-December, Issue , 2016, Pages 2583-2591

  Murdock, Calvin ^a   Li, Zhen ^b   Zhou, Howard ^b   Duerig, Tom ^b  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BACKPROPAGATION; CLASSIFICATION (OF INFORMATION); COMPUTER VISION; HEURISTIC METHODS; PATTERN RECOGNITION;

CLASSIFICATION ACCURACY; DEEP ARCHITECTURES; DYNAMIC MODEL SELECTIONS; HIERARCHICAL ARCHITECTURES; HIERARCHICAL NETWORK STRUCTURE; IMAGE REPRESENTATIONS; MODEL ARCHITECTURE; STRUCTURE-LEARNING;

NETWORK ARCHITECTURE;

EID: 84986277838     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2016.283     Document Type: Conference Paper

References (30)

1. S. Arora, A. Bhaskara, R. Ge, and T. Ma. Provable bounds for learning some deep representations. In International Conference on Machine Learning (ICML), 2014.
2. J. Bergstra and Y. Bengio. Random search for hyperparameter optimization. Journal of Machine Learning Research (JMLR), 13(1):281-305, 2012.
3. S. Branson, G. Van Horn, S. Belongie, and P. Perona. Bird species categorization using pose normalized deep convolutional nets. In British Machine Vision Conference (BMVC), 2014.
4. A. Coates, A. Karpathy, and A. Y. Ng. Emergence of objectselective features in unsupervised feature learning. In Advances in Neural Information Processing Systems (NIPS), 2012.
5. J. Deng, N. Ding, Y. Jia, A. Frome, K. Murphy, S. Bengio, Y. Li, H. Neven, and H. Adam. Large-scale object classification using label relation graphs. In European Conference on Computer Vision (ECCV), 2014.
6. G. Desjardins, K. Simonyan, R. Pascanu, and K. Kavukcuoglu. Natural neural networks. In Advances in Neural Information Processing Systems (NIPS), 2015.
7. B. Graham. Fractional max-pooling. arXiv preprint arXiv:1412.6071, 2014.
8. K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In International Conference on Computer Vision (ICCV), 2015.
9. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.
10. S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning (ICML), 2015.
11. J. Krause, H. Jin, J. Yang, and L. Fei-Fei. Fine-grained recognition without part annotations. In Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
12. A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images, 2009.
13. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NIPS), 2012.
14. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradientbased learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
15. H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In International Conference on Machine Learning (ICML), 2009.
16. T.-Y. Lin, A. RoyChowdhury, and S. Maji. Bilinear cnn models for fine-grained visual recognition. In International Conference on Computer Vision (ICCV), 2015.
17. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), pages 1-42, April 2015.
18. A. Saxe, P.W. Koh, Z. Chen, M. Bhand, B. Suresh, and A. Y. Ng. On random weights and unsupervised feature learning. In International Conference on Machine Learning (ICML), 2011.
19. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations (ICLR), 2014.
20. J. Snoek, H. Larochelle, and R. P. Adams. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems (NIPS), 2012.
21. J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller. Striving for simplicity: The all convolutional net. In International Conference on Learning Representations (ICLR), 2014.
22. R. K. Srivastava, K. Greff, and J. Schmidhuber. Training very deep networks. arXiv preprint arXiv:1507.06228, 2015.
23. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
24. A.-M. Tousch, S. Herbin, and J.-Y. Audibert. Semantic hierarchies for image annotation: A survey. Pattern Recognition, 45(1):333-345, 2012.
25. S. Wager, S. Wang, and P. S. Liang. Dropout training as adaptive regularization. In Advances in Neural Information Processing Systems (NIPS), 2013.
26. L. Wan, M. Zeiler, S. Zhang, Y. L. Cun, and R. Fergus. Regularization of neural networks using dropconnect. In International Conference on Machine Learning (ICML), 2013.
27. D. Warde-Farley, A. Rabinovich, and D. Anguelov. Selfinformed neural network structure learning. In International Conference on Learning Representations (ICLR), 2015.
28. Z. Yan, V. Jagadeesh, D. DeCoste, W. Di, and R. Piramuthu. Hd-cnn: Hierarchical deep convolutional neural network for image classification. In International Conference on Computer Vision (ICCV), 2015.
29. J. Yosinski, J. Clune, Y. Bengio, and H. Lipson. How transferable are features in deep neural networks? In Advances in Neural Information Processing Systems (NIPS), 2014.
30. M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In European Conference on Computer Vision (ECCV), 2014.