Augmenting supervised neural networks with unsupervised objectives for large-scale image classification

33rd International Conference on Machine Learning, ICML 2016

Volumn 2, Issue , 2016, Pages 939-957

  Zhang, Yuting ^a   Lee, Kibok ^a   Lee, Honglak ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION); COMPUTER VISION; LEARNING SYSTEMS; NETWORK ARCHITECTURE; NEURAL NETWORKS; UNSUPERVISED LEARNING;

ARCHITECTURE DESIGNS; HIGH CAPACITY; LABELED IMAGES; LARGE AMOUNTS; LARGE SCALE CLASSIFICATIONS; RESEARCH TOPICS; SUPERVISED AND UNSUPERVISED LEARNING; SUPERVISED NEURAL NETWORKS;

IMAGE CLASSIFICATION;

EID: 84998996019     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (48)

1. Bengio, Y. Learning deep architectures for ai. Foundation and Trends in Machine Learning, 2(1):1-127, January 2009.
2. Bengio, Y, Lamblin, P., Popovici, D., and Larochelle, H. Greedy layer-wise training of deep networks. In NIPS, 2007.
3. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
4. Doersch, C, Gupta, A., and Efros, A. A. Unsupervised visual representation learning by context prediction. In ICCV, 2015.
5. Dong, D. and McAvoy, T. J. Nonlinear principal component analysis based on principal curves and neural networks. Computers & Chemical Engineering, 20(1):65-78, 1996.
6. Dosovitskiy, A. and Brox, T. Inverting visual representations with convolutional networks. CVPR, 2016.
7. Girshick, R., Donahue, J., Darrell, T., and Malik, J. Region-based convolutional networks for accurate object detection and segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(1):142-158, Jan 2016.
8. Goodfellow, I., Mirza, M., Courville, A., and Bengio, Y Multi-prediction deep boltzmann machines. In NIPS, 2013.
9. He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In CVPR, 2016.
10. Hinton, G. E., Osindero, S., and Teh, Y.-W. A fast learning algorithm for deep belief nets. Neural computation, 18(7):1527-1554, 2006.
11. Hochreiter, S., Bengio, Y, Frasconi, P., and Schmidhuber, J. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. In A Field Guide to Dynamical Recurrent Networks. 2001.
12. Ioffe, S. and Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015.
13. Jia, Y, Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. Caffe: Convolutional architecture for fast feature embedding. arXiv: 1408.5093, 2014.
14. Kavukcuoglu, K., Ranzato, M. A., and LeCun, Y Fast inference in sparse coding algorithms with applications to object recognition. arXiv: 1010.3467, 2010.
15. Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
16. Larochelle, H. and Bengio, Y. Classification using discriminative restricted boltzmann machines. In ICML, 2008.
17. LeCun, Y, Bengio, Y, and Hinton, G. Deep learning. Nature, 521(7553):436-444, 2015.
18. Lee, C.-Y, Xie, S., Gallagher, P., Zhang, Z., and Tu, Z. Deeply-supervised nets. In AISTATS, 2015.
19. Lee, H., Grosse, R., Ranganath, R., and Ng, A. Y. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In ICML, 2009.
20. Mahendran, A. and Vedaldi, A. Understanding deep image representations by inverting them. In CVPR, 2015.
21. Mairal, J., Ponce, J., Sapiro, G., Zisserman, A., and Bach, F. R. Supervised dictionary learning. In NIPS, 2009.
22. Makhzani, A. and Frey, B. J. Winner-take-all autoencoders. In NIPS, 2015.
23. Masci, J., Meier, U., Cireşan, D., and Schmidhuber, J. Stacked convolutional auto-encoders for hierarchical feature extraction. In international Conference on Artificial Neural Networks, 2011.
24. Pezeshki, M., Fan, L., Brakel, P., Courville, A., and Bengio, Y. Deconstructing the ladder network architecture. arXiv: 1506.02351, 2016.
25. Ranzato, M. A. and Szummer, M. Semi-supervised learning of compact document representations with deep networks. In ICML, 2008.
26. Ranzato, M. A., Huang, F. J., Boureau, Y.-L., and LeCun, Y Unsupervised learning of invariant feature hierarchies with applications to object recognition. In CVPR, 2007.
27. Rasmus, A., Valpola, H., Honkala, M., Berglund, M., and Raiko, T. Semi-supervised learning with ladder network. In NIPS, 2015.
28. Salakhutdinov, R. and Hinton, G. E. Deep boltzmann machines. In AISTATS, 2009.
29. Scholz, M. and Vigário, R. Nonlinear pca: a new hierarchical approach. In ESANN, 2002.
30. Simonyan, K. and Zisserman, A. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
31. Sohn, K., Zhou, G., Lee, C, and Lee, H. Learning and selecting features jointly with point-wise gated Boltzmann machines. In ICML, 2013.
32. Suddarth, S. and Kergosien, Y. Rule-injection hints as a means of improving network performance and learning time. Neural Networks, 412:120-129, 1990.
33. Szegedy, C, Liu, W., Jia, Y, Sermanet, R, Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. Going deeper with convolutions. In CVPR, 2015.
34. Szegedy, C, Vanhoucke, V, Ioffe, S., Shlens, J., and Wojna, Z. Rethinking the inception architecture for computer vision. 2016.
35. Torralba, A., Fergus, R., and Freeman, W. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(11):1958-1970, Nov. 2008.
36. Valpola, H. From neural PCA to deep unsupervised learning. In Advances in Independent Component Analysis and Learning Machines (Chapter 8), pp. 143-171. 2015.
37. Vincent, R, Larochelle, H., Bengio, Y, and Manzagol, P-A. Extracting and composing robust features with denoising autoencoders. In ICML, 2008.
38. Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y, and Manzagol, P.-A. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research, 11:3371-3408, December 2010.
39. Wang, L., Lee, C.-Y., Tu, Z., and Lazebnik, S. Training deeper convolutional networks with deep supervision. arXiv. l505.02496, 2015.
40. Wang, X. and Gupta, A. Unsupervised learning of visual representations using videos. In ICCV, 2015.
41. Yang, J., Yu, K., and Huang, T. Supervised translationinvariant sparse coding. In CVPR, 2010.
42. Zeiler, M. D. and Fergus, R. Visualizing and understanding convolutional networks. In ECCV, 2014.
43. Zeiler, M. D., Krishnan, D., Taylor, G. W., and Fergus, R. Deconvolutional networks. CVPR, 2010.
44. Zeiler, M., Taylor, G., and Fergus, R. Adaptive deconvolutional networks for mid and high level feature learning. In ICCV, 2011.
45. Zhao, J., Mathieu, M., Goroshin, R., and Lecun, Y. Stacked what-where auto-encoders. arXiv: 1506.02351, 2015.
46. Dosovitskiy, A. and Brox, T. Inverting visual representations with convolutional networks. CVPR, 2016.
47. Pezeshki, M., Fan, L., Brakel, P., Courville, A., and Bengio, Y. Deconstructing the ladder network architecture. arXiv: 1506.02351, 2016.
48. Rasmus, A., Valpola, H., Honkala, M., Berglund, M., and Raiko, T. Semi-supervised learning with ladder network. In NIPS, 2015.