Regularization with stochastic transformations and perturbations for deep semi-supervised learning

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 1171-1179

  Sajjadi, Mehdi ^a   Javanmardi, Mehran ^a   Tasdizen, Tolga ^a  

^a University of Utah   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONVOLUTION; NEURAL NETWORKS; STOCHASTIC SYSTEMS;

BENCHMARK DATASETS; CONVOLUTIONAL NEURAL NETWORK; DATA AUGMENTATION; DETERMINISTIC BEHAVIOR; GRADIENT DESCENT; SEMI- SUPERVISED LEARNING; STOCHASTIC NATURE; TIME-CONSUMING TASKS;

SUPERVISED LEARNING;

EID: 85019203095     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (40)

1. B. B. Le Cun, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, "Handwritten digit recognition with a back-propagation network," in Advances in neural information processing systems, Citeseer, 1990.
2. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, Vol. 86, No. 11, pp. 2278-2324, 1998.
3. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, "Overfeat: Integrated recognition, localization and detection using convolutional networks," arXiv preprint arXiv:1312.6229, 2013.
4. J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Computer Vision and Pattern Recognition, pp. 3431-3440, 2015.
5. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al., "Imagenet large scale visual recognition challenge," International Journal of Computer Vision, Vol. 115, No. 3, pp. 211-252, 2015.
6. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A Rabinovich, "Going deeper with convolutions," in Computer Vision and Pattern Recognition, pp. 1-9, 2015.
7. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, pp. 1097-1105, 2012.
8. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors," arXiv preprint arXiv:1207.0580, 2012.
9. D. Ciresan, U. Meier, and J. Schmidhuber, "Multi-column deep neural networks for image classification," in Computer Vision and Pattern Recognition, pp. 3642-3649, IEEE, 2012.
10. A. Blum and T. Mitchell, "Combining labeled and unlabeled data with co-training," in Proceedings of the eleventh annual conference on Computational learning theory, pp. 92-100, ACM, 1998.
11. V. R. de Sa, "Learning classification with unlabeled data," in Advances in neural information processing systems, pp. 112-119, 1994.
12. D. J. Miller and H. S. Uyar, "A mixture of experts classifier with learning based on both labelled and unlabelled data," in Advances in neural information processing systems, pp. 571-577, 1997.
13. T. Joachims, "Transductive inference for text classification using support vector machines," in ICML, Vol. 99, pp. 200-209, 1999.
14. K. Bennett, A. Demiriz, et al., "Semi-supervised support vector machines," Advances in Neural Information processing systems, pp. 368-374, 1999.
15. A. Blum and S. Chawla, "Learning from labeled and unlabeled data using graph mincuts," 2001.
16. X. Zhu, Z. Ghahramani, J. Lafferty, et al., "Semi-supervised learning using gaussian fields and harmonic functions," in International Conference on Machine Learning, Vol. 3, pp. 912-919, 2003.
17. X. Zhu and Z. Ghahramani, "Learning from labeled and unlabeled data with label propagation," tech. rep., Citeseer, 2002.
18. Y. LeCun, K. Kavukcuoglu, C. Farabet, et al., "Convolutional networks and applications in vision.," in ISCAS, pp. 253-256, 2010.
19. K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y LeCun, "What is the best multi-stage architecture for object recognition?," in International Conference on Computer Vision, pp. 2146-2153, IEEE, 2009.
20. K. Kavukcuoglu, M. Ranzato, and Y. LeCun, "Fast inference in sparse coding algorithms with applications to object recognition," arXiv preprint arXiv:1010.3467, 2010.
21. P. Agrawal, J. Carreira, and J. Malik, "Learning to see by moving," in International Conference on Computer Vision, pp. 37-45, 2015.
22. C. Doersch, A. Gupta, and A. A. Efros, "Unsupervised visual representation learning by context prediction," in International Conference on Computer Vision, pp. 1422-1430, 2015.
23. X. Glorot and Y. Bengio, "Understanding the difficulty of training deep feedforward neural networks," in International conference on artificial intelligence and statistics, pp. 249-256, 2010.
24. R. Johnson and T. Zhang, "Semi-supervised convolutional neural networks for text categorization via region embedding," in Advances in Neural Information Processing Systems, pp. 919-927, 2015.
25. J. Weston, F. Ratle, H. Mobahi, and R. Collobert, "Deep learning via semi-supervised embedding," in Neural Networks: Tricks of the Trade, pp. 639-655, Springer, 2012.
26. X. Wang and A. Gupta, "Unsupervised learning of visual representations using videos," in International Conference on Computer Vision, pp. 2794-2802, 2015.
27. D. Jayaraman and K. Grauman, "Learning image representations tied to ego-motion," in International Conference on Computer Vision, pp. 1413-1421, 2015.
28. A. Rasmus, M. Berglund, M. Honkala, H. Valpola, and T. Raiko, "Semi-supervised learning with ladder networks," in Advances in Neural Information Processing Systems, pp. 3532-3540, 2015.
29. A. Dosovitskiy, J. T. Springenberg, M. Riedmiller, and T Brox, "Discriminative unsupervised feature learning with convolutional neural networks," in Advances in Neural Information Processing Systems, pp. 766-774, 2014.
30. M. Sajjadi, M. Javanmardi, and T. Tasdizen, "Mutual exclusivity loss for semi-supervised deep learning," in International Conference on Image Processing, IEEE, 2016.
31. P. Y. Simard, Y. A. LeCun, J. S. Denker, and B Victorri, "Transformation invariance in pattern recognition-tangent distance and tangent propagation," in Neural networks: tricks of the trade, pp. 239-274, Springer, 1998.
32. D. Jayaraman and K. Grauman, "Slow and steady feature analysis: higher order temporal coherence in video," Computer Vision and Pattern Recognition, 2016.
33. L. Sun, K. Jia, T.-H. Chan, Y. Fang, G. Wang, and S Yan, "Dl-sfa: deeply-learned slow feature analysis for action recognition," in Computer Vision and Pattern Recognition, pp. 2625-2632, 2014.
34. A. Krizhevsky and G. Hinton, "Learning multiple layers of features from tiny images," 2009.
35. Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, "Reading digits in natural images with unsupervised feature learning," in NIPS workshop on deep learning and unsupervised feature learning, Vol. 2011, p. 4, Granada, Spain, 2011.
36. Y. LeCun, F. J. Huang, and L. Bottou, "Learning methods for generic object recognition with invariance to pose and lighting," in Computer Vision and Pattern Recognition, Vol. 2, pp. II-97, IEEE, 2004.
37. A. Krizhevskey, "Cuda-convnet." code.google.com/p/cuda-convnet, 2014.
38. B. Graham, "Spatially-sparse convolutional neural networks," arXiv preprint arXiv:1409.6070, 2014.
39. B. Graham, "Fractional max-pooling," arXiv preprint arXiv:1412.6071, 2014.
40. J.-R. Chang and Y.-S. Chen, "Batch-normalized maxout network in network," arXiv preprint arXiv:1511.02583, 2015.