Training convolutional networks with noisy labels

3rd International Conference on Learning Representations, ICLR 2015 - Workshop Track Proceedings

Volumn , Issue , 2015, Pages

  Sukhbaatar, Sainbayar ^a   Bruna, Joan ^b   Paluri, Manohar ^b   Bourdev, Lubomir ^b   Fergus, Rob ^b  

^a POLYTECHNIC UNIVERSITY   (United States)

^b FACEBOOK AI RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION (OF INFORMATION); CONVOLUTION; LARGE DATASET;

CONVOLUTIONAL NETWORKS; LABELED DATASETS; LARGE SCALE EXPERIMENTS; MANUAL ANNOTATION; NOISY LABELS; SIMPLE MODIFICATIONS; TRAINING INFRASTRUCTURES; TRAINING PROCESS;

NETWORK LAYERS;

EID: 85083950731     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (37)

1. Barandela, Ricardo and Gasca, Eduardo. Decontamination of training samples for supervised pattern recognition methods. In Advances in Pattern Recognition, volume 1876 of Lecture Notes in Computer Science, pp. 621–630. Springer, 2000.
2. Bootkrajang, Jakramate and Kabn, Ata. Label-noise robust logistic regression and its applications. In Machine Learning and Knowledge Discovery in Databases, volume 7523 of Lecture Notes in Computer Science, pp. 143–158. Springer, 2012.
3. Brodley, Carla E. and Friedl, Mark A. Identifying mislabeled training data. Journal of Artificial Intelligence Research, 11:131–167, 1999.
4. Collobert, Ronan, Kavukcuoglu, Koray, and Farabet, Clément. Torch7: A matlab-like environment for machine learning. In BigLearn, NIPS Workshop, 2011.
5. Deng, Jia, Dong, Wei, Socher, R., Li, Li-Jia, Li, Kai, and Fei-Fei, Li. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp. 248–255, June 2009.
6. Erhan, Dumitru, Bengio, Yoshua, Courville, Aaron, Manzagol, Pierre-Antoine, Vincent, Pascal, and Bengio, Samy. Why does unsupervised pre-training help deep learning? J. Mach. Learn. Res., 11:625–660, March 2010.
7. Fergus, Rob, Weiss, Yair, and Torralba, Antonio. Semi-supervised learning in gigantic image collections. In Bengio, Y., Schuurmans, D., Lafferty, J.D., Williams, C.K.I., and Culotta, A. (eds.), Advances in Neural Information Processing Systems 22, pp. 522–530. 2009.
8. Frenay, B. and Verleysen, M. Classification in the presence of label noise: A survey. Neural Networks and Learning Systems, IEEE Transactions on, 25(5):845–869, May 2014.
9. Guyon, Isabelle, Matic, Nada, and Vapnik, Vladimir. Discovering informative patterns and data cleaning. In Advances in Knowledge Discovery and Data Mining, pp. 181–203. 1996.
10. Hinton, G. E. and Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science, 313(5786):504–507, 2006.
11. Ipeirotis, Panagiotis G., Provost, Foster, and Wang, Jing. Quality management on amazon mechanical turk. In Proceedings of the ACM SIGKDD Workshop on Human Computation, HCOMP’10, pp. 64–67. ACM, 2010.
12. Jia, Yangqing, Shelhamer, Evan, Donahue, Jeff, Karayev, Sergey, Long, Jonathan, Girshick, Ross, Guadarrama, Sergio, and Darrell, Trevor. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093, 2014.
13. Krizhevsky, Alex. cuda-convnet. https://code.google.com/p/cuda-convnet/, 2012.
14. Krizhevsky, Alex and Hinton, Geoffrey. Learning multiple layers of features from tiny images. Computer Science Department, University of Toronto, Tech. Rep, 2009.
15. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25, pp. 1097–1105, 2012.
16. Larsen, J., Nonboe, L., Hintz-Madsen, M., and Hansen, L.K. Design of robust neural network classifiers. In Acoustics, Speech and Signal Processing, 1998. Proceedings of the 1998 IEEE International Conference on, volume 2, pp. 1205–1208 vol.2, May 1998.
17. Le, Q.V. Building high-level features using large scale unsupervised learning. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, pp. 8595–8598, May 2013.
18. LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., and Jackel, L. D. Back-propagation applied to handwritten zip code recognition. Neural Computation, 1(4):541–551, 2014/11/13 1989.
19. Lecun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, Nov 1998.
20. Lee, Honglak, Grosse, Roger, Ranganath, Rajesh, and Ng, Andrew Y. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML’09, pp. 609–616. ACM, 2009.
21. Levin, A., Weiss, Y., Durand, F., and Freeman, W. T. Understanding and evaluating blind deconvolution algorithms. In CVPR, 2009.
22. Mnih, Volodymyr and Hinton, Geoffrey. Learning to label aerial images from noisy data. In Proceedings of the 29th International Conference on Machine Learning (ICML-12), pp. 567–574, 2012.
23. Natarajan, Nagarajan, Dhillon, Inderjit, Ravikumar, Pradeep, and Tewari, Ambuj. Learning with noisy labels. In Advances in Neural Information Processing Systems 26, pp. 1196–1204. 2013.
24. Nettleton, David, Orriols-Puig, Albert, and Fornells, Albert. A study of the effect of different types of noise on the precision of supervised learning techniques. Artificial Intelligence Review, 33(4):275–306, 2010.
25. Netzer, Yuval, Wang, Tao, Coates, Adam, Bissacco, Alessandro, Wu, Bo, and Ng, Andrew Y. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
26. Pechenizkiy, M., Tsymbal, A., Puuronen, S., and Pechenizkiy, O. Class noise and supervised learning in medical domains: The effect of feature extraction. In Computer-Based Medical Systems, 2006. CBMS 2006. 19th IEEE International Symposium on, pp. 708–713, 2006.
27. Sermanet, Pierre, Eigen, David, Zhang, Xiang, Mathieu, Michael, Fergus, Rob, and LeCun, Yann. Overfeat: Integrated recognition, localization and detection using convolutional networks. In International Conference on Learning Representations (ICLR 2014), April 2014.
28. Simonyan, Karen and Zisserman, Andrew. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
29. Taigman, Yaniv, Yang, Ming, Ranzato, Marc’Aurelio, and Wolf, Lior. DeepFace: Closing the Gap to Human-Level Performance in Face Verification. Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
30. Talwalkar, A., Kumar, S., and Rowley, H. Large-scale manifold learning. In CVPR, 2008.
31. Torralba, A., Fergus, R., and Freeman, W.T. 80 million tiny images: A large data set for nonparametric object and scene recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 30(11):1958–1970, Nov 2008.
32. van den Hout, Ardo and van der Heijden, Peter G.M. Randomized response, statistical disclosure control and misclassificatio: a review. International Statistical Review, 70(2):269–288, 2002.
33. Zhou, D., Bousquet, O., Lal, T., Weston, J., and Scholkopf, B. Learning with local and global consistency. In NIPS, 2004.
34. Zhu, X. and Lafferty, J. Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semi-supervised learning. In ICML, 2005.
35. Zhu, X., Ghahramani, Z., and Laffery, J. Semi-supervised learning using gaussian fields and harmonic functions. In ICML, 2003.
36. Zhu, Xiaojin. Semi-supervised learning literature survey. In Computer Sciences TR 1530, University of Wisconsin Madison, 2008.
37. Zhu, Xingquan and Wu, Xindong. Class noise vs. attribute noise: A quantitative study. Artificial Intelligence Review, 22(3):177–210, 2004.