Training deep neural networks on noisy labels with bootstrapping

3rd International Conference on Learning Representations, ICLR 2015 - Workshop Track Proceedings

Volumn , Issue , 2015, Pages

  Reed, Scott E ^a   Lee, Honglak ^a   Anguelov, Dragomir ^b   Szegedy, Christian ^b   Erhan, Dumitru ^b   Rabinovich, Andrew ^b  

^a UNIVERSITY OF MICHIGAN   (United States)

^b GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FORECASTING; IMAGE ENHANCEMENT; OBJECT DETECTION; OBJECT RECOGNITION;

CURRENT PRACTICES; EMOTION RECOGNITION; HANDWRITTEN DIGIT; HIGH RESOLUTION IMAGE; NETWORK FEATURES; STATE OF THE ART; SUPERVISED TRAININGS; VISUAL OBJECT RECOGNITION;

DEEP NEURAL NETWORKS;

EID: 85083952566     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (40)

1. Abney, Steven. Understanding the yarowsky algorithm. Computational Linguistics, 30(3):365–395, 2004. 2
2. Bengio, Yoshua and Thibodeau-Laufer, Eric. Deep generative stochastic networks trainable by backprop. arXiv preprint arXiv:1306.1091, 2013. 3
3. Blum, Avrim and Mitchell, Tom. Combining labeled and unlabeled data with co-training. In Proceedings of the eleventh annual conference on Computational learning theory, pp. 92–100. ACM, 1998. 2
4. Brodley, Carla E and Friedl, Mark A. Identifying mislabeled training data. Journal of Artificial Intelligence Research, 11:131–167, 1999. 2
5. Carlson, Andrew, Betteridge, Justin, Kisiel, Bryan, Settles, Burr, Hruschka Jr, Estevam R, and Mitchell, Tom M. Toward an architecture for never-ending language learning. In AAAI, volume 5, pp. 3, 2010. 3
6. Chen, Xinlei, Shrivastava, Abhinav, and Gupta, Abhinav. Neil: Extracting visual knowledge from web data. In Computer Vision (ICCV), 2013 IEEE International Conference on, pp. 1409–1416. IEEE, 2013. 3
7. Chen, Xinlei, Shrivastava, Abhinav, and Gupta, Abhinav. Enriching visual knowledge bases via object discovery and segmentation. CVPR, 2014. 3
8. Coates, Adam and Ng, Andrew Y. The importance of encoding versus training with sparse coding and vector quantization. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp. 921–928, 2011. 8
9. Erhan, Dumitru, Szegedy, Christian, Toshev, Alexander, and Anguelov, Dragomir. Scalable Object Detection Using Deep Neural Networks. In CVPR, pp. 2155–2162, 2014. 2, 3, 6
10. Girshick, Ross, Donahue, Jeff, Darrell, Trevor, and Malik, Jitendra. Rich feature hierarchies for accurate object detection and semantic segmentation. arXiv preprint arXiv:1311.2524, 2013. 1
11. Goodfellow, Ian, Mirza, Mehdi, Courville, Aaron, and Bengio, Yoshua. Multi-prediction deep boltzmann machines. In Advances in Neural Information Processing Systems, pp. 548–556, 2013. 3
12. Grandvalet, Yves and Bengio, Yoshua. Semi-supervised learning by entropy minimization. In Advances in Neural Information Processing Systems, pp. 529–536, 2005. 2
13. Grandvalet, Yves and Bengio, Yoshua. 9 entropy regularization. 2006. 2, 5
14. Haffari, Gholam Reza and Sarkar, Anoop. Analysis of semi-supervised learning with the yarowsky algorithm. arXiv preprint arXiv:1206.5240, 2012. 2
15. Hinton, Geoffrey E. Training products of experts by minimizing contrastive divergence. Neural computation, 14(8):1771–1800, 2002. 4
16. Hinton, Geoffrey E and Mnih, Volodymyr. Restricted boltzmann machine with hidden multinomial output unit. Unpublished work, 2009. 4
17. Hinton, Geoffrey E, Srivastava, Nitish, Krizhevsky, Alex, Sutskever, Ilya, and Salakhutdinov, Ruslan R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012. 1
18. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097–1105, 2012. 1
19. Larochelle, Hugo and Bengio, Yoshua. Classification using discriminative restricted boltzmann machines. In Proceedings of the 25th international conference on Machine learning, pp. 536–543. ACM, 2008. 3
20. LeCun, Yann and Cortes, Corinna. The mnist database of handwritten digits, 1998. 2
21. Lee, Dong-Hyun. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on Challenges in Representation Learning, ICML, 2013. 3
22. Mnih, Volodymyr and Hinton, Geoffrey E. Learning to label aerial images from noisy data. In Proceedings of the 29th International Conference on Machine Learning (ICML-12), pp. 567–574, 2012. 2
23. Nigam, Kamal and Ghani, Rayid. Analyzing the effectiveness and applicability of co-training. In Proceedings of the ninth international conference on Information and knowledge management, pp. 86–93. ACM, 2000. 2
24. Nigam, Kamal, McCallum, Andrew Kachites, Thrun, Sebastian, and Mitchell, Tom. Text classification from labeled and unlabeled documents using em. Machine learning, 39(2-3):103–134, 2000. 2
25. Ouyang, Wanli, Luo, Ping, Zeng, Xingyu, Qiu, Shi, Tian, Yonglong, Li, Hongsheng, Yang, Shuo, Wang, Zhe, Xiong, Yuanjun, Qian, Chen, et al. Deepid-net: multi-stage and deformable deep convolutional neural networks for object detection. arXiv preprint arXiv:1409.3505, 2014.
26. Reed, Scott, Sohn, Kihyuk, Zhang, Yuting, and Lee, Honglak. Learning to disentangle factors of variation with manifold interaction. In Proceedings of The 31st International Conference on Machine Learning, 2014. 8
27. Rifai, Salah, Bengio, Yoshua, Courville, Aaron, Vincent, Pascal, and Mirza, Mehdi. Disentangling factors of variation for facial expression recognition. In Computer Vision–ECCV 2012, pp. 808–822. Springer, 2012. 8
28. Rosenberg, Chuck, Hebert, Martial, and Schneiderman, Henry. Semi-supervised self-training of object detection models. 2005. 2
29. Russakovsky, Olga, Deng, Jia, Su, Hao, Krause, Jonathan, Satheesh, Sanjeev, Ma, Sean, Huang, Zhiheng, Karpathy, Andrej, Khosla, Aditya, Bernstein, Michael, Berg, Alexander C., and Fei-Fei, Li. ImageNet Large Scale Visual Recognition Challenge, 2014. 2
30. Salakhutdinov, Ruslan and Hinton, Geoffrey E. Deep boltzmann machines. In International Conference on Artificial Intelligence and Statistics, pp. 448–455, 2009. 3
31. Sermanet, Pierre, Eigen, David, Zhang, Xiang, Mathieu, Michaël, Fergus, Rob, and LeCun, Yann. Over-feat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229, 2013. 1
32. Smolensky, Paul. Information processing in dynamical systems: Foundations of harmony theory. 1986. 4
33. Sukhbaatar, Sainbayar and Fergus, Rob. Learning from noisy labels with deep neural networks. arXiv preprint arXiv:1406.2080, 2014. 3
34. Susskind, Josh M, Anderson, Adam K, and Hinton, Geoffrey E. The toronto face database. Department of Computer Science, University of Toronto, Toronto, ON, Canada, Tech. Rep, 2010. 2
35. Szegedy, C., Reed, S., Erhan, D., and Anguelov, D. Scalable, High-Quality Object Detection. ArXiv e-prints, December 2014. 1, 8, 9
36. Szegedy, Christian, Liu, Wei, Jia, Yangqing, Sermanet, Pierre, Reed, Scott, Anguelov, Dragomir, Erhan, Dumitru, Vanhoucke, Vincent, and Rabinovich, Andrew. Going deeper with convolutions. arXiv preprint arXiv:1409.4842, 2014. 8
37. Whitney, Max and Sarkar, Anoop. Bootstrapping via graph propagation. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pp. 620–628. Association for Computational Linguistics, 2012. 2
38. Yarowsky, David. Unsupervised word sense disambiguation rivaling supervised methods. In Proceedings of the 33rd annual meeting on Association for Computational Linguistics, pp. 189–196. Association for Computational Linguistics, 1995. 2, 5, 7
39. Zeiler, Matthew D and Fergus, Rob. Visualizing and understanding convolutional neural networks. arXiv preprint arXiv:1311.2901, 2013. 1
40. Zhu, Xiaojin. Semi-supervised learning literature survey. 2005. 2