Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 2016-December, Issue , 2016, Pages 2930-2939

  Misra, Ishan ^a   Zitnick, C Lawrence ^c   Mitchell, Margaret ^b   Girshick, Ross ^c  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b MICROSOFT RESEARCH   (United States)

^c FACEBOOK AI RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION (OF INFORMATION); COMPUTER VISION; PATTERN RECOGNITION;

HUMAN-CENTRIC; IMAGE CAPTION; IMAGE CAPTIONING; IMAGE CLASSIFIERS; PHOTO-SHARING SITES; REPORTING BIAS;

IMAGE CLASSIFICATION;

EID: 84986300500     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2016.320     Document Type: Conference Paper

References (58)

1. K. Barnard and D. Forsyth. Learning the semantics of words and pictures. In ICCV, 2001.
2. E. Beigman and B. B. Klebanov. Learning with annotation noise. In ACL-IJCNLP, 2009.
3. A. C. Berg, T. L. Berg, H. Daume III, J. Dodge, A. Goyal, X. Han, A. Mensch, M. Mitchell, A. Sood, K. Stratos, et al. Understanding and predicting importance in images. In CVPR, 2012.
4. A. Borji, D. N. Sihite, and L. Itti. What stands out in a scene? a study of human explicit saliency judgment. Vision research, 91, 2013.
5. O. Chapelle. Modeling delayed feedback in display advertising. In KDD, 2014.
6. X. Chen, H. Fang, T.-Y. Lin, R. Vedantam, S. Gupta, P. Dollar, and C. L. Zitnick. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325, 2015.
7. A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society. Series B (methodological), pages 1-38, 1977.
8. J. Deng, W. Dong, R. Socher, L. jia Li, K. Li, and L. Feifei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
9. J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. In CVPR, 2015.
10. C. Elkan and K. Noto. Learning classifiers from only positive and unlabeled data. In SIGKDD, 2008.
11. M. Everingham, L. Van Gool, C. K. I.Williams, J.Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2007 (VOC2007).
12. H. Fang, S. Gupta, F. N. Iandola, R. Srivastava, L. Deng, P. Dollár, J. Gao, X. He, M. Mitchell, J. C. Platt, C. L. Zitnick, and G. Zweig. From captions to visual concepts and back. In CVPR, 2015.
13. R. Fergus, Y.Weiss, and A. Torralba. Semi-supervised learning in gigantic image collections. In NIPS, 2009.
14. B. Frénay and M. Verleysen. Classification in the presence of label noise: a survey. NNLS, 25, 2014.
15. K. Fukushima. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological cybernetics, 36(4):193-202, 1980.
16. J. Gordon and B. V. Durme. Reporting bias and knowledge extraction. In Automated Knowledge Base Construction (AKBC) 2013: The 3rd Workshop on Knowledge Extraction, at CIKM 2013, AKBC'13, 2013.
17. R. L. Gregory. Eye and brain: The psychology of seeing. 1966.
18. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.
19. H. Izadinia, B. C. Russell, A. Farhadi, M. D. Hoffman, and A. Hertzmann. Deep classifiers from image tags in the wild. In Workshop on Community-Organized Multimodal Mining: Opportunities for Novel Solutions. ACM, 2015.
20. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. In ACMM, 2014.
21. A. Joulin, L. van der Maaten, A. Jabri, and N. Vasilache. Learning visual features from large weakly supervised data. arXiv preprint arXiv:1511.02251, 2015.
22. M. Kearns. Efficient noise-tolerant learning from statistical queries. JACM, 45, 1998.
23. R. Koolen, A. Gatt, M. Goudbeek, and E. Krahmer. Journal of Pragmatics, 43(13):3231-3250, 2011.
24. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
25. A. Kronfeld. Conversationally relevant descriptions. Proceedings of the 27th Annual Meeting of the Association for Computational Linguistics, 1989.
26. J. Larsen, L. Nonboe, M. Hintz-Madsen, and L. K. Hansen. Design of robust neural network classifiers. In Acoustics, Speech and Signal Processing, volume 2, 1998.
27. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541-551, 1989.
28. X. Li and B. Liu. Learning to classify texts using positive and unlabeled data. In IJCAI, volume 3, 2003.
29. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. In ECCV. 2014.
30. B. Liu, Y. Dai, X. Li, W. S. Lee, and P. S. Yu. Building text classifiers using positive and unlabeled examples. In ICDM, 2003.
31. M. M., R. E., and van Deemter K. Typicality and object reference. 2013.
32. N. Manwani and P. Sastry. Noise tolerance under risk minimization. Cybernetics, 43, 2013.
33. G. A. Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39-41, 1995.
34. I. Misra, A. Shrivastava, and M. Hebert. Watch and learn: Semi-supervised learning of object detectors from videos. In CVPR, 2015.
35. V. Mnih and G. E. Hinton. Learning to label aerial images from noisy data. In ICML, 2012.
36. N. Natarajan, I. S. Dhillon, P. K. Ravikumar, and A. Tewari. Learning with noisy labels. In NIPS. 2013.
37. D. F. Nettleton, A. Orriols-Puig, and A. Fornells. A study of the effect of different types of noise on the precision of supervised learning techniques. Artificial intelligence review, 33, 2010.
38. NLPcaffe. http://github.com/Russell91/NLPCaffe.
39. D. R. Olson. Language and thought: Aspects of a cognitive theory of semantics. Psychological review, 77, 1970.
40. S. Reed, H. Lee, D. Anguelov, C. Szegedy, D. Erhan, and A. Rabinovich. Training deep neural networks on noisy labels with bootstrapping. arXiv preprint arXiv:1412.6596, 2014.
41. E. Rosch. Principles of categorization. Concepts: core readings, pages 189-206, 1999.
42. D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations by error propagation. Technical report, DTIC Document, 1985.
43. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. IJCV, 115, 2015.
44. J. C. Sedivy. Pragmatic versus form-based accounts of referential contrast: Evidence for effects of informativity expectations. Journal of psycholinguistic research, 32(1), 2003.
45. A. Shrivastava, S. Singh, and A. Gupta. Constrained semisupervised learning using attributes and comparative attributes. In ECCV. 2012.
46. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
47. S. Sukhbaatar, J. Bruna, M. Paluri, L. Bourdev, and R. Fergus. Training convolutional networks with noisy labels. In ICLR Workshop, 2015.
48. J. M. Tenenbaum. On locating objects by their distinguishing features in multisensory images. Computer Graphics and Image Processing, 2, 1973.
49. B. Thomee, D. A. Shamma, G. Friedland, B. Elizalde, K. Ni, D. Poland, D. Borth, and L.-J. Li. The new data and new challenges in multimedia research. arXiv preprint arXiv:1503.01817, 2015.
50. N. Turakhia and D. Parikh. Attribute dominance: What pops out? In ICCV, 2013.
51. I. van der Sluis, A. Gatt, and K. van Deemter. Manual for the tuna corpus: Referring expressions in two domains. Technical Report AUCS/TR0705, 2006.
52. P. Viola, J. C. Platt, and C. Zhang. Multiple instance boosting for object detection. In NIPS, 2006.
53. H. Westerbeek, R. Koolen, and A. Maes. Stored object knowledge and the production of referring expressions: The case of color typicality. Frontiers in Psychology, 6, 2015.
54. T. Xiao, T. Xia, Y. Yang, C. Huang, and X. Wang. Learning from massive noisy labeled data for image classification. In CVPR, 2015.
55. K. Yun, Y. Peng, D. Samaras, G. J. Zelinsky, and T. Berg. Studying relationships between human gaze, description, and computer vision. In CVPR, 2013.
56. D. Zhou, O. Bousquet, T. N. Lal, J. Weston, and B. Schölkopf. Learning with local and global consistency. Advances in neural information processing systems, 16(16):321-328, 2004.
57. X. Zhu. Semi-supervised learning. In Encyclopedia of Machine Learning. 2010.
58. X. Zhu, Z. Ghahramani, J. Lafferty, et al. Semi-supervised learning using Gaussian fields and harmonic functions. In ICML, volume 3, pages 912-919, 2003.