Gaze embeddings for zero-shot image classification

Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017

Volumn 2017-January, Issue , 2017, Pages 6412-6421

  Karessli, Nour ^a,b   Akata, Zeynep ^a,c   Schiele, Bernt ^a   Bulling, Andreas ^a  

^a MAX PLANCK INSTITUTE FOR INFORMATICS   (Germany)

^b Eyeem   (Germany)

^c UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; IMAGE CLASSIFICATION;

ANNOTATED DATASETS; AUXILIARY INFORMATION; COMPATIBILITY FUNCTION; DATA COLLECTION; DISCRIMINATION TASKS; DOMAIN EXPERTS; INFORMATION CONTENTS; OBJECT PROPERTY;

CLASSIFICATION (OF INFORMATION);

EID: 85041917180     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2017.679     Document Type: Conference Paper

References (52)

1. Z. Akata, F. Perronnin, Z. Harchaoui, and C. Schmid. Label embedding for attribute-based classification. In CVPR, 2013.
2. Z. Akata, F. Perronnin, Z. Harchaoui, and C. Schmid. Label-embedding for image classification. TPAMI, 2015.
3. Z. Akata, S. Reed, D. Walter, H. Lee, and B. Schiele. Evaluation of output embeddings for fine-grained image classification. In CVPR, 2015.
4. J. Deng, J. Krause, and L. Fei-Fei. Fine-grained crowdsourc-ing for fine-grained recognition. In CVPR, 2013.
5. A. Farhadi, I. Endres, D. Hoiem, and D. Forsyth. Describing objects by their attributes. CVPR, 2009.
6. V. Ferrari and A. Zisserman. Learning visual attributes. In NIPS, 2007.
7. Y. Gong and S. Lazebnik. Comparing data-dependent and data-independent embeddings for classification and ranking of internet images. In CVPR, 2011.
8. J. Harel, C. Koch, and P. Perona. Graph-based visual saliency. In ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 19, pages 545-552. MIT Press, 2007.
9. M. X. Huang, J. Li, G. Ngai, and H. V. Leong. Screenglint: Practical, in-situ gaze estimation on smartphones. In Proceedings of ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), 2017.
10. J. Hyönä, J. Tommola, and A.-M. Alaja. Pupil dilation as a measure of processing load in simultaneous interpretation and other language tasks. The Quarterly Journal of Experimental Psychology, 48(3):598-612, 1995.
11. M. S. L. F.-F. Jia Deng, Jonathan Krause. Leveraging the wisdom of the crowd for fine-grained recognition. TPAMI, in press.
12. T. Judd, K. Ehinger, F. Durand, and A. Torralba. Learning to predict where humans look. In ICCV, pages 2106-2113, 2009.
13. S. Karthikeyan, V. Jagadeesh, R. Shenoy, M. Ecksteinz, and B. Manjunath. From where and how to what we see. In ICCV, pages 625-632, 2013.
14. S. Karthikeyan, T. Ngo, M. Eckstein, and B. Manjunath. Eye tracking assisted extraction of attentionally important objects from videos. In Proc. CVPR, pages 3241-3250, 2015.
15. N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar. Attribute and simile classifiers for face verification. In ICCV, 2009.
16. C. Lampert, H. Nickisch, and S. Harmeling. Attribute-based classification for zero-shot visual object categorization. In TPAMI, 2013.
17. J. Li, Y. Tian, T. Huang, and W. Gao. A dataset and evaluation methodology for visual saliency in video. In ICME, pages 442-445, 2009.
18. Y. Li, X. Hou, C. Koch, J. M. Rehg, and A. L. Yuille. The secrets of salient object segmentation. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR'14, Washington, DC, USA, 2014. IEEE Computer Society.
19. J. Liu, B. Kuipers, and S. Savarese. Recognizing human actions by attributes. In CVPR, 2011.
20. M. Livne, L. Sigal, N. F. Troje, and D. J. Fleet. Human attributes from 3d pose tracking. Computer Vision and Image Understanding, 116(5), 2012.
21. E. Marinoiu, D. Papava, and C. Sminchisescu. Pictorial Human Spaces. How Well do Humans Perceive a 3D Articulated Pose? In ICCV, 2013.
22. S. Mathe and C. Sminchisescu. Multiple instance reinforcement learning for efficient weakly-supervised detection in images. arXiv preprint arXiv:1412.0100, 2014.
23. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In NIPS, 2013.
24. G. A. Miller. Wordnet: a lexical database for english. CACM, 38:39-41, 1995.
25. A. Mishra, Y. Aloimonos, and C. L. Fah. Active segmentation with fixation. In Proc. ICCV, pages 468-475, 2009.
26. F. Moosmann, D. Larlus, and F. Jurie. Learning saliency maps for object categorization. 2006.
27. M.-E. Nilsback and A. Zisserman. Automated flower classification over a large number of classes. In ICCVGI, 2008.
28. M. Palatucci, D. Pomerleau, G. Hinton, and T. Mitchell. Zero-shot learning with semantic output codes. In NIPS, 2009.
29. D. P. Papadopoulos, A. D. F. Clarke, F. Keller, and V. Ferrari. Training object class detectors from eye tracking data. In ECCV, 2014.
30. D. Parikh and K. Grauman. Relative attributes. In ICCV, 2011.
31. O. M. Parkhi, A. Vedaldi, A. Zisserman, and C. V. Jawahar. Cats and dogs. In CVPR, 2012.
32. J. Pennington, R. Socher, and C. D. Manning. Glove: Global vectors for word representation. In EMNLP, 2014.
33. Q. Qiu, Z. Jiang, and R. Chellappa. Sparse dictionary-based representation and recognition of action attributes. In ICCV, 2011.
34. S. Reed, Z. Akata, H. Lee, and B. Schiele. Learning deep representations of fine-grained visual descriptions. In IEEE Computer Vision and Pattern Recognition (CVPR), 2016.
35. M. Rohrbach, M. Stark, and B.Schiele. Evaluating knowledge transfer and zero-shot learning in a large-scale setting. In CVPR, 2011.
36. U. Rutishauser, D. Walther, C. Koch, and P. Perona. Is bottom-up attention useful for object recognition? In CVPR, volume 2, 2004.
37. D. D. Salvucci and J. H. Goldberg. Identifying fixations and saccades in eye-tracking protocols. In Proceedings of the 2000 symposium on Eye tracking research & applications, pages 71-78. ACM, 2000.
38. H. Sattar, S. Müller, M. Fritz, and A. Bulling. Prediction of search targets from fixations in open-world settings. In CVPR, pages 981-990, 2015.
39. I. Shcherbatyi, A. Bulling, and M. Fritz. GazeDPM: Early Integration of Gaze Information in Deformable Part Models. arxiv:1505.05753, 2015.
40. B. Siddiquie, R. Feris, and L. Davis. Image ranking and retrieval based on multi-attribute queries. In CVPR, 2011.
41. R. Socher, M. Ganjoo, H. Sridhar, O. Bastani, C. Manning, and A. Ng. Zero-shot learning through cross-modal transfer. In NIPS, 2013.
42. R. Subramanian, V. Yanulevskaya, and N. Sebe. Can computers learn from humans to see better?: inferring scene semantics from viewers' eye movements. In MM, pages 33-42, 2011.
43. Y. Sugano and A. Bulling. Seeing with humans: Gaze-assisted neural image captioning. arxiv:1608.05203, 2016.
44. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In CVPR, 2015.
45. I. Tsochantaridis, T. Joachims, T. Hofmann, and Y. Altun. Large margin methods for structured and interdependent output variables. JMLR, 2005.
46. G. Wang and D. Forsyth. Joint learning of visual attributes, object classes and visual saliency. In ICCV, 2009.
47. Y. Wang and G. Mori. A discriminative latent model of object classes and attributes. In ECCV, 2010.
48. P. Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Be-longie, and P. Perona. Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-001, Caltech, 2010.
49. J. Weston, S. Bengio, and N. Usunier. Wsabie: Scaling up to large vocabulary image annotation. In IJCAI, 2011.
50. Y. Xian, Z. Akata, G. Sharma, Q. Nguyen, M. Hein, and B. Schiele. Latent embeddings for zero-shot classification. In CVPR, 2016.
51. B. Yao, X. Jiang, A. Khosla, A. L. Lin, L. J. Guibas, and F.-F. Li. Human action recognition by learning bases of action attributes and parts. In ICCV, 2011.
52. K. Yun, Y. Peng, D. Samaras, G. J. Zelinsky, and T. L. Berg. Studying relationships between human gaze, description, and computer vision. In CVPR, pages 739-746, 2013.