Learning Aligned Cross-Modal Representations from Weakly Aligned Data

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 2016-December, Issue , 2016, Pages 2940-2949

  Castrejon, Lluis ^a   Aytar, Yusuf ^b   Vondrick, Carl ^b   Pirsiavash, Hamed ^c   Torralba, Antonio ^a  

^a UNIVERSITY OF TORONTO   (Canada)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^c UNIVERSITY OF MARYLAND BALTIMORE COUNTY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; CONVOLUTION; NEURAL NETWORKS;

CONVOLUTIONAL NEURAL NETWORK; CROSS-MODAL; CROSS-MODAL REPRESENTATIONS; INTERMEDIATE REPRESENTATIONS; NATURAL IMAGES; SCENE REPRESENTATION; SHARED REPRESENTATIONS;

PATTERN RECOGNITION;

EID: 84986322655     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2016.321     Document Type: Conference Paper

References (47)

1. Y. Aytar and A. Zisserman. Part level transfer regularization for enhancing exemplar svms. In Computer Vision and Image Understanding, 2015.
2. J. Ba, K. Swersky, S. Fidler, and R. Salakhutdinov. Predicting deep zero-shot convolutional neural networks using textual descriptions. arXiv, 2015.
3. E. Bart and S. Ullman. Cross-generalization: Learning novel classes from a single example by feature replacement. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, volume 1, pages 672-679. IEEE, 2005.
4. Y. Bengio, J. Louradour, R. Collobert, and J. Weston. Curriculum learning. In ICML, 2009.
5. J. Bromley, J. W. Bentz, L. Bottou, I. Guyon, Y. LeCun, C. Moore, E. Säckinger, and R. Shah. Signature verification using a siamese time delay neural network. International Journal of Pattern Recognition and Artificial Intelligence, 7(04):669-688, 1993.
6. J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. arXiv preprint arXiv:1310.1531, 2013.
7. A. Dosovitskiy and T. Brox. Inverting convolutional networks with convolutional networks. arXiv, 2015.
8. M. Eitz, K. Hildebrand, T. Boubekeur, and M. Alexa. Sketch-based image retrieval: Benchmark and bag-offeatures descriptors. TVCG, 2011.
9. M. Elhoseiny, B. Saleh, and A. Elgammal. Write a classifier: Zero-shot learning using purely textual descriptions. In ICCV, 2013.
10. L. Fei-Fei, R. Fergus, and P. Perona. One-shot learning of object categories. TPAMI, 2006.
11. M. Fink. Object classification from a single example utilizing class relevance metrics. NIPS, 2005.
12. D. F. Fouhey and C. L. Zitnick. Predicting object dynamics in scenes. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 2027-2034. IEEE, 2014.
13. A. Frome, G. S. Corrado, J. Shlens, S. Bengio, J. Dean, T. Mikolov, et al. Devise: A deep visual-semantic embedding model. In NIPS, 2013.
14. T. Gao, M. Stark, and D. Koller. What makes a good detector?-structured priors for learning from few examples. In Computer Vision-ECCV 2012, pages 354-367. Springer, 2012.
15. R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, 2014.
16. R. Gopalan, R. Li, and R. Chellappa. Domain adaptation for object recognition: An unsupervised approach. In ICCV, 2011.
17. D. R. Hardoon, S. Szedmak, and J. Shawe-Taylor. Canonical correlation analysis: An overview with application to learning methods. Neural computation, 16(12):2639-2664, 2004.
18. D. Hoiem, A. A. Efros, and M. Hebert. Geometric context from a single image. In Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on, volume 1, pages 654-661. IEEE, 2005.
19. Y. Jia, M. Salzmann, and T. Darrell. Learning cross-modality similarity for multinomial data. In ICCV, 2011.
20. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the ACM International Conference on Multimedia, pages 675-678. ACM, 2014.
21. A. Khosla, T. Zhou, T. Malisiewicz, A. A. Efros, and A. Torralba. Undoing the damage of dataset bias. In ECCV, 2012.
22. R. Kiros, R. Salakhutdinov, and R. S. Zemel. Unifying visual-semantic embeddings with multimodal neural language models. arXiv, 2014.
23. R. Kiros, Y. Zhu, R. Salakhutdinov, R. S. Zemel, A. Torralba, R. Urtasun, and S. Fidler. Skip-thought vectors. arXiv, 2015.
24. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
25. C. H. Lampert, H. Nickisch, and S. Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In CVPR, 2009.
26. M. Long and J. Wang. Learning transferable features with deep adaptation networks. arXiv preprint arXiv:1502.02791, 2015.
27. K. Murphy and W. Freeman. Contextual models for object detection using boosted random fields. NIPS, 2004.
28. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng. Multimodal deep learning. In Proceedings of the 28th international conference on machine learning (ICML-11), pages 689-696, 2011.
29. M. Norouzi, T. Mikolov, S. Bengio, Y. Singer, J. Shlens, A. Frome, G. S. Corrado, and J. Dean. Zero-shot learning by convex combination of semantic embeddings. arXiv, 2013.
30. A. Owens, P. Isola, J. McDermott, A. Torralba, E. H. Adelson, and W. T. Freeman. Visually indicated sounds. arXiv preprint arXiv:1512.08512, 2015.
31. M. Palatucci, D. Pomerleau, G. E. Hinton, and T. M. Mitchell. Zero-shot learning with semantic output codes. In NIPS, 2009.
32. V. Ranjan, N. Rasiwasia, and C. Jawahar. Multi-label crossmodal retrieval. In Proceedings of the IEEE International Conference on Computer Vision, pages 4094-4102, 2015.
33. N. Rasiwasia, J. Costa Pereira, E. Coviello, G. Doyle, G. R. Lanckriet, R. Levy, and N. Vasconcelos. A new approach to cross-modal multimedia retrieval. In ICM, 2010.
34. N. Rasiwasia, D. Mahajan, V. Mahadevan, and G. Aggarwal. Cluster canonical correlation analysis. In AISTATS, pages 823-831, 2014.
35. K. Saenko, B. Kulis, M. Fritz, and T. Darrell. Adapting visual category models to new domains. In ECCV, 2010.
36. R. Socher, M. Ganjoo, C. D. Manning, and A. Ng. Zero-shot learning through cross-modal transfer. In Advances in neural information processing systems, pages 935-943, 2013.
37. A. Torralba, A. Efros, et al. Unbiased look at dataset bias. In CVPR, 2011.
38. E. Tzeng, J. Hoffman, T. Darrell, and K. Saenko. Simultaneous deep transfer across domains and tasks. In Proceedings of the IEEE International Conference on Computer Vision, pages 4068-4076, 2015.
39. C. Vondrick, A. Khosla, T. Malisiewicz, and A. Torralba. Hoggles: Visualizing object detection features. In Proceedings of the IEEE International Conference on Computer Vision, pages 1-8, 2013.
40. C. Vondrick, H. Pirsiavash, A. Oliva, and A. Torralba. Learning visual biases from human imagination. In Advances in Neural Information Processing Systems, 2015.
41. F.Wang, L. Kang, and Y. Li. Sketch-based 3d shape retrieval using convolutional neural networks. arXiv, 2015.
42. J. Xiao, J. Hays, K. Ehinger, A. Oliva, A. Torralba, et al. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR, 2010.
43. K. Xu, J. Ba, R. Kiros, A. Courville, R. Salakhutdinov, R. Zemel, and Y. Bengio. Show, attend and tell: Neural image caption generation with visual attention. arXiv preprint arXiv:1502.03044, 2015.
44. B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba. Object detectors emerge in deep scene cnns. arXiv, 2014.
45. B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning deep features for scene recognition using places database. In NIPS, 2014.
46. C. L. Zitnick and D. Parikh. Bringing semantics into focus using visual abstraction. In CVPR, 2013.
47. C. L. Zitnick, D. Parikh, and L. Vanderwende. Learning the visual interpretation of sentences. In Computer Vision (ICCV), 2013 IEEE International Conference on, pages 1681-1688. IEEE, 2013.