Visual bilingual lexicon induction with transferred convnet features

Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing

Volumn , Issue , 2015, Pages 148-158

  Kiela, Douwe ^a   Vulić, Ivan ^b   Clark, Stephen ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^b UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

NEURAL NETWORKS;

BILINGUAL LEXICONS; CONVOLUTIONAL NEURAL NETWORK; IMAGE-BASED FEATURES; IMPROVE PERFORMANCE; LINGUISTIC APPROACH; STATE OF THE ART; STATE-OF-THE-ART PERFORMANCE; VISUAL SIMILARITY;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 84959920494     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.18653/v1/d15-1015     Document Type: Conference Paper

References (61)

1. Eneko Agirre, Enrique Alfonseca, Keith B. Hall, Jana Kravalova, Marius Pasca, and Aitor Soroa. 2009. A study on similarity and relatedness using distributional and WordNet-based approaches. In NAACL, pages 19-27.
2. Lawrence W. Barsalou and Katja Wiemer-Hastings. 2005. Situating abstract concepts. In Grounding cognition: The role of perception and action in memory, language, and thought, pages 129-163.
3. Lawrence W. Barsalou. 2008. Grounded cognition. Annual Review of Psychology, 59(1):617-645.
4. Herbert Bay, Andreas Ess, Tinne Tuytelaars, and Luc J. Van Gool. 2008. Speeded-up robust features (SURF). Computer Vision and Image Understanding, 110(3):346-359.
5. Shane Bergsma and Randy Goebel. 2011. Using visual information to predict lexical preference. In RANLP, pages 399-405.
6. Shane Bergsma and Benjamin Van Durme. 2011. Learning bilingual lexicons using the visual similarity of labeled web images. In IJCAI, pages 1764-1769.
7. Anna Bosch, Andrew Zisserman, and Xavier Mufioz. 2007. Image classification using random forests and ferns. In ICCV, pages 1-8.
8. Elia Bruni, Nam-Khanh Tran, and Marco Baroni. 2014. Multimodal distributional semantics. Journal of Artifical Intelligence Research, 49:1-47.
9. Navneet Dalai and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In CVPR, pages 886-893.
10. lia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li. 2009. ImageNet: A large-scale hierarchical image database. In CVPR, pages 248-255.
11. Thomas Deselaers, Daniel Keysers, and Hermann Ney. 2008. Features for image retrieval: An experimental comparison. Information Retrieval, 11(2):77-107.
12. Xavier Driancourt and Leon Bottou. 1990. TDNN-extracted features. In Neuro Nimes 90.
13. Yansong Feng and Mirella Lapata. 2010. Visual information in semantic representation. In NAACL, pages 91-99.
14. Robert Fergus, Fei-Fei Li, Pietro Perona, and Andrew Zisserman. 2005. Learning object categories from Google's image search. In ICCV, pages 1816-1823.
15. Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. 2002. Placing Search in Context: The Concept Revisited. ACM Transactions on Information Systems, 20(1):116-131.
16. Andrea Frome, Gregory S. Corrado, lonathon Shlens, Samy Bengio, leffrey Dean, Marc'Aurelio Ranzato, and Tomas Mikolov. 2013. Devise: A deep visualsemantic embedding model. In NIPS, pages 2121-2129.
17. Pascale Fung and Lo Yuen Yee. 1998. An IR approach for translating new words from nonparallel, comparable texts. In ACL, pages 414-420.
18. Eric Gaussier, Jean-Michel Renders, Irina Matveeva, Cyril Goutte, and Herve Dejean. 2004. A geometric view on bilingual lexicon extraction from comparable corpora. In ACL, pages 526-533.
19. Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR.
20. Aria Haghighi, Percy Liang, Taylor Berg-Kirkpatrick, and Dan Klein. 2008. Learning bilingual lexicons from monolingual corpora. In ACL, pages 771-779.
21. Felix Hill and Anna Korhonen. 2014. Learning abstract concept embeddings from multi-modal data: Since you probably can't see what I mean. In EMNLP, pages 255-265.
22. Felix Hill, Roi Reichart, and Anna Korhonen. 2014. SimLex-999: Evaluating semantic models with (genuine) similarity estimation. CoRR, abs/1408.3456.
23. Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross B. Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional architecture for fast feature embedding. In ACM Multimedia, pages 675-678.
24. Douwe Kiela and Leon Bottou. 2014. Learning image embeddings using convolutional neural networks for improved multi-modal semantics. In EMNLP, pages 36-45.
25. Douwe Kiela, Felix Hill, Anna Korhonen, and Stephen Clark. 2014. Improving multi-modal representations using image dispersion: Why less is sometimes more. In ACL, pages 835-841.
26. Ryan Kiros, Ruslan Salakhutdinov, and Richard S. Zemel. 2014. Multimodal neural language models. In ICML, pages 595-603.
27. Philipp Koehn and Kevin Knight. 2002. Learning a translation lexicon from monolingual corpora. In ULA'02 Workshop, pages 9-16.
28. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In NIPS, pages 1106-1114.
29. George Lakoff and Mark Johnson. 1999. Philosophy in the flesh: The embodied mind and its challenge to Western thought.
30. Victor Lavrenko, Martin Choquette, and W. Bruce Croft. 2002. Cross-lingual relevance models. In SIGIR, pages 175-182.
31. Angeliki Lazaridou, Elia Bruni, and Marco Baroni. 2014. Is this a wampimuk? Cross-modal mapping between distributional semantics and the visual world. In ACL, pages 1403-1414.
32. Chee Wee Leong and Rada Mihalcea. 2011. Going beyond text: A hybrid image-text approach for measuring word relatedness. In IJCNLP, pages 1403-1407.
33. Gina-Anne Levow, Douglas Oard, and Philip Resnik. 2005. Dictionary-based techniques for crosslanguage information retrieval. Information Processing & Management, 41:523-547, 2005/05//.
34. Dekang Lin, Kenneth Ward Church, Heng Ji, Satoshi Sekine, David Yarowsky, Shane Bergsma, Kailash Patil, Emily Pitler, Rachel Lathbury, Vikram Rao, Kapil Dalwani, and Sushant Narsale. 2010. New tools for Web-scale N-grams. In LREC, pages 2221-2227.
35. Xiaodong Liu, Kevin Duh, and Yuji Matsumoto. 2013. Topic models + word alignment = A flexible framework for extracting bilingual dictionary from comparable corpus. In CoNLL, pages 212-221.
36. Max M. Louwerse. 2008. Symbol interdependency in symbolic and embodied cognition. Topics in Cognitive Science, 59(1):617-645.
37. David G. Lowe. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91-110.
38. David M. Mimno, Hanna M. Wallach, Jason Naradowsky, David A. Smith, and Andrew McCallum. 2009. Polylingual topic models. In EMNLP, pages 880-889.
39. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In ICML, pages 807-814.
40. Gonzalo Navarro. 2001. A guided tour to approximate string matching. ACM Computing Surveys, 33(1):31-88.
41. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19-51.
42. Maxime Oquab, Leon Bottou, Ivan Laptev, and Josef Sivic. 2014. Learning and transferring mid-level image representations using convolutional neural networks. In CVPR, pages 1717-1724.
43. Reinhard Rapp. 1999. Automatic identification of word translations from unrelated English and German corpora. In ACL.
44. Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. 2014. CNN features offthe-shelf: an astounding baseline for recognition. CoRR, abs/1403.6382.
45. Stephen Roller and Sabine Schulte im Walde. 2013. A multimodal LDA model integrating textual, cognitive and visual modalities. In EMNLP, pages 1146-1157.
46. Charles Schafer and David Yarowsky. 2002. Inducing translation lexicons via diverse similarity measures and bridge languages. In CoNLL, pages 1-7.
47. Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Gregoire Mesnil. 2014. Learning semantic representations using convolutional neural networks for Web search. In WWW, pages 373-374.
48. Daphna Shezaf and Ari Rappoport. 2010. Bilingual lexicon generation using non-aligned signatures. In ACL, pages 98-107.
49. Carina Silberer and Mirella Lapata. 2012. Grounded models of semantic representation. In EMNLP, pages 1423-1433.
50. Carina Silberer and Mirella Lapata. 2014. Learning grounded meaning representations with autoencoders. In ACL, pages 721-732.
51. Carina Silberer, Vittorio Ferrari, and Mirella Lapata. 2013. Models of semantic representation with visual attributes. In ACL, pages 572-582.
52. Josef Sivic and Andrew Zisserman. 2003. Video google: A text retrieval approach to object matching in videos. In ICCV, pages 1470-1477.
53. Richard Socher, Andrej Karpathy, Quoc V. Le, Christopher D. Manning, and Andrew Y. Ng. 2014. Grounded compositional semantics for finding and describing images with sentences. Transactions of ACL, 2:207-218.
54. Nitish Srivastava and Ruslan Salakhutdinov. 2014. Multimodal learning with deep Boltzmann machines. Journal of Machine Learning Research, 15(1):2949-2980.
55. Akihiro Tamura, Taro Watanabe, and Eiichiro Sumita. 2012. Bilingual lexicon extraction from comparable corpora using label propagation. In EMNLP, pages 24-36.
56. Luis von Ahn and Laura Dabbish. 2004. Labeling images with a computer game. In CHI, pages 319-326.
57. Ivan Vulic and Marie-Francine Moens. 2013a. Crosslingual semantic similarity of words as the similarity of their semantic word responses. In NAACL, pages 106-116.
58. Ivan Vulic and Marie-Francine Moens. 2013b. A study on bootstrapping bilingual vector spaces from nonparallel data (and nothing else). In EMNLP, pages 1613-1624.
59. Ivan Vulic, Wim De Smet, and Marie-Francine Moens. 2011. Identifying word translations from comparable corpora using latent topic models. In ACL, pages 479-484.
60. Pengcheng Wu, Steven C. H. Hoi, Hao Xia, Peilin Zhao, Dayong Wang, and Chunyan Miao. 2013. Online multimodal deep similarity learning with application to image retrieval. In ACM Multimedia, pages 153-162.
61. Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014. How transferable are features in deep neural networks? In NIPS, pages 3320-3328.