Deep multilingual correlation for improved word embeddings

NAACL HLT 2015 - 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference

Volumn , Issue , 2015, Pages 250-256

  Lu, Ang ^a   Wang, Weiran ^b   Bansal, Mohit ^b   Gimpel, Kevin ^b   Livescu, Karen ^b  

^a TSINGHUA UNIVERSITY   (China)

^b TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; CORRELATION METHODS; LINEAR TRANSFORMATIONS; MATHEMATICAL TRANSFORMATIONS; NATURAL LANGUAGE PROCESSING SYSTEMS; QUALITY CONTROL; SPEECH RECOGNITION;

CANONICAL CORRELATION ANALYSIS; MULTILINGUAL CONTEXT; NAMED ENTITY RECOGNITION; NON-LINEAR TRANSFORMATIONS; PART OF SPEECH TAGGING;

EMBEDDINGS;

EID: 84960116111     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/n15-1028     Document Type: Conference Paper

References (49)

1. Eneko Agirre, Enrique Alfonseca, Keith Hall, Jana Kravalova, Marius Pacsca, and Aitor Soroa. 2009. A study on similarity and relatedness using distributional and wordnet-based approaches. In Proceedings of HLT-NAACL.
2. Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. 2013. Deep canonical correlation analysis. In Proceedings of ICML.
3. Mohit Bansal, John DeNero, and Dekang Lin. 2012. Unsupervised translation sense clustering. In Proceedings of NAACL-HLT.
4. M. Bansal, K. Gimpel, and K. Livescu. 2014. Tailoring continuous word representations for dependency parsing. In Proceedings of ACL.
5. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A neural probabilistic language model. JMLR, 3:1137-1155,March.
6. Jordan Boyd-Graber and David M Blei. 2009. Multilingual topic models for unaligned text. In Proceedings of UAI.
7. Elia Bruni, Nam-Khanh Tran, and Marco Baroni. 2014. Multimodal distributional semantics. J. Artif. Intell. Res.(JAIR), 49:1-47.
8. Sarath Chandar, Stanislas Lauly, Hugo Larochelle, Mitesh Khapra, Balaraman Ravindran, Vikas Raykar, and Amrita Saha. 2014. An autoencoder approach to learning bilingual word representations. In Proceeddings of NIPS.
9. Kai-Wei Chang, Wen-tau Yih, and Christopher Meek. 2013. Multi-relational latent semantic analysis. In Proceedings of EMNLP.
10. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of ICML.
11. Tijl De Bie and Bart De Moor. 2003. On the regularization of canonical correlation analysis. Int. Sympos. ICA and BSS, pages 785-790.
12. Scott C. Deerwester, Susan T. Dumais, Thomas K. Landauer, George W. Furnas, and Richard A. Harshman. 1990. Indexing by latent semantic analysis. JASIS, 41(6):391-407.
13. Paramveer Dhillon, Dean P. Foster, and Lyle H. Ungar. 2011. Multi-view learning of word embeddings via CCA. In Proceedings of NIPS.
14. Mona Diab and Philip Resnik. 2002. An unsupervised method for word sense tagging using parallel corpora. In Proceedings of ACL.
15. C. Dyer, A. Lopez, J. Ganitkevitch, J. Weese, F. Ture, P. Blunsom, H. Setiawan, V. Eidelman, and P. Resnik. 2010. cdec: A decoder, alignment, and learning framework for finite-state and context-free translation models. In Proceedings of ACL.
16. Manaal Faruqui and Chris Dyer. 2014. Improving vector space word representations using multilingual correlation. In Proceedings of EACL.
17. Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. 2001. Placing search in context: The concept revisited. In Proceedings of WWW.
18. Stephan Gouws, Yoshua Bengio, and Greg Corrado. 2014. Bilbowa: Fast bilingual distributed representations without word alignments. arXiv preprint arXiv:1410.2455.
19. Aria Haghighi, Percy Liang, Taylor Berg-Kirkpatrick, and Dan Klein. 2008. Learning bilingual lexicons from monolingual corpora. In Proceedings of ACL.
20. David R. Hardoon, Sandor Szedmak, and John Shawe-Taylor. 2004. Canonical correlation analysis: An overview with application to learning methods. Neural Computation, 16(12):2639-2664, December.
21. Felix Hill, Roi Reichart, and Anna Korhonen. 2014. Simlex-999: Evaluating semantic models with (genuine) similarity estimation. arXiv preprint arXiv:1408.3456.
22. Harold Hotelling. 1936. Relations between two sets of variates. Biometrika, 28(3/4):321-377, December.
23. Fei Huang, Arun Ahuja, Doug Downey, Yi Yang, Yuhong Guo, and Alexander Yates. 2014. Learning representations for weakly supervised natural language processing tasks. Computational Linguistics, 40(1).
24. Ann Irvine and Chris Callison-Burch. 2013. Supervised bilingual lexicon induction with multiple monolingual signals. In Proceedings of HLT-NAACL, pages 518-523.
25. Alexandre Klementiev, Ivan Titov, and Binod Bhattarai. 2012. Inducing crosslingual distributed representations of words.
26. Tomá?s Ko?cisky, Karl Moritz Hermann, Phil Blunsom 2014 Learning bilingual word representations by marginalizing alignments arXiv preprint arXiv: 1405.0947
27. Philipp Koehn and Kevin Knight. 2002. Learning a translation lexicon from monolingual corpora. In Proceedings of the ACL Workshop on Unsupervised Lexical Acquisition.
28. Terry Koo, Xavier Carreras, and Michael Collins. 2008. Simple semi-supervised dependency parsing. In Proceedings of ACL.
29. P. L. Lai and C. Fyfe. 2000. Kernel and nonlinear canonical correlation analysis. Int. J. Neural Syst., 10(5):365-377, October.
30. Yann LeCun, Leon Bottou, Genevieve B. Orr, and Klaus-Robert Müller. 1998. Efficient backprop. volume 1524 of Lecture Notes in Computer Science, pages 9-50, Berlin. Springer-Verlag.
31. Minh-Thang Luong, Richard Socher, and Christopher D. Manning. 2013. Better word representations with recursive neural networks for morphology. In Proceedings of CoNLL.
32. TomasMikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of NIPS.
33. George A Miller and Walter G Charles. 1991. Contextual correlates of semantic similarity. Language and cognitive processes, 6(1):1-28.
34. Scott Miller, Jethran Guinness, and Alex Zamanian. 2004. Name tagging with word clusters and discriminative training. In Proceedings of HLT-NAACL.
35. Jeff Mitchell and Mirella Lapata. 2010. Composition in distributional models of semantics. Cognitive Science, 34(8):1388-1439.
36. Andriy Mnih and Geoffrey Hinton. 2007. Three new graphicalmodels for statistical languagemodelling. In Proceedings of ICML.
37. Yves Peirsman and Sebastian Padó. 2010. Crosslingual induction of selectional preferences with bilingual vector spaces. In Proceedings of HLT-NAACL.
38. Kira Radinsky, Eugene Agichtein, Evgeniy Gabrilovich, and Shaul Markovitch. 2011. A word at a time: computing word relatedness using temporal semantic analysis. In Proceedings of WWW.
39. Herbert Rubenstein and John B Goodenough. 1965. Contextual correlates of synonymy. Communications of the ACM, 8(10):627-633.
40. Eiichiro Sumita. 2000. Lexical transfer using a vectorspace model. In Proceedings of ACL.
41. Oscar Täckström, Ryan McDonald, and Jakob Uszkoreit. 2012. Cross-lingual word clusters for direct transfer of linguistic structure. In Proceedings of HLT-NAACL.
42. Joseph Turian, Lev-Arie Ratinov, and Yoshua Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. In Proceedings of ACL.
43. Alexei Vinokourov, Nello Cristianini, and John Shawe-Taylor. 2003. Inferring a semantic representation of text via cross-language correlation analysis. In Proceedings of NIPS.
44. Ivan Vulic and Marie-Francine Moens. 2013. A study on bootstrapping bilingual vector spaces from nonparallel data (and nothing else). In Proceedings of EMNLP.
45. Weiran Wang, Raman Arora, Karen Livescu, and Jeff Bilmes. 2015. Unsupervised learning of acoustic features via deep canonical correlation analysis. In Proceedings of ICASSP.
46. Dongqiang Yang and David MW Powers. 2006. Verb similarity on the taxonomy of wordnet. Proceedings of GWC-06, pages 121-128.
47. Bing Zhao and Eric P Xing. 2006. Bitam: Bilingual topic admixture models for word alignment. In Proceedings of the COLING/ACL on Main conference poster sessions, pages 969-976. ACL.
48. Bing Zhao, Eric P Xing, and Alex Waibel. 2005. Bilingual word spectral clustering for statistical machine translation. In Proceedings of the ACL Workshop on Building and Using Parallel Texts.
49. Will Y Zou, Richard Socher, Daniel M Cer, and Christopher D Manning. 2013. Bilingual word embeddings for phrase-based machine translation. In Proceedings of EMNLP.