Diachronie word embeddings reveal statistical laws of semantic change

54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Long Papers

Volumn 3, Issue , 2016, Pages 1489-1501

  Hamilton, William L ^a   Leskovec, Jure ^a   Jurafsky, Dan ^a  

^a Stanford University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATION THEORY; COMPUTATIONAL LINGUISTICS; LINGUISTICS;

CULTURAL EVOLUTION; EMBEDDINGS; HISTORICAL CHANGES; HISTORICAL DATA; INVERSE POWER LAW; SEMANTIC EVOLUTION; STATISTICAL LAWS; WORD FREQUENCIES;

SEMANTICS;

EID: 85011796775     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (61)

1. James S. Adelman, Gordon D. A. Brown, and José F. Quesada. 2006. Contextual diversity, not word frequency, determines word-naming and lexical decision times. Psychol. Sel, 17(9):814-823.
2. Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python. O'Reilly Media, Inc.
3. Andreas Blank. 1999. Why do new meanings occur? A cognitive typology of the motivations for lexical semantic change. In Peter Koch and Andreas Blank, editors, Historical Semantics and Cognition. Walter de Gruyter, Berlin, Germany.
4. Robert Boyd and Peter J Richerson. 1988. Culture and the Evolutionary Process. University of Chicago Press, Chicago, IL.
5. Elia Bruni, Gemma Boleda, Marco Baroni, and Nam-Khanh Tran. 2012. Distributional semantics in technicolor. In Proc. ACL, pages 136-145.
6. Michel Bréal. 1897. Essai de Sémantique: Science des significations. Hachette, Paris, France.
7. John A. Bullinaria and Joseph P. Levy. 2007. Extracting semantic representations from word cooccurrence statistics: A computational study. Behav. Res. Methods, 39(3):510-526.
8. John A. Bullinaria and Joseph P. Levy. 2012. Extracting semantic representations from word cooccurrence statistics: stop-lists, stemming, and SVD. Behav. Res. Methods, 44(3):890-907.
9. J. L. Bybee. 2007. Frequency of Use And the Organization of Language. Oxford University Press, New York City, NY.
10. Paul Cook, Jey Han Lau, Diana McCarthy, and Timothy Baldwin. 2014. Novel Word-sense Identification. In Proc. COLING, pages 1624-1635.
11. Scott Crossley, Tom Salsbury, and Danielle McNamara. 2010. The development of polysemy and frequency use in english second language speakers. Language Learning, 60(3):573-605.
12. Mark Davies. 2010. The Corpus of Historical American English: 400 million words, 1810-2009. http://corpus.byu.edu/coha/.
13. Beate Dorow and Dominic Widdows. 2003. Discovering corpus-specific word senses. In Proc. EACL, pages 79-82.
14. Christiane Fellbaum. 1998. WordNet. Wiley Online Library.
15. Olivier Ferret. 2004. Discovering word senses from a network of lexical cooccurrences. In Proc. COLING, page 1326.
16. J. R. Firth. 1957. A Synopsis of Linguistic Theory, 1930-1955. In Studies in Linguistic Analysis. Special volume of the Philological Society. Basil Blackwell, Oxford, UK.
17. Lea Frermann and Mirella Lapata. 2016. A Bayesian Model of Diachronie Meaning Change. Trans. ACL, 4:31-45.
18. Dirk Geeraerts. 1997. Diachronie Prototype Semantics: A Contribution to Historical Lexicology. Clarendon Press, Oxford, UK.
19. Michael H. Graham. 2003. Confronting multicollinearity in ecological multiple regression. Ecology, 84(11):2809-2815.
20. Kristina Gulordava and Marco Baroni. 2011. A distributional similarity approach to the detection of semantic change in the Google Books Ngram corpus. In Proc. GEMS 2011 Workshop on Geometrical Models of Natural Language Semantics, pages 67-71. Association for Computational Linguistics.
21. Zellig S. Harris. 1954. Distributional structure. Word, 10:146-162.
22. Paul J. Hopper and Elizabeth Closs Traugott. 2003. Grammaticalization. Cambridge University Press, Cambridge, UK.
23. Adam Jatowt and Kevin Duh. 2014. A framework for analyzing semantic change of words across time. In Proc. ACM/IEEE-CS Conf. on Digital Libraries, pages 229-238. IEEE Press.
24. R. Jeffers and Use Lehiste. 1979. Principles and Methods for Historical Linguistics. MIT Press, Cambridge, MA.
25. Adam Kilgarriff. 2004. How dominant is the commonest sense of a word? In Text, Speech and Dialogue, pages 103-111. Springer.
26. Yoon Kim, Yi-I. Chiu, Kentaro Hanaki, Darshan Heg de, and Slav Petrov. 2014. Temporal analysis of language through neural language models. arXiv preprint arXiv:1405.3515.
27. Vivek Kulkarni, Rami Al-Rfou, Bryan Perozzi, and Steven Skiena. 2014. Statistically significant detection of linguistic change. In Proc. WWW, pages 625-635.
28. Thomas K. Landauer and Susan T. Dumais. 1997. A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev, 104(2):211.
29. Jey Han Lau, Paul Cook, Diana McCarthy, David Newman, and Timothy Baldwin. 2012. Word sense induction for novel sense detection. In Proc. EACL, pages 591-601.
30. Omer Levy, Yoav Goldberg, and Ido Dagan. 2015. Improving distributional similarity with lessons learned from word embeddings. Trans. ACL, 3.
31. Qi Li and Jeffrey Scott Racine. 2007. Nonparametric econometrics: theory and practice. Princeton University Press, Princeton, NJ.
32. Erez Lieberman, Jean-Baptiste Michel, Joe Jackson, Tina Tang, and Martin A. Nowak. 2007. Quantifying the evolutionary dynamics of language. Nature, 449(7163):713-716.
33. Yuri Lin, Jean-Baptiste Michel, Erez Lieberman Aiden, Jon Orwant, Will Brockman, and Slav Petrov. 2012. Syntactic annotations for the google books ngram corpus. In Proc. ACL, System Demonstrations, pages 169-174.
34. Charles E McCulloch and John M Neuhaus. 2001. Generalized linear mixed models. Wiley-Interscience, Hoboken, NJ.
35. Jean-Baptiste Michel, Yuan Kui Shen, Aviva Presser Aiden, Adrian Veres, Matthew K. Gray, Joseph P. Pickett, Dale Hoiberg, Dan Clancy, Peter Norvig, Jon Orwant, and others. 2011. Quantitative analysis of culture using millions of digitized books. Science, 331(6014):176-182.
36. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, pages 3111-3119.
37. Sunny Mitra, Ritwik Mitra, Martin Riedl, Chris Biemann, Animesh Mukherjee, and Pawan Goyal. 2014. That's sick dude!: Automatic identification of word sense change across different timescales. In Proc. ACL.
38. Sunny Mitra, Ritwik Mitra, Suman Kalyan Maity, Martin Riedl, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2015. An automatic approach to identify word sense changes in text media across timescales. Natural Language Engineering, 21(05):773-798.
39. Shinichi Nakagawa and Holger Schielzeth. 2013. A general and simple method for obtaining R 2 from generalized linear mixed-effects models. Methods Ecol. Evol, 4(2):133-142.
40. Mark Pagel, Quentin D. Atkinson, and Andrew Meade. 2007. Frequency of word-use predicts rates of lexical evolution throughout Indo-European history. Nature, 449(7163):717-720.
41. Eitan Adam Pechenick, Christopher M. Danforth, and Peter Sheridan Dodds. 2015. Characterizing the Google Books corpus: Strong limits to inferences of socio-cultural and linguistic evolution. PLoS ONE, 10 (10).
42. F. Reali and T. L. Griffiths. 2010. Words as alleles: connecting language evolution with Bayesian learners to models of genetic drift. Proc. R. Soc. B, 277(1680):429-436.
43. Eyal Sagi, Stefan Kaufmann, and Brady Clark. 2011. Tracing semantic change with latent semantic analysis. In Kathryn Allan and Justyna A. Robinson, editors, Current Methods in Historical Semantics, page 161. De Gruyter Mouton, Berlin, Germany.
44. Benoit Sagot, Lionel Clément, Eric de La Clergerie, and Pierre Boullier. 2006. The Lefff 2 syntactic lexicon for French: architecture, acquisition, use. In Proc. LREC, pages 1-4.
45. Gerold Schneider and Martin Volk. 1998. Adding manual constraints and lexical look-up to a Brilltagger for German. In Proceedings of the ESSLLI-98 Workshop on Recent Advances in Corpus Annotation, Saarbrücken.
46. Peter H Schönemann. 1966. A generalized solution of the orthogonal Procrustes problem. Psychometrika, 31(1):1-10.
47. J. S. Seabold and J. Perktold. 2010. Statsmodels: Econometric and statistical modeling with python. In Proc. 9th Python in Science Conference.
48. John Andrew Simpson, Edmund SC Weiner, et al. 1989. The Oxford English Dictionary, volume 2. Clarendon Press Oxford, Oxford, UK.
49. Elizabeth Closs Traugott and Richard B Dasher. 2001. Regularity in Semantic Change. Cambridge University Press, Cambridge, UK.
50. Peter D. Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. J. Artif. Intell. Res., 37(1):141-188.
51. S. Ullmann. 1962. Semantics: An Introduction to the Science of Meaning. Barnes & Noble, New York City, NY.
52. Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research, 9(2579-2605):85.
53. Duncan J Watts and Steven H Strogatz. 1998. Collective dynamics of 'small-world'networks. Nature, 393(6684):440-442.
54. Derry Tanti Wijaya and Reyyan Yeniterzi. 2011. Understanding semantic change of words over centuries. In Proc. Workshop on Detecting and Exploiting Cultural Diversity on the Social Web, pages 35-40. ACM.
55. David P Wilkins. 1993. From part to person: Natural tendencies of semantic change and the search for cognates. Cognitive Anthropology Research Group at the Max Planck Institute for Psycholinguistics.
56. B. Winter, Graham Thompson, and Matthias Urban. 2014. Cognitive Factors Motivating The Evolution Of Word Meanings: Evidence From Corpora, Behavioral Data And Encyclopedic Network Structure. In Proc. EVOLANG, pages 353-360.
57. Yang Xu and Charles Kemp. 2015. A computational evaluation of two laws of semantic change. In Proc. Annual Conf. of the Cognitive Science Society.
58. Yang Xu, Terry Regier, and Barbara C. Malt. 2015. Historical Semantic Chaining and Efficient Communication: The Case of Container Names. Cognitive Science.
59. Naiwen Xue, Fei Xia, Fu-Dong Chiou, and Marta Palmer. 2005. The Penn Chinese TreeBank: Phrase structure annotation of a large corpus. Natural language engineering, 11(02):207-238.
60. George Kingsley Zipf. 1945. The meaning-frequency relationship of words. J. Gen. Psychol, 33(2):251-256.
61. Bahar ílgen and Bahar Karaoglan. 2007. Investigation of Zipf s 'law-of-meaning'on Turkish corpora. In International Symposium on Computer and Information Sciences, pages 1-6. IEEE.