Identifying missing dictionary entries with frequency-conserving context models

Physical Review E - Statistical, Nonlinear, and Soft Matter Physics

Volumn 92, Issue 4, 2015, Pages

  Williams, Jake Ryland ^a   Clark, Eric M ^a   Bagrow, James P ^a   Danforth, Christopher M ^a   Dodds, Peter Sheridan ^a  

^a Department of Mathematics and Statistics Vermont Complex Systems Center Computational Story Lab and Vermont Advanced Computing Core University of Vermont ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONTEXT MODELS; EXTRACTION TECHNIQUES; LEXICAL UNIT; MOBILITY PATTERN; NATURAL LANGUAGE TEXT; OPEN SOURCES; ORDERED CATEGORICAL DATA; PHRASE FREQUENCY;

CONDENSED MATTER PHYSICS;

EID: 84945207102     PISSN: 15393755     EISSN: 15502376     Source Type: Journal    
DOI: 10.1103/PhysRevE.92.042808     Document Type: Article

References (41)

1. C. E. Shannon, SIGMOBILE Mob. Comput. Commun. Rev. 5, 3 (2001). 1559-1662 10.1145/584091.584093
2. E. T. Jaynes, Phys. Rev. 106, 620 (1957). PHRVAO 0031-899X 10.1103/PhysRev.106.620
3. J. D. Becker, in Proceedings of the 1975 Workshop on Theoretical Issues in Natural Language Processing (Association for Computational Linguistics, Stroudsburg, 1975), pp. 60-63.
4. K. W. Church and P. Hanks, Comput. Linguist. 16, 22 (1990).
5. F. Smadja, Comput. Linguist. 19, 143 (1993).
6. C. Ramisch, Multiword Expressions Acquisition: A Generic and Open Framework (Springer, Berlin, 2014).
7. J. Justeson and S. Katz, Nat. Lang. Eng. 1, 9 (1995). 10.1017/S1351324900000048
8. V. Seretan, Collocation Extraction Based on Syntactic Parsing (Springer, Dordrecht, 2011).
9. P. Pecina, Lang. Resour. Eval. 44, 137 (2010). 1574-020X 10.1007/s10579-009-9101-4
10. S. T. Piantadosi, H. Tily, and E. Gibson, Proc. Natl. Acad. Sci. USA 108, 3526 (2011). PNASA6 0027-8424 10.1073/pnas.1012551108
11. G. K. Zipf, The Psycho-Biology of Language (Houghton-Mifflin, Boston, 1935).
12. J. Reilly and J. Kean, Proc. Natl. Acad. Sci. USA 108, E108 (2011). PNASA6 0027-8424 10.1073/pnas.1103035108
13. S. Piantadosi, H. Tily, and E. Gibson, Proc. Natl. Acad. Sci. USA 108, E109 (2011). PNASA6 0027-8424 10.1073/pnas.1103550108
14. R. Ferrer-i-Cancho and F. M. del Prado Martín, J. Stat. Mech. Theor. Exp. (2011) L12002. 10.1088/1742-5468/2011/12/L12002
15. S. T. Piantadosi, H. Tily, and E. Gibson, arXiv:1307.6726.
16. D. Garcia, A. Garas, and F. Schweitzer, EPJ Data Sci. 1, 3 (2012). 2193-1127 10.1140/epjds3
17. Google research blog: All our n-gram are belong to you, http://googleresearch.blogspot.com/2006/08/all-our-n-gram-are-belong-to-you.html.
18. J. Michel, Y. K. Shen, A. P. Aiden, A. Veres, M. K. Gray, T. G. B. Team, J. P. Pickett, D. Hoiberg, D. Clancy, P. Norvig, Science 331, 176 (2011). SCIEAS 0036-8075 10.1126/science.1199644
19. Y. Lin, J. Michel, E. L. Aiden, J. Orwant, W. Brockman, and S. Petrov, in Proceedings of the ACL 2012 System Demonstrations (Association for Computational Linguistics, Stroudsburg, 2012), pp. 169-174.
20. E. A. Pechenick, C. M. Danforth, and P. S. Dodds, arXiv:1501.00960.
21. R. Reisenauer, K. Smith, and R. A. Blythe, Phys. Rev. Lett. 110, 258701 (2013). PRLTAO 0031-9007 10.1103/PhysRevLett.110.258701
22. P. Resnik, AAAI Report No. WS-92-01, pp. 48-56 (1992), http://www.aaaipress.org/Papers/Workshops/1992/WS-92-01/WS92-01-006.pdf.
23. M. A. Montemurro and D. H. Zanette, Adv. Complex Syst. 13, 135 (2010). 0219-5259 10.1142/S0219525910002530
24. P. S. Dodds and C. M. Danforth, J. Happiness Stud. 11, 441 (2010). 1389-4978 10.1007/s10902-009-9150-9
25. P. S. Dodds, K. D. Harris, I. M. Kloumann, C. A. Bliss, and C. M. Danforth, PLoS ONE 6, e26752 (2011). 1932-6203 10.1371/journal.pone.0026752
26. L. Mitchell, M. R. Frank, K. D. Harris, P. S. Dodds, and C. M. Danforth, PLoS ONE 8, e64417 (2013). 1932-6203 10.1371/journal.pone.0064417
27. P. S. Dodds, E. M. Clark, S. Desu, M. R. Frank, A. J. Reagan, J. R. Williams, L. Mitchell, K. D. Harris, I. M. Kloumann, J. P. Bagrow, Proc. Natl. Acad. Sci. USA 112, 2389 (2015). 10.1073/pnas.1411678112
28. D. Benedetto, E. Caglioti, and V. Loreto, Phys. Rev. Lett. 88, 048702 (2002). PRLTAO 0031-9007 10.1103/PhysRevLett.88.048702
29. J. R. Williams, P. R. Lessard, S. Desu, E. M. Clark, J. P. Bagrow, C. M. Danforth, and P. S. Dodds, Sci. Rep. 5, 12209 (2015). 10.1038/srep12209
30. http://dumps.wikimedia.org/enwiktionary/.
31. A. E. Motter, A. P. S. de Moura, Y.-C. Lai, and P. Dasgupta, Phys. Rev. E 65, 065102 (R) (2002). 1063-651X 10.1103/PhysRevE.65.065102
32. The Collins Cobuild English Language Dictionary, edited by G. Sinclair, P. Hanks, G. Fox, R. Moon, and P. Stock (Collins, Glasgow, 1987).
33. http://www.collinsdictionary.com/dictionary/english-cobuild-learners.
34. https://www.wiktionary.org/.
35. http://www.urbandictionary.com/.
36. http://onlineslangdictionary.com/.
37. E. Sandhaus, The New York Times Annotated Corpus (Linguistic Data Consortium, Philadelphia, 2008).
38. Twitter application programming interface: http://dev.twitter.com/.
39. http://dumps.wikimedia.org/enwiki/.
40. http://www.gutenberg.org.
41. Track the potential missing entries that we have proposed: https://en.wiktionary.org/wiki/User:Jakerylandwilliams/Potential-missing-entries.