Stochastic model for the vocabulary growth in natural languages

Physical Review X

Volumn 3, Issue 2, 2013, Pages

  Gerlach, Martin ^a   Altmann, Eduardo G ^a  

^a MAX PLANCK INSTITUT FÜR PHYSIK KOMPLEXER SYSTEME   (Germany)

Author keywords

Complex systems; Interdisciplinary physics; Statistical physics

Indexed keywords

DESCRIPTIVE MODEL; HIGHER FREQUENCIES; HISTORICAL CHANGES; INFINITE NUMBERS; INTERDISCIPLINARY PHYSICS; LOWER FREQUENCIES; NATURAL LANGUAGES; STATISTICAL PHYSICS;

LARGE SCALE SYSTEMS; PHYSICS;

DATABASE SYSTEMS;

EID: 84883519135     PISSN: None     EISSN: 21603308     Source Type: Journal    
DOI: 10.1103/PhysRevX.3.021006     Document Type: Article

References (58)

1. J.-B. Michel, Y. K. Shen, A. P. Aiden, A. Veres, M. K. Gray, J. P. Pickett, D. Hoiberg, D. Clancy, P. Norvig, J. Orwant, S. Pinker, M.A. Nowak, and E. L. Aiden, Quantitative Analysis of Culture Using Millions of Digitized Books, Science 331, 176 (2011).
2. J. Gao, J. Hu, X. Mao, and M. Perc, Culturomics Meets Random Fractal Theory: Insights into Long-Range Correlations of Social and Natural Phenomena over the Past Two Centuries, J. R. Soc. Interface 9, 1956 (2012).
3. A. M. Petersen, J. Tenenbaum, S. Havlin, and H. E. Stanley, Statistical Laws Governing Fluctuations in Word Use from Word Birth to Word Death, Sci. Rep. 2, 313 (2012).
4. M. Pagel, Q. D. Atkinson, and A. Meade, Frequency of Word-Use Predicts Rates of Lexical Evolution throughout Indo-European History, Nature (London) 449, 717 (2007).
5. E. Lieberman, J.-P. Michel, J. Jackson, T. Tang, and M. A. Nowak, Quantifying the Evolutionary Dynamics of Language, Nature (London) 449, 713 (2007).
6. D. Levary, J.-P. Eckmann, E. Moses, and T. Tlusty, Loops and Self-Reference in the Construction of Dictionaries, Phys. Rev. X 2, 031018 (2012).
7. G. Wimmer and G. Altmann, Review Article: On Vocabulary Richness, J. Quant. Linguist. 6, 1 (1999).
8. R. H. Baayen, Word Frequency Distributions (Kluwer Academic Publishers, Dordrecht, Netherlands, 2001).
9. R. Baeza-Yates and G. Navarro, Block Addressing Indices for Approximate Text Retrieval, J. Am. Soc. Inf. Sci. 51, 69 (2000).
10. H. E. Williams and J. Zobel, Searchable Words on the Web, Int. J. Digit. Libr. 5, 99 (2005).
11. B. Croft, D. Metzler, and T. Strohmann, Search Engines: Information Retrieval in Practice (Addison-Wesley, Boston, MA, 2009).
12. H. S. Heaps, Information Retrieval: Computational and Theoretical Aspects (Academic Press, New York, 1978).
13. M. A. Serrano, A. Flammini, and F. Menczer, Modeling Statistical Properties of Written Text, PLoS ONE 4, e5372 (2009).
14. S. Bernhardsson, L. E. Correa da Rocha, and P. Minnhagen, The Meta Book and Size-Dependent Properties of Written Language, New J. Phys. 11, 123015 (2009).
15. Y. Sano, H. Takayasu, and M. Takayasu, Zipf's Law and Heaps' Law Can Predict the Size of Potential Words, Prog. Theor. Phys. Suppl. 194, 202 (2012).
16. C. Cattuto, A. Barrat, A. Baldassarri, G. Schehr, and V. Loreto, Collective Dynamics of Social Annotation, Proc. Natl. Acad. Sci. U.S.A. 106, 10511 (2009).
17. R.W. Benz, S. J. Swamidass, and P. Baldi, Discovery of Power-Laws in Chemical Space, J. Chem. Inf. Model. 48, 1138 (2008).
18. G. K. Zipf, The Psycho-Biology of Language (Routledge, London, England, 1936).
19. D. C. van Leijenhorst and T. P. van der Weide, A Formal Derivation of Heaps' Law, Information Sciences (NY) 170, 263 (2005).
20. D. H. Zanette and M. A. Montemurro, Dynamics of Text Generation with Realistic Zipf's Distribution, J. Quant. Linguist. 12, 29 (2005).
21. I. Eliazar, The Growth Statistics of Zipfian Ensembles: Beyond Heaps' Law, Physica (Amsterdam) 390, 3189 (2011).
22. M. A. Montemurro, Beyond the Zipf-Mandelbrot Law in Quantitative Linguistics, Physica (Amsterdam) 300, 567 (2001).
23. W. Li, P. Miramontes, and G. Cocho, Fitting Ranked Linguistic Data with Two-Parameter Functions, Entropy 12, 1743 (2010).
24. G. Jäger, Power Laws and Other Heavy-Tailed Distributions in Linguistic Typology, Adv. Compl. Syst. 15, 1150019 (2012).
25. M. P. H. Stumpf and M. A. Porter, Critical Truths About Power Laws, Science 335, 665 (2012).
26. M. E. J. Newman, Power Laws, Pareto Distributions and Zipf's law, Contemp. Phys. 46, 323 (2005).
27. G. U. Yule, A Mathematical Theory of Evolution, Based on the Conclusions of Dr. J.C. Willis, F.R.S., Phil. Trans. R. Soc. B 213, 21 (1925).
28. M. Mitzenmacher, A Brief History of Generative Models for Power Law and Log-normal Distributions, Internet Math. 1, 226 (2004).
29. M.V. Simkin and V. P. Roychowdhury, Re-inventing Willis, Phys. Rep. 502, 1 (2011).
30. C. Castellano, S. Fortunato, and V. Loreto, Statistical Physics of Social Dynamics, Rev. Mod. Phys. 81, 591 (2009).
31. A. Baronchelli, V. Loreto, and F. Tria, Language Dynamics, Adv. Compl. Syst. 15, 1203002 (2012).
32. See Supplemental Material at http://link.aps.org/ supplemental/10.1103/PhysRevX.3.021006 for further details.
33. B. Mandelbrot, An Informational Theory of the Statistical Structure of Language, Communication Theory (Butterworth, Woburn, MA, 1953), p. 486.
34. J. Tuldava, The Frequency Spectrum of Text and Vocabulary, J. Quant. Linguist. 3, 38 (1996).
35. S. K. Baek, S. Bernhardsson, and P. Minnhagen, Zipf's Law Unzipped, New J. Phys. 13, 043004 (2011).
36. A. Cohen, R. N. Mantegna, and S. Havlin, Numerical Analysis of Word Frequencies in Artificial and Natural Language Texts, Fractals 05, 95 (1997).
37. R. Ferrer i Cancho, The Variation of Zipf's Law in Human Language, Eur. Phys. J. B 44, 249 (2005).
38. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes (Cambridge University Press, Cambridge, England, 2007), 3rd ed.
39. R. B. D'Agostino and M. A. Stephens, Goodness-of-Fit-Techniques (Marcel Dekker, New York, 1986).
40. J. R. Taylor, An Introduction to Error Analysis (University Science Books, Sausalito, CA, 1997).
41. H. Akaike, A New Look at the Statistical Model Identification, IEEE Trans. Autom. Control 19, 716 (1974).
42. K. P. Burnham and D. R. Anderson, Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach (Springer, New York, 2002), 2nd ed.
43. F. Johansson et al. mpmath: A Python Library for Arbitrary-Precision Floating-Point Arithmetic (Version 0.14), 2010, http://code.google.com/p/mpmath/
44. E. Jones et al. SciPy: Open Source Scientific Tools for Python, 2001, http://www.scipy.org/
45. Wikimedia: Dump of the English Wikipedia on 02/06/2012, http://dumps.wikimedia.org/enwiki/20120601/
46. University of Pisa Multimedia Lab: Wikipedia Extractor, http://medialab.di.unipi.it/wiki/Wikipedia_Extractor. See Ref. [32] for access dates.
47. The sharp transition between the two regimes in Eq. (1) might seem artificial. We believe that alternative distributions which interpolate between the two scalings could provide a similarly good account of the data. The advantage of the distribution Eq. (1) is that the transition point r = b appears explicitly as a free parameter and can be independently estimated from data.
48. S. Naranan and V. Balasubrahmanyan, Models for Power Law Relations in Linguistics and Information Science, J. Quant. Linguist. 5, 35 (1998).
49. R. Ferrer i Cancho and R.V. Solé, Two Regimes in the Frequency of Words and the Origins of Complex Lexicons: Zipf's Law Revisited, J. Quant. Linguist. 8, 165 (2001).
50. A. M. Petersen, J. Tenenbaum, S. Havlin, H. E. Stanley, and M. Perc, Languages Cool as They Expand: Allometric Scaling and the Decreasing Need for New Words, Sci. Rep. 2, 943 (2012).
51. B. Mandelbrot, On the Theory of Word Frequencies and on Related Markovian Models of Discourse, Structure of Language and Its Mathematical Aspects: Proceedings of Symposia in Applied Mathematics Vol. XII (American Mathematical Society, Providence, RI, 1961).
52. H. A. Simon, On a Class of Skew Distribution Functions, Biometrika 42, 425 (1955).
53. A. P. Masucci, A. Kalampokis, V. M. Eguíluz, and E. Hernández-García, Wikipedia Information Flow Analysis Reveals the Scale-Free Architecture of the Semantic Space, PLoS ONE 6, e17333 (2011).
54. B. Corominas-Murtra, J. Fortuny, and R.V. Solé, Emergence of Zipf's Law in the Evolution of Communication, Phys. Rev. E 83, 036115 (2011).
55. W. Feller, An Introduction to Probability Theory and Its Applications (Wiley, New York, 1968), 3rd ed., Vol. I.
56. *= 7873) in every single year in one decade and in none of the years in the other decade (ordered by the average frequency in the decade in which they belonged to the core vocabulary).
57. S. N. Dorogovtsev and J. F. F. Mendes, Language as an Evolving Word Web, Proc. R. Soc. Lond. B Biol. Sci. 268, 2603 (2001).
58. N. Blumm, G. Ghoshal, Z. Forró, M. Schich, G. Bianconi, J.-P. Bouchaud, and A.-L. Barabási, Dynamics of Ranking Processes in Complex Systems, Phys. Rev. Lett. 109, 128701 (2012).