Numerical analysis of word frequencies in artificial and natural language texts

Fractals

Volumn 5, Issue 1, 1997, Pages 95-104

  Cohen, A ^a   Mantegna, R N ^b,c   Havlin, S ^a  

^a BAR ILAN UNIVERSITY   (Israel)

^b INFM   (Italy)

^c UNIVERSITY OF PALERMO   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0002999358     PISSN: 0218348X     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0218348X97000103     Document Type: Article

References (23)

1. G. K. Zipf, Human Behavior and the Principle of Least Effort (Addison-Wesley Press, 1949).
2. G. K. Zipf, The Psycho-Biology of Language, An Introduction to Dynamic Philology (MIT Press, Cambridge MA, 1965).
3. B. Mandelbrot, Oligopoly, Mergers, and Paretian Size Distribution of Firms, Research note #NC-246 (Thomas J. Watson Research Center, Yorktown Heights, NY, 1963).
4. B. Mandelbrot, A Class of Long-tailed Probability Distributions and the Empirical Distribution of City Sizes, in Mathematical Explorations in Behavioral Science, eds. F. Massarik and P. Ratoosh (R. D. Irwin, Inc and The Dorsey Press, Homewood, IL, 1965).
5. G. Nicolis, C. Nicolis and J. S. Nicolis, "Chaotic Dynamics, Markov Partitions and Zipf's Law," J. Stat. Phys. 54, 915 (1989).
6. M. Yu. Borodovsky and S. M. Gusein-Zade, "A General Rule for Ranged Series of Codon Frequencies in Different Genomes," J. Biomolecular Struc. Dyn. 6, 1001 (1989).
7. R. N. Mantegna, S. V. Buldyrev, A. L. Goldberger, S. Havlin, C. K. Peng, M. Simons and H. E. Stanley, "Linguistic Features of Noncoding DNA Sequences," Phys. Rev. Lett. 73, 3169 (1994); "Systematic Analysis of Coding and Noncoding DNA Sequences Using Methods of Statistical Linguistics," Phys. Rev. E52, 2939 (1995).
8. R. N. Mantegna, S. V. Buldyrev, A. L. Goldberger, S. Havlin, C. K. Peng, M. Simons and H. E. Stanley, "Linguistic Features of Noncoding DNA Sequences," Phys. Rev. Lett. 73, 3169 (1994); "Systematic Analysis of Coding and Noncoding DNA Sequences Using Methods of Statistical Linguistics," Phys. Rev. E52, 2939 (1995).
9. J. P. Bouchaud, "More Lévy Distributions in Physics," in Lévy Flights and Related Topics in Physics, eds. M. F. Shlesinger, G. M. Zaslavsky and U. Frish (Springer, Berlin, 1995), pp. 239-250; H. E. Stanley, S. V. Buldyrev, A. L. Goldberger, S. Havlin, R. N. Mantegna, C. K. Peng, M. Simons and M. H. R. Stanley, "Long-Range Correlations and Generalized Lévy Walks in DNA Sequences," in Lévy Flights and Related Topics in Physics, eds. M. F. Shlesinger, G. M. Zaslavsky and U. Frish (Springer, Berlin, 1995), pp. 331-347.
10. J. P. Bouchaud, "More Lévy Distributions in Physics," in Lévy Flights and Related Topics in Physics, eds. M. F. Shlesinger, G. M. Zaslavsky and U. Frish (Springer, Berlin, 1995), pp. 239-250; H. E. Stanley, S. V. Buldyrev, A. L. Goldberger, S. Havlin, R. N. Mantegna, C. K. Peng, M. Simons and M. H. R. Stanley, "Long-Range Correlations and Generalized Lévy Walks in DNA Sequences," in Lévy Flights and Related Topics in Physics, eds. M. F. Shlesinger, G. M. Zaslavsky and U. Frish (Springer, Berlin, 1995), pp. 331-347.
11. M. H. R. Stanley, S. V. Buldyrev, S. Havlin, R. N. Mantegna, M. A. Salinger and H. E. Stanley, "Zipf Plots and the Size Distribution of Firms," Economic Lett. 49, 453 (1995).
12. B. Mandelbrot, "An Informational Theory of the Statistical Structure of Language," in Communication Theory, ed. W. Jackson (Butterworths Scientific Publications, London, 1953).
13. B. Mandelbrot, "On the Theory of Word Frequencies and on Related Markovian Models of Discourse," in Proceedings of Symposia in Applied Mathematics, Vol. XII: Structure of Language and Its Mathematical Aspects, ed., R. Jakobson (American Mathematical Society, 1961), pp. 190-219.
14. B. Mandelbrot, Information Theory and Psycholinguistics: A Theory of Words Frequencies, in Readings in Mathematical Social Science, eds. P. F. Lazersfeld and N. W. Henry (MIT Press, 1966).
15. H. A. Simon, "On a Class of Skew Distribution Functions," Biometrika 42, 435-440 (1955).
16. W. Li, "Random Texts Exhibit Zipf's-Law-Like Word Frequency Distribution," IEEE Trans. on Inf. Theory 38, No. 6 (1992).
17. M. Damashek, "Gauging Similarity with n-Grams: Language-Independent Categorization of Text," Science 267, 843 (1995).
18. S. Havlin, "The Distance Between Zipf Plots," Physica A216, 148-150 (1995).
19. Ref. 2, pp. 40-44.
20. C. E. Shannon, "A Mathematical Theory of Communication," Bell System Tech. J. 27, 379-423 (1948).
21. L. Brillouin, Science and Information Theory (Academic Press, New York, 1956).
22. G. A. Miller and E. B. Newman, "Test of A Statistical Explanation of the Rank-Frequency Relation for Words in Written English," Am. J. Psychology 71, 209-218 (1958).
23. G. Herdan, The Advanced Theory of Language as Choice and Chance (Springer-Verlag. New York, Inc. 1966).