Characterizing the Google Books corpus: Strong limits to inferences of socio-cultural and linguistic evolution

PLoS ONE

Volumn 10, Issue 10, 2015, Pages

  Pechenick, Eitan Adam ^a,b   Danforth, Christopher M ^a,b   Dodds, Peter Sheridan ^a,b  

^a University of Vermont ^*  (United States)

^b UNIVERSITY OF VERMONT   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONTROLLED STUDY; HUMAN; HUMAN EXPERIMENT; THEORETICAL MODEL; BOOK; LANGUAGE; LIBRARY; LINGUISTICS; READING; TRENDS;

BOOKS; HUMANS; LANGUAGE; LIBRARIES; LINGUISTICS; READING;

EID: 84948657708     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0137041     Document Type: Article

References (17)

1. Michel JB, Shen YK, Aiden AP, Veres A, Gray MK, et al. (2011) Quantitative analysis of culture using millions of digitized books. Science 331:176-182. doi: 10.1126/science.1199644 PMID: 21163965
2. Lin Y, Michel JB, Aiden EL, Orwant J, Brockman W, et al. (2012) Syntactic annotations for the google books ngram corpus. In: Proceedings of the ACL 2012 System Demonstrations. Association for Computational Linguistics, pp. 169-174.
3. Salganik MJ, Dodds PS, Watts DJ (2006) An experimental study of inequality and unpredictability in an artificial cultural market. Science 311:854-856. doi: 10.1126/science.1121066 PMID: 16469928
4. Google Ngram Viewer: 'Frodo', 1800-2000 in English Fiction. https://books.google.com/ngrams/graph?content=Frodo&year-start=1800&year-end=2000&corpus=16&smoothing=1; Accessed April 25, 2015.
5. Twenge JM, Campbell WK, Gentile B (2012) Increases in individualistic words and phrases in american books, 1960-2008. PloS one 7. doi: 10.1371/journal.pone.0040181 PMID: 22808113
6. Twenge JM, Campbell WK, Gentile B (2012) Male and female pronoun use in us books reflects women status, 1900-2008. Sex roles 67:488-493. doi: 10.1007/s11199-012-0194-7
7. Greenfield PM (2013) The changing psychology of culture from 1800 through 2000. Psychological science 24:1722-1731. doi: 10.1177/0956797613479387 PMID: 23925305
8. Petersen AM, Tenenbaum J, Havlin S, Stanley HE (2012) Statistical laws governing fluctuations in word use from word birth to word death. Scientific reports 2. doi: 10.1038/srep00313
9. Gerlach M, Altmann EG (2013) Stochastic model for the vocabulary growth in natural languages. Physical Review X 3:021006. doi: 10.1103/PhysRevX.3.021006
10. Petersen AM, Tenenbaum JN, Havlin S, Stanley HE, Perc M (2012) Languages cool as they expand: Allometric scaling and the decreasing need for new words. Scientific reports 2. doi: 10.1038/srep00943
11. Bentley RA, Acerbi A, Ormerod P, Lampos V (2014) Books average previous decade of economic misery. PloS ONE 9:e83147. doi: 10.1371/journal.pone.0083147 PMID: 24416159
12. Koplenig A (2015) The impact of lacking metadata for the measurement of cultural and linguistic change using the Google Ngram datasets reconstructing the composition of the German corpus in times of WWII. Digital Scholarship in the Humanities.
13. Kullback S, Leibler RA (1951) On information and sufficiency. The Annals of Mathematical Statistics: 79-86.
14. Shannon CE (2001) A mathematical theory of communication. ACM SIGMOBILE Mobile Computing and Communications Review 5:3-55. doi: 10.1145/584091.584093
15. Lin J (1991) Divergence measures based on the shannon entropy. Information Theory, IEEE Transactions on 37:145-151. doi: 10.1109/18.61115
16. Dodds PS, Harris KD, Kloumann IM, Bliss CA, Danforth CM (2011) Temporal patterns of happiness and information in a global social network: Hedonometrics and Twitter. PLoS ONE 6:e26752. doi: 10. 1371/journal.pone.0026752 PMID: 22163266
17. de Solla Price DJ (1963) Little Science, Big Science. New York: Columbia University Press.