Evaluating Google, Twitter, and Wikipedia as tools for influenza surveillance using Bayesian change point analysis: A comparative analysis

JMIR Public Health and Surveillance

Volumn 2, Issue 2, 2016, Pages

  Sharpe, J Danielle ^a,b   Hopkins, Richard S ^a   Cook, Robert L ^a   Striley, Catherine W ^a  

^a UNIVERSITY OF FLORIDA COLLEGE OF MEDICINE   (United States)

^b EMORY UNIVERSITY   (United States)

Author keywords

Bayes theorem; Human; Influenza; Internet; Public health surveillance; Social media

Indexed keywords

EID: 85026276223     PISSN: None     EISSN: 23692960     Source Type: Journal    
DOI: 10.2196/publichealth.5901     Document Type: Article

References (58)

1. Chunara R, Goldstein E, Patterson-Lomba O, Brownstein JS. Estimating influenza attack rates in the United States using a participatory cohort. Sci Rep 2015;5:9540 [FREE Full text] [doi: 10.1038/srep09540] [Medline: 25835538]
2. Centers for Disease Control and Prevention. Estimates of deaths associated with seasonal influenza - United States, 1976-2007. MMWR Morb Mortal Wkly Rep 2010;59(33):1057-1062. [Medline: 20798667]
3. Centers for Disease Control and Prevention. 2015. Overview of influenza surveillance in the United States URL:http://www. cdc.gov/flu/weekly/overview.htm [accessed 2016-03-26] [WebCite Cache ID 6gHxdKs79]
4. Fairchild G, De Silva L, Del Valle S, Segre A. Eliciting disease data from Wikipedia articles. 2015 Presented at: Ninth International AAAI Conference on Web and Social Media; 2015; Oxford, England p. 26-33.
5. Buehler JW, Hopkins RS, Overhage JM, Sosin DM, Tong V. Framework for evaluating public health surveillance systems for early detection of outbreaks: recommendations from the CDC Working Group. MMWR Recomm Rep 2004 May 7;53(RR-5):1-11 [FREE Full text] [Medline: 15129191]
6. Buehler JW, Sonricker A, Paladini M, Soper P, Mostashari F. Syndromic surveillance practice in the United States: findings from a survey of state, territorial, and selected local health departments. Adv Dis Surveill 2008;6(3):1-20.
7. Henning KJ. What is syndromic surveillance? MMWR Suppl 2004 Sep 24;53:5-11 [FREE Full text] [Medline: 15714620]
8. Bernardo TM, Rajic A, Young I, Robiadek K, Pham MT, Funk JA. Scoping review on search queries and social media for disease surveillance: a chronology of innovation. J Med Internet Res 2013;15(7):e147 [FREE Full text] [doi: 10.2196/jmir.2740] [Medline: 23896182]
9. Eysenbach G. Infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the Internet. J Med Internet Res 2009;11(1):e11 [FREE Full text] [doi: 10.2196/jmir.1157] [Medline: 19329408]
10. Salathé M, Bengtsson L, Bodnar TJ, Brewer DD, Brownstein JS, Buckee C, et al. Digital epidemiology. PLoS Comput Biol 2012;8(7):e1002616 [FREE Full text] [doi: 10.1371/journal.pcbi.1002616] [Medline: 22844241]
11. International Telecommunication Union. ICT facts and figures: the world in 2015 URL:http://www.itu.int/en/ITU-D/ Statistics/Documents/facts/ICTFactsFigures2015.pdf [accessed 2016-03-26] [WebCite Cache ID 6gHxY2tQX]
12. Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS, Brilliant L. Detecting influenza epidemics using search engine query data. Nature 2009 Feb 19;457(7232):1012-1014. [doi: 10.1038/nature07634] [Medline: 19020500]
13. Cook S, Conrad C, Fowlkes AL, Mohebbi MH. Assessing Google flu trends performance in the United States during the 2009 influenza virus A (H1N1) pandemic. PLoS One 2011;6(8):e23610 [FREE Full text] [doi: 10.1371/journal.pone.0023610] [Medline: 21886802]
14. Butler D. When Google got flu wrong. Nature 2013 Feb 14;494(7436):155-156. [doi: 10.1038/494155a] [Medline: 23407515]
15. Lazer D, Kennedy R, King G, Vespignani A. Big data. The parable of Google Flu: traps in big data analysis. Science 2014 Mar 14;343(6176):1203-1205. [doi: 10.1126/science.1248506] [Medline: 24626916]
16. Olson DR, Konty KJ, Paladini M, Viboud C, Simonsen L. Reassessing Google Flu Trends data for detection of seasonal and pandemic influenza: a comparative epidemiological study at three geographic scales. PLoS Comput Biol 2013;9(10):e1003256 [FREE Full text] [doi: 10.1371/journal.pcbi.1003256] [Medline: 24146603]
17. Google. Google Flu Trends URL:http://www.google.org/flutrends/about/ [accessed 2016-03-26] [WebCite Cache ID 6gHx8E3SD]
18. Broniatowski DA, Paul MJ, Dredze M. Twitter: big data opportunities. Science 2014 Jul 11;345(6193):148. [doi: 10.1126/science.345.6193.148-a] [Medline: 25013052]
19. Aslam AA, Tsou M, Spitzberg BH, An L, Gawron JM, Gupta DK, et al. The reliability of tweets as a supplementary method of seasonal influenza surveillance. J Med Internet Res 2014;16(11):e250 [FREE Full text] [doi: 10.2196/jmir.3532] [Medline: 25406040]
20. Broniatowski DA, Paul MJ, Dredze M. National and local influenza surveillance through Twitter: an analysis of the 2012-2013 influenza epidemic. PLoS One 2013;8(12):e83672 [FREE Full text] [doi: 10.1371/journal.pone.0083672] [Medline: 24349542]
21. Signorini A, Segre AM, Polgreen PM. The use of Twitter to track levels of disease activity and public concern in the U.S. during the influenza A H1N1 pandemic. PLoS One 2011;6(5):e19467 [FREE Full text] [doi: 10.1371/journal.pone.0019467] [Medline: 21573238]
22. Nagar R, Yuan Q, Freifeld CC, Santillana M, Nojima A, Chunara R, et al. A case study of the New York City 2012-2013 influenza season with daily geocoded Twitter data from temporal and spatiotemporal perspectives. J Med Internet Res 2014;16(10):e236 [FREE Full text] [doi: 10.2196/jmir.3416] [Medline: 25331122]
23. Aramaki E, Maskawa S, Morita M. Twitter catches the flu: detecting influenza epidemics using Twitter. 2011 Presented at: Conference on Empirical Natural Language Processing Conference (EMNLP); 2011; Edinburgh, Scotland p. 1568-1576.
24. Broniatowski DA, Dredze M, Paul MJ, Dugas A. Using Social Media to Perform Local Influenza Surveillance in an Inner-City Hospital: A Retrospective Observational Study. JMIR Public Health Surveill 2015;1(1):e5 [FREE Full text] [doi: 10.2196/publichealth.4472] [Medline: 27014744]
25. Generous N, Fairchild G, Deshpande A, Del Valle SY, Priedhorsky R. Global disease monitoring and forecasting with Wikipedia. PLoS Comput Biol 2014 Nov;10(11):e1003892 [FREE Full text] [doi: 10.1371/journal.pcbi.1003892] [Medline: 25392913]
26. Hickmann KS, Fairchild G, Priedhorsky R, Generous N, Hyman JM, Deshpande A, et al. Forecasting the 2013-2014 influenza season using Wikipedia. PLoS Comput Biol 2015 May;11(5):e1004239 [FREE Full text] [doi: 10.1371/journal.pcbi.1004239] [Medline: 25974758]
27. McIver DJ, Brownstein JS. Wikipedia usage estimates prevalence of influenza-like illness in the United States in near real-time. PLoS Comput Biol 2014 Apr;10(4):e1003581 [FREE Full text] [doi: 10.1371/journal.pcbi.1003581] [Medline: 24743682]
28. Laurent MR, Vickers TJ. Seeking health information online: does Wikipedia matter? J Am Med Inform Assoc 2009;16(4):471-479 [FREE Full text] [doi: 10.1197/jamia.M3059] [Medline: 19390105]
29. Tausczik Y, Faasse K, Pennebaker JW, Petrie KJ. Public anxiety and information seeking following the H1N1 outbreak: blogs, newspaper articles, and Wikipedia visits. Health Commun 2012;27(2):179-185. [doi: 10.1080/10410236.2011.571759] [Medline: 21827326]
30. Centers for Disease Control and Prevention. Influenza activity - United States, 2012-2013 season and composition of the 2013-14 influenza vaccine. MMWR Morb Mortal Wkly Rep 2013;62(23):473-479. [Medline: 23760189]
31. Epperson S, Blanton L, Kniss K, Mustaquim D, Steffens C, Wallis T, Influenza Division‚ National Center for Immunization and Respiratory Diseases‚ CDC. Influenza activity - United States, 2013-14 season and composition of the 2014-15 influenza vaccines. MMWR Morb Mortal Wkly Rep 2014 Jun 6;63(22):483-490 [FREE Full text] [Medline: 24898165]
32. Appiah GD, Blanton L, D'Mello T, Kniss K, Smith S, Mustaquim D, Centers for Disease Control and Prevention (CDC). Influenza activity - United States, 2014-15 season and composition of the 2015-16 influenza vaccine. MMWR Morb Mortal Wkly Rep 2015 Jun 5;64(21):583-590 [FREE Full text] [Medline: 26042650]
33. Centers for Disease Control and Prevention. MMWR weeks URL:https://wwwn.cdc.gov:443/nndss/document/ MMWR_Week_overview.pdf [accessed 2016-03-26] [WebCite Cache ID 6gHxPOqx0]
34. Centers for Disease Control and Prevention. FluView Interactive URL:http://gis.cdc.gov/grasp/fluview/fluportaldashboard. html [accessed 2016-03-26] [WebCite Cache ID 6gHxKR4m4]
35. Dredze M, Cheng R, Paul M, Broniatowski D. HealthTweets.org: a platform for public health surveillance using Twitter. 2014 Presented at: Twenty-Eighth AAAI Conference on Artificial Intelligence; 2014; Quebec City, Quebec, Canada.
36. Lamb A, Paul M, Dredze M. Separating fact from fear: tracking flu infections on Twitter. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL): Human Language Technologies. 2013 Presented at: Conference of the North American Chapter of the Association for Computational Linguistics (NAACL): Human Language Technologies; 2013; Atlanta, GA p. 789-795.
37. Wikimedia. Page view statistics for Wikimedia projects URL:http://dumps.wikimedia.org/other/pagecounts-raw/ [accessed 2016-03-26] [WebCite Cache ID 6gHwztskJ]
38. De Mauro A, Greco M, Grimaldi M. What is big data? A consensual definition and a review of key research topics. In: Proceedings of AIP Conference.: AIP Publishing LLC; 2015 Presented at: 4th International Conference on Integrated Information; September 5-8, 2014; Madrid, Spain p. 97-104. [doi: 10.1063/1.4907823]
39. Erdman C, Emerson JW. bcp: an R package for performing a Bayesian analysis of change point problems. J. Stat. Soft 2007;23(3). [doi: 10.18637/jss.v023.i03]
40. Erdman C, Emerson JW. A fast Bayesian change point analysis for the segmentation of microarray data. Bioinformatics 2008 Oct 1;24(19):2143-2148 [FREE Full text] [doi: 10.1093/bioinformatics/btn404] [Medline: 18667443]
41. Schluter PJ, Hamilton GJ, Deely JM, Ardagh MW. Impact of integrated health system changes, accelerated due to an earthquake, on emergency department attendances and acute admissions: a Bayesian change-point analysis. BMJ Open 2016;6(5):e010709 [FREE Full text] [doi: 10.1136/bmjopen-2015-010709] [Medline: 27169741]
42. Ellis BK, Stanford JA, Goodman D, Stafford CP, Gustafson DL, Beauchamp DA, et al. Long-term effects of a trophic cascade in a large lake ecosystem. Proc Natl Acad Sci U S A 2011 Jan 18;108(3):1070-1075 [FREE Full text] [doi: 10.1073/pnas.1013006108] [Medline: 21199944]
43. Kass-Hout TA, Xu Z, McMurray P, Park S, Buckeridge DL, Brownstein JS, et al. Application of change point analysis to daily influenza-like illness emergency department visits. J Am Med Inform Assoc 2012;19(6):1075-1081 [FREE Full text] [doi: 10.1136/amiajnl-2011-000793] [Medline: 22759619]
44. Wang X, Emerson J. arxiv. 2015. Bayesian change point analysis of linear models on general graphs URL:https://arxiv. org/pdf/1509.00817 [accessed 2016-10-17] [WebCite Cache ID 6lJxaahQr]
45. Rstudio. Boston, MA: RStudio, Inc; 2015. RStudio: integrated development for R URL:https://www.rstudio.com/ [accessed 2016-03-26] [WebCite Cache ID 6gHzyHxHu]
46. Barry D, Hartigan J. A Bayesian analysis for change point problems. JASA 1993;88(421):309-319.
47. German RR, Lee LM, Horan JM, Milstein RL, Pertowski CA, Waller MN, Guidelines Working Group Centers for Disease Control and Prevention (CDC). Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group. MMWR Recomm Rep 2001 Jul 27;50(RR-13):1-35; quiz CE1-7. [Medline: 18634202]
48. Paul MJ, Dredze M, Broniatowski D. Twitter improves influenza forecasting. PLoS Curr 2014;6 [FREE Full text] [doi: 10.1371/currents.outbreaks.90b9ed0f59bae4ccaa683a39865d9117] [Medline: 25642377]
49. Kass-Hout T. Google. 2013. Change point analysis: Introduction URL:https://sites.google.com/site/changepointanalysis/ [accessed 2016-03-26] [WebCite Cache ID 6gHwpzPHi]
50. Pervaiz F, Pervaiz M, Abdur RN, Saif U. FluBreaks: early epidemic detection from Google flu trends. J Med Internet Res 2012;14(5):e125 [FREE Full text] [doi: 10.2196/jmir.2102] [Medline: 23037553]
51. Wu-Chen S. Integrating and mining virtual communities across multiple online social networks: concepts, approaches and challenges. In: Proceedings of the Fourth International Conference on Digital Information and Communication Technology and its Applications. 2014 Presented at: The Fourth International Conference on Digital Information and Communication Technology and its Applications; 2014; Bangkok, Thailand p. 199-204. [doi: 10.1109/DICTAP.2014.6821682]
52. Pew Research Center. Internet user demographics URL:http://www.pewinternet.org/data-trend/internet-use/latest-stats/ [accessed 2016-03-26] [WebCite Cache ID 6gHwYYhqI]
53. Cavazos-Rehg PA, Krauss MJ, Sowles S, Connolly S, Rosas C, Bharadwaj M, et al. A content analysis of depression-related Tweets. Comput Human Behav 2016 Jan 1;54:351-357. [doi: 10.1016/j.chb.2015.08.023] [Medline: 26392678]
54. Reavley NJ, Pilkington PD. Use of Twitter to monitor attitudes toward depression and schizophrenia: an exploratory study. PeerJ 2014;2:e647 [FREE Full text] [doi: 10.7717/peerj.647] [Medline: 25374786]
55. Chen AT, Zhu S, Conway M. What Online Communities Can Tell Us About Electronic Cigarettes and Hookah Use: A Study Using Text Mining and Visualization Techniques. J Med Internet Res 2015;17(9):e220 [FREE Full text] [doi: 10.2196/jmir.4517] [Medline: 26420469]
56. Krauss MJ, Sowles SJ, Moreno M, Zewdie K, Grucza RA, Bierut LJ, et al. Hookah-Related Twitter Chatter: A Content Analysis. Prev Chronic Dis 2015;12:E121 [FREE Full text] [doi: 10.5888/pcd12.150140] [Medline: 26226068]
57. Krueger EA, Young SD. Twitter: A Novel Tool for Studying the Health and Social Needs of Transgender Communities. JMIR Ment Health 2015;2(2):e16 [FREE Full text] [doi: 10.2196/mental.4113] [Medline: 26082941]
58. Santillana M, Nguyen AT, Dredze M, Paul MJ, Nsoesie EO, Brownstein JS. Combining Search, Social Media, and Traditional Data Sources to Improve Influenza Surveillance. PLoS Comput Biol 2015 Oct;11(10):e1004513 [FREE Full text] [doi: 10.1371/journal.pcbi.1004513] [Medline: 26513245]