Garbage in, garbage out: Data collection, quality assessment and reporting standards for social media data use in health research, infodemiology and digital disease detection

Journal of Medical Internet Research

Volumn 18, Issue 2, 2016, Pages

  Kim, Yoonsang ^a   Huang, Jidong ^a   Emery, Sherry ^a  

^a UNIVERSITY OF ILLINOIS AT CHICAGO   (United States)

Author keywords

Digital disease detection; Infodemiology; Infoveillance; Precision and recall; Search filter; Sensitivity and specificity; Social media; Standard reporting; Twitter

Indexed keywords

EPIDEMIC; HEALTH CARE QUALITY; HUMAN; INFORMATION PROCESSING; INTERNET; PROCEDURES; SEARCH ENGINE; SOCIAL MEDIA; TRENDS; UTILIZATION;

DATA COLLECTION; DISEASE OUTBREAKS; HUMANS; INTERNET; QUALITY ASSURANCE, HEALTH CARE; SEARCH ENGINE; SOCIAL MEDIA;

EID: 84962081670     PISSN: None     EISSN: 14388871     Source Type: Journal    
DOI: 10.2196/jmir.4738     Document Type: Article

References (44)

1. Fox S, Duggan M. Internet & American Life Project.: Pew Research Center; 2013 Jan. Health online 2013 URL: http://www.pewinternet.org/2013/01/15/health-online-2013/ [accessed 2015-01-10] [WebCite Cache ID 6YeJ27UsV]
2. Koch-Weser S, Bradshaw YS, Gualtieri L, Gallagher SS. The Internet as a health information source: findings from the 2007 Health Information National Trends Survey and implications for health communication. J Health Commun 2010;15 Suppl 3:279-293. [doi: 10.1080/10810730.2010.522700] [Medline: 21154099]
3. McCully SN, Don BP, Updegraff JA. Using the Internet to help with diet, weight, and physical activity: results from the Health Information National Trends Survey (HINTS). J Med Internet Res 2013;15(8):e148 [FREE Full text] [doi: 10.2196/jmir.2612] [Medline: 23906945]
4. Anderson M, Caumont A. Fact Tank.: Pew Research Center; 2014 Sep 24. How social media is reshaping news URL: http://www.pewresearch.org/fact-tank/2014/09/24/how-social-media-is-reshaping-news/ [accessed 2015-05-18] [WebCite Cache ID 6YeJY7Ho2]
5. Somaiya R. The New York Times. 2014 Oct 26. How Facebook is changing the way its users consume journalism URL: http://www.nytimes.com/2014/10/27/business/media/how-facebook-is-changing-the-way-its-users-consume-journalism.html [accessed 2015-05-19] [WebCite Cache ID 6YeJtn0H2]
6. Eysenbach G. Infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the Internet. J Med Internet Res 2009;11(1):e11 [FREE Full text] [doi: 10.2196/jmir.1157] [Medline: 19329408]
7. Chew C, Eysenbach G. Pandemics in the age of Twitter: content analysis of Tweets during the 2009 H1N1 outbreak. PLoS One 2010;5(11):e14118 [FREE Full text] [doi: 10.1371/journal.pone.0014118] [Medline: 21124761]
8. Signorini A, Segre AM, Polgreen PM. The use of Twitter to track levels of disease activity and public concern in the U.S. during the influenza A H1N1 pandemic. PLoS One 2011;6(5):e19467 [FREE Full text] [doi: 10.1371/journal.pone.0019467] [Medline: 21573238]
9. Myslín M, Zhu S, Chapman W, Conway M. Using twitter to examine smoking behavior and perceptions of emerging tobacco products. J Med Internet Res 2013;15(8):e174 [FREE Full text] [doi: 10.2196/jmir.2534] [Medline: 23989137]
10. Heaivilin N, Gerbert B, Page JE, Gibbs JL. Public health surveillance of dental pain via Twitter. J Dent Res 2011 Sep;90(9):1047-1051 [FREE Full text] [doi: 10.1177/0022034511415273] [Medline: 21768306]
11. Kim AE, Hopper T, Simpson S, Nonnemaker J, Lieberman AJ, Hansen H, et al. Using Twitter Data to Gain Insights into E-cigarette Marketing and Locations of Use: An Infoveillance Study. J Med Internet Res 2015;17(11):e251 [FREE Full text] [doi: 10.2196/jmir.4466] [Medline: 26545927]
12. Sanders-Jackson A, Brown CG, Prochaska JJ. Applying linguistic methods to understanding smoking-related conversations on Twitter. Tob Control 2015 Mar;24(2):136-138 [FREE Full text] [doi: 10.1136/tobaccocontrol-2013-051243] [Medline: 24227540]
13. Zhang N, Campo S, Janz KF, Eckler P, Yang J, Snetselaar LG, et al. Electronic word of mouth on twitter about physical activity in the United States: exploratory infodemiology study. J Med Internet Res 2013;15(11):e261 [FREE Full text] [doi: 10.2196/jmir.2870] [Medline: 24257325]
14. Collier N, Son NT, Nguyen NM. OMG U got flu? Analysis of shared health messages for bio-surveillance. J Biomed Semantics 2011;2 Suppl 5:S9 [FREE Full text] [doi: 10.1186/2041-1480-2-S5-S9] [Medline: 22166368]
15. Cole-Lewis H, Pugatch J, Sanders A, Varghese A, Posada S, Yun C, et al. Social Listening: A Content Analysis of E-Cigarette Discussions on Twitter. J Med Internet Res 2015;17(10):e243 [FREE Full text] [doi: 10.2196/jmir.4969] [Medline: 26508089]
16. Chan B, Lopez A, Sarkar U. The Canary in the Coal Mine Tweets: Social Media Reveals Public Perceptions of Non-Medical Use of Opioids. PLoS One 2015;10(8):e0135072 [FREE Full text] [doi: 10.1371/journal.pone.0135072] [Medline: 26252774]
17. Eichstaedt JC, Schwartz HA, Kern ML, Park G, Labarthe DR, Merchant RM, et al. Psychological language on Twitter predicts county-level heart disease mortality. Psychol Sci 2015 Feb;26(2):159-169 [FREE Full text] [doi: 10.1177/0956797614557867] [Medline: 25605707]
18. Gittelman S, Lange V, Gotway Crawford CA, Okoro CA, Lieb E, Dhingra SS, et al. A new source of data for public health surveillance: Facebook likes. J Med Internet Res 2015;17(4):e98 [FREE Full text] [doi: 10.2196/jmir.3970] [Medline: 25895907]
19. Kietzmann JH, Hermkens K, McCarthy IP, Silvestre BS. Social media? Get serious! Understanding the functional building blocks of social media. Business Horizons 2011 May;54(3):241-251. [doi: 10.1016/j.bushor.2011.01.005]
20. Petrova M, Sutcliffe P, Fulford KWM, Dale J. Search terms and a validated brief search filter to retrieve publications on health-related values in Medline: a word frequency analysis study. J Am Med Inform Assoc 2012;19(3):479-488 [FREE Full text] [doi: 10.1136/amiajnl-2011-000243] [Medline: 21846778]
21. Mollema L, Harmsen IA, Broekhuizen E, Clijnk R, De MH, Paulussen T, et al. Disease detection or public opinion reflection? Content analysis of tweets, other social media, and online newspapers during the measles outbreak in The Netherlands in 2013. J Med Internet Res 2015;17(5):e128 [FREE Full text] [doi: 10.2196/jmir.3863] [Medline: 26013683]
22. Adrover C, Bodnar T, Huang Z, Telenti A, Salathé M. Identifying Adverse Effects of HIV Drug Treatment and Associated Sentiments Using Twitter. JMIR Public Health Surveill 2015 Jul 27;1(2):e7 [FREE Full text] [doi: 10.2196/publichealth.4488]
23. Huang J, Kornfield R, Szczypka G, Emery SL. A cross-sectional examination of marketing of electronic cigarettes on Twitter. Tob Control 2014 Jul;23 Suppl 3:iii26-iii30 [FREE Full text] [doi: 10.1136/tobaccocontrol-2014-051551] [Medline: 24935894]
24. Ghosh DD, Guha R. What are we 'tweeting' about obesity? Mapping tweets with Topic Modeling and Geographic Information System. Cartogr Geogr Inf Sci 2013;40(2):90-102 [FREE Full text] [doi: 10.1080/15230406.2013.776210] [Medline: 25126022]
25. Stryker JE, Wray RJ, Hornik RC, Yanovitzky I. Validation of Database Search Terms for Content Analysis: The Case of Cancer News Coverage. Journalism & Mass Communication Quarterly 2006 Jun 01;83(2):413-430. [doi: 10.1177/107769900608300212]
26. White VJ, Glanville JM, Lefebvre C, Sheldon TA. A statistical approach to designing search filters to find systematic reviews: objectivity enhances accuracy. Journal of Information Science 2001 Dec 01;27(6):357-370. [doi: 10.1177/016555150102700601]
27. Staquet M, Rozencweig M, Lee YJ, Muggia FM. Methodology for the assessment of new dichotomous diagnostic tests. J Chronic Dis 1981;34(12):599-610. [Medline: 6458624]
28. Shuai W, Zhiyuan C, Bing L, Sherry E. Identifying Keywords for Searching Relevant Social Media Posts. AAAI-16; 2016 Feb 12 Presented at: 30th AAAI Conference on Artificial Intelligence; Feb. 2016; Phoenix, Arizona.
29. Urban Dictionary. URL: http://www.urbandictionary.com/ [accessed 2015-05-19] [WebCite Cache ID 6YhNIyRcN]
30. Gordis L. Epidemiology. Philadelphia: Elsevier Saunders; 2004.
31. Liu B. Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data (Data-Centric Systems and Applications). Heidelberg: Springer; 2011.
32. Gelman A, Carlin J, Stern H, Rubin D. Bayesian Data Analysis, Second Edition. Boca Raton, FL: Chapman & Hall; 2003.
33. Agresti A. Categorical data analysis. New York: Wiley-Interscience; 2002.
34. Joseph L, Gyorkos TW, Coupal L. Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard. Am J Epidemiol 1995 Feb 1;141(3):263-272. [Medline: 7840100]
35. Twitter Search. URL: https://twitter.com/search-home [accessed 2016-02-03] [WebCite Cache ID 6f24ZCZqD]
36. Smith B. Bayesian output analysis program (BOA) Version 1.1 Users Manual. Iowa City, IA: University of Iowa; 2005 Jan 08. URL: http://www.public-health.uiowa.edu/boa/boa.pdf [accessed 2016-02-03] [WebCite Cache ID 6f24iO3OH]
37. Fung IC, Tse ZTH, Cheung C, Miu AS, Fu K. Ebola and the social media. Lancet 2014 Dec 20;384(9961):2207. [doi: 10.1016/S0140-6736(14)62418-1] [Medline: 25625391]
38. Lazer D, Kennedy R, King G, Vespignani A. Big data. The parable of Google Flu: traps in big data analysis. Science 2014 Mar 14;343(6176):1203-1205. [doi: 10.1126/science.1248506] [Medline: 24626916]
39. Huang J, Kim Y, Vera L, Emery SL. Electronic Cigarettes Among Priority Populations: Role of Smoking Cessation and Tobacco Control Policies. Am J Prev Med 2016 Feb;50(2):199-209. [doi: 10.1016/j.amepre.2015.06.032] [Medline: 26410185]
40. Arrazola R, Neff L, Kennedy S, Holder-Hayes E, Jones C. Tobacco use among middle and high school students - United States, 2011-2014. Morbidity and Mortality Weekly Report 2014 Nov 14;63(45):1021-1026 [FREE Full text] [WebCite Cache]
41. Mitchell T. Machine learning. New York: McGraw-Hill; 1997.
42. Hastie T, Tibshirani R, Friedman J. The elements of statistical learning: data mining, inference and prediction. New York, NY: Springer; 2009.
43. Capurro D, Cole K, Echavarría MI, Joe J, Neogi T, Turner AM. The use of social networking sites for public health practice and research: a systematic review. J Med Internet Res 2014;16(3):e79 [FREE Full text] [doi: 10.2196/jmir.2679] [Medline: 24642014]
44. Chu K, Sidhu A, Valente T. Electronic Cigarette Marketing Online: a Multi-Site, Multi-Product Comparison. JMIR Public Health Surveill 2015 Sep 11;1(2):e11 [FREE Full text] [doi: 10.2196/publichealth.4777]