The impact of search engine selection and sorting criteria on vaccination beliefs and attitudes:Two experiments manipulating google output

Journal of Medical Internet Research

Volumn 16, Issue 4, 2014, Pages

  Allam, Ahmed ^a   Schulz, Peter Johannes ^a   Nakamoto, Kent ^b  

^a UNIVERSITY OF LUGANO   (Switzerland)

^b PAMPLIN COLLEGE OF BUSINESS   (United States)

Author keywords

Consumer health information; Health communication; Information storage and retrieval; Internet; Online systems; Public health informatics; Search engine; Searching behavior; Vaccination

Indexed keywords

ADULT; AGED; ARTICLE; ATTITUDE TO HEALTH; CONSUMER HEALTH INFORMATION; FACTORIAL ANALYSIS; FEMALE; HEALTH CARE QUALITY; HUMAN; HUMAN COMPUTER INTERACTION; INFORMATION RETRIEVAL; INTERNET; MALE; MEDICAL INFORMATICS; MEDICAL INFORMATION; MIDDLE AGED; ONLINE SYSTEM; SEARCH ENGINE; SEARCHING BEHAVIOR; STANDARD; STATISTICAL BIAS; VACCINATION; YOUNG ADULT;

CONSUMER HEALTH INFORMATION; HEALTH COMMUNICATION; INFORMATION STORAGE AND RETRIEVAL; INTERNET; ONLINE SYSTEMS; PUBLIC HEALTH INFORMATICS; SEARCH ENGINE; SEARCHING BEHAVIOR; VACCINATION;

ADULT; AGED; ATTITUDE TO HEALTH; BIAS (EPIDEMIOLOGY); COMPUTER LITERACY; CONSUMER HEALTH INFORMATION; FACTOR ANALYSIS, STATISTICAL; FEMALE; HUMANS; INFORMATION STORAGE AND RETRIEVAL; INTERNET; MALE; MIDDLE AGED; QUALITY INDICATORS, HEALTH CARE; SEARCH ENGINE; VACCINATION; YOUNG ADULT;

EID: 84901466254     PISSN: None     EISSN: 14388871     Source Type: Journal    
DOI: 10.2196/jmir.2642     Document Type: Article

References (68)

1. Internet World Stats. 2012. Internet Users in the World URL: http://www.internetworldstats.com/stats.htm [accessed 2013-03-10] [WebCite Cache ID 6F0YkwNot]
2. Fox S, Duggan M. Health Online 2013. Washington, DC: Pew Internet & American Life Project; 2013 Jan 15. URL: http:/ /pewinternet.org/Reports/2013/ Health-online.aspx [accessed 2013-07-23] [WebCite Cache ID 6IKHMUFmS]
3. Dutton WH, Blank G. Oxford Internet Survey 2011 Report. Oxford: Oxford Internet Institute, University of Oxford; 2011. Next Generation Users: The Internet in Britain URL: http://microsites.oii.ox.ac.uk/oxis/ [accessed 2013-07-23] [WebCite Cache ID 6IKGy0tu1]
4. Hesse BW, Nelson DE, Kreps GL, Croyle RT, Arora NK, Rimer BK, et al. Trust and sources of health information: the impact of the Internet and its implications for health care providers: findings from the first Health Information National Trends Survey. Arch Intern Med 2005;165(22):2618-2624. [doi: 10.1001/archinte.165.22.2618] [Medline: 16344419]
5. Hesse BW, Moser RP, Rutten LJ. Surveys of physicians and electronic health information. N Engl J Med 2010 Mar 4;362(9):859-860. [doi: 10.1056/NEJMc0909595] [Medline: 20200398]
6. Higgins O, Sixsmith J, Barry MM, Domegan C. ECDC Technical Report. Stockholm: ECDC; 2011. A literature review on health information-seeking behaviour on the web: a health consumer and health professional perspective URL: http:/ /ecdc.europa.eu/en/publications/Publications/ Literature%20review%20on%20health%20information- seeking%20behaviour%20on%20the%20web.pdf [accessed 2013-07-23] [WebCite Cache ID 6IKHshPyy]
7. Andreassen HK, Bujnowska-Fedak MM, Chronaki CE, Dumitru RC, Pudule I, Santana S, et al. European citizens' use of E-health services: a study of seven countries. BMC Public Health 2007;7:53 [FREE Full text] [doi: 10.1186/1471-2458-7-53] [Medline: 17425798]
8. Bader JL, Theofanos MF. Searching for cancer information on the internet: analyzing natural language search queries. J Med Internet Res 2003 Dec 11;5(4):e31 [FREE Full text] [doi: 10.2196/jmir.5.4.e31] [Medline: 14713659]
9. O'Keeffe J, Willinsky J, Maggio L. Public access and use of health research: an exploratory study of the National Institutes of Health (NIH) Public Access Policy using interviews and surveys of health personnel. J Med Internet Res 2011;13(4):e97 [FREE Full text] [doi: 10.2196/jmir.1827] [Medline: 22106169]
10. Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ 2002 Mar 9;324(7337):573-577 [FREE Full text] [Medline: 11884321]
11. Graham L, Tse T, Keselman A. Exploring user navigation during online health information seeking. AMIA Annu Symp Proc 2006:299-303 [FREE Full text] [Medline: 17238351]
12. Google Zeitgeist 2012. All 2012 search trends in the world URL: http://www.google.com/zeitgeist/2012/ [accessed 2013-03-10] [WebCite Cache ID 6F0Y2Lu0j]
13. Wang L, Wang J, Wang M, Li Y, Liang Y, Xu D. Using Internet search engines to obtain medical information: a comparative study. J Med Internet Res 2012;14(3):e74 [FREE Full text] [doi: 10.2196/jmir.1943] [Medline: 22672889]
14. Maurer H. Institute for Information Systems and Computer Media, Graz University of Technology. Austria: Graz University of Technology; 2007 Sep 30. Report on dangers and opportunities posed by large search engines, particularly Google URL: http://www.iicm.tugraz.at/iicmpapers/dangersgoogle.pdf [accessed 2013-03-14] [WebCite Cache ID 6F7JDMqVR]
15. Schulz PJ, Nakamoto K. "Bad" literacy, the Internet, and the limits of patient empowerment. In: AAAI Spring Symposium Series: Artificial Intelligence & Health Communication. 2011 Mar Presented at: AAAI Spring Symposium Series: Artificial Intelligence & Health Communication, 21st-23rd March; March 21-23, 2011; Palo Alto, CA URL: http://aaai.org/ocs/ index.php/SSS/SSS11/paper/view/2430 [WebCite Cache]
16. Murray E, Lo B, Pollack L, Donelan K, Catania J, Lee K, et al. The impact of health information on the Internet on health care and the physician-patient relationship: national U.S. survey among 1.050 U.S. physicians. J Med Internet Res 2003;5(3):e17 [FREE Full text] [doi: 10.2196/jmir.5.3.e17] [Medline: 14517108]
17. Stvilia B, Mon L, Yi YJ. A model for online consumer health information quality. J Am Soc Inf Sci 2009 Sep;60(9):1781-1791. [doi: 10.1002/asi.21115]
18. Gustafson DH, Wyatt JC. Evaluation of ehealth systems and services. BMJ 2004 May 15;328(7449):1150 [FREE Full text] [doi: 10.1136/bmj.328.7449.1150] [Medline: 15142895]
19. Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA 2001;285(20):2612-2621. [Medline: 11368735]
20. Eysenbach G, Diepgen TL. Towards quality management of medical information on the internet: evaluation, labelling, and filtering of information. BMJ 1998 Nov 28;317(7171):1496-1500 [FREE Full text] [Medline: 9831581]
21. Kunst H, Groot D, Latthe PM, Latthe M, Khan KS. Accuracy of information on apparently credible websites: survey of five common health topics. BMJ 2002 Mar 9;324(7337):581-582 [FREE Full text] [Medline: 11884323]
22. Roberts JM, Copeland KL. Clinical websites are currently dangerous to health. Int J Med Inform 2001 Jul;62(2-3):181-187. [Medline: 11470621]
23. Cline RJ, Haynes KM. Consumer health information seeking on the Internet: the state of the art. Health Educ Res 2001 Dec;16(6):671-692 [FREE Full text] [Medline: 11780707]
24. Eysenbach G, Powell J, Kuss O, Sa ER. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA 2002 May;287(20):2691-2700. [Medline: 12020305]
25. Bernstam EV, Walji MF, Sagaram S, Sagaram D, Johnson CW, Meric-Bernstam F. Commonly cited website quality criteria are not effective at identifying inaccurate online information about breast cancer. Cancer 2008 Mar 15;112(6):1206-1213 [FREE Full text] [doi: 10.1002/cncr.23308] [Medline: 18266210]
26. Hardey M. 'E-health': the internet and the transformation of patients into consumers and producers of health knowledge. Info., Comm. & Soc 2001 Oct;4(3):388-405. [doi: 10.1080/713768551]
27. Wilson P. How to find the good and avoid the bad or ugly: a short guide to tools for rating quality of health information on the internet. BMJ 2002 Mar 9;324(7337):598-602 [FREE Full text] [Medline: 11884329]
28. Eysenbach G. Consumer health informatics. BMJ 2000;320(7251):1713-1716. [doi: 10.1136/bmj.320.7251.1713]
29. Purcell GP, Wilson P, Delamothe T. The quality of health information on the internet. BMJ 2002;324(7337):557-558. [doi: 10.1136/bmj.324.7337.557]
30. Boyer C, Selby M, Scherrer JR, Appel RD. The Health On the Net Code of Conduct for medical and health Websites. Comput Biol Med 1998 Sep;28(5):603-610. [Medline: 9861515]
31. World Health Organization. Geneva: World Health Organization; 1999. Medical products and the Internet: a guide to finding reliable information URL: http://apps.who.int/medicinedocs/en/d/Js2277e/ [accessed 2014-03-29] [WebCite Cache ID 6OR1rn8Bo]
32. e-Health Ethics Initiative. e-Health Code of Ethics (May 24). J Med Internet Res 2000;2(2):E9 [FREE Full text] [doi: 10.2196/jmir.2.2.e9] [Medline: 11720928]
33. Zulman DM, Kirch M, Zheng K, An LC. Trust in the internet as a health resource among older adults: analysis of data from a nationally representative survey. J Med Internet Res 2011;13(1):e19 [FREE Full text] [doi: 10.2196/jmir.1552] [Medline: 21324832]
34. Fogg BJ, Marshall J, Laraki O, Osipovich A, Varma C, Fang N, et al. What makes Web sites credible?: A report on a large quantitative study. In: CHI '01 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2001 Presented at: SIGCHI Conference on Human Factors in Computing Systems; March 31-April 4, 2001; Seattle, WA p. 61-68. [doi: 10.1145/365024.365037]
35. Fogg BJ, Marshall J, Kameda T, Solomon J, Rangnekar A, Boyd J, et al. Web credibility research: a method for online experiments and early study results. In: CHI '01 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2001 Presented at: SIGCHI Conference on Human Factors in Computing Systems; March 31-April 4. 2001; Seattle, WA p. 295-296. [doi: 10.1145/634067.634242]
36. Kaicker J, Debono VB, Dang W, Buckley N, Thabane L. Assessment of the quality and variability of health information on chronic pain websites using the DISCERN instrument. BMC Med 2010;8:59 [FREE Full text] [doi: 10.1186/1741-7015-8-59] [Medline: 20939875]
37. Griffiths KM, Christensen H. Website quality indicators for consumers. J Med Internet Res 2005;7(5):e55 [FREE Full text] [doi: 10.2196/jmir.7.5.e55] [Medline: 16403719]
38. Ilic D, Bessell TL, Silagy CA, Green S. Specialized medical search-engines are no better than general search-engines in sourcing consumer information about androgen deficiency. Hum Reprod 2003 Mar;18(3):557-561 [FREE Full text] [Medline: 12615824]
39. Maloney S, Ilic D, Green S. Accessibility, nature and quality of health information on the Internet: a survey on osteoarthritis. Rheumatology (Oxford) 2005 Mar;44(3):382-385 [FREE Full text] [doi: 10.1093/rheumatology/keh498] [Medline: 15572390]
40. Ilic D, Risbridger G, Green S. Searching the Internet for information on prostate cancer screening: an assessment of quality. Urology 2004 Jul;64(1):112-116. [doi: 10.1016/j.urology.2004.02.027] [Medline: 15245946]
41. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999 Feb;53(2):105-111 [FREE Full text] [Medline: 10396471]
42. Khazaal Y, Chatton A, Cochand S, Coquard O, Fernandez S, Khan R, et al. Brief DISCERN, six questions for the evaluation of evidence-based content of health-related websites. Patient Educ Couns 2009 Oct;77(1):33-37. [doi: 10.1016/j.pec.2009.02.016] [Medline: 19372023]
43. Brin S, Page L. Anatomy of a large-scale hypertextual web search engine. In: WWW7 Proceedings from the 7th International World Wide Web Conference. 1998 Presented at: WWW7: The 7th International World Wide Web Conference; April 14-18, 1998; Brisbane, Australia URL: http://ilpubs.stanford.edu:8090/361/ [WebCite Cache]
44. Birru MS, Monaco VM, Charles L, Drew H, Njie V, Bierria T, et al. Internet usage by low-literacy adults seeking health information: an observational analysis. J Med Internet Res 2004 Sep 3;6(3):e25 [FREE Full text] [doi: 10.2196/jmir.6.3.e25] [Medline: 15471751]
45. Knapp C, Madden V, Wang H, Sloyer P, Shenkman E. Internet use and eHealth literacy of low-income parents whose children have special health care needs. J Med Internet Res 2011;13(3):e75 [FREE Full text] [doi: 10.2196/jmir.1697] [Medline: 21960017]
46. Lorence D, Hatton TL. Assessing domains of uncertainty in critical Web-based healthcare communities. J Med Syst 2006 Apr;30(2):117-122. [Medline: 16705996]
47. Keselman A, Browne AC, Kaufman DR. Consumer health information seeking as hypothesis testing. J Am Med Inform Assoc 2008;15(4):484-495 [FREE Full text] [doi: 10.1197/jamia.M2449] [Medline: 18436912]
48. Sillence E, Briggs P, Harris PR, Fishwick L. How do patients evaluate and make use of online health information? Soc Sci Med 2007 May;64(9):1853-1862. [doi: 10.1016/j.socscimed.2007.01.012] [Medline: 17328998]
49. Zeng QT, Kogan S, Plovnick RM, Crowell J, Lacroix EM, Greenes RA. Positive attitudes and failed queries: an exploration of the conundrums of consumer health information retrieval. Int J Med Inform 2004 Feb;73(1):45-55. [doi: 10.1016/j.ijmedinf.2003.12.015] [Medline: 15036078]
50. Ivanitskaya L, O'Boyle I, Casey AM. Health information literacy and competencies of information age students: results from the interactive online Research Readiness Self-Assessment (RRSA). J Med Internet Res 2006;8(2):e6 [FREE Full text] [doi: 10.2196/jmir.8.2.e6] [Medline: 16867969]
51. Ericsson KA, Simon HA. Protocol Analysis: Verbal Reports as Data. Cambridge, MA: MIT Press; 1993.
52. Ziebland S, Chapple A, Dumelow C, Evans J, Prinjha S, Rozmovits L. How the internet affects patients' experience of cancer: a qualitative study. BMJ 2004 Mar 6;328(7439):564 [FREE Full text] [doi: 10.1136/bmj.328.7439.564] [Medline: 15001506]
53. Hanik B, Stellefson M. Int Electronic J of Health Educ. 2011. EHealth literacy competencies among undergraduate health education students URL: http://www.aahperd.org/aahe/publications/iejhe/upload/11B-Hanik.pdf [accessed 2011-07-22] [WebCite Cache ID 60NJfktbm]
54. Centers for Disease Control and Prevention. 2004. Smallpox fact sheet URL: http://www.bt.cdc.gov/agent/smallpox/overview/disease-facts.asp [accessed 2013-03-10] [WebCite Cache ID 6F0ZjHPqx]
55. Fenner F, Henderson DA, Arita I, Jezekv Z, Ladnyi ID. Smallpox and its Eradication. Geneva: World Health Organization; 1988. URL: https://extranet.who. int/iris/restricted/bitstream/10665/39485/1/9241561106.pdf [accessed 2013-07-23] [WebCite Cache ID 6IKeK4RKN]
56. Stern AM, Markel H. The history of vaccines and immunization: familiar patterns, new challenges. Health Aff (Millwood) 2005;24(3):611-621 [FREE Full text] [doi: 10.1377/hlthaff.24.3.611] [Medline: 15886151]
57. Omer SB, Salmon DA, Orenstein WA, deHart MP, Halsey N. Vaccine refusal, mandatory immunization, and the risks of vaccine-preventable diseases. N Engl J Med 2009 May 7;360(19):1981-1988. [doi: 10.1056/NEJMsa0806477] [Medline: 19420367]
58. Crutzen R, Cyr D, de Vries NK. The role of user control in adherence to and knowledge gained from a website: randomized comparison between a tunneled version and a freedom-of-choice version. J Med Internet Res 2012;14(2):e45 [FREE Full text] [doi: 10.2196/jmir.1922] [Medline: 22532074]
59. Bennett GG, Glasgow RE. The delivery of public health interventions via the Internet: actualizing their potential. Annu Rev Public Health 2009;30:273-292. [doi: 10.1146/annurev.publhealth.031308.100235] [Medline: 19296777]
60. Powell J, Inglis N, Ronnie J, Large S. The characteristics and motivations of online health information seekers: cross-sectional survey and qualitative interview study. J Med Internet Res 2011;13(1):e20 [FREE Full text] [doi: 10.2196/jmir.1600] [Medline: 21345783]
61. Wantland DJ, Portillo CJ, Holzemer WL, Slaughter R, McGhee EM. The effectiveness of Web-based vs. non-Web-based interventions: a meta-analysis of behavioral change outcomes. J Med Internet Res 2004 Nov 10;6(4):e40 [FREE Full text] [doi: 10.2196/jmir.6.4.e40] [Medline: 15631964]
62. Hovland CI, Weiss W. The influence of source credibility on communication effectiveness. Public Opin Q 1951;4:635-650. [doi: 10.1086/266350]
63. Hovland CI, Janis IL, Kelly HH. Communication and Persuasion: Psychological Studies of Opinion Change. New Haven, CT: Yale University Press; 1953.
64. Nikolaus J. Credibility effects. In: Donsbach W, editor. Blackwell International Encyclopedia of Communication. Oxford: Wiley; 2008:1044-1047.
65. Google Developers. Google Custom Search API URL: https://developers. google.com/custom-search/ [accessed 2013-03-10] [WebCite Cache ID 6F0Z3bdiF]
66. Google Developers. Google custom search rendering URL: https://developers.google.com/custom-search/docs/js/rendering [accessed 2013-03-14] [WebCite Cache ID 6F7Mya9Sh]
67. Tinsley HE, Tinsley T. Uses of factor analysis in counseling psychology research. Journal of Counseling Psychology 1987;34(4):414-424. [doi: 10.1037/0022-0167.34.4.414]
68. Lorence DP, Greenberg L. The zeitgeist of online health search. Implications for a consumer-centric health system. J Gen Intern Med 2006 Feb;21(2):134-139 [FREE Full text] [doi: 10.1111/j.1525-1497.2005.0295.x] [Medline: 16336621]