Web evaluation at the US National Institutes of Health: Use of the American customer satisfaction index online customer survey

Journal of Medical Internet Research

Volumn 10, Issue 1, 2008, Pages

  Wood, Fred B ^a,c   Siegel, Elliot R ^a   Feldman, Sue ^b   Love, Cynthia B ^a   Rodrigues, Dennis ^c   Malamud, Mark ^d   Lagana, Marie ^e   Crafts, Jennifer ^f  

^a NATIONAL INSTITUTES OF HEALTH   (United States)

^b NATIONAL CANCER INSTITUTE   (United States)

^c NATIONAL INSTITUTES OF HEALTH   (United States)

^d NATIONAL HEART LUNG AND BLOOD INSTITUTE   (United States)

^e NATIONAL INSTITUTES OF HEALTH   (United States)

^f WESTAT   (United States)

Author keywords

Consumer health information; Evaluation studies; Internet; Satisfaction; Surveys; World Wide Web

Indexed keywords

ARTICLE; E-MAIL; HEALTH CARE ORGANIZATION; INTERNET; MEDICAL INFORMATION; MEDICAL RESEARCH; SATISFACTION; UNITED STATES; CONSUMER; EVALUATION; HUMAN; INFORMATION PROCESSING; METHODOLOGY; STANDARD;

CONSUMER SATISFACTION; DATA COLLECTION; HUMANS; INTERNET; NATIONAL INSTITUTES OF HEALTH (U.S.); UNITED STATES;

EID: 40949125727     PISSN: None     EISSN: 14388871     Source Type: Journal    
DOI: 10.2196/jmir.944     Document Type: Article

References (57)

1. Lacroix E, Mehnert R. The US National Library of Medicine in the 21st century: expanding collections, nontraditional formats, new audiences. Health Info Libr J 2002 Sep;19(3):126-132.
2. Hesse BW, Nelson DE, Kreps GL, Croyle RT, Arora NK, Rimer BK, et al. Trust and sources of health information: the impact of the Internet and its implications for health care providers: findings from the first Health Information National Trends Survey. Arch Intern Med 2005 Dec 12;165(22):2618-2624
3. Bright MA, Fleisher L, Thomsen C, Morra ME, Marcus Al, Gehring W. Exploring e-Health usage and interest among cancer information service users: the need for personalized interactions and multiple channels remains. J Health Commun 2005;10 Suppl 1(7):35-52.
4. Grama LM, Beckwith M, Bittinger W, Blais D, Lollar C, Middleswarth A, et al. The role of user input in shaping online information from the National Cancer Institute. J Med Internet Res 2005 Jul 1;7(3):e25
5. McCray AT, Dorfman E, Ripple A, Ide NC, Jha M, Katz DG, et al. Usability issues in developing a Web-based consumer health site. Proc AMIA Symp 2000:556-560.
6. Wood FB, Siegel ER, LaCroix EM, Lyon BL, Benson DA, Cid V, et al. A practical approach to e-government Web evaluation. IT Professional 2003 May;5(3).
7. Sterne J. Web Metrics: Proven Methods for Measuring Website Success. New York: Wiley Publishing Inc; 2002.
8. Bader JL, Theofanos MF. Searching for cancer information on the internet: analyzing natural language search queries. J Med Internet Res 2003 Dec 11;5(4):e31
9. Redalen A, Miller N. Evaluating website modifications at the National Library of Medicine through search log analysis. D-Lib Magazine 2000 Jan;6(1).
10. Miller N. Analysis of user messages to MedlinePlus.gov. J Med Libr Assoc 2007 Jan;95(1):81-83
11. Wood FB, Benson D, LaCroix E, Siegel ER, Fariss S. Use of Internet audience measurement data to gauge market share for online health information services. J Med Internet Res 2005 Jul 1;7(3):e31
12. National Library of Medicine. MedlinePlus Quality Guidelines. 2006. NLM/NIH. URL: http://www.nlm.nih.gov/medlineplus/criteria.html [accessed 2007 Nov 7]
13. Miller N, Lacroix EM, Backus JE. MEDLINEplus: building and maintaining the National Library of Medicine's consumer health Web service. Bull Med Libr Assoc 2000 Jan;88(1):11-17
14. Wilson FL, Baker LM, Brown-Syed C, Gollop C. An analysis of the readability and cultural sensitivity of information on the National Cancer Institute's Web site: CancerNet. Oncol Nurs Forum 2000 Oct;27(9):1403-1409.
15. International Council on Active Aging. NIHSeniorHealth.gov: empowering older adults with health information. J Act Aging 2005 Mar;4(2):62-64
16. Boyer C, Geissbuhler A. A decade devoted to improving online health information quality. Stud Health Technol Inform 2005;116:891-896.
17. Office of Management and Budget. Government Performance Results Act of 1993. whitehouse.gov. URL: http://www.whitehouse.gov/omb/mgmt-gpra/gplaw2m.html [accessed 2007 Nov 7]
18. Office of Management and Budget. Circular No. A-130 Revised. Management of Federal Information Resources. whitehouse.gov. URL: http://www.whitehouse. gov/omb/circulars/a130/a130trans4.html [accessed 2007 Nov 7]
19. Office of Management and Budget. Part IX: Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies; Notice; Republication. Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001: Public Law 106-554. Federal Register. 2002 Feb 22. (36) p. 8451-8460. URL: http://www.whitehouse.gov/omb/fedreg/reproducible2.pdf [accessed 2007 Nov 7]
20. Web Managers Advisory Council, Interagency Committee on Government Information (2004). Recommended Policies and Guidelines for Federal Public Websites. Final Report of the Interagency Committee on Government Information. Submitted to The Office of Management and Budget June 9, 2004. usa.gov. URL: http://www.usa.gov/webcontent/about/documents/icgi_report.html [accessed 2007 Nov 19]
21. Memorandum for the Heads of Executive Departments and Agencies. December 17, 2004. Policies for Federal Agency Public Websites. Washington, DC: Office of Management and Budget, Executive Office of the President; 2004. URL: http://www.whitehouse.gov/omb/memoranda/fy2005/m05-04.pdf [accessed 2007 Nov 19]
22. National Library of Medicine's MedlinePlus Survey Results 2001. NLM/NIH. URL: http://www.nlm.nih.gov/medlineplus/survey/index.html [accessed 2007 Nov 19]
23. MedlinePlus Survey Results 2003. NLM/NIH. URL: http://www.nlm.nih.gov/ medlineplus/survey2003/index.html [accessed 2007 Nov 19]
24. Fornell CG, Johnson MD, Anderson EW, Cha J, Bryant BE. The American Customer Satisfaction Index: nature, purpose and findings. J Mark 1996;60(4):7-18.
25. Stowers G. Measuring the Performance of E-Government. Washington DC: IBM Center for the Business of Government; 2004. URL: http://www. businessofgovernment.org/pdfs/8493_Stowers_Report.pdf [accessed 2007 Nov 19]
26. Freed L. American Customer Satisfaction Index E-Government Satisfaction Index. Ann Arbor, MI: ForeSee Results; 2003. URL: http://www.fcg.gov/pdf/e- gov_comm_092003.pdf [accessed 2007 Nov 19]
27. Yao ML. The President's Management Council: An Important Management Innovation. Arlington, VA: PricewaterhouseCoopers Endowment for The Business of Government; 2000. URL: http://www.businessofgovernment.org/pdfs/yao.pdf [accessed 2007 Nov 19]
28. US General Services Administration. Vice President Launches First Governmentwide Customer Satisfaction Survey. 1999 May 26. gsa.gov. URL: http://www.gsa.gov/Portal/gsa/ep/contentView.do?contentType=GSA_BASIC& contentId=9177&noc=T [accessed 2008 Feb 6]
29. Fornell C. ACSI Commentary: Federal Government Scores. 2004 Dec 14. theacsi.org. URL: http://www.theacsi.org/index.php?option=com_content&task= view&id=152 [accessed 2007 Nov 19]
30. Fornell C. ACSI Commentary: Federal Government Scores. 2005 Dec 15. theacsi.org. URL: http://www.theacsi.org/index.php?option=com_content&task= view&id=151&Itemid=62 [accessed 2007 Nov 19]
31. Fornell C. ACSI Commentary: Federal Government Scores. 2006 Dec 15. theacsi.org. URL: http://www.theacsi.org/index.php?option=com_content&task= view&id=162&Itemid=62 [accessed 2007 Nov 19]
32. Office of Portfolio Analysis and Strategic Initiatives. Evaluation Branch. OPASI/NIH. URL: http://opasi.nih.gov/desa/eb/[accessed 2008 Feb 3]
33. Office of Portfolio Analysis and Strategic Initiatives. Evaluation Branch. NIH Evaluation Set-Aside Program. OPASI/NIH. URL: http://opasi.nih.gov/ desa/eb/setaside.asp [accessed 2008 Feb 3]
34. ForeSee Results home page. ForeSee. URL: http://www.foreseeresults.com/ [accessed 2007 Nov 19]
35. US Department of the Treasury. Federal Consulting Group home page. fcg.gov. URL: http://www.fcg.gov/ [accessed 2007 Nov 19]
36. Fornell C, Van Amburg D, Morgeson F, et al. The American Customer Satisfaction Index at Ten Years, ACSI 1994-2004. A Summary of Findings: Implications for the Economy, Stock Returns and Management. Ann Arbor, MI: University of Michigan, Stephen M. Ross School of Business, National Quality Research Center; 2005.
37. American Customer Satisfaction Index Methodology Report. Ann Arbor, MI: University of Michigan, Stephen M. Ross School of Business, National Quality Research Center; 2005.
38. National Institutes of Health, National Library of Medicine. Programs and Services FY 2005. Bethesda, MD: US Department of Health & Human Services; 2005. URL: http://www.nlm.nih.gov/ocpl/anreports/fy2005.pdf [accessed 2007 Nov 19]
39. National Cancer Institute. NCI Web site rated best e-government main site. NCI Cancer Bull. 2005 Jan 4. (1) URL: http://www.cancer.gov/ ncicancerbulletin/NCI_Cancer_Bulletin_10405/page9 [accessed 2007 Nov 19]
40. National Institute of Diabetes and Digestive and Kidney Diseases (2005). NIDDK health information Web sites rate high in customer satisfaction. In: NIDDK Director's Update (June 15, 2005). NIDDK/NIH. URL: http://www2.niddk.nih.gov/ AboutNIDDK/Director/Update/Archives/20050615.htm [accessed 2007 Nov 19]
41. National Library of Medicine. FY 2006 Budget. NLM/NIH. URL: http://www.nlm.nih.gov/about/2006CJ.pdf [accessed 2008 Feb 6]
42. Wood F, Lacroix EM. NLM User Studies. In: Minutes of the Board of Regents February 10-11, 2004. Bethesda, MD: National Library of Medicine; 2004. p. 9-11. URL: http://www.nlm.nih.gov/od/bor/bor204.pdf [accessed 2007 Nov 19]
43. National Institute of Dental and Craniofacial Research. Satisfaction with NIDCR Web site compares favorably with other government and private sector sites. In: Director's Report to the National Advisory Dental and Craniofacial Research Council. June 2005. NODCR/NIH. URL: http://www.nidcr.nih.gov/ NewsAndReports/ReportsPresentation/DirectorsReportJune2005.htm [accessed 2007 Nov 19]
44. Fariss S, Love CB. Now you've built it...Does it work? Paper presented at: E-Government Citizen Satisfaction Summit; September 21, Washington, DC; 2004
45. Feldman S. Evaluation of NCI Website. Paper presented at: E-Government Citizen Satisfaction Summit; September 21,. Washington, DC; 2004. URL: http://www.fcg.gov/documents/e-gov_satisfaction_summit.ppt [accessed 2007 Nov 19]
46. Martin KL, Feldman S. Who's behind the screen & what are they looking for? Paper presented at: National Cancer Institute HINTS Data Users Conference; January 20, 2005, St. Pete Beach, FL. URL: http://hints.cancer.gov/hints/docs/ Martin_Feldman.pdf [accessed 2007 Nov 19]
47. Feldman S. Demonstrating the value, impact and return on investment from evaluating Web-based services: An overview of strategies. In: Paper presented at: Digital Government Institute. Website Evaluation: Making Web Services Integral to Agency Mission; November 7-8, 2005; Washington, DC.
48. Rodrigues D. CSI survey results and how they are used. In: Paper presented at: Healthfinder Steering Committee Meeting; December 11, 2006; Rockville, MD.
49. Cadden C. How physicians are using the federal Treatment Guidelines: Data from AIDSinfo's online American Customer Satisfaction Index survey. In: Paper presented at: American Conference for the Treatment of HIV (ACTHIV); May 31 to June 3 2007; Dallas, TX.
50. Office of Management and Budget. Memorandum for the Heads of Executive Departments and Agencies. June 22, 2000. Privacy Policies and Data Collection on Federal Web sites. whitehouse.gov. URL: http://www.whitehouse.gov/omb/ memoranda/m00-13.html [accessed 2007 Nov 19]
51. Freed L. American Customer Satisfaction Index E-Government Satisfaction Index. Ann Arbor, MI: ForeSee Results: 2006 Dec 15. URL: http://www.fcg.gov/ documents/acsi-results-12-2006.pdf [accessed 2007 Nov 19]
52. Freed L. American Customer Satisfaction Annual E-Business Report. Ann Arbor MI: ForeSee Results; 2006. URL: http://www.foreseeresults.com/_downloads/ acsicommentary/ACSI_E-Business_Report_Aug06.pdf
53. Chabrow E. High marks for government jobs and health sites: the University of Michigan's quarterly customer-satisfaction survey gives some government sites higher scores than commercial sites. InformationWeek. 2004 Mar 15. URL: http://www.informationweek.com/story/showArticle.jhtml?articleID= 18312139 [accessed Nov 19]
54. Michael S. Consumers wowed by Web sites. FCW.com. 2004 Mar 21. URL: http://www.fcw.com/print/10_7/news/82380-1.html [accessed 2007 Nov 19]
55. Pulliam D. Customer satisfaction with federal Web sites nudges up a notch. GovernmentExecutive.com. 2005 Jun 13. URL: http://www.govexec.com/ dailyfed/0605/061305p1.htm [accessed 2007 Nov 19]
56. Perera D. Health sites spur satisfaction. Government Health IT. 2005 Jun 14. URL: http://govhealthit.com/article89241-06-14-05-Web
57. NIH Web sites win customer satisfaction. 2007 Jan 12. (1) p. 12. URL: http://nihrecord.od.nih.gov/pdfs/2007/01122007_Record.pdf [accessed 2007 Nov 19]