User evaluation of the effects of a text simplification algorithm using term familiarity on perception, understanding, learning, and information retention

Journal of Medical Internet Research

Volumn 15, Issue 7, 2013, Pages

  Leroy, Gondy ^a,b   Endicott, James E ^a   Kauchak, David ^c   Mouradi, Obay ^a   Just, Melissa ^d  

^a CLAREMONT GRADUATE UNIVERSITY   (United States)

^b UNIVERSITY OF ARIZONA   (United States)

^c MIDDLEBURY COLLEGE   (United States)

^d Rutgers University Libraries   (United States)

Author keywords

Consumer health information; Evaluation study; Health literacy; Natural language processing; Text simplification

Indexed keywords

ADULT; AGED; ALGORITHM; ARTICLE; CONSUMER HEALTH INFORMATION; EVALUATION STUDY; FEMALE; HEALTH LITERACY; HUMAN; INFORMATION SERVICE; MALE; MIDDLE AGED; NATURAL LANGUAGE PROCESSING; TEXT SIMPLIFICATION; UNIFIED MEDICAL LANGUAGE SYSTEM; WRITING; YOUNG ADULT;

CONSUMER HEALTH INFORMATION; EVALUATION STUDY; HEALTH LITERACY; NATURAL LANGUAGE PROCESSING; TEXT SIMPLIFICATION;

ADULT; AGED; ALGORITHMS; FEMALE; HUMANS; INFORMATION SERVICES; MALE; MIDDLE AGED; UNIFIED MEDICAL LANGUAGE SYSTEM; WRITING; YOUNG ADULT;

EID: 84883422193     PISSN: None     EISSN: 14388871     Source Type: Journal    
DOI: 10.2196/jmir.2569     Document Type: Article

References (52)

1. Fox S. Health Topics.: Pew Research Center; 2011. URL: http://www.pewinternet.org/Reports/2011/HealthTopics.aspx[accessed 2013-07-07] [WebCite Cache ID 6HwPg8amf]
2. Wang Y. Automatic Recognition of Text Difficulty from Consumers Health Information. In: Proceedings of the 19th IEEE International Symposium on Computer-Based Medical Systems. 2006 Presented at: 19th IEEE International Symposium on Computer-Based Medical Systems; June 22-23, 2006; Salt Lake City.
3. Nielsen-Bohlman L, Institute of Medicine /Committee on Health. 'Health literacy: a prescription to end confusion'. Maryland: National Academies Press; 2004.
4. Yan X, Song D, Li X. Concept-based Document Readability in Domain Specific Information Retrieval. In: Proceedings of the 15th ACM International Conference on Information and Knowledge Management. 2006 Presented at: 15th ACM International Conference on Information and Knowledge Management; Nov. 5-11, 2006; Arlington, Virginia p. 540-549.
5. Zarcadoolas C. The simplicity complex: exploring simplified health messages in a complex world. Health Promot Int 2011 Sep;26(3):338-350 [FREE Full text] [doi: 10.1093/heapro/daq075] [Medline: 21149317]
6. Low health literacy report. URL: http://sphhs.gwu.edu/departments/ healthpolicy/CHPR/downloads/ LowHealthLiteracyReport10-4-07.pdf [accessed 2013-02-06] [WebCite Cache ID 6EEgqYeox]
7. Diakidoy IA, Mouskounti T, Ioannides C. Comprehension and Learning from Refutation and Expository Texts. Reading Research Quarterly 2011;43(1):22. [doi: 10.1598/RRQ.46.1.2]
8. Leroy G, Miller T. Perils of providing visual health information overviews for consumers with low health literacy or high stress. J Am Med Inform Assoc 2010;17(2):220-223 [FREE Full text] [doi: 10.1136/jamia.2009.002717] [Medline: 20190068]
9. Van Servellen G, Brown JS, Lombardi E, Herrera G. Health literacy in low-income Latino men and women receiving antiretroviral therapy in community-based treatment centers. AIDS Patient Care and STDs 2003 Jun;17(6):283-298. [doi: 10.1089/108729103322108166] [Medline: 12880492]
10. Kintsch W. Learning from Text. Cognition and Instruction 1986;3(2):87.
11. Landi N. An examination of the relationship between reading comprehension, higher-level and lower-level reading sub-skills in adults. Read Writ 2010 Jul 1;23(6):701-717 [FREE Full text] [doi: 10.1007/s11145-009-9180-z] [Medline: 21691452]
12. Stanovich KE, West RF. Exposure to Print and Orthographic Processing. Reading Research Quarterly 1989;24(4):402.
13. Leroy G, Lauchak D, Mouradi O. A User-study Measuring the Effects of Lexical Simplification and Coherence Enhancement on Perceived and Actual Text Difficulty. Int J Med Inform 2013 (forthcoming).
14. Janz NK, Becker MH. The Health Belief Model: a decade later. Health Educ Q 1984;11(1):1-47. [Medline: 6392204]
15. Trafimow D, Sheeran P, Conner M, Finlay KA. Evidence that perceived behavioural control is a multidimensional construct: perceived control and perceived difficulty. Br J Soc Psychol 2002 Mar;41(Pt 1):101-121. [Medline: 11970777]
16. Velayo RS. Retention of Content as a Function of Presentation Mode and Perceived Difficulty. Reading Improvement 1993;30(4):216.
17. Leroy G, Endicott J. Term Familiarity to Indicate Perceived and Actual Difficulty of Text in Medical Digital Libraries. In: Proceedings of the International Conference on Asia-Pacific Digital Libraries (ICADL) - Digital Libraries - for Culture Heritage, Knowledge Dissemination, and Future Creation. 2011 Presented at: International Conference on Asia-Pacific Digital Libraries (ICADL) - Digital Libraries - for Culture Heritage, Knowledge Dissemination, and Future Creation; October 2011; Beijing, China p. 24-27.
18. Leroy G, Endicott J. Combining NLP with Evidence-based Methods to Find Text Metrics related to Perceived and Actual Text Difficulty. In: Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium (ACM IHI). 2012 Presented at: 2nd ACM SIGHIT International Health Informatics Symposium (ACM IHI); January 2012; Miami, FL p. 28-30.
19. Pitler E, Nenkova A. Revisiting Readability: A Unified Framework for Predicting Text Quality. In: Proceedings of the Empirical Methods in Natural Language Processing. 2008 Presented at: Empirical Methods in Natural Language Processing; 2008; Honolulu.
20. Soleimani H, Mohammadi E. The Effect of Text Typographical Features on Legibility,Comprehension, and Retrieval of EFL Learners. English Language Teaching 2012;5(8):207.
21. Goldman SR, Murray JD. Knowledge of Connectors as Cohesion Devices in Text: A Comparative Study of Native-English and English-as-a-Second-Language Speakers. Journal of Educational Psychology 1992;84(4):504.
22. Boscolo P, Mason L. Topic Knowledge, Text Coherence, and Interest: How they Interact in Learning from Instructional Texts. The Journal of Experimental Education 2003;7(2):126.
23. McNamara DS, Kintsch E, Songer NB, Kintsch W. Are Good Texts Always Better? Interactions of Text Coherence, Background Knowledge, and Levels of Understanding in Learning from Text. Cognition and Instruction 1996;14(1):1.
24. DuBay WH. The Principles of Readability. Impact Information 2004.
25. Bruce B, Rubin A, Starr K. Why Readability Formulas Fail. IEEE Transactions on Professional Communication 1981;24(1):50.
26. Wang LW, Miller MJ, Schmitt MR, Wen FK. Assessing Readability Formula Differences with Written Health Information Materials: Application, Results, and Recommendations. Research in Social & Administrative Pharmacy 2012 (forthcoming).
27. Sirico LJ. Readability Studies: How Technocentrism Can Compromise Research and Legal Determinations. Villanova University Legal Working Paper Series 2008.
28. Polishchuk DL, Hashem J, Sabharwal S. Readability of online patient education materials on adult reconstruction Web sites. J Arthroplasty 2012 May;27(5):716-719. [doi: 10.1016/j.arth.2011.08.020] [Medline: 22000573]
29. Ahmed OH, Sullivan SJ, Schneiders AG, McCrory PR. Concussion information online: evaluation of information quality, content and readability of concussion-related websites. Br J Sports Med 2012 Jul;46(9):675-683. [doi: 10.1136/bjsm.2010.081620] [Medline: 21504964]
30. Bailey M, Coughlin PA, Sohrabi S, Griffin KJ, Rashid ST, Troxler MA, et al. Quality and readability of online patient information for abdominal aortic aneurysms. J Vasc Surg 2012 Jul;56(1):21-26. [doi: 10.1016/j.jvs.2011.12.063] [Medline: 22521801]
31. Hendrick PA, Ahmed OH, Bankier SS, Chan TJ, Crawford SA, Ryder CR, et al. Acute low back pain information online: an evaluation of quality, content accuracy and readability of related websites. Man Ther 2012 Aug;17(4):318-324. [doi: 10.1016/j.math.2012.02.019] [Medline: 22464886]
32. Kim H, Goryachev S, Rosemblat G, Browne A, Keselman A, Zeng-Treitler Q. Beyond Surface Characteristics: A New Health Text-Specific Readability Measurement. AMIA Annu Symp Proc 2007:418-422.
33. Gemoets D, Rosemblat G, Tse T, Logan R. Assessing readability of consumer health information: an exploratory study. Stud Health Technol Inform 2004;107(Pt 2):869-873. [Medline: 15360936]
34. Clinton V, Broek P. Interest, inferences, and learning from texts. Learning and Individual Differences 2012;22(6):650.
35. Erçetin G. Effects of Topic Interest and Prior Knowledge on Text Recall and Annotation use in Reading a Hypermedia Text in the L2. ReCALL 2010;22(2):228.
36. Leroy G, Endicott J, Mouradi O, Kauchak D, Just M. Improving Perceived and Actual Text Difficulty for Health Information Consumers using Semi-Automated Methods. In: Proceedings of the American Medical Informatics Association (AMIA) Fall Symposium. 2012 Presented at: American Medical Informatics Association (AMIA) Fall Symposium; Nov. 3-7, 2012; Chicago, IL.
37. Web 1T 5-gram Corpus Version 1.1. URL: http://www.ldc.upenn.edu/Catalog/ docs/LDC2006T13/readme.txt [accessed 2013-07-07] [WebCite Cache ID 6HwSRwun2]
38. Fellbaum C. WordNet: An Electronic Lexical Database. Cambridge, MA: MIT Press; 1998.
39. Miller GA. WordNet: a lexical database for English. Commun. ACM 1995;38(11):39.
40. How the GRE tests are scored. URL: http://www.ets.org/gre/institutions/ scores/how [accessed 2013-02-06] [WebCite CacheID 6EEh5ILJE]
41. Kittur A, Chi E, Suh B. Crowdsourcing User Studies with Mechanical Turk. In: Proceedings of the SIGCHI Conference on Human Factors in System Computing. 2008 Presented at: Proceedings of the SIGCHI Conference on Human Factors in System Computing; 2008; Florence, Italy p. 453-456.
42. Ross J, Irani L, Silberman MS, Zaldivar A, Tomlinson B. Who are the Crowdworkers? Shifting Demographics in Mechanical Turk. In: Proceedings of the CHI '10 Extended Abstracts on Human Factors in Computing Systems. 2010 Presented at: CHI '10 Extended Abstracts on Human Factors in Computing Systems; 2010; Atlanta, Georgia.
43. Paolacci G, Changler J, Ipeirotis PG. Running Experiments on Amazon Mechanical Turk. Judgment and Decision-making 2010;5(5):411.
44. Buhrmester M, Kwang T, Gosling SD. Amazon's Mechanical Turk A New Source of Inexpensive, Yet High-Quality, Data? Perspectives on Psychological Science 2011;6(1).
45. Cohen S, Kamarck T, Mermelstein R. A Global Measure of Perceived Stress. Journal of Health and Social Behavior 1983;24(4):385.
46. Nurss JR, Parker RM, Williams MV, Baker DW. Test of Functional Health Literacy in Adults. Hartford, MI: Peppercorn Books & Press; 1995.
47. Parker RM, Baker DW, Williams MV, Nurss Jr. The Test of Functional Health Literacy in Adults: A New Instrument for Measuring Patients' Literacy Skills. Journal of General Internal Medicine 1995;10:537-541.
48. Leroy G, Lally A, Chen H. The Use of Dynamic Contexts to Improve Casual Internet Searching. ACM Transactions on Information Systems 2003;21(3):229.
49. Leroy G, Rindflesch TC. Effects of information and machine learning algorithms on word sense disambiguation with small datasets. Int J Med Inform 2005 Aug;74(7-8):573-585. [doi: 10.1016/j.ijmedinf.2005.03.013] [Medline: 15897005]
50. Zeng-Treitler Q, Goryachev S, Tse T, Keselman A, Boxwala A. Estimating consumer familiarity with health terminology: a context-based approach. J Am Med Inform Assoc 2008;15(3):349-356 [FREE Full text] [doi: 10.1197/jamia.M2592] [Medline: 18308983]
51. Keselman A, Smith CA, Divita G, Kim H, Browne A, Leroy G, et al. Consumer Health Concepts that do not Map to the UMLS: Where Do They Fit? Journal of the American Medical Informatics Association 2008;15(4):496-505.
52. Keselman A, Logan R, Smith CA, Leroy G, Zeng-Treitler Q. Developing informatics tools and strategies for consumer-centered health communication. J Am Med Inform Assoc 2008;15(4):473-483 [FREE Full text] [doi: 10.1197/jamia.M2744] [Medline: 18436895]