Towards a framework for developing semantic relatedness reference standards

Journal of Biomedical Informatics

Volumn 44, Issue 2, 2011, Pages 251-265

  Pakhomov, Serguei V S ^a,c   Pedersen, Ted ^b   McInnes, Bridget ^a   Melton, Genevieve B ^c   Ruggieri, Alexander ^e   Chute, Christopher G ^d  

^a UNIVERSITY OF MINNESOTA   (United States)

^b UNIVERSITY OF MINNESOTA DULUTH   (United States)

^c UNIVERSITY OF MINNESOTA   (United States)

^d Mayo Clinic College of Medicine   (United States)

^e Wellpoint Inc   (United States)

Author keywords

Inter annotator agreement; Reference standards; Reliability; Semantic relatedness

Indexed keywords

AUTOMATIC CLASSIFICATION; DATA-DRIVEN APPROACH; FACTOR ANALYSIS; FUNCTIONAL TESTING; INTER-ANNOTATOR AGREEMENT; MEASURES OF SEMANTIC RELATEDNESS; MEDICAL INFORMATICS; MEDICAL RECORD; MEDICAL TERMS; ONTOLOGY-BASED; OPEN-SOURCE; REFERENCE STANDARD; RESEARCH AREAS; SEMANTIC RELATEDNESS;

COMPUTATIONAL LINGUISTICS; INFORMATION RETRIEVAL; INFORMATION SCIENCE; NATURAL LANGUAGE PROCESSING SYSTEMS; ONTOLOGY; RELIABILITY; SEMANTICS;

STANDARDIZATION;

ARTICLE; AUTOMATION; COMPUTER PROGRAM; ELECTRONIC MEDICAL RECORD; EVALUATION; FACTORIAL ANALYSIS; INFORMATION RETRIEVAL; MEDICAL EDUCATION; MEDICAL INFORMATICS; MEDICAL RECORD; NOMENCLATURE; PARAMETER; PRIORITY JOURNAL; REFERENCE STANDARD; RELIABILITY; REPRODUCIBILITY; SEMANTIC RELATEDNESS; SEMANTICS; STANDARD; TECHNIQUE;

CLINICAL CODING; DATABASES, FACTUAL; MEDICAL INFORMATICS; MEDICAL RECORDS SYSTEMS, COMPUTERIZED; REFERENCE STANDARDS; SEMANTICS; SOFTWARE;

EID: 79952759135     PISSN: 15320464     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jbi.2010.10.004     Document Type: Article

References (59)

1. Pakhomov S.V., Buntrock J., Chute C.G. Prospective recruitment of patients with congestive heart failure using an ad-hoc binary classifier. J Biomed Inform 2005, 38:145-153.
2. Resnik P. Semantic similarity in a taxonomy: an information-based measure and its application to problems of ambiguity in natural language. J Artificial Intell Res 1999, 11:95-130.
3. Rada R., Mili F., Bicknell E., Blettner A.M. Development and application of a metric on semantic nets. IEEE Trans Syst, Man, Cybernet 1989, 19:17-30.
4. Pakhomov SV, McInnes B, Adam T, Liu Y, Pedersen T, Melton GB. Semantic similarity and relatedness between clinical terms: an experimental study. In: Proceedings of American medical informatics association symposium. Washington (DC); 2010. in press.
5. Mozzicato P. Standardised MedDRA queries: their role in signal detection. Drug Saf 2007, 30:617-619.
6. Pearson R.K., Hauben M., Goldsmith D.I., et al. Influence of the MedDRA((R)) hierarchy on pharmacovigilance data mining results. Int J Med Inform 2009.
7. Bousquet C, Jaulent M, Chantellier G, Degoulet P. Using semantic distance for the efficient coding of medical concepts. In: Proceedings of the American medical informatics association symposium; 2000. p. 96-100.
8. Bousquet C., Lagier G., Lillo-Le L.A., Le Beller C., Venot A., Jaulent M.C. Appraisal of the MedDRA conceptual structure for describing and grouping adverse drug reactions. Drug Saf 2005, 28:19-34.
9. Caviedes J., Cimino J. Towards the development of a conceptual distance metric for the UMLS. J Biomed Inform 2004, 37:77-85.
10. Pedersen T., Pakhomov S.V., Patwardhan S. Measures of semantic similarity and relatedness in the biomedical domain. J Biomed Inform 2006, 40:288-299.
11. Al-Mubaid H., Nguyen H.A. A cluster-based approach for semantic similarity in the biomedical domain. Conf Proc IEEE Eng Med Biol Soc 2006, 1:2713-2717.
12. Lee WN, Shah N, Sundlass K, Musen M. Comparison of ontology-based semantic-similarity measures. In: Proceedings of the American medical informatics association symposium; 2008. p. 384-8.
13. Budanitsky A., Hirst G. Evaluating WordNet-based measures of semantic distance. Comput Linguist 2006, 32:13-47.
14. Resnik P. WordNet and class-based probabilities. WordNet: an electronic lexical database 1998, 239-263. MIT Press, Cambridge (MA). C. Fellbaum (Ed.).
15. Patwardhan S, Banerjee S, Pedersen T. Using measures of semantic relatedness for word sense disambiguation. In: Proceedings of the 4th international conference on intelligent text processing and computational linguistics; 2003. p. 241-57.
16. WordNet: an electronic lexical database 1998, MIT Press, Cambridge (MA). C. Fellbaum (Ed.).
17. Miller G., Charles W. Contextual correlates of semantic similarity. Language Cognitive Process 1991, 6:1-28.
18. Burgun A, Bodenreider O. Comparing terms, concepts and semantic classes in WordNet and the unified medical language system. In: Proceedings of the workshop on WordNet and other lexical resources: applications, extensions, and customizations; 2001. p. 77-82.
19. Bodenreider O, Burgun A. Characterizing the definitions of anatomical concepts in WordNet and specialized sources. In: Proceedings of the first global WordNet conference; 2002. p. 223-30.
20. Lord P.W., Stevens R.D., Brass A., Goble C.A. Investigating semantic similarity measures across the gene ontology: the relationship between sequence and annotation. Bioinformatics 2003, 19:1275-1283.
21. Guo X., Liu R., Shriver C.D., Hu H., Liebman M.N. Assessing semantic similarity measures for the characterization of human regulatory pathways. Bioinformatics 2006, 22:967-973.
22. Rada R., Mili H., Bicknell E., Blettner M. Development and application of a metric on semantic nets. IEEE Trans Syst, Man Cybernet 1989, 19:17-30.
23. Rodriguez M., Egenhofer M. Determining semantic similarity among entity classes from different ontologies. IEEE Trans Knowledge Data Eng 2003, 15:442-456.
24. Wilbur W., Yang Y. An analysis of statistical term strength and its use in the indexing and retrieval of molecular biology texts. Comput Biol Med 1996, 26:209-222.
25. Spasic I, Ananiadou S. A flexible measure of contextual similarity for biomedical terms. In: Proceedings of the Pacific symposium on biocomputing, vol. 10; 2005. p. 197-208.
26. Riensche R, Baddley B, Sanfilippo A, Posse C, Gopalan B. XOA: web-enabled cross-ontological analytics. In: IEEE SCW; 2007. p. 99-105.
27. McInnes B, Pedersen T, Pakhomov S. UMLS-interface and UMLS-similarity: open source software for measuring paths and semantic similarity. In: Proceedings of the American medical informatics symposium, San Francisco (CA); November 2009. p. 431-35.
28. Stevenson M, Greenwood M. A semantic approach to IE pattern induction. In: Proceedings of the 43rd annual meeting of the association for computational linguistics; 2005. p. 379-86.
29. Raina R, Ng A, Manning C. Robust textual inference via learning and abductive reasoning. In: Proceedings of the twentieth national conference on artificial intelligence; 2005. p. 1099-105.
30. Bodenreider O, Burgun A. Aligning knowledge sources in the UMLS: methods, quantitative results, and applications. In: Proceedings of Medinfo symposium; 2004. p. 327-31.
31. Melton G.B., Parsons S., Morrison F.P., Rothschild A.S., Markatou M., Hripcsak G. Inter-patient distance metrics using SNOMED CT defining relationships. J Biomed Inform 2006, 39:697-705.
32. Neveol A., Zeng K., Bodenreider O. Besides precision & recall: exploring alternative approaches to evaluating an automatic indexing tool for MEDLINE. AMIA Annu Symp Proc 2006, 589-593.
33. Zhu S., Zeng J., Mamitsuka H. Enhancing MEDLINE document clustering by incorporating MeSH semantic similarity. Bioinformatics 2009, 25:1944-1951.
34. Cao H., Melton G.B., Markatou M., Hripcsak G. Use abstracted patient-specific features to assist an information-theoretic measurement to assess similarity between medical cases. J Biomed Inform 2008, 41:882-888.
35. Rubenstein H., Goodenough J. Contextual correlates of synonymy. Commun ACM 1965, 8:627-633.
36. Krippendorff K. Reliability in content analysis: some common misconceptions and recommendations. Human Commun Res 2004, 30:411-433.
37. Pedersen T. Empiricism is not a matter of faith. Comput Linguist 2008, 34:465-470.
38. Melton L.J. History of the Rochester epidemiology project. Mayo Clin Proc 1996, 7:266-274.
39. H-ICDA. H-ICDA Hospital adaptation of ICD-A-HICDA, 2nd ed. Ann Arbor, MI: CPHA (Commission on Professional and Hospital Activities); 1973.
40. Wu Z, Palmer M. Verb semantics and lexical selection. In: 32nd annual meeting of the association for computational linguistics; 1994. p. 133-8.
41. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Measure 1960, 20:37-46.
42. Krippendorff K. Content analysis: an introduction to its methodology 2004, Sage, Thousand Oaks (CA). 2nd ed.
43. Cronbach L.J. Coefficient alpha and the internal structure of tests. Psychometrica 1951, 16.
44. Kendall M.G., Babington Smith B. The problem of m rankings. Ann Math Stat 1939, 10:275-287.
45. Shrout P.E., Fleiss J.L. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979, 86:420-428.
46. McGraw K.O., Wong S.P. Forming inferences about some intraclass correlations coefficients (vol. 1, 1996. p. 30). Psychol Methods 1996, 1:390.
47. Ward J.H. Hierarchical grouping to optimize an objective function. J Am Stat Assoc 1963, 58:236-244.
48. Hartigan J. Clustering algorithms 1975, Wiley and Sons.
49. Pedersen T, Kulkarni A. Automatic cluster stopping with criterion functions and the gap statistic. In: Proceedings of the demonstration session of the human language technology conference and the sixth annual meeting of the north American chapter of the association for computational linguistics; 2006. p. 276-9.
50. Tibshirani R., Walther G., Hastie T. Estimating the number of clusters in a data set via the gap statistic. J Roy Stat Soc Ser B - Stat Method 2001, 63:411-423.
51. Aronson A.R. Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. Proc/AMIA Annu Symp 2001, 17-21.
52. Rindflesch T.C., Tanabe L., Weinstein J.N., Hunter L. EDGAR: extraction of drugs, genes and relations from the biomedical literature. Pacific Symposium on Biocomput 2000, 517-528.
53. Chapman W.W., Fiszman M., Dowling J.N., Chapman B.E., Rindflesch T.C. Identifying respiratory findings in emergency department reports for biosurveillance using MetaMap. Stud Health Technol Inform 2004, 107:487-491.
54. Ferrand L., Ric F., Augustinova M. Affective priming: a case of semantic priming?. Annee Psychol 2006, 106:79-104.
55. Thompson-Schill S.L., Kurtz K.J., Gabrieli J.D.E. Effects of semantic and associative relatedness on automatic priming. J Memory Language 1998, 38:440-458.
56. Weber M., Thompson-Schill S.L., Osherson D., Haxby J., Parsons L. Predicting judged similarity of natural categories from their neural representations. Neuropsychologia 2009, 47:859-868.
57. Mitchell T.M., Shinkareva S.V., Carlson A., et al. Predicting human brain activity associated with the meanings of nouns. Science 2008, 320:1191-1195.
58. Artstein R., Poesio M. Inter-coder agreement for computational linguistics. Comput Linguist 2008, 34:555-596.
59. Landis J.R., Koch G.G. The measurement of observer agreement for categorical data. Biometrics 1977, 33:159-174.