A French clinical corpus with comprehensive semantic annotations: development of the Medical Entity and Relation LIMSI annOtated Text corpus (MERLOT)

Language Resources and Evaluation

Volumn 52, Issue 2, 2018, Pages 571-601

  Campillos, Leonardo ^a   Deléger, Louise ^a   Grouin, Cyril ^a   Hamon, Thierry ^a   Ligozat, Anne Laure ^a   Névéol, Aurélie ^a  

^a UFR 919 Laboratoire d'Informatique Pour la Mécanique et les Sciences de l'Ingénieur   (France)

Author keywords

Clinical narrative; Inter annotator agreement; Personal health information; Semantic annotations

Indexed keywords

EID: 85012941458     PISSN: 1574020X     EISSN: 15728412     Source Type: Journal    
DOI: 10.1007/s10579-017-9382-y     Document Type: Article

References (45)

1. Albright, D., Lanfranchi, A., Fredriksen, A., Styler, W., Warner, C., Hwang, J., et al. (2013). Towards comprehensive syntactic and semantic annotations of the clinical narrative. JAMIA, 20(5), 922–930.
2. Alex, B., Haddow, B., & Grover, C. (2007). Recognising nested named entities in biomedical text. In Proceedings of BioNLP 2007 workshop (pp. 65–72). ACL.
3. Altman, D. G. (1990). Practical statistics for medical research. Boca Raton: CRC Press.
4. Bada, M., Eckert, M., Evans, D., Garcia, K., Shipley, K., Sitnikov, D., et al. (2012). Concept annotation in the CRAFT corpus. BMC Bioinformatics, 13(1), 1.
5. Bethard, S., Derczynski, L., Savova, G., Savova, G., Pustejovsky, J., & Verhagen, M. (2015). SemEval-2015 task 6: Clinical tempeval. In Proceedings of the international workshop on semantic evaluation (SemEval) (pp. 806–814). ACL.
6. Bethard, S., Savova, G., Chen, W.-T., Derczynski, L., Pustejovsky, J., & Verhagen, M. (2016). SemEval-2016 task 12: Clinical tempeval. In Proceedings of SemEval Workshop (pp. 1052–1062). ACL.
7. Bodenreider, O. (2004). The Unified Medical Language System (UMLS): Integrating biomedical terminology. Nucleic Acids Research, 32(suppl 1), D267–D270.
8. Bodenreider, O., & McCray, A. T. (2003). Exploring semantic groups through visual approaches. Journal of Biomedical Informatics, 36(6), 414–432.
9. Cohen, K., & Demner-Fushman, D. (2014). Biomedical natural language processing. Natural Language Processing Series. John Benjamins.
10. Deléger, L., Grouin, C., Ligozat, A.-L., Zweigenbaum, P., & Névéol, A. (2014a). Annotation of specialized corpora using a comprehensive entity and relation scheme. In Proceedings of LREC 2014 Reikjavik, Iceland.
11. Deléger, L., Grouin, C., & Névéol, A. (2014b). Automatic content extraction for designing a french clinical corpus. In Proceedings of AMIA annual symposium Washington, DC: American Medical Informatics Association (AMIA).
12. Deléger, L., & Névéol, A. (2014). Identification automatique de zones dans des documents pour la constitution d’un corpus médical en français. In Traitement Automatique de la Langue Naturelle–TALN (pp. 568–573).
13. Elhadad, N., Pradhan, S., Chapman, W., Manandhar, S., & Savova, G. (2015). SemEval-2015 task 14: Analysis of clinical text. In Proceedings of SemEval workshop. (pp. 303–10). ACL.
14. Grishman, R., & Sundheim, B. (1996). Message Understanding Conference-6: A brief history. In Proceedings of the 16th COLING conference, vol. 1 (pp. 466–471). Stroudsburg, PA: Association for Computational Linguistics.
15. Grouin, C., Lavergne, T., & Névéol, A. (2014). Optimizing annotation efforts to build reliable annotated corpora for training statistical models. In LAW VIII–the 8th linguistic annotation workshop, 2014 (pp. 54–58).
16. Grouin, C., & Névéol, A. (2014). De-identification of clinical notes in French: Towards a protocol for reference corpus development. Journal of Biomedical Informatics, 50, 151–161.
17. Habert, B., Grabar, N., Jacquemart, P., & Zweigenbaum, P. (2001). Building a text corpus for representing the variety of medical language. In Proceedings Corpus Linguistics, Lancaster.
18. Hirohata, K., Okazaki, N., Ananiadou, S., & Ishizuka, M. (2008).Identifying sections in scientific abstracts using conditionalrandom fields. In Proceedings of the IJCNLP 2008.
19. Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. Annals of Mathematical Statistics, 22, 49–86.
20. Lavergne, T., Cappé, O., & Yvon, F. (2010). Practical very large scale CRFs. In Proceedings of the 48th annual meeting of the association for computational linguistics (ACL) (pp. 504–513). Association for Computational Linguistics.
21. McCray, A. T., Burgun, A., & Bodenreider, O. (2001). Aggregating UMLS semantic types for reducing conceptual complexity. In Proceedings of MedInfo, (vol. 10, pp. 216–20).
22. Mork, J., Bodenreider, O., Demner-Fushman, D., Islamaj Doğan, R., Lang, F., Lu, Z., et al. (2010). Extracting Rx information from clinical narrative. JAMIA, 17(5), 536–9.
23. Namer, F. (2004). Flemm: un analyseur flexionnel de français à base de règles. Traitement Automatique des langues naturelles (TALN), 41, 523–47.
24. Neves, M., & Leser, U. (2012). A survey on annotation tools for the biomedical literature. Brief Bioinformatics, (pp. 523–47).
25. Névéol, A., Kim, W., Wilbur, W., & Lu, Z. (2009). Exploring two biomedical text genres for disease recognition. In Proceedings of the NAACL HLT 2009 BioNLP workshop (pp. 144–52). ACL.
26. Ogren, P., Savova, G., & Chute, C. (2008). Constructing evaluation corpora for automated clinical named entity recognition. In Proceedings of LREC 2008 Marrakech, Morocco.
27. Oronoz, M., Gojenola, K., Pérez, A., de Ilarraza, A. D., & Casillas, A. (2015). On the creation of a clinical gold standard corpus in Spanish: Mining adverse drug reactions. Journal of Biomedical Informatics, 56, 318–332.
28. Pradhan, S., Elhadad, N., Chapman, W., Manandhar, S., & Savova, G. (2014). SemEval-2014 task 7: Analysis of clinical text. In Proceedings of SemEval workshop Vol. 199, no. 99 (p 54).
29. Pustejovsky, J., Castano, J. M., Ingria, R., Sauri, R., Gaizauskas, R. J., Setzer, A., et al. (2003). TimeML: Robust specification of event and temporal expressions in text. New Directions in Question Answering, 3, 28–34.
30. Roberts, A., Gaizauskas, R., Hepple, M., Demetriou, G., Guo, Y., Roberts, I., et al. (2009). Building a semantically annotated corpus of clinical texts. Journal of Biomedical Semantics, 42, 950–966.
31. Savkov, A., Carroll, J., Koeling, R., & Cassell, J. (2016). Annotating patient clinical records with syntactic chunks and named entities: The Harvey corpus. Language Resources and Evaluation, (pp. 1–26).
32. Savova, G., Styler, W., Albright, D., Palmer, M., Harris, D., & Zaramba, G., et al. (2012). SHARP template annotations: Guidelines. Technical report, Mayo Clinic.
33. Schmid, H. (1995). Treetagger| a language independent part-of-speech tagger. Institut für Maschinelle Sprachverarbeitung, Universität Stuttgart, 43, 28.
34. Sinclair, J. (2005). Corpus and text–basic principles. Developing Linguistic Corpora: A Guide to Good Practice, (pp. 1–16).
35. Stenetorp, P., Pyysalo, S., Topić, G., Ohta, T., Ananiadou, S., & Tsujii, J. (2012). BRAT: A Web-based tool for NLP-assisted text annotation. In Proceedings of the Demonstrations Session at EACL 2012 (pp. 102–7).
36. Styler, W, I. V., Bethard, S., Finan, S., Palmer, M., Pradhan, S., de Groen, P., et al. (2014). Temporal annotation in the clinical domain. Transactions of the ACL, 2, 143–154.
37. Sun, W., Rumshisky, A., & Uzuner, O. (2013). Evaluating temporal relations in clinical text: 2012 i2b2 challenge. JAMIA, 20(5), 806–813.
38. Tao, C., Jiang, G., Oniki, T., Freimuth, R., Zhu, Q., Sharma, D., et al. (2013). A semantic-web oriented representation of the clinical element model for secondary use of electronic health records data. JAMIA, 20(3), 554–62.
39. Tapi Nzali, M.D., Névéol, A., & Tannier, X. (2015). Analyse d’expressions temporelles dans les dossiers électroniques patients. In Actes de TALN 2015 (pp. 144–52).
40. Tepper, M., Capurro, D., Xia, F., Vanderwende, L., & Yetisgen-Yildiz, M. (2012). Statistical section segmentation in free-text clinical records. In Proceedings of LREC 2012 Istanbul, Turkey.
41. Uzuner, Ö., Solti, I., & Cadag, E. (2010). Extracting medication information from clinical text. JAMIA, 17(5), 514–518.
42. Uzuner, Ö., South, B. R., Shen, S., & DuVall, S. L. (2011). 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. JAMIA, 18(5), 552–556.
43. Verspoor, K., Yepes, A.J., Cavedon, L., McIntosh, T., Herten-Crabb, A., Thomas, Z., & Plazzer, J.-P. (2013). Annotating the biomedical literature for the human variome. Database.
44. Wu, S., Kaggal, V., Dligach, D., Masanz, J., Chen, P., Becker, L., et al. (2013). A common type system for clinical natural language processing. Journal of Biomedical Semantics, 4(1), 1.
45. Zweigenbaum, P., Baud, R. H., Burgun, A., Namer, F., Jarrousse, E., Grabar, N., et al. (2005). A unified medical lexicon for French. International Journal of Medical Informatics, 74(2–4), 119–124.