By all these lovely tokens... Merging conflicting tokenizations

Language Resources and Evaluation

Volumn 46, Issue 1, 2012, Pages 53-74

  Chiarcos, Christian ^a   Ritz, Julia ^a   Stede, Manfred ^a  

^a UNIVERSITY OF POTSDAM   (Germany)

Author keywords

Conflicting tokenizations; Corpus linguistics; Linguistic annotation; Multi layer annotation; Tokenization alignment

Indexed keywords

EID: 84859105948     PISSN: 1574020X     EISSN: 15728412     Source Type: Journal    
DOI: 10.1007/s10579-011-9161-0     Document Type: Article

References (45)

1. Brants, T. (2000). TnT-A statistical part-of-speech tagger. In Proceedings of the sixth applied natural language processing (ANLP-2000), Seattle, WA, pp. 224-231.
2. Brill, E. (1995). Transformation-based error-driven learning and natural language processing: A case study in part-of-speech tagging. Computational Linguistics, 21(4), 543-565.
3. Burnard, L. (2007). Reference guide for the British national corpus (XML Edition). http://www. natcorp. ox. ac. uk/XMLedition/URG/bnctags. html (August 6, 2011).
4. Carletta, J., Evert, S., Heid, U., Kilgour, J., Robertson, J., & Voormann, H. (2003). The NITE XML toolkit: Flexible annotation for multi-modal language data. Behavior Research Methods, Instruments, and Computers, 35(3), 353-363.
5. Carlson, L., Marcu, D., & Okurowski, M. E. (2003), Building a discourse-tagged corpus in the framework of rhetorical structure theory. In J. van Kuppevelt & R. W. Smith (Eds.), Current and new directions in discourse and dialogue, text, speech, and language technology; 22 (pp. 85-112). Dordrecht: Kluwer.
6. Cheng, L., & Demirdache, H. (1990). Superiority violations. In L. Cheng & H. Demirdache (Eds.), Papers on Wh-movement, MIT working papers in linguistics; 13, MITWPL, pp. 27-46.
7. Chiarcos, C., Dipper, S., Götze, M., Leser, U., Lüdeling, A., Ritz, J., & Stede, M. (2008). A flexible framework for integrating annotations from different tools and tagsets. TAL (Traitement automatique des langues), 49(2), 217-246.
8. Christ, O. (1994). A modular and flexible architecture for an integrated corpus query system. In Proceedings of the 3rd conference on computational lexicography and text research (COMPLEX 94), Budapest, Hungary, pp. 23-32.
9. Cunningham, H., Maynard, D., Bontcheva, K., & Tablan, V. (2002). GATE: An architecture for development of robust HLT applications. In Proceedings of the 40th anniversary meeting of the association for computational linguistics (ACL-2002), Philadelphia, Pennsylvania, pp. 168-175.
10. Dipper, S. (2005). XML-based stand-off representation and exploitation of multi-level linguistic annotation. In Proceedings of berliner XML tage 2005 (BXML 2005), Berlin, Germany, pp. 39-50.
11. Dipper, S., & Götze, M. (2005) Accessing heterogeneous linguistic data - Generic XML-based representation and flexible visualization. In Proceedings of the 2nd language and technology conference (L&T'05), Poznan, Poland, pp. 23-30.
12. Dipper, S., & Götze, M. (2005) Accessing heterogeneous linguistic data - Generic XML-based representation and flexible visualization. In Proceedings of the 2nd language and technology conference (L&T'05), Poznan, Poland, pp. 23-30.
13. Ferrucci, D., & Lally, A. (2004). UIMA: An architectural approach to unstructured information processing in the corporate research environment. Natural Language Engineering, 10(3/4), 327-348.
14. Guo, J. (1997). Critical tokenization and its properties. Computational Linguistics, 4(23), 569-596.
15. Henderson, J. C. (2000). A DTD for reference key annotation of EDT entities and RDC relations in the ACE evaluations (v. 5. 2. 0, 2000/01/05). http://projects. ldc. upenn. edu/ace/annotation/apf. v5. 2. 0. dtd. Accessed 6 August 2011.
16. Heycock, C. (1995). Asymmetries in reconstruction. Linguistic Inquiry, 26(4), 547-570.
17. Hovy, E., Marcus, M., Palmer, M., Ramshaw, L., & Weischedel, R. (2006) OntoNotes: The 90% solution. In Proceedings of the human language technology conference of the NAACL (HLT 2006), New York City, USA, pp. 57-60.
18. Ide, N. (2008). The American national corpus: Then, now and tomorrow. Keynote paper presented at the HCSNet workshop on designing the Australian national corpus, 4-5 December, UNSW, Sydney, Australia.
19. Ide, N., & Suderman, K. (2007). GrAF: A graph-based format for linguistic annotations. In Proceedings of the linguistic annotation workshop (LAW) 2007, Prague, Czech Republic, pp. 1-8.
20. Jiampojamarn, S., & Kondrak, G. (2009). Online discriminative training for grapheme-to-phoneme conversion. In Proceedings of the 10th annual conference of the international speech communication association (Interspeech 2009), Brighton, pp. 1303-1306.
21. Junghanns, U., & Zybatow, G., (1995). Fokus im Russischen. In Proceedings of the Göttingen focus workshop at the 17th annual conference of the German linguistic society (DGfS 1995), Göttingen, Germany, pp. 113-136.
22. Kaplan, R., & Newman, P. (1997). Lexical resource reconciliation in the xerox linguistic environment. In Proceedings of the ACL'97 workshop on computational environments for grammar development and linguistic engineering, Madrid, Spain, pp. 54-61.
23. Kingsbury, P., & Palmer, M. (2002). From TreeBank to PropBank. In Proceedings of the third international conference on language resources and evalution (LREC 2002), Las Palmas, Spain, pp. 1989-1993.
24. Kohler, K. (1996). Labelled data bank of spoken standard German. The Kiel Corpus of read/spontaneous speech. In Proceedings of the fourth international conference on spoken language processing (ICSLP'96), Philadelphia, pp. 1938-1941.
25. König, E., & Lezius, W. (2000). A description language for syntactically annotated corpora. In Proceedings of the 18th international conference on computational linguistics (COLING 2000), Saarbrücken, Germany, pp. 1056-1060.
26. Lezius, W. (2002). TIGERSearch. Ein Suchwerkzeug für Baumbanken. In Proceedings of the 6th Konferenz zur Verarbeitung natürlicher Sprache (KONVENS 2002), Saarbrücken, Germany, pp. 107-114.
27. Manning, C., & Schütze, H. (1999). Foundations of statistical natural language processing. Cambridge: MIT Press.
28. Marcus, M. P., Santorini, B., & Marcinkiewicz, M. A. (1993). Building a large annotated corpus of English: The PennTreeBank. Computational Linguistics, 19, 313-330.
29. Meyers, A., Reeves, R., Macleod, C., Szekely, R., Zilinska, V., & Young, B. (2004). The NomBank project: An interim report. In HLT-NAACL workshop on frontiers in corpus Annotation, Boston, Massachusetts, pp. 24-31.
30. Müller, S. (2005). Zur Analyse der scheinbar mehrfachen Vorfeldbesetzung. Linguistische Berichte, 203, 297-330.
31. Müller, C., & Strube, M. (2006). Multi-level annotation of linguistic data with MMAX2. In S. Braun, K. Kohn, & J. Mukherjee (Eds.), Corpus technology and language pedagogy: New resources, new tools, new methods (pp. 197-214). Frankfurt, Germany: Peter Lang.
32. Poesio, M., & Artstein, R. (2008). Anaphoric annotation in the ARRAU corpus. In N. Calzolari, K. Choukri, B. Maegaard, J. Mariani, J. Odjik, S. Piperidis, & D. Tapias (Eds.), Proceedings of the sixth international language resources and evaluation (LREC 2008), Marrakech, Morocco.
33. Prasad, R., Dinesh, N., Lee, A., Miltsakaki, E., Robaldo, L., Joshi, A., & Webber, B. (2008). The Penn Discourse TreeBank 2. 0. In Proceedings of the sixth international language resources and evaluation (LREC 2008), Marrakech, Morocco.
34. Pustejovsky, J., Hanks, P., Saurí, R., See, A., Gaizauskas, R., Setzer, A., Radev, D., Beth Sundheim, D. D., Ferro, L., & Lazo, M. (2003). The TIMEBANK corpus. In Corpus linguistics, pp. 647-656.
35. Rehm, G., Schonefeld, O., Witt, A., Chiarcos, C., & Lehmberg, T. (2008). SPLICR: A sustainability platform for linguistic corpora and resources. In A. Storrer, A. Geyken, A. Siebert, & K. M. Würzner (Eds.), Text resources and lexical knowledge (pp. 85-96). Berlin, Germany: Mouton de Gruyter.
36. Sampson, G. R. (1999). CHRISTINE corpus, stage I: Documentation. http://www. grsampson. net/ChrisDoc. htm.
37. Schmidt, T. (2004). Transcribing and annotating spoken language with EXMARaLDA. In Proceedings of the LREC 2004 workshop on XML based richly annotated corpora, Lisboa, Portugal.
38. Sekerina, I. (1997). The syntax and processing of scrambling constructions in Russian. PhD thesis, The City University of New York.
39. Stede, M., Bieler, H., Dipper, S., & Suriyawongkul, A. (2006). Summar: Combining linguistics and statistics for text summarization. In Proceedings of the 17th European conference on artificial intelligence (ECAI-06), Riva del Garda, Italy, pp. 827-828.
40. Vilain, M., Burger, J., Aberdeen, J., Connolly, D., & Hirschman, L. (1995) A model-theoretic coreference scoring scheme. In MUC6: Proceedings of the 6th conference on message understanding, Morristown, NJ, USA, pp. 45-52.
41. Wolf, F., & Gibson, E. (2005). Representing discourse coherence: A corpus-based study. Computational Linguistics, 31(2), 249-287.
42. Wu, D. (1998). A position statement on chinese segmentation. In Proceedings of the Chinese language processing workshop, University of Pennsylvania, Pennsylvania, Philadelphia.
43. Yamamoto, K., Kudo. T., Konagaya, A., & Matsumoto, Y. (2003). Protein name tagging for biomedical annotation in text. In Proceedings of the ACL 2003 workshop on natural language processing in biomedicine, Morristown, NJ, USA, pp. 65-72.
44. Zeldes, A., Ritz, J., Lüdeling, A., & Chiarcos, C. (2009). ANNIS: A search tool for multi-layer annotated corpora. In Proceedings of corpus linguistics 2009, Liverpool, UK.
45. Zipser, F., & Romary, L. (2010). A model oriented approach to the mapping of annotation formats using standards. In Proceedings of the 7th international conference on language resources and evaluation (LREC 2010), Valetta, Malta.