Learning multilingual named entity recognition from Wikipedia

Artificial Intelligence

Volumn 194, Issue , 2013, Pages 151-175

  Nothman, Joel ^a,b   Ringland, Nicky ^a   Radford, Will ^a,b   Murphy, Tara ^a   Curran, James R ^a,b  

^a UNIVERSITY OF SYDNEY   (Australia)

^b CAPITAL MARKETS CRC LIMITED   (Australia)

Author keywords

Annotated corpora; Information extraction; Named entity recognition; Semi structured resources; Semi supervised learning; Wikipedia

Indexed keywords

ANNOTATED CORPORA; INFORMATION EXTRACTION; NAMED ENTITY RECOGNITION; SEMI-STRUCTURED; SEMI-SUPERVISED LEARNING; WIKIPEDIA;

GOLD; SUPERVISED LEARNING;

WEBSITES;

EID: 84870294311     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.artint.2012.03.006     Document Type: Article

References (89)

1. S.F. Adafre, M. de Rijke, Finding similar sentences across multiple languages in Wikipedia, in: Proceedings of the Workshop on NEW TEXT, 11th Conference of the European Chapter of the Association for Computational Linguistics, 2006, pp. 62-69.
2. E. Adar, M. Skinner, D.S. Weld, Information arbitrage across multi-lingual Wikipedia, in: Proceedings of the Second ACM International Conference on Web Search and Data Mining, Barcelona, Spain, 2009, pp. 94-103.
3. J. An, S. Lee, G.G. Lee, Automatic acquisition of named entity tagged corpus from world wide web, in: The Companion Volume to the Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, Sapporo, Japan, 2003, pp. 165-168.
4. R.K. Ando, T. Zhang, A high-performance semi-supervised learning method for text chunking, in: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, Ann Arbor, Michigan, 2005, pp. 1-9.
5. D. Balasuriya, N. Ringland, J. Nothman, T. Murphy, J.R. Curran, Named entity recognition in Wikipedia, in: Proceedings of the Workshop on the Peoples Web Meets NLP: Collaboratively Constructed Semantic Resources, Singapore, 2009, pp. 10-18.
6. A. Bhole, B. Fortuna, M. Grobelnik, D. Mladenić, Mining Wikipedia and relating named entities over time, in: Proceedings of the 10th International Multiconference on Information Society, Jožef Stefan Institute, Ljubljana, 2007, pp. 117-180.
7. F. Biadsy, J. Hirschberg, E. Filatova, An unsupervised approach to biography production using Wikipedia, in: Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Columbus, Ohio, 2008, pp. 807-815.
8. D.M. Bikel, R. Schwartz, and R.M. Weischedel An algorithm that learns whats in a name Machine Learning 34 1999 211 231
9. The British National Corpus, Version 3 (BNC XML edition), distributed by Oxford University Computing Services on behalf of the BNC Consortium, http://www.natcorp.ox.ac.uk/, 2007.
10. G. Bouma, S. Duarte, Z. Islam, Cross-lingual alignment and completion of Wikipedia templates, in: Proceedings of the Third International Workshop on Cross Lingual Information Access: Addressing the Information Need of Multilingual Societies, 2009, pp. 21-29.
11. A. Brunstein, Annotation guidelines for answer types, LDC2005T33, Linguistic Data Consortium, Philadelphia, 2002.
12. R. Bunescu, M. Paşca, Using encyclopedic knowledge for named entity disambiguation, in: Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics, Trento, Italy, 2006, pp. 9-16.
13. N. Chinchor, MUC-7 test scores introduction (Appendix B), in: Proceedings of the 7th Message Understanding Conference, 1998.
14. N. Chinchor, Overview of MUC-7, in: Proceedings of the 7th Message Understanding Conference, 1998.
15. N. Chinchor, P. Robinson, MUC-7 named entity task definition (version 3.5), in: Proceedings of the 7th Message Understanding Conference, 1998.
16. M. Ciaramita, Y. Altun, Named-entity recognition in novel domains with external lexical knowledge, in: Proceedings of the NIPS Workshop on Advances in Structured Learning for Text and Speech Processing, Whistler, British Columbia, 2005.
17. S. Cucerzan, Large-scale named entity disambiguation based on Wikipedia data, in: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Prague, Czech Republic, 2007, pp. 708-716.
18. J.R. Curran, S. Clark, Language independent NER using a maximum entropy tagger, in: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, Edmonton, Canada, 2003, pp. 164-167.
19. W. Dakka, S. Cucerzan, Augmenting Wikipedia with named entity tags, in: Proceedings of the 3rd International Joint Conference on Natural Language Processing, Hyderabad, India, 2008, pp. 545-552.
20. M. Dickinson, W.D. Meurers, Detecting errors in part-of-speech annotation, in: Proceedings of the 10th Conference of the European Chapter of the Association for Computational Linguistics, Budapest, Hungary, 2003, pp. 107-114.
21. M. Ehrmann, M. Turchi, R. Steinberger, Building a multilingual named entity-annotated corpus using annotation projection, in: Proceedings of the International Conference Recent Advances in Natural Language Processing, Hissar, Bulgaria, 2011, pp. 118-124.
22. O. Etzioni, M. Cafarella, D. Downey, A.M. Popescu, T. Shaked, S. Soderland, D.S. Weld, and A. Yates Unsupervised named-entity extraction from the web: An experimental study Artificial Intelligence 165 2005 91 134 (Pubitemid 40556657)
23. R.E. Fan, K.W. Chang, C.J. Hsieh, X.R. Wang, and C.J. Lin LIBLINEAR: A library for large linear classification Journal of Machine Learning Research 9 2008 1871 1874
24. M. Faruqui, S. Padó, Training and evaluating a German named entity recognizer with semantic generalization, in: Proceedings of Die Konferenz zur Verarbeitung Natürlicher Sprache (KONVENS) 2010, Saarbrücken, Germany, 2010.
25. E. Fernandes, and U. Brefeld Learning from partially annotated sequences Machine Learning and Knowledge Discovery in Databases Lecture Notes in Computer Science vol. 6911 2011 407 422
26. S. Ferrández, A. Toral, Ó. Ferrández, A. Ferrández, R. Muñoz, Applying Wikipedias multilingual knowledge to cross-lingual question answering, in: Proceedings of the 12th International Conference on Applications of Natural Language to Information Systems, Paris, France, 2007, pp. 352-363.
27. E. Filatova, Multilingual Wikipedia, summarization, and information trustworthiness, in: Proceedings of the SIGIR Workshop on Information Access in a Multilingual World, Boston, Massachusetts, 2009, pp. 19-24.
28. R. Fu, B. Qin, T. Liu, Generating Chinese named entity data from a parallel corpus, in: Proceedings of 5th International Joint Conference on Natural Language Processing, Chiang Mai, Thailand, 2011, pp. 264-272.
29. E. Gabrilovich, S. Markovitch, Overcoming the brittleness bottleneck using Wikipedia: Enhancing text categorization with encyclopedic knowledge, in: Proceedings of the 21st National Conference on Artificial Intelligence, Boston, Massachusetts, 2006, pp. 1301-1306. (Pubitemid 44705454)
30. M. Hu, E.P. Lim, A. Sun, H.W. Lauw, B.Q. Vuong, Measuring article quality in Wikipedia: Models and evaluation, in: Proceedings of the 16th ACM Conference on Information and Knowledge Management, Lisbon, Portugal, 2007, pp. 243-252.
31. J. Kazama, K. Torisawa, Exploiting Wikipedia as external knowledge for named entity recognition, in: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Prague, Czech Republic, 2007, pp. 698-707.
32. J. Kazama, K. Torisawa, Inducing gazetteers for named entity recognition by large-scale clustering of dependency relations, in: Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Columbus, Ohio, 2008, pp. 407-415.
33. T. Kiss, and J. Strunk Unsupervised multilingual sentence boundary detection Computational Linguistics 32 2006 485 525 (Pubitemid 44916187)
34. F. Laws, C. Scheible, H. Schütze, Active learning with amazon mechanical turk, in: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, Edinburgh, Scotland, UK, 2011, pp. 1546-1556.
35. N. Lawson, K. Eustice, M. Perkowitz, M. Yetisgen-Yildiz, Annotating large email datasets for named entity recognition with mechanical turk, in: Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazons Mechanical Turk, Los Angeles, 2010, pp. 71-79.
36. ACE (Automatic Content Extraction) English Annotation Guidelines for Entities, Linguistic Data Consortium, Version 5.6.1, http://www.ldc.upenn.edu/ Projects/ACE/docs/English-Entities-Guidelines-v5.6.1.pdf, 2005.
37. W. Liao, S. Veeramachaneni, A simple semi-supervised algorithm for named entity recognition, in: Proceedings of the NAACL HLT 2009 Workshop on Semi-supervised Learning for Natural Language Processing, Boulder, Colorado, 2009, pp. 58-65.
38. E. Loper, S. Bird, NLTK: The natural language toolkit, in: Proceedings of the ACL-02 Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics, Philadelphia, Pennsylvania, 2002, pp. 63-70.
39. X. Ma, Toward a name entity aligned bilingual corpus, in: Proceedings of the Workshop on Methods for the Automatic Acquisition of Language Resources and Their Evaluation Methods, Valletta, Malta, 2010.
40. 1, in: NLPers Blog, http://nlpers.blogspot.com, 25 August 2006.
41. M. Marcus, B. Santorini, and M. Marcinkiewicz Building a large annotated corpus of English: The Penn Treebank Computational Linguistics 19 1993 313 330
42. O. Medelyan, C. Legg, Integrating Cyc and Wikipedia: Folksonomy meets rigorously defined common-sense, in: Proceedings of the AAAI 2008 Workshop on Wikipedia and Artificial Intelligence: An Evolving Synergy, Chicago, Illinois, 2008.
43. G. de Melo, G. Weikum, MENTA: Inducing multilingual taxonomies from Wikipedia, in: Proceedings of the Nineteenth ACM Conference on Information and Knowledge Management, Toronto, Canada, 2010, pp. 1099-1108.
44. R. Merchant, M.E. Okurowski, N. Chinchor, The multilingual entity task (MET) overview, in: TIPSTER Text Program Phase II: Proceedings of a Workshop held at Vienna, Virginia, 1996, pp. 445-448.
45. R. Mihalcea, A. Csomai, Wikify!: Linking documents to encyclopedic knowledge, in: Proceedings of the 16th ACM Conference on Information and Knowledge Management, 2007, pp. 233-242.
46. P. Mika, M. Ciaramita, H. Zaragoza, and J. Atserias Learning to tag and tagging to learn: A case study on Wikipedia IEEE Intelligent Systems 23 2008 26 33
47. A. Mikheev, M. Moens, C. Grover, Named entity recognition without gazetteers, in: Proceedings of the Ninth Conference of European Chapter of the Association for Computational Linguistics, Bergen, Norway, 1999, pp. 1-8.
48. D. Nadeau, and S. Sekine A survey of named entity recognition and classification Lingvisticae Investigationes 30 2007 3 26
49. D. Nadeau, P.D. Turney, S. Matwin, Unsupervised named-entity recognition: Generating gazetteers and resolving ambiguity, in: Proceedings of the 19th Canadian Conference on Artificial Intelligence, 2006, pp. 266-277. (Pubitemid 44077472)
50. V. Nastase, M. Strube, B. Börschinger, C. Zirn, A. Elghafari, WikiNet: A very large scale multi-lingual concept network, in: Proceedings of the 7th International Conference on Language Resources and Evaluation, Valletta, Malta, 2010, pp. 19-21.
51. R. Navigli, S.P. Ponzetto, BabelNet: Building a very large multilingual semantic network, in: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, 2010, pp. 216-225.
52. C. Nobata, N. Collier, J. Tsuji, Comparison between tagged corpora for the named entity task, in: Proceedings of the Workshop on Comparing Corpora, Seattle, Washington, 2000, pp. 20-27.
53. E.W. Noreen Computer Intensive Methods for Testing Hypotheses 1989 John Wiley & Sons
54. J. Nothman, Learning named entity recognition from Wikipedia, Honours Thesis, School of IT, University of Sydney, 2008.
55. J. Nothman, J.R. Curran, T. Murphy, Transforming Wikipedia into named entity training data, in: Proceedings of the Australian Language Technology Workshop, Hobart, 2008, pp. 124-132.
56. J. Nothman, T. Murphy, J.R. Curran, Analysing Wikipedia and gold-standard corpora for NER training, in: Proceedings of the 12th Conference of the European Chapter of the ACL, Athens, Greece, 2009, pp. 612-620.
57. S.P. Ponzetto, R. Navigli, Large-scale taxonomy mapping for restructuring and integrating Wikipedia, in: Proceedings of the 21st International Joint Conference on Artificial Intelligence, Pasadena, California, 2009, pp. 2083-2088.
58. M. Potthast, B. Stein, M. Anderka, A Wikipedia-based multilingual retrieval model, in: Proceedings of Advances in Information Retrieval: 30th European Conference on IR Research, Glasgow, UK, 2008, pp. 522-530. (Pubitemid 351499100)
59. L. Ratinov, D. Roth, Design challenges and misconceptions in named entity recognition, in: Proceedings of the Thirteenth Conference on Computational Natural Language Learning, Boulder, Colorado, 2009, pp. 147-155.
60. A.E. Richman, personal communication, 2010.
61. A.E. Richman, P. Schone, Mining wiki resources for multilingual named entity recognition, in: 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Columbus, Ohio, 2008, pp. 1-9.
62. N. Ringland, J. Nothman, T. Murphy, J.R. Curran, Classifying articles in English and German Wikipedia, in: Proceedings of the Australasian Language Technology Association Workshop, Sydney, Australia, 2009, pp. 20-28.
63. M. Ruiz-Casado, E. Alfonseca, and P. Castells Automatic assignment of Wikipedia encyclopedic entries to WordNet synsets Advances in Web Intelligence Lecture Notes in Computer Science vol. 3528 2005 380 386
64. D. Samy, A. Moreno, J.M. Guirao, A proposal for an Arabic named entity tagger leveraging a parallel corpus, in: Proceedings of the International Conference Recent Advances in Natural Language Processing, Borovets, Bulgaria, 2005, pp. 459-465.
65. H. Schmid, Probabilistic part-of-speech tagging using decision trees, Technical report, Institut für maschinelle Sprachverarbeitung, Universität Stuttgart, 1994.
66. P. Schönhofen, Identifying document topics using the Wikipedia category network, in: Proceedings of the 2006 IEEE/WIC/ACM International Conference on Web Intelligence, Hong Kong, 2006, pp. 456-462.
67. P. Schönhofen, A. Benczúr, I. Bíró, and K. Csalogány Cross-language retrieval with Wikipedia Advances in Multilingual and Multimodal Information Retrieval Lecture Notes in Computer Science vol. 5152 2008 72 79
68. S. Sekine, K. Sudo, C. Nobata, Extended named entity hierarchy, in: Proceedings of the 3rd International Conference on Language Resources and Evaluation, Las Palmas, Canary Islands, 2002, pp. 1818-1824.
69. R. Shah, B. Lin, A. Gershman, R. Frederking, SYNERGY: A named entity recognition system for resource-scarce languages such as Swahili using online machine translation, in: Proceedings of the Second Workshop on African Language Technology, Valletta, Malta, 2010, pp. 21-26.
70. P. Sorg, P. Cimiano, Enriching the crosslingual link structure of Wikipedia: A classification-based approach, in: Proceedings of the AAAI 2008 Workshop on Wikipedia and Artificial Intelligence, Chicago, Illinois, 2008.
71. M. Strube, S.P. Ponzetto, WikiRelate! Computing semantic relatedness using Wikipedia, in: Proceedings of the 21st National Conference on Artificial Intelligence, Boston, Massachusetts, 2006, pp. 1419-1424. (Pubitemid 44705473)
72. F.M. Suchanek, G. Kasneci, G. Weikum, YAGO: A core of semantic knowledge - unifying WordNet and Wikipedia, in: Proceedings of the 16th International Conference on World Wide Web, Banff, Alberta, 2007, pp. 697-706. (Pubitemid 47582299)
73. F.M. Suchanek, G. Kasneci, and G. Weikum YAGO: A large ontology from Wikipedia and WordNet Web Semantics: Science, Services and Agents on the World Wide Web 6 2008 203 217
74. J. Suzuki, H. Isozaki, Semi-supervised sequential labeling and segmentation using giga-word scale unlabeled data, in: Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Columbus, Ohio, 2008, pp. 665-673.
75. S. Tardif, J.R. Curran, T. Murphy, Improved text categorisation for Wikipedia named entities, in: Proceedings of the Australian Language Technology Workshop, Sydney, Australia, 2009, pp. 104-108.
76. E.F. Tjong Kim Sang, Introduction to the CoNLL-2002 shared task: Language-independent named entity recognition, in: Proceedings of the 6th Conference on Natural Language Learning, Taipei, Taiwan, 2002, pp. 1-4.
77. E.F. Tjong Kim Sang, F. De Meulder, Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition, in: Proceedings of the 7th Conference on Natural Language Learning, Edmonton, Canada, 2003, pp. 142-147.
78. M. Tkatchenko, A. Ulanov, A. Simanovsky, Classifying Wikipedia entities into fine-grained classes, in: Proceedings of the IEEE 27th International Conference on Data Engineering Workshops, 2011, pp. 212-217.
79. A. Toral, S. Ferrández, M. Monachini, and R. Muñoz Web 2.0, language resources and standards to automatically build a multilingual named entity lexicon Language Resources and Evaluation 2011 1 37
80. A. Toral, R. Muñoz, A proposal to automatically build and maintain gazetteers for named entity recognition using Wikipedia, in: Proceedings of the Workshop on NEW TEXT, 11th Conference of the European Chapter of the Association for Computational Linguistics, Trento, Italy, 2006.
81. D. Urbansky, J. Thom, D. Schuster, and A. Schill Training a named entity recognizer on the web Web Information System Engineering - WISE 2011 Lecture Notes in Computer Science vol. 6997 2011 87 100
82. R. Voyer, V. Nygaard, W. Fitzgerald, H. Copperman, A hybrid model for annotating named entity training corpora, in: Proceedings of the Fourth Linguistic Annotation Workshop, 2010, pp. 243-246.
83. Y. Watanabe, M. Asahara, Y. Matsumoto, A graph-based approach to named entity categorization in Wikipedia using conditional random fields, in: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Prague, Czech Republic, 2007, pp. 649-657.
84. R. Weischedel, A. Brunstein, BBN pronoun coreference and entity type corpus, LDC2005T33, Linguistic Data Consortium, Philadelphia, 2005.
85. W. Wentland, J. Knopp, C. Silberer, M. Hartung, Building a multilingual lexical resource for named entity disambiguation, translation and transliteration, in: Proceedings of the 6th International Conference on Language Resources and Evaluation, Marrakech, Morocco, 2008, pp. 28-30.
86. Y. Wong, H.T. Ng, One class per named entity: Exploiting unlabeled text for named entity recognition, in: Proceedings of the 20th International Joint Conference on Artificial Intelligence, Hyderabad, India, 2007, pp. 1763-1768.
87. D. Wu, W.S. Lee, N. Ye, H.L. Chieu, Domain adaptive bootstrapping for named entity recognition, in: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, Singapore, 2009, pp. 1523-1532.
88. F. Wu, D.S. Weld, Autonomously semantifying Wikipedia, in: Proceedings of the 16th ACM Conference on Information and Knowledge Management, Lisbon, Portugal, 2007, pp. 41-50.
89. D. Yarowsky, G. Ngai, R. Wicentowski, Inducing multilingual text analysis tools via robust projection across aligned corpora, in: Proceedings of the First International Conference on Human Language Technology Research, San Francisco, 2001, pp. 109-116.