Let's agree to disagree: On the evaluation of vocabulary alignment

KCAP 2011 - Proceedings of the 2011 Knowledge Capture Conference

Volumn , Issue , 2011, Pages 65-72

  Tordai, Anna ^a   Van Ossenbruggen, Jacco ^a   Schreiber, Guus ^a   Wielinga, Bob ^a  

^a VU UNIVERSITY AMSTERDAM   (Netherlands)

Author keywords

empirical study; inter rater agreement; manual evaluation; vocabulary alignment

Indexed keywords

EMPIRICAL STUDIES; EVALUATION GUIDELINES; GOLD STANDARDS; INTER-RATER AGREEMENT; MANUAL EVALUATION; MAPPING RELATION; NATURAL LANGUAGES; ONTOLOGY ALIGNMENT; THINK ALOUD; VOCABULARY ALIGNMENT;

EXPERIMENTS; MAPPING; ONTOLOGY; RATING;

ALIGNMENT;

EID: 79960270087     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1999676.1999689     Document Type: Conference Paper

References (16)

1. Guidelines for the Construction, Format, and Management of Monolingual Controlled Vocabularies. Technical report, National Information Standards Organization, 2005.
2. R. Artstein and M. Poesio. Inter-coder Agreement for Computational Linguistics. Computational Linguistics, 34:555-596, 2008.
3. P. Bailey, N. Craswell, I. Soboroff, P. Thomas, A. P. de Vries, and E. Yilmaz. Relevance Assessment: Are Judges Exchangeable and Does It Matter?. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR '08, pages 667-674. ACM, 2008.
4. J. Carletta. Assessing Agreement on Classification Tasks: The Kappa Statistic. Computational Linguistics, 22:249-254, June 1996.
5. W. J. Clancey. Situated Cognition: On Human Knowledge and Computer Representations. Cambridge University Press, New York, NY, USA, 1997.
6. J. Cohen. A Coefficient of Agreement For Nominal Scales. Educational and Psychological Measurement, 20(1):37-46, April 1960.
7. J. Euzenat. An Api for Ontology Alignment. In S. A. McIlraith, D. Plexousakis, and F. van Harmelen, editors, The Semantic Web - ISWC 2004, volume 3298 of Lecture Notes in Computer Science, pages 698-712. Springer Berlin / Heidelberg, 2004.
8. J. Euzenat, A. Ferrara, L. Hollink, A. Isaac, C. Joslyn, V. Malaisé, C. Meilicke, A. Nikolov, J. Pane, M. Sabou, F. Scharffe, P. Shvaiko, V. Spiliopoulos, H. Stuckenschmidt, O. Sváb-Zamazal, V. Svátek, C. Trojahn dos Santos, G. Vouros, and S. Wang. Results of the Ontology Alignment Evaluation Initiative 2009. In P. S. et al., editor, Proc. 4th ISWC workshop on ontology matching (OM), pages 73-126, 2009.
9. H. Halpin, P. J. Hayes, J. P. McCusker, D. L. McGuiness, and H. S. Thompson. When owl:sameAs isn't the Same: An Analysis of Identity in Linked Data. In Proceedings of the 9th International Semantic Web Conference (ISWC), November 2010.
10. A. Isaac, L. Van Der Meij, S. Schlobach, and S. Wang. An Empirical Study of Instance-based Ontology Matching. In ISWC'07/ASWC'07, pages 253-266, Berlin, Heidelberg, 2007. Springer-Verlag.
11. G. Lakoff. Women, Fire and Dangerous Things; What Categories Reveal About the Mind
12. J. R. Landis and G. G. Koch. The Measurement of Observer Agreement for Categorical Data. Biometrics, 33:159-174, 1977.
13. A. Miles and S. Bechhofer. SKOS Simple Knowledge Organization System Reference, August 2009.
14. J. R. Taylor. Linguistic Categorization. Oxford University Press, third edition, 2003.
15. A. Tordai, J. van Ossenbruggen, and G. Schreiber. Combining Vocabulary Alignment Techniques. In Y. Gil and N. F. Noy, editors, K-CAP, pages 25-32. ACM, 2009.
16. A. Tordai, J. van Ossenbruggen, G. Schreiber, and B. J. Wielinga. Aligning Large SKOS-like Vocabularies: Two Case Studies. In The Semantic Web: Research and Applications, 7th Extended Semantic Web Conference, ESWC 2010, Heraklion, Crete, Greece, May 30-June 3, 2010, Proceedings, Part I, volume 6088 of Lecture Notes in Computer Science, pages 198-212. Springer, 2010.