Comparison of ontology alignment systems across single matching task via the McNemar's test

ACM Transactions on Knowledge Discovery from Data

Volumn 12, Issue 4, 2018, Pages

  Mohammadi, Majid ^a   Atashin, Amir Ahooye ^b   Hofman, Wout ^c   Tan, Yaohua ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^b FERDOWSI UNIVERSITY OF MASHHAD   (Iran)

^c TNO   (Netherlands)

Author keywords

Anatomy; Family wise error rate; McNemar's test; OAEI; Ontology alignment

Indexed keywords

DIRECTED GRAPHS; ERRORS; SEMANTICS; STATISTICAL METHODS; TESTING;

ANATOMY; ERROR RATE; MCNEMAR'S TESTS; OAEI; ONTOLOGY ALIGNMENT;

ONTOLOGY;

EID: 85052596763     PISSN: 15564681     EISSN: 1556472X     Source Type: Journal    
DOI: 10.1145/3193573     Document Type: Article

References (47)

1. Manel Achichi, Michelle Cheatham, Zlatan Dragisic, Jérôme Euzenat, Daniel Faria, Alfio Ferrara, Giorgos Flouris, Irini Fundulaki, Ian Harrow, Valentina Ivanova, and others. 2016. Results of the ontology alignment evaluation initiative 2016. In Proceedings of the 11th ISWC Workshop on Ontology Matching (OM'16). No Commercial Editor, 73-129.
2. Siham Amrouch, Sihem Mostefai, and Muhammad Fahad. 2016. Decision trees in automatic ontology matching. International Journal of Metadata, Semantics and Ontologies 11, 3 (2016), 180-190.
3. Beate Bergmann and Gerhard Hommel. 1988. Improvements of general multiple test procedures for redundant systems of hypotheses. In Proceedings of the Multiple Hypothesenprüfung/Multiple Hypotheses Testing. Springer, 100-115.
4. Michelle Cheatham and Pascal Hitzler. 2013. String similarity metrics for ontology alignment. In Proceedings of the International Semantic Web Conference. Springer, 294-309.
5. William Cohen, Pradeep Ravikumar, and Stephen Fienberg. 2003. A comparison of string metrics for matching names and records. In Proceedings of the KDD Workshop on Data Cleaning and Object Consolidation, vol. 3, 73-78.
6. Jomar da Silva. 2016. ALIN results for OAEI 2016. In Proceedings of the Twelfth International Workshop on Ontology Matching (OM'17).
7. Janez Demšar. 2006. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7(Jan. 2006), 1-30.
8. Warith Eddine Djeddi and Mohammed Tarek Khadir. 2010. XMAP: A novel structural approach for alignment of OWL-full ontologies. In Proceedings of the International Conference on Machine and Web Intelligence (ICMWI'10). IEEE, 368-373.
9. Olive Jean Dunn. 1961. Multiple comparisons among means. Journal of the American Statistical Association 56, 293 (1961), 52-64.
10. Allen L. Edwards. 1948. Note on the correction for continuity in testing the significance of the difference between correlated proportions. Psychometrika 13, 3 (1948), 185-187.
11. Marc Ehrig and York Sure. 2004. Ontology mapping-an integrated approach. In Proceedings of the European Semantic Web Symposium. Springer, 76-91.
12. Jérôme Euzenat. 2007. Semantic precision and recall for ontology alignment evaluation. In Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI'07). 348-353.
13. Jérôme Euzenat, Pavel Shvaiko, and others. 2007. Ontology Matching, Vol. 18. Springer.
14. Morten W. Fagerland, Stian Lydersen, and Petter Laake. 2013. The McNemar test for binary matched-pairs data: Mid-p and asymptotic are better than exact conditional. BMC Medical Research Methodology 13, 1 (2013), 1.
15. Daniel Faria, Catia Pesquita, Emanuel Santos, Matteo Palmonari, Isabel F. Cruz, and Francisco M. Couto. 2013. The agree-mentmakerlight ontology matching system. In Proceedings of the OTM Confederated International Conferences“On the Move to Meaningful Internet Systems”. Springer, 527-541.
16. H. Finner. 1993. On a monotonicity problem in step-down multiple test procedures. Journal of the American Statistical Association 88, 423 (1993), 920-923.
17. Yosef Hochberg. 1988. A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75, 4 (1988), 800-802.
18. Burt S. Holland and Margaret DiPonzio Copenhaver. 1987. An improved sequentially rejective Bonferroni test procedure. Biometrics 43, 2 (1987), 417-423.
19. Sture Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics (1979), 65-70.
20. Sadaqat Jan, Maozhen Li, Hamed Al-Raweshidy, Alireza Mousavi, and Man Qi. 2012. Dealing with uncertain entities in ontology alignment using rough sets. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 42, 6 (2012), 1600-1612.
21. Matthew A. Jaro. 1995. Probabilistic linkage of large public health data files. Statistics in Medicine 14, 5-7 (1995), 491-498.
22. Ernesto Jiménez-Ruiz and Bernardo Cuenca Grau. 2011. Logmap: Logic-based and scalable ontology matching. In Proceedings of the International Semantic Web Conference. Springer, 273-288.
23. Grzegorz Kondrak. 2005. N-gram similarity and distance. In Proceedings of the International Symposium on String Processing and Information Retrieval. Springer, 115-126.
24. H. O. Lancaster. 1961. Significance tests in discrete distributions. Journal of the American Statistical Association 56, 294 (1961), 223-234.
25. Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady 10 (1966), 707-710.
26. Iti Mathur, Nisheeth Joshi, Hemant Darbari, and Ajai Kumar. 2014. Shiva: A framework for graph based ontology matching. arxiv:1403.7465.
27. Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12, 2 (1947), 153-157.
28. Imen Megdiche, Olivier Teste, and Cassia Trojahn. 2016. LPHOM results for OAEI 2016. Ontology Matching, 190.
29. George A. Miller. 1995. WordNet: A lexical database for english. Communications of the ACM 38, 11 (1995), 39-41.
30. Majid Mohammadi, Wout Hofman, and Yaohua Tan. 2018. A comparative study of ontology matching systems via inferential statistics. IEEE Transactions on Knowledge and Data Engineering. (In press).
31. James Munkres. 1957. Algorithms for the assignment and transportation problems. Journal of the Society for Industrial and Applied Mathematics 5, 1 (1957), 32-38.
32. Miklos Nagy, Maria Vargas-Vera, and Enrico Motta. 2006. Dssim-ontology mapping with uncertainty. In Proceedings of the 1st International Workshop on Ontology Matching (OM'06).
33. Saul B. Needleman and Christian D. Wunsch. 1970. A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of Molecular Biology 48, 3 (1970), 443-453.
34. P. Nemenyi. 1963. Distribution-Free Multiple Comparisons. Ph.D. Dissertation. Princeton University.
35. Dominique Ritze, Heiko Paulheim, and Kai Eckert. 2013. Evaluation measures for ontology matchers in supervised matching scenarios. In Proceedings of the International Semantic Web Conference. Springer, 392-407.
36. Juliet Popper Shaffer. 1986. Modified sequentially rejective multiple test procedures. Journal of the American Statistical Association 81, 395 (1986), 826-831.
37. Juliet Popper Shaffer. 1995. Multiple hypothesis testing. Annual Review of Psychology 46, 1 (1995), 561-584.
38. David J. Sheskin. 2003. Handbook of Parametric and Nonparametric Statistical Procedures. CRC Press.
39. Giorgos Stoilos, Giorgos Stamou, and Stefanos Kollias. 2005. A string metric for ontology alignment. In Proceedings of the International Semantic Web Conference. Springer, 624-637.
40. Samy Suissa and Jonathan J. Shuster. 1991. The 2 x 2 matched-pairs trial: Exact unconditional design and analysis. Biometrics (1991), 361-372.
41. Peng Wang and Baowen Xu. 2008. Lily: Ontology alignment results for OAEI 2008. In Proceedings of the 3rd International Conference on Ontology Matching, vol. 431. CEUR-WS. org, 167-175.
42. William E. Winkler. 1999. The state of record linkage and current research problems. In Statistical Research Division, US Census Bureau. Citeseer.
43. David H. Wolpert. 1996. The lack of a priori distinctions between learning algorithms. Neural Computation 8, 7 (1996), 1341-1390.
44. David H. Wolpert. 2012. What the no free lunch theorems really mean; how to improve search algorithms. Santa fe Institute Working Paper. 12.
45. David H. Wolpert and William G. Macready. 1997. No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation 1, 1 (1997), 67-82.
46. Xingsi Xue and Yuping Wang. 2015. Ontology alignment based on instance using NSGA-II. Journal of Information Science 41, 1 (2015), 58-70.
47. Mengyi Zhao and Songmao Zhang. 2016. FCA-Map results for OAEI 2016. Ontology Matching, 172.