Offspring from reproduction problems: What replication failure teaches us

ACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

Volumn 1, Issue , 2013, Pages 1691-1701

  Fokkens, Antske ^a   Van Erp, Marieke ^a   Postma, Marten ^b   Pedersen, Ted ^c   Vossen, Piek ^a   Freire, Nuno ^d  

^a VU UNIVERSITY AMSTERDAM   (Netherlands)

^b UTRECHT UNIVERSITY   (Netherlands)

^c UNIVERSITY OF MINNESOTA DULUTH   (United States)

^d European Library   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS;

RESEARCH PAPERS; RESEARCH QUESTIONS; SYSTEMATIC TESTING;

EXPERIMENTS;

EID: 84901646141     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (35)

1. Stanjeev Banerjee and Ted Pedersen. 2003. Extended gloss overlaps as a measure of semantic relatedness. In Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, pages 805-810, Acapulco, August.
2. Daniel M. Bikel. 2004. Intricacies of Collins' parsing model. Computational Linguistics, 30(4):479-511.
3. Tomasz Buchert and Lucas Nussbaum. 2012. Leveraging business workflows in distributed systems research for the orchestration of reproducible and scalable experiments. In Anne Etien, editor, 9ème édition de la conférence MAnifestation des JE-unes Chercheurs en Sciences et Technologies de l'Information et de la Communication - MajecSTIC 2012 (2012).
4. Alexander Budanitsky and Graeme Hirst. 2006. Evaluating WordNet-based measures of lexical semantic relatedness. Computational Linguistics, 32(1):13-47.
5. Michael Collins. 1999. Head-Driven Statistical Models for Natural Language Parsing. Phd dissertation, University of Pennsylvania.
6. Irene Cramer. 2008. How well do semantic relatedness measures perform? a meta-study. In Semantics in Text Processing. STEP 2008 Conference Proceedings, volume 1, pages 59-70.
7. Olivier Dalle. 2012. On reproducibility and traceability of simulations. In WSC-Winter Simulation Conference-2012.
8. Chris Drummond. 2009. Replicability is not reproducibility: nor is it good science. In Proceedings of the Twenty-Sixth International Conference on Machine Learning: Workshop on Evaluation Methods for Machine Learning IV.
9. Jenny Finkel, Trond Grenager, and Christopher D. Manning. 2005. Incorporating non-local information into information extraction systems by Gibbs sampling. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 363-370, Ann Arbor, USA.
10. Nuno Freire, José Borbinha, and Pável Calado. 2012. An approach for named entity recognition in poorly structured data. In Proceedings of ESWC 2012.
11. Graeme Hirst and David St-Onge. 1998. Lexical chains as representations of context for the detection and correction of malapropisms. In C. Fellbaum, editor, WordNet: An electronic lexical database, pages 305-332. MIT Press.
12. James Howison and James D. Herbsleb. 2013. Sharing the spoils: incentives and collaboration in scientific software development. In Proceedings of the 2013 conference on Computer Supported Cooperative Work, pages 459-470.
13. Darrel C. Ince, Leslie Hatton, and John Graham-Cumming. 2012. The case for open computer programs. Nature, 482(7386):485-488.
14. Jay J. Jiang and David W. Conrath. 1997. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of the International Conference on Research in Computational Linguistics (RO-CLING X), pages 19-33, Taiwan.
15. Maurice Kendall. 1938. A new measure of rank correlation. Biometrika, 30(1-2):81-93.
16. Claudia Leacock and Martin Chodorow. 1998. Combining local context and WordNet similarity for word sense identification. In C. Fellbaum, editor, WordNet: An electronic lexical database, pages 265-283. MIT Press.
17. Dekang Lin. 1998. An information-theoretic definition of similarity. In Proceedings of the 15th International Conference on Machine Learning, pages 296-304, Madison, USA.
18. Panos Louridas and Georgios Gousios. 2012. A note on rigour and replicability. SIGSOFT Softw. Eng. Notes, 37(5):1-4.
19. Andrew K. McCallum. 2002. MALLET: A machine learning for language toolkit. http://mallet. cs.umass.edu.
20. Thilo Mende. 2010. Replication of defect prediction studies: problems, pitfalls and recommendations. In Proceedings of the 6th International Conference on Predictive Models in Software Engineering. ACM.
21. Lingling Meng, Runqing Huang, and Junzhong Gu. 2013. A review of semantic similarity measures in wordnet. International Journal of Hybrid Information Technology, 6(1):1-12.
22. George A. Miller and Walter G. Charles. 1991. Contextual correlates of semantic similarity. Language and Cognitive Processes, 6(1):1-28.
23. Cameron Neylon, Jan Aerts, C Titus Brown, Simon J Coles, Les Hatton, Daniel Lemire, K Jarrod Millman, Peter Murray-Rust, Fernando Perez, Neil Saunders, Nigam Shah, Arfon Smith, Gaël Varoquaux, and Egon Willighagen. 2012. Changing computational research. the challenges ahead. Source Code for Biology and Medicine, 7 (2).
24. Siddharth Patwardhan and Ted Pedersen. 2006. Using wordnet based context vectors to estimate the semantic relatedness of concepts. In Proceedings of the EACL 2006 Workshop Making Sense of Sense - Bringing Computational Linguistics and Psycholinguistics Together, pages 1-8, Trento, Italy.
25. Ted Pedersen. 2008. Empiricism is not a matter of faith. Computational Linguistics, 34(3):465-470.
26. Ted Pedersen. 2010. Information content measures of semantic similarity perform better without sensetagged text. In Proceedings of the 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT 2010), pages 329-332, Los Angeles, USA.
27. Troy Raeder, T. Ryan Hoens, and Nitesh V. Chawla. 2010. Consequences of variability in classifier performance estimates. In Proceedings of ICDM'2010.
28. Philip Resnik. 1995. Using information content to evaluate semantic similarity in a taxonomy. In Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI), pages 448-453, Montreal, Canada.
29. Herbert Rubenstein and John B. Goodenough. 1965. Contextual correlates of synonymy. Communications of the ACM, 8(10):627-633.
30. Wheeler Ruml. 2010. The logic of benchmarking: A case against state-of-the-art performance. In Proceedings of the Third Annual Symposium on Combinatorial Search (SOCS-10).
31. Charles Spearman. 1904. Proof and measurement of association between two things. American Journal of Psychology, 15:72-101.
32. Marieke Van Erp and Lourens Van Der Meij. 2013. Reusable research? a case study in named entity recognition. CLTL 2013-01, Computational Lexicology & Terminology Lab, VU University Amsterdam.
33. Joaquin Vanschoren, Hendrik Blockeel, Bernhard Pfahringer, and Geoffrey Holmes. 2012. Experiment databases. Machine Learning, 87(2):127-158.
34. Piek Vossen, Isa Maks, Roxane Segers, Hennie Van Der Vliet, Marie-Francine Moens, Katja Hofmann, Erik Tjong Kim Sang, and Maarten De Rijke. 2013. Cornetto: a Combinatorial Lexical Semantic Database for Dutch. In Peter Spyns and Jan Odijk, editors, Essential Speech and Language Technology for Dutch Results by the STEVIN-programme, number XVII in Theory and Applications of Natural Language Processing, chapter 10. Springer.
35. Zhibiao Wu and Martha Palmer. 1994. Verb semantics and lexical selection. In Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics, pages 133-138, Las Cruces, USA.