What’s in a p-value in NLP?

CoNLL 2014 - 18th Conference on Computational Natural Language Learning, Proceedings

Volumn , Issue , 2014, Pages 1-10

  Søgaard, Anders ^a   Johannsen, Anders ^a   Plank, Barbara ^a   Hovy, Dirk ^a   Martinez, Hector ^a  

^a UNIVERSITY OF COPENHAGEN   (Denmark)

Author keywords

[No Author keywords available]

Indexed keywords

NATURAL LANGUAGE PROCESSING SYSTEMS;

COVARIATES; FALSE POSITIVE; P-VALUES; RESEARCH STANDARDS; SAMPLE SIZES; SELECTION BIAS; SENTENCE LENGTH; STANDARD METRICS;

RISK PERCEPTION;

EID: 85000184427     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/w14-1601     Document Type: Conference Paper

References (39)

1. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: an automatic metric for MT evaluation with improved correlation with human judgments. In ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization.
2. Taylor Berg-Kirkpatrick, David Burkett, and Dan Klein. 2012. An empirical investigation of statistical significance in nlp. In EMNLP.
3. John Blitzer, Ryan McDonald, and Fernando Pereira. 2006. Domain adaptation with structural correspondence learning. In EMNLP.
4. Rens Bod. 2000. Parsing with the shortest derivation. In COLING.
5. Sabine Buchholz and Erwin Marsi. 2006. CoNLL-X Shared Task on Multilingual Dependency Parsing. In CoNLL.
6. Katherine Button, John Ioannidis, Claire Mokrysz, Brian Nosek, Jonathan Flint, Emma Robinson, and Marcus Munafo. 2013. Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14:365–376.
7. Wenliang Chen, Youzheng Wu, and Hitoshi Isahara. 2008. Learning Reliable Information for Dependency Parsing Adaptation. In COLING.
8. Hai Leong Chieu and Hwee Tou Ng. 2003. Named entity recognition with a maximum entropy approach. In CoNLL.
9. Jonathan H. Clark, Chris Dyer, Alon Lavie, and Noah A. Smith. 2011. Better hypothesis testing for statistical machine translation: controlling for optimizer instability. In ACL.
10. Hal Daume III. 2007. Frustratingly easy domain adaptation. In ACL.
11. Janez Demsar. 2006. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7:1–30.
12. Pascal Denis and Benoît Sagot. 2009. Coupling an annotated corpus and a morphosyntactic lexicon for state-of-the-art pos tagging with less human effort. In PACLIC.
13. Leon Derczynski and Kalina Bontcheva. 2014. Passive-aggressive sequence labeling with discriminative post-editing for recognising person entities in tweets. In EACL.
14. Bradley Efron and Robert Tibshirani. 1993. An introduction to the bootstrap. Chapman & Hall, Boca Raton, FL.
15. Jakob Elming, Anders Johannsen, Sigrid Klerke, Emanuele Lapponi, Hector Martinez Alonso, and Anders Søgaard. 2013. Down-stream effects of tree-to-dependency conversions. In NAACL.
16. Radu Florian, Abe Ittycheriah, Hongyan Jing, and Tong Zhang. 2003. Named entity recognition through classifier combination. In CoNLL.
17. Jennifer Foster, Ozlem Cetinoglu, Joachim Wagner, Josef Le Roux, Joakim Nivre, Deirde Hogan, and Josef van Genabith. 2011. From news to comments: Resources and benchmarks for parsing the language of Web 2.0. In IJCNLP.
18. John Ioannidis. 2005. Why most published research findings are false. PLoS Medicine, 2(8):696–701.
19. Jing Jiang and ChengXiang Zhai. 2007. Instance weighting for domain adaptation in NLP. In ACL.
20. Chin-Yew Lin. 2004. ROUGE: a package for automatic evaluation of summaries. In WAS.
21. Chris Manning. 2011. Part-of-speech tagging from 97% to 100%: Is it time for some linguistics? In CICLing.
22. Joakim Nivre, Johan Hall, Sandra Kübler, Ryan McDonald, Jens Nilsson, Sebastian Riedel, and Deniz Yuret. 2007a. The CoNLL 2007 Shared Task on Dependency Parsing. In EMNLP-CoNLL.
23. Joakim Nivre, Johan Hall, Jens Nilsson, Atanas Chanev, Gülsen Eryigit, Sandra Kübler, Svetoslav Marinov, and Erwin Marsi. 2007b. MaltParser: a language-independent system for data-driven dependency parsing. Natural Language Engineering, 13(2):95–135.
24. Eric Noreen. 1989. Computer intensive methods for testing hypotheses. Wiley.
25. Kishore Papineni, Salim Roukus, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In ACL, pages 311–318, Philadelphia, Pennsylvania.
26. Barbara Plank and Gertjan van Noord. 2011. Effective measures of domain similarity for parsing. In ACL.
27. Peter Rankel, John Conroy, Eric Slud, and Dianne O’Leary. 2011. Ranking human and machine summarization systems. In EMNLP.
28. Stefan Riezler and John Maxwell. 2005. On some pitfalls in automatic evaluation and significance testing for MT. In ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization.
29. Laura Rimell, Stephen Clark, and Mark Steedman. 2009. Unbounded dependency recovery for parser evaluation. In EMNLP.
30. Roy Schwartz, and Omri Abend, Roi Reichart, and Ari Rappoport. 2011. Neutralizing linguistically problematic annotations in unsupervised dependency parsing evaluation. In ACL.
31. Joseph Simmons, Leif Nelson, and Uri Simonsohn. 2011. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11):1359–1366.
32. Richard Socher, John Bauer, Chris Manning, and Andrew Ng. 2013. Parsing with compositional vector grammars. In ACL.
33. Anders Søgaard. 2013. Estimating effect size across datasets. In NAACL.
34. Erik F Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. In In CoNLL.
35. Kristina Toutanova, Dan Klein, Chris Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In NAACL.
36. Reut Tsarfaty, Joakim Nivre, and Evelina Andersson. 2012. Cross-framework evaluation for statistical parsing. In EACL.
37. Yoshimasa Tsuruoka, Yusuke Miyao, and Jun’ichi Kazama. 2011. Learning with lookahead: can history-based models rival globally optimized models? In CoNLL.
38. Shulamit Umansky-Pesin, Roi Reichart, and Ari Rappoport. 2010. A multi-domain web-based algorithm for POS tagging of unknown words. In COLING.
39. Alexander Yeh. 2000. More accurate tests for the statistical significance of result differences. In ACL.