Words and echoes: Assessing and mitigating the non-randomness problem in word frequency distribution modeling

ACL 2007 - Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics

Volumn , Issue , 2007, Pages 904-911

  Baroni, Marco ^a   Evert, Stefan ^b  

^a UNIVERSITY OF TRENTO   (Italy)

^b UNIVERSITY OF OSNABRÜCK   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

CORRECTION TECHNIQUES; CROSS VALIDATION; FREQUENCY DISTRIBUTIONS; NON-RANDOMNESS; OVER FITTING PROBLEM; PRE-PROCESSING METHOD; PREDICTION ACCURACY; RANDOM SAMPLING; RIGOROUS EVALUATION; TEST DATA; WORD FREQUENCIES;

COMPUTATIONAL LINGUISTICS;

EID: 84859999046     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (6)

1. Baayen, Harald. 1992. Quantitative aspects of morphological productivity. Yearbook of Morphology 1991, 109-150.
2. Baayen, Harald. 2001. Word frequency distributions. Dordrecht: Kluwer.
3. Church, KennethW. andWilliam A. Gale. 1995. Poisson mixtures. Journal of Natural Language Engineering 1, 163-190.
4. Evert, Stefan. 2004. A simple LNRE model for random character sequences. Proceedings of JADT 2004, 411-422.
5. Evert, Stefan and Marco Baroni. 2006. Testing the extrapolation quality of word frequency models. Proceedings of Corpus Linguistics 2005.
6. Katz, Slava M. 1996. Distribution of content words and phrases in text and language modeling. Natural Language Engineering, 2(2) 15-59.