Frequent term distribution measures for dataset profiling

Proceedings of the 4th International Conference on Language Resources and Evaluation, LREC 2004

Volumn , Issue , 2004, Pages 1647-1650

  De Roeck, Anne ^a   Sarkar, Avik ^a   Garthwaite, Paul ^a  

^a OPEN UNIVERSITY   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

DISTRIBUTION PATTERNS; TERM DISTRIBUTIONS;

EID: 84862304065     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (17)

1. Barbu, C. and R. Mitkov (2001) Evaluation Tool for rule-based anaphora resolution methods. Proceedings ACL, pp34-41, Budapest.
2. Cavaglia, G. (2002) Measuring corpus homogeneity using a range of measures for inter-document distance. ITRI Report Series, ITRI-02-08, University of Brighton, UK.
3. Donaway, R.L., Drummey, K.W. and L.A. Mather (2000) A Comparison of Rankings produced by summarisation evaluation measures. Proceedings NAACL-ANLP Workshop on Text Summerisation.
4. De Roeck, A., Sarkar, A and Garthwaite, P. (2004a) Defeating the Homogeneity Assumption. Proceedings of JADT04.
5. De Roeck, A., Sarkar, A. and Garthwaite P. (2004b) Profiling Datasets with Very Frequent Term Distribution Measures. Computing Technical Report TR 2004/05. The Open University.
6. Dunning, T. (1993) Accurate Methods for the Statistics of Surprise and Co-incidence. Computational Linguistics, 19(1):61-74.
7. Donna Harman: (1991) How effective is suffixing? JASIS 42(1): 7-15
8. Jurafsky, D. and J.H. Martin (2000) Speech and Language Processing. Prentice Hall. New Jersey.
9. Katz, S. (1996) Distribution of content words and phrases in text and language modelling. Natural Language Engineering, 2(1):15-59.
10. Kilgariff, A. (1996) Which words are particularly characteristic of a text? A survey of statistical approaches. In Proceedings AISB Workshop on Language Engineering for Document Analysis and Recognition, pages 33-40.
11. Kilgariff, A. (1997) Using word frequency lists to measure corpus homogeneity and similarity between corpora. Proceedings ACL-SIGDAT Workshop on very large corpora, Hong Kong.
12. Kilgariff, A. and T. Rose (1998) Measures for Corpus Similarity and Homogeneity. Proc. 3rd Int. Conf. Empirical Methods in NLP. pp 46-52. Granada.
13. Krovetz, R. (1993) Viewing Morphology as an Inference Process. SIGIR-93, pp 191-202. ACM.
14. Manning, Christopher and Hinrich Schuetze. (1999) Foundations of statistical natural language processing. MIT Press, Cambridge, Mass.
15. Rose, T. and N. Haddock (1997) The effects of corpus size and homogeneity on language model quality. Proceedings ACL-SIGDAT workshop on very large corpora, pp178-191, Hong Kong.
16. Wilbur, J. and K. Sirotkin. 1992. The Automatic identification of Stop Words. Journal of Information Science. 18:45-55. Elsevier.
17. Yang, Yiming and J. Wilbur. 1996. Using Corpus Ststistics to Remove Redundant Words in Text Categorisation. JASIS. 47(5):357-369.