A Method to Quantify Corpus Similarity and its Application to Quantifying the Degree of Literality in a Document

International Journal of Technology and Human Interaction (IJTHI)

Volumn 2, Issue 1, 2006, Pages 51-66

  Denoual, Etienne ^a,b  

^a UNIV GRENOBLE ALPES   (France)

^b Department of Cardiovascular Science   (Japan)

Author keywords

corpus design; data management; document management systems; information processing; natural language processing; text database

Indexed keywords

EID: 85001720429     PISSN: 15483908     EISSN: 15483916     Source Type: Journal    
DOI: 10.4018/jthi.2006010104     Document Type: Article

References (13)

1. Biber, D. (1995). Dimensions in register variation. Cambridge: Cambridge University Press.
2. Biber, D. (1988). Variation across speech and writing. Cambridge: Cambridge University Press.
3. Cavaglià, G. (2002). Measuring corpus homogeneity using a range of measures for inter-document distance. In Proceedings of the LREC (pp. 426–431).
4. Charniak, E. (1993). Statistical language learning. Cambridge, MA: MIT Press.
5. Dunning, T. (1994a). Accurate methods for the statistics of surprise and coincidence. Computational Linguistics, 19(2), 219–241.
6. Dunning, T. (1994b). Statistical identification of language. New Mexico: CRL New Mexico State University.
7. Kilgarriff, A. (2001). Comparing corpora. International Journal of Corpus Linguistics, 6(1), 1–37.
8. Kilgarriff, A., & Rose, T. (1998). Measures for corpus similarity and homogeneity. In Proceedings of the 3rd Conference on Empirical Methods in Natural Language Processing (pp. 46–52), Granada, Spain.
9. Liebscher, R. A. (2003). New corpora, new tests, and new data for frequency-based corpus comparisons. Center for Research in Language Newsletter, 15(2).
10. Matsumoto, Y., et al. (2002). Morphological analysis system ChaSen version 2.2.9 mannual. Japan: Nara Institute of Science and Technology.
11. Nakamura, A., Matsunaga, S., Shimizu, T., Tonomura, M., & Sagisaka, Y. (1996). Japanese speech databases for robust speech recognition. In Proceedings of the ICSLP, Volume 4, 96, (pp. 2199–2202). Philadelphia, PA.
12. Sproat, R., & Emerson, T. (2003). The first international Chinese word segmentation bakeoff. In Proceedings of the Second SIGHAN Workshop on Chinese Language Processing, Sapporo, Japan.
13. Teahan, W. J., & Cleary, J. G. (1997). Applying compression to natural language processing. In Proceedings of the 6th Applied NLP Conference (ANLP-1997).