Algorithmic stemmers or morphological analysis? An evaluation

Journal of the American Society for Information Science and Technology

Volumn 60, Issue 8, 2009, Pages 1616-1624

  Fautsch, Claire ^a   Savoy, Jacques ^a  

^a UNIVERSITY OF NEUCHÂTEL   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

CLASS NUMBERS; CLASSIFICATION TASKS; CROSS-LANGUAGE EVALUATION FORUMS; ENGLISH LANGUAGES; INFORMATION EXTRACTION; IR MODELS; MORPHOLOGICAL ANALYSIS; MORPHOLOGICAL ANALYZER; RETRIEVAL PERFORMANCE; STOP WORD; TEST COLLECTION;

ALGORITHMS; INFORMATION RETRIEVAL; INFORMATION SERVICES; LINGUISTICS; MORPHOLOGY;

ALGORITHMIC LANGUAGES;

EID: 67650860199     PISSN: 15322882     EISSN: 15322890     Source Type: Journal    
DOI: 10.1002/asi.21093     Document Type: Article

References (31)

1. Amati, G., & van Rijsbergen, C.J. (2002). Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Transactions on Information Systems, 20, 357-389.
2. Buckley, C. (2007). Why current IR engines fail. In Proceedings of ACM- SIGIR'2004 (pp. 584-585). New York: ACM Press.
3. Buckley, C., & Voorhees, E.M. (2005). Retrieval system evaluation. In E.M. Voorhees & D.K. Harman (Eds.), TREC: Experiment and Evaluation in Information Retrieval (pp. 53-75). Cambridge, MA: MIT Press.
4. Chan, Y.S., Ng, H.T., & Zhong, Z. (2007). NUS-PT: Exploiting parallel texts for word sense disambiguation in the English all-words tasks. In Proceedings of the 4th International Workshop on Semantic Evaluations (pp. 253-256), Stroudsburg, PA: ACL.
5. Fellbaum, C. (1998). WordNet: An electronic lexical database. Cambridge, MA: MIT Press.
6. Fox, C. (1990). A stop list for general text. ACM-SIGIR Forum, 24, 19-35.
7. Harman, D. (1991). How effective is suffixing? Journal of the American Society for Information Science, 42, 7-15.
8. Harter, S.P. (1986). Online information retrieval. Concepts, principles, and techniques. San Diego, CA: Academic Press.
9. Hiemstra, D. (2000). Using language models for information retrieval. Unpublished doctoral dissertation, Centre for Telematics and Information
10. Technology, University of Twente, The Netherlands.
11. Hiemstra, D. (2002). Term-specific smoothing for the language modeling approach to information retrieval: The importance of a query term. In Proceedings of the ACM-SIGIR (pp. 35-41). New York: ACM Press.
12. Hull, D. (1996). Stemming algorithms: A case study for detailed evaluation. Journal of the American Society for Information Science, 47, 70-84.
13. Kettunen, K. (2007). Reductive and generative approaches to morphological variation of keywords in monolingual information retrieval. Unpublished doctoral dissertation, University of Tampere, Finland.
14. Krovetz, R. (1993). Viewing morphology as an inference process. In Proceedings of the ACM-SIGIR'93 (pp. 191-202). New York: ACM Press.
15. Lovins, J.B. (1968). Development of a stemming algorithm. Mechanical Translation and Computational Linguistics, 11, 22-31.
16. Manning, C.D., Raghavan, P., & Schütze, H. (2008). Introduction to information retrieval. Cambridge, England: Cambridge University Press.
17. Marcus, M.P., Santorini, B., & Marcinkiewicz, M.A. (1993). Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19, 313-330.
18. McNamee, P., & Mayfield, J. (2004). Character N-gram tokenization for European language text retrieval. Information Retrieval Journal, 7, 73-97.
19. Peng, F., Ahmed, N., Li, X., & Lu, Y. (2007). Context sensitive stemming for web search. In Proceedings of the ACM-SIGIR (pp. 639-646). New York: ACM Press.
20. Peters, C., Jijkoun, V., Mandl, T., Müller, H., Oard, D.W., Peñas, A., & Santos, D. (Eds.). (2008). Advances in multilingual and multimodal information retrieval. Lecture Notes in Computer Science (Vol. 5152). Berlin, Germany: Springer-Verlag.
21. Porter, M.F. (1980). An algorithm for suffix stripping. Program, 14, 130-137.
22. Ratnaparkhi, A. (1996). A maximum entropy part-of-speech tagger. In Proceedings of the Empirical Methods in Natural Language Processing Conference (pp. 133-142). Stroudsburg, PA: ACL.
23. Robertson, S.E., Walker, S., & Beaulieu, M. (2000). Experimentation as a way of life: Okapi at TREC Information Processing & Management, 36, 95-108.
24. Salton, G. (1971). The SMART retrieval system-Experiments in automatic document processing. Englewood Cliffs, NJ: Prentice-Hall.
25. Savoy, J. (1993). Stemming of French words based on grammatical category. Journal of the American Society for Information Science, 44, 1-9.
26. Savoy, J. (1997). Statistical inference in retrieval effectiveness evaluation. Information Processing & Management, 33, 495-512.
27. Savoy J. (2007). Why do successful search systems fail for some topics? In Proceedings of the ACM Symposium in Applied Computing (pp. 872-877). New York: ACM Press.
28. Tomlinson, S. (2004). Lexical and algorithmic stemming compared for 9 European languages with Hummingbird SearchServer at CLEF 2003. In C. Peters, J. Gonzalo, M. Braschler, & M. Kluck (Eds.), Comparative evaluation of multilingual information access systems (pp. 286-300). Lecture Notes in Computer Science (Vol. 3237). Berlin, Germany: Springer-Verlag.
29. Voorhees, E.M. (1993). Using WordNet to disambiguate word senses for text retrieval. In Proceedings of theACM-SIGIR'93 (pp. 171-180). New York: ACM Press.
30. Xu, J., & Croft, B. (1998). Corpus-based stemming using cooccurrence of word variants. ACM Transactions on Information Systems, 16, 61-81.
31. Zhai, C., & Lafferty, J. (2004). A study of smoothing methods for language models applied to information retrieval. ACM Transactions on Information Systems, 22, 179-214.