A novel method for stemmer generation based on Hidden Markov Models

International Conference on Information and Knowledge Management, Proceedings

Volumn , Issue , 2003, Pages 131-138

  Melucci, Massimo ^a   Orio, Nicola ^a  

^a UNIVERSITY OF PADOVA   (Italy)

Author keywords

Hidden Markov Models; HMM; Multi lingual Information Retrieval; Stemming; Unsupervised Training

Indexed keywords

ALGORITHMS; COMPUTATIONAL LINGUISTICS; MARKOV PROCESSES; MATHEMATICAL MODELS; STATISTICAL METHODS; WORD PROCESSING;

HIDDEN MARKOV MODELS (HMM); MULTI-LINGUAL INFORMATION RETRIEVAL; STEMMER GENERATION; UNSUPERVISED TRAINING;

INFORMATION RETRIEVAL;

EID: 18744402487     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/956863.956889     Document Type: Conference Paper

References (23)

1. J. Allan, editor. Challenges in Information Retrieval and Language Modeling, Report of a Workshop held at the Center for Intelligent Information Retrieval on September, 2002, University of Massachusetts, Amherst, MA, Published on April, 24th 2003 at http://ciir.cs.umass.edu/irchallenges/.
2. M. Bacchin, N. Ferro, and M. Melucci. The effectiveness of a graph-based algorithm for stemming. In Proceedings of the International Conference on Asian Digital Libraries, pages 117-128, Singapore, 2002.
3. M. Braschler and B. Ripplinger. Stemming and decompounding for German text retrieval. In Proceedings of the European Conference on Information Retrieval Research (ECIR), pages 177-192, Pisa, Italy, 2003.
4. G. Di Nunzio, N. Ferro, M. Melucci, and N. Orio The University of Padova at CLEF 2003: Experiments to evaluate probabilistic models for automatic stemmer generation and query word translation Working Notes of the Cross-Language Evaluation Forum Workshop, in press.
5. W.B. Frakes. Stemming algorithms. In W.B. Frakes and R. Baeza-Yates, editors, Information Retrieval-data structures and algorithms, chapter 8. Prentice Hall, Englewood Cliffs, NJ, 1992.
6. J. Goldsmith. Unsupervised learning of the morphology of a natural language. Computational Linguistics, 27(2):154-198, 2001.
7. D. Harman. How effective is suffixing. Journal of the American Society for Information Science, 42(1):7-15, 1991.
8. D.A. Hull. Stemming algorithms: A case study for detailed evaluation. Journal of the American Society for Information Science, 47(1):70-84, 1996.
9. W. Kraaij and R. Pohlmann. Viewing stemming as recall enhancement. In Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pages 40-48, Zürich, Switzerland, 1996.
10. R. Krovetz. Viewing Morphology as an Inference Process,. In Proceedings of the Sixteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1-203, 1993.
11. J. Lovins. Development of a stemming algorithm. Mechanical Translation and Computational Linguistics, 11:22-31, 1968.
12. Lucene. The Lucene Search Engine, http://jakarta.apache.org/lucene/docs/ index.html, 2003.
13. A.M. Mood, F.A. Graybill, and D.C. Boes. Introduction to the Theory of Statistics. McGraw-Hill, Inc., 1974.
14. C.D. Paice. Constructing literature abstract by computer: techniques and prospects. Information Processing & Management, 26(1):171-186, 1990.
15. M. Popovic and P. Willett. The effectiveness of stemming for natural language access to Slovene textual data. Journal of the American Society for Information Science, 43(5):384-390, 1992.
16. G. Salton. Automatic Text Processing Addison Wesley, 1989.
17. M. Porter. Snowball. http://www.snowball.tartarus.org/, May 2003.
18. M.F. Porter. An algorithm for suffix stripping. Program, 14(3):130-137, 1980.
19. L. Rabiner and B.H. Juang. Fundamentals of speech recognition Prentice Hall, Englewood Cliffs, NJ, 321-389, 1993.
20. G. Salton and M.J. McGill. Introduction to modern Information Retrieval. McGraw-Hill, New York, NY, 1983.
21. J. Savoy. A stemming procedure and stopword list for general French corpora. Journal of the American Society for Information Science, 50(10):944-952, 1999.
22. P. Sheridan and J.P. Ballerini. Experiments in multilingual information retrieval using the SPIDER system. In Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pages 58-65, Zürich, Switzerland, 1996.
23. J. Xu and W.B. Croft. Corpus-Based Stemming Using Cooccurrence of Word Variants. ACM Transactions on Information Systems, 16(1):61-81, January 1998.