HPS: High precision stemmer

Information Processing and Management

Volumn 51, Issue 1, 2015, Pages 68-91

  Brychcín, Tomáš ^a   Konopík, Miloslav ^a  

^a UNIVERSITY OF WEST BOHEMIA   (Czech Republic)

Author keywords

Information retrieval; Language modeling; Maximum entropy; Maximum mutual information; Morphology; Stemming

Indexed keywords

MAXIMUM ENTROPY METHODS; MORPHOLOGY;

LANGUAGE MODEL; MAXIMUM MUTUAL INFORMATION; STEMMING;

INFORMATION RETRIEVAL;

EID: 84908102154     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipm.2014.08.006     Document Type: Article

References (36)

1. G. Amati, and C.J. Van Rijsbergen Probabilistic models of information retrieval based on measuring the divergence from randomness ACM Transactions on Information Systems 20 4 2002 357 389
2. G. Andrew, and J. Gao Scalable training of L1-regularized log-linear models Proceedings of the 24th international conference on machine learning 2007 ACM New York 33 40
3. M. Bacchin, N. Ferro, and M. Melucci A probabilistic model for stemmer generation Information Processing and Management 41 2005 121 137
4. L.R. Bahl, F. Jelinek, and R.L. Mercer A maximum likelihood approach to continuous speech recognition IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-5 2 1983 179 190
5. A.L. Berger, V.J.D. Pietra, and S.A.D. Pietra A maximum entropy approach to natural language processing Computational Linguistics 22 1996 39 71
6. P.F. Brown, P.V. deSouza, R.L. Mercer, V.J.D. Pietra, and J.C. Lai Class-based n-gram models of natural language Computational Linguistics 18 1992 467 479
7. Brychcín, T., & Konopík, M. (2011). Morphological based language models for inflectional languages. In Proceedings of IEEE international conference on intelligent data acquisition and advanced computing systems.
8. T. Brychcín, and M. Konopík Semantic spaces for improving language modeling Computer Speech & Language 28 1 2014 192 209
9. W.G. Charles Contextual correlates of meaning Applied Psycholinguistics 21 04 2000 505 524
10. Chen, S. F., & Goodman, J. T. (1998). An empirical study of smoothing techniques for language modeling. Tech. rep., Computer Science Group, Harvard University.
11. A.P. Dempster, N.M. Laird, and D.B. Rubin Maximum likelihood from incomplete data via the EM algorithm Journal of the Royal Statistical Society. Series B 39 1 1977 1 38
12. L. Dolamic, and J. Savoy Indexing and stemming approaches for the Czech language Information Processing and Management 45 2009 714 720
13. J. Goldsmith Unsupervised learning of the morphology of a natural language Computational Linguistics 27 2001 153 198
14. J. Goldsmith An algorithm for the unsupervised learning of morphology Natural Language Engineering 12 2006 353 371
15. I. Habernal, and T. Brychcín Semantic spaces for sentiment analysis I. Habernal, V. Matoušek, Text, speech, and dialogue Lecture notes in computer science Vol. 8082 2013 Springer Berlin Heidelberg 484 491
16. Habernal, I., Ptáček, T., Steinberger, J. (June 2013). Sentiment analysis in Czech social media using supervised machine learning. In Proceedings of the 4th workshop on computational approaches to subjectivity, sentiment and social media analysis (pp. 65-74). Association for Computational Linguistics, Atlanta, Georgia.
17. I. Habernal, T. Ptáček, and J. Steinberger Supervised sentiment analysis in czech social media Information Processing & Management 50 5 2014 693 707
18. H. Hammarström, and L. Borin Unsupervised learning of morphology Computational Linguistics 37 2011 309 350
19. R. Huddleston English grammar: An outline 1988 Cambridge University Press
20. D. Hull Using statistical testing in the evaluation of retrieval experiments Proceedings of the 16th annual international ACM SIGIR conference on research and development in information retrieval 1993 ACM New York 329 338
21. Koehn, P., & Hoang, H. (2007). Factored translation models. In EMNLP-CoNLL (pp. 868-876).
22. M. Konkol Brainy: A machine learning library L. Rutkowski, M. Korytkowski, R. Scherer, R. Tadeusiewicz, L.A. Zadeh, J.M. Zurada, Artificial intelligence and soft computing Lecture notes in computer science Vol. 8468 2014 Springer International Publishing 490 499
23. P. Kroeger Analyzing grammar 2005 Cambridge University Press
24. J.B. Lovins Development of a stemming algorithm Mechanical Translation and Computational Linguistics 11 1968 22 31
25. P. Majumder, M. Mitra, S.K. Parui, G. Kole, P. Mitra, and K. Datta YASS: Yet another suffix stripper ACM Transactions on Information Systems 2007 25
26. D.W. Oard, G. Levow, and C.I. Cabezas CLEF experiments at Maryland: Statistical stemming and backoff translation Revised papers from the workshop of cross-language evaluation forum on cross-language information retrieval and evaluation 2001 Springer-Verlag London 176 187
27. C.D. Paice An evaluation method for stemming algorithms Proceedings of the 17th annual international ACM SIGIR conference on research and development in information retrieval 1994 Springer-Verlag New York 42 50
28. J.H. Paik, M. Mitra, S.K. Parui, and K. Järvelin GRAS: An effective and efficient stemming algorithm for information retrieval ACM Transactions on Information Systems 29 2011 19:1 19:24
29. J.H. Paik, D. Pal, and S.K. Parui A novel corpus-based stemming algorithm using co-occurrence statistics Proceedings of the 34th international ACM SIGIR conference on research and development in information retrieval 2011 ACM New York 863 872
30. M.F. Porter An algorithm for suffix stripping Program 14 3 1980 130 137
31. H. Rubenstein, and J.B. Goodenough Contextual correlates of synonymy Communications of the ACM 8 10 1965 627 633
32. J. Savoy Searching strategies for the Hungarian language Information Processing & Management 44 1 2008 310 324
33. J. Steinberger, T. Brychcín, and M. Konkol Aspect-level sentiment analysis in czech Proceedings of the 5th workshop on computational approaches to subjectivity, sentiment and social media analysis 2014 Association for Computational Linguistics Baltimore, Maryland 24 30
34. M. Taulé, M.A Martí, and M. Recasens AnCora: Multilevel annotated corpora for Catalan and Spanish Proceedings of the 6th international conference on language resources and evaluation 2008 European Language Resources Association Marrakech, Morocco
35. S. Watanabe, T. Iwata, T. Hori, A. Sako, and Y. Ariki Topic tracking language model for speech recognition Computer Speech & Language 25 2011 440 461
36. J. Xu, and W.B. Croft Corpus-based stemming using cooccurrence of word variants ACM Transactions on Information Systems 16 1998 61 81