Stemming via distribution-based word segregation for classification and retrieval

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 37, Issue 2, 2007, Pages 350-360

  Bhamidipati, Narayan L ^a   Pal, Sankar K ^a  

^a INDIAN STATISTICAL INSTITUTE   (India)

Author keywords

Precision and recall; Prototype selection; Stemming; Text categorization

Indexed keywords

ALGORITHMS; CLASSIFICATION (OF INFORMATION); DATA STRUCTURES; INFORMATION RETRIEVAL; TEXT PROCESSING;

PRECISION AND RECALL; PROTOTYPE SELECTION; TEXT CATEGORIZATION;

WORD PROCESSING;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; EVALUATION; INFORMATION RETRIEVAL; LINGUISTICS; METHODOLOGY; NATURAL LANGUAGE PROCESSING;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; INFORMATION STORAGE AND RETRIEVAL; NATURAL LANGUAGE PROCESSING; PATTERN RECOGNITION, AUTOMATED; VOCABULARY, CONTROLLED;

EID: 34047191837     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2006.885307     Document Type: Article

References (39)

1. G. Salton, "Developments in automatic text retrieval," Science, vol. 253, no. 5023, pp. 974-980, Aug. 1991.
2. W. Kraaij and R. Pohlmann, "Viewing stemming as recall enhancement," in Proc. l7th ACM SIGIR Conf., Zurich, Switzerland, Aug. 1996, pp. 40-48.
3. W. B. Frakes and C. J. Fox, "Strength and similarity of affix removal stemming algorithms," ACM SIGIR Forum, vol. 37, no. 1, pp. 26-30, 2003.
4. F. Yamout, R. Demachkieh, G. Hamdan, and R. Sabra, "Further enhancement to Porter algorithm," in Proc. Workshop Mach. Learn. and Interact. Text-Based Inf. Retrieval, Ulm, Germany,Sep. 2004, pp. 7-24.
5. N. L. Johnson, S. Kotz, and N. Balakrishnan, Discrete Multivariate Distributions. Hoboken, NJ: Wiley-Interscience, 1997.
6. M. F. Porter, "An algorithm for suffix stripping," Program, vol. 14, no. 3, pp. 130-137, 1980.
7. J. Xu and W. B. Croft, "Corpus-based stermning using coocurrence of word variants," ACM Trans. Inf. Syst., vol. 16, no. 1, pp. 61-81, 1998.
8. L. D. Baker and A. K. McCallum, "Distributional clustering of words for text classification," in Proc. 21stACMSIGIR Conf., Melbourne, Australia, 1998, pp. 96-103.
9. G. Salton, A. Wong, and C. S. Yang, "A vector space model for automatic indexing," Commun. ACM, vol. 18, no. 11, pp. 613-620, Nov. 1975.
10. D. Harman, "How effective is suffixing?" J. Amer. Soc. Inf. Sci., vol. 42, no. 1, pp. 7-15.
11. J. B. Lovins, "Development of a stemming algorithm," Mech. Transl. Comput. Linguist., vol. 11, no. 1/2, pp. 22-31, 1968.
12. J. L. Dawson, "Suffix removal for word conflation," Bull. Assoc. Lit. Linguist. Comput., vol. 2, no. 3, pp. 33-46, 1974.
13. C. D. Paice, "Another stemmer," SIGIR Forum, vol. 24, no. 3, pp. 56-61, 1990.
14. R. Krovetz, "Viewing morphology as an inference process," in Proc. l6th ACM SIGIR Conf., Pittsburgh, PA, 1993, pp. 191-202.
15. M. Kantrowitz, B. Mohit, and V. Mittal, "Stemming and its effects on TFIDF ranking," in Proc. 23rd Annu. SIGIR Conf., Athens, Greece, 2000, pp. 357-359.
16. T. Gustad and G. Bouma, "Accurate stemming of Dutch for text classification, " Lang. Comput., vol. 45, no. 1, pp. 104-117, 2002.
17. M. Bacchin, N. Ferro, and M. Melucci, "A probabilistic model for stemmer generation," Inf. Process. Manage., vol. 41, no. 1, pp. 121-137, 2005.
18. F. Pereira, N. Tishby, and L. Lee, "Distributional clustering of English words," in Proc. 31st Annu. Meeting Assoc. Comput. Linguist., 1993, pp. 183-190.
19. D. Lin, "Automatic retrieval and clustering of similar words," in Proc. l7th Int. Conf. Comput. Linguist., 1998, pp. 768-774.
20. L. Lee, "Measures of distributional similarity," in Proc. 37th Annu. Meeting Assoc. Comput. Linguist., 1999, pp. 25-32.
21. Y. Yang, "A re-examination of text categorization methods," in Proc. 22nd ACM SIGIR Conf., Berkeley, CA, 1999, pp. 42-49.
22. E. Riloff, "Little words can make a big difference for text classification," in Proc. l8th ACM SIGIR Conf., Seattle, WA, 1995, pp. 130-136.
23. M. Spitters, "Comparing feature sets for learning text categorization," in Proc. RIAO, Paris, France, 2000, pp. 1124-1135.
24. A. M. Cohen, J. Yang, and W. R. Hersh, "A comparison of techniques for classification and ad hoc retrieval of biomedical documents," in Proc. l4th Annu. Text REtrieval Conf., Gaithersburg, MD, 2005. [Online]. Available: http://trec.nist.gov/pubs/trec14/papers/ohsu-geo.pdf
25. A. Wald, Sequential Analysis. New York: Wiley, 1947.
26. G. Salton and M. J. McGill, Introduction to Modem Information Retrieval. New York: McGraw-Hill, 1983.
27. S. Kullback, Information Theory and Statistics. New York: Wiley, 1959.
28. E. L. Lehmann, Testing Statistical Hypotheses. New York: Springer-Verlag, 1997.
29. The 20 News Groups (20NG) data set. [Online]. Available: http://kdd.ics.uci.edu/databases/20newsgroups/20newsgroups.html
30. The WebKB data set. [Online]. Available: http://www-2.cs.cmu.edu/afs/ cs.cmu.edu/project/theo-20/www/data/
31. D. Harman, "Overview of the Third Text Retrieval Conference," in Proc. 3rd TREC-3, 1995, pp. 1-20.
32. C. D. Paice, "Method for evaluation of stemming algorithms based on error counting," J. Amer. Soc. Inf. Sci., vol. 47, no. 8, pp. 632-649, 1996.
33. R. S. de Madariaga, J. R. F. del Castillo, and J. R. Hilera, "A generalization of the method for evaluation of stemming algorithms based on error counting," in Proc. l2th Int. Conf. String Process. and Inf. Retrieval, Buenos Aires, Argentina, 2005, pp. 228-233.
34. A. K. McCallum, Bow: A Toolkit for Statistical Language Modeling, Text Retrieval, Classification and Clustering, 1996.[Online]. Available: http://www.cs.cmu.edu/~mccallum/bow
35. D. D. Lewis, "Naive (Bayes) at forty: The independence assumption in information retrieval," in Proc. l0th Eur Conf. Mach.Learn., Chemnitz, Germany, 1998, pp. 4-15.
36. V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
37. T. Joachims, "Text categorization with support vector machines: Learning with many relevant features," in Proc. l0th Eur. Conf. Mach. Learn., Chemnitz, Germany, 1998, pp. 137-142.
38. K. Nigam, J. Lafferty, and A. McCallum, "Using maximum entropy for text classification," in Proc. lJCAI Workshop Mach. Learn. Inf. Filtering, 1999, pp. 61-67.
39. S. Chakrabarti, Mining the Web: Discovering Knowledge From Hypertext Data. San Mateo, CA: Morgan Kaufmann, 2002.