Research on the construction and filter method of stop-word list in text preprocessing

Proceedings - 4th International Conference on Intelligent Computation Technology and Automation, ICICTA 2011

Volumn 1, Issue , 2011, Pages 217-221

  Yao, Zhou ^a   Ze Wen, Cao ^a  

^a NATIONAL UNIVERSITY OF DEFENSE TECHNOLOGY   (China)

Author keywords

hash algorithm; stop word list; stopword filter; text mining; text preprocessing

Indexed keywords

HASH ALGORITHM; STOP-WORD LIST; STOPWORD FILTER; TEXT MINING; TEXT PREPROCESSING;

HASH FUNCTIONS;

FILTRATION;

EID: 79956003324     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICICTA.2011.64     Document Type: Conference Paper

References (17)

1. Feldman R, Dagan I. Knowledge Discovery in Textual Databases(KDT)[C].In: Proceedings of the 1st Annual Conference on Knowledge Discovery and Data Mining,1995:112-117.
2. Helena A., Oskari H., Mika K. and Inkeri V.A. Applying Data Mining Techniques in Text Analysis[J],1997(2):4-8.
3. Yang Y., Pedersen J O. A Comparative Study on Feature Selection in Text Categorization[A].Proceedings of ICML-97,14th International Conference on Machine Learning[C]. San Francisco: Morgan Kaufmann Publishers Inc.,1997.412-420.
4. Silva C, Ribeiro B. The Importance of Stop Word Removal on Recall Values in Text Categorization[J].Neural NetWorks,2003,3:20-24.
5. Luhn H.P. A Statistical Approach to Mechanized Encoding and Searching of Literary Information. IBM Journal of Research and Development, 1957, 1(4):309-317.
6. Luhn,H.P. The Automatic Creation of Literature Abstracts.IBM Journal, April, presented at IRE National Convention, New York, 1958, March 24.
7. VAN RIJSBERGEN, C. J. Information Retrieval[M]. London: Butterworths. 1975
8. FRANCIS, W., and KUCERA, H. Frequency Analysis of English Usage. Journal of English Linguistic. 1982,18(1):64-70.
9. Fox C. Lexical analysis and stop-lists. Information retrieval: Data structures and algorithms[M].Upper Saddle River, New Jersey: Prentice Hall,1992.
10. th Duth-belgian Information Retrieval Workshop. Utrecht, the Neterlands.2005,141-148.
11. Ljiljana D and Jacques S. When Stopword Lists Make the Difference. JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 2010, 61(1):200-203.
12. C.X.Cui. Research on the Effect of Stop Words Selection onText CategOrization[J].JOURNAL OF TAIYUAN NORMAL UNIVERSITY(Natural Science Edition),DEC,2008,VOL.7(4).pp.91-93(In Chinese)
13. Fox C. A Stop List for General Text. ACM-SIGIR Forum, 1990, VOL.24.pp. 19-35.
14. Y.J.Gu,X.Z.Fan,J.H.Wang,T.Wang,and W.J.Huang.Automatic Selection of Chinese Stoplist[J].Transaction of Beijing Institute of Technology.2005.VOL. 25(04).pp.337-340.(In Chinese )
15. Q.Q.Zhou,B.D.Sun and Y. Wang. Study on New Pretreatment Method for Chinese Text Classofication System[J].APPLICATION RESEARCH OF COMPUTER.2005.VOL.22(2).pp.85-86(In Chinese)
16. Blanchard A. Understanding and customizing stopword lists for enhanced patent mapping. World Patent Information, 2007, 29:308-316.
17. Brian W . Kernighan and Dennis M. Ritchie. The C Programming Language. Prentice-Hall. 1988: Chapter 6-Structures.