An automated domain specific stop word generation method for natural language text classification

INISTA 2011 - 2011 International Symposium on INnovations in Intelligent SysTems and Applications

Volumn , Issue , 2011, Pages 500-503

  Ayral, Hakan ^a   Yavuz, Sirma ^a  

^a YILDIZ TECHNICAL UNIVERSITY   (Turkey)

Author keywords

natural language processing; stop word; text mining

Indexed keywords

AUTOMATED METHODS; BAG OF WORDS; DOMAIN SPECIFIC; ENGLISH LANGUAGES; GENERATION METHOD; MAXIMUM A POSTERIORI PROBABILITIES; NATURAL LANGUAGE PROCESSING; NATURAL LANGUAGE TEXT; NATURAL LANGUAGES; STOP WORD; TEXT MINING; WORD FREQUENCIES; ZIPF'S LAW;

COMPUTATIONAL LINGUISTICS; INTELLIGENT SYSTEMS; PROBABILITY DISTRIBUTIONS; STATISTICAL TESTS; TEXT PROCESSING; USER INTERFACES;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 79961176846     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/INISTA.2011.5946149     Document Type: Conference Paper

References (14)

1. C. Silva and B. Ribeiro, "The importance of stop word removal on recall values in text categorization," Proceedings of the International Joint Conference on Neural Networks, 2003., pp. 1661-1666. [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1223656
2. T. K. Ho, "Stop word location and identification for adaptive text recognition," International Journal on Document Analysis and Recognition, vol. 3, no. 1, pp. 16-26, Aug. 2000. [Online]. Available: http://www. springerlink.com/index/10.1007/PL00013551
3. -, "Fast identification of stop words for font learning and keyword spotting," in icdar. Published by the IEEE Computer Society, 1999, p. 333. [Online]. Available: http://www.computer.org/portal/web/csdl/doi/10.1109/ICDAR. 1999.791792
4. E. C. Dragut, F. Fang, A. P. Sistla, C. T. Yu, and W. Meng, "Stop word and related problems in web interface integration," PVLDB, vol. 2, no. 1, pp. 349-360, 2009. [Online]. Available: http://www.vldb.org/pvldb/2/vldb09- 384.pdf
5. Z. Yao and C. Ze-wen, "Research on the Construction and Filter Method of Stop-word List in Text Preprocessing," 2011 Fourth International Conference on Intelligent Computation Technology and Automation, pp. 217-221, Mar. 2011. [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/ wrapper.htm?arnumber=5750595
6. L. Hao and L. Hao, "Automatic identification of stop words in chinese text classification," in Proceedings of the 2008 International Conference on Computer Science and Software Engineering - Volume 01. Washington, DC, USA: IEEE Computer Society, 2008, pp. 718-722.
7. F. Zou, F. Wang, X. Deng, and S. Han, "Automatic identification of Chinese stop words," Research on Computing Science, vol. 18, pp. 151-162, 2006. [Online]. Available: http://site.cicling.org/2006/Proceedings/RCS-18- Page151.pdf
8. G. Zheng and G. Gaowa, "The selection of Mongolian stop words," in Intelligent Computing and Intelligent Systems (ICIS), 2010 IEEE International Conference on, vol. 2. IEEE, 2010, pp. 71-74. [Online]. Available: http://ieeexplore.ieee.org/xpls/abs all.jsp?arnumber=5658841
9. R. Al-Shalabi, G. Kanaan, J. Jaam, a. Hasnah, and E. Hilat, "Stop-word removal algorithm for arabic language," Proceedings. 2004 International Conference on Information and Communication Technologies: From Theory to Applications, 2004., pp. 545-545. [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1307875
10. M. R. Davarpanah, M. Sanji, and M. Aramideh, "Farsi lexical analysis and stop word list," Library Hi Tech, vol. 27, no. 3, pp. 435-449, 2009. [Online]. Available: http://www.emeraldinsight.com/10.1108/07378830910988559
11. U. Garain and A. K. Das, "Stop word detection in compressed textual images: An experiment on indic script documents," 2008 19th International Conference on Pattern Recognition, pp. 1-4, Dec. 2008. [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4761529
12. Pascal ontology learning challenge. [Online]. Available: http://olc.ijs.si/
13. C. Hauff and D. Hiemstra, "University of twente at trec 2009: Indexing half a billion web pages," in The Eighteenth Text Retrieval Conference (TREC 2009) : Proceedings, ser. NIST Special Publication, E. Voorhees and L. Buckland , Eds., no. SP 500-278. Gaithersburg, USA: National Institute of Standards and Technology, 2010.
14. L. Q. Ha, P. Hanna, J. Ming, and F. J. Smith, "Extending zipf's law to n-grams for large corpora," Artif. Intell. Rev., vol. 32, pp. 101-113, December 2009.