Optimizing document similarity detection in Persian information retrieval

Journal of Convergence Information Technology

Volumn 5, Issue 2, 2010, Pages 101-106

  Kashefi, Omid ^a   Mohseni, Nina ^a   Minaei, Behrouz ^a  

^a IRAN UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Iran)

Author keywords

Affix; Document similarity; Information retrieval; Lemmatization; NLP; Persian language; Stemming; Suffix

Indexed keywords

AFFIX; CO-OCCURRENCE; DOCUMENT SIMILARITY; EVALUATION RESULTS; LATENT SEMANTIC INDEXING; LEMMATIZATION; PERSIANS; VECTOR SPACE MODELS; WEB USERS;

LINGUISTICS; NATURAL LANGUAGE PROCESSING SYSTEMS; SEARCH ENGINES; SECURITY OF DATA; TEXT PROCESSING; VECTOR SPACES; WORLD WIDE WEB;

INFORMATION RETRIEVAL;

EID: 78651593888     PISSN: 19759320     EISSN: None     Source Type: Journal    
DOI: 10.4156/jcit.vol5.issue2.11     Document Type: Article

References (27)

1. W. S. Al halabi, M. Kubat, M. Tapia, "A Tool to Personalize the Ranking of the Documents Returned by an Internet Search Engine", Journal of Convergence Information Technology, Vol.2 No.3, pp. 6-10, 2007.
2. S. Antonia, and H. Leong, "Duplicate data detection", in Proceedings of the 6th International Web Conference, 1997.
3. G. A. Singhal, "Modern Information Retrieval: A brief overview", Journal of the American Society for Information Science, vol. 24, pp. 115-139, 2001.
4. J. Becker, and D. Kuropka, "Topic-based Vector Space Model", In Proceedings of the 6th International Conference on Business Information Systems, 2003.
5. E. M. Voorhees, "Natural Language Processing and Information Retrieval", Lecture Notes in Computer Science, Springer Berlin/Heidelberg, pp. 32-48, 1999.
6. J. W.Cooper, A. R.Coden, and E. W.Bro, "Detecting similar documents using salient terms", in Proceedings of the 11th international conference on Information and knowledge management, Virginia, USA, 2002.
7. M. Azadnia, "Presenting an expert system for automatic correcting Persian texts", International Journal of Computer Science and Network Security, Vol. 8, No. 3, pp. 27-31, 2008.
8. H. Al-Mubaid, and P. Chen, "Context-based similar words detection and its application in specialized search engines", in Proceedings of the 10th international conference on Intelligent user interfaces, San Diego, California, USA, 2005.
9. W. Meng, C. YU and K. Lup Liu, "Building efficient and effective meta search engines", ACM Computing Surveys, Volume 34, pp. 48-89, 2002.
10. S. Helmer, "Measuring the structural similarity of semistructured documents using entropy", in Proceedings of the 33rd international conference on Very large data bases, 2007.
11. M. Sankupellay, "Malay-Language Stemmer," Sunway Academic Journal, vol. 3, pp. 147-153, 2006.
12. A. Zamanifar, B. Minaei-Bidgoli, and O. Kashefi, "A New Technique for Detecting Similar Documents based on Term Co-occurrence and Conceptual Property of the Text", in Proceedings of the 3th International Conference on Digital Information Management, London, England, 2008.
13. G. Salton, A. Wong, and C. S. Yang "A Vector Space Model for Automatic Indexing" In Proceedings of the 14th ACM international conference on Computational Linguistics, 1975.
14. S. Deerwester, S. T. Dumais, G. W. Furnas, T.K. Landauer, and R. Harshman, "Indexing by Latent Semantic Analysis", Journal of the American Society for Information Science, Vol. 41, pp. 391-407, 1990.
15. Broder, Glassman, Manasse, and Zweig, Syntactic Clustering of the Web, SRC Technical Note, 1997.
16. A. Broder, "Identifying and filtering near-duplicate documents" in Proceedings of the 11th Annual Symposium on Combinatorial Pattern Matching, Springer, 2000.
17. A. Aleahmad, P. Hakimian, F. Mahdikhani, and F. Oroumchian, "N-Gram and Local Context Analysis For Persian Text Retrieval", in Proceedings of the Third Text Retrieval Conference (TREC-3), 2007.
18. F. Oroumchian, and F. Mazhar Garamaleki, "An Evaluation of Retrieval performance Using Farsi Text", in Proceedings First Eurasia Conference on Advances in Information and Communication Technology, Tehran, Iran, 2002.
19. A. Nayyeri, and F. Oroumchian, "FuFaIR: a Fuzzy Farsi Information Retrieval System", in Proceedings of IEEE International Conference on Computer Systems and Applications, 2006.
20. K. Taghva, R. Beckley, and M. Sadeh. "A Stemming Algorithm for the Farsi Language". in Proceedings International Conference on Information Technology: Coding and Computing, 2005.
21. R. Hessami Fard, and G. Ghasem Sani, "Stemmer Algorithm Design for Persian Language", in Proceedings 11th International CSI Computer Conference, Tehran, Iran, 2006.
22. A. Mokhtaripour, S. Jahanpour, "Introduction to a New Farsi Stemmer", in CIKM, Virginia, USA, 2006.
23. A. Usefan, S. Salehi, and B. Minaei-Bidgoli, "Stemming Chalenges and Stemming Algorithm for Farsi Verbs", In Proceedings of the First Workshop on Persian Language and Computers, Tehran University, Iran, 1993.
24. M. Sadiqi, K. Zamanifar, "A method for overcoming Persian web content mining", Iran Documents, Vol. 2, Iran, 2005.
25. M. Nasri, and O. Kashefi, Defects of Persian Official Writing Rules from Computational View, Technical Report PMF-1-I, IUST, Tehran, 2008.
26. O. Kashefi, B. Minaei-Bidgoli, M. Sharifi, "A novel string distance metric for ranking Persian spelling error corrections", Language Resource and Evaluation, 2010.
27. H. Anvari, H.A. Givi, Persian Language, 3rd ed., Tehran: Fatemi, 2006.