Evaluation of n-gram conflation approaches for arabic text retrieval

Journal of the American Society for Information Science and Technology

Volumn 60, Issue 7, 2009, Pages 1448-1465

  Ahmed, Farag ^a   Nürnberger, Andreas ^a  

^a OTTO VON GUERICKE UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE USER INTERFACE; ARABIC TEXTS; METASEARCH; N-GRAM MODELS; N-GRAMS; PRECISION AND RECALL; PRIOR KNOWLEDGE; RELATED WORD; SIMILARITY MEASURE; SPECIFIC LOCATION; SUBWORDS; TARGET LANGUAGE; UNSUPERVISED METHOD;

SEARCH ENGINES; USER INTERFACES;

LINGUISTICS;

EID: 66749186635     PISSN: 15322882     EISSN: 15322890     Source Type: Journal    
DOI: 10.1002/asi.21063     Document Type: Article

References (48)

1. Abu-Salem, H. (2004). Comparison of stemming and n-gram matching for term-conflation in Arabic text. International Journal of Computer Processing of Oriental languages. 17(2), 61-81.
2. Adamson, G.W., & Boreham, J. (1974). The use of an association measure based on character structure to identify semantically related pairs of words and document titles. Information Storage and Retrieval, 10, 253-260.
3. Ahmed, F., De Luca. E.W. & Nürnberger, A. (2007. February). MultiSpell: An n-gram-based language-independent spellchecker. Poster presented at the Eighth International Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2007), Mexico City, Mexico. Retrieved March 19, 2009, from http://wwwiti.es.unimagdeburg. de/-fahmed/AhmDeLNue-MultiSpell- CicLing07.pdf.
4. Ahmed, F., & Nürnberger, A. (2008). araSearch: Improving Arabic text retrieval via detection of word form variations. In Proceedings of the First International Conference on Information Systems and Economic Intelligence (SIIE'2008) (pp. 309-323). Hmmamet, Tunisia: IHE.
5. Al-Fedaghi, S.S., & Al-Anzi, F.S. (1989). A new algorithm to generate Arabic root-pattern forms. In Proceedings of the 11th National Computer Conference (pp. 04-07). Dhahran. Saudi Arabia: King Fahd University of Petroleum and Minerals.
6. Aqeel, S.U., Beitzel, S.M., Jensen E.C, Grossman, D., & Frieder, O. (2006). On the development of name search techniques for Arabic. Journal of the American Society for Information Science and Technology, 57, 728-739.
7. Berlian, V., Vega, S.N., &.Bressan, S. (2001). Indexing the Indonesian Web: Language identification and miscellaneous issues. Paper presented at the 10th International World Wide Web Conference, Hong Kong.
8. Bordag, S. (2005, September). Unsupervised knowledge-free morpheme boundary detection. Paper presented at the International Conference on Recent Advances in Natural Language Processing (RANLP 05). Retrieved March 17, 2009, from http://wortschatz.unileipzig.de/-sbordag/papers/BordagMorphy05.pdf.
9. Buckwalter, T. (2002). Buckwalter Arabic Morphological Analyzer Version 1.0. Retrieved February 26, 2009, from http://www.ldc.upenn.edu/Catalog/ CatalogEntry.jsp?catalogId=LDC2002L49.
10. Carlberger, J., Dalianis, H., Hassel, M., & Knutsson, O. (2001, May). Improving precision, in information retrieval for Swedish using stemming. Paper presented at NODALIDA '01: The 13th Nordic Conference on Computational Linguistics, Uppsala, Sweden. Retrieved March 17, 2009, from http://stp.ling.uu.se/nodalida01/pdf/carlberger.pdf.
11. Dang, M.T., & Choudri, S. (2005). Simple unsupervised morphology analysis algorithm. In Unsupervised Segmentation of Words into Morphemes: Challenge 2005, Laboratory of Computer and Information Science. Retrieved February 26, 2009, from www.cis.hut.fi/morphochallenge 2005/P09-DangChoudri.pdf.
12. Darwish, K., & Oard, D.W. (2002). Term selection for searching printed. Arabic. In Proceedings of the 25th ACM SIGIR International Conference on Research and Development in Information Retrieval (pp. 261-268). New York: ACM.
13. Davis, M. W., & Ogden, W.C. ( 1998). Free resources and advanced alignment for cross-language text retrieval. In E.M. Voorhees & D.K. Harman (Eds.), NIST Special Publication 500-240: Proceedings of the Sixth Text Retrieval Conference (TREC-6) (pp. 385-394). Gaithersburg, MD: NIST.
14. De Roeck, A.N., & Al-Fares, W. (2000). A morphologically sensitive clustering algorithm for identifying Arabic roots. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics (pp. 199-206). Morristown, NJ: ACL.
15. Ekmekcioglu, F.C., Lynch, M.F., & Willett, P. (1996). Stemming and n-gram matching for term, conflation in Turkish texts. Information Research News, 2, pp. 2-6. Retrieved February 26, 2009, from http://informationr.net/ir/ 2-2/paper13.html.
16. Frakes, W.B. (1992). Stemming algorithms. In W.B. Frakes & R. Baeza-Yates (Eds.), Information retrieval: Data structures and algorithms. Englewood Cliffs, NJ: Prentice-Hall.
17. Gelbukh, A., Alexandrov, M., & Han, S.Y. (2004). Detecting inflection patterns in natural language by minimization of the morphological model. In A. Sanfeliu, J.F. Martinez Trinidad, & J.A. Carrasco Ochoa (Eds.), Lecture Notes on Computer Science, Vol.3287: Progress in Pattern Recognition, Image Analysis, and Applications: Ninth Iberoamerican Congress on Pattern Recognition (CIARP 2004) (pp. 432-438). Berlin: Springer.
18. Ghaoui, A., Yvon, F., Mokbel, C., & Chollet, G. (2005). On the use of morphological constraints in n-gram. statistical language model. In Proceedings of the Eighth International Conference on Spoken Language Processing (INTERSPEECH-2005) (pp. 1281-1284). Retrieved March 1, 2009, from http://www.isca-speech.org/archive/interspeech- 2005/i05-1281.ok?html.
19. Gispert, A., & Marino, J.B. (2006). Linguistic tuple segmentation in ngrambased statistical machine translation. In Proceedings of the Ninth International Conference on Spoken Language Processing (INTERSPEECH'06) (pp. 1149-1152). Retrieved March 1, 2009, from http://www.iscaspeech.org/archive/ interspeech-2006/i06-.1049.html.
20. Greengrass, M., Robertson, A.M., Robyn, S., & Willett, P. (1996). Processing morphological variants in searches of Latin text. Information Research News, 6(4), 2-5.
21. Hafer, M., & Weiss, S. (1974). Word segmentation by letter successor varieties. Information Processing & Management. 10, 371-386.
22. Khaltar, B.Q., Fujii, A., & Ishikawa, T. (2006). Extracting loanwords from. Mongolian corpora and producing a Japanese-Mongolian bilingual, dictionary. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the ACL (pp. 657-664). Morristown, NJ: ACL.
23. Khoja, S., & Garside, R. (1999). Arabic stemmer. Retrieved March 17,2009, from. http://zeus.cs.pacificu.edu/shereen/research.htm#stemming.
24. Khreisat, L. (2006). Arabic text classification using n-gram frequency statistics: A comparative study. In S.F. Crone, S. Lessmann, & R. Stahlbock (Eds), Proceedings of the 2006 International Conference on Data Mining (DMIN 2006) (pp. 78-82). Las Vegas, NV: CSREA Press.
25. Kosinov, S. (2001). Evaluation of n-grams conflation approach in text-based information retrieval. In Proceedings of the Eighth International Symposium on String Processing and Information Retrieval (SPIRE 2001) (pp. 136-142). Washington, DC: IEEE.
26. Kraaij, W., & Pohlmann, R. (1996). Viewing stemming as recall enhancement. In Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 40-48). New York: ACM.
27. Larkey, L., Ballesteros, L., & Connell, M. (2005). Light stemming for Arabic IR. In A. Soudi, A. van den Bosch, & G. Neumann (Eds.), Arabic computational morphology: Knowledge-based and empirical methods (pp. 221-243). Dordrecht, Netherlands: Springer.
28. Levenshtein, V.I. (1966). Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Dokladi 10, 8, 707-710.
29. Lovins J.B. (1968). Development of a stemming algorithm. Mechanical. Translation and Computational Linguistics, 11, 22-31.
30. Martins, B., & Silva, M.J. (2004). Spelling correction for search engine queries. In J.G. Carbonell & J. Siekmann (Eds.), Lecture Notes in Computer Science:, Vol.3230: Advances in Natural Language Processing (pp. 372-383). Berlin: Springer.
31. Mayfield, J., McNam.ee, P., Costello, C., Piatko, C., & Banerjee, A. (2001). JHU/APL at TREC 2001: Experiments in filtering and in Arabic, video, and Web retrieval. In E.M Voorhees & D. Harman (Eds.), NIST Special Publication 500-250: Proceedings of the Tenth Text Retrieval Conference (TREC 2001). Gaithersburg, MD; NIST.
32. Monz, C., & de Rijke, M. (2001). Shallow morphological, analysis in monolingual information retrieval for Dutch, German, and Italian. In C Peters, M. Braschler, J. Gonzalo, & M. Kluck (Eds), Lecture Notes in Computer Science, Vol.2406: Evaluation of Cross-Language Information Retrieval Systems (CLEF 2001) (pp. 262-277). London: Springer.
33. Moukdad, H. (2004). Lost in cyberspace: How do search engines handle Arabic queries?, In Access to Information: Technologies, Skills, and Sociopolitical Context. Proceedings of the 32nd Annual Conference of the Canadian Association for Information Science. Retreived March 1, 2009, from http://www.cais-acsi.ca/proceedings/2004/moukdad-2004.pdf.
34. Moukdad,H.,&Large,A.(2001).Informationretrievalfromfull- textArabicdatabases:CansearchenginesdesignedforEnglish,dothejob?Libri,51(2), 63-74.(Pubitemid33778535)
35. Moulinier, I., McCulloh, A., & Lund, E. (2000). West group at CLEF 2000: Non-English monolingual retrieval. In C. Peters (Ed.), Revised Papers from the Workshop on Cross Language Information Retrieval and Evaluation (pp. 253-260). Heidelberg, Germany: Springer.
36. Mustafa, S.H. (2004). Character contiguity in n-gram-based word matching: The case for Arabic text searching. Information Processing & Management, 41, 819-827.
37. Mustafa, S.H., & Al-Radaideh, Q. (2004). Using n-grams for Arabic text searching. Journal of the American Society for Information Science and Technology, 55, 1002-1007.
38. Paice, C.D. (1990). Another stemmer. ACM SIGIR Forum, 24(3), 56-61.
39. Pfeifer, U., Poersch, T., & Fuhr, N. (1996). Retrieval effectiveness of proper name search methods. Information Processing & Management, 32, 667-679. (Pubitemid 126371199)
40. Popovic, M., & Willett, P. (1992). The effectiveness of stemming for naturallanguage access to Slovenetextual data. Journal of the American Society for Information Science, 43, 384-390.
41. Porter, M.F. (1980). An algorithm for suffix stripping. Program, 14(3), 130-137.
42. Sabah, S., & Al-Anzi. (1989). A new algorithm, to generate Arabic rootpattern forms. In Proceedings of the 11th National Computer Conference (pp. 04-07). Dhahran, Saudi. Arabia: King Fahd University of Petroleum and Minerals.
43. Tai, S.Y., Ong, C.S., & Abdullah, N.A. (2000). On designing an automated Malaysian stemmer for the Malay language. In Proceedings of the Fifth International Workshop on Information Retrieval with Asian Languages (pp. 207-208). New York: ACM.
44. Xu, J., & Croft, W.B. (1996). Corpus-based stemming using cooccurrence of word variants, ACM Transactions on Information Systems, 16(1), 61-81.
45. Wagner, R.A., & Fisher, M.J. (1974). The string to string correction problem. Journal of the Association for Computing Machinery, 21, 168-173.
46. Whitaker, B. (2002a). Arabic words and the Roman alphabet. Retrieved March 2, 2009, from http://www.al-bab.com/arab/language/romanl.htm.
47. Whitaker, B. (2002b). Lost in translation. Retrieved March 2, 2009, from http://www.guardian.co.uk/world/2002/jun/10/israel1 .
48. Zobel, J., & Dart, P. (1995). Finding approximate matches in large lexicons. Software: Practice and Experience, 25, 331-345.