Arabic script web page language identifications using decision tree neural networks

Pattern Recognition

Volumn 44, Issue 1, 2011, Pages 133-144

  Selamat, A ^a   Ng, C C ^a  

^a UNIVERSITI TEKNOLOGI MALAYSIA   (Malaysia)

Author keywords

Adaptive neural networks; Arabic script web documents; ARTMAP; Decision tree; Language identification

Indexed keywords

ADAPTIVE NEURAL NETWORKS; ARABIC SCRIPTS; ARTMAP; HYBRID APPROACH; LANGUAGE IDENTIFICATION; PERSIANS; WEB DOCUMENT; WEB PAGE;

DECISION TREES; LINGUISTICS; QUERY LANGUAGES; SPEECH RECOGNITION; WORLD WIDE WEB;

NEURAL NETWORKS;

EID: 77956468640     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2010.07.009     Document Type: Article

References (57)

1. M.Z. Abd Rozan, Y. Mikami, A.Z. Abu Bakar, O. Vikas, Multilingual ict education: language observatory as a monitoring instrument, in: Proceedings of the South East Asia Regional Computer Confederation 2005: ICT Building Bridges Conference, vol. 46, Sydney, Australia, pp. 5361.
2. M.H. Aghdam, N. Ghasem-Aghaee, and M.B. Basiri Text feature selection using ant colony optimization Expert Systems with Applications 36 2009 6843 6853
3. J.D. Allen The Unicode Standard 5.0 2006 Addison-Wesley
4. H. Altinay, Z. Erenel, Analytical evaluation of term weighting schemes for text categorization, Pattern Recognition Letters 23 (2010) 13101323, doi: 10.1016/j.patrec.2010.03.012.
5. N.E.B. Amara, and F. Bouslama Classification of arabic script using multiple sources of information: state of the art and perspectives International Journal on Document Analysis and Recognition 5 2003 195 212
6. O. Artemenko, T. Mandl, M. Shramko, C. Womser-Hacker, Evaluation of a language identification system for mono-and multilingual text documents, in: Proceedings of the 2006 ACM Symposium on Applied Computing, pp. 859860.
7. D. Benedetto, E. Caglioti, and V. Loreto Language trees and zipping Physical Review Letters 88 2002 48702
8. C. Biemann, S. Teresniak, Disentangling from babylonian confusionunsupervised language identification, in: Proceedings of CICLing-2005, Mexico City, Mexico and Springer Lecture Notes in Computer Science, Proceedings of Computational Linguistics and Intelligent Text Processing, vol. 3406, Springer, 2005, pp. 762773.
9. E. Bilcu, J. Astola, A hybrid neural network for language identification from text, in: Proceedings of the 2006 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing, pp. 253258.
10. G. Botha, V. Zimu, E. Barnard, Text-based language identification for the south African languages, in: Proceedings of the 17th Annual Symposium of the Pattern Recognition Association of South Africa, 2006, Parys, South Africa, pp. 713.
11. British Broadcasting Corporation, BBC news 〈 http://www.bbc.co.uk/ 〉, 2010. Accessed on 15 November 2009.
12. G.A. Carpenter, Default artmap, in: Proceedings of the International Joint Conference on Neural Networks 2003, vol. 2, 2003, pp. 13961401.
13. W.B. Cavnar, J.M. Trenkle, N-gram-based text categorization, in: Proceedings of the 3rd Annual Symposium on Document Analysis and Information Retrieval 1994, Las Vegas, Nevada, USA, 1994, pp. 161175.
14. K. Chai, H. Chieu, H. Ng, Bayesian online classifiers for text classification and filtering, in: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval 2002, ACM New York, NY, SA, pp. 97104.
15. A. De Souza, F. Pedroni, E. Oliveira, P. Ciarelli, W. Henrique, L. Veronese, and C. Badue Automated multi-label text categorization with vg-ram weightless neural networks Neurocomputing 72 1012 2009 2209 2217
16. E. Dougherty Feature-selection overfitting with small-sample classifier design IEEE Intelligent Systems 20 2005 64 66
17. E. Giguet, The stakes of multilinguality: multilingual text tokenization in natural language diagnosis, in: Proceedings of the 4th Pacific Rim International Conference on Artificial Intelligence Workshop Future Issues for Multilingual Text Processing, 1996, Cairns, Australia.
18. G. Grefenstette, Comparing two language identification schemes, in: Proceedings of the 3rd International Conference on the Statistical Analysis of Textual Data, Rome, Italy, December 1995.
19. Y. Guo, S. Zhiqing, and N. Hua Automatic text categorization based on content analysis with cognitive situation models Information Sciences: an International Journal 180 2010 613 630
20. J. Hakkinen, J. Tian, N-gram and decision tree based language identification for written words, in: Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding 2001, 2001, pp. 335338.
21. S. Haykin Neural Networks: A Comprehensive Foundation 2008 Prentice-Hall
22. T. Hidayet, and S. brahim Letter based text scoring method for language identification Lecture Notes in Computer Science vol. 3261/2005 2005 283 290
23. C. Jiang, F. Coenen, R. Sanderson, and M. Zito Text classification using graph mining-based feature extraction Knowledge-Based Systems 23 2010 302 308
24. S.Y. Jung, S.L. Hong, E. Paek, An english to Korean transliteration model of extended Markov window, in: Proceedings of the 18th Conference on Computational linguistics 2000, vol. 1, 2000, pp. 383389.
25. R. Kohavi, A study of cross-validation and bootstrap for accuracy estimation and model selection, in: Proceedings of the International Joint Conference on Artificial Intelligence 1995, vol. 14, 1995, pp. 11371145.
26. C. Lee, and H. Yang Construction of supervised and unsupervised learning systems for multilingual text categorization Expert Systems with Applications 36 2009 2400 2410
27. D.D. Lewis, Evaluating and optimizing autonomous text classification systems, in: Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1995, pp. 246254.
28. H.Z. Li, B. Ma, and C.H. Lee A vector space modeling approach to spoken language identification IEEE Transactions on Audio, Speech, and Language Processing 15 2007 271 284
29. Q. Li, and P. Chen Personalized text snippet extraction using statistical language models Pattern Recognition 43 2010 378 386
30. S. Li, K. Momoi, A composite approach to language/encoding detection, in: Proceedings of the 19th International Unicode Conference 2001, 2001, pp. 114.
31. H. Liu, and H. Motoda Computational methods of feature selection Computational Methods of Feature Selection 2006 Chapman & Hall/CRC 3 17
32. Y. Liua, H.T. Loh, and A. Sun Imbalanced text classification: a term weighting approach Expert Systems with Applications 36 2009 690 701
33. S. MacNamara, P. Cunningham, and J. Byrne Neural networks for language identification: a comparative study Information Processing and Management 34 1998 395 403
34. B. Martins, M.J. Silva, Language identification in web pages, in: Proceedings of the 2005 ACM Symposium on Applied Computing, 2005, pp. 764768.
35. P. McNamee Character n-gram tokenization for European language text retrieval Information Retrieval 7 2004 73 97
36. Miniwatts Marketing Group, Internet world users by language: top ten internet languages used in the web 〈 http://www.internetworldstats.com/ 〉, 2010, Accessed on 15 April 2010.
37. M.S. Mohamed Ould, Arabic language script and encoding identification with support vector machines and rough set theory, Master's Thesis, Universiti Teknologi Malaysia, 2007.
38. K. Murthy, and G. Kumar Language identification from small text samples Journal of Quantitative Linguistics 13 2006 57 80
39. P.J. Payack, The global language monitor 〈 http://www. languagemonitor.com/ 〉, 2007, Accessed on 20 November 2007.
40. F. Peng, D. Schuurmans, S. Wang, Language and task independent text categorization with simple language models, in: Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, vol. 1, 2003, pp. 110117.
41. P. Refaeilzadeh, L. Tang, H. Liu, On comparison of feature selection algorithms, in: Association for the Advancement of Artificial Intelligence (AAAI), Workshop on Evaluation Methods for Machine Learning II, The AAAI Press, Menlo Park, California, 2007, pp. 3439.
42. S. SantoshKumar, V. Ramasubramanian, Automatic language identification using ergodic-hmm, in: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, IEEE, 2005, pp. 609612.
43. A. Selamat, C.C. Ng, S.N.A. Ibrahim, Arabic script web document language identification using neural network, in: Proceedings of the Ninth International Conference on Information Integration and Web Based Applications and Services 2007, 2007, pp. 329338.
44. A. Selamat, and S. Omatu Web page feature selection and classification using neural networks Information Sciences 158 2004 69 88
45. P. Sibun, J.C. Reynar, Language identification: examining the issues, in: Proceedings of the Symposium on Document Analysis and Information Retrieval 2006, 2006, pp. 125135.
46. S. Singhi, H. Liu, Feature subset selection bias for classification learning, in: Proceedings of the 23rd International Conference on Machine Learning 2006, ACM, New York, NY, USA, 2006, pp. 849856.
47. Y. Song, L. Dai, R. Wang, An automatic language identification method based on subspace analysis, in: Multimedia and Expo, 2009. ICME 2009. IEEE International Conference on Digital Object Identifier: 10.1109/ICME.2009. 5202567, pp. 598601.
48. I. Suzuki, Y. Mikami, A. Ohsato, and Y. Chubachi A language and character set determination method based on n-gram statistics ACM Transactions on Asian Language Information Processing 1 2002 269 278
49. S. Tan An effective refinement strategy for knn text classifier Expert Systems with Applications 30 2006 290 298
50. J. Tian, J. Suontausta, Scalable neural network based language identification from written text, in: Proceedings of the 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, 2003, pp. 4851.
51. I. Unicode, Unicode home page, the unicode consortium 〈 http://www.unicode.org/ 〉, 2010, Accessed 28 April 2010.
52. L. Wang, E. Ambikairajah, E. Choi, Robust language identification based on fused phonotactic information with mlksfm pre-classifier, in: Multimedia and Expo, 2009. ICME 2009. IEEE International Conference on Digital Object Identifier: 10.1109/ICME.2009.5202451, Publication Year: 2009, pp. 121124.
53. G. Windisch, L. Csink, Language identification using global statistics of natural languages, in: Proceedings of the 2nd Romanian-Hungarian Joint Symposium on Applied Computational Intelligence (SACI), 2005, pp. 243255.
54. A. Xafopoulos, C. Kotropoulos, G. Almpanidis, and I. Pitas Language identification in web documents using discrete hmms Pattern Recognition 37 2004 583 594
55. B. Yin, E. Ambikairajah, A novel weighting technique for combining likelihood scores in language identification systems, in: Information, Communications and Signal Processing, 2007 6th International Conference on Digital Object Identifier: 10.1109/ICICS.2007.4449628, pp. 15.
56. L.F. Zhai, M.H. Siu, X. Yang, H. Gish, Discriminatively trained language models using support vector machines for language identification, in: Proceedings of the IEEE Odyssey 2006: The Speaker and Language Recognition Workshop, 2006, pp. 16.
57. J. Zou, G. Chen, W. Guo, Chinese web page classification using noise-tolerant support vector machines, in: Proceedings of the 2005 IEEE International Conference on Natural Language Processing and Knowledge Engineering, 2005, pp. 785790.