Language identification in Web pages

Proceedings of the ACM Symposium on Applied Computing

Volumn 1, Issue , 2005, Pages 764-768

  Martins, Bruno ^a   Silva, Mário J ^a  

^a UNIVERSITY OF LISBON   (Portugal)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTER PROGRAMMING LANGUAGES; HEURISTIC METHODS; SEARCH ENGINES;

GRAM BASED ALGORITHM; WEB DOCUMENT;

WORLD WIDE WEB;

EID: 33644526889     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1066677.1066852     Document Type: Conference Paper

References (28)

1. E. Amitay. Hypertext - the importance of being different. Master's thesis, Centre for Cognitive Science, Edinburgh University, 1997.
2. E. Amitay. Using common hypertext links to identify the best phrasal description of target Web documents. In Proceedings of the SIGIR-98 Post-Conference Workshop on Hypertext Information Retrieval for the Web, 1998.
3. J. A. Aslam and M. Frost. An information-theoretic measure for document similarity. In Proceedings of SIGIR-03, the 26th annual international conference on Research and development in informaion retrieval, pages 449-450. ACM Press, July 2003.
4. I. Biskri and S. Delisle. Text classification and multilinguism: Getting at words vian-grams of characters. In Proceedings of SCI-2002, 6th World Multiconference on Systemics, Cybernetics and Informatics, volume 5, pages 110-115, July 2002.
5. A. Budanitsky and G. Hirst. Semantic distance in WordNet: An experimental, application-oriented evaluation of five measures. In Workshop on WordNet and Other Lexical Resources, 2nd meeting of the North American Chapter of the Association for Computational Linguistics (NAACL-2000), June 2001.
6. W. B. Cavnar and J. M. Trenkle. N-gram-based text categorization. In Proceedings of SDAIR-94, the 3rd Annual Symposium on Document Analysis and Information Retrieval, pages 161-175, Las Vegas, Nevada, U.S.A. 1994.
7. S. Chakrabarti, B. E. Dom, S. R. Kumar, P. Raghavan, S. Rajagopalan, A. Tomkins, D. Gibson, and J. Kleinberg. Mining the Web's link structure. Computer, 32(8):60-67, August 1999.
8. S. F. Chen and J. Goodman. An empirical study of smoothing techniques for language modeling. In A. Joshi and M. Palmer, editors, Proceedings of ACL-96, the 34th Annual Meeting of the Association for Computational Linguistics, pages 310-318, San Francisco, 1996. Morgan Kaufmann Publishers.
9. Y. S. M. Cutler and W. Meng. Using the structure of HTML documents to improve retrieval. In Proceedings of the USENIX Symposium on Internet Technologies and Systems, 1997.
10. M. Darnashek. Gauging similarity with n-grams: language independent categorization of text. Science. 267(5199):843-848, 1995.
11. T. Dunning. Statistical identification of language. Technical Report MCCS 94-273, New Mexico State University, 1994.
12. I. J. Good. The population frequencies of species and the estimation of population parameters. Biometrika, 40:237-264, 1953.
13. G. Grefenstette and P. Tapanainen. What is a word, what is a sentence? problems of tokenization. In Proceedings of COMPLEX-94, the 3rd International Conference on Computational Lexicography, pages 79-87, 1994.
14. P. Henrich. Language identification for the automatic grapheme-to-phoneme conversion of foreign words in a german text-to-speech system. In Proceedings of Eurospeech 1989, European Speech Communication and Technology, pages 220-223, September 1989.
15. C. Hill. Information space based on html structure. In E. M. Voorhees and D. K. Harman, editors, Proceedings of TREC-9, the 9th Text REtrieval Conference. Department of Commerce of National Institute of Standards and Technology, 2000.
16. J. Y. Jiang and D. W. Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of ROCLING-X, the ROCUNG 1997 International Conference on Research on Computational Linguistics, 1997.
17. D. Lin. An information-theoretic definition of similarity. In Proceedings of ICML-98, the 15th International Conference on Machine Learning, pages 296-304. Morgan Kaufmann, San Francisco, CA, 1998.
18. R. D. Lins and P. Conçoives. Automatic language identification of written texts. In Proceedings of SAC-2004, the 2004 ACM symposium on Applied computing, pages 1128-1133. ACM Press, 2004.
19. M. Marchiori. The limits of web metadata, and beyond. In Proceedings of WWW-98, the 7th International World Wide Web Conference, April 1998.
20. P. McNamee and J. Mayfleld. Character n-gram tokenization for european language text retrieval. Information Retrieval, 7, April 2004.
21. E. Miller, D. Shen, J. Liu, and C. Nicholas. Performance and scalability of a large-scale n-gram based information retrieval system. Journal of Digital Information, 1(21), 2000.
22. P. Newman. Foreign language identification - first step in the translation process. In K. Kummer, editor, Proceedings of the 28th Annual Conference of the American Translators Association, pages 509-516, 1987.
23. C. Pearce and B. Rye. N-gram term weighting: A comparative analysis. Technical Report TR-R52-001-98, National Security Agency Technical, January 1998.
24. P. Sibun and J. C. Reynar. Language identification: Examining the issues. In Proceedings of SDAIR-96, the 5th Symposium on Document Analysis and Information Retrieval, pages 125-135, 1996.
25. P. Sibun and A. L. Spitz. Language determination: natural language processing from scanned document images. In Proceedings of ANLP-94, the 4th conference on Applied natural language processing, pages 15-21. Morgan Kaufmann Publishers Inc., 1994.
26. C. Souter, G. Churcher, J. Hayes, J. Hughes, and S. Johnson. Natural language identification using corpus-based models. Hermes Journal of Linguistics, 13:183-203, 1994.
27. L. Wittmann, T. Pêgo, and D. Santos. Português do Brasil e de Portugal: alguns contrastes. In Actas do XI Encontro da Associação Portuguesa de Linguística, pages 465-487, 1995.
28. D.-V. Ziegler. The Automatic Identification of Languages Using Linguistic Recognition Signals. PhD thesis, State University of New York, 1991.