Linguini: Language Identification for Multilingual Documents

Journal of Management Information Systems

Volumn 16, Issue 3, 1999, Pages 71-101

  Prager, John M ^a,b,c  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^c UNIVERSITY OF MASSACHUSETTS   (United States)

Author keywords

Categorization; Information retrieval; Language identification; Vector space models

Indexed keywords

EID: 0033277432     PISSN: 07421222     EISSN: None     Source Type: Journal    
DOI: 10.1080/07421222.1999.11518257     Document Type: Article

References (28)

1. Apte, C.; Damerau, F.; and Weiss, S. Automated learning of decision rules for text categorization, ACM Transactions on Information Systems, 12, 3 (1994), 233-251.
2. Baeza-Yates, R., and Ribeiro-Neto, B. Modern Information Retrieval. New York: ACM Press, 1999.
3. Boguraev, B., and Kennedy, C. Salience-based content characterization of text documents. In I. Mani and M. Maybury (eds.), Advances in Automatic Text Summarization. Cambridge, MA: MIT Press, 1999.
4. Byrd, R.J., and Ravin, Y. Identifying and extracting relations in text. In Proceedings of the Fourth International Conference on Applications of Natural Language to Information Systems (NLDB 99), Klagenfurt, Austria, 1999.
5. Cavnar, W.B., and Trenkle., J.M. N-gram-based text categorization. In Symposium on Document Analysis and Information Retrieval, University of Nevada, Las Vegas, 1994, pp. 161-176.
6. Damashek, M. Gauging similarity with N-grams: language-independent categorization of text. Science, 267 (1995), 843-848.
7. Duda, R.O., and Hart, P.E. Pattern Classification and Scene Analysis. New York: John Wiley & Sons, 1973.
8. Grefenstette, G. Comparing two language identification schemes. http:// www.rxrc.xerox.com/publis/mltt/jadt/jadt.html (1996).
9. Grefenstette, G., ed. Cross-Language Information Retrieval. Boston: Kluwer Academic Publishers, 1998.
10. Harmon, D. How effective is suffixing? Journal of the American Society for Information Science, 42, 1 (1991), 7-15.
11. Harmon, D. Ranking algorithms. In W. Frakes and R. Baeza-Yates (eds.), Information Retrieval: Data Structures and Algorithms. Upper Saddle River, NJ: Prentice-Hall, 1992, pp. 363-392.
12. Hearst, M.A. Automated discovery of WordNet relations. In C. Fellbaum (ed.), WordNet: an Electronic Lexical Database. Cambridge, MA: MIT Press, 1998.
13. Hull, D.A. Stemming algorithms: a case study for detailed evaluation. Journal of the American Society for Information Science, 47, 1 (1996), 70-84.
14. IBM Intelligent Miner for Text, http://www.software.ibm.com/data/iminer/fortext (1997).
15. Kupiec. J. MURAX: a robust linguistic approach for question answering using an on-line encyclopedia. In Proceedings of the 16th International ACM SIGIR Conference on Research and Development in Information Retrieval, Pittsburgh, 1993, pp. 181-190.
16. Lawrence, S., and Giles, C.L. Searching the World Wide Web. Science, 280 (1998), 98-100.
17. Lewis, D.D.; Shapire, R.E.; Callan J.P.; and Papka R. Training algorithms for linear text classifiers. In Proceedings of the Nineteenth International ACM SIGIR Conference on Research and Development in Information Retrieval, 1996, pp. 298-306.
18. Ling, C.X., and Wang, H. Computing optimal attribute weight settings for Nearest Neighbor algorithms. Artificial Intelligence Review, 11 (1997), 255-272.
19. Maarek, Y. and Smadja, F. Full text indexing based on lexical relations. an application: Software libraries. In Proceedings of the Seventeenth International ACM SIGIR Conference on Research and Development in Information Retrieval, 1989, pp. 198-206.
20. MUC. Proceedings of the Seventh Message Understanding Conference (MUC-7). DARPA Software and Intelligent Systems Technology Office, 1998.
21. Porter, M. F. An algorithm for suffix stripping. Program, 14, 3 (1980), 130-137.
22. Ravin, Y.; Wacholder, N.; and Choi, M. Disambiguation of names in text. In Proceedings of the Fifth Conference on Applied Natural Language Processing, Washington, DC, 1997, pp. 202-208.
23. Salton, G., and McGill, M. Introduction to Modern Information Retrieval. New York: McGraw-Hill, 1983.
24. TDT. Proceedings of DARPA Broadcast News Workshop (TDT-2), Washington, DC, February-March 1999.
25. Teufel, S., and Moens, M. Argumentative classification of extracted sentences as a first step towards flexible abstracting. In I. Mani and M. Maybury (eds.), Advances in Automatic Text Summarization. Cambridge, MA: MIT Press, 1999.
26. Yang, Y. Expert Network: effective and efficient learning from human decisions in text categorization and retrieval. In Proceedings of the Seventeenth International ACM SIGIR Conference on Research and Development in Information Retrieval, Dublin, 1994, pp. 13-22.
27. Yang, Y. An evaluation of statistical approaches to text categorization. Carnegie Mellon University School of Computer Science Technical Report CMU-CS-97-127, Pittsburgh, 1997.
28. Yang, Y., and Pedersen, J.O. A comparative study on feature selection in text categorization. In Proceedings of the Fourteenth International Conference on Machine Learning, Vanderbilt University, Nashville, 1997.