Embedding Web-Based Statistical Translation Models in Cross-Language Information Retrieval

Computational Linguistics

Volumn 29, Issue 3, 2003, Pages 381-419

  Kraaij, Wessel ^a   Nie, Jian Yun ^b   Simard, Michel ^b  

^a TNO   (Netherlands)

^b UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; MACHINE TRANSLATION; WEBSITES;

CROSS-LANGUAGE INFORMATION RETRIEVAL; CURRENT MODELING; EMBEDDINGS; LANGUAGE PAIRS; MACHINE TRANSLATIONS; STATISTICAL TRANSLATION MODEL; TRANSLATION MODELS; TRANSLATION RESOURCES; TRANSLATION SERVICES; WEB BASED;

INFORMATION RETRIEVAL;

EID: 0344154402     PISSN: 08912017     EISSN: None     Source Type: Journal    
DOI: 10.1162/089120103322711587     Document Type: Review

References (72)

1. Allen, James, editor. 2002. Event-Based Information Organization. Kluwer Academic, Boston.
2. Baum, L. E. 1972. An inequality and associated maximization technique in statistical estimations of probabilistic functions of Markov processes. Inequalities, 3:1-8.
3. Berger, Adam and John Lafferty. 2000. The Weaver system for document retrieval. In Ellen M. Voorhees and Donna K. Harman, editors, The Eighth Text Retrieval Conference (TREC-8), volume 8. National Institute of Standards and Technology Special Publication 500-246, Gaithersburg, MD.
4. Braschler, Martin and Bärbel Ripplinger. 2003. Stemming and decompounding for German text retrieval. In Fabrizio Sebastiani, editor, Advances in Information Retrieval: 25th European Conference on IR Research (ECIR 2003), Pisa, Italy, April 2003, Proceedings. Lecture Notes in Computer Science 2633. Springer, Berlin.
5. Broglio, John, James P. Callan, W. Bruce Croft, and Daniel W. Nachbar. 1995. Document retrieval and routing using the INQUERY system. In Donna K. Harman, editor, The Third Text Retrieval Conference, volume 4. National Institute of Standards and Technology Special Publication 500-236, Gaithersburg, MD, pages 29-38.
6. Brown, Peter F., John Cocke, Stephen A. Della Pietra, Vincent J. Della Pietra, Fredrick Jelinek, John D. Lafferty, Robert L. Mercer, and Paul S. Roossin. 1990. A statistical approach to machine translation. Computational Linguistics, 16(2):79-85.
7. Chen, Jiang and Jian-Yun Nie. 2000. Web parallel text mining for Chinese-English cross-language information retrieval. In Proceedings of NAACL-ANLP, Seattle.
8. Clark, James. 2001. SP - An SGML System Conforming to International Standard ISO 8879 - Standard Generalized Markup Language. Available at 〈http://www.jclark.com/sp/〉.
9. Conover, William Jay. 1980. Practical Nonparametric Statistics. Wiley, London.
10. Croft, W. Bruce, Alistair Moffat, C. J. "Keith" van Rijsbergen, Ross Wilkinson, and Justin Zobel, editors. 1998. Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'98). ACM Press.
11. Dumais, Susan T., Todd A. Letsche, Michael L. Littman, and Thomas K. Landauer. 1997. Automatic cross-language retrieval using latent semantic indexing. In AAAI Spring Symposium on Cross-Language Text and Speech Retrieval, Palo Alto, CA.
12. Federico, Marcello and Nicola Bertoldi. 2002. Statistical cross-language information retrieval using N-best query translations. In Micheline Beaulieu, Ricardo Baeza-Yates, Sung Hyon Myaeng, and Kalervo Järvelin, editors, Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2002). ACM Press, New York.
13. Foster, George F. 1991. Statistical lexical disambiguation. Master's thesis, McGill University, School of Computer Science.
14. Foster, George. 2000. A maximum entropy/minimum divergence translation model. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics (ACL), Hong Kong.
15. Franz, Martin, J. Scott McCarley, and R. Todd Ward. 2000. Ad hoc, cross-language and spoken document retrieval at IBM. In Ellen M. Voorhees and Donna K. Harman, editors, The Eighth Text Retrieval Conference (TREC-8), volume 8. National Institute of Standards and Technology Special Publication 500-246, Gaithersburg, MD.
16. Franz, Martin, J. Scott McCarley, Todd Ward, and Wei-Jing Zhu. 2001. Quantifying the utility of parallel corpora. In W. Bruce Croft, David J. Harper, Donald H. Kraft, and Justin Zobel, editors, Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001). ACM Press, New York.
17. Gao, Jianfeng, Jian-Yun Nie, Endong Xun, Jian Zhang, Ming Zhou, and Changning Huang. 2001. Improving query translation for cross-language information retrieval using statistical models. In W. Bruce Croft, David J. Harper, Donald H. Kraft, and Justin Zobel, editors, Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001). ACM Press, New York.
18. Gollins, Tim and Mark Sanderson. 2001. Improving cross language retrieval with triangulated translation. In W. Bruce Croft, David J. Harper, Donald H. Kraft, and Justin Zobel, editors, Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001). ACM Press, New York.
19. Grefenstette, Gregory. 1998. The problem of cross-language information retrieval. In Gregory Grefenstette, editor, Cross-Language Information Retrieval. Kluwer Academic, Boston, pages 1-9.
20. Harman, Donna K., editor. 1995. The Third Text Retrieval Conference (TREC-3), volume 4. National Institute of Standards and Technology Special Publication 500-236.
21. Hearst, Marti, Fred Gey, and Richard Tong, editors. 1999. Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '99). ACM Press.
22. Hiemstra, Djoerd. 1998. A linguistically motivated probabilistic model of information retrieval. In Christos Nicolaou and Constantine Stephanides, editors, Research and Advanced Technology for Digital Libraries - Second European Conference (ECDL'98), Proceedings. Lecture Notes in Computer Science 1513. Springer Verlag, Berlin.
23. Hiemstra, Djoerd. 2001. Using Language Models for Information Retrieval. Ph.D. thesis, University of Twente, Enschede, the Netherlands.
24. Hiemstra, Djoerd and Franciska de Jong. 1999. Disambiguation strategies for cross-language information retrieval. In European Conference on Digital Libraries, pages 274-293.
25. Hiemstra, Djoerd and Wessel Kraaij. 1999. Twenty-one at TREC-7: Ad hoc and cross language track. In Ellen M. Voorhees and Donna K. Harman, editors, The Seventh Text Retrieval Conference (TREC-7), volume 7. National Institute of Standards and Technology Special Publication 500-242, Gaithersburg, MD.
26. Hiemstra, Djoerd, Wessel Kraaij, Renée Pohlmann, and Thijs Westerveld. 2001. Translation resources, merging strategies and relevance feedback. In Carol Peters, editor, Cross-Language Information Retrieval and Evaluation. Lecture Notes in Computer Science 2069. Springer Verlag, Berlin.
27. Hull, David. 1993. Using statistical testing in the evaluation of retrieval experiments. In Robert Korfhage, Edie Rasmussen, and Peter Willett, editors, Proceedings of the 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '93), pages 329-338. ACM Press, New York.
28. Hull, David. 1996. Stemming algorithms - a case study for detailed evaluation. Journal of the American Society for Information Science, 47(1): 47-84.
29. Hull, David. 1997. Using structured queries for disambiguation in cross-language information retrieval. In David Hull and Douglas Oard, editors, AAAI Symposium on Cross-Language Text and Speech Retrieval. American Association for Artificial Intelligence. Available at 〈http://www.aaai.org/Press/Reports/Symposia/Spring/ss-97-05.html〉.
30. Hull, David and Gregory Grefenstette. 1996. Querying across languages: A dictionary-based approach to multilingual information retrieval. In Hans-Peter Frei, Donna Harman, Peter Schäuble, and Ross Wilkinson, editors, Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '96). ACM Press, New York, pages 49-57.
31. Hull, David, Paul B. Kantor, and Kwong Bor Ng. 1999. Advanced approaches to the statistical analysis of TREC information retrieval experiments. Unpublished report, Rutgers University, New Brunswick, NJ.
32. Ide, Nancy, G. Priest-Dorman and Jean Véronis. 1995. Corpus encoding standard. Available at 〈http://www.cs.vassar.edu/CES/〉.
33. Isabelle, Pierre, Michel Simard, and Pierre Plamondon. 1998. Demo. Available at 〈http://www.rali.iro.umontreal.ca/SILC/SILC.en.cgi〉.
34. Jansen, Bernard J., Amanda Spink, Deitmar Wolfram, and Tefko Saracevic. 2001. Searching the Web: The public and their queries, Journal of the American Society for Information Science and Technology, 53(3):226-234.
35. Kraaij, Wessel. 2002. TNO at CLEF-2001: Comparing translation resources. In Carol Peters, Martin Braschler, Julio Gonzalo, and Michael Kluck, editors, Evaluation of Cross-Language Information Retrieval Systems: Second Workshop of the Cross-Language Evaluation Forum (CLEF 2001). Springer Verlag, Berlin.
36. Kraaij, Wessel and Renée Pohlmann. 1996. Viewing stemming as recall enhancement. In Hans-Peter Frei, Donna Harman, Peter Schäuble, and Ross Wilkinson, editors, Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '96). ACM Press, New York, pages 40-48.
37. Kraaij, Wessel, Renée Pohlmann, and Djoerd Hiemstra. 2000. Twenty-one at TREC-8: Using language technology for information retrieval. In Ellen M. Voorhees and Donna K. Harman, editors, The Eighth Text Retrieval Conference (TREC-8), volume 8. National Institute of Standards and Technology Special Publication 500-246, Gaithersburg, MD.
38. Kraaij, Wessel and Martijn Spitters. 2003. Language models for topic tracking. In Bruce Croft and John Lafferty, editors, Language Models for Information Retrieval. Kluwer Academic, Boston.
39. Kraaij, Wessel, Thijs Westerveld, and Djoerd Hiemstra. 2002. The importance of prior probabilities for entry page search. In Micheline Beaulieu, Ricardo Baeza-Yates, Sung Hyon Myaeng, and Kalervo Järvelin, editors, Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2002). ACM Press, New York.
40. Kwok, K. L. 1999. English-Chinese cross-language retrieval based on a translation package. In Workshop: Machine Translation for Cross Language Information Retrieval, Singapore, Machine Translation Summit VII, pages 8-13.
41. Lafferty, John and Chengxiang Zhai. 2001a. Document language models, query models, and risk minimization for information retrieval. In W. Bruce Croft, David J. Harper, Donald H. Kraft, and Justin Zobel, editors, Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001). ACM Press, New York.
42. Lafferty, John and Chengxiang Zhai. 2001b. Probabilistic IR models based on document and query generation. In Jamie Callan, Bruce Croft, and John Lafferty, editors, Proceedings of the Workshop on Language Modeling and Information Retrieval, Pittsburgh.
43. Laffling, John. 1992. On constructing a transfer dictionary for man and machine. Target, 4(1):17-31.
44. Lavrenko, Victor, Martin Choquette, and W. Bruce Croft. 2002. Cross-lingual relevance models. In Micheline Beaulieu, Ricardo Baeza-Yates, Sung Hyon Myaeng, and Kalervo Järvelin, editors, Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2002). ACM Press, New York.
45. Lavrenko, Victor and W. Bruce Croft. 2001. Relevance-based language models. In W. Bruce Croft, David J. Harper, Donald H. Kraft, and Justin Zobel, editors, Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001). ACM Press, New York.
46. Lehtokangas, Raija and Eija Airio. 2002. Translation via a pivot language challenges direct translation in CLIR. In Proceedings of the SIGIR 2002 Workshop: Cross-Language Information Retrieval: A Research Roadmap, Tampere, Finland.
47. Ma, Xiaoyi. 1999. Parallel text collections at the Linguistic Data Consortium. In Machine Translation Summit VII, Singapore.
48. McNamee, Paul and James Mayfield. 2001. A language-independent approach to European text retrieval. In Carol Peters, editor, Cross-Language Information Retrieval and Evaluation. Lecture Notes in Computer Science 2069. Springer Verlag, Berlin.
49. McNamee, Paul and James Mayfield. 2002. Comparing cross-language query expansion techniques by degrading translation reources. In Micheline Beaulieu, Ricardo Baeza-Yates, Sung Hyon Myaeng, and Kalervo Järvelin, editors, Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2002). ACM Press, New York.
50. Miller, David R. H., Tim Leek, and Richard M. Schwartz. 1999. A hidden Markov model information retrieval system. In Marti Hearst, Fred Gey, and Richard Tong, editors, Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '99). ACM Press, New York, pages 214-221.
51. Ng, Kenney. 2000. A maximum likelihood ratio information retrieval model. In Ellen M. Voorhees and Donna K. Harman, editors, The Eighth Text Retrieval Conference (TREC-8), volume 8. National Institute of Standards and Technology Special Publication 500-246, Gaithersburg, MD.
52. Nie, Jian-Yun. 2002. Query expansion and query translation as logical inference. Journal of the American Society for Information Science and Technology, 54(4):340-351.
53. Nie, Jian-Yun and Jian Cai. 2001. Filtering noisy parallel corpora of Web pages. In IEEE Symposium on NLP and Knowledge Engineering, Tucson, AZ, pages 453-458.
54. Nie, Jian-Yun and Jean-François Dufort. 2002. Combining words and compound terms for monolingual and cross-language information retrieval. In Proceedings of Information 2002, Beijing, pages 453-458.
55. Nie, Jian-Yun, Michel Simard, Pierre Isabelle, and Richard Durand. 1999. Cross-language information retrieval based on parallel texts and automatic mining of parallel texts from the Web. In Marti Hearst, Fred Gey, and Richard Tong, editors, Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '99). ACM Press, New York, pages 74-81.
56. Pirkola, Ari. 1998. The effects of query structure and dictionary setups in dictionary-based cross-language information retrieval. In W. Bruce Croft, Alistair Moffat, C. J. "Keith" van Rijsbergen, Ross Wilkinson, and Justin Zobel, editors, Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '98). ACM Press, New York, pages 55-63.
57. Ponte, Jay M. and W. Bruce Croft. 1998. A language modeling approach to information retrieval. In W. Bruce Croft, Alistair Moffat, C. J. "Keith" van Rijsbergen, Ross Wilkinson, and Justin Zobel, editors, Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '98). ACM Press, New York, pages 275-281.
58. Porter, Martin F. 1980. An algorithm for suffix stripping. Program, 14(3):130-137.
59. Ragget, Dave. 1998. Clean up your Web pages with HTML TIDY. Available at 〈http://www.w3.org/People/Raggett/tidy/〉.
60. Resnik, Philip. 1998. Parallel stands: A preliminary investigation into mining the Web for bilingual text. In Proceedings of AMTA. Lecture Notes in Computer Science 1529. Springer, Berlin.
61. Robertson, Stephen E. and Steve Walker. 1994. Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. In W. Bruce Croft and C. J. "Keith" van Rijsbergen, editors, Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '94), pages 232-241. ACM Press, New York.
62. Salton, G. and M. J. McGill. 1983. Introduction to Modern Information Retrieval. McGraw-Hill, New York.
63. Savoy, Jacques. 2002. Report on CLEF-2001 experiments. In Carol Peters, Martin Braschler, Julio Gonzalo, and Michael Kluck, editors, Evaluation of Cross-Language Information Retrieval Systems: Second Workshop of the Cross-Language Evaluation Forum (CLEF 2001). Springer Verlag, Berlin.
64. Sheridan, Paraic, Jean Paul Ballerini, and Peter Schäuble. 1998. Building a large multilingual text collection from comparable news documents. In Gregory Grefenstette, editor, Cross-Language Information Retrieval. Kluwer Academic, pages 137-150.
65. Simard, Michel, George Foster, and Pierre Isabelle. 1992. Using Cognates to Align Sentences in Bilingual Corpora. In Proceedings of the Fourth Conference on Theoretical and Methodological Issues in Machine Translation (TMI), pages 67-82, Montréal, Québec.
66. Spitters, Martijn and Wessel Kraaij. 2001. Using language models for tracking events of interest over time. In Proceedings of the Workshop on Language Models for Information Retrieval (LMIR2001), Pittsburgh.
67. Tague-Sutcliffe, Jean and James Blustein. 1995. A statistical analysis of the TREC-3 data. In Donna K. Harman, editor, The Third Text Retrieval Conference, volume 4. National Institute of Standards and Technology Special Publication 500-236, Gaithersburg, MD, pages 385-398.
68. Véronis, Jean, editor. 2000. Parallel Text Processing. Kluwer Academic, Dordrecht, the Netherlands.
69. Voorhees, Ellen M. 1998. Variations in relevance judgements and the measurement of retrieval effectiveness. In W. Bruce Croft, Alistair Moffat, C. J. "Keith" van Rijsbergen, Ross Wilkinson, and Justin Zobel, editors, Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '98). ACM Press, New York, pages 315-323.
70. Xu, Jinxi, Ralph Weischedel, and Chanh Nguyen. 2001. Evaluating a probabilistic model for cross-lingual information retrieval. In W. Bruce Croft, David J. Harper, Donald H. Kraft, and Justin Zobel, editors, Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001). ACM Press, New York.
71. Yang, Yiming, Jaime G. Carbonell, Ralph Brown, and Robert E. Frederking. 1998. Translingual information retrieval: Learning from bilingual corpora. Artificial Intelligence Journal, 103(1-2):323-345.
72. Zhai, ChengXiang and John Lafferty. 2002. Two-stage language models for information retrieval. In Micheline Beaulieu, Ricardo Baeza-Yates, Sung Hyon Myaeng, and Kalervo Järvelin, editors, Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2002). ACM Press, New York.