Statistical query translation models for cross-language information retrieval

ACM Transactions on Asian Language Information Processing

Volumn 5, Issue 4, 2006, Pages 323-359

  Gao, Jianfeng ^a   Nie, Jian Yun ^b   Zhou, Ming ^c  

^a MICROSOFT RESEARCH   (United States)

^b UNIVERSITÉ DE MONTRÉAL   (Canada)

^c MICROSOFT RESEARCH ASIA   (China)

Author keywords

CLIR; Linguistic structures; Query translation; Statistical models

Indexed keywords

CROSS LANGUAGE INFORMATION RETRIEVAL (CLIR); LINGUISTIC DEPENDENCIES; LINGUISTIC STRUCTURES; QUERY TRANSLATION; STATISTICAL MODELS;

COMPUTATIONAL LINGUISTICS; QUERY LANGUAGES; QUERY PROCESSING; STATISTICAL METHODS;

INFORMATION RETRIEVAL;

EID: 34147207726     PISSN: 15300226     EISSN: 15583430     Source Type: Journal    
DOI: 10.1145/1236181.1236184     Document Type: Article

References (59)

1. ADRIANI, M. 2000. Using statistical term similarity for sense disambiguation in cross-language information retrieval. Inform, Retriev. 2, 69-80.
2. BALLESTEROS, L. AND CROFT, W. B. 1997. Phrasal translation and query expansion techniques for cross-language information retrieval. In Proceedings of the 20th International Conference on Research and Development in Information Retrieval. 84-91.
3. BALLESTEROS, L. AND CROFT, W. B. 1998. Resolving ambiguity for cross-language retrieval. In Proceedings of the 21st International Conference on Research and Development in Information Retrieval. Melbourne, Australia.
4. BIAN, G. W. AND CHEN, H. H. 1998. Integrating query translation and document translation in a cross-language information retrieval system. Machine Translation and Information Soup. Lecture Notes in Computer Science, vol. 1529, Spring-Verlag, 250-265.
5. BROWN, P. F., DELLA PIETRA, S. A., DELLA PIETRA, V. J., AND MERCER, R. L. 1993. The mathematics of statistical machine translation: Parameter estimation. Computat. Linguis. 19, 2, 263-311.
6. CHEN, A., JIANG, H., AND GEY, F. 2000. Combining multiple sources for short query translation in Chinese-English cross-language retrieval. In Proceedings of the 5th International Workshop Information Retrieval with Asian Languages. 17-23.
7. CHEN, S. F. AND GOODMAN, J. 1998. An empirical study of smoothing techniques for language modeling. Tech. rep., TR-10-98, Harvard University, Cambridge, MA.
8. CHIANG. D. 2005. A hierarchical phrase-based model for statistical machine translation. In Proceedings of the Association for Computational Linguistics (ACL'05).
9. CHURCH, K. 1988. A stochastic parts program and noun phrase parser for unrestricted text. In Proceedings of the 2nd Conference on Applied Natural Language Processing Association of Computational Linguistics. 136-143.
10. CLARKSON, P. AND ROBINSON, A. 1997. Language model adaptation using mixtures and an exponentially decaying cache. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'97). 799-802.
11. DAVIS, M. W. AND OGDEN, W. C. 1997. Free resources and advanced alignment for cross-language text retrieval. In Proceedings of the 6th Text Retrieval Conference (TREC-6). NIST, Gaithersbury, MD.
12. DING, Y. AND PALMER, M. 2005. Machine translation using probabilistic synchronous dependency insertion grammars. In Proceedings of the Association for Computational Linguistics (ACL'05), 541-548.
13. DUDA, R. O., HART, P. E., AND STORK, D. G. 2001. Pattern Classification. John Wiley and Sons.
14. FAGAN, J. 1988. Experiments in automatic phrase indexing for document retrieval: A comparison of syntactic and non-syntactic methods. Tech. rep. TR87-868, Department of Computer Science, Cornell University, Ithaea, NY.
15. FOX. H. J. 2002. Phrasal cohesion and statistical machine translation. In Proceedings of the Conference on Empirical Methods in National Language Processing (EMNLP'02).
16. FUNG, P., LIU, X., AND CHEUNG, C. S. 1999. Mixed language query disambiguation. In the 37th Annual Meeting of the Association for Computational Linguistics. College Park, MD, 333-340.
17. GAO, J., GOODMAN, J., AND MIAO, J. 2001a. The use of clustering techniques for language modeling - application to Asian languages. Computat. Linguis. Chinese Lang. Process. 6, 1, 27-60.
18. GAO, J., GOODMAN, J., LI, M., AND LEE, K. F. 2002a. Toward a unified approach to statistical language modeling for Chinese. ACM Trans. Asian Lang. Inform. Process. 1, 1, 3-33.
19. GAO, J., LI, M., WU, A., AND HUANG, C. N. 2005a. Chinese word segmentation and named entity recognition: A pragmatic approach. Comput. Linguis. 31, 4.
20. GAO, J., NIE, J. Y., HE, H., CHEN, W., AND ZHOU, M. 2002b. Resolving query translation ambiguity using a decaying co-occurrence model and syntactic dependence relations. In SIGIR 2002. 183-190.
21. GAO, J., NIE, J. Y., ZHANG, J., XUN, E., SU, Y., ZHOU, M., AND HUANG, C. 2000. TREC-9 CLIR experiments at MSRCN. In TREC-9. 343-353.
22. GAO, J., NIE, J. Y., ZHANG, J., XUN, E., ZHOU, M., AND HUANG, C. 2001b. Improving query translation for CLIR using statistical models. In ACM SIGIR2001, New Orleans, LO. 96-104.
23. GAO, J., YU, H., YUAN, W., AND XU, P. 2005b. Minimum sample risk methods for language modeling. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP'05).
24. HEIDORN, G. 2000. Intelligent writing assistance. In Dale et al., Ed. Handbook of Natural Language Processing. Marcel Dekker.
25. HIEMSTRA, D. AND JONG, F. 1999. Disambiguation strategies for cross-language information retrieval. In Proceedings of the 3rd European Conference on Research and Advanced Technology for Digital Libraries. 274-293.
26. HUANG, X., ACERO, A., AND HON, H.-W. 2001. Spoken Language Processing: A Guide to Theory, Algorithm, and System Development, Prentice Hall.
27. HULL, D. A. 1997. Using structured queries for disambiguation in cross-language information retrieval. In AAAI Symposium on Cross-Language Text and Speech Retrieval.
28. HULL, D. A. AND GREEENSTETTE, G. 1996. Querying across languages: A dictionary-based approach to multilingual information retrieval. In Research and Development in Information Retrieval, ACM-SIGIR. 46-57.
29. JANG, M. G., MYAENG, S. H., AND PARK S. Y. 1999. Using mutual information to resolve query translation ambiguities and query term weighting. In Proceedings of the Association for Computational Linguistics (ACL'99). College Park, MD, 223-229.
30. JORDAN, M. I., GHAHRAMANI, Z., JAAKKOLA, T. S., AND SAUL, L. K. 1999. An introduction to variational methods for graphical models. In Jordan, M. I., Ed. Learning in Graphical Models. MIT Press, Cambridge, MA.
31. KOEHN, P. 2003. Noun phrase translation. Ph.D. thesis, University of Southern California.
32. KOWK, K. L. 2000. Exploiting a Chinese-English bilingual wordlist for English-Chinese cross language information retrieval. In 5th International Workshop on Information Retrieval with Asian Languages (IRAL'00). Hong Kong.
33. LIN, D. 1997. Using syntactic dependency as local context to resolve word sense ambiguity. In Proceedings of the European Chapter of the Association for Computational Linguistics (EACL'97), Madrid, Spain, 64-71.
34. LIN, D. 1998. Automatic retrieval and clustering of similar words. In COLING. Montreal, Canada. 768-774.
35. LIU, Y., JIN, R., AND CHAI, J. Y. 2005. A maximum coherence model for dictionary-based cross-language information retrieval. In SIGIR 2005. 536-543.
36. MANDALA, R., TOKUNAGA, T., AND TANAKA, H. 1999. Combining multiple evidence from different types of thesaurus for query expansion. In ACM SIGIR. 191-197.
37. MARCUS, M., MABCINKIEWICX, M., AND SANTOHINI, B. 1993. Building a large annotated corpus of English: The Penn Treebank. Computat. Linguist. 19, 2, 313-330.
38. NIE, J. Y., SIMARD, M., ISABELLE, P., AND DURAND, R. 1999. Cross-language information retrieval based on parallel texts and automatic mining of parallel texts from the web. In Conference on Research and Development in Information Retrieval (SIGIR). 74-81
39. NIE, J. Y. 2003. Query expansion and query translation as logical inference. J. Amer. Soc. Inform. Science Techn. B4, 4, 335-346.
40. OCH, F. J. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the Association for Computational Linguistics (ACL'03).
41. OCH, F. J. AND NEY, H. 2004. The alignment template approach to statistical machine translation. Computat. Linguist. 30, 417-449.
42. PETERS, C. AND PICCHI, E. 1996. Cross language information retrieval: A system for comparable corpus querying. In Workshop on Cross-linguistic Information Retrieval (SIGIR'96). 24-33.
43. PRESS, W. H., TEUKOLSKY, S. A., VETTEHLING, W. T., AND FLANNERY, B. P. 1992. Numerical Recipes In C: The Art of Scientific Computing, Cambridge University Press, Cambridge, UK.
44. QUIRK, C., MENEZES, A., AND CHERRY, C. 2005. Dependency treelet translation: Syntactically informed phrasal SMT. In Proceedings of the Association for Computational Linguistics (ACL'05). 271-279.
45. RAMSHAW, L. A. AND MARCUS, M. 1998. Text chunking using transformation-based learning. In Natural Language Processing Using Very large Corpora, Kluwer.
46. RESNIK, P. AND SMITH, N. A. 2003. The Web as a parallel corpus. Computat. Linguist. 29, 349-380.
47. ROBERTSON, S. E. AND WALKER, S. 2000. Microsoft Cambridge at TREC-9: Filtering track. In TREC-9 361-368.
48. TJONG KIM SANG, E. AND BUCHHOLZ, B. S. 2000. Introduction to the CoNLL-2000 shared task: Chunking. In Proceedings of CoNLL 2000 and LLL 2000. Lisbon, Portugal. 127-132.
49. VAN RIJSBERGEN, C. J. 1979. Information Retrieval, 2nd Ed. Butterworths, London, UK.
50. VAN RIJSBERGEN, C. J. 1986. A Non-Classical Logic for Information Retrieval. Comput. J. 29, 6, 481-485.
51. VOORHEES, E. AND HARMAN, D. 2001. Overview of the ninth text retrieval conference (TREC-9). In TREC-9. 1-14.
52. WU, D. 1997. Stochastic inversion transduction grammars and bilingual parsing of parallel corpora. Computat. Linguist. 23, 3, 377-403.
53. XU, J. AND WEISCHEDEL, R. 2000a. Cross-lingual information retrieval using hidden Markov models. In Proceeding of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora. Hong Kong.
54. XU, J. AND WEISCHEDEL, R. 2000b. TREC-9 cross-language retrieval at BBN. In the 9th Text Retrieval Conference (TREC-9). NIST, Gaithersbury, MD.
55. XUN, E., ZHOU, M., AND HUANG, C. 2000. A unified statistical model for the identification of English base NP. In the 38th Annual Meeting of the Association for Computational Linguistics (ACL'00). Hong Kong.
56. YAMADA, K. AND KNIGHT, K. 2001. A syntax based statistical translation model. In Proceedings of the Association for Computational Linguistics (ACL'01).
57. ZHANG, Y. AND VINES, P. 2004. Using the web for automated translation extraction in cross-language information retrieval. In SIGIR'04. 162-169.
58. ZHANG, Y., WU, K., GAO, J., AND VINES, P. 2006. Automatic acquisition of Chinese-English parallel corpus from the web. In Proceedings of the European Conference on Information Retrieval (ECIR'06).
59. ZHOU, M., DING, Y., AND HUANG, C. 2001. Improving translation selection with a new translation model trained by independent monolingual corpora. Computat. Linguist. Chinese Lang. Process. 6, 1, 1-26.