A large scale ranker-based system for search query spelling correction

Coling 2010 - 23rd International Conference on Computational Linguistics, Proceedings of the Conference

Volumn 2, Issue , 2010, Pages 358-366

  Gao, Jianfeng ^a   Li, Xiaolong ^b   Micol, Daniel ^b   Quirk, Chris ^a   Sun, Xu ^c  

^a MICROSOFT RESEARCH   (United States)

^b MICROSOFT   (United States)

^c UNIVERSITY OF TOKYO   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

BASELINE METHODS; DISTRIBUTED INFRASTRUCTURE; ERROR MODEL; MULTI-WORD; N-GRAM LANGUAGE MODELS; NOISY CHANNEL; SEARCH LOGS; SEARCH QUERIES; SPELLING CORRECTION; WRITTEN TEXTS;

COMPUTATIONAL LINGUISTICS; USER INTERFACES;

PROBABILITY DISTRIBUTIONS;

EID: 80052127939     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. Agichtein, E., Brill, E. and Dumais, S. 2006. Improving web search ranking by incorporating user behavior information. In SIGIR, pp. 19-26.
2. Ahmad, F., and Kondrak, G. 2005. Learning a spelling error model from search query logs. In HLTEMNLP, pp. 955-962.
3. Brants, T., and Franz, A. 2006. Web 1T 5-gram corpus version 1.1. Technical report, Google Research.
4. Brants, T., Popat, A. C., Xu, P., Och, F. J., and Dean, J. 2007. Large language models in machine translation. In EMNLP-CoNLL, pp. 858 - 867.
5. Brill, E., and Moore, R. C. 2000. An improved error model for noisy channel spelling correction. In ACL, pp. 286-293.
6. Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, and Hullender, G. 2005. Learning to rank using gradient descent. In ICML, pp. 89-96. (Pubitemid 43183320)
7. Chaiken, R., Jenkins, B., Larson, P., Ramsey, B., Shakib, D., Weaver, S., and Zhou, J. 2008. SCOPE: easy and efficient parallel processing f massive data sets. In Proceedings of the VLDB Endowment, pp. 1265-1276.
8. Charniak, E. 2001. Immediate-head parsing for language models. In ACL/EACL, pp. 124-131.
9. Chen, S. F., and Goodman, J. 1999. An empirical study of smoothing techniques for language modeling. Computer Speech and Language, 13(10):359-394. (Pubitemid 30518216)
10. Chen, Q., Li, M., and Zhou, M. 2007. Improving query spelling correction using web search results. In EMNLP-CoNLL, pp. 181-189.
11. Church, K., Hard, T., and Gao, J. 2007. Compressing trigram language models with Golomb coding. In EMNLP-CoNLL, pp. 199-207.
12. Cucerzan, S., and Brill, E. 2004. Spelling correction as an iterative process that exploits the collective knowledge of web users. In EMNLP, pp. 293-300.
13. Freund, Y. and Schapire, R. E. 1999. Large margin classification using the perceptron algorithm. In Machine Learning, 37(3): 277-296. (Pubitemid 32210619)
14. Gao, J., Goodman, J., and Miao, J. 2001. The use of clustering techniques for language modeling - application to Asian languages. Computational Linguistics and Chinese Language Processing, 6(1):27-60, 2001.
15. Gao, J., Yuan, W., Li, X., Deng, K., and Nie, J-Y. 2009. Smoothing clickthrough data for web search ranking. In SIGIR, pp. 355-362.
16. Golding, A. R., and Roth, D. 1996. Applying winnow to context-sensitive spelling correction. In ICML, pp. 182-190.
17. Joachims, T. 2002. Optimizing search engines using clickthrough data. In SIGKDD, pp. 133-142.
18. Kernighan, M. D., Church, K. W., and Gale, W. A. 1990. A spelling correction program based on a noisy channel model. In COLING, pp. 205-210.
19. Koehn, P., Och, F., and Marcu, D. 2003. Statistical phrase-based translation. In HLT/NAACL, pp. 127-133.
20. Kucich, K. 1992. Techniques for automatically correcting words in text. ACM Computing Surveys, 24(4):377-439.
21. Levenshtein, V. I. 1966. Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady, 10(8):707-710.
22. Li, M., Zhu, M., Zhang, Y., and Zhou, M. 2006. Exploring distributional similarity based models for query spelling correction. In ACL, pp. 1025-1032.
23. Mangu, L., and Brill, E. 1997. Automatic rule acquisition for spelling correction. In ICML, pp. 187-194.
24. Microsoft Microsoft web n-gram services. 2010. http://research.microsoft. com/web-ngram.
25. Nguyen, P., Gao, J., and Mahajan, M. 2007. MSRLM: a scalable language modeling toolkit. Technical report TR-2007-144, Microsoft Research.
26. Och, F. 2002. Statistical machine translation: from single-word models to alignment templates. PhD thesis, RWTH Aachen.
27. Och, F., and Ney, H. 2004. The alignment template approach to statistical machine translation. Computational Linguistics, 30(4): 417-449. (Pubitemid 41399543)
28. Philips, L. 1990. Hanging on the metaphone. Computer Language Magazine, 7(12):38-44.
29. Sun, X., Gao, J., Micol, D., and Quirk, C. 2010. Learning phrase-based spelling error models from clickthrough data. In ACL.
30. Toutanova, K., and Moore, R. 2002. Pronunciation modeling for improved spelling correction. In ACL, pp. 144-151.
31. Whitelaw, C., Hutchinson, B., Chung, G. Y., and Ellis, G. 2009. Using the web for language independent spellchecking and autocorrection. In EMNLP, pp. 890-899.
32. Zhang, Y., Hildebrand, Al. S., and Vogel, S. 2006. Distributed language modeling for n-best list reranking. In EMNLP, pp. 216-233.