Chinese word segmentation and its effect on information retrieval

Information Processing and Management

Volumn 40, Issue 1, 2004, Pages 161-190

  Foo, Schubert ^a   Li, Hui ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Chinese; Information retrieval; Retrieval effectiveness; Word segmentation

Indexed keywords

DATA PROCESSING; INFORMATION RETRIEVAL; QUERY LANGUAGES; STATISTICAL TESTS; WORD PROCESSING;

IDEOGRAPHIC CHARACTERS; RETRIEVAL EFFECTIVENESS; WORD SEGMENTATION;

INFORMATION MANAGEMENT;

EID: 0344496050     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0306-4573(02)00079-1     Document Type: Article

References (55)

1. Allan, J., et al. (1996). INQUERY at TREC-5. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
2. Allan, J., et al. (1997). INQUERY does battle with TREC-6. Available: http://trec.nist.gov/pubs/trec6/t6_proceedings.html, Maryland.
3. Beaulieu, M., et al. (1996). Okapi at TREC-5. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
4. Chen, A., Jiang, H., & Gey, F. (2000). English-Chinese cross-language IR using bilingual dictionaries. Available: http://trec.nist.gov/pubs/trec9/t9_proceedings.html, Maryland.
5. City University. (1996). Chinese results. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
6. Claritech Corporation. (1996). Chinese results. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
7. Cornell University. (1996). Chinese results. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
8. Cuny (1996). Chinese results. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
9. Dai, Y.B. (1997). Developing a new statistical method for Chinese text segmentation. First year report of Master of Science, School of Applied Science, Nanyang Technological University, Singapore.
10. Foo S.B., Li H. An integrated bigram approach with single-character word list for Chinese word segmentation. TEXT Technology. 8(4):1998;17-28.
11. Franz, M., McCarley, J. S., & Zhu, W.-J. (2000) English-Chinese information retrieval at IBM. Available: http://trec.nist.gov/pubs/trec9/papers/ibm_clir.pdf, Maryland.
12. Fuller, M., et al. (1997). MDS TREC6 report. Available: http://trec.nist.gov/pubs/trec6/t6_proceedings.html, Maryland.
13. Gao, J., Xun, E., Zhou, M., Huang, H., Nie, J.-Y., Zhang, J.-Y., & Su, Y. (2000). TREC-9 CLIR experiments at MSRCN. Available: http://trec.nist.gov/pubs/trec9/papers/trec-9.pdf, Maryland.
14. George Manson University. (1996). Chinese results. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
15. Harman, D. (1993). Overview of the second text retrieval conference. Available: http://trec.nist.gov/pubs/trec6/t6_proceedings.html, Maryland.
16. He, J., Xu, J., Chen, A., Meggs, J., & Gey, F. C. (1996). Berkeley Chinese information retrieval at TREC-5: Technical report. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
17. Huang, X. J., & Robertson, S. E. (1997). Okapi Chinese text retrieval experiments at TREC-6. Available: http://trec.nist.gov/pubs/trec6/t6_proceedings.html, Maryland.
18. Hull, D. (1993). Using statistical testing in the evaluation of retrieval experiments. In Proceedings of SIGIR'93 (pp. 329-338), Zurich, USA.
19. Information Technology Institute. (1997). Chinese results. Available: http://trec.nist.gov/pubs/trec6/t6_proceedings.html, Maryland.
20. Jie C.Y., Liu Y., Liang N.Y. The design and realization Chinese automatic segmenting system CASS. Journal of Chinese Information Processing. 5(4):1991;27-34. (in Chinese).
21. Kwok, K. L. (1997a). Comparing representations in Chinese information retrieval. Available: http://ir.cs.qc.edu/#publi_.
22. Kwok, K. L. (1997b). Lexicon effects on Chinese information retrieval. Available: http://ir.cs.qc.edu/#publi_.
23. Kwok, K. L., & Grunfeld, L. (1996). TREC-5 English and Chinese retrieval experiments using PIrCS. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
24. Lane, D. (1997). MG pages. Available: http://www.mds.rmit.edu.au/mg.
25. Leong, M. K., & Zhou, H. (1997). Preliminary qualitative analysis of segmented vs bigram indexing in Chinese. Available: http://trec.nist.gov/pubs/trec6/t6_proceedings.html, Maryland.
26. Leung C.H., Kan W.K. Parallel Chinese word segmentation algorithm based on maximum matching. Neural, Parallel and Science Computations. 4(3):1996;291-303.
27. Lim, H. K. (1999). Chinese text retrieval system. Thesis of Master of Applied Science, School of Applied Science, Nanyang Technological University, Singapore.
28. Liu Y. The rules of modern Chinese segmentation for the purpose of information processing and approaches of automatic Chinese segmentation. 1994;36-63 Tsinghua University Press, Beijing. (in Chinese).
29. Mateev, B., Munteanu, E., Sheridan, P., Wechsler, M., & Schuble, P. (1997). ETH TREC-6: routing, Chinese, cross-language and spoken document retrieval. Available: http://trec.nist.gov/pubs/trec6/t6_proceedings.html, Maryland.
30. McNamee, P., Mayfield, J., & Piatko, C. (2000). The HAIRCUT system at TREC-9. Available: http://trec.nist.gov/pubs/trec9/papers/jhuapl.pdf, Maryland.
31. Motulsky, H. (1995). Intuitive biostatistics. Available: http://www.graphpad.com/www/pvalue.htm.
32. Ngo, C. W., & Lai, K. F. (1996). Experiments on routing, filtering and Chinese text retrieval in TREC-5. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
33. Nie, J. Y., Brisebois, M., & Ren, X. B. (1996). On Chinese text retrieval. In Proceedings of SIGIR'96 (pp. 225-233), Zurich, Switzerland.
34. Nie, J. Y., Chevallet, J. P., & Bruandet, M. F. (1997). Between terms and words for European language Ir and between words and bigrams for Chinese IR. Available: http://trec.nist.gov/pubs/trec6/t6_proceedings.html, Maryland.
35. Nie, J.-Y., Gao, J., Zhang, J., & Zhou, M. (2000). On the use of words and N-grams for Chinese information retrieval. In Proceedings of IRAL2000, Fifth international workshop on information retrieval with Asian languages, Hong Kong.
36. rijsbergen C.J. Information retrieval. 1997;Butterworths, London.
37. Royal Melbourne Institute of Technology. (1996). Chinese results. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
38. Royal Melbourne Institute of Technology. (1997). Chinese results. Available: http://trec.nist.gov/pubs/trec6/t6_proceedings.html, Maryland.
39. Salton G., McGill M.J. Introduction to modern information retrieval. 1983;McGraw-Hill Book Company, New York.
40. Smeaton, A., & Wilkinson, R. (1996). Spanish and Chinese document retrieval in TREC-5. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
41. Sproat R., Shih C. A statistical method for finding word boundaries in Chinese text. Computer Processing of Chinese and Oriental Languages. 4(4):1990;336-351.
42. Sproat R., Shih C. A stochastic finite-state word-segmentation algorithm for Chinese. Computational Linguistics. 22(3):1996;377-404.
43. SPSS. (2002). SPSS 11.0. Available: http://www.spss.com/spssbi/spss/.
44. Tong, X., Zai, C., Milic-Frayling, C., & Evans, D. A. (1996). Experiments on Chinese text indexing - CLARIT TREC-5 Chinese track report. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
45. University of California, Berkeley (1996). Chinese results. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
46. University of Massachusetts (1996). Chinese results. Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland.
47. University of Massachusetts (1997). Chinese results. Available: http://trec.nist.gov/pubs/trec6/t6_proceedings.html, Maryland.
48. Voorhees, E. M., & Harman, D. (1996). Overview of the fifth text retrieval conference (TREC-5). Available: http://trec.nist.gov/pubs/trec5/t5_proceedings.html, Maryland, 1996.
49. Wang Y.C., Su H.J., Mo Y. Automatic processing of Chinese words. Journal of Chinese Information Processing. 4(4):1990;1-11. (in Chinese).
50. Wilkinson, R. (1997). Chinese document retrieval at TREC-6. Available: http://trec.nist.gov/pubs/trec6/t6_proceedings.html, Maryland.
51. Witten I.H., Moffat A., Bell T.C. Managing gigabytes: compressing and indexing documents and images. 1994;Van Nostrand reinhold, New York.
52. Wu, L., Huang, X.-J., Guo, Y., Liu, B., & Zhang, Y. (2000). FDU at TREC-9: CLIR, Filtering and QA Tasks. Available: http://trec.nist.gov/pubs/trec9/papers/FduT9report.pdf, Maryland.
53. Wu Z.M., Tseng G. Chinese text segmentation for text retrieval: achievements and problems. Journal of the American Society for Information Science. 44(9):1993;532-542.
54. Wu Z.M., Tseng G. ACTS: An automatic Chinese text segmentation system for full text retrieval. Journal of the American Society for Information Science. 46(2):1995;83-96.
55. Xu, J., & Weischedel, R. (2000). TREC-9 cross lingual retrieval at BBN. Available: http://trec.nist.gov/pubs/trec9/papers/bbn-trec9.pdf, Maryland.