A Comparison of Chinese Document Indexing Strategies and Retrieval Models

ACM Transactions on Asian Language Information Processing

Volumn 1, Issue 3, 2002, Pages 225-268

  Luk, Robert W P ^a   Kwok, K L ^b  

^a HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

^b CITY UNIVERSITY OF NEW YORK   (United States)

Author keywords

Algorithms; Chinese information retrieval; comparison; Experimentation; indexing strategies; Languages; Performance

Indexed keywords

EID: 34247334591     PISSN: 15300226     EISSN: 15583430     Source Type: Journal    
DOI: 10.1145/772755.772758     Document Type: Article

References (60)

1. Allan, J., Callan, J., Croft, B.W., Ballesteros, L., Broglio, J., Xu, J., and Shu, H. 1996. Inquery at TREC-5. In Proceedings of the TREC-5 Conference (Gaithersburg, MD), 119-132
2. Allan, J., Callan, J., Croft, B.W., Ballesteros, L., Byrd, D., and Xu, J. 1997. Inquery at TREC-6. In Proceedings of the TREC-6 Conference (Gaithersburg, MD), 169-206
3. Allan, J., Connell, M.E., Croft, W.B., Feng, F-F., Fisher, D., and Li, X. 2000. INQUERY and TREC-9. In Proceedings of the TREC-9 Conference (Gaithersburg, MD), 1-12
4. Beaulieu, M. M., Gatford, M., Huang, X., Robertson, S. E., Walker, S., and Williams, P. 1996. Okapi at TREC-5. In Proceedings of the TREC-5 Conference (Gaithersburg, MD), 143-166
5. Chan, S. K., Wong, C. Y., and Luk, R. W. P. 1998. Variable bit-block compression signature for English-Chinese information retrieval. In Proceedings of the Conference on Information Retrieval with Asian Languages 98 (KRDL, National University of Singapore), 61-66
6. Chen, A., Gey, F. C., and Jiang, H. 2001. Berkeley at NTCIR-2: Chinese, Japanese, and English IR experiments. In Proceedings of the 2nd NTCIR Workshop on Research in Chinese & Japanese Text Retrieval and Text Summarization (National Institute of Informatics, Tokyo, Japan), 137-145
7. Chen, A., He, J., Xu, L., Gey, F. C., and Meggs, J. 1997. Chinese text retrieval without using a dictionary. In Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (Philadelphia, PA), 42-49
8. Chen, K.J., and Huang, C. R., Eds. 1993. Chinese word class analysis. Tech. Rep. 93-05, Chinese Knowledge Information Processing Group, Institute of Information Science, Academia Sinica, Taiwan
9. Chen, A., Jiang, H., and Gey, F. C. 2000. English-Chinese cross-language IR using bilingual dictionaries. In Proceedings of the TREC-9 Conference (Gaithersburg, MD), 15-21
10. Chien, L.-F. 1994. A model-based signature file approach for full-text retrieval of Chinese document databases. Comput. Process. Chinese Oriental Lang. 8, Supplement, 59-76
11. Chien, L.-F. 1997. PAT-tree-based keyword extraction for Chinese information retrieval. In Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, (Philadelphia, PA), 50-58
12. Chinese National Standards. 1994. GB13715: Information processing based on modern Chinese word segmentation. Tsinghua University Press, Beijing
13. Chow, K.C.W., Luk, R.W.P., Wong, K.F., and Kwok, K.L. 2001. Hybrid term indexing for weighted Boolean and vector space models. Int, J. Comput. Process. Chinese Oriental Lang. 14, 2, 133-152
14. Cooper, W., Gey, F. C., and Chen, A. 1992. Probabilistic retrieval in the TIPSTER collections: an application of staged logistic regression. In Proceedings of the TREC-1 Conference (Gaithersburg, MD), 73-88
15. Cooper, W. S., Chen, A., and Gey, F. C. 1994. Full text retrieval based on probabilistic equations with coefficients fitted by logistic regression. In Proceedings of theTREC-2 Conference (Gaithersburg, MD), 57-66
16. Croft, W. B., Adharper, D. J. 1979. Using probabilistic models of document retrieval without relevance. J. Doc. 35, 285-295
17. D'Souza, D., Fuller, M., Thom, J., Vines, P., Zobel, J., De Kretser, O., Wilkinson, R., and Wu, M. 2000. Melbourne TREC-9 experiments. In Proceedings of the TREC-9 Conference (Gaithersburg, MD), 437-452
18. Gao, J., Nie, J.-Y., Zhang, J., Xun, E., Su, Y., Zhou, M., and Huang, C. 2000. TREC-9 CLIR and experiments at MSRCN. In Proceedings of the TREC-9 Conference (Gaithersburg, MD), 343-354
19. Gey, F. C. Adchen, A. 1997. Phrase discovery for English and cross-language retrieval at TREC-6. In Proceedings of theTREC-6 Conference (Gaithersburg, MD), 637-648
20. He, J., L. Xu, Chen, A., Meggs, J., and Gey, F. C. 1996. Berkeley Chinese information retrieval at TREC-5: technical report. In Proceedings of the TREC-5 Conference (Gaithersburg, MD), 181-186
21. Huang, C. -R., Chen, K. -J., Chen, F. -Y., and Chang, L. -L. 1997. Segmentation standard for Chinese natural language processing. Comput. Linguist. Chinese Natural Lang. Process. 2, 2, 47-62
22. Huang, X., and Robertson, S. E. 1997. Okapi Chinese text retrieval experiments at TREC-6. In Proceedings of the TREC-6 Conference (Gaithersburg, MD), 137-142
23. Jin, H., and Wong, K. F. 2000. TREC-9 CLIR at CUHK: Disambiguation by similarity values between adjacent words. In Proceedings of the TREC-9 Conference (Gaithersburg, MD), 151-156
24. Kando, N. 2002. NTCIR (NII-NACSIS test collection for IR systems). Project NTCIR Home
25. Kit, C., Liu, Y., and Liang, N. 1989. On methods of Chinese automatic word segmentation. J. Chinese Inf. Process. 3, 1, 13-20
26. Kwok, K.L. 1996. A network approach to probabilistic information retrieval. ACM Trans. Inf. Syst. 13, 325-353
27. Kwok, K. L. 1997. Comparing representations in Chinese information retrieval. In Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (Philadelphia, PA), 34-41
28. Kwok, K. L. 1999. Employing multiple representations for Chinese information retrieval. J. Am. Soc. Inf. Sci. 50, 8, 709-723
29. Kwok, K. L. 2001. NTCIR-2 Chinese, cross language retrieval experiments using Pircs. In Proceedings of the 2“’ NTCIR Workshop on Research in Chinese & Japanese Text Retrieval and Text Summarization (National Institute of Informatics, Tokyo, Japan), 111-118
30. Kwok, K. L. and Grunfield, L. 1996. TREC-5 English and Chinese retrieval experiments using Pircs. In Proceedings of the TREC-5 Conference (Gaithersburg, MD), 1-10
31. Kwok, K. L., Grunfeld, L., Dinstl, N., and Chan, M. 2000. TREC-9 cross language, web and question-answering track experiments using Pircs. In Proceedings of the TREC-9 Conference (Gaithersburg, MD), 419-429
32. Kwok, K. L., Grunfield, L., and Xu, J. H. 1997. TREC-6 English and Chinese retrieval experiments using Pircs. In Proceedings of the TREC-6 Conference (Gaithersburg, MD), 207-214
33. Lam, W., Wong, K.F., and Wong, C.-Y. 2001. Chinese document indexing based on a new partitioned signature file: model and evaluation. J. Am. Soc. Inf. Sci. 52, 7, 584-597
34. Lam, W., Wong, C. -Y., and Wong, K. F. 1997. Performance evaluation of character-, word- and n-gram-based indexing for Chinese text retrieval. In Proceedings of the Information Retrieval with Asian Languages 97 Conference (Ibaraki-ken, Japan), 68-80
35. Lee, C. -F. 1997. Chinese information retrieval: using characters or words. Inf. Process. Manage. 35. 443-462
36. Lee, D. L., Chuang, H., and Seamons, K. 1997. Document ranking and the vector-space model. IEEE Software 14, 2, 67-75
37. Leong, M. -K. and Zhou, H. 1997. Preliminary qualitative analysis of segmented vs bigram indexing in Chinese. In Proceedings of the TREC-6 Conference (Gaithersburg, MD, Nov.), 19-21
38. Lin, D., Zhang, Y., Sun, L., Sun, Y., and Han, J. 2000. PM-based indexing for Chinese text retrieval. In Proceedings of the Information Retrieval with Asian Language. 2000 Conference (Hong Kong), 55-60
39. Luk, R. W. P., Wong, K. F., and Kwok, K. L. 2001. Hybrid term indexing: an evaluation. In Proceedings of the 2“’ NTCIR Workshop on Research in Chinese & Japanese Text Retrieval and Text Summarization, (National Institute of Informatics, Tokyo, Japan), 130-136
40. Mcnamee, P., Mayfield, J., and Piatko, C. 2000. The haircut system at TREC-9. In Proceedings of the TREC-9 Conference (Gaithersburg, MD), 273-294
41. Mcnamee, P. 2001. Experiments in the retrieval of unsegmented Japanese text at the NTCIR-2 workshop. In Proceedings of the 2nd NTCIR Workshop on Research in Chinese & Japanese Text Retrieval and Text Summarization (National Institute of Informatics, Tokyo, Japan), 263-269
42. Murata, M., Utiyama, M., Ma, Q., Ozaku, H., and Isahara, H. CRL At NTCIR2. In Proceedings of the 2nd NTCIR Workshop on Research in Chinese & Japanese Text Retrieval and Text Summarization (National Institute of Informatics, Tokyo, Japan), 119-130
43. Nie, J-Y., Chevallet, J. -P., and Bruandet, M. -F. 1997. Between terms and words for European language IR and between words and bigrams for Chinese IR. In Proceedings of the TREC-6 Conference (Gaithersburg, MD), 697-710
44. Nie, J.-Y., Gao, J., Zhang, J., and Zhou, M. 2000. On the use of words and n-grams for Chinese information retrieval. In Proceedings of the Information Retrieval with Asian Languages 2000 Conference (Hong Kong), 141-148
45. Nie, J-Y., and Ren, F. 1997. Chinese information retrieval: using characters or words. Inf. Process. Manage. 35, 443-462
46. Robertson, S. E. and Sparck Jones, K. 1976. Relevance weighting of search terms. J. Am. Soc. Inf. Sci. 27, 129-146
47. Robertson, S. E., and Walker, S. 1992. Some simple effe ctive approximations to the 2-Poisson model for probabilistic weighted retrieval. In Proceedings of the 15t’’ Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (Philadelphia, PA), 232-241
48. Robertson, S., Walker, S., Jones, S., Hancock-Beaulieu, M., and Gatford, M. 1993. Okapi at TREC-2. In Proceedings of the TREC-2 Conference (Gaithersburg, MD}, 21-25
49. Robertson, S. E., Walker, S., Jones, S., Hancock-Beaulieu, M., and Gatford, M. 1994. Okapi at TREC- 3. In Proceedings of the TREC-3 Conference (Gaithersburg, MD), 109-128
50. Salton, G. and Buckley, C. 1998. Term-weighting approaches in automatic text retrieval. Inf. Process. Manage. 24, 5, 513-523
51. Shishibori, M., Fiketa, M., Ando, K., and Aoe, J. -I. 1997. A construction method for the index represented by a pointerless patricia trie. In Proceedings of the Information Retrieval with Asian Languages 97 Conference (Ibaraki-ken, Japan)
52. Spink, A., Jansen, B. J., Wolfram, D., and Sap. Acevic, T. 2002. From e-sex to e-commerce: web search changes. IEEE Computer 35, 3, 107-109
53. Sproat, R. and Shih, C. 1990. A statistical method for finding word boundaries in Chinese text. Comput. Process. Chinese Oriental Lang. 4, 4, 336-351
54. Tsang, T. F., Luk, R. W. P., and Wong, K. F. 1999. Hybrid term indexing using words and bigrams. In Proceedings of the Information Retrieval with Asian Languages 1999 Conference (Academia Sinica, Taiwan), 112-117
55. Vines, P. and Zobel, J. 1999. Efficient building and querying of Asian language document databases. In Proceedings of the Information Retrieval with Asian Languages 1999 Conference (Academia Sinica, Taiwan), 118-125
56. Voorhees, E. M. and Harman, D. K. 2002. Text Retrieval Conferences
57. Witten, I. H., Moffat, A., and Bell, T. C. 1999. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann, San Francisco, CA
58. Wu, D. and Fung, P. 1994. Improving Chinese tokenization with linguistic filters on statistical lexical acquisition, In Proceedings of the 4“’ Conference on Applied Natural Language Processing (Stuttgart, Germany), 180-181
59. Xu, J. and Weischedel, R. 2000. TREC-9 Cross-lingual retrieval at BNN. In Proceedings of the TREC-9 Conference, 551-561
60. Zhang, Y. Sun, L., Du, L., Jin, Y., and Sun, Y. 2001. ISCAS’ Text retrieval in NTCIR workshop II. In Proceedings of the 2nd NTCIR Workshop on Research in Chinese & Japanese Text Retrieval and Text Summarization (National Institute of Informatics, Tokyo, Japan), 146-153