A new PPM variant for Chinese text compression

Natural Language Engineering

Volumn 14, Issue 3, 2008, Pages 417-430

  Wu, Peiliang ^a   Teahan, W J ^a  

^a BANGOR UNIVERSITY   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

CHARACTER RECOGNITION; PATTERN MATCHING; PROBABILITY; TEXT PROCESSING;

PREDICTION BY PARTIAL MATCH MODEL; TEXT COMPRESSION;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 44949246419     PISSN: 13513249     EISSN: 14698110     Source Type: Journal    
DOI: 10.1017/S1351324907004597     Document Type: Article

References (24)

1. Bassiou, N., and Kotropoulos, C. 2005. Interpolated distanced bigram language models for robust word clustering. In International Workshop on Nonlinear Signal and Image Processing (NSIP 2005).
2. Bell, T. C., Cleary, J. G., and Witten, I. H. 1990. Text Compression Upper Saddle Rivee, NJ: Prentice Hall.
3. Bodden, E., Clasen, M., and Kneis, J. 2004. Arithmetic Coding revealed. Germany: RWTH Aachen University.
4. Burrows, M., and Wheeler, D. J. 1994. A block-sorting lossless data compression algorithm. Technical Report 124, Digital Equipment Corporation, Palo Alto, CA.
5. Cheng, K.-S., Young, G. H., and Wong, K.-F. 1999. A study of word-based and integralbit chinese text compression algorithms. Journal of the American Society for information Science 50(3):218-228.
6. Cleary, J. G., and Witten, I. H. 1984. Data compression using adaptive coding and partial string matching. IEEE Transactions on Communication COM-32(4):396-402.
7. Cleary, J. G., and Teahan, W. J. 1997. Unbounded length contexts for PPM. The Computer Journal 40(2/3):67-75.
8. Gu, H. Y. 1995. Large-alphabet Chinese text compression using adaptive Markov model and arithmetic coder. Computer Processing of Chinese and Oriental Languages 9(2):111-124
9. Gu, H.-Y. 2005. A large-alphabet-oriented scheme for Chinese and English text compression. Software Practice and Experience 35 1027-1039.
10. Jelinek, F. 19885. Self-organized language modeling for speech recognition. In Readings in Speech Recognition, A. Waibel and K. Lee (eds.). Morgan Kaufmann, Weshington, DC vol. 28, pp. 2591-2594.
11. Jin, G. 1992. PH Corpus of Mandarin Chinese. ftp://ftp.cogsci.ed.ac.uk/ pub/chinese. Date accessed 30 June, 2007.
12. Lua, K. T. 1994. Compression of Chinese text. In International Conference on Chinese Computing. pp. 367-375.
13. McEnery, T., and Xiao, R. 2004. The Lancaster Corpus of Mandarin Chinese. European Language Resources Association. http:// bowland-files.lancs.ac.uk/corplang/lcmc/. Date accessed 30 June, 2007.
14. Moffat, A. 1989. Word-based text compression. Software Practice and Experience 19(2):185-198.
15. Moffat, A. 1990. Implementing the PPM data compression scheme. IEEE Transaction on Communication 38(11):1917-1921.
16. Moffat, A., Neal R., and Witten, I. 1998. Arithmetic Coding Revisited. ACM Transactions on Information Systems 16(3):256-294.
17. Ong, G. H., and Ng, J. P. 2005. Dynamic Markov Compression using a crossbar-like tree initial structure for Chinese texts. In ICITA '05: Proceedings of the Third International Conference on Information Technology and Applications (ICITA'05), vol. 2, pp. 407-410.
18. Shkarin, D. 2002 PPM : One step to practicality. In Data Compression Conference 2002. pp. 202-211.
19. Teahan, W. J. 1998. Modelling English Text, PhD thesis. New Zealand: University of Waikato.
20. Teahan, W. J., and Harper, D. J. 2001. Combining PPM models using a text mining approach. In Data Compression Conference 2001 pp. 153-162.
21. TREC Mandarin Corpus. 2000 Text Retrieval Conference test data. http:// www.ldc.upenn.edu. Date accessed 30 June, 2007.
22. Vines, P., and Zobel, J. 1998 Compression techniques for Chinese text. Software - Practice and Experience 28(12):1299-1314.
23. Witten, I. H., Bray, Z., Mahoui, and Teahan, W. J. 1999. Text mining: A new frontier for lossless compression. In Data Compression Conference 1999. pp. 198-207.
24. Wu, P., and Teahan, W. J. 2005. Modelling Chinese for text compression. In Data Compression Conference, 2005, Proceedings. DCC 2005. p. 488.