Arithmetic coding for data compression

Communications of the ACM

Volumn 30, Issue 6, 1987, Pages 520-540

  Witten, Ian H ^a   Neal, Radford M ^a   Cleary, John G ^a  

^a UNIVERSITY OF CALGARY   (Canada)

Author keywords

Adaptive modeling; arithmetic coding; Huffman coding

Indexed keywords

COMPUTER PROGRAMS; INFORMATION THEORY - DATA COMPRESSION;

ADAPTIVE MODELS; ARITHMETIC CODING; DECODING; ENCODING; FIXED MODELS;

CODES, SYMBOLIC;

EID: 0023364261     PISSN: 00010782     EISSN: 15577317     Source Type: Journal    
DOI: 10.1145/214762.214771     Document Type: Article

References (23)

1. Abramson, N. Information Theory and Coding. McGraw-Hill, New York, 1963. This textbook contains the first reference to what was to become the method of arithmetic coding (pp. 61–62).
2. Bentley, J.L., Sleator, D.D., Tarjan, R.E., and Wei, V.K. A locally adaptive data compression scheme. Commun. ACM 29, 4 (Apr. 1986), 320–330. Shows how recency effects can be incorporated explicitly into a text compression system.
3. Cleary, J.G., and Witten, I.H. A comparison of enumerative and adaptive codes. IEEE Trans. Inf. Theory IT-30, 2 (Mar. 1984), 306–315. Demonstrates under quite general conditions that adaptive coding outperforms the method of calculating and transmitting an exact model of the message first.
4. Cleary, J.G., and Witten, I.H. Data compression using adaptive coding and partial string matching. IEEE Trans. Commun. COM-32, 4 (Apr. 1984), 396–402. Presents an adaptive modeling method that reduces a large sample of mixed-case English text to around 2.2 bits/character when arithmetically coded.
5. Cormack, G.V., and Horspool, R.N. Algorithms for adaptive Huffman codes. Inf. Process. Lett. 18, 3 (Mar. 1984), 159–166. Describes how adaptive Huffman coding can be implemented efficiently.
6. Cormack, G.V., and Horspool, R.N. Data compression using dynamic Markov modeling. Res. Rep., Computer Science Dept., Univ. of Waterloo, Ontario, Apr. 1985. Also to be published in Comput. ]. Presents an adaptive state-modeling technique that, in conjunction with arithmetic coding, produces results competitive with those of [4].
7. Gallager, R.G. Variations on a theme by Huffman. IEEE Trans. Inf. Theory IT-24, 6 (Nov. 1978), 668–674. Presents an adaptive Huffman coding algorithm, and derives new bounds on the redundancy of Huffman codes.
8. Held, G. Data Compression: Techniques and Applications. Wiley, New York, 1984. Explains a number of ad hoc techniques for compressing text.
9. Hester, J.H., and Hirschberg, D.S. Self-organizing linear search. ACM Comput. Surv. 37, 3 (Sept. 1985), 295–311. A general analysis of the technique used in the present article to expedite access to an array of dynamically changing frequency counts.
10. Huffman, D.A. A method for the construction of minimum-redundancy codes. Proc. Inst. Electr. Radio Eng. 40, 9 (Sept. 1952), 1098–1101. The classic paper in which Huffman introduced his famous coding method.
11. Hunter, R., and Robinson, A.H. International digital facsimile coding standards. Proc. Inst. Electr. Electron. Eng. 68, 7 (July 1980), 854–867. Describes the use of Huffman coding to compress run lengths in black/white images.
12. Kucera, H., and Francis, W.N. Computational Analysis of Present-Day American English. Brown University Press, Providence, R.I., 1967. This large corpus of English is often used for computer experiments with text, including some of the evaluation reported in the present article.
13. Langdon, G.G. An introduction to arithmetic coding. IBM J. Res. Dev. 28, 2 (Mar. 1984), 135–149. Introduction to arithmetic coding from the point of view of hardware implementation.
14. Langdon, G.G., and Rissanen, J. Compression of black-white images with arithmetic coding. IEEE Trans. Commun. COM 29, 6 (June 1981), 858–867. Uses a modeling method specially tailored to black/white pictures, in conjunction with arithmetic coding, to achieve excellent compression results.
15. Pasco, R. Source coding algorithms for fast data compression. Ph.D. thesis, Dept. of Electrical Engineering, Stanford Univ., Stanford, Calif., 1976. An early exposition of the idea of arithmetic coding, but lacking the idea of incremental operation.
16. Rissanen, J.J. Generalized Kraft inequality and arithmetic coding. IBM J. Res. Dev. 20 (May 1976), 198–203. Another early exposition of the idea of arithmetic coding.
17. Rissanen, J.J. Arithmetic codings as number representations. Acta Polytech. Scand. Math. 31 (Dec. 1979), 44–51. Further develops arithmetic coding as a practical technique for data representation.
18. Rissanen, J., and Langdon, G.G. Arithmetic coding. IBM J. Res. Dev. 23, 2 (Mar. 1979), 149–162. Describes a broad class of arithmetic codes.
19. Rissanen, J., and Langdon, G.G. Universal modeling and coding, IEEE Trans. Inf. Theory IT-27,1 (Jan. 1981), 12–23. Shows how data compression can be separated into modeling for prediction and coding with respect to a model.
20. Rubin, F. Arithmetic stream coding using fixed precision registers. IEEE Trans. Inf. Theory IT-25, 6 (Nov. 1979), 672–675. One of the first papers to present all the essential elements of practical arithmetic coding, including fixed-point computation and incremental operation.
21. Shannon, C.E., and Weaver, W. The Mathematical Theory of Communication. University of Illinois Press, Urbana, III., 1949. A classic book that develops communication theory from the ground up.
22. Welch, T.A. A technique for high-performance data compression. Computer 17, 6 (June 1984), 8–19. A very fast coding technique based on the method of [23], but whose compression performance is poor by the standards of [4] and [6]. An improved implementation of this method is widely used in UNIX systems under the name compress.
23. Ziv, J., and Lempel, A. Compression of individual sequences via variable-rate coding, IEEE Trans. Inf. Theory IT-24, 5 (Sept. 1978), 530–536. Describes a method of text compression that works by replacing a substring with a pointer to an earlier occurrence of the same substring. Although it performs quite well, it does not provide a clear separation between modeling and coding.