Hierarchical universal coding

IEEE Transactions on Information Theory

Volumn 42, Issue 5, 1996, Pages 1354-1364

  Feder, Meir ^a,b   Merhav, Neri ^a,c  

^a IEEE

^b TEL AVIV UNIVERSITY   (Israel)

^c TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Arbitrarily varying sources; Capacity; Inaximin redundancy; Minimax redundancy; Mixtures; Redundancy capacity theorem; Universal coding

Indexed keywords

CHANNEL CAPACITY; DATA COMPRESSION; HIERARCHICAL SYSTEMS; INFORMATION THEORY; MARKOV PROCESSES; PROBABILITY; REDUNDANCY; SET THEORY;

ARBITRARILY VARYING PROCESSES; HIERARCHICAL UNIVERSAL CODING; MAXIMIN REDUNDANCY; MINIMAX REDUNDANCY; REDUNDANCY CAPACITY THEOREM;

SIGNAL ENCODING;

EID: 0030241059     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.532877     Document Type: Article

References (26)

1. A. R. Barron, "Logically smooth density estimation." Ph.D dissertation. Stanford University, Stanford, CA, 1985.
2. A. R. Barron and T. M. Cover, "Minimum complexity density estimation," IEEE Trans. Inform. Theory, vol. 37, no. 4, 1034-1054, July 1991.
3. A. R. Barron and C. H. Sheu, "Approximation of density functions by sequences of exponential families." Ann. Statist., vol. 1, no. 3, pp. 1347-1369, 1991.
4. T. Berger, Rate Distortion Theory. Englewood Cliffs, NJ: Prentice-Hall, 1971.
5. B. S. Clarke and A. R. Barron, "Information-theoretic asymptotics of Bayesian methods," IEEE Trans. Inform. Theory, vol. 36. pp. 453-471, 1990.
6. _, "Jeffrey's prior is asymptotically least favorable under entropy risk," J. Statist. Plan. Inform., vol. 41, pp. 37-60, 1994.
7. T. M. Cover, "On the competitive optimality of Huffman codes," IEEE Trans. Inform. Theory, vol. 37, no. 1, pp. 172-174, Jan. 1991.
8. I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic Press, 1981.
9. L. D. Davisson, "Universal noiseless coding." IEEE Trans. Inform. Theory, vol. IT-19, no. 6, pp. 783-795, Nov. 1973.
10. _, "Minimax noiseless universal coding for Markov sources," IEEE Trans. inform. Theory, vol. IT-29, no. 2, pp. 211-215, Mar. 1983.
11. L. D. Davisson and A. Leon-Garcia, "A source matching approach to finding minimax codes," IEEE Trans. Inform. Theory, vol. IT-26, no. 2, pp. 166-174, Mar. 1980.
12. R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.
13. _, "Source coding with side information and universal coding," unpublished manuscript, Sept. 1976. (Also, presented at the International Symposium on Inform. Theory, Oct. 1974.)
14. E. J. Hannan and B. G. Quinn, "The determination of the order of an autorcgression," J. Roy Statist. Soc. Ser. B, vol. 41, pp. 190-195, 1979.
15. J. C. Kieffer, "Strongly consistent MDL-based of a model class for a finite alphabet source," preprint, 1993.
16. N. Merhav and M. Feder, "A strong version of the redundancy-capacity theorem of universal coding," IEEE Trans. Inform. Theory, vol. 41, no. 3, pp. 714-722, May 1995.
17. J. Rissanen, "A universal data compression system, IEEE Trans. Inform. Theory, vol. IT-29, no. 5, pp. 656-664, Sept. 1983.
18. _, "Universal coding, information, prediction, and estimation," IEEE Trans. Inform. Theory, vol. IT-30, no. 4, pp. 629-636, July 1984.
19. _, "Stochastic complexity and modeling," Ann. Statist., vol. 14, pp. 1080-1100, 1986.
20. _, "Complexity of strings in the class of Markov sources," IEEE Trans. Inform. Theory, vol. IT-32, no. 4, pp. 526-532, July 1986.
21. S. Rudich, "Inferring the structure of a Markov chain from its output," in Proc. 26th IEEE Symp. on Foundations of Computer Science, 1985, pp. 321-326.
22. B. Ya. Ryahko, "Encoding a source with unknown but ordered probabilities," Probl. Inform. Transm., pp. 134-138, Oct. 1979.
23. _, "Twice-universal coding," Probl. Inform. Transm., pp. 173-177, July-Sept., 1984.
24. M. J. Weinberger and M. Feder, "Predictive stochastic complexity and model estimation for finite-state processes," J. Statist. Plan. Inf., Vol. 39, pp. 353-372, 1994.
25. M. J. Weinberger, N. Merhav, and M. Feder, "Optimal sequential probability assignment for individual sequences," IEEE Trans, Inform. Theory, vol. 40, no. 2, pp. 384-396, Mar. 1994.
26. F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "The context-tree weighting methods: basic properties," IEEE Trans. Inform. Theory, vol. 41, no. 3, pp. 653-664, May 1995.