Universal estimation of erasure entropy

IEEE Transactions on Information Theory

Volumn 55, Issue 1, 2009, Pages 350-357

  Yu, Jiming ^a   Verdú, Sergio ^a  

^a Princeton University   (United States)

Author keywords

Bidirectional context tree; Context tree weighting; Data compression; Entropy rate; Universal algorithms; Universal modeling

Indexed keywords

COMPUTATIONAL METHODS; ENTROPY; SIGNAL THEORY; TREES (MATHEMATICS);

BIDIRECTIONAL CONTEXT TREE; CONTEXT-TREE WEIGHTING; ENTROPY RATE; UNIVERSAL ALGORITHMS; UNIVERSAL MODELING;

DATA COMPRESSION;

EID: 57349093159     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2008.2008117     Document Type: Article

References (23)

1. S. Verdú and T. Weissman, "Erasure entropy," in Proc. 2006 IEEE Int. Symp. Information Theory, Seattle, WA, Jul. 2006, pp. 98-102.
2. S. Verdú and T. Weissman, "The information lost in erasures," IEEE Trans. Inf. Theory, vol. 54, no. 11, pp. 5030-5058, Nov. 2008.
3. F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "The context-tree weighting method: Basic properties," IEEE Trans. Inf. Theory, vol. 41, no. 3, pp. 653-664, May 1995.
4. F. M. J. Willems, "The context-tree weighting method: Extensions," IEEE Trans. Inf. Theory, vol. 44, no. 2, pp. 792-798, Mar. 1998.
5. H. Cai, S. R. Kulkarni, and S. Verdú, "A universal lossless compressor with side information based on context tree weighting," in Proc. IEEE Int. Symp. Information Theory, Adelaide, Australia, Sep. 2005, pp. 2340-2344.
6. H. Cai, S. R. Kulkarni, and S. Verdú, "An algorithm for universal lossless compression with side information," IEEE Trans. Inf. Theory vol. 52, no. 9, pp. 4008-4016, Sep. 2006.
7. T. Weissman, E. Ordentlich, G. Seroussi, S. Verdú, and M. Weinberger, "Universal discrete denoising: Known channel," IEEE Trans. Inf. Theory, vol. 51, no. 1, pp. 5-28, Jan. 2005.
8. E. Ordentlich, M. J. Weinberger, and T. Weissman, "Efficient pruning of bi-directional context trees with applications to universal denoising and compression," in Proc. IEEE Information Theory Workshop, San Antonio, TX, Oct. 2004.
9. E. Ordentlich, M. J. Weinberger, and T. Weissman, "Multi-directional context sets with applications to universal denoising and compression," in Proc. IEEE Int. Symp. Information Theory, Adelaide, Australia, Sep. 2005, pp. 1270-1274.
10. J. Yu and S. Verdú, "Schemes for bi-directional modeling of discrete stationary sources," in Proc. 39th Annual Conf. Information Science and Systems, Baltimore, MD, Mar. 2005.
11. J. Yu and S. Verdú, "Schemes for bidirectional modeling of discrete stationary sources," IEEE Trans. Inf. Theory, vol. 52, no. 11, pp. 4789-4807, Nov. 2006.
12. F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "Context weighting for general finite-context sources," IEEE Trans. Inf. Theory, vol. 42, no. 6, pp. 1514-1520, Sep. 1996.
13. J. Yu and S. Verdú, "Universal erasure entropy estimation," in Proc. IEEE Int. Symp. Information Theory, Seattle, WA, Jul. 2006, pp. 2358-2362.
14. H. Cai, S. R. Kulkarni, and S. Verdú, "Universal entropy estimation via block sorting," IEEE Trans. Inf. Theory, vol. 50, no. 7, pp. 1551-1561, Jul. 2004.
15. H. Cai, "Universal Estimation of Information Measures for Finite Alphabet Sources.," Ph.D. dissertation, Princeton Univ., Princeton, NJ, 2005.
16. H. Cai, S. R. Kulkarni, and S. Verdú, "Universal divergence estimation for finite-alphabet sources," IEEE Trans. Inf. Theory, vol. 52, no. 8, pp. 3456-3475, Aug. 2006.
17. J. Yu, "Bidirectional Modeling of Finite-Alphabet Sources.," Ph.D. dissertation, Princeton Univ., Princeton, NJ, 2007.
18. R. E. Krichevsky and V. K. Trofimov, "The performance of universal encoding," IEEE Trans. Inf. Theory, vol. IT-27, no. 2, pp. 199-207, Mar. 1981.
19. F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "Context-tree maximizing," in Proc. 2000 Conf. Information Sciences and Systems Princeton, NJ, Mar. 2000, pp. TP6 7-12.
20. L. Paninski, "Estimation of entropy and mutual information," Neural Comput., vol. 15, pp. 1191-1253, 2003.
21. F. M. J. Willems and T. J. Tjalkens, "Complexity reduction of the context-tree weighting algorithm: A study for KPN research," Tech. Univ. Eindhoven, Eindhoven, The Netherlands, EIDMA Rep. RS.97.01, Jan. 1997.
22. P. Volf, "Weighting Techniques in Data Compression: Theory and Algorithm," Ph.D. dissertation, Tech. Univ. Eindhoven, Eindhoven, The Netherlands, 2002.
23. N. Chomsky, "Three models for the description of language," IRE Trans. Inf. Theory, vol. IT-2, pp. 113-124, 1956.