The confidence interval of entropy estimation through a noisy channel

2010 IEEE Information Theory Workshop, ITW 2010 - Proceedings

Volumn , Issue , 2010, Pages

  Ho, Siu Wai ^a   Chan, Terence ^a   Grant, Alex ^a  

^a UNIVERSITY OF SOUTH AUSTRALIA   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

ALPHABET SIZE; CONFIDENCE INTERVAL; DISCRETE MEMORYLESS CHANNELS; ENTROPY ESTIMATION; LOWER BOUNDS; MEMORYLESS SOURCE; NOISY CHANNEL; NUMBER OF SAMPLES; PRIOR KNOWLEDGE; SOURCE DISTRIBUTION; SOURCE ENTROPY; TRANSITION MATRICES;

CHANNEL CAPACITY; COMMUNICATION CHANNELS (INFORMATION THEORY); ESTIMATION; FINANCIAL DATA PROCESSING;

ENTROPY;

EID: 80051932385     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CIG.2010.5592695     Document Type: Conference Paper

References (17)

1. L. Paninski, "Estimation of entropy and mutual information," Neural Comput., vol. 15, pp. 1191-1253, 2003. (Pubitemid 37049793)
2. A. Kraskov, H. Stogbauer, and P. Grassberger, "Estimating mutual information," Phys. Rev. E, vol. 69, 066138, (2004).
3. G.P. Basarin, "On a Statistical Estimate for the Entropy of a Sequence of Independent Random Variables," Theory Probab. Appln., vol. 4, pp. 333-336, 1959.
4. R. C. Conant, "Estimation of Entropy of a Binary Variable" Satisfying a Reliability Criterion," Kybernetik, Vol. 12, pp. 55-57, 1973.
5. M. B. Kennel, J. Shlens, H. D. I. Abarbanel and E. J. Chichilnisky, "Estimating Entropy Rates with Bayesian Confidence Intervals," Neural Comput., vol. 17, pp. 1531-1576, 2005. (Pubitemid 40669911)
6. H. Cai, S. R. Kulkarni, and S. Verdú, "Universal entropy estimation via block sorting," IEEE Trans. Inform. Theory, vol. 50, pp. 1551-1561, July 2004.
7. A. J. Wyner and D. Foster, "On the lower limits of entropy estimation," IEEE Trans. Inform. Theory, preprint.
8. S.-W. Ho and R. W. Yeung, "The Interplay between Entropy and Variational Distance," in Proc. IEEE Int. Symp. Inform. Theory, (Nice, France), Jun. 24-29, 2007.
9. ∼hosw/Full-Hestimate.pdf.
10. R. W. Yeung, Information Theory and Network Coding, Springer, 2008.
11. 1 Deviation of the Empirical Distribution," Tech. Rept., HP Laboratories Palo Alto, HPL-2003-97 (R.1), Jun. 2003.
12. D. Angluin and M. Csürös, "Learning Markov chains with variable memory length from noisy output," in Proc. 10th Annual Conf. Computational Learning Theory, (Nashville), Jul. 1997.
13. T. Weissman, E. Ordentlich, G. Seroussi, S. Verdu and M. Weinberger, "Universal Discrete Denoising: Known Channel," IEEE Trans. Inform. Theory, vol. 51, no. 1, pp. 5-28, Jan. 2005. (Pubitemid 40145984)
14. T. Moon and T. Weissman, "Universal Filtering Via Hidden Markov Modeling," IEEE Trans. Inform. Theory, vol. 54, no. 2, pp. 692-708, Feb. 2008. (Pubitemid 351314660)
15. T. M. Cover and J. A. Thomas, Elements of Information Theory, New York: Wiley-Interscience, 1991.
16. A. Antos and I. Kontoiannis, "Convergence Properties of Functional Estimates of Discrete Distributions," Random Structures and Algorithms, 2002.
17. S.-W. Ho and R. W. Yeung, "On Information Divergence Measures and a Unified Typicality," in Proc. IEEE Int. Symp. Inform. Theory, (Seattle), July 9-14, 2006.