Estimation of the information by an adaptive partitioning of the observation space

IEEE Transactions on Information Theory

Volumn 45, Issue 4, 1999, Pages 1315-1321

  Darbellay, Georges A ^a   Vajda, Igor ^a  

^a INSTITUTE OF INFORMATION THEORY AND AUTOMATION   (Czech Republic)

Author keywords

[No Author keywords available]

Indexed keywords

MAXIMUM LIKELIHOOD ESTIMATION; PROBABILITY; RANDOM PROCESSES;

ADAPTIVE PARTITIONING; MUTUAL INFORMATION;

INFORMATION THEORY;

EID: 0032641874     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.761290     Document Type: Article

References (16)

1. A. R. Barren, L. Györfi, and E. C. van der Meulen Distribution estimation consistent in total variation and in two types of information divergence," IEEE Trans. Inform. Theory, vol. 35, pp. 14371454, Sept. 1992.
2. J. Beirlant, E. J. Dudewicz, L. Györfi, and E. C. van der Meulen Nonparametric entropy estimation: An overview," Int. J. Math. Stat. Sei., vol. 6, no. 1, pp. 1739, 1997.
3. E. Bofinger Goodnessoffit using sample quantiles," J. Roy. Stat. Soc., Ser. B, vol. 35, pp. 277284, 1973.
4. I. Cizova Test of a histogram estimator for the differential entropy," Master's thesis, Czech Tech. Univ. (CVUT), Prague, 1997 (in Czech).
5. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
6. G. A. Darbellay An adaptive histogram estimator for the mutual information," Res. Rep. 1889, UTIA, Academy of Sciences, Prague, Czech Republic, 1996. Also, Computat. Statist. Data Anal, to be published.
7. _ The mutual information as a measure of statistical dependence," in Proc. Int. Symp. Information Theory (Ulm, Germany, June 29July 4, 1997). Piscataway, NJ: IEEE Press, 1997, p. 405.
8. _ Predictability: An informationtheoretic perspective," in Signal Analysis and Prediction, A. Prochâzka, J. Uhh'f, P. J. W. Rayner, and N. G. Kingsbury, Eds. Boston, MA: BirkhäuserVerlag, 1998, pp. 249262.
9. _ Statistical dependences in IR"1: An informationtheoretic approach," in Proc. 3rd European IEEE Workshop Computationally Intensive Methods in Control and Data Processing (Prague, Czech Republic, Sept. 79, 1998). Available via email at library@utia.cas.cz.
10. G. A. Darbellay and I. Vajda Entropy expressions for multivariate continuous distributions," Res. Rep. 1920, UTIA, Academy of Sciences, Prague, Czech Republic, 1998. Available via email at library@utia.cas.cz.
11. _ Estimation of the mutual information with datadependent partitions," Res. Rep. 1921, UTIA, Academy of Sciences, Prague, Czech Republic, 1998. Available via email at library@utia.cas.cz.
12. R. L. Dobrushin General formulation of Shannon's main theorem in information theory," Usp. Mat. Nauk, vol. 14, pp. 3104, 1959 (in Russian). Translated in Amer. Math. Soc. Trans., vol. 33, pp. 323438.
13. L. Györfi and E. C. van der Meulen Densityfree convergence properties of various estimators of entropy," Comput. Statist. Data Anal., vol. 5, pp. 425436, 1987.
14. F. Liese and I. Vajda, Convex Statistical Distances. Leipzig, Germany: Teubner, 1987.
15. D. W. Scott, Multivariate Density Estimation. New York: Wiley, 1992.
16. Y. Shao and M. G. Hahn Limit theorems for the logarithm of sample spacings," Statist. Probab. Lett., vol. 24, pp. 121132, 1995.