An estimator of the mutual information based on a criterion for conditional independence

Computational Statistics and Data Analysis

Volumn 32, Issue 1, 1999, Pages 1-17

  Darbellay, Georges A ^a  

^a INSTITUTE OF INFORMATION THEORY AND AUTOMATION   (Czech Republic)

Author keywords

Data dependent partitions; Mutual information; Nonparametric estimation; Statistical dependence

Indexed keywords

ALGORITHMS; COMPUTER SIMULATION; MAXIMUM LIKELIHOOD ESTIMATION; NORMAL DISTRIBUTION; RANDOM PROCESSES; STATISTICAL TESTS;

DATA DEPENDENT PARTITIONS; MUTUAL INFORMATION; NON PARAMETRIC ESTIMATION; STATISTICAL DEPENDENCE;

INFORMATION THEORY;

EID: 0033225867     PISSN: 01679473     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0167-9473(99)00020-1     Document Type: Article

References (22)

1. Beirlant J., Dudewicz E.J., Györfi L., van der Meulen E.C. Nonparametric entropy estimation: an overview. Internat. J. Math. Statist. Sci. 6(1):1997;17-39.
2. Darbellay, G.A., 1998a. Predictability: an information-theoretic perspective. In: Procházka, A., Uhlíř, J., Rayner, P.J.W., Kingsbury, N.G. (Eds.), Signal Analysis and Prediction. Birkhäuser, Boston, pp. 249-262.
3. Darbellay, G.A., 1998b. An adaptive histogram estimator for the mutual information (Extended Version). Research Report no. 1936, UTIA, Academy of Sciences, Prague (library@utia.cas.cz).
4. d : an information-theoretic approach. Proceedings of the Third European IEEE workshop. Computationally Intensive Methods in Control and Data Processing. Prague, 7-9 September 1998 (library@utia.cas.cz).
5. Darbellay, G.A., Vajda, I., 1998. Entropy expressions for multivariate continuous distributions. Research Report no 1920, UTIA, Academy of Sciences, Prague (library@utia.cas.cz).
6. Darbellay, G.A., Vajda, I., 1999. Estimation of the mutual information with data-dependent partitions. IEEE Trams. Inform. THeory, May issue.
7. Dobrushin, R.L., 1959. General formulation of Shannon's main theorem in information theory. Uspekhi Mat. Nauk 14, 3-104 (in Russian). Translated in Amer. Math. Soc. Trans. 33, 323-438.
8. Dobrushin, R.L., 1959. General formulation of Shannon's main theorem in information theory. Uspekhi Mat. Nauk 14, 3-104 (in Russian). Translated in Amer. Math. Soc. Trans. 33, 323-438.
9. L'Ecuyer P. Combined multiple recursive random number generators. Oper. Res. 44:1996;816-822.
10. Fraser, A.M., Swinney, H.L., 1986. Independent coordinates for strange attractors from mutual information. Phys. Rev. A 33(2), 1134-1140.
11. Gelfand, I.M., Kolmogorov, A.N., Yaglom, A.M., 1956. On the general definition of an amount of information. Dokl. Akad. Nauk SSSR 111, 745-748 (in Russian).
12. Granger, C.W.J., Teräsvirta, T., 1993. Modelling Nonlinear Economic Relationships, Oxford Univ. Press, Oxford.
13. Lilliefors H.W. On the Kolmogorov-Smirnov test for normality with mean and variance unknown. J. Amer. Stat. Assoc. 62:1967;399-402.
14. Perez, A., 1957a. Notions généralisées d'incertitude, d'entropie et d'information du point de vue de la théorie des martingales. Transactions of the 1st Prague Conference in Information Theory, Statistical Decision Functions, Random Processes, Publishing House of the Czechoslovak Acad. of Sciences, Prague, pp. 183-208.
15. Perez, A., 1957b. Sur la convergence des incertitudes, entropies et informations échantillon vers leurs valeurs vraies, as previous reference but pp. 245-252.
16. Rényi A. On measures of dependence. Acta Math. Acad. Sci. Hungar. 10:1959;441-451.
17. Shannon, C.E., 1948. A mathematical theory of communication. Bell Systems Technol. J. 27 (part I), 379-423; (part II), 623-656.
18. Snedecor, G.W., Cochran, W.G., 1967. Statistical Methods. Iowa State University Press, USA.
19. Wichmann, B.A., Hill, I.D., 1984. An efficient and portable pseudorandom number generator. Appl. Statist., 31, 188-190. (See also corrections and remarks in the same journal by Wichmann and Hill ( 1984 ), 31, 123; Mc Leod (1985) 34, 198-200; Zeisel ( 1986 ) 35, 89.).
20. Wichmann, B.A., Hill, I.D., 1984. An efficient and portable pseudorandom number generator. Appl. Statist., 31, 188-190. (See also corrections and remarks in the same journal by Wichmann and Hill ( 1984 ), 31, 123; Mc Leod (1985) 34, 198-200; Zeisel ( 1986 ) 35, 89.).
21. Wichmann, B.A., Hill, I.D., 1984. An efficient and portable pseudorandom number generator. Appl. Statist., 31, 188-190. (See also corrections and remarks in the same journal by Wichmann and Hill ( 1984 ), 31, 123; Mc Leod (1985) 34, 198-200; Zeisel ( 1986 ) 35, 89.).
22. Wichmann, B.A., Hill, I.D., 1984. An efficient and portable pseudorandom number generator. Appl. Statist., 31, 188-190. (See also corrections and remarks in the same journal by Wichmann and Hill ( 1984 ), 31, 123; Mc Leod (1985) 34, 198-200; Zeisel ( 1986 ) 35, 89.).