Estimating mutual information

Physical Review E - Statistical, Nonlinear, and Soft Matter Physics

Volumn 69, Issue 6 2, 2004, Pages

  Kraskov, Alexander ^a   Stögbauer, Harald ^a   Grassberger, Peter ^a  

^a FORSCHUNGSZENTRUM JÜLICH   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; CONVERGENCE OF NUMERICAL METHODS; CORRELATION METHODS; DATABASE SYSTEMS; ENTROPY; INDEPENDENT COMPONENT ANALYSIS; LAGRANGE MULTIPLIERS; PROBABILITY; RANDOM PROCESSES; RELIABILITY; SYNCHRONIZATION;

CORRELATION COEFFICIENTS; MUTUAL INFORMATION (MI); PROBABILITY DENSITY; RANDOM VARIABLES;

ESTIMATION;

EID: 39749164774     PISSN: 15393755     EISSN: None     Source Type: Journal    
DOI: 10.1103/PhysRevE.69.066138     Document Type: Article

References (42)

1. T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley, New York, 1991).
2. A. Renyi, Probability Theory (North Holland, Amsterdam, 1971).
3. Independent Component Analysis: Principles and Practice, edited by S. Roberts and R. Everson (Cambridge Univ. Press, Cambridge, 2001).
4. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis (Wiley, New York, 2001).
5. Y.-I. Moon, B. Rajagopalan, and U. Lall, Phys. Rev. E 52, 2318 (1995).
6. R. Steuer, J. Kurths, C. O. Daub, J. Weise, and J. Selbig, Bioinformatics 18 (Suppl. 2), S231 (2002).
7. A. M. Fraser and H. L. Swinney, Phys. Rev. A 33, 1134 (1986).
8. G. A. Darbellay and I. Vajda, IEEE Trans. Inf. Theory 45, 1315 (1999).
9. P. Grassberger, Phys. Lett. A 128, 369 (1988).
10. M. S. Roulston, Physica D 125, 285 (1999).
11. R. L. Dobrushin, Theor. Probab. Appl. 3, 462 (1958).
12. O. Vasicek, J. R. Stat. Soc. Ser. B. Methodol. 38, 54 (1976).
13. A. Kaiser and T. Schreiber, Physica D 166, 43 (2002).
14. E. S. Dudewicz and E. C. van der Meulen, J. Am. Stat. Assoc. 76, 967 (1981).
15. B. van Es, Scand. J. Stat. 19, 61 (1992).
16. N. Ebrahimi, K. Pflughoeft, and E. S. Soon, Stat. Probab. Lett. 20, 225 (1994).
17. J. C. Correa, Commun. Stat: Theory Meth. 24, 2439 (1995).
18. A. B. Tsybakov and E. C. van der Meulen, Scand. J. Stat. 23, 75 (1996).
19. R. Wieczorkowski and P. Grzegorzewksi, Commun. Stat.-Simul. Comput. 28, 541 (1999).
20. L. F. Kozachenko and N. N. Leonenko, Probl. Inf. Transm. 23, 95 (1987).
21. P. Grassberger, Phys. Lett. 107A, 101 (1985).
22. R. L. Somorjai, Methods for Estimating the Intrinsic Dimensionality of High-Dimensional Point Sets, in Dimensions and Entropies in Chaotic Systems, edited by G. Mayer-Kress (Springer, Berlin, 1986).
23. J. D. Victor, Phys. Rev. E 66, 051903 (2002).
24. G. A. Darbellay and I. Vajda, IEEE Trans. Inf. Theory 46, 709 (2000).
25. W. H. Press et al., Numerical Recipes (Cambridge Univ. Press, New York, 1993).
26. P. Grassberger, Phys. Lett. A 148, 63 (1990).
27. R. Hegger, H. Kantz, and T. Schreiber, TISEAN: Nonlinear Time Series Analysis Software Package (ULR: www.mpipksdresden.mpg.de//~rtisean).
28. G. A. Darbellay, Comput. Stat. Data Anal. 32, 1 (1999).
29. G. A. Darbellay and I. Vajda, Inst. of Information Theory and Automation, Technical Rep. No. 1921 (1998), to be obtained from http://siprint.utia.cas.cz/darbellay/
30. G. A. Darbellay, 3rd IEEE European Workshop on Computer-intensive Methods in Control and Data Processing, Prague, 1998 (IEEE, Piscataway, NJ, 1999), p. 83.
31. T. R. Hughes et al., Cell 102, 109 (2000); see also www.rii.com/publications/2000/cell_hughes.htm
32. H. Stögbauer, A. Kraskov, S. A. Astakhov, and P. Grassberger, e-print physics/0405044.
33. B. W. Silverman, Density Estimation for Statistics and Data Analysis (Chapman and Hall, London, 1986).
34. F. Meinecke, A. Ziehe, M. Kawanabe, and K.-R. Mller, IEEE Trans. Biomed. Eng. 49, 1514 (2002).
35. J.-F. Cardoso, Multidimensional Independent Component Analysis, Proceedings of ICASSP '98 (IEEE, Piscataway, NJ, 1998).
36. J.-F. Cardoso and A. Souloumiac, IEE Proc. F, Radar Signal Process. 140, 362 (1993).
37. Daisy: Database for the identification of systems, edited by B. L. R. De Moor, www.esat.kuleuven.ac.be/sista/daisy (1997).
38. A. Ziehe and K.-R. Mller, TDSEP - An Efficient Algorithm for Blind Separation Using Time Structure, in Proceedings of the 8th International Conferences on Artificial Neural Networks, ICANN'98, edited by L. Niklasson et al. (Springer, Berlin, 1998), p. 675.
39. In Figs. 18-20 we used k=1, since the time sequences were sufficiently long to give very small statistical errors. To find components as independent as possible, we should have used much larger k, since this would reduce statistical errors at the cost of increased but nevertheless very small systematic errors. We checked that basically the same results were obtained with k up to 100.
40. B. Pompe, P. Blidh, D. Hoyer, and M. Eiselt, IEEE Eng. Med. Biol, Mag. 17, 32 (1998).
41. H. Matsuda, Phys. Rev. E 62, 3096 (2000).
42. See http://siprint.utia.cas.cz/timeseries/