Evaluation of mutual information estimators for time series

International Journal of Bifurcation and Chaos

Volumn 19, Issue 12, 2009, Pages 4197-4215

  Papana, Angeliki ^a   Kugiumtzis, Dimitris ^a  

^a ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

Author keywords

Mutual information; Nonlinear systems; Probability density; Time series

Indexed keywords

INTELLIGENT SYSTEMS; LINEAR SYSTEMS; MEMBERSHIP FUNCTIONS; MONTE CARLO METHODS; MOTION COMPENSATION; NEAREST NEIGHBOR SEARCH; NONLINEAR SYSTEMS; PROBABILITY DENSITY FUNCTION; TEXT PROCESSING; TIME SERIES;

ADAPTIVE CRITERION; COMPARATIVE STUDIES; K-NEAREST NEIGHBORS; LINEAR AND NONLINEAR SYSTEMS; MUTUAL INFORMATIONS; OPTIMIZATION OF PARAMETERS; PARAMETER SELECTION; PROBABILITY DENSITIES;

PARAMETER ESTIMATION;

EID: 77950348287     PISSN: 02181274     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0218127409025298     Document Type: Article

References (56)

1. Abarbanel, H., Masuda, N., Rabinovich, M. &Tumer, E. [2001] "Distribution of mutual information," Phys. Lett. A 281, 368-373.
2. Bendat, S. &Piersol, A. [1966] Measurements and Analysis of Random Data (John Wiley and Sons, NY).
3. Blinnikov, S. &Moessner, R. [1998] "Expansions for nearly Gaussian distributions," Astron. Astrophys. Sup. 130, 193-205.
4. Bonnlander, B. &Weigend, A. [1994] "Selecting input variables using mutual information and nonparametric density estimation," Proc. 1994 Int. Symp. Artificial Neural Networks (ISANN '94), Taiwan, pp. 42-50.
5. Cellucci, C., Albano, A. &Rapp, P. [2005] "Statistical validation of mutual information calculations: Comparison of alternative numerical algorithms," Phys. Rev. E 71, 066208.
6. Cochran, W. [1954] "Some methods for strengthening the common X2 tests," Biometrics 10, 417-451.
7. Cohen, A. &Procaccia, I. [1985] "Computing the kolmogorov entropyfrom time signals of dissipative and conservative dynamical systems," Phys. Rev. A 31, 1872-1882.
8. Darbellay, G. &Vajda, I. [1999] "Estimation of the information by an adaptive partitioning of the observation space," IEEE T. Inform. Th. 45, 1315-1321.
9. Daub, C., Steuer, R., Selbig, J. &Kloska, S. [2004] "Estimating mutual information using B-spline functions: An improved similarity measure for analysing gene expression data," Bioinformatics 5, 118.
10. Diks, C. &Manzan, S. [2002] "Tests for serial independence and linearity based on correlation integrals," Stud. Nonlin. Dyn. E 6, 2.
11. Diks, C. &Panchenko, V. [2008] "Rank-based entropy tests for serial independence," Stud. Nonlin. Dyn. E 12, 2.
12. Doane, D. [1976] "Aesthetic frequency classifications," Am. Stat. 30, 181-183.
13. Fraser, A. &Swinney, H. [1986] "Independent coordinates for strange attractors from mutual information," Phys. Rev. A 33, 1134-1140.
14. Freedman, D. &Diaconis, P. [1981] "On the histogram as a density estimator: L2 theory," Z. Wahrscheinlichkeit. 57, 453-476.
15. Grassberger, P. &Procaccia, I. [1983] "Measuring the strangeness of strange attractor," Physica D 9, 189-208.
16. Grassberger, P. [1988] "Finite sample corrections to entropy and dimension estimates," Phys. Lett. A 128, 369-373.
17. Harrold, T., Sharma, A. &Sheather, S. [2001] "Selection of a kernel bandwidth for measuring dependence in hydrologic time series using the mutual information criterion," Stoch. Env. Res. Risk A 15, 310-324.
18. Henon, M. [1976] "A two dimensional mapping with a strange attractor," Commun. Math. Phys. 50, 69-77.
19. Hively, L., Protopopescu, V. &Gailey, P. [2000] "Timely detection of dynamical change in scalp EEG signals," Chaos 10, 864-875.
20. Hutter, M. &Zaffalon,M. [2005] "Distribution of mutual information from complete and incomplete data," Comput. Stat. Data An. 48, 633-657.
21. Ikeda, K., Kondo, K. &Akimoto, O. [1980] "Optical turbulence: Chaotic behaviour of transmitted light from a ring cavity," Phys. Rev. Lett. 45, 709-712.
22. Jones, M., Marron, J. &Sheather, S. [1996] "A brief survey of bandwidth selection for density estimation," J. Amer. Statist. Assoc. 91, 401-407.
23. Kantz, H. &Sḧurmann, T. [1996] "Enlarged scaling ranges in entropy and dimension estimates," Chaos 6, 167-171.
24. Kantz, H. &Schreiber, T. [1997] Nonlinear Time Series Analysis. (Cambridge University Press, Reading, MA)
25. -[2007] "Relative performance of mutual information estimation methods for quantifying the dependence among short and noisy data," Phys. Rev. E 76, 026209.
26. Khan, S., Bandyopadhyay, S., Ganguly, A., Saigal, S., Erickson, D., Protopopescu, V. &Ostrouchov, G.Knuth, K. [2006] "Optimal data-based binning for histograms," url: http://arxiv.org/abs/physics/ 0605197v1.
27. Kraskov, A., Sẗogbauer, H. &Grassberger, P. [2004] "Estimating mutual information," Phys. Rev. E 69, 066138.
28. Kreuz, T., Mormann, F., Andrzejak, R., Kraskov, A., Lehnertz, K. &Grassberger, P. [2007] "Measuring synchronization in coupled model systems: A comparison of different approaches," Physica D 225, 29-42.
29. Mackey, M. &Glass, L. [1977] "Oscillation and chaos in physiological control systems," Science 197, 287-289.
30. Micheas, A. &Zografos, K. [2006] "Measuring stochastic dependence using ?-divergence," J. Multivariate Anal. 97, 765-784.
31. Moddemeijer, R. [1989] "On estimation of entropy and mutual information of continuous distributions," Sign. Process. 16, 233-248.
32. Moon, Y., Rajagopalan, B. &Lall, U. [1995] "Estimation of mutual information using kernel density estimators," Phys. Rev. E 52, 2318-2321.
33. Naa, S., Jina, S.-H., Kima, S. &Hamb, B.-J. [2002] "EEG in schizophrenic patients: Mutual information analysis," Clin. Neurophysiol. 113, 1954-1960.
34. Nicolaou, N. &Nasuto, S. [2005] "Mutual information for EEG analysis," Proc. 4th IEEE EMBSS UKRI Postgraduate Conf. Adv. in Biomed. Eng. and Med. Phys. (PGBIOMED'05), Reading, UK, pp. 23-24.
35. Palus, M. [1993] "Identifying and quantifying chaos by using information-theoretic functionals," in Time Series Prediction: Forecasting the Future and Understanding the Past, Santa Fe Institute Studies in the Sciences of Complexity, Vol.XV, eds. Weigend, A. &Gershenfeld, N. (Addison-Wesley, Reading), pp. 387- 413.
36. Palus, M. [1995] "Testing for nonlinearity using redundancies: Quantitative and qualitative aspects," Physica D 80, 186-205.
37. Paninski, L. [2003] "Estimation of entropy and mutual information," Neural Comput. 15, 1191-1253.
38. Pardo, J. [1995] "Some applications of the useful mutual information," Appl. Math. Comput. 72, 33-50.
39. Priness, I., Maimon, O. &Ben-Gal, I. [2007] "Evaluation of gene-expression clustering via mutual information distance measure," Bioinformatics 8, 111.
40. Rissanen, J. [1992] Stochastic Complexity in Statistical Inquiry (World Scientific, Singapore).
41. Roulston, M. [1997] "Significance testing of information theoretic functionals," Physica D 110, 62-66.
42. Schmid, M., Conforto, S., Bibbo, D. &D'Alessio, T. [2004] "Respiration and postural sway: Detection of phase synchronizations and interactions," Hum. Movement Sci. 23, 105-119.
43. Scott, D. [1979] "On optimal and data-based histograms," Biometrika 66, 605-610.
44. Sheather, S. &Jones, M. [1991] "A reliable data-based bandwidth selection method for kernel density estimation," J. Roy. Stat. Soc. B 53, 683-690.
45. Silverman, B. [1986] Density Estimation for Statistics and Data Analysis (Chapman and Hall, London)
46. Steuer, R., Kurths, J., Daub, C., Weise, J. &Selbig, J. [2002] "The mutual information: Detecting and evaluating dependences between variables," Bioinformatics 18, S231-S240.
47. Sturge, H. [1926] "The choice of a class interval," J. Am. Stat. Assoc. 21, 65-66.
48. Terrell, G. &Scott, D. [1985] "Oversmooth nonparametric density estimates," J. Am. Stat. Assoc. 80, 209- 214
49. Tourassi, G., Frederick, E., Markey, M. &Floyd, C. [2001] "Application of the mutual information criterion for feature selection in computer-aided diagnosis," Med. Phys. 28, 2394-2402.
50. Trappenberg, T., Ouyang, J. &Back, A. [2006] "Input variable selection: Mutual information and linear mixing measures," IEEE T. Knowl. Data Eng. 18, 37-46.
51. Treves, A. &Panzeri, S. [1995] "The upward bias in measures of information derived from limited data samples," Neural Comput. 7, 399-407.
52. Tukey, J. &Mosteller, F. [1977] Data Analysis and Regression (Addison-Wesley, Reading, MA).
53. Walter, P. [1975] Ergodic Theory - Introductory Lectures Notes (Springer, Berlin).
54. Wand, M. &Jones, M. [1993] "Comparison of smoothing parameterizations in bivariate kernel density estimation," J. Am. Stat. Assoc. 88, 520-528.
55. Wand, M. &Jones, M. [1995] Kernel Smoothing (Chapman and Hall, London).
56. Wicks, R., Chapman, S. &Dendy, R. [2007] "Mutual information as a tool for identifying phase transitions in dynamical complex systems with limited data," Phys. Rev. E 75, 051125.