Efficient estimation of information transfer

Understanding Complex Systems

Volumn , Issue , 2014, Pages 37-58

  Vicente, Raul ^a   Wibral, Michael ^b  

^a MAX PLANCK INSTITUTE FOR BRAIN RESEARCH   (Germany)

^b GOETHE UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84901756489     PISSN: 18600832     EISSN: 18600840     Source Type: Book Series    
DOI: 10.1007/978-3-642-54474-3_2     Document Type: Review

References (79)

1. Hubel, D.H., Wiesel, T.N.: Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology 195(1), 215-243 (1968)
2. Gray, C.M., Knig, P., Engel, A.K., Singer, W.: Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature 338(6213), 334-337 (1989) (Pubitemid 19082973)
3. Canolty, R.T., Knight, R.T.: The functional role of cross-frequency coupling. Trends in Cognitive Sciences 14(11), 506-515 (2010)
4. Victor, J.D.: Approaches to information-theoretic analysis of neural activity. Biological Theory 1(3), 302 (2006)
5. Lizier, J.T.: The Local Information Dynamics of Distributed Computation in Complex Systems. Springer theses. Springer (2013)
6. Lehmann, E.L., Casella, G.: Theory of point estimation, vol. 31. Springer (1998)
7. Niso, G., Brua, R., Pereda, E., Gutirrez, R., Bajo, R., Maest, F., Del-Pozo, F.: Hermes: Towards an integrated toolbox to characterize functional and effective brain connectivity. Neuroinformatics 11, 405-434 (2013)
8. Pereda, E., Quiroga, R.Q., Bhattacharya, J.: Nonlinear multivariate analysis of neurophysiological signals. Progress in Neurobiology 77(1), 1-37 (2005) (Pubitemid 41654771)
9. Cover, T.M., Thomas, J.A.: Elements of information theory. Wiley-Interscience, New York (1991)
10. Latham, P.E., Nirenberg, S.: Synergy, redundancy, and independence in population codes, revisited. J. Neurosci. 25(21), 5195-5206 (2005) (Pubitemid 40731562)
11. Williams, P.L., Beer, R.D.: Nonnegative decomposition of multivariate information. arXiv preprint arXiv:1004.2515 (2010)
12. Rieke, F., Warland, D., Deruytervansteveninck, R., Bialek, W.: Spikes: exploring the neural code (computational neuroscience). MIT Press (1999)
13. Shannon, C.E.: The bell technical journal. A Mathematical Theory of Communication 27(4), 379-423 (1948)
14. Shannon, C.E., Weaver, W.: The mathematical theory of communication, urbana, il, vol. 19(7), p. 1. University of Illinois Press (1949)
15. Barlow, H.B.: Possible principles underlying the transformation of sensory messages. Sensory Communication, 217-234 (1961)
16. de Ruyter van Steveninck, R.R., Laughlin, S.B.: The rate of information transfer at graded-potential synapses. Nature 379(6566), 642-645 (1996) (Pubitemid 26052825)
17. Lewicki, M.S.: Efficient coding of natural sounds. Nature Neuroscience 5(4), 356-363 (2002) (Pubitemid 34279816)
18. Olshausen, B.A., Field, D.J.: Sparse coding of sensory inputs. Current Opinion in Neurobiology 14(4), 481-487 (2004) (Pubitemid 39078497)
19. Johnson, D.H.: Information theory and neuroscience: Why is the intersection so small? In: IEEE Information Theory Workshop, ITW 2008, pp. 104-108 (2008)
20. Shannon, C.E.: The bandwagon. IRE Transactions on Information Theory 2(1), 3 (1956)
21. Nirenberg, S.H., Victor, J.D.: Analyzing the activity of large populations of neurons: how tractable is the problem? Current Opinion in Neurobiology 17(4), 397-400 (2007) (Pubitemid 47531946)
22. Johnson, D.H.: Information theory and neural information processing. IEEE Transactions on Information Theory 56(2), 653-666 (2010)
23. Schreiber, T.: Measuring information transfer. Phys. Rev. Lett. 85(2), 461-464 (2000)
24. Wiener, N.: The theory of prediction. In: Beckmann, E.F. (ed.) Modern Mathematics for the Engineer. McGraw-Hill, New York (1956)
25. Vicente, R., Wibral, M., Lindner, M., Pipa, G.: Transfer entropy - a model-free measure of effective connectivity for the neurosciences. J. Comput. Neurosci. 30(1), 45-67 (2011)
26. Ay, N., Polani, D.: Information flows in causal networks. Adv. Complex Syst. 11, 17 (2008)
27. Kaiser, A., Schreiber, T.: Information transfer in continuous processes. Physica D 166, 43 (2002)
28. Chicharro, D., Ledberg, A.: When two become one: the limits of causality analysis of brain dynamics. PLoS One 7(3), e32466 (2012)
29. Chávez, M., Martinerie, J., Le Van Quyen, M.: Statistical assessment of nonlinear causality: application to epileptic EEG signals. J. Neurosci. Methods 124(2), 113-128 (2003) (Pubitemid 36442693)
30. Wibral, M., Rahm, B., Rieder, M., Lindner, M., Vicente, R., Kaiser, J.: Transfer entropy in magnetoencephalographic data: Quantifying information flow in cortical and cerebellar networks. Prog. Biophys. Mol. Biol. 105(1-2), 80-97 (2011)
31. Vicente, R., Gollo, L.L., Mirasso, C.R., Fischer, I., Pipa, G.: Dynamical relaying can yield zero time lag neuronal synchrony despite long conduction delays. Proceedings of the National Academy of Sciences 105(44), 17157-17162 (2008)
32. Kay, S.M.: Fundamentals of statistical signal processing. In: Estimation Theory, vol. 1 (1993)
33. Hlaváčková-Schindler, K., Paluš, M., Vejmelka, M., Bhattacharya, J.: Causality detection based on information-theoretic approaches in time series analysis. Physics Reports 441(1), 1-46 (2007)
34. Gourevitch, B., Eggermont, J.J.: Evaluating information transfer between auditory cortical neurons. J. Neurophysiol. 97(3), 2533-2543 (2007) (Pubitemid 46411182)
35. Ito, S., Hansen, M.E., Heiland, R., Lumsdaine, A., Litke, A.M., Beggs, J.M.: Extending transfer entropy improves identification of effective connectivity in a spiking cortical network model. PLoS One 6(11), e27431 (2011)
36. Li, Z., Li, X.: Estimating temporal causal interaction between spike trains with permutation and transfer entropy. PLoS One 8(8), e70894 (2013)
37. Barnett, L., Barrett, A.B., Seth, A.K.: Granger causality and transfer entropy are equivalent for Gaussian variables. Phys. Rev. Lett. 103(23), 238701 (2009)
38. Hlavácková-Schindler, K.: Equivalence of Granger causality and transfer entropy: A generalization. Applied Mathematical Sciences 5(73), 3637-3648 (2011)
39. Nichols, J.M., Seaver, M., Trickey, S.T., Todd, M.D., Olson, C., Overbey, L.: Detecting nonlinearity in structural systems using the transfer entropy. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 72(4 Pt. 2), 046217 (2005)
40. Hahs, D.W., Pethel, S.D.: Transfer entropy for coupled autoregressive processes. Entropy 15(3), 767-788 (2013)
41. Barnett, L., Bossomaier, T.: Transfer entropy as a log-likelihood ratio. Physical Review Letters 109(13), 138105 (2012)
42. Miller, G.A.: Note on the bias of information estimates. Information Theory in Psychology: Problems and Methods 2, 95-100 (1955)
43. Efron, B., Stein, C.: The jackknife estimate of variance. The Annals of Statistics, 586-596 (1981)
44. Pompe, B., Runge, J.: Momentary information transfer as a coupling measure of time series. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 83(5 Pt. 1), 051122 (2011)
45. Wibral, M., Pampu, N., Priesemann, V., Siebenhhner, F., Seiwert, H., Lindner, M., Lizier, J.T., Vicente, R.: Measuring information-transfer delays. PLoS One 8(2), e55809 (2013)
46. Paluš, M.: Testing for nonlinearity using redundancies: Quantitative and qualitative aspects. Physica D: Nonlinear Phenomena 80(1), 186-205 (1995)
47. Fraser, A.M., Swinney, H.L.: Independent coordinates for strange attractors from mutual information. Phys. Rev. A. 33, 1134 (1986)
48. Darbellay, G.A., Vajda, I.: Estimation of the information by an adaptive partitioning of the observation space. IEEE Transactions on Information Theory 45(4), 1315-1321 (1999)
49. Cellucci, C.J., Albano, A.M., Rapp, P.E.: Statistical validation of mutual information calculations: Comparison of alternative numerical algorithms. Physical Review E 71(6), 066208 (2005)
50. Daub, C.O., Steuer, R., Selbig, J., Kloska, S.: Estimating mutual information using bspline functions-an improved similarity measure for analysing gene expression data. BMC Bioinformatics 5(1), 118 (2004)
51. Victor, J.: Binless strategies for estimation of information from neural data. Phys. Rev. E 72, 051903 (2005)
52. Silverman, B.W.: Density estimation for statistics and data analysis, vol. 26. CRC Press (1986)
53. Young-Il, M., Rajagopalan, B., Lall, U.: Estimation of mutual information using kernel density estimators. Physical Review E 52(3), 2318 (1995)
54. Steuer, R., Kurths, J., Daub, C.O., Weise, J., Selbig, J.: The mutual information: detecting and evaluating dependencies between variables. Bioinformatics 18(suppl. 2), S231-S240 (2002) (Pubitemid 41073239)
55. Kozachenko, L.F., Leonenko, N.N.: Sample estimate of entropy of a random vector. Probl. Inform. Transm. 23, 95-100 (1987)
56. Knuth, D.E.: The art of computer programming. In: Sorting and Searching, vol. 3 (1973)
57. Vaidya, P.M.: An O(n logn) algorithm for the all-nearest-neighbors problem. Discrete & Computational Geometry 4(1), 101-115 (1989)
58. Zezula, P., Amato, G., Dohnal, V., Batko, M.: Similarity search: The metric space approach. Advances in Database Systems, vol. 32. Springer, Secaucus (2005)
59. Heineman, G.T., Pollice, G., Selkow, S.: Algorithms in a Nutshell. O'Reilly Media, Inc. (2009)
60. Merkwirth, P., Lauterborn, W.: Fast nearest-neighbor searching for nonlinear signal processing. Phys. Rev. E Stat. Phys. Plasmas Fluids Relat. Interdiscip. Topics 62(2 Pt. A), 2089-2097 (2000)
61. Wollstadt, P., Martinez-Zarzuela, M., Vicente, R., Wibral, M.: Efficient transfer entropy analysis of nonstationary neural time series. arXiv preprint arXiv:1401.4068 (2014)
62. Kraskov, A., Stoegbauer, H., Grassberger, P.: Estimating mutual information. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 69(6 Pt. 2), 066138 (2004)
63. Kraskov, A.: Synchronization and Interdependence measures and their application to the electroencephalogram of epilepsy patients and clustering of data. PhD thesis, University of Wuppertal (February 2004)
64. Gomez-Herrero, G., Wu, W., Rutanen, K., Soriano, M.C., Pipa, G., Vicente, R.: Assessing coupling dynamics from an ensemble of time series. arXiv preprint arXiv:1008.0539 (2010)
65. Takens, F.: Detecting Strange Attractors in Turbulence. In: Dynamical Systems and Turbulence, Warwick, 1980. Lecture Notes in Mathematics, vol. 898, pp. 366-381. Springer (1981)
66. Kantz, H., Schreiber, T.: Nonlinear Time Series Analysis, 2nd edn. Cambridge University Press (November 2003)
67. Cao, L.Y.: Practical method for determining the minimum embedding dimension of a scalar time series. Physica A 110, 43-50 (1997)
68. Ragwitz, M., Kantz, H.: Markov models from data by simple nonlinear time series predictors in delay embedding spaces. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 65(5 Pt. 2), 056201 (2002)
69. Theiler, J.: Spurious dimension from correlation algorithms applied to limited time-series data. Physical Review A 34(3), 2427 (1986)
70. Vejmelka, M., Hlaváčková-Schindler, K.: Mutual information estimation in higher dimensions: A speed-up of a k-nearest neighbor based estimator. In: Beliczynski, B., Dzielinski, A., Iwanowski, M., Ribeiro, B. (eds.) ICANNGA 2007, Part I. LNCS, vol. 4431, pp. 790-797. Springer, Heidelberg (2007)
71. Lindner, M., Vicente, R., Priesemann, V., Wibral, M.: Trentool: A Matlab open source toolbox to analyse information flow in time series data with transfer entropy. BMC Neurosci. 12(119), 1-22 (2011)
72. Lindner, M., Vicente, R., Wibral, M., Pampu, N., Wollstadt, P., Martinez-Zarzuela, M.: TRENTOOL, http://www.trentool.de
73. Rutanen, K.: TIM 1.2.0, http://www.cs.tut.fi/~timhome/tim-1.2.0/tim.htm
74. Lizier, J.: Java Information Dynamics Toolkit, http://code.google.com/p/ information-dynamics-toolkit/
75. Faes, L., Nollo, G., Porta, A.: Non-uniform multivariate embedding to assess the information transfer in cardiovascular and cardiorespiratory variability series. Comput. Biol. Med. 42(3), 290-297 (2012)
76. Lizier, J.T., Rubinov, M.: Inferring effective computational connectivity using incrementally conditioned multivariate transfer entropy. BMC Neuroscience 14(suppl. 1), P337 (2013)
77. Wiener, N.: Cybernetics. Hermann, Paris (1948)
78. Davies, P.C.W., Gregersen, N.H.: Information and the Nature of Reality, vol. 3. Cambridge University Press, Cambridge (2010)
79. Barnett, L., Lizier, J.T., Harré, M., Seth, A.K., Bossomaier, T.: Information flow in a kinetic Ising model peaks in the disordered phase. Physical Review Letters 111(17), 177203 (2013)