Measuring the dynamics of information processing on a local scale in time and space

Understanding Complex Systems

Volumn , Issue , 2014, Pages 161-193

  Lizier, Joseph T ^a  

^a CSIRO   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84901776855     PISSN: 18600832     EISSN: 18600840     Source Type: Book Series    
DOI: 10.1007/978-3-642-54474-3_7     Document Type: Review

References (98)

1. Ash, R.B.: Information Theory. Dover Publishers, Inc., New York (1965)
2. Ay, N., Polani, D.: Information Flows in Causal Networks. Advances in Complex Systems 11(1), 17-41 (2008)
3. Bandt, C., Pompe, B.: Permutation entropy: A natural complexity measure for time series. Physical Review Letters 88(17), 174102 (2002)
4. Barnett, L., Barrett, A.B., Seth, A.K.: Granger Causality and Transfer Entropy Are Equivalent for Gaussian Variables. Physical Review Letters 103(23), 238701 (2009)
5. Barnett, L., Bossomaier, T.: Transfer Entropy as a Log-Likelihood Ratio. Physical Review Letters 109, 138105 (2012)
6. Barnett, L., Buckley, C.L., Bullock, S.: Neural complexity and structural connectivity. Physical Review E 79(5), 051914 (2009)
7. Boedecker, J., Obst, O., Lizier, J.T., Mayer, N.M., Asada, M.: Information processing in echo state networks at the edge of chaos. Theory in Biosciences 131(3), 205-213 (2012)
8. Bressler, S.L., Tang, W., Sylvester, C.M., Shulman, G.L., Corbetta, M.: Top-Down Control of Human Visual Cortex by Frontal and Parietal Cortex in Anticipatory Visual Spatial Attention. Journal of Neuroscience 28(40), 10056-10061 (2008)
9. Ceguerra, R.V., Lizier, J.T., Zomaya, A.Y.: Information storage and transfer in the synchronization process in locally-connected networks. In: Proceedings of the 2011 IEEE Symposium on Artificial Life (ALIFE), pp. 54-61. IEEE (2011)
10. Chávez, M., Martinerie, J., Le Van Quyen, M.: Statistical assessment of nonlinear causality: application to epileptic EEG signals. Journal of Neuroscience Methods 124(2), 113-128 (2003) (Pubitemid 36442693)
11. Chicharro, D., Ledberg, A.: When Two Become One: The Limits of Causality Analysis of Brain Dynamics. PLoS One 7(3), e32466 (2012)
12. Couzin, I.D., James, R., Croft, D.P., Krause, J.: Social Organization and Information Transfer in Schooling Fishes. In: Brown, C., Laland, K.N., Krause, J. (eds.) Fish Cognition and Behavior, Fish and Aquatic Resources, pp. 166-185. Blackwell Publishing (2006)
13. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley-Interscience, New York (1991)
14. Crutchfield, J.P., Feldman, D.P.: Regularities Unseen, Randomness Observed: Levels of Entropy Convergence. Chaos 13(1), 25-54 (2003) (Pubitemid 36419900)
15. Crutchfield, J.P., Young, K.: Inferring statistical complexity. Physical Review Letters 63(2), 105-108 (1989)
16. Dasan, J., Ramamohan, T.R., Singh, A., Nott, P.R.: Stress fluctuations in sheared Stokesian suspensions. Physical Review E 66(2), 021409 (2002)
17. Derdikman, D., Hildesheim, R., Ahissar, E., Arieli, A., Grinvald, A.: Imaging spatiotemporal dynamics of surround inhibition in the barrels somatosensory cortex. The Journal of Neuroscience 23(8), 3100-3105 (2003) (Pubitemid 36531949)
18. De Weese, M.R., Meister, M.: How to measure the information gained from one symbol. Network: Computation in Neural Systems 10, 325-340 (1999)
19. Effenberger, F.: A primer on information theory, with applications to neuroscience, arXiv:1304.2333 (2013), http://arxiv.org/abs/1304.2333
20. Faes, L., Nollo, G., Porta, A.: Information-based detection of nonlinear Granger causality in multivariate processes via a nonuniform embedding technique. Physical Review E 83, 051112 (2011)
21. Faes, L., Nollo, G., Porta, A.: Non-uniform multivariate embedding to assess the information transfer in cardiovascular and cardiorespiratory variability series. Computers in Biology and Medicine 42(3), 290-297 (2012)
22. Fano, R.M.: Transmission of information: a statistical theory of communications. MIT Press, Cambridge (1961)
23. Flecker, B., Alford, W., Beggs, J.M., Williams, P.L., Beer, R.D.: Partial information decomposition as a spatiotemporal filter. Chaos: An Interdisciplinary Journal of Nonlinear Science 21(3), 037104 (2011)
24. Frenzel, S., Pompe, B.: Partial Mutual Information for Coupling Analysis of Multivariate Time Series. Physical Review Letters 99(20), 204101 (2007)
25. Friston, K.J., Harrison, L., Penny, W.: Dynamic causal modelling. Neuro Image 19(4), 1273-1302 (2003) (Pubitemid 37025831)
26. Gomez-Herrero, G., Wu, W., Rutanen, K., Soriano, M.C., Pipa, G., Vicente, R.: Assessing coupling dynamics from an ensemble of time series. arXiv:1008.0539 (2010), http://arxiv.org/abs/1008.0539
27. Gong, P., van Leeuwen, C.: Distributed Dynamical Computation in Neural Circuits with Propagating Coherent Activity Patterns. PLoS Computational Biology 5(12) (2009)
28. Granger, C.W.J.: Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, 424-438 (1969)
29. Grassberger, P.: New mechanism for deterministic diffusion. Physical Review A 28(6), 3666 (1983)
30. Grassberger, P.: Long-range effects in an elementary cellular automaton. Journal of Statistical Physics 45(1-2), 27-39 (1986)
31. Grassberger, P.: Toward a quantitative theory of self-generated complexity. International Journal of Theoretical Physics 25(9), 907-938 (1986)
32. Griffith, V., Koch, C.: Quantifying synergistic mutual information. In: Prokopenko, M. (ed.) Guided Self-Organization: Inception, pp. 159-190. Springer, Heidelberg (2014)
33. Hanson, J.E., Crutchfield, J.P.: The Attractor-Basin Portait of a Cellular Automaton. Journal of Statistical Physics 66, 1415-1462 (1992)
34. Hanson, J.E., Crutchfield, J.P.: Computational mechanics of cellular automata: An example. Physica D 103(1-4), 169-189 (1997) (Pubitemid 127701408)
35. Harder, M., Salge, C., Polani, D.: Bivariate Measure of Redundant Information. Physical Review E 87, 012130 (2013)
36. Helvik, T., Lindgren, K., Nordahl, M.G.: Local information in one-dimensional cellular automata. In: Sloot, P.M.A., Chopard, B., Hoekstra, A.G. (eds.) ACRI 2004. LNCS, vol. 3305, pp. 121-130. Springer, Heidelberg (2004) (Pubitemid 39742826)
37. Helvik, T., Lindgren, K., Nordahl, M.G.: Continuity of Information Transport in Surjective Cellular Automata. Communications in Mathematical Physics 272(1), 53-74 (2007)
38. Hinrichs, H., Heinze, H.J., Schoenfeld, M.A.: Causal visual interactions as revealed by an information theoretic measure and fMRI. Neuro Image 31(3), 1051-1060 (2006) (Pubitemid 44183793)
39. Honey, C.J., Kotter, R., Breakspear, M., Sporns, O.: Network structure of cerebral cortex shapes functional connectivity on multiple time scales. Proceedings of the National Academy of Science 104(24), 10,240-10,245 (2007) (Pubitemid 47175293)
40. Ito, S., Hansen, M.E., Heiland, R., Lumsdaine, A., Litke, A.M., Beggs, J.M.: Extending Transfer Entropy Improves Identification of Effective Connectivity in a Spiking Cortical Network Model. PLoS One 6(11), e27431 (2011)
41. Kantz, H., Schreiber, T.: Nonlinear Time Series Analysis. Cambridge University Press, Cambridge (1997)
42. Katare, S., West, D.H.: Optimal complex networks spontaneously emerge when information transfer is maximized at least expense: A design perspective. Complexity 11(4), 26-35 (2006)
43. Kerr, C.C., Van Albada, S.J., Neymotin, S.A., Chadderdon, G.L., Robinson, P.A., Lytton, W.W.: Cortical information flow in parkinson's disease: a composite network/field model. Frontiers in Computational Neuroscience 7(39) (2013)
44. Kraskov, A.: Synchronization and Interdependence Measures and their Applications to the Electroencephalogram of Epilepsy Patients and Clustering of Data. Publication Series of the John von Neumann Institute for Computing, vol. 24. John von Neumann Institute for Computing, Jülich (2004)
45. Kraskov, A., Stögbauer, H., Grassberger, P.: Estimating mutual information. Physical Review E 69(6), 066138 (2004)
46. Langton, C.G.: Computation at the edge of chaos: phase transitions and emergent computation. Physica D 42(1-3), 12-37 (1990)
47. Levina, A., Herrmann, J.M., Geisel, T.: Dynamical synapses causing self-organized criticality in neural networks. Nature Physics 3(12), 857-860 (2007)
48. Liang, H., Ding, M., Bressler, S.L.: Temporal dynamics of information flow in the cerebral cortex. Neurocomputing 38-40, 1429-1435 (2001) (Pubitemid 35276791)
49. Lindner, M., Vicente, R., Priesemann, V., Wibral, M.: TRENTOOL: A Matlab open source toolbox to analyse information flow in time series data with transfer entropy. BMC Neuroscience 12(1), 119 (2011)
50. Lizier, J., Heinzle, J., Soon, C., Haynes, J.D., Prokopenko, M.: Spatiotemporal information transfer pattern differences in motor selection. BMC Neuroscience 12(Suppl. 1), P261 (2011)
51. Lizier, J.T.: JIDT: An information-theoretic toolkit for studying the dynamics of complex systems (2012), https://code.google.com/p/information- dynamics-toolkit/
52. Lizier, J.T.: The Local Information Dynamics of Distributed Computation in Complex Systems. Springer Theses. Springer, Heidelberg (2013)
53. Lizier, J.T., Flecker, B., Williams, P.L.: Towards a synergy-based approach to measuring information modification. In: Proceedings of the 2013 IEEE Symposium on Artificial Life (ALIFE), pp. 43-51. IEEE (2013)
54. Lizier, J.T., Heinzle, J., Horstmann, A., Haynes, J.D., Prokopenko, M.: Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity. Journal of Computational Neuroscience 30(1), 85-107 (2011)
55. Lizier, J.T., Pritam, S., Prokopenko, M.: Information dynamics in small-world Boolean networks. Artificial Life 17(4), 293-314 (2011)
56. Lizier, J.T., Prokopenko, M.: Differentiating information transfer and causal effect. European Physical Journal B 73(4), 605-615 (2010)
57. Lizier, J.T., Prokopenko, M., Tanev, I., Zomaya, A.Y.: Emergence of Glider-like Structures in a Modular Robotic System. In: Bullock, S., Noble, J., Watson, R., Bedau, M.A. (eds.) Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems (ALife XI), Winchester, UK, pp. 366-373. MIT Press, Cambridge (2008)
58. Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Detecting Non-trivial Computation in Complex Dynamics. In: Almeida e Costa, F., Rocha, L.M., Costa, E., Harvey, I., Coutinho, A. (eds.) ECAL 2007. LNCS (LNAI), vol. 4648, pp. 895-904. Springer, Heidelberg (2007)
59. Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Local information transfer as a spatiotemporal filter for complex systems. Physical Review E 77(2), 026110 (2008)
60. Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Information modification and particle collisions in distributed computation. Chaos 20(3), 037109 (2010)
61. Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Coherent information structure in complex computation. Theory in Biosciences 131(3), 193-203 (2012)
62. Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Local measures of information storage in complex distributed computation. Information Sciences 208, 39-54 (2012)
63. Lizier, J.T., Rubinov, M.: Multivariate construction of effective computational networks from observational data. Tech. Rep. Preprint 25/2012, Max Planck Institute for Mathematics in the Sciences (2012)
64. Lungarella, M., Sporns, O.: Mapping Information Flow in Sensorimotor Networks. PLoS Computational Biology 2(10), e144 (2006)
65. MacKay, D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge (2003)
66. Mahoney, J.R., Ellison, C.J., James, R.G., Crutchfield, J.P.: How hidden are hidden processes? A primer on crypticity and entropy convergence. Chaos 21(3), 037112 (2011)
67. Manchanda, K., Yadav, A.C., Ramaswamy, R.: Scaling behavior in probabilistic neuronal cellular automata. Physical Review E 87, 012704 (2013)
68. Manning, C.D., Schütze, H.: Foundations of Statistical Natural Language Processing. The MIT Press, Cambridge (1999)
69. Marinazzo, D., Wu, G., Pellicoro, M., Angelini, L., Stramaglia, S.: Information flow in networks and the law of diminishing marginal returns: evidence from modeling and human electroencephalographic recordings. PLoS One 7(9), e45026 (2012)
70. Mitchell, M.: Computation in Cellular Automata: A Selected Review. In: Gramss, T., Bornholdt, S., Gross, M., Mitchell, M., Pellizzari, T. (eds.) Non-Standard Computation, pp. 95-140. VCH Verlagsgesellschaft, Weinheim (1998)
71. Mitchell, M., Crutchfield, J.P., Hraber, P.T.: Evolving Cellular Automata to Perform Computations: Mechanisms and Impediments. Physica D 75, 361-391 (1994)
72. Nakajima, K., Li, T., Kang, R., Guglielmino, E., Caldwell, D.G., Pfeifer, R.: Local information transfer in soft robotic arm. In: 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1273-1280. IEEE (2012)
73. Obst, O., Boedecker, J., Asada, M.: Improving Recurrent Neural Network Performance Using Transfer Entropy. In:Wong, K.W., Mendis, B.S.U., Bouzerdoum, A. (eds.) ICONIP 2010, Part II. LNCS, vol. 6444, pp. 193-200. Springer, Heidelberg (2010)
74. Pearl, J.: Causality: Models, Reasoning, and Inference. Cambridge University Press, Cambridge (2000)
75. Priesemann, V., Munk, M., Wibral, M.: Subsampling effects in neuronal avalanche distributions recorded in vivo. BMC Neuroscience 10(1), 40 (2009)
76. Prokopenko, M., Boschietti, F., Ryan, A.J.: An Information-Theoretic Primer on Complexity, Self-Organization, and Emergence. Complexity 15(1), 11-28 (2009)
77. Prokopenko, M., Gerasimov, V., Tanev, I.: Evolving Spatiotemporal Coordination in a Modular Robotic System. In: Nolfi, S., Baldassarre, G., Calabretta, R., Hallam, J.C.T., Marocco, D., Meyer, J.-A., Miglino, O., Parisi, D. (eds.) SAB 2006. LNCS (LNAI), vol. 4095, pp. 558-569. Springer, Heidelberg (2006) (Pubitemid 44553386)
78. Prokopenko, M., Lizier, J.T., Obst, O., Wang, X.R.: Relating Fisher information to order parameters. Physical Review E 84, 41116 (2011)
79. Prokopenko, M., Lizier, J.T., Price, D.C.: On thermodynamic interpretation of transfer entropy. Entropy 15(2), 524-543 (2013)
80. Rubinov, M., Lizier, J., Prokopenko, M., Breakspear, M.: Maximized directed information transfer in critical neuronal networks. BMC Neuroscience 12(supp.l 1), P18 (2011)
81. Schreiber, T.: Interdisciplinary application of nonlinear time series methods - the generalized dimensions. Physics Reports 308, 1-64 (1999)
82. Schreiber, T.: Measuring Information Transfer. Physical Review Letters 85(2), 461-464 (2000)
83. Shalizi, C.R.: Causal Architecture, Complexity and Self-Organization in Time Series and Cellular Automata. Ph.D. thesis, University of Wisconsin-Madison (2001)
84. Shalizi, C.R., Haslinger, R., Rouquier, J.B., Klinkner, K.L., Moore, C.: Automatic filters for the detection of coherent structure in spatiotemporal systems. Physical Review E 73(3), 036104 (2006)
85. Shannon, C.E.: A mathematical theory of communication. Bell System Technical Journal 27, 379-423, 623-656 (1948)
86. Soon, C.S., Brass, M., Heinze, H.J., Haynes, J.D.: Unconscious determinants of free decisions in the human brain. Nature Neuroscience 11(5), 543-545 (2008) (Pubitemid 351595206)
87. Staniek, M., Lehnertz, K.: Symbolic transfer entropy. Physical Review Letters 100(15), 158101 (2008)
88. Stramaglia, S., Wu, G.R., Pellicoro, M., Marinazzo, D.: Expanding the transfer entropy to identify information subgraphs in complex systems. In: Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 3668-3671. IEEE (2012)
89. Ver Steeg, G., Galstyan, A.: Information-theoretic measures of influence based on content dynamics. In: Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, pp. 3-12 (2013)
90. Verdes, P.F.: Assessing causality from multivariate time series. Physical Review E 72(2), 026222 (2005)
91. Vicente, R., Wibral, M., Lindner, M., Pipa, G.: Transfer entropy-a model-free measure of effective connectivity for the neurosciences. Journal of Computational Neuroscience 30(1), 45-67 (2011)
92. Wang, X.R., Miller, J.M., Lizier, J.T., Prokopenko, M., Rossi, L.F.: Quantifying and Tracing Information Cascades in Swarms. PLoS One 7(7), e40084 (2012)
93. Wibral, M., Pampu, N., Priesemann, V., Siebenhühner, F., Seiwert, H., Lindner, M., Lizier, J.T., Vicente, R.: Measuring Information-Transfer delays. PLoS One 8(2), e55809 (2013)
94. Wibral, M., Rahm, B., Rieder, M., Lindner, M., Vicente, R., Kaiser, J.: Transfer entropy in magnetoencephalographic data: quantifying information flow in cortical and cerebellar networks. Progress in Biophysics and Molecular Biology 105(1-2), 80-97 (2011)
95. Williams, P.L., Beer, R.D.: Nonnegative Decomposition of Multivariate Information. arXiv:1004.2515 (2010), http://arxiv.org/abs/1004.2515
96. Williams, P.L., Beer, R.D.: Generalized Measures of Information Transfer. arXiv:1102.1507 (2011), http://arxiv.org/abs/1102.1507
97. Wolfram, S.: A New Kind of Science. Wolfram Media, Champaign (2002)
98. Wuensche, A.: Classifying cellular automata automatically: Finding gliders, filtering, and relating space-time patterns, attractor basins, and the Z parameter. Complexity 4(3), 47-66 (1999) (Pubitemid 129180503)