Local active information storage as a tool to understand distributed neural information processing

Frontiers in Neuroinformatics

Volumn 8, Issue JAN, 2014, Pages

  Wibral, Michael ^a   Lizier, Joseph T ^b   Vögler, Sebastian ^c   Priesemann, Viola ^d   Galuske, Ralf ^c  

^a GOETHE UNIVERSITY   (Germany)

^b CSIRO   (Australia)

^c DARMSTADT UNIVERSITY OF TECHNOLOGY   (Germany)

^d MAX PLANCK INSTITUTE FOR DYNAMICS AND SELF ORGANIZATION   (Germany)

Author keywords

Complex systems; Distributed computation; Information storage; Local information dynamics; Neural dynamics; Predictive coding; Visual system; Voltage sensitive dye imaging

Indexed keywords

ARTICLE; ARTIFICIAL NEURAL NETWORK; CODING ALGORITHM; COMPUTER PROGRAM; CORRELATION ANALYSIS; ENTROPY; EXCITATION AND STIMULATION; INFORMATION PROCESSING; INFORMATION STORAGE; LOCAL ACTIVE INFORMATION STORAGE; MATHEMATICAL COMPUTING; MEDICAL INFORMATICS; NERVE CELL; SPATIOTEMPORAL ANALYSIS; SPEARMAN RANK CORRELATION ANALYSIS; TEMPORAL SUMMATION; VISUAL CORTEX; VISUAL STIMULATION; VOLTAGE SENSITIVE DYE IMAGING;

EID: 84893355380     PISSN: 16625196     EISSN: None     Source Type: Journal    
DOI: 10.3389/fninf.2014.00001     Document Type: Article

References (52)

1. Ay, N., and Polani, D. (2008). Information flows in causal networks. Adv. Complex Syst. 11, 17. doi: 10.1142/S0219525908001465
2. Bastos, A. M., Usrey, W. M., Adams, R. A., Mangun, G. R., Fries, P., and Friston, K. J. (2012). Canonical microcircuits for predictive coding. Neuron 76, 695-711. doi: 10.1016/j.neuron.2012.10.038
3. Butts, D. A. (2003). How much information is associated with a particular stimulus? Network 14, 177-187. doi: 10.1088/0954-898X/14/2/301
4. Butts, D. A., and Goldman, M. S. (2006). Tuning curves, neuronal variability, and sensory coding. PLoS Biol. 4:e92. doi: 10.1371/journal.pbio.0040092
5. Cover, T. M., and Thomas, J. A. (1991). Elements of Information Theory. New York, NY: Wiley-Interscience. doi: 10.1002/0471200611
6. Crutchfield, J. P., and Feldman, D. P. (2003). Regularities unseen, randomness observed: levels of entropy convergence. Chaos 13, 25-54. doi: 10.1063/1.1530990
7. Dasgupta, S., Wörgötter, F., and Manoonpong, P. (2013). Information dynamics based self-adaptive reservoir for delay temporal memory tasks. Evol. Syst. 4, 235-249. doi: 10.1007/s12530-013-9080-y
8. Deacon, T. W. (2010). "What is missing from theories of information?" (chapter 8) in Information and the Nature of Reality, eds P. Davies and N. H. Gregersen (Cambridge: Cambridge University Press), 146.
9. DeWeese, M. R., and Meister, M. (1999). How to measure the information gained from one symbol. Network 10, 325-340. doi: 10.1088/0954-898X/10/4/303
10. Faes, L., Nollo, G., and Porta, A. (2012). Non-uniform multivariate embedding to assess the information transfer in cardiovascular and cardiorespiratory variability series. Comput. Biol. Med. 42, 290-297. doi: 10.1016/j.compbiomed.2011.02.007
11. Faes, L., Porta, A., Rossato, G., Adami, A., Tonon, D., Corica, A., et al. (2013). Investigating the mechanisms of cardiovascular and cerebrovascular regulation in orthostatic syncope through an information decomposition strategy. Auton. Neurosci. 178, 76-82. doi: 10.1016/j.autneu.2013.02.013
12. Fano, R. (1961). Transmission of Information: A Statistical Theory of Communications. Cambridge, MA: The MIT Press.
13. Friston, K. (2005). A theory of cortical responses. Philos. Trans. R. Soc. Lond. B Biol. Sci. 360, 815-836. doi: 10.1098/rstb.2005.1622
14. Gómez, C., and Hornero, R. (2010). Entropy and complexity analyses in alzheimer's disease: an MEG study. Open Biomed. Eng. J. 4, 223. doi: 10.2174/1874120701004010223
15. Granger, C. W. J. (1969). Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, 424-438. doi: 10.2307/1912791
16. Kantz, H., and Schreiber, T. (2003). Nonlinear Time Series Analysis. 2nd Edn. Cambridge: Cambridge University Press. doi: 10.1017/CBO9780511755798
17. Knill, D. C., and Pouget, A. (2004). The bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 27, 712-719. doi: 10.1016/j.tins.2004.10.007
18. Kraskov, A., Stoegbauer, H., and Grassberger, P. (2004). Estimating mutual information. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 69, 066138. doi: 10.1103/PhysRevE.69.066138
19. Langton, C. G. (1990). Computation at the edge of chaos: phase transitions and emergent computation. Physica D 42, 12-37. doi: 10.1016/0167-2789(90)90064-V
20. Lindner, M., Vicente, R., Priesemann, V., and Wibral, M. (2011). Trentool: a Matlab open source toolbox to analyse information flow in time series data with transfer entropy. BMC Neurosci. 12:1-22. doi: 10.1186/1471-2202-12-119
21. Lizier, J. T. (2012). JIDT: an information-theoretic toolkit for studying the dynamics of complex systems. Available online at: http://code.google.com/p/information-dynamics-toolkit/
22. Lizier, J. T. (2013). The Local Information Dynamics of Distributed Computation in Complex Systems. Springer theses. Springer. doi: 10.1007/978-3-642-32952-4_2
23. Lizier, J. T., Atay, F. M., and Jost, J. (2012a). Information storage, loop motifs, and clustered structure in complex networks. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 86(2 Pt 2), 026110. doi: 10.1103/PhysRevE.86.026110
24. Lizier, J. T., Prokopenko, M., and Zomaya, A. Y. (2012b). Local measures of information storage in complex distributed computation. Inform. Sci. 208, 39-54. doi: 10.1016/j.ins.2012.04.016
25. Lizier, J. T., Flecker, B., and Williams, P. L. (2013). "Towards a synergy-based approach to measuring information modification, " in Proceedings of the 2013 IEEE Symposium on Artificial Life (ALIFE) (Singapore), 43-51. doi: 10.1109/ALIFE.2013.6602430
26. Lizier, J. T., and Prokopenko, M. (2010). Differentiating information transfer and causal effect. Eur. Phys. J. B 73, 605-615. doi: 10.1140/epjb/e2010-00034-5
27. MacKay, D. J. (2003). Information Theory, Inference and Learning Algorithms. Cambridge: Cambridge University Press.
28. Mitchell, M. (1998). "Computation in cellular automata: a selected review, " in Non-Standard Computation, eds T. Gramß, S. Bornholdt, M. Groß, M. Mitchell, and T. Pellizzari (Weinheim: Wiley-VCH Verlag GmbH & Co. KGaA), 95-140.
29. Mitchell, M., Hraber, P., and Crutchfield, J. P. (1993). Revisiting the edge of chaos: evolving cellular automata to perform computations. Complex Systems 7, 89-130.
30. Obst, O., Boedecker, J., Schmidt, B., and Asada, M. (2013). On active information storage in input-driven systems. arXiv: 1303.5526.
31. Pincus, S. M. (1991). Approximate entropy as a measure of system complexity. Proc. Natl. Acad. Sci. U.S.A. 88, 2297-2301. doi: 10.1073/pnas.88.6.2297
32. Porta, A., Baselli, G., Liberati, D., Montano, N., Cogliati, C., Gnecchi-Ruscone, T., et al. (1998). Measuring regularity by means of a corrected conditional entropy in sympathetic outflow. Biol. Cybernet. 78, 71-78. doi: 10.1007/s004220050414
33. Porta, A., Guzzetti, S., Montano, N., Pagani, M., Somers, V., Malliani, A., et al. (2000). Information domain analysis of cardiovascular variability signals: evaluation of regularity, synchronisation and co-ordination. Med. Biol. Eng. Comput. 38, 180-188. doi: 10.1007/BF02344774
34. Priesemann, V., Munk, M. H. J., and Wibral, M. (2009). Subsampling effects in neuronal avalanche distributions recorded in vivo. BMC Neurosci. 10:40. doi: 10.1186/1471-2202-10-40
35. Priesemann, V., Valderrama, M., Wibral, M., and Le Van Quyen, M. (2013). Neuronal avalanches differ from wakefulness to deep sleep-evidence from intracranial depth recordings in humans. PLoS Comput. Biol. 9:e1002985. doi: 10.1371/journal.pcbi.1002985
36. Prokopenko, M., Boschietti, F., and Ryan, A. J. (2009). An information-theoretic primer on complexity, self-organization, and emergence. Complexity 15, 11-28. doi: 10.1002/cplx.20249
37. Prokopenko, M., Gerasimov, V., and Tanev, I. (2006). "Evolving spatiotemporal coordination in a modular robotic system, " in From Animals to Animats 9: Proceedings of the Ninth International Conference on the Simulation of Adaptive Behavior (SAB'06). Lecture notes in computer science. Vol. 4095, eds S. Nolfi, G. Baldassarre, R. Calabretta, J. C. T. Hallam, D. Marocco, J.-A. Meyer, et al. (Berlin: Springer), 558-569.
38. Ragwitz, M., and Kantz, H. (2002). Markov models from data by simple nonlinear time series predictors in delay embedding spaces. Phys. Rev. E Stat. Nonlin. Soft Matter. Phys. 65(5 Pt 2), 056201. doi: 10.1103/PhysRevE.65.056201
39. Rao, R. P., and Ballard, D. H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2, 79-87. doi: 10.1038/4580
40. Richman, J. S., and Moorman, J. R. (2000). Physiological time-series analysis using approximate entropy and sample entropy. Am. J. Physiol. Heart. Circ. Physiol. 278, H2039-H2049.
41. Schreiber (2000). Measuring information transfer. Phys. Rev. Lett. 85, 461-464. doi: 10.1103/PhysRevLett.85.461
42. Stetter, O., Battaglia, D., Soriano, J., and Geisel, T. (2012). Model-free reconstruction of excitatory neuronal connectivity from calcium imaging signals. PLoS Comput. Biol. 8:e1002653. doi: 10.1371/journal.pcbi.1002653
43. Takens, F. (1981). "Detecting strange attractors in turbulence" (chapter 21) in Dynamical Systems and Turbulence, Warwick 1980. Lecture notes in mathematics. Vol. 898, eds D. Rand and L.-S. Young (Berlin: Springer), 366-381.
44. Turing, A. M. (1936). On computable numbers, with an application to the Entscheidungsproblem. Proc. Lond. Math. Soc. 42, 230-265.
45. Turney, P. D., and Pantel, P. (2010). From frequency to meaning: vector space models of semantics. J. Artif. Intell. Res. 37, 141-188.
46. Vakorin, V. A., Mišić, B., Krakovska, O., and McIntosh, A. R. (2011). Empirical and theoretical aspects of generation and transfer of information in a neuromagnetic source network. Front. Syst. Neurosci. 5:96. doi: 10.3389/fnsys.2011.00096
47. Vicente, R., Wibral, M., Lindner, M., and Pipa, G. (2011). Transfer entropy-a model-free measure of effective connectivity for the neurosciences. J. Comput. Neurosci. 30, 45-67. doi: 10.1007/s10827-010-0262-3
48. Wang, X. R., Miller, J. M., Lizier, J. T., Prokopenko, M., and Rossi, L. F. (2012). Quantifying and tracing information cascades in swarms. PLoS ONE 7:e40084. doi: 10.1371/journal.pone.0040084
49. Wibral, M., Pampu, N., Priesemann, V., Siebenhühner, F., Seiwert, H., Lindner, M., et al. (2013). Measuring information-transfer delays. PLoS ONE 8:e55809. doi: 10.1371/journal.pone.0055809
50. Wibral, M., Rahm, B., Rieder, M., Lindner, M., Vicente, R., and Kaiser, J. (2011). Transfer entropy in magnetoencephalographic data: quantifying information flow in cortical and cerebellar networks. Prog. Biophys. Mol. Biol. 105, 80-97. doi: 10.1016/j.pbiomolbio.2010.11.006
51. Wiener, N. (1956). "The theory of prediction, " in In Modern Mathematics for the Engineer, ed E. F. Beckmann (New York, NY: McGraw-Hill).
52. Zipser, D., Kehoe, B., Littlewort, G., and Fuster, J. (1993). A spiking network model of short-term active memory. J. Neurosci. 13, 3406-3420