On thermodynamic interpretation of transfer entropy

Entropy

Volumn 15, Issue 2, 2013, Pages 524-543

  Prokopenko, Mikhail ^a,b   Lizier, Joseph T ^a   Price, Don C ^a  

^a CSIRO   (Australia)

^b UNIVERSITY OF SYDNEY   (Australia)

Author keywords

Boltzmann's principle; Causal effect; Entropy production; Information transfer; Irreversibility; Kullback Leibler divergence; Thermodynamic equilibrium; Transfer entropy

Indexed keywords

EID: 84874769692     PISSN: None     EISSN: 10994300     Source Type: Journal    
DOI: 10.3390/e15020524     Document Type: Article

References (68)

1. Schreiber, T. Measuring information transfer. Phys. Rev. Lett. 2000, 85, 461-464.
2. Jakubowski, M.H.; Steiglitz, K.; Squier, R. Information transfer between solitary waves in the saturable Schr̈odinger equation. Phys. Rev. E 1997, 56, 7267.
3. Baek, S.K.; Jung, W.S.; Kwon, O.; Moon, H.T. Transfer Entropy Analysis of the Stock Market, 2005, arXiv:physics/0509014v2.
4. Moniz, L.J.; Cooch, E.G.; Ellner, S.P.; Nichols, J.D.; Nichols, J.M. Application of information theory methods to food web reconstruction. Ecol. Model. 2007, 208, 145-158.
5. Ch́avez, M.; Martinerie, J.; Le Van Quyen, M. Statistical assessment of nonlinear causality: Application to epileptic EEG signals. J. Neurosci. Methods 2003, 124, 113-128.
6. Pahle, J.; Green, A.K.; Dixon, C.J.; Kummer, U. Information transfer in signaling pathways: A study using coupled simulated and experimental data. BMC Bioinforma. 2008, 9, 139.
7. Lizier, J.T.; Prokopenko, M.; Zomaya, A.Y. Local information transfer as a spatiotemporal filter for complex systems. Phys. Rev. E 2008, 77, 026110.
8. Lizier, J.T.; Prokopenko, M.; Zomaya, A.Y. Information modification and particle collisions in distributed computation. Chaos 2010, 20, 037109.
9. Lizier, J.T.; Prokopenko, M.; Zomaya, A.Y. Coherent information structure in complex computation. Theory Biosci. 2012, 131, 193-203.
10. Lizier, J.T.; Prokopenko, M.; Zomaya, A.Y. Local measures of information storage in complex distributed computation. Inf. Sci. 2012, 208, 39-54.
11. Lizier, J.T.; Prokopenko, M.; Tanev, I.; Zomaya, A.Y. Emergence of Glider-like Structures in a Modular Robotic System. In Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems (ALife XI), Winchester, UK, 5-8 August 2008
12. Bullock, S., Noble, J., Watson, R., Bedau, M.A., Eds.; MIT Press: Cambridge, MA, USA, 2008; pp. 366-373.
13. Lizier, J.T.; Prokopenko, M.; Zomaya, A.Y. The Information Dynamics of Phase Transitions in Random Boolean Networks. In Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems (ALife XI), Winchester, UK, 5-8 August 2008
14. Bullock, S., Noble, J., Watson, R., Bedau, M.A., Eds.; MIT Press: Cambridge, MA, USA, 2008; pp. 374-381.
15. Lizier, J.T.; Pritam, S.; Prokopenko, M. Information dynamics in small-world Boolean networks. Artif. Life 2011, 17, 293-314.
16. Lizier, J.T.; Heinzle, J.; Horstmann, A.; Haynes, J.D.; Prokopenko, M. Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity. J. Comput. Neurosci. 2011, 30, 85-107.
17. Wang, X.R.; Miller, J.M.; Lizier, J.T.; Prokopenko, M.; Rossi, L.F. Quantifying and tracing information cascades in swarms. PLoS One 2012, 7, e40084.
18. Lizier, J.T.; Prokopenko, M.; Cornforth, D.J. The Information Dynamics of Cascading Failures in Energy Networks. In Proceedings of the European Conference on Complex Systems (ECCS), Warwick, UK, 21-25 October 2009; p. 54. ISBN: 978-0-9554123-1-8.
19. Kaiser, A.; Schreiber, T. Information transfer in continuous processes. Physica D 2002, 166, 43-62.
20. Ay, N.; Polani, D. Information flows in causal networks. Adv. Complex Syst. 2008, 11, 17-41.
21. Bennett, C.H. Notes on Landauer's principle, reversible computation, and Maxwell's Demon. Stud. History Philos. Sci. Part B 2003, 34, 501-510.
22. Piechocinska, B. Information erasure. Phys. Rev. A 2000, 61, 062314.
23. Lloyd, S. Programming the Universe; Vintage Books: New York, NY, USA, 2006.
24. Parrondo, J.M.R.; den Broeck, C.V.; Kawai, R. Entropy production and the arrow of time. New J. Phys. 2009, 11, 073008.
25. Prokopenko, M.; Lizier, J.T.; Obst, O.;Wang, X.R. Relating Fisher information to order parameters. Phys. Rev. E 2011, 84, 041116.
26. Landauer, R. Irreversibility and heat generation in the computing process. IBM J. Res. Dev. 1961, 5, 183-191.
27. Maroney, O.J.E. Generalizing Landauer's principle. Phys. Rev. E 2009, 79, 031105.
28. Jaynes, E.T. Information theory and statistical mechanics. Phys. Rev. 1957, 106, 620-630.
29. Jaynes, E.T. Information theory and statistical mechanics. II. Phys. Rev. 1957, 108, 171-190.
30. Crooks, G. Measuring thermodynamic length. Phys. Rev. Lett. 2007, 99, 100602+.
31. Liang, X.S. Information flow within stochastic dynamical systems. Phys. Rev. E 2008, 78, 031113.
32. L̈udtke, N.; Panzeri, S.; Brown, M.; Broomhead, D.S.; Knowles, J.; Montemurro, M.A.; Kell, D.B. Information-theoretic sensitivity analysis: A general method for credit assignment in complex networks. J. R. Soc. Interface 2008, 5, 223-235.
33. Auletta, G.; Ellis, G.F.R.; Jaeger, L. Top-down causation by information control: From a philosophical problem to a scientific research programme. J. R. Soc. Interface 2008, 5, 1159-1172.
34. Hlav́ackov́a-Schindler, K.; Palus, M.; Vejmelka, M.; Bhattacharya, J. Causality detection based on information-theoretic approaches in time series analysis. Phys. Rep. 2007, 441, 1-46.
35. Lungarella, M.; Ishiguro, K.; Kuniyoshi, Y.; Otsu, N. Methods for quantifying the causal structure of bivariate time series. Int. J. Bifurc. Chaos 2007, 17, 903-921.
36. Ishiguro, K.; Otsu, N.; Lungarella, M.; Kuniyoshi, Y. Detecting direction of causal interactions between dynamically coupled signals. Phys. Rev. E 2008, 77, 026216.
37. Sumioka, H.; Yoshikawa, Y.; Asada, M. Causality Detected by Transfer Entropy Leads Acquisition of Joint Attention. In Proceedings of the 6th IEEE International Conference on Development and Learning (ICDL 2007), London, UK, 11-13 July 2007; pp. 264-269.
38. Vejmelka, M.; Palus, M. Inferring the directionality of coupling with conditional mutual information. Phys. Rev. E 2008, 77, 026214.
39. Verdes, P.F. Assessing causality from multivariate time series. Phys. Rev. E 2005, 72, 026222:1- 026222:9.
40. Tung, T.Q.; Ryu, T.; Lee, K.H.; Lee, D. Inferring Gene Regulatory Networks from Microarray Time Series Data Using Transfer Entropy. In Proceedings of the Twentieth IEEE International Symposium on Computer-Based Medical Systems (CBMS '07), Maribor, Slovenia, 20-22 June 2007
41. Kokol, P., Podgorelec, V., Mǐcetǐc-Turk, D., Zorman, M., Verlǐc, M., Eds.; IEEE: Los Alamitos, CA, USA, 2007; pp. 383-388.
42. Van Dijck, G.; van Vaerenbergh, J.; van Hulle, M.M. Information Theoretic Derivations for Causality Detection: Application to Human Gait. In Proceedings of the International Conference on Artificial Neural Networks (ICANN 2007), Porto, Portugal, 9-13 September 2007; Śa, J.M.d., Alexandre, L.A., Duch,W., Mandic, D., Eds.; Springer-Verlag: Berlin/Heidelberg, Germany, 2007; Volume 4669, Lecture Notes in Computer Science, pp. 159-168.
43. Hung, Y.C.; Hu, C.K. Chaotic communication via temporal transfer entropy. Phys. Rev. Lett. 2008, 101, 244102.
44. Lizier, J.T.; Prokopenko, M. Differentiating information transfer and causal effect. Eur. Phys. J. B 2010, 73, 605-615.
45. Wuensche, A. Classifying cellular automata automatically: Finding gliders, filtering, and relating space-time patterns, attractor basins, and the Z parameter. Complexity 1999, 4, 47-66.
46. Soĺe, R.V.; Valverde, S. Information transfer and phase transitions in a model of internet traffic. Physica A 2001, 289, 595-605.
47. Soĺe, R.V.; Valverde, S. Information Theory of Complex Networks: On Evolution and Architectural Constraints. In Complex Networks; Ben-Naim, E., Frauenfelder, H., Toroczkai, Z., Eds.; Springer: Berlin/Heidelberg, Germany, 2004; Volume 650, Lecture Notes in Physics, pp. 189-207.
48. Prokopenko, M.; Boschietti, F.; Ryan, A.J. An information-theoretic primer on complexity, self-organization, and emergence. Complexity 2009, 15, 11-28.
49. Takens, F. Detecting Strange Attractors in Turbulence. In Dynamical Systems and Turbulence, Warwick 1980; Rand, D., Young, L.S., Eds.; Springer: Berlin/Heidelberg, Germany, 1981; pp. 366-381.
50. Granger, C.W.J. Investigating causal relations by econometric models and cross-spectral methods. Econometrica 1969, 37, 424-438.
51. Fano, R. Transmission of Information: A Statistical Theory of Communications; The MIT Press: Cambridge, MA, USA, 1961.
52. Manning, C.D.; Scḧutze, H. Foundations of Statistical Natural Language Processing; The MIT Press: Cambridge, MA, USA, 1999.
53. Dasan, J.; Ramamohan, T.R.; Singh, A.; Nott, P.R. Stress fluctuations in sheared Stokesian suspensions. Phys. Rev. E 2002, 66, 021409.
54. MacKay, D.J. Information Theory, Inference, and Learning Algorithms; Cambridge University Press: Cambridge, MA, USA, 2003.
55. Pearl, J. Causality: Models, Reasoning, and Inference; Cambridge University Press: Cambridge, MA, USA, 2000.
56. Goyal, P. Information physics-towards a new conception of physical reality. Information 2012, 3, 567-594.
57. Sethna, J.P. Statistical Mechanics: Entropy, Order Parameters, and Complexity; Oxford University Press: Oxford, UK, 2006.
58. Seife, C. Decoding the Universe; Penguin Group: New York, NY, USA, 2006.
59. Einstein, A. Ü ber einen die Erzeugung und Verwandlung des Lichtes betreffenden heuristischen Gesichtspunkt. Ann. Phys. 1905, 322, 132-148.
60. Norton, J.D. Atoms, entropy, quanta: Einstein's miraculous argument of 1905. Stud. History Philos. Mod. Phys. 2006, 37, 71-100.
61. Barnett, L.; Buckley, C.L.; Bullock, S. Neural complexity and structural connectivity. Phys. Rev. E 2009, 79, 051914.
62. Ay, N.; Bernigau, H.; Der, R.; Prokopenko, M. Information-driven self-organization: The dynamical system approach to autonomous robot behavior. Theory Biosci. 2012, 131, 161-179.
63. Evans, D.J.; Cohen, E.G.D.; Morriss, G.P. Probability of second law violations in shearing steady states. Phys. Rev. Lett. 1993, 71, 2401-2404.
64. Searles, D.J.; Evans, D.J. Fluctuations relations for nonequilibrium systems. Aus. J. Chem. 2004, 57, 1129-1123.
65. Crooks, G.E. Entropy production fluctuation theorem and the nonequilibrium work relation for free energy differences. Phys. Rev. E 1999, 60, 2721-2726.
66. Jarzynski, C. Nonequilibrium work relations: Foundations and applications. Eur. Phys. J. B-Condens. Matter Complex Syst. 2008, 64, 331-340.
67. Schl̈ogl, F. Information Measures and Thermodynamic Criteria for Motion. In Structural Stability in Physics: Proceedings of Two International Symposia on Applications of Catastrophe Theory and Topological Concepts in Physics, T̈ubingen, Germany, 2-6 May and 11-14 December 1978
68. G̈uttinger, W., Eikemeier, H., Eds.; Springer: Berlin/Heidelberg, Germany, 1979; Volume 4, Springer series in synergetics, pp. 199-209.