Multi-information in the thermodynamic limit

Journal of Statistical Physics

Volumn 115, Issue 3-4, 2004, Pages 949-976

  Erb, Ionas ^a,b   Ay, Nihat ^b,c,d  

^a UNIVERSITY OF LEIPZIG   (Germany)

^b MAX PLANCK INSTITUTE FOR MATHEMATICS IN THE SCIENCES   (Germany)

^c SANTA FE INSTITUTE   (United States)

^d UNIVERSITY OF ERLANGEN NUREMBERG   (Germany)

Author keywords

Complexity; Excess entropy; Ising model; Mutual information; Phase transitions

Indexed keywords

EID: 3543067020     PISSN: 00224715     EISSN: None     Source Type: Journal    
DOI: 10.1023/b:joss.0000022375.49904.ea     Document Type: Article

References (43)

1. S. Amari, Information geometry on hierarchy of probability distributions, IEEE Trans. Inform. Theory 47:1701-1711 (2001).
2. S. Amari and H. Nagaoka, in Methods of information geometry, AMS Translations of Mathematical Monographs, Vol. 191 (Oxford University Press, Oxford, 2000).
3. D. V. Arnold, Information-theoretic analysis of phase transitions, Complex Systems 10:143-155 (1996).
4. N. Ay, An information geometric approach to a theory of pragmatic structuring, Ann. Probab. 30:416-436 (2002).
5. N. Ay, Locality of global stochastic interaction in directed acyclic networks, Neural Comput. 14:2959-2980 (2002).
6. N. Ay and A. Knauf, Maximizing Multi-Information, Max-Planck-Institute of Mathematics in the Sciences, Preprint No. 42/2003 (2003).
7. H. Bauer, Probability Theory (de Gruyter, Berlin/New York, 1996).
8. A. J. Bell, The Co-Information Lattice, preprint Redwood Neuroscience Institute RNI-TR-02-1 (2002).
9. W. Bialek, I. Nemenman, and N. Tishby, Predictability, complexity, and learning, Neural Comput. 13:2409-2463 (2001).
10. P. M. Binder and J. Plazas, Multiscale analysis of complex systems, Phys. Rev. E 63:065203R (2001).
11. I. N. Bronstein and K. A. Semendjajew, Taschenbuch der Mathematik (Teubner, Leipzig, 1989).
12. D. E. Catlin, Estimation, Control, and the Discrete Kalman Filter (Springer-Verlag, New York, 1989).
13. G. Choquet, Lectures on Analysis, Vols. I-III (Benjamin, London, 1969).
14. T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley, New York, 1991).
15. J. P. Crutchfield and K. Young, Computation at the onset of chaos, in Complexity, Entropy, and the Physics of Information, W. H. Zurek, ed. (Addison-Wesley, Reading, 1990).
16. J. P. Crutchfield and D. P. Feldman, Statistical complexity of simple one-dimensional spin systems, Phys. Rev. E 55:R1239-R1242 (1997).
17. J. P. Crutchfield and D. P. Feldman, Regularities unseen, randomness observed: Levels of entropy convergence, Chaos 13:25-54 (2003).
18. R. L. Dobrushin, The description of a random field by means of conditional probabilities and condition of its regularities, Theory Probab. Appl. 13:458-486 (1968).
19. W. Ebeling, Prediction and entropy of nonlinear dynamical systems and symbolic sequences with LRO, Physica D 109:42-52 (1997).
20. D. P. Feldman and J. P. Crutchfield, Discovering noncritical organization: Statistical mechanical, information theoretic, and computational views of patterns in one-dimensional spin systems, Santa Fe Institute Working Paper 98-04-026.
21. D. P. Feldman and J. P. Crutchfield, Structural information in two-dimensional patterns: Entropy convergence and excess entropy, Phys. Rev. E 67:051104 (2003).
22. H.-O. Georgii, Gibbs Measures and Phase Transitions (de Gruyter, Berlin/New York, 1988).
23. P. Grassberger, Toward a quantitative theory of self-generated complexity, Internat. J. Theoret. Phys. 25:907-938 (1986).
24. E. Greenfield and H. Lecar, Mutual information in a dilute, asymmetric neural network model, Phys. Rev. E 63:041905 (2001).
25. D. Griffeath, Introduction to random fields, in Denumerable Markov Chains, J. G. Kemeney, J. L. Snell, and A. W. Knapp, eds. (Springer, New York/Heidelberg/Berlin, 1976).
26. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis (Wiley, 2001).
27. R. B. Israel, Convexity in the Theory of Lattice Gases (Princeton University Press, Princeton, 1979).
28. E. T. Jaynes, Information theory and statistical mechanics, Phys. Rev. 106:620-630 (1957).
29. W. Li, Mutual information versus correlation functions, J. Stat. Phys. 60:823-837 (1990).
30. W. Li, On the relationship between complexity and entropy for Markov chains and regular languages, Complex Systems 5:381-399 (1991).
31. R. Linsker, Self-organization in a perceptual network, IEEE Comput. 21:105-117 (1988).
32. O. E. Lanford and D. Ruelle, Observables at infinity and states with short range correlations in statistical mechanics, Comm. Math. Phys. 13:194-215 (1969).
33. B. Luque and A. Ferrera, Measuring mutual information in random Boolean networks, Complex Systems 12:241-246 (2001).
34. H. Matsuda, K. Kudo, R. Nakamura, O. Yamakawa, and T. Murata, Mutual information of Ising systems, Internat. J. Theoret. Phys. 35:839-845 (1996).
35. B. M. McCoy and T. T. Wu, The Two-Dimensional Ising Model (Harvard University Press, Cambridge, Massachusetts, 1973).
36. R. K. Pathria, Statistical Mechanics (Butterworth-Heinemann, Oxford, 1996).
37. D. Petritis, Thermodynamic formalism of neural computing, in Dynamics of Complex Interacting Systems, E. Goles and M. Servet, eds. (Kluwer Academic Publishers, Dordrecht/Boston/London, 1996).
38. D. Ruelle, Statistical Mechanics - Rigorous Results (World Scientific, Singapore, 1999).
39. C. E. Shannon, A mathematical theory of communication, Bell System Tech. J. 27:379-423, 623-656 (1948).
40. B. Simon, The Statistical Mechanics of Lattice Gases (Princeton University Press, Princeton, 1993).
41. M. Studeny and J. Vejnarova, The multiinformation function as a tool for measuring stochastic dependence, in Learning in Graphical models, M. I. Jordan, ed. (Kluwer, Dordrecht, 1998).
42. G. Tononi, O. Sporns, and G. Edelman, A measure for brain complexity: Relating functional segregation and integration in the nervous system, Proc. Natl. Acad. Sci. USA 91:5033-5037 (1994).
43. G. H. Wannier, Statistical Physics (Dover, New York, 1987).