Information geometry of mean-field approximation

Neural Computation

Volumn 12, Issue 8, 2000, Pages 1951-1968

  Tanaka, Toshiyuki ^a  

^a TOKYO METROPOLITAN UNIVERSITY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; ARTIFICIAL INTELLIGENCE; INFORMATION SCIENCE; PHYSICS; STATISTICAL MODEL;

ARTIFICIAL INTELLIGENCE; INFORMATION THEORY; LINEAR MODELS; MODELS, STATISTICAL; PHYSICS;

EID: 0034241901     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976600300015213     Document Type: Article

References (27)

1. Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann machines. Cognitive Science, 9, 147-169.
2. Amari, S. (1985). Differential-geometrical method in statistics. Berlin: Springer-Verlag.
3. Amari, S., & Nagaoka, H. (1998). Introduction to information geometry. (D. Harada, trans.). New York: AMS and Oxford University Press.
4. de Almeida, J. R. L., & Thouless, D. J. (1978). Stability of the Sherrington-Kirkpatrick solution of a spin glass model. J. Phys. A: Math. Gen., 11(5), 983-990.
5. Dunmur, A. P., & Titterington, D. M. (1997). On a modification to the mean field EM algorithm in factorial learning. In M. C. Mozer, M. I. Jordan, & T. Petsche (Eds.), Advances in neural information processing systems, 9 (pp. 431-437). Cambridge, MA: MIT Press.
6. Fischer, K. H., & Hertz, J. A. (1991). Spin glasses. Cambridge: Cambridge University Press.
7. Galland, C. C. (1993). The limitations of deterministic Boltzmann machine learning. Network, 4(3), 355-379.
8. Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Patt. Anal. Machine Intell., PAMI-6(6), 721-741.
9. Georges, A., & Yedida, J. S. (1991). How to expand around mean-field theory using high-temperature expansions. J. Phys. A: Math. Gen., 24(9), 2173-2192.
10. Haykin, S. (1994). Neural network - A comprehensive foundation. New York: Macmillan.
11. Hofmann, T., & Buhmann, J. M. (1997). Pairwise data clustering by deterministic annealing. IEEE Trans. Patt. Anal. Machine Intell., 19(1), 1-14. Errata, ibid., 19(2), 197, 1997.
12. Kappen, H. J., & Rodríguez, F. B. (1998a). Efficient learning in Boltzmann machines using linear response theory. Neural Computation, 10(5), 1137-1156.
13. Kappen, H. J., & Rodríguez, F. B. (1998b). Boltzmann machine learning using mean field theory and linear response correction. In M. I. Jordan, M. J. Kearns, & S. A. Solla (Eds.), Advances in neural information processing systems, 10, (pp. 280-286). Cambridge, MA: MIT Press.
14. Lee, T. D., & Yang, C. N. (1952). Statistical theory of equations of state and phase transitions II lattice gas and Ising model. Phys. Rev., 87(3), 410-419.
15. Leisink, M. A. R., & Kappen, H. J. (1998). Learning higher order Boltzmann machines using linear response. In L. Niklasson, M. Boden, & T. Ziemke (Eds.), Proc. 8th Int. Conf. Artificial Neural Networks (pp. 511-516). Berlin: Springer-Verlag.
16. Nakanishi, K., & Takayama, H. (1997). Mean-field theory for a spin-glass model of neural networks: TAP free energy and the paramagnetic to spin-glass transition. J. Phys. A: Math. Gen., 30(23), 8085-8094.
17. Parisi, G. (1988). Statistical field theory. Reading, MA: Addison-Wesley.
18. Parisi, G., & Potters, M. (1995). Mean-field equations for spin models with orthogonal interaction matrices. J. Phys. A: Math. Gen., 28(18), 5267-5285.
19. Peterson, C., & Anderson, J. R. (1987). A mean field theory learning algorithm for neural networks. Complex Systems, 1, 995-1019.
20. Plefka, T. (1982). Convergence condition of the TAP equation for the infinite-ranged Ising spin glass model. J. Phys. A: Math. Gen., 15(6), 1971-1978.
21. Saul, L. K., & Jordan, M. I. (1996). Exploiting tractable structures in intractable networks. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in neural information processing systems, 8, (pp. 486-492). Cambridge, MA: MIT Press.
22. Sherrington, D., & Kirkpatrick, S. (1975). Solvable odel of a spin-glass. Phys. Rev. Lett., 35(26), 1792-1796.
23. Tanaka, T. (1996). Information geometry of mean field theory. IEICE Trans. Fundamentals, E79-A(5), 709-715.
24. Tanaka, T. (1998a). Mean field theory of Boltzmann machine learning. Phys. Rev. E, 58(2), 2302-2310.
25. Tanaka, T. (1998b). Estimation of third-order correlations within mean field approximation. In S. Usui & T. Omori (Eds.), Proc. Fifth Int. Conf. Neural Info. Process. '98 Kitakyushu (Vol. 1, pp. 554-557).
26. Thouless, D. J., Anderson, P. W., & Palmer, R. G. (1977). Solution of "Solvable model of a spin glass." Phil. Mag., 35(3), 593-601.
27. Yedida, J. S., & Georges, A. (1990). The fully frustrated Ising model in infinite dimensions. J. Phys. A: Math. Gen., 23(11), 2165-2171.