Approximate learning algorithm in Boltzmann machines

Neural Computation

Volumn 21, Issue 11, 2009, Pages 3130-3178

  Yasuda, Muneki ^a   Tanaka, Kazuyuki ^a  

^a TOHOKU UNIVERSITY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; ENERGY TRANSFER; PROBABILITY; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; ENERGY TRANSFER; MARKOV CHAINS; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER);

EID: 77953737086     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2009.08-08-844     Document Type: Article

References (22)

1. Ackley, D.H., Hinton, G.E., & Sejnowski, T.J. (1985). A learning algorithm for Boltzmann machines. Cognitive Science, 9, 147-169.
2. Hinton, G.E. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14, 1771-1800.
3. Hinton, G.E., & Sejnowski, T.J. (1983). Optimal perceptual learning. IEEE Conference on Computer Vision and Pattern Recognition (pp. 448-453). Piscataway, NJ: IEEE Press.
4. Horiguchi, T. (1981). On the Bethe approximation for the random bond Ising model. Physica A, 107, 360-370.
5. Jaakkola, T.S., Wainwright, M.J., & Willsky, A.S. (2003). Tree-reweighted belief propagation algorithms and approximate ML estimation via pseudo-moment matching. In Proceedings of the 9th International Conference on Artificial Intelligence and Statistics.N.p.
6. Kappen, H.J., & Rodríguez, F. B. (1998). Efficient learning in Boltzmann machines using linear response theory. Neural Comput., 10, 1137-1156.
7. Morita, T., & Horiguchi, T. (1976). Exactly solvable model of a spin glass. Solid State Comm., 19, 833-835.
8. Opper, M., & Saad, D. (Eds.). (2001). Advanced mean field methods-theory and practice. Cambridge, MA: MIT Press.
9. Parise, S., & Welling, M. (2005). Learning in Markov random fields: An empirical study. In Proc. of the Joint Statistical Meeting.N.p.
10. Pelizzola, A. (2005). Cluster variation method in statistical physics and probabilistic graphical models. J. Phys. A: Math. and Gen., 38, R309-R339.
11. Peterson, C., & Anderson, J.R. (1987). A mean field theory learning algorithm for neural networks. Complex Systems, 1, 995-1019.
12. Plefka, T. (1982). Convergence condition of the tap equation for the infinite-ranged Ising spin glass model. J. Phys. A: Math. and Gen., 15, 1971-1978.
13. Tanaka, K. (2003). Probabilistic inference by mean of cluster variation method and linear response theory. IEICE Trans. on Information and Systems, E86-D, 1228-1242.
14. Tanaka, T. (1998). Mean-field theory of Boltzmann machine learning. Phys. Rev. E, 58, 2302-2310.
15. Thouless, D.J., Anderson, P.W., & Palmer, R.G. (1977). Solution of "solvable model of a spin glass" Philosophical Magazine, 35, 593-601.
16. Wainwright, M.J. (2006). Estimating the "wrong" graphical model: Benefits in the computation-limited setting. Journal of Machine Learning Research, 7, 1829-1859.
17. Welling, M., & Teh, Y.W. (2003). Approximation inference in Boltzmann machines. Artificial Intelligence, 143, 19-50.
18. Welling, M., & Teh, Y.W. (2004). Linear response algorithms for approximate inference in graphical models. Neural Comput., 16, 197-221.
19. Yasuda, M., & Horiguchi, T. (2006). Triangular approximation for Ising model and its application to Boltzmann machine. Physica A, 836, 83-95.
20. Yasuda, M., & Tanaka, K. (2006). The relationship between Plefka's expansion and the cluster variation method. J. Phys. Soc. Jpn., 75, 1-8.
21. Yasuda, M., & Tanaka, K. (2007). The mathematical structure of the approximate linear response relation. J. Phys. A: Math. and Theor., 40, 9993-10007.
22. Yedidia, J.S., Freeman, W.T., & Weiss, Y. (2005). Constructing free-energy approximations and generalized belief propagation algorithms. IEEE Transactions on Information Theory, 51, 2282-2312.