Mean field theory for sigmoid belief networks

Journal of Artificial Intelligence Research

Volumn 4, Issue , 1996, Pages 61-76

  Saul, Lawrence K ^a   Jaakkola, Tommi ^a   Jordan, Michael I ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; COMPUTATION THEORY; PATTERN RECOGNITION; PROBABILITY; STATISTICAL MECHANICS;

MEAN FIELD THEORY; SIGMOID BELIEF NETWORKS;

NEURAL NETWORKS;

EID: 0029679189     PISSN: 10769757     EISSN: None     Source Type: Journal    
DOI: 10.1613/jair.251     Document Type: Article

References (27)

1. Ackley, D., Hinton, G., & Sejnowski, T. (1985) A learning algorithm for Boltzmann machines. Cognitive Science, 9, 147-169.
2. Buntine, W. (1994) Operations for learning with graphical models. Journal of Artificial Intelligence Research, 2, 159-225.
3. Cooper, G. (1990) Computational complexity of probabilistic inference using Bayesian belief networks. Artificial Intelligence, 42, 393-405.
4. Cover, T., & Thomas, J. (1991) Elements of Information Theory. New York: John Wiley & Sons.
5. Dagum, P., & Luby, M. (1993) Approximately probabilistic reasoning in Bayesian belief networks is NP-hard. Artificial Intelligence, 60, 141-153.
6. Dayan, P., Hinton, G., Neal, R., & Zemel, R. (1995) The Helmholtz machine. Neural Computation, 7, 889-904.
7. Dempster, A., Laird, N., and Rubin, D. (1977) Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B39, 1-38.
8. Frey, B., Hinton, G., & Dayan, P. (1995) Does the wake-sleep algorithm learn good density estimators? In D. Touretzky, M. Mozer, and M. Hasselmo (eds). Advances of Neural Information Processing Systems: Proceedings of the 1995 Conference.
9. Geman, S., & Geman, D. (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721-741.
10. Hertz, J., Krogh, A., and Palmer, R. G. (1991) Introduction to the Theory of Neural Computation. Redwood City, CA: Addison-Wesley.
11. Hinton, G., Dayan, P., Frey, B., & Neal, R. (1995) The wake-sleep algorithm for unsupervised neural networks. Science, 268, 1158-1161.
12. Itzykson, C., & Drouffe, J.M. (1991). Statistical Field Theory. Cambridge: Cambridge University Press.
13. Jaakkola, T., Saul, L., & Jordan, M. (1995) Fast learning by bounding likelihoods in sigmoid-type belief networks. In D. Touretzky, M. Mozer, and M. Hasselmo (eds). Advances of Neural Information Processing Systems: Proceedings of the 1995 Conference.
14. Jaakkola, T., & Jordan, M. (1996a) Mixture model approximations for belief networks. Manuscript in preparation.
15. Jaakkola, T., & Jordan, M. (1996b) Computing upper and lower bounds on likelihoods in intractable networks. Submitted.
16. Jensen, C. S., Kong, A., & Kjaerulff, U. (1995) Blocking Gibbs sampling in very large probabilistic expert systems. International Journal of Human Computer Studies. Special Issue on Real-World Applications of Uncertain Reasoning.
17. Lauritzen, S., & Spiegelhalter, D. (1988). Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society B, 50, 157-224.
18. McCullagh, P., & Nelder, J. A. (1983) Generalized Linear Models. London: Chapman and Hall.
19. Neal, R. (1992) Connectionist learning of belief networks. Artificial Intelligence, 56, 71-113.
20. Neal, R., & Hinton, G. (1993) A new view of the EM algorithm that justifies incremental and other variants. Submitted for publication.
21. Parisi, G. (1988) Statistical Field Theory. Redwood City, CA: Addison-Wesley.
22. Pearl, J. (1988) Probabilistic Reasoning in Intelligent Systems. San Mateo, CA: Morgan Kaufmann.
23. Peterson, C., & Anderson, J.R. (1987) A mean field theory learning algorithm for neural networks. Complex Systems, 1, 995-1019.
24. Press, W. H., Flannery, B. P., Teukolsky, S.A., & Vetterling, W. T. (1986) Numerical Recipes. Cambrige: Cambridge University Press.
25. Russell, S., Binder, J., Koller, D., & Kanazawa, K. (1995). Local learning in probabilistic networks with hidden variables. In Proceedings of IJCAI-95.
26. Saul, L., & Jordan, M. (1995) Exploiting tractable substructures in intractable networks. In D. Touretzky, M. Mozer, and M. Hasselmo (eds). Advances of Neural Information Processing Systems: Proceedings of the 1995 Conference.
27. Seung, H. (1995). Annealed theories of learning. In J.-H. Oh, C. Kwon, and S. Cho, eds. Neural Networks: The Statistical Mechanics Perspective, Proceedings of the CTP-PRSRI Joint Workshop on Theoretical Physics. Singapore, World Scientific.