Learning in Markov random fields with contrastive free energies

AISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics

Volumn , Issue , 2005, Pages 397-404

  Welling, Max ^a   Sutton, Charles ^b  

^a Ringgold Standard Institution Informatics   (United States)

^b Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONDITIONAL RANDOM FIELD; CONTRASTIVE DIVERGENCE; DATA DISTRIBUTION; EQUILIBRIUM DISTRIBUTIONS; GLOBAL NORMALIZATION; LONG RANGE INTERACTIONS; MARKOV RANDOM FIELD MODELS; MARKOV RANDOM FIELDS; MEAN FIELD; OBJECTIVE FUNCTIONS; PSEUDO-LIKELIHOOD; TEXT DATA; TRAINING TIME;

ARTIFICIAL INTELLIGENCE; FREE ENERGY; LEARNING ALGORITHMS; MAXIMUM LIKELIHOOD;

IMAGE SEGMENTATION;

EID: 78651077964     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. J. Besag. Efficiency of pseudo-likelihood estimation for simple Gaussian fields. Biometrika, 64:616-618, 1977.
2. Dayne Freitag. Machine Learning for Information Extraction in Informal Domains. PhD thesis, Carnegie Mellon University, 1998.
3. B. Gidas. Consistency of maximum likelihood and pseudo-likelihood estimators for Gibbs distributions. In W. Fleming and eds. P.L. Lions, editors, Stochastic Differential Systems, Stochastic Control Theory and Applications. New York: Springer, 1988.
4. T. Heskes. Stable fixed points of loopy belief propagation are minima of the Bethe free energy. In Advances in Neural Information Processing Systems, volume 15, Vancouver, CA, 2003.
5. G.E. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14:1771-1800, 2002.
6. G.E. Hinton, K. Osindero, M. Welling, and Y.W. Teh. Unsupervised discovery of non-linear structure using contrastive backpropagation, 2004. in prepration.
7. G.E. Hinton, M. Welling, and A. Mnih. Wormholes improve contrastive divergence. In Advances in Neural Information Processing Systems, volume 16, 2004.
8. John Lafferty, Andrew McCallum, and Fernando Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proc. 18th International Conf. on Machine Learning, pages 282-289. Morgan Kaufmann, San Francisco, CA, 2001.
9. I. Murray and Z. Ghahramani. Bayesian learning in undirected graphical models: approximate MCMC algorithms. In Proceedings of the 14th Annual Conference on Uncertainty in Artificial Intelligence (UAI-04), San Francisco, CA, 2004. Morgan Kaufmann Publishers.
10. C. Peterson and J. Anderson. A mean field theory learning algorithm for neural networks. Complex Systems, 1:995-1019, 1987.
11. Charles Sutton and Andrew McCallum. Collective segmentation and labeling of distant entities in information extraction. Technical Report TR # 04-49, University of Massachusetts, July 2004. Presented at ICML Workshop on Statistical Relational Learning and Its Connections to Other Fields.
12. Charles Sutton, Khashayar Rohanimanesh, and Andrew McCallum. Dynamic conditional random fields: Factorized probabilistic models for labeling and segmenting sequence data. In Proceedings of the Twenty-First International Conference on Machine Learning (ICML-2004), 2004.
13. Ben Taskar, Pieter Abbeel, and Daphne Koller. Discriminative probabilistic models for relational data. In Eighteenth Conference on Uncertainty in Artificial Intelligence (UAI02), 2002.
14. Y.W. Teh and M. Welling. The unified propagation and scaling algorithm. In Advances in Neural Information Processing Systems, 2002.
15. Y.W. Teh, M. Welling, S. Osindero, and G.E. Hinton. Energy-based models for sparse overcomplete representations. Journal of Machine Learning Research - Special Issue on ICA, 4:1235-1260, 2003.
16. M.J. Wainwright, T. Jaakkola, and A.S. Willsky. Tree-based reparameterization for approximate estimation on loopy graphs. In Advances Neural Information Processing Systems, volume 14, vancouver, Canada, 2001.
17. M. Welling and G.E. Hinton. A new learning algorithm for mean field Boltzmann machines. In Proceedings of the International Conference on Artificial Neural Networks, Madrid, Spain, 2001.
18. M. Welling and Y.W. Teh. Belief optimization for binary networks: a stable alternative to loopy belief propagation. In Proceedings of the Conference on Uncertainty in Artificial Intelligence, pages 554-561, Seattle, USA, 2001.
19. J.S. Yedidia, W. Freeman, and Y. Weiss. Constructing free energy approximations and generalized belief propagation algorithms. Technical report, MERL, 2002. Technical Report TR-2002-35.
20. A.L. Yuille. CCCP algorithms to minimize the Bethe and Kikuchi free energies: Convergent alternatives to belief propagation. Neural Computation, 14(7):1691-1722, 2002.
21. A.L. Yuille. A comment on contrastive divergence. Technical report, Department Statistics and Psychology UCLA, 2004. Technical Report.