Asymptotic efficiency of deterministic estimators for discrete energy-based models: Ratio matching and pseudolikelihood

Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence, UAI 2011

Volumn , Issue , 2011, Pages 497-505

  Marlin, Benjamin M ^a   De Freitas, Nando ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPUTATION THEORY; COVARIANCE MATRIX;

ASYMPTOTIC COVARIANCE MATRIX; ASYMPTOTIC EFFICIENCY; ASYMPTOTIC THEORIES; GENERIC EXPRESSION; MODEL PROBABILITIES; PSEUDO-LIKELIHOOD; RECENT RESEARCHES; STATISTICAL EFFICIENCY;

MAXIMUM LIKELIHOOD ESTIMATION;

EID: 80053168693     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. D.H. Ackley, G.E. Hinton, and T.J. Sejnowski. A learning algorithm for Boltzmann machines. Cognitive science, 9(1):147-169, 1985.
2. J. Besag. Statistical analysis of non-lattice data. The Statistician, 24(3):179-195, 1975.
3. R. A. Fisher. On the mathematical foundations of theoretical statistics. Philosophical Transactions of the Royal Society of London Series A., 222:309-368, 1922.
4. Aapo Hyvärinen. Some extensions of score matching. Computational Statistics & Data Analysis, 51 (5):2499-2512, 2007. (Pubitemid 44751259)
5. E. Ising. Beitrag zur theorie des ferromagnetismus. Zeitschrift für Physik, 31(1):253-258, 1925.
6. R. Kindermann, J.L. Snell, and American Mathematical Society. Markov random fields and their applications. American Mathematical Society, 1980.
7. J.D. Lafferty, A. McCallum, and F.C.N. Pereira. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proceedings of the 18th International Conference on Machine Learning, pages 282-289, 2001.
8. P. Liang and M.I. Jordan. An asymptotic analysis of generative, discriminative, and pseudolikelihood estimators. In Proceedings of the 25th International Conference on Machine learning, pages 584-591, 2008.
9. B.G. Lindsay. Composite likelihood methods. Contemporary Mathematics, 80(1):221-239, 1988.
10. S. Lyu. Interpretation and generalization of score matching. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, pages 359- 366, 2009.
11. B.M. Marlin, K. Swersky, B. Chen, and N. de Freitas. Inductive principles for restricted Boltzmann machine learning. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, pages 509-516, 2009.
12. J. Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, 1988.
13. N.A. Smith and J. Eisner. Contrastive estimation: Training log-linear models on unlabeled data. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pages 354-362, 2005.
14. P. Smolensky. Information processing in dynamical systems: Foundations of harmony theory. Parallel distributed processing: explorations in the microstructure of cognition, Vol. 1, pages 194-281, 1986.
15. A.W. van der Vaart. Asymptotic statistics. Cambridge University Press, 2000.
16. D. Vickrey, C. Lin, and D. Koller. Non-Local Contrastive Objectives. In Proceedings of the 27th International Conference on Machine Learning, 2010.
17. M. Wainwright, T. Jaakkola, and A. Willsky. Tree-reweighted belief propagation algorithms and approximate ML estimation via pseudo-moment matching. In Workshop on Artificial Intelligence and Statistics 9, 2003.
18. M.Welling, M. Rosen-Zvi, and G. Hinton. Exponential family harmoniums with an application to information retrieval. In Advances in Neural Information Processing Systems 17, pages 1481-1488, 2005.
19. L. Younes. Parametric inference for imperfectly observed Gibbsian fields. Probability Theory and Related Fields, 82(4):625-645, 1989.