Empirical risk minimization of graphical model parameters given approximate inference, decoding, and model structure

Journal of Machine Learning Research

Volumn 15, Issue , 2011, Pages 725-733

  Stoyanov, Veselin ^a   Ropson, Alexander ^a   Eisner, Jason ^a  

^a Johns Hopkins University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATE INFERENCE; EMPIRICAL RISK MINIMIZATION; EMPIRICAL RISKS; GRAPHICAL MODEL; IMPERFECT SYSTEMS; STOCHASTIC META DESCENTS; TEST DATA;

ARTIFICIAL INTELLIGENCE; GRAPHIC METHODS;

SPEECH RECOGNITION;

EID: 84862282438     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (29)

1. L. R. Bahl, P. F. Brown, P. V. de Souza, and R. L. Mercer. A new algorithm for the estimation of hidden Markov model parameters. In Proceedings of ICASSP, pages 493-496, 1988.
2. J. Besag. Statistical analysis of non-lattice data. The Statistician, 24(3):179-195, 1975. ISSN 0039-0526.
3. J. Besag. Efficiency of pseudolikelihood estimation for simple Gaussian fields. Biometrika, 64(3):616-618, 1977. ISSN 0006-3444.
4. F. Casacuberta and C. De La Higuera. Computational complexity of problems on probabilistic grammars and transducers. In Proceedings of the 5th International Colloquium on Grammatical Inference: Algorithms and Applications, pages 15-24, 2000.
5. J. Domke. Implicit Differentiation by Perturbation. In Advances in Neural Information Processing Systems, 2010.
6. F. Eaton and Z. Ghahramani. Choosing a variable to clamp: Approximate inference using conditioned belief propagation. In Proceedings of AISTATS, 2009.
7. A. Griewank and G. Corliss, editors. Automatic Differentiation of Algorithms. SIAM, Philadelphia, 1991.
8. A. Griewank and A. Walther. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. SIAM, 2008.
9. S. Gross, O. Russakovsky, C. Do, and S. Batzoglou. Training conditional random fields for maximum labelwise accuracy. Advances in Neural Information Processing Systems, 19:529, 2007.
10. G.E. Hinton, S. Osindero, and Y.W. Teh. A fast learning algorithm for deep belief nets. Neural computation, 18(7):1527-1554, 2006. ISSN 0899-7667.
11. S. Kakade, Y.W. Teh, and S.T. Roweis. An alternate objective function for Markovian fields. In International Conference on Machine Learning, pages 275-282, 2002.
12. A. Kulesza and F. Pereira. Structured learning with approximate inference. In Advances in Neural Information Processing Systems, 2008.
13. S. Lacoste-Julien, F. Huszr, and Z. Ghahramani. Approximate inference for the loss-calibrated Bayesian. In Proceedings of AISTATS, 2011.
14. Y. LeCun, S. Chopra, R. Hadsell, M. Ranzato, and F. Huang. A tutorial on energy-based learning. In G. Bakir, T. Hofman, B. Schlkopf, A. Smola, and B. Taskar, editors, Predicting Structured Data. MIT Press, 2006.
15. S.-I. Lee, V. Ganapathi, and D. Koller. Efficient structure learning of markov networks using l1regularization. In Advances in Neural Information Processing Systems, pages 817-824, 2006.
16. Z. Li and J. Eisner. First- and second-order expectation semirings with applications to minimum-risk training on translation forests. In Proc. of EMNLP, pages 40-51, 2009.
17. T. Minka. Empirical risk minimization is an incomplete inductive principle. MIT Media Lab note, August 2000.
18. J. Mooij. libDAI: A free and open source C++ library for discrete approximate inference in graphical models. Journal of Machine Learning Research, 11:2169-2173, Aug 2010.
19. F. Och. Minimum error rate training in statistical machine translation. In Proceedings of ACL, 2003.
20. J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988. ISBN 1558604790.
21. B. Pearlmutter. Fast exact multiplication by the Hessian. Neural Computation, 6:147-160, 1994.
22. N.N. Schraudolph. Local gain adaptation in stochastic gradient descent. In Proceedings of the Ninth International Conference on Artificial Neural Networks, volume 2, pages 569-574, 1999.
23. Veselin Stoyanov and Jason Eisner. Minimum-risk training of approximate CRF-based NLP systems, in review.
24. C. Sutton and A. McCallum. Piecewise training of undirected models. In 21st Conference on Uncertainty in Artificial Intelligence, 2005.
25. J. Suzuki, E. McDermott, and H. Isozaki. Training conditional random fields with multivariate evaluation measures. In Proceedings of COLING/ACL, pages 217-224, 2006.
26. S.V.N. Vishwanathan, N.N. Schraudolph, M.W. Schmidt, and K.P. Murphy. Accelerated training of conditional random fields with stochastic gradient methods. In Proceedings of the 23rd International Conference on Machine Learning, pages 969-976. ACM, 2006.
27. M. J. Wainwright. Estimating the "wrong" graphical model: Benefits in the computation-limited setting. Journal of Machine Learning Research, 7: 1829-1859, September 2006.
28. R.J. Williams and D. Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1(2):270-280, 1989. ISSN 0899-7667.
29. J. Yedidia, W. Freeman, and Y. Weiss. Bethe free energy, Kikuchi approximations and belief propagation algorithms. Technical Report TR2001-16, Mitsubishi Electric Research Laboratories, 2000.