Conditional random field autoencoders for unsupervised structured prediction

Advances in Neural Information Processing Systems

Volumn 4, Issue January, 2014, Pages 3311-3319

  Ammar, Waleed ^a   Dyer, Chris ^a   Smith, Noah A ^a  

^a Carnegie Mellon University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION SCIENCE; LEARNING SYSTEMS; NATURAL LANGUAGE PROCESSING SYSTEMS;

CONDITIONAL RANDOM FIELD; GENERATIVE MODEL; INDEPENDENCE ASSUMPTION; LATENT STRUCTURES; MULTI-VIEW LEARNING; NATURAL LANGUAGE PROCESSING; PART-OF-SPEECH INDUCTIONS; STRUCTURED PREDICTION;

RANDOM PROCESSES;

EID: 84937844873     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (47)

1. T. L. Bailey and C. Elkan. Unsupervised learning of multiple motifs in biopolymers using expectation maximization. Machine learning, 1995.
2. K. Bellare, G. Druck, and A. McCallum. Alternating projections for learning with expectation constraints. In Proc. of UAI, 2009.
3. T. Berg-Kirkpatrick, A. Bouchard-Côté, J. DeNero, and D. Klein. Painless unsupervised learning with features. In Proc. of NAACL, 2010.
4. P. Blunsom and T. Cohn. Discriminative word alignment with conditional random fields. In Proc. of Proceedings of ACL, 2006.
5. P. F. Brown, P. V. de Souza, R. L. Mercer, V. J. D. Pietra, and J. C. Lai. Class-based n-gram models of natural language. Computational Linguistics, 1992.
6. P. F. Brown, V. J. D. Pietra, S. A. D. Pietra, and R. L. Mercer. The mathematics of statistical machine translation: parameter estimation. In Computational Linguistics, 1993.
7. S. Buchholz and E. Marsi. CoNLL-X shared task on multilingual dependency parsing. In CoNLL-X, 2006.
8. R. Collobert and J. Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proc. of ICML, 2008.
9. H. Daumé III. Unsupervised search-based structured prediction. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 209-216. ACM, 2009.
10. A. P. Dempster, N. M. Laird, D. B. Rubin, et al. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal statistical Society, 39(1): 1-38, 1977.
11. G. Druck and A. McCallum. High-performance semi-supervised learning using discriminatively constrained generative models. In Proc. of ICML, 2010.
12. J. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for online learning and stochastic optimization. JMLR, 2011.
13. C. Dyer, A. Lopez, J. Ganitkevitch, J. Weese, F. Ture, P. Blunsom, H. Setiawan, V. Eidelman, and P. Resnik. cdec: A decoder, alignment, and learning framework for finite-state and context-free translation models. In Proc. of ACL, 2010.
14. C. Dyer, J. Clark, A. Lavie, and N. A. Smith. Unsupervised word alignment with arbitrary features. In Proc. of ACL-HLT, 2011.
15. C. Dyer, V. Chahuneau, and N. A. Smith. A simple, fast, and effective reparameterization of IBM Model 2. In Proc. of NAACL, 2013.
16. K. Ganchev, J. Graça, J. Gillenwater, and B. Taskar. Posterior regularization for structured latent variable models. Journal of Machine Learning Research, 11: 2001-2049, 2010.
17. Q. Gao and S. Vogel. Parallel implementations of word alignment tool. In Proc. of the ACL workshop, 2008.
18. A. Haghighi and D. Klein. Prototype-driven learning for sequence models. In Proc. of NAACLHLT, 2006.
19. F. Jelinek. Statistical Methods for Speech Recognition. MIT Press, 1997.
20. M. Johnson. Why doesn't EM find good HMM POS-taggers? In Proc. of EMNLP, 2007.
21. D. Klein and C. D. Manning. Corpus-based induction of syntactic structure: Models of dependency and constituency. In Proc. of ACL, 2004.
22. P. Koehn. Statistical Machine Translation. Cambridge, 2010.
23. P. Koehn, F. J. Och, and D. Marcu. Statistical phrase-based translation. In Proc. of NAACL, 2003.
24. J. Lafferty, A. McCallum, and F. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proc. of ICML, 2001.
25. P. Liang. Semi-supervised learning for natural language. In Thesis, MIT, 2005.
26. C.-C. Lin, W. Ammar, C. Dyer, and L. Levin. The cmu submission for the shared task on language identification in code-switched data. In First Workshop on Computational Approaches to Code Switching at EMNLP, 2014.
27. A. V. Lukashin and M. Borodovsky. Genemark. hmm: new solutions for gene finding. Nucleic acids research, 26(4): 1107-1115, 1998.
28. B. Merialdo. Tagging english text with a probabilistic model. In Comp. Ling., 1994.
29. T. Minka. Discriminative models, not discriminative training. Technical report, Technical Report MSR-TR-2005-144, Microsoft Research, 2005.
30. A. Mnih and Y. W. Teh. A fast and simple algorithm for training neural probabilistic language models. In Proc. of ICML, 2012.
31. J. Nivre, J. Hall, S. Kubler, R. McDonald, J. Nilsson, S. Riedel, and D. Yuret. The CoNLL 2007 shared task on dependency parsing. In Proc. of CoNLL, 2007.
32. F. Och and H. Ney. A systematic comparison of various statistical alignment models. Computational Linguistics, 2003.
33. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. Bleu: a method for automatic evaluation of machine translation. In Proc. of ACL, 2002.
34. S. Petrov, D. Das, and R. McDonald. A universal part-of-speech tagset. In Proc. of LREC, May 2012.
35. H. Poon and P. Domingos. Joint unsupervised coreference resolution with Markov logic. In Proc. of EMNLP, 2008.
36. S. Reddy and S. Waxmonsky. Substring-based transliteration with conditional random fields. In Proc. of the Named Entities Workshop, 2009.
37. A. Rosenberg and J. Hirschberg. V-measure: A conditional entropy-based external cluster evaluation measure. In EMNLP-CoNLL, 2007.
38. N. A. Smith and J. Eisner. Contrastive estimation: Training log-linear models on unlabeled data. In Proc. of ACL, 2005.
39. R. Socher, C. D. Manning, and A. Y. Ng. Learning continuous phrase representations and syntactic parsing with recursive neural networks. In NIPS workshop, 2010.
40. V. Stoyanov, A. Ropson, and J. Eisner. Empirical risk minimization of graphical model parameters given approximate inference, decoding, and model structure. In Proc. of AISTATS, 2011.
41. J. Suzuki and H. Isozaki. Semi-supervised sequential labeling and segmentation using giga-word scale unlabeled data. In Proc. of ACL, 2008.
42. R. Swier and S. Stevenson. Unsupervised semantic role labelling. In Proc. of EMNLP, 2004.
43. D. Vickrey, C. C. Lin, and D. Koller. Non-local contrastive objectives. In Proc. of ICML, 2010.
44. P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol. Extracting and composing robust features with denoising autoencoders. In Proc. of ICML, 2008.
45. S. Vogel, H. Ney, and C. Tillmann. Hmm-based word alignment in statistical translation. In Proc. of COLING, 1996.
46. M. Weber, M. Welling, and P. Perona. Unsupervised learning of models for recognition. 2000.
47. J. Yamato, J. Ohya, and K. Ishii. Recognizing human action in time-sequential images using hidden Markov model. In Proc. of CVPR, pages 379-385. IEEE, 1992.