Lateen EM: Unsupervised training with multiple objectives, applied to dependency grammar induction

EMNLP 2011 - Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

Volumn , Issue , 2011, Pages 1269-1280

  Spitkovsky, Valentin I ^a   Alshawi, Hiyan ^a   Jurafsky, Daniel ^b  

^a GOOGLE INC   (United States)

^b STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DEPENDENCY GRAMMAR; DEPENDENCY PARSING; EM ALGORITHMS; EXPECTATION-MAXIMIZATION ALGORITHMS; FIXED POINTS; LOCAL OPTIMA; MULTIPLE OBJECTIVES; TRAINING METHODS; UNSUPERVISED TRAINING;

COMPUTATIONAL GRAMMARS; NATURAL LANGUAGE PROCESSING SYSTEMS;

ALGORITHMS;

EID: 80053220392     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (62)

1. S. Abney. 1996. Statistical methods and linguistics. In J. L. Klavans and P. Resnik, editors, The Balancing Act: Combining Symbolic and Statistical Approaches to Language. MIT Press.
2. E. L. Allgower and K. Georg. 1990. Numerical Continuation Methods: An Introduction. Springer-Verlag.
3. S. Arlot and A. Celisse. 2010. A survey of cross-validation procedures for model selection. Statistics Surveys, 4.
4. J. K. Baker. 1979. Trainable grammars for speech recognition. In Speech Communication Papers for the 97th Meeting of the Acoustical Society of America.
5. M.-F. Balcan, A. Blum, and K. Yang. 2004. Co-training and expansion: Towards bridging theory and practice. In NIPS.
6. T. Berg-Kirkpatrick, A. Bouchard-Côté, J. DeNero, and D. Klein. 2010. Painless unsupervised learning with features. In NAACL-HLT.
7. A. Blum and T. Mitchell. 1998. Combining labeled and unlabeled data with co-training. In COLT.
8. P. Blunsom and T. Cohn. 2010. Unsupervised induction of tree substitution grammars for dependency parsing. In EMNLP.
9. P. F. Brown, V. J. Della Pietra, S. A. Della Pietra, and R. L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19.
10. S. Buchholz and E. Marsi. 2006. CoNLL-X shared task on multilingual dependency parsing. In CoNLL.
11. J. Chan, I. Koprinska, and J. Poon. 2004. Co-training with a single natural feature set applied to email classification. In WI.
12. M.-W. Chang, L. Ratinov, and D. Roth. 2007. Guiding semi-supervision with constraint-driven learning. In ACL.
13. E. Charniak. 1997. Statistical techniques for natural language parsing. AI Magazine, 18.
14. S. B. Cohen and N. A. Smith. 2009. Shared logistic normal distributions for soft parameter tying in unsu-pervised grammar induction. In NAACL-HLT.
15. M. Collins and Y. Singer. 1999. Unsupervised models for named entity classification. In EMNLP.
16. M. Collins. 1999. Head-Driven Statistical Models for Natural Language Parsing. Ph.D. thesis, University of Pennsylvania.
17. J. Crim. 2002. Co-training re-rankers for improved parser accuracy.
18. S. Dasgupta, M. L. Littman, and D. McAllester. 2001. PAC generalization bounds for co-training. In NIPS.
19. A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B, 39.
20. I. S. Dhillon, Y. Guan, and J. Kogan. 2002. Iterative clustering of high dimensional text data augmented by local search. In ICDM.
21. D. Elworthy. 1994. Does Baum-Welch re-estimation help taggers? In ANLP.
22. S. Geisser. 1975. The predictive sample reuse method with applications. Journal of the American Statistical Association, 70.
23. J. Gillenwater, K. Ganchev, J. Graça, F. Pereira, and B. Taskar. 2010. Posterior sparsity in unsupervised dependency parsing. Technical report, University of Pennsylvania.
24. S. Goldman and Y. Zhou. 2000. Enhancing supervised learning with unlabeled data. In ICML.
25. W. P. Headden, III, M. Johnson, and D. McClosky. 2009. Improving unsupervised dependency parsing with richer contexts and smoothing. In NAACL-HLT.
26. S. Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6.
27. J. P. A. Ioannidis. 2005. Why most published research findings are false. PLoS Medicine, 2.
28. M. Johnson. 2009. How the statistical revolution changes (computational) linguistics. In EACL: Interaction between Linguistics and Computational Linguistics: Virtuous, Vicious or Vacuous?
29. M. Kearns, Y. Mansour, and A. Y. Ng. 1997. An information-theoretic analysis of hard and soft assignment methods for clustering. In UAI.
30. D. Klein and C. D. Manning. 2004. Corpus-based induction of syntactic structure: Models of dependency and constituency. In ACL.
31. D. Klein. 2005. The Unsupervised Learning of Natural Language Structure. Ph.D. thesis, Stanford University.
32. S. C. Larson. 1931. The shrinkage of the coefficient of multiple correlation. Journal of Educational Psychology, 22.
33. P. Liang and D. Klein. 2008. Analyzing the errors of unsupervised learning. In HLT-ACL.
34. D. C. Liu and J. Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Mathematical Programming. Series B, 45.
35. B. T. Lowerre. 1976. The HARPY Speech Recognition System. Ph.D. thesis, CMU.
36. L. B. Lucy. 1974. An iterative technique for the rectification of observed distributions. The Astronomical Journal, 79.
37. M. P. Marcus, B. Santorini, and M. A. Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19.
38. X.-L. Meng. 2007. EM and MCMC: Workhorses for scientific computing (thirty years of EM and much more). Statistica Sinica, 17.
39. B. Merialdo. 1994. Tagging English text with a probabilistic model. Computational Linguistics, 20.
40. D. C. Montgomery. 2005. Design and Analysis of Experiments. John Wiley & Sons, 6th edition.
41. M. Mylonakis and K. Sima'an. 2010. Learning probabilistic synchronous CFGs for phrase-based translation. In CoNLL.
42. R. M. Neal and G. E. Hinton. 1999. A view of the EM algorithm that justifies incremental, sparse, and other variants. In M. I. Jordan, editor, Learning in Graphical Models. MIT Press.
43. V. Ng and C. Cardie. 2003. Weakly supervised natural language learning without redundant views. In HLT-NA ACL.
44. K. Nigam and R. Ghani. 2000. Analyzing the effectiveness and applicability of co-training. In CIKM.
45. J. Nivre, J. Hall, S. Kübler, R. McDonald, J. Nilsson, S. Riedel, and D. Yuret. 2007. The CoNLL 2007 shared task on dependency parsing. In EMNLP-CoNLL.
46. M. A. Paskin. 2001. Grammatical bigrams. In NIPS.
47. F. Pereira and Y. Schabes. 1992. Inside-outside reestimation from partially bracketed corpora. In ACL.
48. R Development Core Team, 2011. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing.
49. S. Ravi and K. Knight. 2009. Minimized models for un-supervised part-of-speech tagging. In ACL-IJCNLP.
50. K. Rose. 1998. Deterministic annealing for clustering, compression, classification, regression and related optmization problems. Proceedings of the IEEE, 86.
51. A. Sarkar. 2001. Applying co-training methods to statistical parsing. In NAACL.
52. R. Schwartz, O. Abend, R. Reichart, and A. Rappoport. 2011. Neutralizing linguistically problematic annotations in unsupervised dependency parsing evaluation. In ACL.
53. N. A. Smith and J. Eisner. 2004. Annealing techniques for unsupervised statistical language learning. In ACL.
54. N. A. Smith and J. Eisner. 2005a. Contrastive estimation: Training log-linear models on unlabeled data. In ACL.
55. N. A. Smith and J. Eisner. 2005b. Guiding unsupervised grammar induction using contrastive estimation. In IJ-CAI: Grammatical Inference Applications.
56. A. Søgaard and C. Rishøj. 2010. Semi-supervised dependency parsing using generalized tri-training. In COLING.
57. V. I. Spitkovsky, H. Alshawi, and D. Jurafsky. 2009. Baby Steps: How "Less is More" in unsupervised dependency parsing. In NIPS: Grammar Induction, Representation of Language and Language Learning.
58. V. I. Spitkovsky, H. Alshawi, D. Jurafsky, and C. D. Manning. 2010a. Viterbi training improves unsupervised dependency parsing. In CoNLL.
59. V. I. Spitkovsky, D. Jurafsky, and H. Alshawi. 2010b. Profiting from mark-up: Hyper-text annotations for guided parsing. In ACL.
60. M. Stone. 1974. Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society. Series B, 36.
61. A. Yessenalina, Y. Yue, and C. Cardie. 2010. Multilevel structured models for document-level sentiment classification. In EMNLP.
62. Y. Zhou and S. Goldman. 2004. Democratic co-learning. In ICTAI.