Breaking out of local optima with count transforms and model recombination: A study in grammar induction

EMNLP 2013 - 2013 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

Volumn , Issue , 2013, Pages 1983-1995

  Spitkovsky, Valentin I ^a   Alshawi, Hiyan ^a   Jurafsky, Daniel ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

COMPLEX NETWORKS; COMPUTATIONAL LINGUISTICS; NATURAL LANGUAGE PROCESSING SYSTEMS;

BUILDING BLOCKES; DEPENDENCY GRAMMAR; GRAMMAR INDUCTION; LOCAL OPTIMIZERS; OPTIMAL SOLUTIONS; ORGANIZED NETWORKS; STATE OF THE ART; STATISTICAL LEARNING;

COMPUTATIONAL GRAMMARS;

EID: 84926302248     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (99)

1. H. Alshawi. 1996. Head automata for speech translation. In ICSLP.
2. F. Attneave. 1953. Psychological probability as a function of experienced frequency. Experimental Psychology, 46.
3. M. Bar-Hillel. 1980. The base-rate fallacy in probability judgments. Acta Psychologica, 44.
4. A. Belz. 1998. Discovering phonotactic finite-state automata by genetic search. In COLING-ACL.
5. Y. Bengio, J. Louradour, R. Collobert, and J. Weston. 2009. Curriculum learning. In ICML.
6. A. Bhargava and G. Kondrak. 2009. Multiple word alignment with profile hidden Markov models. In NAACL-HLT: Student Research and Doctoral Consortium.
7. Y. Bisk and J. Hockenmaier. 2012. Simple robust grammar induction with combinatory categorial grammars. In AAAI.
8. A. Blum and T. Mitchell. 1998. Combining labeled and unla-beled data with co-training. In COLT.
9. P. Blunsom and T. Cohn. 2010. Unsupervised induction of tree substitution grammars for dependency parsing. In EMNLP.
10. S. Buchholz and E. Marsi. 2006. CoNLL-X shared task on multilingual dependency parsing. In CoNLL.
11. X. Carreras and L. Màrquez. 2005. Introduction to the CoNLL-2005 shared task: Semantic role labeling. In CoNLL.
12. L. J. Chapman. 1967. Illusory correlation in observational report. Verbal Learning and Verbal Behavior, 6.
13. A. Clark. 2000. Inducing syntactic categories by context distribution clustering. In CoNLL-LLL.
14. A. Clark. 2003. Combining distributional and morphological information for part of speech induction. In EACL.
15. S. B. Cohen and N. A. Smith. 2010. Viterbi training for PCFGs: Hardness results and competitiveness of uniform initialization. In ACL.
16. T. Cohn, P. Blunsom, and S. Goldwater. 2011. Inducing tree-substitution grammars. JMLR.
17. M. Collins and Y. Singer. 1999. Unsupervised models for named entity classification. In EMNLP.
18. M. Collins. 1999. Head-Driven Statistical Models for Natural Language Parsing. Ph.D. thesis, University of Pennsylvania.
19. E. Corlett and G. Penn. 2010. An exact Å method for deciphering letter-substitution ciphers. In ACL.
20. F. I. M. Craik and E. Bialystok. 2006. Cognition through the lifespan: mechanisms of change. TRENDS in Cognitive Sciences, 10.
21. C. de Marcken. 1995. Lexical heads, phrase structure and the induction of grammar. In WVLC.
22. K. Duh and K. Kirchhoff. 2004. Automatic learning of language model structure. In COLING.
23. J. Eisner. 2012. Grammar induction: Beyond local search. In ICGI.
24. G. Elidan, M. Ninio, N. Friedman, and D. Schuurmans. 2002. Data perturbation for escaping local maxima in learning. In AAAI.
25. J. L. Elman. 1993. Learning and development in neural networks: The importance of starting small. Cognition, 48.
26. M. Elsner and W. Schudy. 2009. Bounding and comparing methods for correlation clustering beyond ILP. In NAACL-HLT: Integer Linear Programming for NLP.
27. C. M. Fonseca and P. J. Fleming. 1993. Genetic algorithms for multiobjective optimization: Formulation, discussion and generalization. In ICGA.
28. C. J. Geyer. 1991. Markov chain Monte Carlo maximum likelihood. In Interface Symposium.
29. J. Gillenwater, K. Ganchev, J. Graça, F. Pereira, and B. Taskar. 2010. Posterior sparsity in unsupervised dependency parsing. Technical report, University of Pennsylvania.
30. K. Gimpel and N. A. Smith. 2012. Concavity and initialization for unsupervised dependency parsing. In NAACL-HLT.
31. F. Glover and M. Laguna. 1993. Tabu search. In C. R. Reeves, editor, Modern Heuristic Techniques for Combinatorial Problems. Blackwell Scientific Publications.
32. F. Glover. 1989. Tabu search - Part I. ORSA Journal on Computing, 1.
33. D. E. Goldberg. 1989. Genetic Algorithms In Search, Optimization & Machine Learning. Addison-Wesley.
34. D. Golland, J. DeNero, and J. Uszkoreit. 2012. A feature-rich constituent context model for grammar induction. In EMNLP-CoNLL.
35. M. R. Gormley and J. Eisner. 2013. Nonconvex global optimization for latent-variable models. In ACL.
36. W. P. Headden, III, M. Johnson, and D. McClosky. 2009. Improving unsupervised dependency parsing with richer contexts and smoothing. In NAACL-HLT.
37. G. Hinton and S. Roweis. 2003. Stochastic neighbor embedding. In NIPS.
38. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. In ArXiv.
39. J. H. Holland. 1975. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. University of Michigan Press.
40. H. H. Hoos and T. Stützle. 2004. Stochastic Local Search: Foundations and Applications. Morgan Kaufmann.
41. C. R. Houck, J. A. Joines, and M. G. Kay. 1996. Comparison of genetic algorithms, random restart, and two-opt switching for solving large location-allocation problems. Computers & Operations Research, 23.
42. X. Hu, R. Shonkwiler, and M. C. Spruill. 1994. Random restarts in global optimization. Technical report, GT.
43. Y. Huang, M. Zhang, and C. L. Tan. 2012. Improved constituent context model with features. In PACLIC.
44. F. Jelinek and R. L. Mercer. 1980. Interpolated estimation of Markov source parameters from sparse data. In Pattern Recognition in Practice.
45. D. S. Johnson, C. H. Papadimitriou, and M. Yannakakis. 1988. How easy is local search? Journal of Computer and System Sciences, 37.
46. D. Kahneman and A. Tversky. 1972. Subjective probability: A judgment of representativeness. Cognitive Psychology, 3.
47. D. Kahneman and A. Tversky. 1973. On the psychology of prediction. Psychological Review, 80.
48. D. Kahneman and A. Tversky. 1982. Evidential impact of base rates. In D. Kahneman, P. Slovic, and A. Tversky, editors, Judgment under uncertainty: Heuristics and biases. Cambridge University Press.
49. J. Ke, M. Ogura, and W. S.-Y. Wang. 2003. Optimization models of sound systems using genetic algorithms. Computational Linguistics, 29.
50. J. Kim and R. J. Mooney. 2010. Generative alignment and semantic parsing for learning from ambiguous supervision. In COLING.
51. S. Kirkpatrick, C. D. Gelatt, Jr., and M. P. Vecchi. 1983. Optimization by simulated annealing. Science, 220.
52. D. Klein and C. D. Manning. 2002. A generative constituentcontext model for improved grammar induction. In ACL.
53. D. Klein and C. D. Manning. 2004. Corpus-based induction of syntactic structure: Models of dependency and constituency. In ACL.
54. D. Koller and N. Friedman. 2009. Probabilistic Graphical Models: Principles and Techniques. MIT Press.
55. K. A. Krueger and P. Dayan. 2009. Flexible shaping: How learning in small steps helps. Cognition, 110.
56. D. Lashkari and P. Golland. 2008. Convex clustering with exemplar-based models. In NIPS.
57. P. Liang and D. Klein. 2009. Online EM for unsupervised models. In NAACL-HLT.
58. L. K. Low and H.-J. Cheng. 2006. Axon pruning: an essential step underlying the developmental plasticity of neuronal connections. Royal Society of London Philosophical Transactions Series B, 361.
59. M. P. Marcus, B. Santorini, and M. A. Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19.
60. D. Mareček and M. Straka. 2013. Stop-probability estimates computed on a large corpus improve unsupervised dependency parsing. In ACL.
61. D. Mareček and Z. Žabokrtský. 2011. Gibbs sampling with treeness constraint in unsupervised dependency parsing. In ROBUS.
62. D. Mareček and Z. Žabokrtský. 2012. Exploiting reducibility in unsupervised dependency parsing. In EMNLP-CoNLL.
63. R. Martin-Brualla, E. Alfonseca, M. Pasca, K. Hall, E. Robledo-Arnuncio, and M. Ciaramita. 2010. Instance sense induction from attribute sets. In COLING.
64. N. McIntyre and M. Lapata. 2010. Plot induction and evolutionary search for story generation. In ACL.
65. X.-d. Mei, S.-h. Sun, J.-s. Pan, and T.-Y. Chen. 2001. Optimization of HMM by the tabu search algorithm. In RO-CLING.
66. C. Mellish, A. Knott, J. Oberlander, and M. O'Donnell. 1998. Experiments using stochastic search for text planning. In INLG.
67. R. C. Moore and C. Quirk. 2008. Random restarts in minimum error rate training for statistical machine translation. In COLING.
68. T. Naseem and R. Barzilay. 2011. Using semantic cues to learn syntax. In AAAI.
69. R. M. Neal and G. E. Hinton. 1999. A view of the EM algorithm that justifies incremental, sparse, and other variants. In M. I. Jordan, editor, Learning in Graphical Models. MIT Press.
70. J. Nivre, J. Hall, S. Kübler, R. McDonald, J. Nilsson, S. Riedel, and D. Yuret. 2007. The CoNLL 2007 shared task on dependency parsing. In EMNLP-CoNLL.
71. M. A. Paskin. 2001a. Cubic-time parsing and learning algorithms for grammatical bigram models. Technical report, UCB.
72. M. A. Paskin. 2001b. Grammatical bigrams. In NIPS.
73. F. Pereira and Y. Schabes. 1992. Inside-outside reestimation from partially bracketed corpora. In ACL.
74. S. Petrov. 2010. Products of random latent variable grammars. In NAACL-HLT.
75. E. Ponvert, J. Baldridge, and K. Erk. 2011. Simple unsuper-vised grammar induction from raw text with cascaded finite state models. In ACL-HLT.
76. K. V. Price, R. M. Storn, and J. A. Lampinen. 2005. Differential Evolution: A Practical Approach to Global Optimization. Springer.
77. S. Ravi and K. Knight. 2009. Minimized models for unsuper-vised part-of-speech tagging. In ACL-IJCNLP.
78. K. Rose. 1998. Deterministic annealing for clustering, compression, classification, regression and related optmization problems. Proceedings of the IEEE, 86.
79. Y. Seginer. 2007. Fast unsupervised incremental parsing. In ACL.
80. B. Selman, H. Levesque, and D. Mitchell. 1992. A new method for solving hard satisfiability problems. In AAAI.
81. B. Selman, H. A. Kautz, and B. Cohen. 1994. Noise strategies for improving local search. In AAAI.
82. F. J. Solis and R. J.-B. Wets. 1981. Minimization by random search techniques. Mathematics of Operations Research, 6.
83. V. I. Spitkovsky, H. Alshawi, and D. Jurafsky. 2009. Baby Steps: How "Less is More" in unsupervised dependency parsing. In GRLL.
84. V. I. Spitkovsky, H. Alshawi, D. Jurafsky, and C. D. Manning. 2010. Viterbi training improves unsupervised dependency parsing. In CoNLL.
85. V. I. Spitkovsky, H. Alshawi, and D. Jurafsky. 2011a. Lateen EM: Unsupervised training with multiple objectives, applied to dependency grammar induction. In EMNLP.
86. V. I. Spitkovsky, H. Alshawi, and D. Jurafsky. 2011b. Punctuation: Making a point in unsupervised dependency parsing. In CoNLL.
87. V. I. Spitkovsky, A. X. Chang, H. Alshawi, and D. Jurafsky. 2011c. Unsupervised dependency parsing without gold part-of-speech tags. In EMNLP.
88. V. I. Spitkovsky, H. Alshawi, and D. Jurafsky. 2012a. Bootstrapping dependency grammar inducers from incomplete sentence fragments via austere models. In ICGI.
89. V. I. Spitkovsky, H. Alshawi, and D. Jurafsky. 2012b. Three dependency-and-boundary models for grammar induction. In EMNLP-CoNLL.
90. W. Stadler, editor. 1988. Multicriteria Optimization in Engineering and in the Sciences. Plenum Press.
91. A. Sun, R. Grishman, and S. Sekine. 2011. Semi-supervised relation extraction with large-scale word clustering. In ACL.
92. M. Surdeanu and C. D. Manning. 2010. Ensemble models for dependency parsing: Cheap and good? In NAACL-HLT.
93. K. Tu and V. Honavar. 2011. On the utility of curricula in unsupervised learning of probabilistic grammars. In IJCAI.
94. K. Tu and V. Honavar. 2012. Unambiguity regularization for unsupervised learning of probabilistic grammars. In EMNLP-CoNLL.
95. A. Tversky and D. Kahneman. 1973. Availability: A heuristic for judging frequency and probability. Cognitive Psychology, 5.
96. S. I. Wang and C. D. Manning. 2013. Fast dropout training. In ICML.
97. Q. I. Wang, D. Schuurmans, and D. Lin. 2008. Semi-supervised convex training for dependency parsing. In HLT-ACL.
98. T. Xiao, J. Zhu, M. Zhu, and H. Wang. 2010. Boosting-based system combination for machine translation. In ACL.
99. D. Yarowsky. 1995. Unsupervised word sense disambiguation rivaling supervised methods. In ACL.