Spectral learning of latent-variable PCFGs: Algorithms and sample complexity

Journal of Machine Learning Research

Volumn 15, Issue , 2014, Pages 2399-2449

  Cohen, Shay B ^a   Stratos, Karl ^b   Collins, Michael ^b   Foster, Dean P ^c   Ungar, Lyle ^d  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b Department of Earth and Environmental Sciences   (United States)

^c YAHOO RESEARCH   (United States)

^d UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Latent variable PCFGs; Spectral learning algorithms

Indexed keywords

LEARNING ALGORITHMS;

CONVERGENCE BOUNDS; ESTIMATION METHODS; LATENT-VARIABLE PCFGS; PARAMETER ESTIMATE; SAMPLE COMPLEXITY; SPECTRAL LEARNING; SPECTRAL LEARNING ALGORITHMS; TRAINING EXAMPLE;

TENSORS;

EID: 84907302464     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (36)

1. S. Arora, R. Ge, Y. Halpern, D. M. Mimno, A. Moitra, D. Sontag, Y. Wu, and M. Zhu. A practical algorithm for topic modeling with provable guarantees. In Proceedings of ICML, 2013.
2. R. Bailly, A. Habrar, and F. Denis. A spectral approach for probabilistic grammatical inference on trees. In Proceedings of ALT, 2010.
3. R. Bailly, Carreras P. X., F. M. Luque, and A. J. Quattoni. Unsupervised spectral learning of WCFG as low-rank matrix completion. In Proceedings of EMNLP, 2013.
4. J. Baker. Trainable grammars for speech recognition. In Proceedings of ASA, 1979.
5. B. Balle, A. Quattoni, and X. Carreras. A spectral learning algorithm for finite state transducers. In Proceedings of ECML, 2011.
6. P.J. Bickel and K.A. Doksum. Mathematical Statistics: Basic Ideas And Selected Topics. Mathematical Statistics: Basic Ideas and Selected Topics. Pearson Prentice Hall, 2006.
7. E. Charniak. Statistical parsing with a context-free grammar and word statistics. In Proceedings of AAAI-IAAI, 1997.
8. S. B. Cohen and M. Collins. A provably correct learning algorithm for latent-variable PCFGs. In Proceedings of ACL, 2014.
9. S. B. Cohen, K. Stratos, M. Collins, D. P. Foster, and L. Ungar. Spectral learning of latent-variable PCFGs. In Proceedings of ACL, 2012.
10. S. B. Cohen, K. Stratos, M. Collins, D. P. Foster, and L. Ungar. Experiments with spectral learning of latent-variable PCFGs. In Proceedings of NAACL, 2013.
11. M. Collins. Three generative, lexicalised models for statistical parsing. In Proceedings of ACL, 1997.
12. S. Dasgupta. Learning mixtures of Gaussians. In Proceedings of FOCS, 1999.
13. A. Dempster, N. Laird, and D. Rubin. Maximum likelihood estimation from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B, 39:1-38, 1977.
14. P. Dhillon, D. Foster, and L. Ungar. Multi-view learning of word embeddings via CCA. In Proceedings of NIPS, 2011.
15. P. Dhillon, J. Rodu, M. Collins, D. P. Foster, and L. H. Ungar. Spectral dependency parsing with latent variables. In Proceedings of EMNLP, 2012.
16. D. P. Foster, J. Rodu, and L. H. Ungar. Spectral dimensionality reduction for HMMs. arXiv:1203.6130, 2012.
17. J. Goodman. Parsing algorithms and metrics. In Proceedings of ACL, 1996.
18. D. Hsu, S. M. Kakade, and T. Zhang. A spectral algorithm for learning hidden Markov models. In Proceedings of COLT, 2009.
19. H. Jaeger. Observable operator models for discrete stochastic time series. Neural Computation, 12(6), 2000.
20. M. Johnson. PCFG models of linguistic tree representations.
21. D. Klein and C.D. Manning. Accurate Unlexicalized Parsing. In Proceedings of ACL, pages 423-430, 2003.
22. F. M. Luque, A. Quattoni, B. Balle, and X. Carreras. Spectral learning for non-deterministic dependency parsing. In Proceedings of EACL, 2012.
23. M.P. Marcus, B. Santorini, and M.A. Marcinkiewicz. Building a large annotated corpus of english: The penn treebank. Computational Linguistics, 19(2):313-330, 1993.
24. T. Matsuzaki, Y. Miyao, and J. Tsujii. Probabilistic CFG with latent annotations. In Proceedings of ACL, 2005.
25. C. McDiarmid. On the method of bounded differences. Surveys in Combinatorics, pages 148-188, 1989.
26. A. Moitra and G. Valiant. Settling the polynomial learnability of mixtures of Gaussians. IEEE Annual Symposium on Foundations of Computer Science, pages 93-102, 2010. ISSN 0272-5428.
27. A. Parikh, L. Song, and E. P. Xing. A spectral algorithm for latent tree graphical models. In Proceedings of ICML, 2011.
28. S. Petrov, L. Barrett, R. Thibaux, and D. Klein. Learning accurate, compact, and interpretable tree annotation. In Proceedings of COLING-ACL, 2006.
29. S. Siddiqi, B. Boots, and G. Gordon. Reduced-rank hidden Markov models. Journal of Machine Learning Research, 9:741-748, 2010.
30. L. Song, B. Boots, S. M. Siddiqi, G. J. Gordon, and A. J. Smola. Hilbert space embeddings of hidden Markov models. In Proceedings of ICML, 2010.
31. L. Song, A. P. Parikh, and E. P. Xing. Kernel embeddings of latent tree graphical models. In NIPS, pages 2708-2716, 2011.
32. K. Stratos, A. M. Rush, S. B. Cohen, and M. Collins. Spectral learning of refinement HMMs. In Proceedings of CoNLL, 2013.
33. S. A. Terwijn. On the learnability of hidden Markov models. In Grammatical Inference: Algorithms and Applications (Amsterdam, 2002), volume 2484 of Lecture Notes in Artificial Intelligence, pages 261-268, Berlin, 2002. Springer.
34. A. Tropp, N. Halko, and P. G. Martinsson. Finding structure with randomness: Stochastic algorithms for constructing approximate matrix decompositions. In Technical Report No. 2009-05, 2009.
35. L. Valiant. A theory of the learnable. Communications of the ACM, 27:1134-1142, 1984.
36. S. Vempala and G. Wang. A spectral algorithm for learning mixtures of distributions. Journal of Computer and System Sciences, 68(4):841-860, 2004.