Spectral learning of weighted automata: A forward-backward perspective

Machine Learning

Volumn 96, Issue 1-2, 2014, Pages 33-63

  Balle, Borja ^a   Carreras, Xavier ^a   Luque, Franco M ^b   Quattoni, Ariadna ^a  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

^b UNIVERSIDAD NACIONAL DE CÓRDOBA   (Argentina)

Author keywords

Dependency parsing; Spectral learning; Weighted finite automata

Indexed keywords

ALGEBRA; FINITE AUTOMATA; MATRIX ALGEBRA;

DEPENDENCY PARSING; LEARNING COMPLEXITY; NATURAL LANGUAGE PARSING; PARTICULAR DISTRIBUTION; REAL APPLICATIONS; SPECTRAL ALGORITHM; SPECTRAL LEARNING; WEIGHTED FINITE AUTOMATON;

LEARNING ALGORITHMS;

EID: 84903884130     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-013-5416-x     Document Type: Article

References (52)

1. Anandkumar, A., Foster, D. P., Hsu, D., Kakade, S., & Liu, Y. K. (2012a). A spectral algorithm for latent Dirichlet allocation. In P. L. Bartlett, F. C. N. Pereira, C. J. C. Burges, L. Bottou, & K. Q. Weinberger (Eds.), NIPS (pp. 926-934).
2. Anandkumar, A., Ge, R., Hsu, D., Kakade, S. M., & Telgarsky, M. (2012b). Tensor decompositions for learning latent variable models. arXiv:1210.7559.
3. Anandkumar, A., Hsu, D., & Kakade, S. M. (2012c). A method of moments for mixture models and hidden Markov models. Journal of Machine Learning Research - Proceedings Track, 23, 33.1-33.34.
4. Bailly, R. (2011). Quadratic weighted automata: spectral algorithm and likelihood maximization. Journal of Machine Learning Research - Proceedings Track, 20, 147-163.
5. Bailly, R., Denis, F., & Ralaivola, L. (2009). Grammatical inference as a principal component analysis problem. In L. Bottou & M. Littman (Eds.), Proceedings of the 26th international conference on machine learning (pp. 33-40). Montreal: Omnipress.
6. Bailly, R., Habrard, A., & Denis, F. (2010). A spectral approach for probabilistic grammatical inference on trees. In M. Hutter, F. Stephan, V. Vovk, & T. Zeugmann (Eds.), Lecture notes in computer science (Vol. 6331, pp. 74-88). Berlin: Springer.
7. Baker, J. K. (1979). Trainable grammars for speech recognition. In D. H. Klatt & J. J. Wolf (Eds.), Speech communication papers for the 97th meeting of the Acoustical Society of America (pp. 547-550).
8. Balle, B. (2013). Learning finite-state machines: algorithmic and statistical aspects. PhD thesis, Universitat Politècnica de Catalunya.
9. Balle, B., & Mohri, M. (2012). Spectral learning of general weighted automata via constrained matrix completion. In Advances in neural information processing systems (Vol. 25, pp. 2168-2176).
10. Balle, B., Quattoni, A., & Carreras, X. (2011). A spectral learning algorithm for finite state transducers. In D. Gunopulos, T. Hofmann, D. Malerba, & M. Vazirgiannis (Eds.), Lecture notes in computer science: Vol. 6911. ECML/PKDD (1) (pp. 156-171). Berlin: Springer.
11. Balle, B., Quattoni, A., & Carreras, X. (2012). Local loss optimization in operator models: a new insight into spectral learning. In J. Langford & J. Pineau (Eds.), Proceedings of the 29th international conference on machine learning (ICML-2012), ICML'12 (pp. 1879-1886). New York: Omnipress.
12. Balle, B., Castro, J., & Gavaldà, R. (2013). Learning probabilistic automata: a study in state distinguishability. Theoretical Computer Science, 473, 46-60.
13. Beimel, A., Bergadano, F., Bshouty, N., Kushilevitz, E., & Varricchio, S. (2000). Learning functions represented as multiplicity automata. Journal of the ACM, 47, 506-530.
14. Berstel, J., & Reutenauer, C. (1988). Rational series and their languages. Berlin: Springer.
15. Boots, B., Siddiqi, S., & Gordon, G. (2011). Closing the learning planning loop with predictive state representations. The International Journal of Robotics Research, 30(7), 954-966.
16. Carlyle, J. W., & Paz, A. (1971). Realizations by stochastic finite automata. Journal of Computer and System Sciences, 5(1), 26-40.
17. Castro, J., & Gavaldà, R. (2008). Towards feasible PAC-learning of probabilistic deterministic finite automata. In Proceedings of the 9th international colloquium on grammatical inference (ICGI) (pp. 163-174).
18. Clark, A., & Thollard, F. (2004). PAC-learnability of probabilistic deterministic finite state automata. Journal of Machine Learning Research, 5, 473-497.
19. Clark, S., & Curran, J. R. (2004). Parsing the WSJ using CCG and log-linear models. In Proceedings of the 42nd meeting of the association for computational linguistics (ACL'04), main volume, Barcelona, Spain (pp. 103-110).
20. Cohen, S. B., Stratos, K., Collins, M., Foster, D. P., & Ungar, L. (2012). Spectral learning of latent-variable PCFGS. In Proceedings of the 50th annual meeting of the association for computational linguistics (Volume 1: Long papers) (pp. 223-231). Jeju Island: Association for Computational Linguistics.
21. Cohen, S. B., Stratos, K., Collins, M., Foster, D. P., & Ungar, L. (2013). Experiments with spectral learning of latent-variable pcfgs. In Proceedings of the 2013 conference of the North American chapter of the Association for Computational Linguistics: human language technologies (pp. 148-157). Atlanta: Association for Computational Linguistics.
22. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39(1), 1-38.
23. Denis, F., & Esposito, Y. (2008). On rational stochastic languages. Fundamenta Informaticae, 86(1-2), 41-77.
24. Dhillon, P., Rodu, J., Collins, M., Foster, D., & Ungar, L. (2012). Spectral dependency parsing with latent variables. In Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning (pp. 205-213). Jeju Island: Association for Computational Linguistics.
25. Eisner, J. (2000). Bilexical grammars and their cubic-time parsing algorithms. In H. Bunt & A. Nijholt (Eds.), Advances in probabilistic and other parsing technologies (pp. 29-62). Norwell: Kluwer Academic.
26. Eisner, J., & Satta, G. (1999). Efficient parsing for bilexical context-free grammars and head-automaton grammars. In Proceedings of the 37th annual meeting of the Association for Computational Linguistics (ACL), University of Maryland (pp. 457-464).
27. Eisner, J., & Smith, N. A. (2010). Favor short dependencies: parsing with soft and hard constraints on dependency length. In H. Bunt, P. Merlo, & J. Nivre (Eds.), Trends in parsing technology: dependency parsing, domain adaptation, and deep parsing (Vol. 8, pp. 121-150). Berlin: Springer.
28. Fliess, M. (1974). Matrices de Hankel. Journal de Mathématiques Pures et Appliquées, 53, 197-222.
29. Goodman, J. (1996). Parsing algorithms and metrics. In Proceedings of the 34th annual meeting of the Association for Computational Linguistics (pp. 177-183). Santa Cruz: Association for Computational Linguistics.
30. Hsu, D., Kakade, S. M., & Zhang, T. (2009). A spectral algorithm for learning hidden Markov models. In Proceedings of the annual conference on computational learning theory (COLT).
31. Hsu, D., Kakade, S. M., & Liang, P. (2012). Identifiability and unmixing of latent parse trees. Advances in neural information processing systems (NIPS).
32. Kearns, M., Mansour, Y., Ron, D., Rubinfeld, R., Schapire, R. E., & Sellie, L. (1994). On the learnability of discrete distributions. STOC '94. In Proceedings of the twenty-sixth annual ACM symposium on theory of computing (pp. 273-282). New York: ACM.
33. Luque, F. M., Quattoni, A., Balle, B., & Carreras, X. (2012). Spectral learning for non-deterministic dependency parsing. In Proceedings of the 13th conference of the European chapter of the Association for Computational Linguistics (pp. 409-419). Avignon: Association for Computational Linguistics.
34. Manning, C. D., Raghavan, P., & Schütze, H. (2008). Introduction to information retrieval (1st ed.). Cambridge: Cambridge University Press.
35. Marcus, M. P., Santorini, B., & Marcinkiewicz, M. A. (1993). Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics, 19, 313-330.
36. McDonald, R., & Pereira, F. (2006). Online learning of approximate dependency parsing algorithms. In Proceedings of the 11th conference of the European chapter of the Association for Computational Linguistics (pp. 81-88).
37. McDonald, R., Pereira, F., Ribarov, K., & Hajic, J. (2005). Non-projective dependency parsing using spanning tree algorithms. In Proceedings of human language technology conference and conference on empirical methods in natural language processing (pp. 523-530). Vancouver: Association for Computational Linguistics.
38. Mohri, M. (2009). Weighted automata algorithms. In M. Droste, W. Kuich, & H. Vogler (Eds.), Monographs in theoretical computer science. An EATCS series. Handbook of weighted automata (pp. 213-254). Berlin: Springer.
39. Mossel, E., & Roch, S. (2005). Learning nonsingular phylogenies and hidden Markov models. In Proceedings of the 37th annual ACM symposium on theory of computing (STOC) (pp. 366-375).
40. Palmer, N., & Goldberg, P. W. (2007). PAC-learnability of probabilistic deterministic finite state automata in terms of variation distance. Theoretical Computer Science, 387(1), 18-31.
41. Parikh, A., Song, L., & Xing, E. (2011). A spectral algorithm for latent tree graphical models. In Proceedings of the 28th international conference on machine learning, ICML 2011 (ICML) (pp. 1065-1072).
42. Park, J. D., & Darwiche, A. (2004). Complexity results and approximation strategies for map explanations. Journal of Artificial Intelligence Research, 21, 101-133.
43. Petrov, S., & Klein, D. (2007). Improved inference for unlexicalized parsing. Proceedings of the main conference, Association for Computational Linguistics. In Human language technologies 2007: the conference of the North American chapter of the Association for Computational Linguistics (pp. 404-411). Rochester: Association for Computational Linguistics.
44. Petrov, S., Das, D., & McDonald, R. (2012). A universal part-of-speech tagset. In Proceedings of LREC.
45. Ron, D., Singer, Y., & Tishby, N. (1998). On the learnability and usage of acyclic probabilistic finite automata. Journal of Computing Systems Science, 56(2), 133-152.
46. Salomaa, A., & Soittola, M. (1978). Automata-theoretic aspects of formal power series. New York: Springer.
47. Schützenberger, M. (1961). On the definition of a family of automata. Information and Control, 4, 245-270.
48. Siddiqi, S. M., Boots, B., & Gordon, G. J. (2010). Reduced-rank hidden Markov models. In Proceedings of the thirteenth international conference on artificial intelligence and statistics (AISTATS) (pp. 741-748).
49. Song, L., Siddiqi, S. M., Gordon, G., & Smola, A. (2010). Hilbert space embeddings of hidden Markov models. In J. Fürnkranz & T. Joachims (Eds.), Proceedings of the 27th international conference on machine learning (ICML-10) (pp. 991-998). Haifa: Omnipress.
50. Titov, I., & Henderson, J. (2006). Loss minimization in parse reranking. In Proceedings of the 2006 conference on empirical methods in natural language processing (pp. 560-567). Sydney: Association for Computational Linguistics.
51. Valiant, L. G. (1984). A theory of the learnable. Communications of the ACM, 27(11), 1134-1142.
52. Wiewiora, E. (2005). Learning predictive representations from a history. In Proceedings of the 22nd international conference on machine learning (pp. 964-971). New York: ACM. Técnicas