Links between multiplicity automata, observable operator models and predictive state representations | a unified learning framework

Journal of Machine Learning Research

Volumn 16, Issue , 2015, Pages 103-147

  Thon, Michael ^a   Jaeger, Herbert ^a  

^a JACOBS UNIVERSITY BREMEN   (Germany)

Author keywords

Hidden markov models; Multiplicity automata; Observable operator models; Predictive state representations; Spectral learning algorithms

Indexed keywords

LEARNING ALGORITHMS; MARKOV PROCESSES; RANDOM PROCESSES; SPEECH RECOGNITION; STOCHASTIC MODELS; STOCHASTIC SYSTEMS;

GRAMMATICAL INFERENCES; LEARNING FRAMEWORKS; MULTIPLICITY AUTOMATA; OBSERVABLE OPERATOR MODELS; PREDICTIVE STATE REPRESENTATION; PROBABILISTIC AUTOMATA; SPECTRAL LEARNING ALGORITHMS; WEIGHTED FINITE AUTOMATON;

HIDDEN MARKOV MODELS;

EID: 84923822469     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (87)

1. Proceedings of the 37th Annual Symposium on Foundations of Computer Science (FOCS 1996), 1996. IEEE Computer Society.
2. Naoki Abe and Manfred K. Warmuth. On the computational complexity of approximating distributions by probabilistic automata. Machine Learning, 9:205{260, 1992.
3. Animashree Anandkumar, Daniel Hsu, and Sham M. Kakade. A method of moments for mixture models and hidden markov models. In Shie Mannor, Nathan Srebro, and Robert C. Williamson, editors, Proceedings of the 25th Annual Conference on Learning Theory (COLT 2012), volume 23 of JMLR Workshop & Conference Proceedings, pages 33.1{33.34, 2012.
4. Dana Angluin. Queries and concept learning. Machine Learning, 2(4):319{342, 1987.
5. Raphael Bailly. Quadratic weighted automata: Spectral algorithm and likelihood maximization.In Chun-Nan Hsu and Wee Sun Lee, editors, Proceedings of the 3rd Asian Conference on Machine Learning (ACML 2011), volume 20 of JMLR Workshop & Con- ference Proceedings, pages 147{163, 2011.
6. Raphael Bailly, Fran-cois Denis, and Liva Ralivola. Grammatical inference as a principal component analysis problem. In Andrea Pohoreckyj Danyluk, L-eon Bottou, and Michael L. Littman, editors, Proceedings of the 26th International Conference on Ma- chine Learning (ICML 2009), volume 382 of ACM Proceedings, pages 33{40, 2009.
7. Raphael Bailly, Xavier Carreras, and Ariadna Quattoni. Unsupervised spectral learning of nite state transducers. In C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 26 (NIPS 2013), pages 800{808. Curran Associates, Inc., 2013.
8. Borja Balle and Mehryar Mohri. Spectral learning of general weighted automata via constrained matrix completion. In Peter L. Bartlett, Fernando C. N. Pereira, Christopher J. C. Burges, L-eon Bottou, and Kilian Q. Weinberger, editors, Advances in Neural Infor- mation Processing Systems 25 (NIPS 2012), pages 2168{2176, 2012.
9. Borja Balle, Ariadna Quattoni, and Xavier Carreras. A spectral learning algorithm for nite state transducers. In Dimitrios Gunopulos, Thomas Hofmann, Donato Malerba, and Michalis Vazirgiannis, editors, Machine Learning and Knowledge Discovery in Databases - European Conference (ECML/PKDD 2011), Proceedings, Part I, volume 6911 of Lecture Notes in Computer Science, pages 156{171. Springer, 2011.
10. Borja Balle, Ariadna Quattoni, and Xavier Carreras. Local loss optimization in operator models: A new insight into spectral learning. In Proceedings of the 29th International Conference on Machine Learning (ICML 2012). icml.cc/Omnipress, 2012.
11. Borja Balle, Xavier Carreras, Franco M. Luque, and Ariadna Quattoni. Spectral learning of weighted automata { a forward-backward perspective. Machine Learning, 96(1-2):33{63, 2014.
12. Amos Beimel, Francesco Bergadano, Nader H. Bshouty, Eyal Kushilevitz, and Stefano Varricchio. On the applications of multiplicity automata in learning. In Proceedings of the 37th Annual Symposium on Foundations of Computer Science (FOCS 1996) FOC (1996), pages 349{358.
13. Amos Beimel, Francesco Bergadano, Nader H. Bshouty, Eyal Kushilevitz, and Stefano Varricchio. Learning functions represented as multiplicity automata. Journal of the ACM, 47(3):506{530, 2000.
14. Francesco Bergadano and Stefano Varricchio. Learning behaviors of automata from multiplicity and equivalence queries. In Maurizio A. Bonuccelli, Pierluigi Crescenzi, and Rossella Petreschi, editors, Proceedings of the 2nd Italian conference on Algorithms and Complexity (CIAC 1994), volume 778 of Lecture Notes in Computer Science, pages 54{62. Springer, 1994.
15. Francesco Bergadano, Dario Catalano, and Stefano Varricchio. Learning sat-k-DNF formulas from membership queries. In Proceedings of the 28th Annual ACM Symposium on Theory of Computing (STOC 1996), pages 126{130. ACM, 1996.
16. Jean Berstel Jr. and Christophe Reutenauer. Rational Series and Their Languages, volume 12 of EATCS Monographs on Theoretical Computer Science. Springer, 1988.
17. Byron Boots and Geo-rey J. Gordon. Predictive state temporal dierence learning. In J. La-erty, C. K. I. Williams, J. Shawe-Taylor, R.S. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems 23 (NIPS 2010), pages 271{279. MIT Press, 2010.
18. Byron Boots and Georey J. Gordon. An online spectral learning algorithm for partially observable nonlinear dynamical systems. In Wolfram Burgard and Dan Roth, editors, Proceedings of the 25th AAAI Conference on Arti-cial Intelligence (AAAI 2011). AAAI Press, 2011.
19. Byron Boots, Sajid M. Siddiqi, and Georey J. Gordon. Closing the learning-planning loop with predictive state representations. In Proceedings of the 9th International Confer- ence on Autonomous Agents and Multiagent Systems (AAMAS 2010), pages 1369{1370. IFAAMAS, 2010.
20. Byron Boots, Georey J. Gordon, and Arthur Gretton. Hilbert space embeddings of predictive state representations. In Proceedings of the 29th Conference on Uncertainty in Articial Intelligence (UAI 2013), pages 92{101. AUAI Press, 2013.
21. Michael Bowling, Peter McCracken, Michael James, James Neufeld, and Dana F. Wilkinson. Learning predictive state representations using non-blind policies. In Cohen and Moore (2006), pages 129{136.
22. Carla E. Brodley, editor., Proceedings of the 21st International Conference on Machine Learning (ICML 2004) volume 69 of ACM Proceedings, 2004.
23. Jack W. Carlyle and Azaria Paz. Realizations by stochastic nite automata. Journal of Computer and System Sciences, 5(1):26{40, 1971.
24. William W. Cohen and Andrew Moore, editors. Proceedings of the 23rd International Conference on Machine Learning (ICML 2006), volume 148 of ACM Proceedings, 2006
25. Corinna Cortes and Mehryar Mohri. Context-free recognition with weighted automata. Grammars, 3(2/3):133{150, 2000.
26. Arthur P. Dempster, Nan M. Laird, and Donald B. Rubin. Maximum likelihood estimation from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39 (1):1{38, 1977.
27. Francois Denis and Yann Esposito. Learning classes of probabilistic automata. In Proceed- ings of the 17th Annual Conference on Learning Theory (COLT 2004), volume 3120 of Lecture Notes in Computer Science, pages 124{139. Springer, 2004.
28. Francois Denis and Yann Esposito. On rational stochastic languages. Fundamenta Informaticae, 86(1):41{77, 2008.
29. Francois Denis, Yann Esposito, and Amaury Habrard. Learning rational stochastic languages.In Gabor Lugosi and Hans-Ulrich Simon, editors, Proceedings of the 19th Annual Conference on Learning Theory (COLT 2006), volume 4005 of Lecture Notes in Computer Science, pages 274{288. Springer, 2006.
30. Manfred Droste, Werner Kuich, and Heiko Vogler. Handbook of Weighted Automata. Springer, 2009.
31. Pierre Dupont, Francois Denis, and Yann Esposito. Links between probabilistic automata and hidden Markov models: probability distributions, learning models and induction algorithms. Pattern Recognition, 38(9):1349{1371, 2005.
32. Carl Eckart and Gale Young. The approximation of one matrix by another of lower rank. Psychometrika, 1(3):211{218, 1936.
33. Michel Fliess. Matrices de Hankel. Journal de Mathematiques Pures et Appliquees, 53: 197{222, 1974.
34. Edgar J. Gilbert. On the identiability problem for functions of nite Markov chains. The Annals of Mathematical Statistics, 30(3):688{697, 1959.
35. Robert M. Gray. Probability, Random Processes, and Ergodic Properties. Springer, 1988.
36. William L. Hamilton, Mahdi M. Fard, and Joelle Pineau. Modelling sparse dynamical systems with compressed predictive state representations. In Sanjoy Dasgupta and David Mcallester, editors, Proceedings of the 30th International Conference on Machine Learn- ing (ICML 2013), volume 28 of JMLR Workshop & Conference Proceedings, pages 178{ 186, 2013.
37. Per Christian Hansen. Rank-De-cient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 1998.
38. Alex Heller. On stochastic processes derived from Markov chains. The Annals of Mathe- matical Statistics, 36(4):1286{1291, 1965.
39. Daniel Hsu, Sham M. Kakade, and Tong Zhang. A spectral algorithm for learning hidden Markov models. In Proceedings of the 22nd Annual Conference on Learning Theory (COLT 2009), 2009.
40. Hisashi Ito. An Algebraic Study of Discrete Stochastic Systems. Unpublished doctoral dissertation, University of Tokyo, Bunkyo-ku, Tokyo, 1992.
41. Hisashi Ito, Shun ichi Amari, and Kingo Kobayashi. Identi-ability of hidden Markov information sources and their minimum degrees of freedom. IEEE Transactions on Informa tion Theory, 38(2):324{333, 1992.
42. Herbert Jaeger. Observable operator models and conditioned continuation representations. Arbeitspapiere der GMD 1043, GMD Forschungszentrum Informationstechnik, Sankt Augustin, Germany, 1997.
43. Herbert Jaeger. Discrete-time, discrete-valued observable operator models: A tutorial. Technical Report 42, GMD Forschungszentrum Informationstechnik, Sankt Augustin, Germany, 1998.
44. Herbert Jaeger. Modeling and learning continuous-valued stochastic processes with OOMs. GMD Report 102, GMD Forschungszentrum Informationstechnik, Sankt Augustin, Germany, 2000a.
45. Herbert Jaeger. Observable operator models for discrete stochastic time series. Neural Computation, 12(6):1371{1398, 2000b.
46. Herbert Jaeger, MingJie Zhao, and Andreas Kolling. Ecient estimation of ooms. In Weiss et al. (2006), pages 555{562.
47. Herbert Jaeger, MingJie Zhao, Klaus Kretzschmar, Tobias Oberstein, Dan Popovici, and Andreas Kolling. Learning observable operator models via the ES algorithm. In Simon Haykin, Jose C. Prncipe, Terrence J. Sejnowski, and John McWhirter, editors, New Directions in Statistical Signal Processing: From Systems to Brains, Neural Information Processing, chapter 14, pages 417{464. MIT Press, Cambridge, MA, USA, 2006b.
48. Michael R. James and Satinder P. Singh. Learning and discovery of predictive state representations in dynamical systems with reset. In Brodley (2004), pages 53{60.
49. Michael R. James and Satinder P. Singh. Planning in models that combine memory with predictive representations of state. In Manuela M. Veloso and Subbarao Kambhampati, editors, Proceedings of the 20th National Conference on Arti-cial Intelligence (AAAI 2005), pages 987{992. AAAI Press, 2005.
50. Michael R. James, Satinder Singh, and Michael L. Littman. Planning with predictive state representations. In Proceedings of the 3rd International Conference on Machine Learning and Applications (ICMLA 2004), pages 304{311. IEEE Computer Society, 2004.
51. Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra. Planning and acting in partially observable stochastic domains. Articial Intelligence, 101(1-2):99{134, 1998.
52. Attila Kondacs and John Watrous. On the power of quantum -nite state automata. In Proceedings of the 37th Annual Symposium on Foundations of Computer Science (FOCS 1996) FOC (1996), pages 66{75.
53. Klaus Kretzschmar. Learning symbol sequences with observable operator models. GMD Report 161, GMD Forschungszentrum Informationstechnik, Sankt Augustin, Germany, 2001.
54. Michael L. Littman, Richard S. Sutton, and Satinder P. Singh. Predictive representations of state. In Thomas G. Dietterich, Suzanna Becker, and Zoubin Ghahramani, editors, Advances in Neural Information Processing Systems 14 (NIPS 2001), pages 1555{1561. MIT Press, 2001.
55. Ivan Markovsky and Sabine Van Huel. Left vs right representations for solving weighted low-rank approximation problems. Linear Algebra and its Applications, 422(2-3):540{552, 2007a.
56. Ivan Markovsky and Sabine Van Huel. Overview of total least-squares methods. Signal Processing, 87(10):2283{2302, 2007b.
57. Peter McCracken and Michael H. Bowling. Online discovery and learning of predictive state representations. In Weiss et al. (2006).
58. Mehryar Mohri, Fernando Pereira, and Michael Riley. Weighted nite-state transducers in speech recognition. Computer Speech & Language, 16(1):69{88, 2002.
59. Cristopher Moore and James P. Crutcheld. Quantum automata and quantum grammars. Theoretical Computer Science, 237(1-2):275{306, 2000.
60. Hiroyuki Ohnishi, Hiroyuki Seki, and Tadao Kasami. A polynomial time learning algorithm for recognizable series. IEICE Transactions on Information and Systems, E77-D(10): 1077{1085, 1994.
61. Lawrence R. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257{286, 1989.
62. Adria Recasens and Ariadna Quattoni. Spectral learning of sequence taggers over continuous sequences. In Hendrik Blockeel, Kristian Kersting, Siegfried Nijssen, and Filip Zelezn-y, editors, Machine Learning and Knowledge Discovery in Databases - European Conference (ECML/PKDD 2013), Proceedings, Part I, volume 8188 of Lecture Notes in Computer Science, pages 289{304. Springer, 2013.
63. Matthew Rosencrantz, Geo-rey J. Gordon, and Sebastian Thrun. Learning low dimensional predictive representations. In Brodley (2004), pages 695{702.
64. Sam Roweis. EM algorithms for PCA and SPCA. In Michael I. Jordan, Michael J. Kearns, and Sara A. Solla, editors, Advances in Neural Information Processing Systems 10 (NIPS 1997), pages 626{632. MIT Press, 1998.
65. Matthew Rudary and Satinder P. Singh. Predictive linear-Gaussian models of controlled stochastic dynamical systems. In Cohen and Moore (2006), pages 777{784.
66. Matthew Rudary and Satinder P. Singh. Predictive linear-Gaussian models of stochastic dynamical systems with vector-value actions and observations. In Proceedings of the 10th International Symposium on Arti-cial Intelligence and Mathematics (ISAIM 2008), 2008.
67. Matthew Rudary, Satinder P. Singh, and David Wingate. Predictive linear-Gaussian models of stochastic dynamical systems. In Fahiem Bacchus and Tommi Jaakkola, editors, Proceedings of the 21st Conference in Uncertainty in Arti-cial Intelligence (UAI 2005), pages 501{508. AUAI Press, 2005.
68. Matthew R. Rudary and Satinder Singh. A nonlinear predictive state representation. In S. Thrun S. Becker and K. Obermayer, editors, Advances in Neural Information Process- ing Systems 15 (NIPS 2002), pages 855{862. MIT Press, 2003.
69. Arto Salomaa and Matti Soittola. Automata-Theoretic Aspects of Formal Power Series. Texts and Monographs in Computer Science. Springer, 1978.
70. Marcel Paul Schutzenberger. On the de-nition of a family of automata. Information and Control, 4(2-3):245{270, 1961.
71. Sajid M. Siddiqi, Byron Boots, and Georey J. Gordon. Reduced-rank hidden markov models. In Yee Whye Teh and D. Mike Titterington, editors, Proceedings of the 13th International Conference on Arti-cial Intelligence and Statistics (AISTATS 2010), volume 9 of JMLR Workshop & Conference Proceedings, pages 741{748, 2010.
72. Satinder Singh, Michael R. James, and Matthew R. Rudary. Predictive state representations: A new theory for modeling dynamical systems. In Joseph Halpern, editor, Proceedings of the 20th Conference on Uncertainty in Arti-cial Intelligence (UAI 2004), pages 512{519. AUAI Press, 2004.
73. Le Song, Byron Boots, Sajid M. Siddiqi, Geo-rey J. Gordon, and Alex J. Smola. Hilbert space embeddings of hidden Markov models. In Johannes Furnkranz and Thorsten Joachims, editors, Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pages 991{998. Omnipress, 2010.
74. Daniel R. Upper. Theory and Algorithms for Hidden Markov Models and Generalized Hidden Markov Models. PhD thesis, University of California at Berkeley, 1997.
75. Leslie G. Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134{ 1142, 1984.
76. Y. Weiss, B. Scholkopf, and J. Platt, editors. Advances in Neural Information Processing Systems 18 (NIPS 2005), 2006. MIT Press.
77. Eric W. Wiewiora. Modeling Probability Distributions with Predictive State Representations. PhD thesis, University of California, San Diego, 2008.
78. David Wingate and Satinder P. Singh. Kernel predictive linear Gaussian models for nonlinear stochastic dynamical systems. In Cohen and Moore (2006), pages 1017{1024.
79. David Wingate and Satinder P. Singh. Mixtures of predictive linear Gaussian models for nonlinear, stochastic dynamical systems. In Anthony Cohn, editor, Proceedings of the 21st National Conference on Arti-cial Intelligence (AAAI 2006). AAAI Press, 2006b.
80. David Wingate and Satinder P. Singh. Exponential family predictive representations of state. In J.C. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20 (NIPS 2007), pages 1617{1624. MIT Press, 2008a.
81. David Wingate and Satinder P. Singh. Eciently learning linear-linear exponential family predictive representations of state. In William W. Cohen, Andrew McCallum, and Sam T. Roweis, editors, Proceedings of the 25th International Conference on Machine Learning (ICML 2008), volume 307 of ACM Proceedings, pages 1176{1183, 2008b.
82. David Wingate, Vishal Soni, Britton Wolfe, and Satinder P. Singh. Relational knowledge with predictive state representations. In Manuela M. Veloso, editor, Proceedings of the 20th International Joint Conference on Arti-cial Intelligence (IJCAI 2007), pages 2035{ 2040. AAAI Press, 2007.
83. Britton Wolfe and Satinder P. Singh. Predictive state representations with options. In Cohen and Moore (2006), pages 1025{1032.
84. LotAsker Zadeh. The concept of system, aggregate, and state in system theory. InLot Asker Zadeh and Elijah Polak, editors, System Theory, volume 8 of Inter-University Electronics Series, pages 3{42. McGraw-Hill, New York, 1969.
85. MingJie Zhao and Herbert Jaeger. Norm observable operator models. Neural Computation, 22(7):1927{1959, 2010.
86. MingJie Zhao, Herbert Jaeger, and Michael Thon. A bound on modeling error in observable operator models and an associated learning algorithm. Neural Computation, 21(9):2687{ 2712, 2009a.
87. MingJie Zhao, Herbert Jaeger, and Michael Thon. Making the error-controlling algorithm of observable operator models constructive. Neural Computation, 21(12):3460{3486, 2009b.