A spectral algorithm for learning hidden Markov models

COLT 2009 - The 22nd Conference on Learning Theory

Volumn , Issue , 2009, Pages

  Hsu, Daniel ^a   Kakade, Sham M ^b   Zhang, Tong ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO   (United States)

^c RUTGERS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CRYPTOGRAPHIC ASSUMPTIONS; DISCRETE TIME SERIES; HIDDEN MARKOV MODELS (HMMS); MATRIX MULTIPLICATION; NATURAL LANGUAGE PROCESSING; SEPARATION CONDITION; SPECTRAL ALGORITHM; SPECTRAL PROPERTIES;

HEURISTIC ALGORITHMS; HIDDEN MARKOV MODELS; NATURAL LANGUAGE PROCESSING SYSTEMS; STATISTICAL MECHANICS;

LEARNING ALGORITHMS;

EID: 84898066687     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (26)

1. S. Andersson, T. Ryden, and R. Johansson. Linear optimal prediction and innovations representations of hidden markov models. Stochastic Processes and their Applications, 108: 131-149, 2003.
2. Leonard E. Baum and J.A. Eagon. An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology. Bull. Amer. Math. Soc., 73(3): 360-363, 1967.
3. Leonard E. Baum, Ted Petrie. George Soules, and Norman Weiss. A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Annals of Mathematical Statistics, 41(1): 164-171, 1970.
4. S. Charles Brubaker and Santosh Vempala. Isotropic PCA and affine-invariant clustering. In FOCS, 2008.
5. G. Cybenko and V. Crespi. Learning hidden markov models using non-negative matrix factorization. Technical report, 2008. arXiv: 0809.4086.
6. J.W. Carlyle and A. Paz. Realization by stochastic finite automaton. J. Comput. Syst. Sci., 5: 26-40, 1971.
7. Kamalika Chaudhuri and Satish Rao. Learning mixtures of product distributions using correlations and independence. In COLT, 2008.
8. Sanjoy Dasgupta. Learning mixutres of Gaus-sians. In FOCS, 1999.
9. A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39(1): 1-38, 1977.
10. Sanjoy Dasgupta and Leonard Schulman. A probabilistic analysis of EM for mixtures of separated, spherical Gaussians. JMLR, 8(Feb): 203-226, 2007.
11. Eyal Even-Dar, Sham M. Kakade, and Yishay Mansour. Planning in POMDPs using multiplicity automata. In UAI, 2005.
12. Eyal Even-Dar, Sham M. Kakade, and Yishay Mansour. The value of observation for monitoring dynamic systems. In IJCAI, 2007.
13. M. Fliess. Matrices deHankel. J. Math. Pures Appl., 53: 197-222, 1974.
14. H. Hotelling. The most predictable criterion. Journal of Educational Psychology, 1935.
15. Herbert Jaeger. Observable operator models for discrete stochastic time series. Neural Comput., 12(6), 2000.
16. T. Katayama. Subspace Methods for System Identification. Springer, 2005.
17. M. Kearns, Y. Mansour, D. Ron, R. Rubinfeld, R. Schapire, and L. Sellie. On the learnability of discrete distributions. In STOC, pages 273-282, 1994.
18. L. Ljung. System Identification: Theory for the User. NJ: Prentice-Hall Englewood Cliffs, 1987.
19. Michael Littman, Richard Sutton, and Satinder Singh. Predictive representations of state. In Advances in Neural Information Processing Systems 14 (NIPS), pages 1555-1561, 2001.
20. E. Mossel and S. Roch. Learning nonsingular phylogenies and hidden Markov models. Annals of Applied Probability, 16(2): 583-614, 2006.
21. P. V. Overschee and B. De Moor. Subspace Identification of Linear Systems. Kluwer Academic Publishers, 1996.
22. M.P. Schützenberger. On the definition of a family of automata. Inf. Control, 4: 245-270, 1961.
23. Sebastiaan Terwijn. On the learnability of hidden Markov models. In International Colloquium on Grammatical Inference, 2002.
24. Santosh Vempala and Grant Wang. A spectral algorithm for learning mixtures of distributions. In FOCS, 2002.
25. B. Vanluyten, J. Willems, and B. De Moor. A new approach for the identification of hidden markov models. In Conference on Decision and Control, 2007.
26. Ming Jie Zhao and Herbert Jaeger. The error controlling algorithm for learning OOMs. Technical Report 6, International University Bremen, 2007.